LBRYCommunity/ytsync
1
0
Fork 0
Code Issues 39 Pull requests 8 Projects Releases 197 Packages Wiki Activity
494 commits 17 branches 205 tags 26 MiB
Commit graph

53 commits

Author SHA1 Message Date
Niko Storni
070287716b switch from env vars to a config file 
get rid of stupid stuff
simplify S3 configuration
split wallets from blockchain.db and use separate S3 store
fix bugs
 2021-11-24 05:54:08 +01:00
Niko Storni
519e1e4648 switch to yt-dlp 
add progressbars
avoid unnecessary calls to youtube
update user agents
cookies fixes
bug fixes
introduction of new bugs
 2021-06-17 17:51:21 +02:00
Niko Storni
fecf67118c remove unused flag 
always go through syncing first
don't sync videos shorter than 7 seconds
refactor code in video error handling
add interface to handle hard video failures (incomplete)
 2020-11-03 21:41:39 +01:00
Niko Storni
beade71aa6 fix bugs 2020-11-03 02:14:01 +01:00
Niko Storni
f5f12e1560 add support for multiple queues 
add support for blockchain.db pruning via new status
 2020-10-27 19:50:10 +01:00
Niko Storni
7c02c5b92d refactor structures 
fix various bugs
 2020-08-08 01:12:55 +02:00
Niko Storni
5d230a6b54 potentially fix looping 
add limits by db
 2020-08-06 20:32:49 +02:00
Mark Beamer Jr
5be3551abe
Mark video failed if we cannot get video info instead of failing the sync 
Turn on error log after fixing lockup
Try different user agents if calls fail
Prevent double processing a channel in a row, just fail the sync instead
 2020-08-05 15:47:53 -04:00
Alex Grintsvayg
0eef62b5fd
i made a huge mess. sorry future me, when you're digging through this 2020-07-28 21:34:08 -04:00
Alex Grintsvayg
70ad891dfa
use durations 2020-07-27 19:35:07 -04:00
Niko Storni
749d5095c9 fix semver 2020-06-11 18:45:56 +02:00
Niko Storni
1d5b69bfe6 add automatic sleep until quota reset 
re-enable ipv6
 2020-06-10 03:32:45 +02:00
Niko Storni
4d915587e0 temporarily disable ipv6 2020-05-30 02:18:42 +02:00
Niko Storni
9bfb1656fe fix length checks on single syncs 2020-04-10 17:46:07 +02:00
Niko Storni
5d40d24804 fix max length issue for bigger youtubers 
fix issue with no-playlist-found channels
reduce the max db size limit
 2020-04-08 23:14:10 +02:00
Niko Storni
259df8d257 restrict length amount on small youtubers 2020-03-12 04:08:50 +01:00
Niko Storni
ef62242f42 add cookies to youtube-dl 
minor fixes
 2020-01-03 15:59:05 +01:00
Niko Storni
ac95ca57af halt when quota is exceeded 2019-12-27 04:15:48 +01:00
Niko Storni
59471f9c26 reduce sync delay 
fix stopper bug
 2019-12-27 01:27:29 +01:00
Niko Storni
8028c6621c add temporary workaround to release all IPs 
speed up one failure
 2019-12-26 17:52:44 +01:00
Niko Storni
1f9a645c57 support create fixes 
various bug fixes
 2019-12-24 05:00:16 +01:00
Niko Storni
77d0efcf3c refactor tags_mapping name 
remove failure state for transfers
 2019-10-21 15:44:24 +02:00
Niko Storni
30af4a0136 update lbry.go to v2 library 
update modules
add slightly more LBCs for the channel claim to ensure fees are covered
 2019-10-10 16:51:40 +02:00
Niko Storni
44e6cb5ddc fix bug where newly published and transferred videos caused an error 
rename variable for better clarity
 2019-10-08 01:59:18 +02:00
Niko Storni
849ff11bfd import public keys if available 2019-09-24 20:42:17 +02:00
Niko Storni
1b08bb0e61 fix bug with non latin titles for claim names 
hard exit when a duplicate name is attempted
 2019-09-10 11:43:20 +02:00
Niko Storni
14668c339e refactor sync flags 
add disable-transfer flag
 2019-08-30 21:31:58 +02:00
Niko Storni
1b9ed266e0 move channel status call to proper region 
fix shadowed errors
refactor code to be more readable and consistent
get video transfer state
 2019-08-28 15:25:53 +02:00
Mark Beamer Jr
eb2f6273e4 Add transfer of channel after all video's have been transferred 
Add channel status call to notify external db that the channel has been transferred.
 2019-08-28 15:25:53 +02:00
Mark Beamer Jr
45cea808ed Add marking video status when transfer of claim is complete. 
Add error handling when transfer of claim fails.
 2019-08-28 15:25:53 +02:00
Mark Beamer Jr
d63aba568d Add transfer logic as post sync process step. 
Add authtoken for youtuber in test data setup to call transfer api
Update e2e test to include scenario where we transfer channel.
 2019-08-28 15:25:53 +02:00
Niko Storni
4cc0b71279 process failures sequentially 2019-08-20 11:33:06 +02:00
Niko Storni
5cb7f72ef7 remove useless logging 2019-08-14 05:18:57 +02:00
Mark Beamer Jr
b90e367e82 Addressed comments 2019-08-14 04:22:59 +02:00
Mark Beamer Jr
e477759395 change to master tag of internal-apis docker image 2019-08-14 04:22:59 +02:00
Mark Beamer Jr
6438048f40 Add CleanOnStartup for local testing and e2e testing 
Update methods to be refactored into util package to be used in many locations.
Add end to end test of ytsync for CI
 2019-08-14 04:22:59 +02:00
Niko Storni
0aedf80386 limit throttling to only queued channels 2019-07-22 16:24:13 -04:00
Niko Storni
c193dae95a temporary throttling of channels to sync for tom to review them 2019-07-22 16:16:23 -04:00
Niko Storni
60f2585f33 improve throttling 
refactor slack wrapper
cleanup dependencies
 2019-07-15 16:16:02 -04:00
Niko Storni
51cb7f5e34 improve youtube antithrottle 
fix logging
 2019-07-11 19:14:15 +02:00
Niko Storni
b34d328cba halt ytsync when youtube throttles us 2019-07-10 22:47:01 +02:00
Niko Storni
5dbad2d2e4 add automatic blob reflection 
retry videos that fail due to tx issues
 2019-07-10 15:46:54 +02:00
Niko Storni
8c349cf7ac fix bug in queues 2019-06-24 04:14:27 -04:00
Niko Storni
e893f68bc1 fix queue pulling 
fix address reuse
 2019-06-12 22:42:42 +02:00
Niko Storni
f7f0231c80 add VideoUpgradeFailed staus 2019-06-10 21:59:42 +02:00
Niko Storni
10ad46f464 various bug fixes 2019-06-06 23:25:31 +02:00
Niko Storni
8dce052fe6 change logic used to upgrade videos 2019-06-06 16:24:20 +02:00
Niko Storni
76e653fb9b add Fee support 
lowercase all tags
refactor code
implement and test regexps for tags
 2019-06-06 02:16:07 +02:00
Niko Storni
cfe8ff0879
add flag to remove unpublished videos 
refactor code
add workaround for SDK bug
add code to remove unpublished videos
fix alt downloader to only produce mp4
add missing height and width
add tags sorting
add a few channel wide tags
 2019-06-04 22:21:44 +02:00
Niko Storni
23b8fb4e54 add support for new SDK 
add support for direct thumbnails upload
remove UCB code
add support for republishing
refactoring
 2019-05-03 05:11:52 +02:00