LBRYCommunity/ytsync
1
0
Fork 0
Code Issues 39 Pull requests 8 Projects Releases 197 Packages Wiki Activity
522 commits 17 branches 205 tags 26 MiB
Commit graph

211 commits

Author SHA1 Message Date
Mark Beamer Jr
d99e200178
Add check for already published videos using new video-state api. 2020-07-29 00:12:23 -04:00
Alex Grintsvayg
357aebbcce
less loud 2020-07-28 22:40:56 -04:00
Alex Grintsvayg
0eef62b5fd
i made a huge mess. sorry future me, when you're digging through this 2020-07-28 21:34:08 -04:00
Alex Grintsvayg
a3dd3dc626
WIP: trying to get the accurate api to work 2020-07-28 12:47:28 -04:00
Alex Grintsvayg
70ad891dfa
use durations 2020-07-27 19:35:07 -04:00
Mark Beamer Jr
e7fdd21bac
commit some more stuff 2020-07-27 17:19:15 -04:00
Alex Grintsvayg
1369ed0b48
replace yt channel video count with a scrape of socialblade.com 2020-07-27 15:51:06 -04:00
Alex Grintsvayg
bffc0823be
more 2020-07-27 14:52:21 -04:00
Alex Grintsvayg
edbb22fcf0
rename 2020-07-27 14:49:37 -04:00
Alex Grintsvayg
843303301a
move all youtube api calls into a single place 2020-07-27 14:48:05 -04:00
Alex Grintsvayg
4c4da93ef3
bump lbry.go dependency to be compatible with internal-apis dependency bump 2020-07-21 13:55:34 -04:00
Niko Storni
c2b5878daa
don't delete lbry first videos from the db 2020-07-21 13:54:28 -04:00
Niko Storni
749d5095c9 fix semver 2020-06-11 18:45:56 +02:00
Niko Storni
1d5b69bfe6 add automatic sleep until quota reset 
re-enable ipv6
 2020-06-10 03:32:45 +02:00
Niko Storni
4d915587e0 temporarily disable ipv6 2020-05-30 02:18:42 +02:00
Niko Storni
47e467057d change timeouts to avoid long wait times 2020-05-20 04:28:29 +02:00
Niko Storni
7907ee3579 add prometheus metrics 2020-05-19 23:13:01 +02:00
Niko Storni
414ed1c130 or to and 2020-05-14 17:28:15 +02:00
Niko Storni
3bb17b09d3 fix never ending condition 2020-05-14 06:47:19 +02:00
Niko Storni
76301d40ba only wait a maximum of 2 blocks before transfering 
fix debug string
 2020-05-08 16:33:29 +02:00
Niko Storni
95890cbabf prevent live streams from syncing 
reduce timeout to 20 minutes
 2020-04-29 16:15:22 +02:00
Niko Storni
69c2a91ec7 fix bug in video length 2020-04-23 16:37:35 +02:00
Niko Storni
9bfb1656fe fix length checks on single syncs 2020-04-10 17:46:07 +02:00
Niko Storni
5d40d24804 fix max length issue for bigger youtubers 
fix issue with no-playlist-found channels
reduce the max db size limit
 2020-04-08 23:14:10 +02:00
Niko Storni
79d3aadda8 add utxo_release as hotfix for stuck balances 2020-04-01 04:49:24 +02:00
Niko Storni
eba59a0806 change utxo_list signature 2020-04-01 04:44:05 +02:00
Niko Storni
3fd4909cd1 replace support abandon with txo_spend 
upgrade lbry.go library
upgrade lbrynet support
possibly introduce new bugs ;)
 2020-03-31 04:30:02 +02:00
Niko Storni
b59372c1d8 don't prevent channels with 0 youtube videos from transferring 2020-03-31 02:45:53 +02:00
Niko Storni
c9e713e9df do not attempt upgrading channels no longer in our control 2020-03-27 18:16:35 +01:00
Niko Storni
259df8d257 restrict length amount on small youtubers 2020-03-12 04:08:50 +01:00
Niko Storni
5460224cd6 upgrade SDK 
fix UTXO creation time waste
 2020-02-24 15:52:32 +01:00
Niko Storni
45e87c3007 don't ever re-create a channel if one was previously created 2020-01-14 04:34:33 +01:00
Niko Storni
cdee0d0de7 fix typo 2020-01-14 04:14:14 +01:00
Niko Storni
e30f21dc85 fail sync when channel is abandoned 2020-01-14 04:10:20 +01:00
Niko Storni
93f9185c71
fix bug when publishing 2020-01-11 22:01:40 -05:00
Niko Storni
ef62242f42 add cookies to youtube-dl 
minor fixes
 2020-01-03 15:59:05 +01:00
Niko Storni
4a356e86da add quicksync for better quota usage 2019-12-27 18:12:41 +01:00
Niko Storni
ac95ca57af halt when quota is exceeded 2019-12-27 04:15:48 +01:00
Niko Storni
c28d5a716b send warning to slack when video processing panics 2019-12-27 01:39:27 +01:00
Niko Storni
59471f9c26 reduce sync delay 
fix stopper bug
 2019-12-27 01:27:29 +01:00
Niko Storni
8028c6621c add temporary workaround to release all IPs 
speed up one failure
 2019-12-26 17:52:44 +01:00
Niko Storni
1f9a645c57 support create fixes 
various bug fixes
 2019-12-24 05:00:16 +01:00
Niko Storni
b6861dae9b don't fail on abandoned videos 2019-12-19 03:13:22 +01:00
Niko Storni
cff147bb95 work around missing inputs error 2019-12-18 20:24:50 +01:00
Niko Storni
54d04dcf2c add IP debugging 
improve process halting
rename improper language code for hebrew
 2019-12-18 18:22:15 +01:00
Niko Storni
65c4f99c1c make sure abandons spend from default account only 2019-12-16 04:53:15 +01:00
Niko Storni
2bde06e4b9 fixes to throttling 2019-12-14 14:58:04 +01:00
Niko Storni
5df6db6e96 fix bug with non-related publishes 2019-12-14 06:43:01 +01:00
Niko Storni
fe55304184 fix anti-throttling 
fix duplicates with multi-channels
 2019-12-13 18:20:41 +01:00
Niko Storni
36f539ef5d change ip throttling management 2019-12-10 23:02:56 +01:00
Niko Storni
7bdc117ba0
upgrade ytsync to latest SDK 
upgrade e2e tests
 2019-12-08 11:49:27 -05:00
Niko Storni
fc575dac80
improve logging 
fix ipv6 switching
 2019-12-08 10:31:15 -05:00
Niko Storni
77d0efcf3c refactor tags_mapping name 
remove failure state for transfers
 2019-10-21 15:44:24 +02:00
Niko Storni
291a105269 improve getDefaultAccount 
use default account on more calls
improve API logging
fix memory leak
 2019-10-16 19:38:45 +02:00
Niko Storni
6d2e4aa6e6 list all claims at once (faster) 2019-10-15 00:29:26 +02:00
Niko Storni
10a855a8ba fix nil ptr dereference 
fix bug in transfers
 2019-10-12 09:49:49 +02:00
Niko Storni
a9e76149e8 parallelize support abandons 
update lbrynet support in e2e test
stop using reflector on regtest!
change e2e channel target for more videos
possibly fix a deadlock
 2019-10-11 19:42:03 +02:00
Niko Storni
dd4aebdba4 make video transferring async 
fix bug in videosLimit
change test subject to have more videos
 2019-10-11 01:00:04 +02:00
Niko Storni
b409d36de0 remove unnecessary output 2019-10-10 18:17:13 +02:00
Niko Storni
30af4a0136 update lbry.go to v2 library 
update modules
add slightly more LBCs for the channel claim to ensure fees are covered
 2019-10-10 16:51:40 +02:00
Niko Storni
65daeec216 resolve issue with channels attempting re-transfer 2019-10-08 14:42:57 +02:00
Niko Storni
ad1704fb39 remove obsolete check 2019-10-08 02:13:17 +02:00
Niko Storni
44e6cb5ddc fix bug where newly published and transferred videos caused an error 
rename variable for better clarity
 2019-10-08 01:59:18 +02:00
Niko Storni
5171acc007 refactor integrity check 
add checks for improper transfer state
add checks for unsent supports
only use default account for operations that only work on it
add more logging
 2019-10-08 01:38:39 +02:00
Niko Storni
5961327adf reduce bid even further to avoid lockups 2019-10-03 21:06:47 +02:00
Niko Storni
d2bbb36312 improve detection for own claims vs their claims 2019-10-01 16:55:43 +02:00
Niko Storni
c7272e988a reduce amount to avoid dust 2019-09-27 20:17:49 +02:00
Niko Storni
5b9846bcca fix pointer issue 2019-09-27 19:29:42 +02:00
Niko Storni
8d849b8f06 do not retry unavailable videos 2019-09-27 16:12:21 +02:00
Niko Storni
c41c368f34 fix transfer bugs 2019-09-27 16:10:57 +02:00
Niko Storni
7ce4c4a2ce fix lbrynet shutdown check in e2e 2019-09-25 05:07:19 +02:00
Niko Storni
0d473cc4f6 add support for transferring videos after transferring everything 2019-09-25 04:38:49 +02:00
Niko Storni
2f701a887a fix double counting 
fix transferred check
 2019-09-25 04:15:16 +02:00
Niko Storni
8d56ef852f fix e2e 
fix database transferred flag
 2019-09-25 03:30:07 +02:00
Niko Storni
849ff11bfd import public keys if available 2019-09-24 20:42:17 +02:00
Niko Storni
07413b499a update SDK support 
fix bug where videos would publish to the wrong address
 2019-09-24 18:51:06 +02:00
Niko Storni
1b08bb0e61 fix bug with non latin titles for claim names 
hard exit when a duplicate name is attempted
 2019-09-10 11:43:20 +02:00
Niko Storni
bdd55c9965 fix balance refill issue 
add additional debugging
 2019-09-04 19:24:11 +02:00
Niko Storni
14668c339e refactor sync flags 
add disable-transfer flag
 2019-08-30 21:31:58 +02:00
Niko Storni
7d38aa7b29 add republishing on non-available videos 
enhanced logging
 2019-08-30 19:35:04 +02:00
Niko Storni
3c3ea2138e adjust support management 
adjust e2e script
 2019-08-28 21:08:25 +02:00
Mark Beamer Jr
bacb91e82a Add supporty to create supports 
Update lbry.go dep to get lbrycrd commands
 2019-08-28 15:25:53 +02:00
Niko Storni
577411cf50 fix e2e 
add support for supports (heh)
 2019-08-28 15:25:53 +02:00
Niko Storni
f7c80c2e5d refactor transfer process 
start working on supports
 2019-08-28 15:25:53 +02:00
Niko Storni
4d16a1d6e8 fix e2e tests 
update wallet server
cleanup go.sum
Fix broken balance logging
Prevent failures once channel is transferred
Wait for all UTXOs to confirm before transferring
Fix various bugs
 2019-08-28 15:25:53 +02:00
Niko Storni
7a7de03c0f adjust account balance 
minor fix
 2019-08-28 15:25:53 +02:00
Niko Storni
1b9ed266e0 move channel status call to proper region 
fix shadowed errors
refactor code to be more readable and consistent
get video transfer state
 2019-08-28 15:25:53 +02:00
Mark Beamer Jr
eb2f6273e4 Add transfer of channel after all video's have been transferred 
Add channel status call to notify external db that the channel has been transferred.
 2019-08-28 15:25:53 +02:00
Mark Beamer Jr
45cea808ed Add marking video status when transfer of claim is complete. 
Add error handling when transfer of claim fails.
 2019-08-28 15:25:53 +02:00
Mark Beamer Jr
9a5b6b4e56 Update to leverage lbrycrd v0.17.2.1, lbrynet v0.39.1, walletserver v0.39.1 2019-08-28 15:25:53 +02:00
Mark Beamer Jr
d63aba568d Add transfer logic as post sync process step. 
Add authtoken for youtuber in test data setup to call transfer api
Update e2e test to include scenario where we transfer channel.
 2019-08-28 15:25:53 +02:00
Niko Storni
4cc0b71279 process failures sequentially 2019-08-20 11:33:06 +02:00
Niko Storni
81616c1d96 refactor channel selector 
remove unused code
 2019-08-15 19:31:43 +02:00
Niko Storni
5cb7f72ef7 remove useless logging 2019-08-14 05:18:57 +02:00
Mark Beamer Jr
cd57e1ab08 Always send channel export 2019-08-14 04:22:59 +02:00
Mark Beamer Jr
b90e367e82 Addressed comments 2019-08-14 04:22:59 +02:00
Mark Beamer Jr
e477759395 change to master tag of internal-apis docker image 2019-08-14 04:22:59 +02:00
Mark Beamer Jr
6438048f40 Add CleanOnStartup for local testing and e2e testing 
Update methods to be refactored into util package to be used in many locations.
Add end to end test of ytsync for CI
 2019-08-14 04:22:59 +02:00
Mark Beamer Jr
73937a541b Add var/tmp volume for ytsync tmp files communicated between lbrynet and ytsync 
Add reference to wallet dir env var for when the wallet file is uploaded so it is cleared properly when testing locally
 2019-08-14 04:22:59 +02:00
Mark Beamer Jr
517c2c56a3 Add disabling of blob uploading for local testing until reflector docker image can be created( not urgent now ). 
Add ability to generate blocks when in regtest mode and using docker.
Add refactor of environment variable accessing.
 2019-08-14 04:22:59 +02:00