Commit graph

482 commits

Author SHA1 Message Date
Mark Beamer Jr
6438048f40 Add CleanOnStartup for local testing and e2e testing
Update methods to be refactored into util package to be used in many locations.
Add end to end test of ytsync for CI
2019-08-14 04:22:59 +02:00
Mark Beamer Jr
73937a541b Add var/tmp volume for ytsync tmp files communicated between lbrynet and ytsync
Add reference to wallet dir env var for when the wallet file is uploaded so it is cleared properly when testing locally
2019-08-14 04:22:59 +02:00
Mark Beamer Jr
517c2c56a3 Add disabling of blob uploading for local testing until reflector docker image can be created( not urgent now ).
Add ability to generate blocks when in regtest mode and using docker.
Add refactor of environment variable accessing.
2019-08-14 04:22:59 +02:00
Mark Beamer Jr
c84e1e6fb9 Fixed bugs around sending channel cert and enabling local testing with docker. 2019-08-14 04:22:59 +02:00
Mark Beamer Jr
f26712b022 Added e2e test docker-compose and supporting work so that ytsync can run independently of the mainnet for testing/CI 2019-08-14 04:22:59 +02:00
Mark Beamer Jr
5e56ac0516 Add set channel cert on new channels. 2019-08-14 04:22:59 +02:00
Niko Storni
0308da2e4c add timeout to daemon startup 2019-08-13 23:05:09 +02:00
Niko Storni
df63b91f61 update reflector libs 2019-08-06 20:57:36 +02:00
Niko Storni
03161a5a02 prefer 1080p videos 2019-08-05 20:08:46 +02:00
Niko Storni
6dc2c39ee6 limit description length
rather than limiting lines count
2019-08-02 10:01:33 -04:00
Niko Storni
87211e802d reduce tags length 2019-08-02 09:05:24 -04:00
Niko Storni
3ba0c97634 fix flags 2019-07-25 18:33:05 -04:00
Niko Storni
9d65fce6bc do not apply limits if disabled 2019-07-25 18:22:56 -04:00
Niko Storni
d5ed4ce753 facilitate streaming by moving the moov atom 2019-07-25 18:15:14 -04:00
Niko Storni
0aedf80386 limit throttling to only queued channels 2019-07-22 16:24:13 -04:00
Niko Storni
c193dae95a temporary throttling of channels to sync for tom to review them 2019-07-22 16:16:23 -04:00
Niko Storni
0c47510785 revert quoting paths 2019-07-22 02:51:13 +02:00
Niko Storni
8f556a86c8 fix missing fragments for good
never retry them again
2019-07-22 02:45:38 +02:00
Niko Storni
64ab2490fe fix bugs
improve debugging
2019-07-22 02:27:14 +02:00
Niko Storni
1760935edf don't retry missing fragments
lower syncing quality for now
2019-07-22 02:09:18 +02:00
Niko Storni
60f2585f33 improve throttling
refactor slack wrapper
cleanup dependencies
2019-07-15 16:16:02 -04:00
Niko Storni
df08d42d9b improve ip re-selection 2019-07-13 12:15:19 +02:00
Niko Storni
1b55033bb8 don't retry big videos 2019-07-13 11:40:50 +02:00
Niko Storni
1331e5769f actually use the functions I create... 2019-07-13 11:33:40 +02:00
Niko Storni
43f745ae20 improve ip shuffling (prevent ips in use from being dealt) 2019-07-12 23:54:48 +02:00
Niko Storni
8c2a8262e6 fix bugs 2019-07-12 23:20:01 +02:00
Niko Storni
6f486717da fix stop pattern
fix noretry string
fix youtube-dl param
2019-07-12 22:58:34 +02:00
Niko Storni
a3fcd67611 better handle interruptions by user
refactor IP throttling in its own package
2019-07-12 21:32:49 +02:00
Niko Storni
ae1ffb60c5 add ip throttling and better pooling
fix nil pointer dereference
2019-07-12 20:42:44 +02:00
Niko Storni
fa24d83ae9 add max length filter 2019-07-12 02:39:24 +02:00
Niko Storni
2f11d99a97 stop on rare bug/failure 2019-07-12 02:21:27 +02:00
Niko Storni
bab01a6354 prefer IPv4 over IPv6
youtube sucks and they ban /64 subnet blocks on IPv6
2019-07-12 01:54:34 +02:00
Niko Storni
89fe1e99d6 improve ipv6 validation
fix audio merge issue
2019-07-12 01:28:00 +02:00
Niko Storni
ec81af0857 fix download filters 2019-07-12 01:04:45 +02:00
Niko Storni
51cb7f5e34 improve youtube antithrottle
fix logging
2019-07-11 19:14:15 +02:00
Niko Storni
8ad2acc936 remove original downloader
add locking to IP selection to avoid concurrent access
2019-07-11 17:32:46 +02:00
Niko Storni
b8caf3f82b add IPv6 source pooling to bypass youtube throttling 2019-07-11 16:22:58 +02:00
Niko Storni
b34d328cba halt ytsync when youtube throttles us 2019-07-10 22:47:01 +02:00
Niko Storni
a0dcb2216d close db handles 2019-07-10 22:23:57 +02:00
Niko Storni
9d6a7e03cd update prism dependency 2019-07-10 21:07:48 +02:00
Niko Storni
e44e272689 adjust config name 2019-07-10 18:04:05 +02:00
Niko Storni
87f7d4c9f9 recreate directory when emptied 2019-07-10 17:42:28 +02:00
Niko Storni
173e62d286 add check to avoid deleting unreflected blobs 2019-07-10 17:39:58 +02:00
Niko Storni
013760d48d delete blobs after reflection
remove 15 seconds wait after sync
2019-07-10 17:27:46 +02:00
Niko Storni
5dbad2d2e4 add automatic blob reflection
retry videos that fail due to tx issues
2019-07-10 15:46:54 +02:00
Niko Storni
545ab07013 fail channels that should fail 2019-07-05 17:35:20 +02:00
Niko Storni
c402a1cdef add locking to map reads 2019-06-26 14:40:40 -04:00
Niko Storni
a4c6b2da30 actually count UTXOs 2019-06-25 22:28:43 -04:00
Niko Storni
7f48220a80 remove dist folder... 2019-06-26 03:48:44 +02:00
Niko Storni
9e49110295 fix hash 2019-06-26 03:37:52 +02:00