Commit graph

8864 commits

Author SHA1 Message Date
Victor Shyba
cf73e4599f Merge pull request #3585 from lbryio/fix_blob_db_queries
Fixes bugs on disk space management and stream recovery
2022-03-24 21:05:09 -03:00
Victor Shyba
2a698932da implement announcer as a consumer task on gather 2022-03-15 15:36:00 -03:00
Victor Shyba
ee36162b16 make active an explicit ordered dict 2022-03-15 15:36:00 -03:00
Victor Shyba
97f472d4e1 remove unused search rounds 2022-03-15 15:36:00 -03:00
Victor Shyba
160e227e90 timeout is now supported on dht tests 2022-03-15 15:36:00 -03:00
Victor Shyba
475ba010e3 fix and enable test_blob_announcer 2022-03-15 15:36:00 -03:00
Victor Shyba
57ebbbcb78 simplify dht mock and restore clock after accelerating 2022-03-15 15:36:00 -03:00
Victor Shyba
cbf973bf2d better representation of kademliapeer on debug logs 2022-03-15 15:36:00 -03:00
Victor Shyba
9d26ad96c6 add a way to wait announcements to finish so tests are reliable 2022-03-15 15:36:00 -03:00
Victor Shyba
c2a3ec3265 make timeout handler immune to asyncio time tricks 2022-03-15 15:36:00 -03:00
Victor Shyba
521d783260 allow running some extra probes for k replacements 2022-03-15 15:36:00 -03:00
Victor Shyba
50846cd37e remove all references to bottoming out 2022-03-15 15:36:00 -03:00
Victor Shyba
40ed5f311b no stop condition, let it exhaust 2022-03-15 15:36:00 -03:00
Victor Shyba
293410ac9f bottoming out is now warning and no results for peer search 2022-03-15 15:36:00 -03:00
Victor Shyba
068f46d137 don't probe peers too far from the top closest 2022-03-15 15:36:00 -03:00
Victor Shyba
5c60c09fef use a dict for the active queue 2022-03-15 15:36:00 -03:00
Victor Shyba
aca4a243d6 log bottom out of peer search in debug, show short key id for find value 2022-03-15 15:36:00 -03:00
Victor Shyba
422f292d82 bump bottom out limit of peer search so people can use 100 concurrent announcers 2022-03-15 15:36:00 -03:00
Victor Shyba
c81598aab8 wait until k peers are ready. do not double add peers 2022-03-15 15:36:00 -03:00
Victor Shyba
3f8c8e01c6 only return good (contacted) peers 2022-03-15 15:36:00 -03:00
Victor Shyba
8d90653395 reset closest peer on failure 2022-03-15 15:36:00 -03:00
Victor Shyba
01841694ad add peers from shortlist regardless, but check from other nodes 2022-03-15 15:36:00 -03:00
Victor Shyba
c3d0dd3073 bump split index to 2 2022-03-15 15:36:00 -03:00
Victor Shyba
4851f5c300 fix distance sorting and improve logging 2022-03-15 15:36:00 -03:00
Victor Shyba
aef5f11a18 closest peer is only ready when it was contacted and isn't known to be bad 2022-03-15 15:36:00 -03:00
Victor Shyba
700f1cb8e5 dont probe and ignore bad peers 2022-03-15 15:36:00 -03:00
Victor Shyba
7e5f7c1725 simplify, genaralize to any size and fix tests 2022-03-15 15:36:00 -03:00
Victor Shyba
aa7a17b112 stop after finding what to download 2022-03-15 15:36:00 -03:00
Victor Shyba
838fd71538 replace duplicated code 2022-03-15 15:36:00 -03:00
Victor Shyba
0146044b90 add get_colliding_prefix_bits, docs and tests 2022-03-15 15:36:00 -03:00
Victor Shyba
f8f73ed4a6 extract method and avoid using hash builtin name 2022-03-15 15:36:00 -03:00
Victor Shyba
74bf8c551b extract min_prefix_colliding_bits to a contanst 2022-03-15 15:36:00 -03:00
Victor Shyba
4be346d0bb check that the stored blob is at least 1 prefix byte close to peer id 2022-03-15 15:36:00 -03:00
Victor Shyba
4fd1fe0046 add migrator to set head blobs should_announce=0 2022-03-15 15:36:00 -03:00
Victor Shyba
9524f03eac fix test_announces 2022-03-15 15:36:00 -03:00
Victor Shyba
90c8a8bea3 do not search for the head blob 2022-03-15 15:36:00 -03:00
Victor Shyba
154c41bd8d dont set head blob to announce on save 2022-03-15 15:36:00 -03:00
Victor Shyba
f271511484 break tie by length 2022-03-15 15:36:00 -03:00
Victor Shyba
dde4a5b495 fix and test case for blob_clean after disabling network storage 2022-03-15 15:36:00 -03:00
Victor Shyba
68a940f3a6 put back all the peers, get rid of re_add 2022-03-15 15:36:00 -03:00
Alex Grintsvayg
8c773151a3 let stream_update work on non-stream claims 2022-03-15 15:36:00 -03:00
Jeffrey Picard
f1f5276a04 Update __init__.py
Update go hub binary to fix es sync test.
2022-03-15 15:36:00 -03:00
Lex Berezhny
51490b36e6 fix 2022-03-15 15:36:00 -03:00
Lex Berezhny
e00dc78e1f prevent creation of change which is below the dust threshold of 1000 dewies 2022-03-15 15:36:00 -03:00
Victor Shyba
72beb02ec1 bump DHT peer manager cache to 16384 2022-03-15 15:36:00 -03:00
Victor Shyba
b13c346418 add gauge for queue size 2022-03-15 15:36:00 -03:00
Victor Shyba
2520465fc1 count announcements and how many peers we were able to announce to 2022-03-15 15:36:00 -03:00
Eugene Dubinin
dd868d5148 adds tests for guess_media_type
removes unnecessary comments
2022-03-15 15:36:00 -03:00
Eugene Dubinin
e90fb9a28f adjusts code style 2022-03-15 15:36:00 -03:00
Eugene Dubinin
96d1745232 fixes KeyError on missing synonyms 2022-03-15 15:36:00 -03:00