Madiator2011/lbry-sdk
1
0
Fork 0
forked from LBRYCommunity/lbry-sdk
Code Pull requests Activity
9228 commits 111 branches 641 tags 30 MiB
Commit graph

2133 commits

Author SHA1 Message Date
Jonathan Moody
91a6eae831 Fix lint issue in iterative_find.py. 2022-05-20 17:23:39 -04:00
Jonathan Moody
5852fcd287 Don't wait on running_tasks after cancel(). Sometimes a CancelledError exception is received, which is unhelpful, and complicates shutting down the generator. 2022-05-20 17:23:39 -04:00
Jonathan Moody
4767bb9dee Wrap "async for" over IterativeXXXFinder in try/finally ensuring aclose(). 2022-05-20 17:23:39 -04:00
Jonathan Moody
82d7f81f41 Correct call to _aclose() in response to TransportNotConnected. 2022-05-20 17:23:39 -04:00
Jonathan Moody
b036961954 Tighten up IterativeFinder logic to respect max_records better, and wait after task cancel(). 
Also make IterativeFinder a proper AsyncGenerator. This gives it an offically recognized aclose() method and could help with clean finalization.
 2022-05-20 17:23:39 -04:00
Victor Shyba
4ab29c4d5f tests: fix hub url 2022-05-20 16:50:09 -03:00
Victor Shyba
03b0d5e250 tracker client: extract default timeout and concurreny. Bump concurrency to 100 2022-05-11 21:13:30 -03:00
Victor Shyba
629812337b changes from review 2022-05-11 21:13:30 -03:00
Victor Shyba
e54cc8850c return KademliaPeers directly into the queue instead of exposing Announcement abstraction 2022-05-11 21:13:30 -03:00
Victor Shyba
7cba51ca7d update tests, query with port 0, filter bad ports earlier, make unit tests more reliable 2022-05-11 21:13:30 -03:00
Victor Shyba
3dc145fe68 make peer list query trackers 2022-05-11 21:13:30 -03:00
Victor Shyba
7d560df9fd use same arg name as overriden datagram_received (linting) 2022-05-11 21:13:30 -03:00
Victor Shyba
235cc5dc05 results are indexed by ip, setdefault after resolve 2022-05-11 21:13:30 -03:00
Victor Shyba
c276053301 move server implementation to tracker module 2022-05-11 21:13:30 -03:00
Victor Shyba
2e85e29ef1 peer id PREFIX is a constant 2022-05-11 21:13:30 -03:00
Victor Shyba
1169a02c8b make client server updatable from conf 2022-05-11 21:13:30 -03:00
Victor Shyba
a7cea4082e tracker:log DNS errors as warning instead of trace 2022-05-11 21:13:30 -03:00
Victor Shyba
7e6ea97499 make peer id according to BEP20 2022-05-11 21:13:30 -03:00
Victor Shyba
3c46cc4fdd expire connection id quicker as some trackers have it set low 2022-05-11 21:13:30 -03:00
Victor Shyba
6e5c7a1927 use cache_concurrent to avoid requesting the same connection_id multiple times 2022-05-11 21:13:30 -03:00
Victor Shyba
4e09b35012 remove unused import and dead code 2022-05-11 21:13:30 -03:00
Victor Shyba
16a2023bbd stop tasks before removing transport 2022-05-11 21:13:30 -03:00
Victor Shyba
99fc7178c1 better way to batch announce + handle different intervals for different trackers 2022-05-11 21:13:30 -03:00
Victor Shyba
d4aca89a48 handle multiple results from multiple trackers 2022-05-11 21:13:30 -03:00
Victor Shyba
2918d8c7b4 tracker component is running only if the task is alive 2022-05-11 21:13:30 -03:00
Victor Shyba
407c570f8b tests: lower timeout, add test with bad and good mixed 2022-05-11 21:13:30 -03:00
Victor Shyba
0e4f1eae5b reduce timeout to 10, fix lints 2022-05-11 21:13:30 -03:00
Victor Shyba
eccf0e6234 fix reusing result interval from failed expired attempt 2022-05-11 21:13:30 -03:00
Victor Shyba
a3da041412 fix exceptions on shutdown, stop using cancel_tasks 2022-05-11 21:13:30 -03:00
Victor Shyba
2f1617eee4 less verbose on timeouts, dont count timeouts, fix stop 2022-05-11 21:13:30 -03:00
Victor Shyba
05124d41ae only log when really announcing, stop counting cached ones 2022-05-11 21:13:30 -03:00
Victor Shyba
42fd1c962e stop tracker tasks on shutdown 2022-05-11 21:13:30 -03:00
Victor Shyba
47e432b4bb make it less verbose, only log after all events are fired 2022-05-11 21:13:30 -03:00
Victor Shyba
61c99abcf1 avoid readding the same hash when tracker is busy with too many files 2022-05-11 21:13:30 -03:00
Victor Shyba
28fdd62945 move concurreny control to lower layer 2022-05-11 21:13:30 -03:00
Victor Shyba
3855db6c66 pause announcer for 1 minute each round 2022-05-11 21:13:30 -03:00
Victor Shyba
30acde0afc at most 10 announces concurrently 2022-05-11 21:13:30 -03:00
Victor Shyba
2d9c5742c7 cache results, save interval on tracker 2022-05-11 21:13:30 -03:00
Victor Shyba
43e50f7f04 fix subscribe_hash 2022-05-11 21:13:30 -03:00
Victor Shyba
888e9918a6 improve timeout handling 2022-05-11 21:13:30 -03:00
Victor Shyba
9e9a64d989 evented system for tracker announcements 2022-05-11 21:13:30 -03:00
Victor Shyba
7acaecaed2 managed_stream: remove unused imports 2022-05-11 21:13:30 -03:00
Victor Shyba
2344aca146 fix component property 2022-05-11 21:13:30 -03:00
Victor Shyba
7b425eb2ac add tracker announcer component 2022-05-11 21:13:30 -03:00
Victor Shyba
30e8728f7f use tracker on download 2022-05-11 21:13:30 -03:00
Victor Shyba
3989eef84b return whole announcement so the caller knows the interval 2022-05-11 21:13:30 -03:00
Victor Shyba
dc6f8c4fc4 add arg to announce stopped, removing the announcement 2022-05-11 21:13:30 -03:00
Victor Shyba
2df8a1d99d make a helper function to announce 2022-05-11 21:13:30 -03:00
Victor Shyba
4ea858fdd3 add new conf: tracker_servers 2022-05-11 21:13:30 -03:00
Victor Shyba
006391dd26 move udp server to test file, add link to BEP15 2022-05-11 21:13:29 -03:00
Victor Shyba
4a0bf8a702 add torrent udp tracker client, server and tests 2022-05-11 21:13:29 -03:00
Lex Berezhny
8209eafc6b v0.108.0 2022-04-10 23:25:15 -04:00
Lex Berezhny
23ecbc8ebe set the default per character fee for claims to zero 2022-04-08 10:58:02 -04:00
Daniel Krol
eca69391ef Add wallet json-schema, validate in one test. 2022-04-08 10:56:18 -04:00
Lex Berezhny
84ef52cf4d fix redeem scripthash test 2022-04-08 10:11:11 -04:00
Lex Berezhny
8fb14bf713 remove command not available in lbcd 2022-04-08 09:59:22 -04:00
Lex Berezhny
16eb50a291 working jsonrpc_account_deposit 2022-04-08 09:57:15 -04:00
Lex Berezhny
dd503fbb82 set locktime from script 2022-04-08 09:57:15 -04:00
Lex Berezhny
ae79314869 wip 2022-04-08 09:57:15 -04:00
Lex Berezhny
0cbc514a8e account_deposit command added which accepts time locked TXs 2022-04-08 09:57:15 -04:00
Lex Berezhny
5777f3e15c wip 2022-04-08 09:57:15 -04:00
Lex Berezhny
2d20458bc2 re-use existing constraints cleanup function 2022-04-06 09:09:39 -04:00
zeppi
2bd2088248 bugfix 2022-04-06 09:09:39 -04:00
zeppi
5818270803 fix address_list pagination 2022-04-06 09:09:39 -04:00
Victor Shyba
79a5f0e375 lint 2022-04-05 00:35:48 -03:00
AlessandroSpallina
96490fdb15
Merge branch 'master' into master 2022-03-29 13:50:57 +02:00
Victor Shyba
5a0c225c6f v0.107.0 2022-03-28 15:56:06 -03:00
Jack Robison
9faf6e46ca move lbry.wallet.server to new project called scribe 
switch from using lbrycrd to lbcd
 2022-03-27 23:33:26 -04:00
Victor Shyba
200761ff13 make added_on a required parameter on BlobInfo, fix callers 2022-03-24 19:51:48 -03:00
Victor Shyba
cb78e95e3d add missing space on query, typo 2022-03-23 13:40:01 -03:00
AlessandroSpallina
f01cf98d62 fix #3530 added error log when tcp port is already in use 2022-03-22 17:17:41 +01:00
Victor Shyba
c9c2495611 if a blob file exists but is pending on db, fix on startup 2022-03-21 21:58:36 -03:00
Victor Shyba
aac72fa512 fix bug where recovery doesnt update blob status 2022-03-21 21:33:33 -03:00
Victor Shyba
c5e2f19dde fix bug where added_on is always 0 for downloads 2022-03-21 04:38:51 -03:00
Victor Shyba
34bd9e5cb4 exclude sd blobs from calculation and make them be picked last on removal 2022-03-21 04:26:27 -03:00
Lex Berezhny
ca4ba19a5e fixes #3577 2022-03-13 20:42:34 -04:00
Victor Shyba
f05943ff79 implement announcer as a consumer task on gather 2022-03-02 13:00:34 -03:00
Victor Shyba
7ded8a1333 make active an explicit ordered dict 2022-03-02 13:00:34 -03:00
Victor Shyba
c2478d4add remove unused search rounds 2022-03-02 13:00:34 -03:00
Victor Shyba
8b1009161a better representation of kademliapeer on debug logs 2022-03-02 13:00:34 -03:00
Victor Shyba
868a620e91 add a way to wait announcements to finish so tests are reliable 2022-03-02 13:00:34 -03:00
Victor Shyba
a0e34b0bc8 make timeout handler immune to asyncio time tricks 2022-03-02 13:00:34 -03:00
Victor Shyba
612dbcb2f3 allow running some extra probes for k replacements 2022-03-02 13:00:34 -03:00
Victor Shyba
b3614d965d remove all references to bottoming out 2022-03-02 13:00:34 -03:00
Victor Shyba
5d7137255e no stop condition, let it exhaust 2022-03-02 13:00:34 -03:00
Victor Shyba
6ff867ef55 bottoming out is now warning and no results for peer search 2022-03-02 13:00:34 -03:00
Victor Shyba
c14915df29 don't probe peers too far from the top closest 2022-03-02 13:00:34 -03:00
Victor Shyba
7d4966e2ae use a dict for the active queue 2022-03-02 13:00:34 -03:00
Victor Shyba
3876e0317d log bottom out of peer search in debug, show short key id for find value 2022-03-02 13:00:34 -03:00
Victor Shyba
0b2b10f759 bump bottom out limit of peer search so people can use 100 concurrent announcers 2022-03-02 13:00:34 -03:00
Victor Shyba
9a79b33664 wait until k peers are ready. do not double add peers 2022-03-02 13:00:34 -03:00
Victor Shyba
af1a6edd15 only return good (contacted) peers 2022-03-02 13:00:34 -03:00
Victor Shyba
b78929f4d5 reset closest peer on failure 2022-03-02 13:00:34 -03:00
Victor Shyba
fb6e342043 add peers from shortlist regardless, but check from other nodes 2022-03-02 13:00:34 -03:00
Victor Shyba
0faa2d35da bump split index to 2 2022-03-02 13:00:34 -03:00
Victor Shyba
511e57c231 fix distance sorting and improve logging 2022-03-02 13:00:34 -03:00
Victor Shyba
d762d675c4 closest peer is only ready when it was contacted and isn't known to be bad 2022-03-02 13:00:34 -03:00
Victor Shyba
3fdadee87c dont probe and ignore bad peers 2022-03-02 13:00:34 -03:00
Victor Shyba
1aa4d9d585 simplify, genaralize to any size and fix tests 2022-02-28 13:06:51 -03:00
Victor Shyba
8019f4bdb3 stop after finding what to download 2022-02-28 13:06:51 -03:00
Victor Shyba
ca65c1ebc5 replace duplicated code 2022-02-28 13:06:51 -03:00
Victor Shyba
f0e47aae86 add get_colliding_prefix_bits, docs and tests 2022-02-28 13:06:51 -03:00
Victor Shyba
dc7cd545ba extract method and avoid using hash builtin name 2022-02-28 13:06:51 -03:00
Victor Shyba
76bd59d82e extract min_prefix_colliding_bits to a contanst 2022-02-28 13:06:51 -03:00
Victor Shyba
461687ffb4 check that the stored blob is at least 1 prefix byte close to peer id 2022-02-28 13:06:51 -03:00
Victor Shyba
dd5b9ca81b add migrator to set head blobs should_announce=0 2022-02-20 22:33:57 -03:00
Victor Shyba
ec0d9f06c5 do not search for the head blob 2022-02-20 22:33:57 -03:00
Victor Shyba
03b59ac6fc dont set head blob to announce on save 2022-02-20 22:33:57 -03:00
Victor Shyba
43ac3336d7 break tie by length 2022-02-20 22:24:04 -03:00
Victor Shyba
d12c78db74 fix and test case for blob_clean after disabling network storage 2022-02-20 22:24:04 -03:00
Victor Shyba
bb60c385d5 put back all the peers, get rid of re_add 2022-02-08 21:41:52 -03:00
Alex Grintsvayg
7c7a0d4bdf
let stream_update work on non-stream claims 2022-02-08 09:28:17 -05:00
Jeffrey Picard
e0ea6383e2
Update __init__.py 
Update go hub binary to fix es sync test.
 2022-02-04 12:17:19 -05:00
Lex Berezhny
cba9c16a06 fix 2022-02-04 12:07:41 -05:00
Lex Berezhny
dd68fb077b prevent creation of change which is below the dust threshold of 1000 dewies 2022-02-04 12:07:41 -05:00
Victor Shyba
c0f512ace7 bump DHT peer manager cache to 16384 2022-02-02 16:54:42 -03:00
Victor Shyba
c9d637b4da add gauge for queue size 2022-02-02 11:56:42 -03:00
Victor Shyba
ae3e8fadf5 count announcements and how many peers we were able to announce to 2022-02-02 11:56:42 -03:00
Eugene Dubinin
9b463a8cab adds tests for guess_media_type 
removes unnecessary comments
 2022-01-29 20:49:42 +02:00
Eugene Dubinin
babc54a240 adjusts code style 2022-01-29 15:25:17 +02:00
Eugene Dubinin
5836a93b21 fixes KeyError on missing synonyms 2022-01-29 15:25:17 +02:00
Eugene Dubinin
557348e345 detect media_type from the file contents 2022-01-29 15:25:17 +02:00
Lex Berezhny
3a496902f8 wallet locking/unlocking no longer breaks deterministic channel keys 2022-01-24 09:45:08 -05:00
Lex Berezhny
302461b446 updated based on code review 2022-01-17 11:08:28 -05:00
Lex Berezhny
ac201c718e drop dependency on cryptography library in wallet module 2022-01-17 10:43:59 -05:00
Victor Shyba
0618053bd4 remove request_flight metric 2022-01-12 12:41:04 -03:00
Victor Shyba
fd9dcbf9a8 add granular metric for stored blob prefix, for network announcements calculation 2022-01-12 12:39:23 -03:00
Victor Shyba
beb8583436 change colliding bits metric to gauge 2022-01-12 12:39:23 -03:00
Victor Shyba
b44e2c0b38 count bit collisions between 8 and 16 2022-01-12 12:39:23 -03:00
Victor Shyba
06e94640b5 add counter for peers with colliding bytes 2022-01-12 12:39:23 -03:00
Victor Shyba
ff36bdc802 add requests in flight and error 2022-01-12 12:39:23 -03:00
Victor Shyba
46f576de46 add request received 2022-01-12 12:39:23 -03:00
Victor Shyba
7b09c34fce add request_sent and request_time metric on dht 2022-01-12 12:39:23 -03:00
Victor Shyba
a22f50aa84 add storing_peers and peer_manager_keys 2022-01-12 12:39:23 -03:00
Victor Shyba
2d9130b4e0 prometheus: move blobs_stored and peers to SDK. add buckets_in_routing_table 2022-01-12 12:39:23 -03:00
Victor Shyba
5cb4c06d0c add prefix_neighbors_count to routing table debug api 2022-01-12 12:39:23 -03:00
Lex Berezhny
aaa11c02bf added integration test 2022-01-10 08:46:10 -05:00
vertbyqb
d2ebbf5db6 jsonrpc_channel_sign - Convert hexdata to a string before signing 
Fixes #3533
 2022-01-10 08:46:10 -05:00
Victor Shyba
a8523996a9 extract cache values, increase peer cache to 2048 2022-01-07 12:58:52 -03:00
Victor Shyba
f586de2bbe DHT bugfix: failures tracking should be bound to 2048 LRU cache size 2022-01-07 12:46:00 -03:00
Victor Shyba
7df02303b2 fix missing docopt argument 2022-01-05 17:10:31 -03:00
Victor Shyba
f89c75e642 bump hub version to latest supporting sd_hash search 2022-01-05 17:10:31 -03:00
Victor Shyba
d2c1961101 update hub protobuf including sd_hash field 2022-01-05 17:10:31 -03:00
Victor Shyba
2a4c5a48bf increase indexed sd_hash prefix to 4 chars 2022-01-05 17:10:31 -03:00
Victor Shyba
5f5f39a4aa enable and test prefix search for sd hash 2022-01-05 17:10:31 -03:00
Victor Shyba
df54cc04af sync and search sd_hash 2022-01-05 17:10:31 -03:00
Victor Shyba
19fa274227 add sd hash to API 2022-01-05 17:10:31 -03:00
Lex Berezhny
4822792ee2 create nondetermnistic channel in test to replicate old test behavior 2021-12-22 18:29:46 -05:00
Lex Berezhny
23c10faff5 lint 2021-12-22 18:29:46 -05:00
Lex Berezhny
1eaa195363 reduced crypto dependency in wallet to coincurve 2021-12-22 18:29:46 -05:00