Madiator2011/lbry-sdk
1
0
Fork 0
forked from LBRYCommunity/lbry-sdk
Code Pull requests Activity
9216 commits 111 branches 641 tags 30 MiB
Commit graph

2129 commits

Author SHA1 Message Date
Jonathan Moody
b036961954 Tighten up IterativeFinder logic to respect max_records better, and wait after task cancel(). 
Also make IterativeFinder a proper AsyncGenerator. This gives it an offically recognized aclose() method and could help with clean finalization.
 2022-05-20 17:23:39 -04:00
Victor Shyba
4ab29c4d5f tests: fix hub url 2022-05-20 16:50:09 -03:00
Victor Shyba
03b0d5e250 tracker client: extract default timeout and concurreny. Bump concurrency to 100 2022-05-11 21:13:30 -03:00
Victor Shyba
629812337b changes from review 2022-05-11 21:13:30 -03:00
Victor Shyba
e54cc8850c return KademliaPeers directly into the queue instead of exposing Announcement abstraction 2022-05-11 21:13:30 -03:00
Victor Shyba
7cba51ca7d update tests, query with port 0, filter bad ports earlier, make unit tests more reliable 2022-05-11 21:13:30 -03:00
Victor Shyba
3dc145fe68 make peer list query trackers 2022-05-11 21:13:30 -03:00
Victor Shyba
7d560df9fd use same arg name as overriden datagram_received (linting) 2022-05-11 21:13:30 -03:00
Victor Shyba
235cc5dc05 results are indexed by ip, setdefault after resolve 2022-05-11 21:13:30 -03:00
Victor Shyba
c276053301 move server implementation to tracker module 2022-05-11 21:13:30 -03:00
Victor Shyba
2e85e29ef1 peer id PREFIX is a constant 2022-05-11 21:13:30 -03:00
Victor Shyba
1169a02c8b make client server updatable from conf 2022-05-11 21:13:30 -03:00
Victor Shyba
a7cea4082e tracker:log DNS errors as warning instead of trace 2022-05-11 21:13:30 -03:00
Victor Shyba
7e6ea97499 make peer id according to BEP20 2022-05-11 21:13:30 -03:00
Victor Shyba
3c46cc4fdd expire connection id quicker as some trackers have it set low 2022-05-11 21:13:30 -03:00
Victor Shyba
6e5c7a1927 use cache_concurrent to avoid requesting the same connection_id multiple times 2022-05-11 21:13:30 -03:00
Victor Shyba
4e09b35012 remove unused import and dead code 2022-05-11 21:13:30 -03:00
Victor Shyba
16a2023bbd stop tasks before removing transport 2022-05-11 21:13:30 -03:00
Victor Shyba
99fc7178c1 better way to batch announce + handle different intervals for different trackers 2022-05-11 21:13:30 -03:00
Victor Shyba
d4aca89a48 handle multiple results from multiple trackers 2022-05-11 21:13:30 -03:00
Victor Shyba
2918d8c7b4 tracker component is running only if the task is alive 2022-05-11 21:13:30 -03:00
Victor Shyba
407c570f8b tests: lower timeout, add test with bad and good mixed 2022-05-11 21:13:30 -03:00
Victor Shyba
0e4f1eae5b reduce timeout to 10, fix lints 2022-05-11 21:13:30 -03:00
Victor Shyba
eccf0e6234 fix reusing result interval from failed expired attempt 2022-05-11 21:13:30 -03:00
Victor Shyba
a3da041412 fix exceptions on shutdown, stop using cancel_tasks 2022-05-11 21:13:30 -03:00
Victor Shyba
2f1617eee4 less verbose on timeouts, dont count timeouts, fix stop 2022-05-11 21:13:30 -03:00
Victor Shyba
05124d41ae only log when really announcing, stop counting cached ones 2022-05-11 21:13:30 -03:00
Victor Shyba
42fd1c962e stop tracker tasks on shutdown 2022-05-11 21:13:30 -03:00
Victor Shyba
47e432b4bb make it less verbose, only log after all events are fired 2022-05-11 21:13:30 -03:00
Victor Shyba
61c99abcf1 avoid readding the same hash when tracker is busy with too many files 2022-05-11 21:13:30 -03:00
Victor Shyba
28fdd62945 move concurreny control to lower layer 2022-05-11 21:13:30 -03:00
Victor Shyba
3855db6c66 pause announcer for 1 minute each round 2022-05-11 21:13:30 -03:00
Victor Shyba
30acde0afc at most 10 announces concurrently 2022-05-11 21:13:30 -03:00
Victor Shyba
2d9c5742c7 cache results, save interval on tracker 2022-05-11 21:13:30 -03:00
Victor Shyba
43e50f7f04 fix subscribe_hash 2022-05-11 21:13:30 -03:00
Victor Shyba
888e9918a6 improve timeout handling 2022-05-11 21:13:30 -03:00
Victor Shyba
9e9a64d989 evented system for tracker announcements 2022-05-11 21:13:30 -03:00
Victor Shyba
7acaecaed2 managed_stream: remove unused imports 2022-05-11 21:13:30 -03:00
Victor Shyba
2344aca146 fix component property 2022-05-11 21:13:30 -03:00
Victor Shyba
7b425eb2ac add tracker announcer component 2022-05-11 21:13:30 -03:00
Victor Shyba
30e8728f7f use tracker on download 2022-05-11 21:13:30 -03:00
Victor Shyba
3989eef84b return whole announcement so the caller knows the interval 2022-05-11 21:13:30 -03:00
Victor Shyba
dc6f8c4fc4 add arg to announce stopped, removing the announcement 2022-05-11 21:13:30 -03:00
Victor Shyba
2df8a1d99d make a helper function to announce 2022-05-11 21:13:30 -03:00
Victor Shyba
4ea858fdd3 add new conf: tracker_servers 2022-05-11 21:13:30 -03:00
Victor Shyba
006391dd26 move udp server to test file, add link to BEP15 2022-05-11 21:13:29 -03:00
Victor Shyba
4a0bf8a702 add torrent udp tracker client, server and tests 2022-05-11 21:13:29 -03:00
Lex Berezhny
8209eafc6b v0.108.0 2022-04-10 23:25:15 -04:00
Lex Berezhny
23ecbc8ebe set the default per character fee for claims to zero 2022-04-08 10:58:02 -04:00
Daniel Krol
eca69391ef Add wallet json-schema, validate in one test. 2022-04-08 10:56:18 -04:00
Lex Berezhny
84ef52cf4d fix redeem scripthash test 2022-04-08 10:11:11 -04:00
Lex Berezhny
8fb14bf713 remove command not available in lbcd 2022-04-08 09:59:22 -04:00
Lex Berezhny
16eb50a291 working jsonrpc_account_deposit 2022-04-08 09:57:15 -04:00
Lex Berezhny
dd503fbb82 set locktime from script 2022-04-08 09:57:15 -04:00
Lex Berezhny
ae79314869 wip 2022-04-08 09:57:15 -04:00
Lex Berezhny
0cbc514a8e account_deposit command added which accepts time locked TXs 2022-04-08 09:57:15 -04:00
Lex Berezhny
5777f3e15c wip 2022-04-08 09:57:15 -04:00
Lex Berezhny
2d20458bc2 re-use existing constraints cleanup function 2022-04-06 09:09:39 -04:00
zeppi
2bd2088248 bugfix 2022-04-06 09:09:39 -04:00
zeppi
5818270803 fix address_list pagination 2022-04-06 09:09:39 -04:00
Victor Shyba
79a5f0e375 lint 2022-04-05 00:35:48 -03:00
AlessandroSpallina
96490fdb15
Merge branch 'master' into master 2022-03-29 13:50:57 +02:00
Victor Shyba
5a0c225c6f v0.107.0 2022-03-28 15:56:06 -03:00
Jack Robison
9faf6e46ca move lbry.wallet.server to new project called scribe 
switch from using lbrycrd to lbcd
 2022-03-27 23:33:26 -04:00
Victor Shyba
200761ff13 make added_on a required parameter on BlobInfo, fix callers 2022-03-24 19:51:48 -03:00
Victor Shyba
cb78e95e3d add missing space on query, typo 2022-03-23 13:40:01 -03:00
AlessandroSpallina
f01cf98d62 fix #3530 added error log when tcp port is already in use 2022-03-22 17:17:41 +01:00
Victor Shyba
c9c2495611 if a blob file exists but is pending on db, fix on startup 2022-03-21 21:58:36 -03:00
Victor Shyba
aac72fa512 fix bug where recovery doesnt update blob status 2022-03-21 21:33:33 -03:00
Victor Shyba
c5e2f19dde fix bug where added_on is always 0 for downloads 2022-03-21 04:38:51 -03:00
Victor Shyba
34bd9e5cb4 exclude sd blobs from calculation and make them be picked last on removal 2022-03-21 04:26:27 -03:00
Lex Berezhny
ca4ba19a5e fixes #3577 2022-03-13 20:42:34 -04:00
Victor Shyba
f05943ff79 implement announcer as a consumer task on gather 2022-03-02 13:00:34 -03:00
Victor Shyba
7ded8a1333 make active an explicit ordered dict 2022-03-02 13:00:34 -03:00
Victor Shyba
c2478d4add remove unused search rounds 2022-03-02 13:00:34 -03:00
Victor Shyba
8b1009161a better representation of kademliapeer on debug logs 2022-03-02 13:00:34 -03:00
Victor Shyba
868a620e91 add a way to wait announcements to finish so tests are reliable 2022-03-02 13:00:34 -03:00
Victor Shyba
a0e34b0bc8 make timeout handler immune to asyncio time tricks 2022-03-02 13:00:34 -03:00
Victor Shyba
612dbcb2f3 allow running some extra probes for k replacements 2022-03-02 13:00:34 -03:00
Victor Shyba
b3614d965d remove all references to bottoming out 2022-03-02 13:00:34 -03:00
Victor Shyba
5d7137255e no stop condition, let it exhaust 2022-03-02 13:00:34 -03:00
Victor Shyba
6ff867ef55 bottoming out is now warning and no results for peer search 2022-03-02 13:00:34 -03:00
Victor Shyba
c14915df29 don't probe peers too far from the top closest 2022-03-02 13:00:34 -03:00
Victor Shyba
7d4966e2ae use a dict for the active queue 2022-03-02 13:00:34 -03:00
Victor Shyba
3876e0317d log bottom out of peer search in debug, show short key id for find value 2022-03-02 13:00:34 -03:00
Victor Shyba
0b2b10f759 bump bottom out limit of peer search so people can use 100 concurrent announcers 2022-03-02 13:00:34 -03:00
Victor Shyba
9a79b33664 wait until k peers are ready. do not double add peers 2022-03-02 13:00:34 -03:00
Victor Shyba
af1a6edd15 only return good (contacted) peers 2022-03-02 13:00:34 -03:00
Victor Shyba
b78929f4d5 reset closest peer on failure 2022-03-02 13:00:34 -03:00
Victor Shyba
fb6e342043 add peers from shortlist regardless, but check from other nodes 2022-03-02 13:00:34 -03:00
Victor Shyba
0faa2d35da bump split index to 2 2022-03-02 13:00:34 -03:00
Victor Shyba
511e57c231 fix distance sorting and improve logging 2022-03-02 13:00:34 -03:00
Victor Shyba
d762d675c4 closest peer is only ready when it was contacted and isn't known to be bad 2022-03-02 13:00:34 -03:00
Victor Shyba
3fdadee87c dont probe and ignore bad peers 2022-03-02 13:00:34 -03:00
Victor Shyba
1aa4d9d585 simplify, genaralize to any size and fix tests 2022-02-28 13:06:51 -03:00
Victor Shyba
8019f4bdb3 stop after finding what to download 2022-02-28 13:06:51 -03:00
Victor Shyba
ca65c1ebc5 replace duplicated code 2022-02-28 13:06:51 -03:00
Victor Shyba
f0e47aae86 add get_colliding_prefix_bits, docs and tests 2022-02-28 13:06:51 -03:00
Victor Shyba
dc7cd545ba extract method and avoid using hash builtin name 2022-02-28 13:06:51 -03:00
Victor Shyba
76bd59d82e extract min_prefix_colliding_bits to a contanst 2022-02-28 13:06:51 -03:00
Victor Shyba
461687ffb4 check that the stored blob is at least 1 prefix byte close to peer id 2022-02-28 13:06:51 -03:00
Victor Shyba
dd5b9ca81b add migrator to set head blobs should_announce=0 2022-02-20 22:33:57 -03:00
Victor Shyba
ec0d9f06c5 do not search for the head blob 2022-02-20 22:33:57 -03:00
Victor Shyba
03b59ac6fc dont set head blob to announce on save 2022-02-20 22:33:57 -03:00
Victor Shyba
43ac3336d7 break tie by length 2022-02-20 22:24:04 -03:00
Victor Shyba
d12c78db74 fix and test case for blob_clean after disabling network storage 2022-02-20 22:24:04 -03:00
Victor Shyba
bb60c385d5 put back all the peers, get rid of re_add 2022-02-08 21:41:52 -03:00
Alex Grintsvayg
7c7a0d4bdf
let stream_update work on non-stream claims 2022-02-08 09:28:17 -05:00
Jeffrey Picard
e0ea6383e2
Update __init__.py 
Update go hub binary to fix es sync test.
 2022-02-04 12:17:19 -05:00
Lex Berezhny
cba9c16a06 fix 2022-02-04 12:07:41 -05:00
Lex Berezhny
dd68fb077b prevent creation of change which is below the dust threshold of 1000 dewies 2022-02-04 12:07:41 -05:00
Victor Shyba
c0f512ace7 bump DHT peer manager cache to 16384 2022-02-02 16:54:42 -03:00
Victor Shyba
c9d637b4da add gauge for queue size 2022-02-02 11:56:42 -03:00
Victor Shyba
ae3e8fadf5 count announcements and how many peers we were able to announce to 2022-02-02 11:56:42 -03:00
Eugene Dubinin
9b463a8cab adds tests for guess_media_type 
removes unnecessary comments
 2022-01-29 20:49:42 +02:00
Eugene Dubinin
babc54a240 adjusts code style 2022-01-29 15:25:17 +02:00
Eugene Dubinin
5836a93b21 fixes KeyError on missing synonyms 2022-01-29 15:25:17 +02:00
Eugene Dubinin
557348e345 detect media_type from the file contents 2022-01-29 15:25:17 +02:00
Lex Berezhny
3a496902f8 wallet locking/unlocking no longer breaks deterministic channel keys 2022-01-24 09:45:08 -05:00
Lex Berezhny
302461b446 updated based on code review 2022-01-17 11:08:28 -05:00
Lex Berezhny
ac201c718e drop dependency on cryptography library in wallet module 2022-01-17 10:43:59 -05:00
Victor Shyba
0618053bd4 remove request_flight metric 2022-01-12 12:41:04 -03:00
Victor Shyba
fd9dcbf9a8 add granular metric for stored blob prefix, for network announcements calculation 2022-01-12 12:39:23 -03:00
Victor Shyba
beb8583436 change colliding bits metric to gauge 2022-01-12 12:39:23 -03:00
Victor Shyba
b44e2c0b38 count bit collisions between 8 and 16 2022-01-12 12:39:23 -03:00
Victor Shyba
06e94640b5 add counter for peers with colliding bytes 2022-01-12 12:39:23 -03:00
Victor Shyba
ff36bdc802 add requests in flight and error 2022-01-12 12:39:23 -03:00
Victor Shyba
46f576de46 add request received 2022-01-12 12:39:23 -03:00
Victor Shyba
7b09c34fce add request_sent and request_time metric on dht 2022-01-12 12:39:23 -03:00
Victor Shyba
a22f50aa84 add storing_peers and peer_manager_keys 2022-01-12 12:39:23 -03:00
Victor Shyba
2d9130b4e0 prometheus: move blobs_stored and peers to SDK. add buckets_in_routing_table 2022-01-12 12:39:23 -03:00
Victor Shyba
5cb4c06d0c add prefix_neighbors_count to routing table debug api 2022-01-12 12:39:23 -03:00
Lex Berezhny
aaa11c02bf added integration test 2022-01-10 08:46:10 -05:00
vertbyqb
d2ebbf5db6 jsonrpc_channel_sign - Convert hexdata to a string before signing 
Fixes #3533
 2022-01-10 08:46:10 -05:00
Victor Shyba
a8523996a9 extract cache values, increase peer cache to 2048 2022-01-07 12:58:52 -03:00
Victor Shyba
f586de2bbe DHT bugfix: failures tracking should be bound to 2048 LRU cache size 2022-01-07 12:46:00 -03:00
Victor Shyba
7df02303b2 fix missing docopt argument 2022-01-05 17:10:31 -03:00
Victor Shyba
f89c75e642 bump hub version to latest supporting sd_hash search 2022-01-05 17:10:31 -03:00
Victor Shyba
d2c1961101 update hub protobuf including sd_hash field 2022-01-05 17:10:31 -03:00
Victor Shyba
2a4c5a48bf increase indexed sd_hash prefix to 4 chars 2022-01-05 17:10:31 -03:00
Victor Shyba
5f5f39a4aa enable and test prefix search for sd hash 2022-01-05 17:10:31 -03:00
Victor Shyba
df54cc04af sync and search sd_hash 2022-01-05 17:10:31 -03:00
Victor Shyba
19fa274227 add sd hash to API 2022-01-05 17:10:31 -03:00
Lex Berezhny
4822792ee2 create nondetermnistic channel in test to replicate old test behavior 2021-12-22 18:29:46 -05:00
Lex Berezhny
23c10faff5 lint 2021-12-22 18:29:46 -05:00
Lex Berezhny
1eaa195363 reduced crypto dependency in wallet to coincurve 2021-12-22 18:29:46 -05:00
Lex Berezhny
fb57cfa5d8 moved imports for lint 2021-12-22 18:29:46 -05:00
Lex Berezhny
d815a6f02c use ecdsa for signing/veryfing instead of coincurve due to compatibility issues 2021-12-22 18:29:46 -05:00
Lex Berezhny
8216f4a873 work in progress 2021-12-22 18:29:46 -05:00
Lex Berezhny
e4cc4521d9 channel key generation no longer arbitrarily bounded 2021-12-22 18:29:46 -05:00