Madiator2011/lbry-sdk
1
0
Fork 0
forked from LBRYCommunity/lbry-sdk
Code Pull requests Activity
5165 commits 111 branches 641 tags 30 MiB
Commit graph

3563 commits

Author SHA1 Message Date
Jack Robison
868110a6f2
populate stream manager with a single batch query 2019-02-08 16:06:35 -05:00
Victor Shyba
9ecec52412 check lock acquired to prevent races 2019-02-08 04:39:28 -03:00
Victor Shyba
8b25acff6e do not wait blob to be written, log errors from races 2019-02-08 04:32:53 -03:00
Victor Shyba
e2b06677b5 defer db insert during downloads 2019-02-08 03:45:23 -03:00
Victor Shyba
af1619ebfb disable race limiting for now 2019-02-08 03:32:38 -03:00
Victor Shyba
3352e0e4f4 score peers by speed 2019-02-08 03:08:08 -03:00
Victor Shyba
141d68a2cd fix write race condition 2019-02-08 02:39:29 -03:00
Victor Shyba
c06ec6cd69 manage connections, reusing them when possible 2019-02-08 02:27:58 -03:00
Victor Shyba
1be5dce30e stop racing during long streams 2019-02-08 01:28:03 -03:00
Victor Shyba
73da223ae1 conf: give up first contact earlier, but let the transfer go longer 2019-02-08 01:05:58 -03:00
Victor Shyba
2d7eb83518 change score calculation, wait for active peers too, simplify peer sorting/keeping 2019-02-08 01:04:38 -03:00
Victor Shyba
ad03f91d24 let cancelled errors flow 2019-02-08 01:03:13 -03:00
Victor Shyba
5586a226c2 bypass parser during download 2019-02-07 21:19:31 -03:00
Victor Shyba
a616582733 wait on connection tasks 2019-02-07 20:11:57 -03:00
Victor Shyba
43ac928f0b remove dht locks 2019-02-07 20:11:57 -03:00
Lex Berezhny
a121110743 v0.31.0 2019-02-06 14:28:40 -05:00
Victor Shyba
3a15ff4bcd test incomplete streams, respect real length, delete on incomplete assembly 2019-02-06 15:56:06 -03:00
Jack Robison
dfdc8eda00
improve test, fix stream_finished_event 2019-02-06 13:32:50 -05:00
Jack Robison
58f6cb71c6
test stream manager 2019-02-06 13:08:57 -05:00
Jack Robison
ae11c5bb4b
fix setting finished event 2019-02-06 12:18:24 -05:00
Jack Robison
f22b5da170
fix raised error for a peer address mismatch 2019-02-06 12:11:22 -05:00
Jack Robison
c1c6d5bc99
fix deleting partial downloads when stopped and previous streams when updating a publish 2019-02-06 11:43:05 -05:00
Jack Robison
0e972ec2ae
refactor BlobFile.close to be non async 2019-02-06 11:00:57 -05:00
Victor Shyba
8b5803c945 format error dict on get 2019-02-06 11:55:12 -03:00
Jack Robison
76c8447a56
add stream descriptor tests 2019-02-05 21:25:31 -05:00
Jack Robison
7c8ad0fe30
fix deleting incomplete downloads upon stop 
fix deleting previous streams (not assembled/published files) for a claim being updated
 2019-02-05 15:33:08 -05:00
Jack Robison
073ffdd049
proto and type args 2019-02-05 13:36:25 -05:00
Jack Robison
7fe211d134
fix resolve_host 2019-02-05 13:31:57 -05:00
Victor Shyba
a4ced4dbd2 set verified after callback 2019-02-05 13:17:00 -03:00
Victor Shyba
c6b910a5f3 use time_getter on storage 2019-02-05 13:14:59 -03:00
Victor Shyba
63d1ec3c28 fixes from jack comments 2019-02-05 13:01:12 -03:00
Victor Shyba
7b8f42e6d0 remove dead set_should_announce code 2019-02-05 13:01:12 -03:00
Victor Shyba
2c275efa48 remove redundant calls to should announce 2019-02-05 13:01:12 -03:00
Victor Shyba
6ead932ccb announces naturally, removes redundant calls 2019-02-05 13:01:12 -03:00
Victor Shyba
f766dbefe0 announce and set completed after inserts 2019-02-05 13:01:12 -03:00
Jack Robison
0685bc5689
include torba in log file 2019-02-04 18:25:47 -05:00
Jack Robison
8cda3143f2
fix setting should_announce and blob status 2019-02-04 18:22:44 -05:00
Jack Robison
5af351a234
reflector logging 2019-02-04 17:33:25 -05:00
Victor Shyba
b82d2c3120 handle and log if downloading from bad sendfile implementations 2019-02-04 19:13:31 -03:00
Jack Robison
2bce7cecfa
fix sendfile on windows 2019-02-04 16:48:48 -05:00
Jack Robison
8883587cc7
logging 2019-02-04 16:19:27 -05:00
Jack Robison
c85991704d
catch invalid data error 2019-02-04 16:19:27 -05:00
Jack Robison
377442d9f0
fix request_blob cancellation 2019-02-04 16:19:24 -05:00
Jack Robison
f0b49fc153
always return close peers from findValue 2019-02-04 15:07:42 -05:00
Jack Robison
ae1162f5b0
don't try downloading from oneself 
-don't return the querying peer in results for findValue
 2019-02-04 14:50:53 -05:00
Victor Shyba
ad71a9a89d refactor reflect to close protocol 2019-02-04 13:34:18 -03:00
Jack Robison
13b3e4ba8e v0.31.0rc5 2019-02-04 10:22:43 -05:00
Victor Shyba
736a949cb5 resume streams on startup 2019-02-03 20:49:09 -03:00
Victor Shyba
11bb1cc7ee use with clause to avoid leaking fds 2019-02-03 20:41:47 -03:00
Jack Robison
dcd78bd86e
always delete file from downloads if download was not complete 2019-02-03 16:43:40 -05:00
Jack Robison
e5af041627
fix connection status message 2019-02-03 16:20:52 -05:00
Jack Robison
0a027cf2de
logging 2019-02-03 15:47:19 -05:00
Jack Robison
3f98b16800
fix blob announce time, use time.time instead of loop.time 2019-02-03 15:47:15 -05:00
Jack Robison
8f5a8ab8ba
fix peer_ping 2019-02-02 02:02:19 -05:00
Jack Robison
91e581422c v0.31.0rc4 2019-02-02 00:30:57 -05:00
Jack Robison
f2968aab22
add re-reflect task to stream manager, add concurrent_reflector_uploads to config 2019-02-01 22:59:41 -05:00
Jack Robison
ccbe8aa0eb
create data. download, and wallet directories if they don't exist on startup 2019-02-01 22:45:28 -05:00
Jack Robison
478f79353a
fix bytes error 2019-02-01 22:20:11 -05:00
Jack Robison
c3b0151943
fix dht status error during startup 2019-02-01 22:12:07 -05:00
Jack Robison
e96b75a0d0
fix file/download paths 
move download-from-uri logic into stream manager
 2019-02-01 21:40:35 -05:00
Jack Robison
efe4afd09e
fix blob announce time 2019-02-01 21:28:59 -05:00
Jack Robison
c7cb6822cc
logging 2019-02-01 21:28:59 -05:00
Jack Robison
c75665d3f0
update file_list docs, add blobs_remaining to file list args and to file dict result 2019-02-01 21:28:59 -05:00
Jack Robison
744375b2c0
re-assemble file / resume downloads 2019-02-01 21:28:59 -05:00
Victor Shyba
40da5b6cb6 rework data parser 2019-02-01 20:09:14 -03:00
Victor Shyba
b36cf3e36c remove closing peers from active connections 2019-02-01 20:09:14 -03:00
Jack Robison
0a08f445a4
close blob server transport on sendfile failing 2019-02-01 15:11:37 -05:00
Victor Shyba
f5bf1463af fixes from review 2019-02-01 17:09:37 -03:00
Victor Shyba
ca4a55ef28 set verified event earlier and remove stop awaits 2019-02-01 16:05:14 -03:00
Victor Shyba
832be0758b let cancel propagate 2019-02-01 16:05:14 -03:00
Victor Shyba
25050fdeeb fixes for writers (check inside lock and dont hold verified bytes) 2019-02-01 16:05:14 -03:00
Jack Robison
dabd9cbe1e
faster stream manager startup 2019-02-01 13:53:03 -05:00
Jack Robison
2336015f9a
fix publish 
fixes https://github.com/lbryio/lbry/issues/1826
- remove `sources` argument from `publish`
- only add/update files we have
- delete existing stream if updating with a new one
 2019-02-01 13:35:18 -05:00
Jack Robison
31445c7797
remove lock from PeerManager 2019-01-31 20:57:48 -05:00
Jack Robison
87f5d1bd4a
fix updating claim with existing stream 2019-01-31 20:49:19 -05:00
Jack Robison
2e9acd1baa
remove blob request asyncio.Lock 2019-01-31 20:19:50 -05:00
Jack Robison
df86937f4e
fix deleting a file mid download 2019-01-31 17:59:42 -05:00
Jack Robison
957d0290df
add torba logging 2019-01-31 16:53:54 -05:00
Jack Robison
40b90ce79b
fix hanging download 2019-01-31 16:53:27 -05:00
Jack Robison
aa7f0845cc
catch sendfile errors 2019-01-31 16:10:27 -05:00
Jack Robison
6e69d306de
tests 2019-01-31 15:39:23 -05:00
Jack Robison
d85d3dfaa4
increase default max_connections_per_download 2019-01-31 15:39:20 -05:00
Jack Robison
2b035009ef
handle both urls and ip addresses for fixed nodes and reflector servers 2019-01-31 15:39:20 -05:00
Jack Robison
f9fd62c214
add fixed peers immediately if the dht has no peers 2019-01-31 15:39:20 -05:00
Jack Robison
e8a1953815
add fixed peers immediately if dht is skipped 2019-01-31 15:39:20 -05:00
Jack Robison
34776e7eb9
fix test 2019-01-31 15:39:20 -05:00
Jack Robison
a4419f562f
pylint 2019-01-31 15:39:20 -05:00
Jack Robison
e60e5b3919
fix updating content claim for a file 
-fix file name and download dir not being hex encoded during publish, add repair script
 2019-01-31 15:39:20 -05:00
Jack Robison
3589cc9977
call update_reflected_stream upon reflector finishing 2019-01-31 15:39:20 -05:00
Jack Robison
2e978c00b2
prioritize peers to download from 2019-01-31 15:39:20 -05:00
Jack Robison
2335243de9
pylint 2019-01-31 15:39:20 -05:00
Jack Robison
6ad68eabd9
tests 2019-01-31 15:39:20 -05:00
Jack Robison
262b9a624b
update file_reflect 2019-01-31 15:39:20 -05:00
Jack Robison
05915c540a
set fully reflected if the server indicates it had all of the stream 2019-01-31 15:39:20 -05:00
Jack Robison
6aef6a80b7
refactor downloader 
split peer accumulation out, use Queues instead of locks
 2019-01-31 15:39:20 -05:00
Jack Robison
16efe9ba95
add fixed_peer_delay setting 2019-01-31 15:39:20 -05:00
Jack Robison
fcda9500fe
add accumulate_peers to Node 
simplifies collecting peers during a download by giving a queue of hashes to search for and a queue or peers as they are found
 2019-01-31 15:39:20 -05:00
Victor Shyba
5ee4b2173a fix probes bugs, partial DHT refactor 2019-01-31 01:49:52 -03:00
Victor Shyba
3e1c3d1e4b use node_id parameter instead of your own 2019-01-31 01:11:09 -03:00
Victor Shyba
21f2b2dcd7 cast timestamp to integer before querying 2019-01-31 01:09:57 -03:00
Jack Robison
ffc7fc0793
fix uncaught exception in BlobServerProtocol when client disconnects during a blob transfer 2019-01-30 12:03:05 -05:00
Victor Shyba
2e633c0ee6 fix chunked requests for server 2019-01-30 12:11:05 -03:00
Jack Robison
933d58d49b
logging 2019-01-29 20:47:02 -05:00
Jack Robison
8f7cf0b38f
verbose logging 2019-01-29 13:08:11 -05:00
Jack Robison
418def56ac
fix log 2019-01-28 22:38:17 -05:00
Jack Robison
cefe3eb520
fix file_delete 2019-01-28 22:01:43 -05:00
Jack Robison
64dffa306f
logging, cleanup 2019-01-28 22:01:43 -05:00
Jack Robison
7d33b4f1f3
fix unawaited task 2019-01-28 22:01:43 -05:00
Jack Robison
f507d95198
always add fixed peers if configured after 2s 2019-01-28 22:01:43 -05:00
Jack Robison
2a7d80f8b4
logging 2019-01-28 22:01:43 -05:00
Jack Robison
bfaebf0085
fix timeout error 2019-01-28 17:20:14 -05:00
Jack Robison
71f9f8ae9c
fix race condition in reflector server 2019-01-28 16:51:11 -05:00
Jack Robison
330862e487
fix https://github.com/lbryio/lbry/issues/1297 2019-01-28 16:00:37 -05:00
Jack Robison
778d3826ab
clean up settings and use them 2019-01-28 15:51:27 -05:00
Jack Robison
4a11cf007f
remove unused settings and constants 2019-01-27 19:51:44 -05:00
Jack Robison
573285a5bb
fix error encoding find value response 2019-01-26 17:04:11 -05:00
Lex Berezhny
60ed4e503f refactored settings_set to support all settings 2019-01-26 00:04:36 -05:00
Lex Berezhny
aa1405887e use float instead of decimal for now 2019-01-25 23:20:43 -05:00
Lex Berezhny
cced217e43 update doc format for max_key_fee 2019-01-25 23:20:43 -05:00
Lex Berezhny
8a3b8d2df1 max_key_fee in yaml as null 2019-01-25 23:20:43 -05:00
Lex Berezhny
be0bd3bdea refactored --max-key-fee to be more ergonomic 2019-01-25 23:20:43 -05:00
Lex Berezhny
e01c73a7f8 pylint 2019-01-25 21:00:20 -05:00
Lex Berezhny
134338db4b dropped help, commands and resolve_name commands 2019-01-25 21:00:20 -05:00
Lex Berezhny
f47a685734 help message for start command 2019-01-25 20:28:47 -05:00
Lex Berezhny
6e29f7e99b user is shown help when typing lbrynet help 2019-01-25 20:28:47 -05:00
Lex Berezhny
a55802c1de v0.31.0rc3 2019-01-25 17:05:38 -05:00
Jack Robison
a7610e3d34
async reflector 2019-01-25 16:10:17 -05:00
Jack Robison
69bcb052de
fix blob server protocol factory 2019-01-25 13:10:40 -05:00
Lex Berezhny
b7d62ca084 v0.31.0rc2 2019-01-25 10:46:20 -05:00
Jack Robison
fa97f7bd38 fix written_bytes error 2019-01-25 10:23:03 -05:00
Lex Berezhny
4a35247016 fixes for toggle config, passing servers as list, etc 2019-01-25 10:19:24 -05:00
Lex Berezhny
d0526ad7d5 bring --config back to start command 2019-01-25 10:19:24 -05:00
Lex Berezhny
19b529cf38 removed unused import 2019-01-25 10:19:24 -05:00
Lex Berezhny
0373196707 max_key_fee is plain YAML now 2019-01-25 10:19:24 -05:00
Lex Berezhny
7a601db23f cli fixup 2019-01-25 10:19:24 -05:00
Jack Robison
77ece96356
fix config error if $HOME is None 
-not seen on travis due to HOME being manually set to /tmp
 2019-01-24 12:42:55 -05:00
Lex Berezhny
80440fd6bd v0.31.0rc1 2019-01-24 00:13:55 -05:00
Lex Berezhny
79e4767f69 unused imports 2019-01-24 00:13:15 -05:00
Lex Berezhny
0ee279bfd7 fix startup/shutdown and file_list --full-status argument 2019-01-24 00:13:15 -05:00
Lex Berezhny
5249d2a3ca removed unused methods 2019-01-24 00:13:15 -05:00
Lex Berezhny
4d5823f473 distro package is in setup.py, does not make sense to wrap it in try/except ModuleNotFoundError 2019-01-24 00:13:15 -05:00
Lex Berezhny
2dcd187d69 fixed component manager tests 2019-01-24 00:13:15 -05:00
Lex Berezhny
3d9ea92118 remove unnecessary logger configuration for BitcoinRPC 2019-01-24 00:13:15 -05:00
Lex Berezhny
3b61222e5a pylint 2019-01-24 00:13:15 -05:00
Lex Berezhny
2ca438c36e passing integration tests 2019-01-24 00:13:15 -05:00
Jack Robison
8948d5be86 don't shut down immediately after startup 2019-01-24 00:13:15 -05:00
Jack Robison
a3e64f9cef remove dht requirement from stream manager component 2019-01-24 00:13:15 -05:00
Jack Robison
09afec1961 import 2019-01-24 00:13:15 -05:00
Jack Robison
f9c465a636 f string 2019-01-24 00:13:15 -05:00
Jack Robison
89efe3d1b9 minimal changes to cli.py 2019-01-24 00:13:15 -05:00