Commit graph

119 commits

Author SHA1 Message Date
Jack Robison
dbb6ba6241
recover streams with missing sd blobs, handle previous sd blob bugs
-test download and recover stream with old key sorting
2019-02-15 15:21:29 -05:00
Victor Shyba
a49f479a5f move wait to where its needed on assembler 2019-02-15 10:26:05 -05:00
Jack Robison
56c41b2fea
fix integration test
-cancel running reflector uploads from publishes upon shutdown
-refactor blob.delete to be non-async
-delete blobs synchronously
2019-02-14 15:43:26 -05:00
Victor Shyba
04f70d1f31 stop when data timeout 2019-02-14 00:10:44 -05:00
Victor Shyba
dbc8cfe5ae delete incomplete files on assembler stop 2019-02-14 00:10:44 -05:00
Victor Shyba
b6df2b9d28 missing None for optional field + pylint 2019-02-13 23:33:49 -05:00
Victor Shyba
5c1ab42277 resume should put the stream into wait as done usually on get calls 2019-02-13 23:33:49 -05:00
Victor Shyba
bbdc96e762 file_name as a property of managed stream, to reflect internal changes 2019-02-13 23:33:49 -05:00
Victor Shyba
fc0c949187 properly update file name on downloader/assembler 2019-02-13 23:33:49 -05:00
Victor Shyba
37bb765a2e resume downloads if DHT disabled 2019-02-13 23:33:49 -05:00
Victor Shyba
e2170ec83d fix download timeout types 2019-02-12 20:58:31 -05:00
Jack Robison
c9d54bb049
fix max_key_fee 2019-02-11 15:26:13 -05:00
Jack Robison
6386815a29
fix reflect_streams not being followed 2019-02-08 20:13:26 -05:00
Jack Robison
868110a6f2
populate stream manager with a single batch query 2019-02-08 16:06:35 -05:00
Victor Shyba
e2b06677b5 defer db insert during downloads 2019-02-08 03:45:23 -03:00
Victor Shyba
c06ec6cd69 manage connections, reusing them when possible 2019-02-08 02:27:58 -03:00
Victor Shyba
3a15ff4bcd test incomplete streams, respect real length, delete on incomplete assembly 2019-02-06 15:56:06 -03:00
Jack Robison
dfdc8eda00
improve test, fix stream_finished_event 2019-02-06 13:32:50 -05:00
Jack Robison
58f6cb71c6
test stream manager 2019-02-06 13:08:57 -05:00
Jack Robison
ae11c5bb4b
fix setting finished event 2019-02-06 12:18:24 -05:00
Jack Robison
c1c6d5bc99
fix deleting partial downloads when stopped and previous streams when updating a publish 2019-02-06 11:43:05 -05:00
Jack Robison
0e972ec2ae
refactor BlobFile.close to be non async 2019-02-06 11:00:57 -05:00
Jack Robison
76c8447a56
add stream descriptor tests 2019-02-05 21:25:31 -05:00
Jack Robison
7c8ad0fe30
fix deleting incomplete downloads upon stop
fix deleting previous streams (not assembled/published files) for a claim being updated
2019-02-05 15:33:08 -05:00
Jack Robison
7fe211d134
fix resolve_host 2019-02-05 13:31:57 -05:00
Victor Shyba
63d1ec3c28 fixes from jack comments 2019-02-05 13:01:12 -03:00
Victor Shyba
2c275efa48 remove redundant calls to should announce 2019-02-05 13:01:12 -03:00
Victor Shyba
6ead932ccb announces naturally, removes redundant calls 2019-02-05 13:01:12 -03:00
Victor Shyba
f766dbefe0 announce and set completed after inserts 2019-02-05 13:01:12 -03:00
Jack Robison
8cda3143f2
fix setting should_announce and blob status 2019-02-04 18:22:44 -05:00
Jack Robison
5af351a234
reflector logging 2019-02-04 17:33:25 -05:00
Victor Shyba
ad71a9a89d refactor reflect to close protocol 2019-02-04 13:34:18 -03:00
Victor Shyba
736a949cb5 resume streams on startup 2019-02-03 20:49:09 -03:00
Victor Shyba
11bb1cc7ee use with clause to avoid leaking fds 2019-02-03 20:41:47 -03:00
Jack Robison
dcd78bd86e
always delete file from downloads if download was not complete 2019-02-03 16:43:40 -05:00
Jack Robison
0a027cf2de
logging 2019-02-03 15:47:19 -05:00
Jack Robison
f2968aab22
add re-reflect task to stream manager, add concurrent_reflector_uploads to config 2019-02-01 22:59:41 -05:00
Jack Robison
478f79353a
fix bytes error 2019-02-01 22:20:11 -05:00
Jack Robison
e96b75a0d0
fix file/download paths
move download-from-uri logic into stream manager
2019-02-01 21:40:35 -05:00
Jack Robison
c75665d3f0
update file_list docs, add blobs_remaining to file list args and to file dict result 2019-02-01 21:28:59 -05:00
Jack Robison
744375b2c0
re-assemble file / resume downloads 2019-02-01 21:28:59 -05:00
Victor Shyba
f5bf1463af fixes from review 2019-02-01 17:09:37 -03:00
Victor Shyba
ca4a55ef28 set verified event earlier and remove stop awaits 2019-02-01 16:05:14 -03:00
Jack Robison
dabd9cbe1e
faster stream manager startup 2019-02-01 13:53:03 -05:00
Jack Robison
2336015f9a
fix publish
fixes https://github.com/lbryio/lbry/issues/1826
- remove `sources` argument from `publish`
- only add/update files we have
- delete existing stream if updating with a new one
2019-02-01 13:35:18 -05:00
Jack Robison
df86937f4e
fix deleting a file mid download 2019-01-31 17:59:42 -05:00
Jack Robison
6e69d306de
tests 2019-01-31 15:39:23 -05:00
Jack Robison
2b035009ef
handle both urls and ip addresses for fixed nodes and reflector servers 2019-01-31 15:39:20 -05:00
Jack Robison
f9fd62c214
add fixed peers immediately if the dht has no peers 2019-01-31 15:39:20 -05:00
Jack Robison
e8a1953815
add fixed peers immediately if dht is skipped 2019-01-31 15:39:20 -05:00
Jack Robison
a4419f562f
pylint 2019-01-31 15:39:20 -05:00
Jack Robison
e60e5b3919
fix updating content claim for a file
-fix file name and download dir not being hex encoded during publish, add repair script
2019-01-31 15:39:20 -05:00
Jack Robison
3589cc9977
call update_reflected_stream upon reflector finishing 2019-01-31 15:39:20 -05:00
Jack Robison
6ad68eabd9
tests 2019-01-31 15:39:20 -05:00
Jack Robison
05915c540a
set fully reflected if the server indicates it had all of the stream 2019-01-31 15:39:20 -05:00
Jack Robison
6aef6a80b7
refactor downloader
split peer accumulation out, use Queues instead of locks
2019-01-31 15:39:20 -05:00
Jack Robison
933d58d49b
logging 2019-01-29 20:47:02 -05:00
Jack Robison
cefe3eb520
fix file_delete 2019-01-28 22:01:43 -05:00
Jack Robison
64dffa306f
logging, cleanup 2019-01-28 22:01:43 -05:00
Jack Robison
f507d95198
always add fixed peers if configured after 2s 2019-01-28 22:01:43 -05:00
Jack Robison
71f9f8ae9c
fix race condition in reflector server 2019-01-28 16:51:11 -05:00
Jack Robison
778d3826ab
clean up settings and use them 2019-01-28 15:51:27 -05:00
Jack Robison
a7610e3d34
async reflector 2019-01-25 16:10:17 -05:00
Jack Robison
fa97f7bd38 fix written_bytes error 2019-01-25 10:23:03 -05:00
Lex Berezhny
0ee279bfd7 fix startup/shutdown and file_list --full-status argument 2019-01-24 00:13:15 -05:00
Jack Robison
a3e64f9cef remove dht requirement from stream manager component 2019-01-24 00:13:15 -05:00
Jack Robison
09afec1961 import 2019-01-24 00:13:15 -05:00
Jack Robison
5596eac930 merge conflicts 2019-01-24 00:13:15 -05:00
Jack Robison
4bf7742c19 async lbrynet.stream 2019-01-24 00:13:15 -05:00