132 commits

Author	SHA1	Message	Date
Jack Robison	d95903d219	update aiohttp and pylint	2019-02-28 12:04:07 -05:00
Victor Shyba	9f071bbe36	pylint	2019-02-26 17:42:45 -05:00
Victor Shyba	7a96e742f2	stop trapping CancelledError	2019-02-26 17:42:45 -05:00
Victor Shyba	fcdd9295b4	delete upon failing to decode JSON from sd blob	2019-02-21 20:53:21 -03:00
Victor Shyba	4e7d88311f	raise proper exception on undecode-able blobs	2019-02-21 20:42:51 -03:00
Victor Shyba	38ed212c01	refactor, dont call storage from blob_manager + missing await	2019-02-19 21:45:14 -03:00
Victor Shyba	d035c13883	file_path can be None, add helper to check if stream output file exists instead	2019-02-19 21:45:14 -03:00
Lex Berezhny	0fdd2ae178	added tx to JSON response from get	2019-02-18 22:53:11 -05:00
Victor Shyba	d2150cf142	fix balance check	2019-02-18 22:53:11 -05:00
Victor Shyba	5c217f1544	resovle reflector IP before adding it	2019-02-15 21:20:06 -03:00
Jack Robison	fd081496c0	fix blocking on data being written for resumed streams	2019-02-15 17:35:46 -05:00
Jack Robison	0d558bd552	fix default file list sorting	2019-02-15 17:26:24 -05:00
Jack Robison	ca835f3f80	fix 1883	2019-02-15 16:26:13 -05:00
Jack Robison	dbb6ba6241	recover streams with missing sd blobs, handle previous sd blob bugs ... -test download and recover stream with old key sorting	2019-02-15 15:21:29 -05:00
Victor Shyba	a49f479a5f	move wait to where its needed on assembler	2019-02-15 10:26:05 -05:00
Jack Robison	56c41b2fea	fix integration test ... -cancel running reflector uploads from publishes upon shutdown -refactor blob.delete to be non-async -delete blobs synchronously	2019-02-14 15:43:26 -05:00
Victor Shyba	04f70d1f31	stop when data timeout	2019-02-14 00:10:44 -05:00
Victor Shyba	dbc8cfe5ae	delete incomplete files on assembler stop	2019-02-14 00:10:44 -05:00
Victor Shyba	b6df2b9d28	missing None for optional field + pylint	2019-02-13 23:33:49 -05:00
Victor Shyba	5c1ab42277	resume should put the stream into wait as done usually on get calls	2019-02-13 23:33:49 -05:00
Victor Shyba	bbdc96e762	file_name as a property of managed stream, to reflect internal changes	2019-02-13 23:33:49 -05:00
Victor Shyba	fc0c949187	properly update file name on downloader/assembler	2019-02-13 23:33:49 -05:00
Victor Shyba	37bb765a2e	resume downloads if DHT disabled	2019-02-13 23:33:49 -05:00
Victor Shyba	e2170ec83d	fix download timeout types	2019-02-12 20:58:31 -05:00
Jack Robison	c9d54bb049	fix max_key_fee	2019-02-11 15:26:13 -05:00
Jack Robison	6386815a29	fix reflect_streams not being followed	2019-02-08 20:13:26 -05:00
Jack Robison	868110a6f2	populate stream manager with a single batch query	2019-02-08 16:06:35 -05:00
Victor Shyba	e2b06677b5	defer db insert during downloads	2019-02-08 03:45:23 -03:00
Victor Shyba	c06ec6cd69	manage connections, reusing them when possible	2019-02-08 02:27:58 -03:00
Victor Shyba	3a15ff4bcd	test incomplete streams, respect real length, delete on incomplete assembly	2019-02-06 15:56:06 -03:00
Jack Robison	dfdc8eda00	improve test, fix stream_finished_event	2019-02-06 13:32:50 -05:00
Jack Robison	58f6cb71c6	test stream manager	2019-02-06 13:08:57 -05:00
Jack Robison	ae11c5bb4b	fix setting finished event	2019-02-06 12:18:24 -05:00
Jack Robison	c1c6d5bc99	fix deleting partial downloads when stopped and previous streams when updating a publish	2019-02-06 11:43:05 -05:00
Jack Robison	0e972ec2ae	refactor BlobFile.close to be non async	2019-02-06 11:00:57 -05:00
Jack Robison	76c8447a56	add stream descriptor tests	2019-02-05 21:25:31 -05:00
Jack Robison	7c8ad0fe30	fix deleting incomplete downloads upon stop ... fix deleting previous streams (not assembled/published files) for a claim being updated	2019-02-05 15:33:08 -05:00
Jack Robison	7fe211d134	fix resolve_host	2019-02-05 13:31:57 -05:00
Victor Shyba	63d1ec3c28	fixes from jack comments	2019-02-05 13:01:12 -03:00
Victor Shyba	2c275efa48	remove redundant calls to should announce	2019-02-05 13:01:12 -03:00
Victor Shyba	6ead932ccb	announces naturally, removes redundant calls	2019-02-05 13:01:12 -03:00
Victor Shyba	f766dbefe0	announce and set completed after inserts	2019-02-05 13:01:12 -03:00
Jack Robison	8cda3143f2	fix setting should_announce and blob status	2019-02-04 18:22:44 -05:00
Jack Robison	5af351a234	reflector logging	2019-02-04 17:33:25 -05:00
Victor Shyba	ad71a9a89d	refactor reflect to close protocol	2019-02-04 13:34:18 -03:00
Victor Shyba	736a949cb5	resume streams on startup	2019-02-03 20:49:09 -03:00
Victor Shyba	11bb1cc7ee	use with clause to avoid leaking fds	2019-02-03 20:41:47 -03:00
Jack Robison	dcd78bd86e	always delete file from downloads if download was not complete	2019-02-03 16:43:40 -05:00
Jack Robison	0a027cf2de	logging	2019-02-03 15:47:19 -05:00
Jack Robison	f2968aab22	add re-reflect task to stream manager, add concurrent_reflector_uploads to config	2019-02-01 22:59:41 -05:00
Jack Robison	478f79353a	fix bytes error	2019-02-01 22:20:11 -05:00
Jack Robison	e96b75a0d0	fix file/download paths ... move download-from-uri logic into stream manager	2019-02-01 21:40:35 -05:00
Jack Robison	c75665d3f0	update file_list docs, add blobs_remaining to file list args and to file dict result	2019-02-01 21:28:59 -05:00
Jack Robison	744375b2c0	re-assemble file / resume downloads	2019-02-01 21:28:59 -05:00
Victor Shyba	f5bf1463af	fixes from review	2019-02-01 17:09:37 -03:00
Victor Shyba	ca4a55ef28	set verified event earlier and remove stop awaits	2019-02-01 16:05:14 -03:00
Jack Robison	dabd9cbe1e	faster stream manager startup	2019-02-01 13:53:03 -05:00
Jack Robison	2336015f9a	fix publish ... fixes https://github.com/lbryio/lbry/issues/1826 - remove `sources` argument from `publish` - only add/update files we have - delete existing stream if updating with a new one	2019-02-01 13:35:18 -05:00
Jack Robison	df86937f4e	fix deleting a file mid download	2019-01-31 17:59:42 -05:00
Jack Robison	6e69d306de	tests	2019-01-31 15:39:23 -05:00
Jack Robison	2b035009ef	handle both urls and ip addresses for fixed nodes and reflector servers	2019-01-31 15:39:20 -05:00
Jack Robison	f9fd62c214	add fixed peers immediately if the dht has no peers	2019-01-31 15:39:20 -05:00
Jack Robison	e8a1953815	add fixed peers immediately if dht is skipped	2019-01-31 15:39:20 -05:00
Jack Robison	a4419f562f	pylint	2019-01-31 15:39:20 -05:00
Jack Robison	e60e5b3919	fix updating content claim for a file ... -fix file name and download dir not being hex encoded during publish, add repair script	2019-01-31 15:39:20 -05:00
Jack Robison	3589cc9977	call update_reflected_stream upon reflector finishing	2019-01-31 15:39:20 -05:00
Jack Robison	6ad68eabd9	tests	2019-01-31 15:39:20 -05:00
Jack Robison	05915c540a	set fully reflected if the server indicates it had all of the stream	2019-01-31 15:39:20 -05:00
Jack Robison	6aef6a80b7	refactor downloader ... split peer accumulation out, use Queues instead of locks	2019-01-31 15:39:20 -05:00
Jack Robison	933d58d49b	logging	2019-01-29 20:47:02 -05:00
Jack Robison	cefe3eb520	fix file_delete	2019-01-28 22:01:43 -05:00
Jack Robison	64dffa306f	logging, cleanup	2019-01-28 22:01:43 -05:00
Jack Robison	f507d95198	always add fixed peers if configured after 2s	2019-01-28 22:01:43 -05:00
Jack Robison	71f9f8ae9c	fix race condition in reflector server	2019-01-28 16:51:11 -05:00
Jack Robison	778d3826ab	clean up settings and use them	2019-01-28 15:51:27 -05:00
Jack Robison	a7610e3d34	async reflector	2019-01-25 16:10:17 -05:00
Jack Robison	fa97f7bd38	fix written_bytes error	2019-01-25 10:23:03 -05:00
Lex Berezhny	0ee279bfd7	fix startup/shutdown and file_list --full-status argument	2019-01-24 00:13:15 -05:00
Jack Robison	a3e64f9cef	remove dht requirement from stream manager component	2019-01-24 00:13:15 -05:00
Jack Robison	09afec1961	import	2019-01-24 00:13:15 -05:00
Jack Robison	5596eac930	merge conflicts	2019-01-24 00:13:15 -05:00
Jack Robison	4bf7742c19	async lbrynet.stream	2019-01-24 00:13:15 -05:00