3572 commits

Author	SHA1	Message	Date
Alex Grintsvayg	f65e70df2f	better resolve description	2019-02-11 12:34:57 -05:00
Jack Robison	6386815a29	fix reflect_streams not being followed	2019-02-08 20:13:26 -05:00
Jack Robison	0c93090d2c	faster is_valid_hashcharacter	2019-02-08 19:57:26 -05:00
Jack Robison	45404b6ad4	faster is_valid_blobhash	2019-02-08 19:54:59 -05:00
Jack Robison	aeb2891baa	faster population of channel names	2019-02-08 19:51:13 -05:00
Jack Robison	8abfdc82b2	faster blob manager startup	2019-02-08 17:37:35 -05:00
Jack Robison	da9b4e317a	limit blobs to announce	2019-02-08 16:29:55 -05:00
Jack Robison	e414cc5c48	whitespace	2019-02-08 16:11:56 -05:00
Jack Robison	939edb2f18	logging ... -call loop.set_debug when using --verbose	2019-02-08 16:06:38 -05:00
Jack Robison	868110a6f2	populate stream manager with a single batch query	2019-02-08 16:06:35 -05:00
Victor Shyba	9ecec52412	check lock acquired to prevent races	2019-02-08 04:39:28 -03:00
Victor Shyba	8b25acff6e	do not wait blob to be written, log errors from races	2019-02-08 04:32:53 -03:00
Victor Shyba	e2b06677b5	defer db insert during downloads	2019-02-08 03:45:23 -03:00
Victor Shyba	af1619ebfb	disable race limiting for now	2019-02-08 03:32:38 -03:00
Victor Shyba	3352e0e4f4	score peers by speed	2019-02-08 03:08:08 -03:00
Victor Shyba	141d68a2cd	fix write race condition	2019-02-08 02:39:29 -03:00
Victor Shyba	c06ec6cd69	manage connections, reusing them when possible	2019-02-08 02:27:58 -03:00
Victor Shyba	1be5dce30e	stop racing during long streams	2019-02-08 01:28:03 -03:00
Victor Shyba	73da223ae1	conf: give up first contact earlier, but let the transfer go longer	2019-02-08 01:05:58 -03:00
Victor Shyba	2d7eb83518	change score calculation, wait for active peers too, simplify peer sorting/keeping	2019-02-08 01:04:38 -03:00
Victor Shyba	ad03f91d24	let cancelled errors flow	2019-02-08 01:03:13 -03:00
Victor Shyba	5586a226c2	bypass parser during download	2019-02-07 21:19:31 -03:00
Victor Shyba	a616582733	wait on connection tasks	2019-02-07 20:11:57 -03:00
Victor Shyba	43ac928f0b	remove dht locks	2019-02-07 20:11:57 -03:00
Lex Berezhny	a121110743	v0.31.0	2019-02-06 14:28:40 -05:00
Victor Shyba	3a15ff4bcd	test incomplete streams, respect real length, delete on incomplete assembly	2019-02-06 15:56:06 -03:00
Jack Robison	dfdc8eda00	improve test, fix stream_finished_event	2019-02-06 13:32:50 -05:00
Jack Robison	58f6cb71c6	test stream manager	2019-02-06 13:08:57 -05:00
Jack Robison	ae11c5bb4b	fix setting finished event	2019-02-06 12:18:24 -05:00
Jack Robison	f22b5da170	fix raised error for a peer address mismatch	2019-02-06 12:11:22 -05:00
Jack Robison	c1c6d5bc99	fix deleting partial downloads when stopped and previous streams when updating a publish	2019-02-06 11:43:05 -05:00
Jack Robison	0e972ec2ae	refactor BlobFile.close to be non async	2019-02-06 11:00:57 -05:00
Victor Shyba	8b5803c945	format error dict on get	2019-02-06 11:55:12 -03:00
Jack Robison	76c8447a56	add stream descriptor tests	2019-02-05 21:25:31 -05:00
Jack Robison	7c8ad0fe30	fix deleting incomplete downloads upon stop ... fix deleting previous streams (not assembled/published files) for a claim being updated	2019-02-05 15:33:08 -05:00
Jack Robison	073ffdd049	proto and type args	2019-02-05 13:36:25 -05:00
Jack Robison	7fe211d134	fix resolve_host	2019-02-05 13:31:57 -05:00
Victor Shyba	a4ced4dbd2	set verified after callback	2019-02-05 13:17:00 -03:00
Victor Shyba	c6b910a5f3	use time_getter on storage	2019-02-05 13:14:59 -03:00
Victor Shyba	63d1ec3c28	fixes from jack comments	2019-02-05 13:01:12 -03:00
Victor Shyba	7b8f42e6d0	remove dead set_should_announce code	2019-02-05 13:01:12 -03:00
Victor Shyba	2c275efa48	remove redundant calls to should announce	2019-02-05 13:01:12 -03:00
Victor Shyba	6ead932ccb	announces naturally, removes redundant calls	2019-02-05 13:01:12 -03:00
Victor Shyba	f766dbefe0	announce and set completed after inserts	2019-02-05 13:01:12 -03:00
Jack Robison	0685bc5689	include torba in log file	2019-02-04 18:25:47 -05:00
Jack Robison	8cda3143f2	fix setting should_announce and blob status	2019-02-04 18:22:44 -05:00
Jack Robison	5af351a234	reflector logging	2019-02-04 17:33:25 -05:00
Victor Shyba	b82d2c3120	handle and log if downloading from bad sendfile implementations	2019-02-04 19:13:31 -03:00
Jack Robison	2bce7cecfa	fix sendfile on windows	2019-02-04 16:48:48 -05:00
Jack Robison	8883587cc7	logging	2019-02-04 16:19:27 -05:00
Jack Robison	c85991704d	catch invalid data error	2019-02-04 16:19:27 -05:00
Jack Robison	377442d9f0	fix request_blob cancellation	2019-02-04 16:19:24 -05:00
Jack Robison	f0b49fc153	always return close peers from findValue	2019-02-04 15:07:42 -05:00
Jack Robison	ae1162f5b0	don't try downloading from oneself ... -don't return the querying peer in results for findValue	2019-02-04 14:50:53 -05:00
Victor Shyba	ad71a9a89d	refactor reflect to close protocol	2019-02-04 13:34:18 -03:00
Jack Robison	13b3e4ba8e	v0.31.0rc5	2019-02-04 10:22:43 -05:00
Victor Shyba	736a949cb5	resume streams on startup	2019-02-03 20:49:09 -03:00
Victor Shyba	11bb1cc7ee	use with clause to avoid leaking fds	2019-02-03 20:41:47 -03:00
Jack Robison	dcd78bd86e	always delete file from downloads if download was not complete	2019-02-03 16:43:40 -05:00
Jack Robison	e5af041627	fix connection status message	2019-02-03 16:20:52 -05:00
Jack Robison	0a027cf2de	logging	2019-02-03 15:47:19 -05:00
Jack Robison	3f98b16800	fix blob announce time, use time.time instead of loop.time	2019-02-03 15:47:15 -05:00
Jack Robison	8f5a8ab8ba	fix peer_ping	2019-02-02 02:02:19 -05:00
Jack Robison	91e581422c	v0.31.0rc4	2019-02-02 00:30:57 -05:00
Jack Robison	f2968aab22	add re-reflect task to stream manager, add concurrent_reflector_uploads to config	2019-02-01 22:59:41 -05:00
Jack Robison	ccbe8aa0eb	create data. download, and wallet directories if they don't exist on startup	2019-02-01 22:45:28 -05:00
Jack Robison	478f79353a	fix bytes error	2019-02-01 22:20:11 -05:00
Jack Robison	c3b0151943	fix dht status error during startup	2019-02-01 22:12:07 -05:00
Jack Robison	e96b75a0d0	fix file/download paths ... move download-from-uri logic into stream manager	2019-02-01 21:40:35 -05:00
Jack Robison	efe4afd09e	fix blob announce time	2019-02-01 21:28:59 -05:00
Jack Robison	c7cb6822cc	logging	2019-02-01 21:28:59 -05:00
Jack Robison	c75665d3f0	update file_list docs, add blobs_remaining to file list args and to file dict result	2019-02-01 21:28:59 -05:00
Jack Robison	744375b2c0	re-assemble file / resume downloads	2019-02-01 21:28:59 -05:00
Victor Shyba	40da5b6cb6	rework data parser	2019-02-01 20:09:14 -03:00
Victor Shyba	b36cf3e36c	remove closing peers from active connections	2019-02-01 20:09:14 -03:00
Jack Robison	0a08f445a4	close blob server transport on sendfile failing	2019-02-01 15:11:37 -05:00
Victor Shyba	f5bf1463af	fixes from review	2019-02-01 17:09:37 -03:00
Victor Shyba	ca4a55ef28	set verified event earlier and remove stop awaits	2019-02-01 16:05:14 -03:00
Victor Shyba	832be0758b	let cancel propagate	2019-02-01 16:05:14 -03:00
Victor Shyba	25050fdeeb	fixes for writers (check inside lock and dont hold verified bytes)	2019-02-01 16:05:14 -03:00
Jack Robison	dabd9cbe1e	faster stream manager startup	2019-02-01 13:53:03 -05:00
Jack Robison	2336015f9a	fix publish ... fixes https://github.com/lbryio/lbry/issues/1826 - remove `sources` argument from `publish` - only add/update files we have - delete existing stream if updating with a new one	2019-02-01 13:35:18 -05:00
Jack Robison	31445c7797	remove lock from PeerManager	2019-01-31 20:57:48 -05:00
Jack Robison	87f5d1bd4a	fix updating claim with existing stream	2019-01-31 20:49:19 -05:00
Jack Robison	2e9acd1baa	remove blob request asyncio.Lock	2019-01-31 20:19:50 -05:00
Jack Robison	df86937f4e	fix deleting a file mid download	2019-01-31 17:59:42 -05:00
Jack Robison	957d0290df	add torba logging	2019-01-31 16:53:54 -05:00
Jack Robison	40b90ce79b	fix hanging download	2019-01-31 16:53:27 -05:00
Jack Robison	aa7f0845cc	catch sendfile errors	2019-01-31 16:10:27 -05:00
Jack Robison	6e69d306de	tests	2019-01-31 15:39:23 -05:00
Jack Robison	d85d3dfaa4	increase default max_connections_per_download	2019-01-31 15:39:20 -05:00
Jack Robison	2b035009ef	handle both urls and ip addresses for fixed nodes and reflector servers	2019-01-31 15:39:20 -05:00
Jack Robison	f9fd62c214	add fixed peers immediately if the dht has no peers	2019-01-31 15:39:20 -05:00
Jack Robison	e8a1953815	add fixed peers immediately if dht is skipped	2019-01-31 15:39:20 -05:00
Jack Robison	34776e7eb9	fix test	2019-01-31 15:39:20 -05:00
Jack Robison	a4419f562f	pylint	2019-01-31 15:39:20 -05:00
Jack Robison	e60e5b3919	fix updating content claim for a file ... -fix file name and download dir not being hex encoded during publish, add repair script	2019-01-31 15:39:20 -05:00
Jack Robison	3589cc9977	call update_reflected_stream upon reflector finishing	2019-01-31 15:39:20 -05:00
Jack Robison	2e978c00b2	prioritize peers to download from	2019-01-31 15:39:20 -05:00
Jack Robison	2335243de9	pylint	2019-01-31 15:39:20 -05:00
Jack Robison	6ad68eabd9	tests	2019-01-31 15:39:20 -05:00
Jack Robison	262b9a624b	update file_reflect	2019-01-31 15:39:20 -05:00
Jack Robison	05915c540a	set fully reflected if the server indicates it had all of the stream	2019-01-31 15:39:20 -05:00
Jack Robison	6aef6a80b7	refactor downloader ... split peer accumulation out, use Queues instead of locks	2019-01-31 15:39:20 -05:00
Jack Robison	16efe9ba95	add fixed_peer_delay setting	2019-01-31 15:39:20 -05:00
Jack Robison	fcda9500fe	add accumulate_peers to Node ... simplifies collecting peers during a download by giving a queue of hashes to search for and a queue or peers as they are found	2019-01-31 15:39:20 -05:00
Victor Shyba	5ee4b2173a	fix probes bugs, partial DHT refactor	2019-01-31 01:49:52 -03:00
Victor Shyba	3e1c3d1e4b	use node_id parameter instead of your own	2019-01-31 01:11:09 -03:00
Victor Shyba	21f2b2dcd7	cast timestamp to integer before querying	2019-01-31 01:09:57 -03:00
Jack Robison	ffc7fc0793	fix uncaught exception in BlobServerProtocol when client disconnects during a blob transfer	2019-01-30 12:03:05 -05:00
Victor Shyba	2e633c0ee6	fix chunked requests for server	2019-01-30 12:11:05 -03:00
Jack Robison	933d58d49b	logging	2019-01-29 20:47:02 -05:00
Jack Robison	8f7cf0b38f	verbose logging	2019-01-29 13:08:11 -05:00
Jack Robison	418def56ac	fix log	2019-01-28 22:38:17 -05:00
Jack Robison	cefe3eb520	fix file_delete	2019-01-28 22:01:43 -05:00
Jack Robison	64dffa306f	logging, cleanup	2019-01-28 22:01:43 -05:00
Jack Robison	7d33b4f1f3	fix unawaited task	2019-01-28 22:01:43 -05:00
Jack Robison	f507d95198	always add fixed peers if configured after 2s	2019-01-28 22:01:43 -05:00
Jack Robison	2a7d80f8b4	logging	2019-01-28 22:01:43 -05:00
Jack Robison	bfaebf0085	fix timeout error	2019-01-28 17:20:14 -05:00
Jack Robison	71f9f8ae9c	fix race condition in reflector server	2019-01-28 16:51:11 -05:00
Jack Robison	330862e487	fix https://github.com/lbryio/lbry/issues/1297	2019-01-28 16:00:37 -05:00
Jack Robison	778d3826ab	clean up settings and use them	2019-01-28 15:51:27 -05:00
Jack Robison	4a11cf007f	remove unused settings and constants	2019-01-27 19:51:44 -05:00
Jack Robison	573285a5bb	fix error encoding find value response	2019-01-26 17:04:11 -05:00
Lex Berezhny	60ed4e503f	refactored settings_set to support all settings	2019-01-26 00:04:36 -05:00
Lex Berezhny	aa1405887e	use float instead of decimal for now	2019-01-25 23:20:43 -05:00
Lex Berezhny	cced217e43	update doc format for max_key_fee	2019-01-25 23:20:43 -05:00
Lex Berezhny	8a3b8d2df1	max_key_fee in yaml as null	2019-01-25 23:20:43 -05:00
Lex Berezhny	be0bd3bdea	refactored --max-key-fee to be more ergonomic	2019-01-25 23:20:43 -05:00
Lex Berezhny	e01c73a7f8	pylint	2019-01-25 21:00:20 -05:00
Lex Berezhny	134338db4b	dropped help, commands and resolve_name commands	2019-01-25 21:00:20 -05:00
Lex Berezhny	f47a685734	help message for start command	2019-01-25 20:28:47 -05:00
Lex Berezhny	6e29f7e99b	user is shown help when typing lbrynet help	2019-01-25 20:28:47 -05:00
Lex Berezhny	a55802c1de	v0.31.0rc3	2019-01-25 17:05:38 -05:00
Jack Robison	a7610e3d34	async reflector	2019-01-25 16:10:17 -05:00
Jack Robison	69bcb052de	fix blob server protocol factory	2019-01-25 13:10:40 -05:00
Lex Berezhny	b7d62ca084	v0.31.0rc2	2019-01-25 10:46:20 -05:00
Jack Robison	fa97f7bd38	fix written_bytes error	2019-01-25 10:23:03 -05:00
Lex Berezhny	4a35247016	fixes for toggle config, passing servers as list, etc	2019-01-25 10:19:24 -05:00
Lex Berezhny	d0526ad7d5	bring --config back to start command	2019-01-25 10:19:24 -05:00
Lex Berezhny	19b529cf38	removed unused import	2019-01-25 10:19:24 -05:00
Lex Berezhny	0373196707	max_key_fee is plain YAML now	2019-01-25 10:19:24 -05:00
Lex Berezhny	7a601db23f	cli fixup	2019-01-25 10:19:24 -05:00
Jack Robison	77ece96356	fix config error if $HOME is None ... -not seen on travis due to HOME being manually set to /tmp	2019-01-24 12:42:55 -05:00
Lex Berezhny	80440fd6bd	v0.31.0rc1	2019-01-24 00:13:55 -05:00
Lex Berezhny	79e4767f69	unused imports	2019-01-24 00:13:15 -05:00
Lex Berezhny	0ee279bfd7	fix startup/shutdown and file_list --full-status argument	2019-01-24 00:13:15 -05:00
Lex Berezhny	5249d2a3ca	removed unused methods	2019-01-24 00:13:15 -05:00
Lex Berezhny	4d5823f473	distro package is in setup.py, does not make sense to wrap it in try/except ModuleNotFoundError	2019-01-24 00:13:15 -05:00