60 commits

Author	SHA1	Message	Date
Victor Shyba	521d783260	allow running some extra probes for k replacements	2022-03-15 15:36:00 -03:00
Victor Shyba	50846cd37e	remove all references to bottoming out	2022-03-15 15:36:00 -03:00
Victor Shyba	40ed5f311b	no stop condition, let it exhaust	2022-03-15 15:36:00 -03:00
Victor Shyba	293410ac9f	bottoming out is now warning and no results for peer search	2022-03-15 15:36:00 -03:00
Victor Shyba	068f46d137	don't probe peers too far from the top closest	2022-03-15 15:36:00 -03:00
Victor Shyba	5c60c09fef	use a dict for the active queue	2022-03-15 15:36:00 -03:00
Victor Shyba	aca4a243d6	log bottom out of peer search in debug, show short key id for find value	2022-03-15 15:36:00 -03:00
Victor Shyba	422f292d82	bump bottom out limit of peer search so people can use 100 concurrent announcers	2022-03-15 15:36:00 -03:00
Victor Shyba	c81598aab8	wait until k peers are ready. do not double add peers	2022-03-15 15:36:00 -03:00
Victor Shyba	3f8c8e01c6	only return good (contacted) peers	2022-03-15 15:36:00 -03:00
Victor Shyba	8d90653395	reset closest peer on failure	2022-03-15 15:36:00 -03:00
Victor Shyba	01841694ad	add peers from shortlist regardless, but check from other nodes	2022-03-15 15:36:00 -03:00
Victor Shyba	4851f5c300	fix distance sorting and improve logging	2022-03-15 15:36:00 -03:00
Victor Shyba	aef5f11a18	closest peer is only ready when it was contacted and isn't known to be bad	2022-03-15 15:36:00 -03:00
Victor Shyba	700f1cb8e5	dont probe and ignore bad peers	2022-03-15 15:36:00 -03:00
Victor Shyba	7e5f7c1725	simplify, genaralize to any size and fix tests	2022-03-15 15:36:00 -03:00
Victor Shyba	838fd71538	replace duplicated code	2022-03-15 15:36:00 -03:00
Victor Shyba	72beb02ec1	bump DHT peer manager cache to 16384	2022-03-15 15:36:00 -03:00
Victor Shyba	b13c346418	add gauge for queue size	2022-03-15 15:36:00 -03:00
Victor Shyba	2520465fc1	count announcements and how many peers we were able to announce to	2022-03-15 15:36:00 -03:00
Victor Shyba	44e647d85d	remove request_flight metric	2022-03-15 15:36:00 -03:00
Victor Shyba	b4d144da21	add granular metric for stored blob prefix, for network announcements calculation	2022-03-15 15:36:00 -03:00
Victor Shyba	39c2c0b922	change colliding bits metric to gauge	2022-03-15 15:36:00 -03:00
Victor Shyba	dbf11fe750	count bit collisions between 8 and 16	2022-03-15 15:36:00 -03:00
Victor Shyba	cc391324e4	add counter for peers with colliding bytes	2022-03-15 15:36:00 -03:00
Victor Shyba	273b4f6d3b	add requests in flight and error	2022-03-15 15:36:00 -03:00
Victor Shyba	d359d25935	add request received	2022-03-15 15:36:00 -03:00
Victor Shyba	645a81cec2	add request_sent and request_time metric on dht	2022-03-15 15:36:00 -03:00
Victor Shyba	06da53ef09	add storing_peers and peer_manager_keys	2022-03-15 15:36:00 -03:00
Victor Shyba	692af0cc55	prometheus: move blobs_stored and peers to SDK. add buckets_in_routing_table	2022-03-15 15:36:00 -03:00
Victor Shyba	8adec9d0ad	extract cache values, increase peer cache to 2048	2022-03-15 15:35:59 -03:00
Victor Shyba	030e407584	DHT bugfix: failures tracking should be bound to 2048 LRU cache size	2022-03-15 15:35:59 -03:00
Victor Shyba	a866621ccb	do not limit DHT results by K, respect max_results	2022-03-15 15:34:09 -03:00
Victor Shyba	b45b87e72d	clarify DHT debug logging on key and operation	2022-03-15 15:34:09 -03:00
Victor Shyba	60969e4817	clear cache on test assertions	2022-03-15 15:34:09 -03:00
Victor Shyba	806857c066	download from stored announcements and dont reannounce	2022-03-15 15:34:09 -03:00
Victor Shyba	c3b89a896b	move more logic out of the downloader component	2022-03-15 15:34:09 -03:00
Victor Shyba	8d6a7101f6	download from DHT	2022-03-15 15:34:09 -03:00
Victor Shyba	7c1c04674c	dont lose results on duplicates, just warn	2022-03-15 15:34:08 -03:00
Cristian Vicas	90aaf64b77	Bug [#2070] where blob_get RPC timed out. ... Both stream.downloader and blob_exchange.downloader paths are adding the fixed_peers list to the DHT node. Tested jsonrpc_blob_get daemon call. Bug [#2070] where blob_get RPC timed out. Both stream.downloader and blob_exchange.downloader paths are adding the fixed_peers list to the DHT node. Tested jsonrpc_blob_get daemon call.	2022-03-15 15:34:08 -03:00
Victor Shyba	e7b5b82909	dht: use bytes hex/fromhex instead of binascii	2022-03-15 15:34:07 -03:00
Victor Shyba	a237cbd963	add dht seed node script	2022-03-15 15:34:07 -03:00
Lex Berezhny	dc1d9e1c84	upgrade pylint and fix lint errors	2021-08-20 22:36:35 -04:00
Victor Shyba	0b52d2cc15	log invalid port as a warning instead of an exception	2021-08-03 15:29:52 -03:00
Victor Shyba	a9f6a68952	use LRU caches for DHT metrics	2021-06-04 11:54:37 -03:00
Jack Robison	c465d6a6c2	ignore udp packets with low source ports	2021-04-28 16:47:00 -04:00
Jack Robison	a6d65233f1	fallback to getting external ip from spv servers instead of internal apis	2021-01-21 16:20:53 -05:00
Jack Robison	3a64ceb4d6	add forward compatibility for byte datagram keys	2020-09-28 15:56:13 -04:00
Jack Robison	f030d41dc7	add test_is_valid_ipv4	2020-06-02 15:56:57 -04:00
Jack Robison	8d079bfcd1	Merge branch 'master' into master	2020-06-02 15:52:04 -04:00