Victor Shyba
562d23df11
remove nodes that went ignored during iteration
2018-07-16 17:22:46 -03:00
Victor Shyba
5c7d279104
prune failures during is_ignored calls
2018-07-12 18:24:59 -03:00
Victor Shyba
cfe8e17223
filter out the bad node in results instead of punishing the node we trusted to contact
2018-07-12 18:24:18 -03:00
Victor Shyba
c589e89bf0
greedy search with exclude filtering
2018-07-05 22:03:52 -03:00
Victor Shyba
1cc7ce69ab
set the peer search timeout as configured and raise it to 30s
2018-07-03 13:47:00 -03:00
Victor Shyba
7e0784d8e0
log a warning when peer search times out for real
2018-07-03 12:20:33 -03:00
Victor Shyba
56613a71d5
let find value run until it finds a value
2018-07-03 11:51:39 -03:00
Jack Robison
929a0cc5af
exclude self and querying node from closest contacts
2018-07-02 14:54:03 -04:00
Jack Robison
b6289d101d
fix improper sorting when getting the closest peers to a hash
2018-07-02 14:54:03 -04:00
Jack Robison
7fe92d2df0
load all finished blobs into the node datastore
2018-07-02 14:31:09 -04:00
Jack Robison
03769b94b8
accept invalid store tokens for the first expire-time after startup
...
this is to better handle nodes storing to us after we've restarted our node
2018-07-02 13:26:38 -04:00
Jack Robison
e6df378e9e
fix token validation error when the dht node has just been started ( https://github.com/lbryio/lbry/issues/1248 )
2018-06-14 15:07:10 -04:00
Jack Robison
7f3ead67bf
disable forced bucket refresh during join
2018-06-07 15:02:47 -04:00
Jack Robison
b0e4fc4faa
fix iterative find lockup
2018-06-07 15:02:47 -04:00
Jack Robison
ae631f05c3
fix teardown error
2018-06-07 15:02:47 -04:00
Jack Robison
bc0da5e2d1
only use seeds in iterative bootstrap if no contacts are known yet
2018-06-07 15:02:47 -04:00
Jack Robison
44644673d7
add profiler
2018-06-07 15:02:47 -04:00
Jack Robison
e8b402f998
remove deferredLock from iterativeFind
...
-fire the first iteration right away
2018-06-07 15:02:47 -04:00
Jack Robison
4fbaaac3f3
default new contacts to protocol version 0
2018-06-07 15:02:47 -04:00
Jack Robison
db06191c33
reduce default concurrent announcers to 10
...
-lower rpc timeout to what it originally was
2018-06-07 15:02:47 -04:00
Jack Robison
b0a741b1f4
fix hash announcer semaphore
2018-06-07 15:02:47 -04:00
Jack Robison
537df6c8ad
log socket errors
2018-06-07 15:02:47 -04:00
Jack Robison
42eb172638
refactor announceHaveBlob
...
-add cached `token` to Contact objects to minimize findValue requests
-remove self_store, always store to remote contacts even if we're the closest known node to the hash
-move the store call and error handling from announceHaveBlob to a smaller function of its own
2018-06-07 15:02:47 -04:00
Jack Robison
9a63db4ec6
add protocol version to the dht and migrate old arg format for store
2018-06-07 15:02:47 -04:00
Jack Robison
7d21cc5822
pylint and more tests
2018-06-07 15:02:47 -04:00
Jack Robison
cce3c8c7b5
increase kademlia rpc timeout to 8 seconds
2018-06-07 15:02:47 -04:00
Jack Robison
0e80123615
use 12 minutes instead of 15 as delay in contact_is_good
2018-06-07 15:02:47 -04:00
Jack Robison
659632b66c
fix and update tests
2018-06-07 15:02:47 -04:00
Jack Robison
9582b7fcf5
use maybe_ping for refreshing stale buckets and storing peers
...
-move store refresh to its own looping call
2018-06-07 15:02:47 -04:00
Jack Robison
945da5985e
fix age used in datastore to determine if a value is expired
2018-06-07 15:02:47 -04:00
Jack Robison
921ee3c4c1
use refreshTimeout in getRefreshList
2018-06-07 15:02:47 -04:00
Jack Robison
8efede6ad6
maybe_ping bad and unknown contacts instead of only unknown
2018-06-07 15:02:47 -04:00
Jack Robison
fb3aac15fa
add optional delay argument to enqueue_maybe_ping, fix default value
2018-06-07 15:02:47 -04:00
Jack Robison
73e813f9ec
verify key size in Distance
2018-06-07 15:02:47 -04:00
Jack Robison
0d23c68797
raise attribute error for non-rpc functions in Contact
2018-06-07 15:02:47 -04:00
Jack Robison
ec1b6b2387
comments, cleaner key_bits constant
2018-06-07 15:02:47 -04:00
Jack Robison
29d5750371
pylint
2018-06-07 15:02:47 -04:00
Jack Robison
07f92014d7
omit bad contacts from getPeersForBlob
...
-refresh stored contacts to detect when they go offline or come back
2018-06-07 15:02:47 -04:00
Jack Robison
c521120b17
update and fix hash announcer test
2018-06-07 15:02:47 -04:00
Jack Robison
d02ed29e50
add kademlia store and expiration test
2018-06-07 15:02:47 -04:00
Jack Robison
877da78505
ping contacts right away during refresh instead of using PingQueue
...
-fixes contact expiration test
2018-06-07 15:02:47 -04:00
Jack Robison
a952d2d2c8
reset _listeningPort and _listening Deferred on teardown
2018-06-07 15:02:47 -04:00
Jack Robison
3dfc6bd2cc
update CallLaterManager to be an object
2018-06-07 15:02:47 -04:00
Jack Robison
760417ff3a
pylint
2018-06-07 15:02:47 -04:00
Jack Robison
4f72098cad
use PingQueue to try refresh all contacts
2018-06-07 15:02:47 -04:00
Jack Robison
9920ff59d4
force KBucket refreshes
2018-06-07 15:02:47 -04:00
Jack Robison
aee7a3aa38
simplify announceHaveBlob, remove unused getPeersForBlob
2018-06-07 15:02:47 -04:00
Jack Robison
372fb45e06
refactor dht bootstrap
...
after finding the closest nodes try to populate the buckets out by looking up random ids in their key ranges
2018-06-07 15:02:47 -04:00
Jack Robison
c65274e9e5
add PingQueue to KademliaProtocol
2018-06-07 15:02:47 -04:00
Jack Robison
1adf4f7818
fix constant used to check if a bucket is fresh
2018-06-07 15:02:47 -04:00
Jack Robison
05241012a2
update contact replacement in KBucket to follow BEP0005
...
http://www.bittorrent.org/beps/bep_0005.html
2018-06-07 15:02:47 -04:00
Jack Robison
cf3359044d
fix conditions for when a kbucket should be split
...
https://stackoverflow.com/a/32187456
2018-06-07 15:02:47 -04:00
Jack Robison
5631a24881
improve findCloseNodes, choose closest contacts from higher and lower buckets
2018-06-07 15:02:47 -04:00
Jack Robison
c654bfe296
use reactor clock in TreeRoutingTable instead of time module
2018-06-07 15:02:47 -04:00
Jack Robison
d4e28216a0
sort KBucket.getContacts
2018-06-07 15:02:47 -04:00
Jack Robison
95ed1e044b
raise TransportNotConnected instead of logging a warning
...
-add a _listening Deferred to KademliaProtocol which is called when the protocol is started
2018-06-07 15:02:47 -04:00
Jack Robison
f1e0a784d9
refactor iterativeFind, move to own file
2018-06-07 15:02:47 -04:00
Jack Robison
e5703833cf
prevent duplicate entries in the datastore
2018-06-07 15:02:47 -04:00
Jack Robison
23c202b5e4
refactor Contact class, DHT RPCs, and Contact addition/removal
...
-track contact failures, last replied, and last requested. use this to provide a 'contact_is_good' property on Contact objects
-ensure no duplicate contact objects are created
-remove confusing conflation of node id strings with Contact objects, update docstrings
-move RPC failure tracking to a callback/errback pair in sendRPC (so the contact is only updated once)
-handle seed nodes during the join sequence by setting their node ids after they initially reply to our ping
-name all of the kademlia RPC keyword args, remove confusing **kwargs and dictionary parsing
-add host ip/port to DHT send/receive logging to make the results comprehensible when running many nodes at once
2018-06-07 15:02:47 -04:00
Jack Robison
ad2dcf0893
add the parent node id to KBucket
2018-06-07 15:02:47 -04:00
Jack Robison
406ddaa4ef
use base class to simplify Node init
...
-add looping call helpers which use the same clock as the Node
2018-06-07 15:02:47 -04:00
Jack Robison
e52689a33d
remove OptimizedTreeRoutingTable for now, use TreeRoutingTable
2018-06-07 15:02:47 -04:00
Jack Robison
d65dc0aec3
disable loading DictDataStore in Node.__init__
...
-to be re-done when the datastore uses sqlite
2018-06-07 15:02:47 -04:00
Jack Robison
159e153393
make DataStore clock mockable
2018-06-07 15:02:47 -04:00
Jack Robison
9ed08f8fc9
remove unused constant
2018-06-07 15:02:47 -04:00
Jack Robison
4bd9f3bd68
remove popular hash tracking, simplify DHTPeerFinder
2018-06-07 15:02:47 -04:00
Jack Robison
e6ffd7caf8
remove hashwatcher
2018-06-07 15:02:47 -04:00
Jack Robison
11aad04c31
datagram decode error logging
2018-04-19 15:47:42 -04:00
Jack Robison
b89823c968
fix blob announce locking up the daemon process
2018-04-19 15:47:19 -04:00
Jack Robison
5cea031f38
return list of dictionaries from peer_list, include peer node ids
2018-04-03 13:10:30 -04:00
Jack Robison
51662b1e5b
re-add hash_queue_size function
2018-03-29 15:06:08 -04:00
Jack Robison
492858596e
add single_announce column to blob table
...
-remove deprecated blob_announce_all function
-remove announce_all parameter to blob_announce
-change blob_announce to be asynchronous
2018-03-28 18:50:24 -04:00
Jack Robison
dff1fd3fe9
logging, raise default concurrent announcers
2018-03-28 18:50:24 -04:00
Jack Robison
eabf4a0e40
remove delay from udp write
2018-03-28 18:50:24 -04:00
Jack Robison
5bab6f7d39
remove bandwidth stats
2018-03-28 18:50:24 -04:00
Jack Robison
a41bbd5e27
pylint and tests
2018-03-28 18:50:18 -04:00
Jack Robison
43d3f7c087
add concurrent_announcers to config
2018-03-28 15:53:51 -04:00
Jack Robison
c5bf64cf0a
refactor DHTHashAnnouncer
...
-remove hash_announcer from Node and DiskBlobManager
-remove announcement related functions from DiskBlobManager
-update SQLiteStorage to store announcement times and provide blob hashes needing to be announced
-use dataExpireTimeout from lbrynet.dht.constants for re-announce timing
-use DeferredSemaphore for concurrent blob announcement
2018-03-28 15:53:51 -04:00
Jack Robison
ea0ea704a2
refactor iterativeAnnounceHaveBlob
...
-change to only self_store if the number of contacts to store to is less than k and we are the closest node to the hash
2018-03-28 15:53:51 -04:00
Jack Robison
14f9bb7b82
log EWOULDBLOCK
2018-03-28 15:53:51 -04:00
Jack Robison
a8025b02c6
log invalid vs missing token
2018-03-28 15:53:51 -04:00
Jack Robison
a96d827c0f
use reactor time in Delay
2018-03-28 15:53:50 -04:00
Jack Robison
4eab77fa10
safe start_listening
2018-03-28 15:53:50 -04:00
Jack Robison
5013426e02
logging and docstring
2018-03-28 15:53:50 -04:00
Jack Robison
cb09be5336
remove unused stuff
2018-03-28 15:53:50 -04:00
Jack Robison
5628d0825b
add CallLaterManager
2018-03-28 15:53:49 -04:00
Jack Robison
88970cb0a8
move peer manager class to lbrynet.core
2018-03-28 15:53:49 -04:00
Jack Robison
87c69742cd
log packet encoding errors and warn if the transport is not connected
2018-03-28 15:53:48 -04:00
Jack Robison
b4bc5e2110
cancel callLater on error or timeout
2018-03-28 15:53:48 -04:00
Jack Robison
df78f7ff9f
add response assertion to announce_to_peer
2018-03-28 15:53:48 -04:00
Jack Robison
bdba263224
catch TimeoutError in _IterativeFindHelper
2018-03-28 15:53:48 -04:00
Jack Robison
16fcc3f5c1
findValue inlinecallbacks refactor
2018-03-28 15:53:48 -04:00
Jack Robison
43896c8d17
refactor joinNetwork into smaller functions
...
-try to re-join network if no contacts are known
2018-03-28 15:53:47 -04:00
Jack Robison
e6caedac91
remove DHTHashSupplier class, move former functions into DHTHashAnnouncer
2018-03-28 15:53:47 -04:00
Jack Robison
6666468640
add reactor arguments to Node
...
-adds reactor (clock) and reactor functions listenUDP, callLater, and resolve as arguments to Node.__init__
-set the reactor clock on LoopingCalls to make them easily testable
-convert callLater manage loops to LoopingCalls
2018-03-28 15:53:47 -04:00
Jack Robison
efaa97216f
move dht node setup back into node class
2018-03-28 15:53:47 -04:00
Jack Robison
e30ea50ef4
more
2018-03-28 15:53:47 -04:00
Jack Robison
3296c0fb3d
move dht related classes to lbrynet.dht
2018-03-28 15:53:47 -04:00
Jack Robison
ad6a2bef7f
handle error from old clients with a broken ping command
2018-03-28 15:53:45 -04:00
Jack Robison
446c3a88dc
refactor DHTHashAnnouncer and iterativeAnnounceHaveBlob
...
-use looping call for running manage function rather than a scheduled
callLater
-track announce speed
-retry store requests that failed up to 3 times
-return a dict of {blob_hash: [storing_node_id]} results from
_announce_hashes
_refreshRoutingTable inline cb refactor
-add and use DeferredLockContextManager
-don't trap errback from iterativeFindNode in iterativeAnnounceHaveBlob
2018-03-28 15:53:45 -04:00
