shyba
eab95a6246
DHT fixes from review and an attempt at removing hashing and equals ( #1370 )
...
* use int to_bytes/from_bytes instead of struct
* fix ping queue bug and dht functional tests
* run functional tests on travis
* re-add contact comparison unit test
* dont need __ne__ if its just inverting __eq__ result
2018-08-24 11:42:35 -04:00
Victor Shyba
3f6e928cc2
fix findValue result parsing
2018-08-24 11:40:43 -04:00
Lex Berezhny
10b34d6b33
unnecessary list() added during py3 port
...
instead of recursive bytes2unicode use a proper JSONEncoder to conver bytes->unicode for json.dumps()
removing excessive isinstance(data, bytes) checks
py3: / -> // and list() around .items() that gets modified in loop
moved lbrynet.undecorated to where its actually used and hopefully we can delete it eventually
removed build/upload_assets.py, travis can do all this now
2018-08-24 11:40:43 -04:00
Victor Shyba
31630a84be
reduce entropy on DHT test suite
2018-08-24 11:40:43 -04:00
hackrush
473d2eabfa
curl works with python3 now
2018-08-24 11:37:53 -04:00
Victor Shyba
78c8c8e64d
more porting, plus some functional tests working
2018-08-24 11:37:53 -04:00
Lex Berezhny
4ece422f48
No longer inheriting from object and added proper use of super().
2018-08-24 11:37:53 -04:00
Victor Shyba
e1314a9d1e
working functional test_contact_rpc + more string bans
2018-08-24 11:37:53 -04:00
Victor Shyba
19211d4417
make sure bencoding works for bytes, not strings
2018-08-24 11:37:53 -04:00
Victor Shyba
e4ea1ccbfb
test_node green on py3
2018-08-24 11:37:53 -04:00
Lex Berezhny
5520d518b5
DHT py3 compatibility, mostly commenting out implements() and fixing imports
...
cryptstream py3 support, mostly commenting out implements()
lbry_file py3 support, mostly commenting out implements()
file_manager py3 support, mostly commenting out implements()
core py3 support, mostly commenting out implements() and fixing imports
2018-08-24 11:37:25 -04:00
Jack Robison
26e32472d5
fix upnp and peer port setup
...
-bump txupnp requirement
2018-08-07 19:04:54 -04:00
Jack Robison
f1b19f10cc
Merge branch 'master' into greedy_search
2018-08-03 13:08:31 -04:00
Jack Robison
b06dcf0a0d
cancel starting components if the reactor is stopped before startup has finished
...
-don't block starting the dht component on having found enough peers, only on setting up the protocol
2018-07-24 20:01:38 -04:00
Jack Robison
d29f103910
pylint
2018-07-17 17:38:19 -04:00
Jack Robison
ec51333b56
allow node udp interface to be configured
2018-07-17 17:13:33 -04:00
Victor Shyba
c589e89bf0
greedy search with exclude filtering
2018-07-05 22:03:52 -03:00
Jack Robison
b6289d101d
fix improper sorting when getting the closest peers to a hash
2018-07-02 14:54:03 -04:00
Jack Robison
7fe92d2df0
load all finished blobs into the node datastore
2018-07-02 14:31:09 -04:00
Jack Robison
03769b94b8
accept invalid store tokens for the first expire-time after startup
...
this is to better handle nodes storing to us after we've restarted our node
2018-07-02 13:26:38 -04:00
Jack Robison
e6df378e9e
fix token validation error when the dht node has just been started ( https://github.com/lbryio/lbry/issues/1248 )
2018-06-14 15:07:10 -04:00
Jack Robison
7f3ead67bf
disable forced bucket refresh during join
2018-06-07 15:02:47 -04:00
Jack Robison
ae631f05c3
fix teardown error
2018-06-07 15:02:47 -04:00
Jack Robison
bc0da5e2d1
only use seeds in iterative bootstrap if no contacts are known yet
2018-06-07 15:02:47 -04:00
Jack Robison
44644673d7
add profiler
2018-06-07 15:02:47 -04:00
Jack Robison
42eb172638
refactor announceHaveBlob
...
-add cached `token` to Contact objects to minimize findValue requests
-remove self_store, always store to remote contacts even if we're the closest known node to the hash
-move the store call and error handling from announceHaveBlob to a smaller function of its own
2018-06-07 15:02:47 -04:00
Jack Robison
9a63db4ec6
add protocol version to the dht and migrate old arg format for store
2018-06-07 15:02:47 -04:00
Jack Robison
9582b7fcf5
use maybe_ping for refreshing stale buckets and storing peers
...
-move store refresh to its own looping call
2018-06-07 15:02:47 -04:00
Jack Robison
29d5750371
pylint
2018-06-07 15:02:47 -04:00
Jack Robison
07f92014d7
omit bad contacts from getPeersForBlob
...
-refresh stored contacts to detect when they go offline or come back
2018-06-07 15:02:47 -04:00
Jack Robison
d02ed29e50
add kademlia store and expiration test
2018-06-07 15:02:47 -04:00
Jack Robison
877da78505
ping contacts right away during refresh instead of using PingQueue
...
-fixes contact expiration test
2018-06-07 15:02:47 -04:00
Jack Robison
a952d2d2c8
reset _listeningPort and _listening Deferred on teardown
2018-06-07 15:02:47 -04:00
Jack Robison
3dfc6bd2cc
update CallLaterManager to be an object
2018-06-07 15:02:47 -04:00
Jack Robison
4f72098cad
use PingQueue to try refresh all contacts
2018-06-07 15:02:47 -04:00
Jack Robison
9920ff59d4
force KBucket refreshes
2018-06-07 15:02:47 -04:00
Jack Robison
aee7a3aa38
simplify announceHaveBlob, remove unused getPeersForBlob
2018-06-07 15:02:47 -04:00
Jack Robison
372fb45e06
refactor dht bootstrap
...
after finding the closest nodes try to populate the buckets out by looking up random ids in their key ranges
2018-06-07 15:02:47 -04:00
Jack Robison
95ed1e044b
raise TransportNotConnected instead of logging a warning
...
-add a _listening Deferred to KademliaProtocol which is called when the protocol is started
2018-06-07 15:02:47 -04:00
Jack Robison
f1e0a784d9
refactor iterativeFind, move to own file
2018-06-07 15:02:47 -04:00
Jack Robison
23c202b5e4
refactor Contact class, DHT RPCs, and Contact addition/removal
...
-track contact failures, last replied, and last requested. use this to provide a 'contact_is_good' property on Contact objects
-ensure no duplicate contact objects are created
-remove confusing conflation of node id strings with Contact objects, update docstrings
-move RPC failure tracking to a callback/errback pair in sendRPC (so the contact is only updated once)
-handle seed nodes during the join sequence by setting their node ids after they initially reply to our ping
-name all of the kademlia RPC keyword args, remove confusing **kwargs and dictionary parsing
-add host ip/port to DHT send/receive logging to make the results comprehensible when running many nodes at once
2018-06-07 15:02:47 -04:00
Jack Robison
406ddaa4ef
use base class to simplify Node init
...
-add looping call helpers which use the same clock as the Node
2018-06-07 15:02:47 -04:00
Jack Robison
e52689a33d
remove OptimizedTreeRoutingTable for now, use TreeRoutingTable
2018-06-07 15:02:47 -04:00
Jack Robison
d65dc0aec3
disable loading DictDataStore in Node.__init__
...
-to be re-done when the datastore uses sqlite
2018-06-07 15:02:47 -04:00
Jack Robison
4bd9f3bd68
remove popular hash tracking, simplify DHTPeerFinder
2018-06-07 15:02:47 -04:00
Jack Robison
e6ffd7caf8
remove hashwatcher
2018-06-07 15:02:47 -04:00
Jack Robison
b89823c968
fix blob announce locking up the daemon process
2018-04-19 15:47:19 -04:00
Jack Robison
5cea031f38
return list of dictionaries from peer_list, include peer node ids
2018-04-03 13:10:30 -04:00
Jack Robison
5bab6f7d39
remove bandwidth stats
2018-03-28 18:50:24 -04:00
Jack Robison
a41bbd5e27
pylint and tests
2018-03-28 18:50:18 -04:00
