From 3cfc19063f805465b89644d76e30c52ca4a33e2e Mon Sep 17 00:00:00 2001 From: Alex Grin Date: Fri, 22 Oct 2021 10:34:36 -0400 Subject: [PATCH] Created DHT Resources (markdown) --- DHT-Resources.md | 55 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 DHT-Resources.md diff --git a/DHT-Resources.md b/DHT-Resources.md new file mode 100644 index 0000000..613ea48 --- /dev/null +++ b/DHT-Resources.md @@ -0,0 +1,55 @@ +Some links I found useful while doing DHT work + +[http://xlattice.sourceforge.net/components/protocol/kademlia/specs.html](http://xlattice.sourceforge.net/components/protocol/kademlia/specs.html) + +[http://www.maymounkov.org/papers/maymounkov-kademlia-lncs.pdf](http://www.maymounkov.org/papers/maymounkov-kademlia-lncs.pdf) ([https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf](https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf) alternative as it was offline) + +[http://blog.notdot.net/2009/11/Implementing-a-DHT-in-Go-part-2](http://blog.notdot.net/2009/11/Implementing-a-DHT-in-Go-part-2) + +[http://www.scs.stanford.edu/17au-cs244b/labs/projects/kaplan-nelson_ma_rachleff.pdf](http://www.scs.stanford.edu/17au-cs244b/labs/projects/kaplan-nelson_ma_rachleff.pdf) + +[http://www.bittorrent.org/beps/bep_0005.html](http://www.bittorrent.org/beps/bep_0005.html) + +[blog.libtorrent.org/2016/09/dht-bootstrap-node](https://blog.libtorrent.org/2016/09/dht-bootstrap-node) + +[https://github.com/lbryio/reflector-cluster/issues/41](https://github.com/lbryio/reflector-cluster/issues/41) + +torrent specific: [https://wiki.theory.org/BitTorrentDraftDHTProtocol](https://wiki.theory.org/BitTorrentDraftDHTProtocol) and torrent general: [https://wiki.theory.org/BitTorrentSpecification](https://wiki.theory.org/BitTorrentSpecification) + +## Naive IterativeFind description + +_grin made this up_ + +naive iterativeFind description (bounded parallelism, not loose parallelism): + +`target` = hash that you're looking for + +`activeNodes = []`, always sorted by distance to target + +`toContact = []`, always sorted by distance to target + +`contacted = {}` + +start! + +put all nodes in routing table into the toContact list + +run `ALPHA` workers + +each worker will asynchronously: + +- pop first node in toContact list, mark it as contacted, contact it +- if node does not reply in some time, drop it and move on to next node +- if node replies, insert it into activeNodes +- if node replies with the value for FindValue call, return that value and immediately end the whole process +- if node replies with contacts, insert the contacts that have not been contacted yet into toContact +- if `len(toContact) == 0` (and there are no outstanding request) or `len(activeNodes) >= K && dist(target, activeNodes[K-1]) < dist(target, toContact[0])`, quit + +if FindValue found a value, return it. otherwise return first K activeNodes + +caveats: + +- if you're doing the optimization of caching FindValue results at the closest node that didn't have it, you can't simply stop the search when you get a value. you have to keep going so you can find the closest node that didn't return a value +- this algorithm doesn't stop early when its unlikely to find closer nodes. to do this, you have to change two things + - only add the K closest nodes to the shortlist at the start of the search + - for the termination condition, track the closest node seen so far. if responses from the shortlist are not getting you any closer to the target, stop searching \ No newline at end of file