This commit is a rather large one which implements transaction pool and
relay according to the protocol rules of the reference implementation.
It makes use of btcchain to ensure the transactions are valid for the
block chain and includes several stricter checks which determine if they
are "standard" or not before admitting them into the pool and relaying
them.
There are still a few TODOs around the more strict rules which determine
which transactions are willing to be mined, but the core checks which
are imperative (everything except the all of the "standard" checks really)
to operate as a good citizen on the bitcoin network are in place.
We originally wanted to also not fetch orphan parents in this commit, however,
I have discovered that if you are doing a main sync from a peer, if it
sends you an orphan you must fetch it, else you ahven't fetched
everything it told you about and thus it will nto send you end more invs
from the main sync.
So we always fetch orphan parents, but we still don't fetch from
non-sync peers (all invs from them will be unsolicited). Seems to fix some hangs
with multiple peers.
The "official" regression test tool intentionally sends some unrequested
duplicate blocks to ensure the chain handling code does not fail when
trying to insert them. This commit adds an exception to the block manager
which typically disconnects peers that send unrequested blocks (they are
misbehaving if they do this) for regression test mode.
Really, it would be nice to pass an interface{} into chain to be given
to us when the callback calls, it would avoid the awkward sidchanneling
through the map and should actually be more efffieint (pointer passing >
hashtable insert, lookup, then remove).
Rather than having all of the various places that print peer figure out
the direction and form the string, centralize it by implementing the
Stringer interface on the peer.
Chain is not concurrency safe, so we move the chainNotifySink handling
into the main blockmanager goroutine. Due to a possible deadlock if the
buffer is filled this still has to be a single channel that isn't linked
to the other ones. There is a possible starvation issue where the main
msgChan gets selected more often than the notification sink, but until
chain is concurrency safe this is rather unavoidable.
Only log errors for most cases if the peer is persisent (and thus requested).
Only log by default after version exchange, and after losing a peer that had
completed version exchange. Make most other messages debug.
We would occasionally hang or a while during server shudown, this is due
to an outbound peer waiting on a connection or a sleep. However, we
don't actually require to wait for the peers to finish at all. So just
let them finish.
Secondly, make peer.disconnnect and server.shutdown atomic varaibles so
that checking them from multiple goroutines isn't race, and clean up
their usage.
Use this information so that we do not request a block per peer we got
an inv for it, makes multi peer much quieter and rather more bandwidth
efficient.
In order to remove a number of possible races we combine blockhandling
an synchandler and use one channel for all messages. This ensures that
all messages from a single peer will be recieved in order. It also
removes the need for a lot of locking between the peer removal code and
the block/inv handlers.
Implement the bucketing by source group and group using essentially the
same algorithm as the address maanger in bitcoind.
Fix up the saving of peer.json to do so in a json format that keeps bucket
metadata.
If we fail to load the some of the data we asssume that we have
incomplete information, so we nuke the existing file and reinitialise so
we have a clean slate.
Previously a new goroutine was launched for each notification in order to
avoid blocking chain from continuing while the notification is being
processed. This approach had a couple of issues.
First, since goroutines are not guaranteed to execute in any given order,
the notifications were no longer handled in the same order as they were
sent. For the current code, this is not a problem, but upcoming code that
handles a transaction memory pool, the order needs to be correct.
Second, goroutines are relatively cheap, but it's still quite a bit of
overhead to launch 3-4 goroutines per block.
This commit modifies the handling code to have a single sink executing in
a separate goroutine. The main handler then adds the notifications to a
queue which is processed by the sink. This approach retains the
non-blocking behavior of the previous approach, but also keeps the order
correct and, as an additional benefit, is also more efficient.
This removes a horrible case of reach-around from per into the guts of
the blockmaanger to frob the chain. Soon, when we try to deduplicate the
fetching of blocks from multiple peers this will need decisions made in
a central point.
Discussed at length with davec.
