We would occasionally hang or a while during server shudown, this is due
to an outbound peer waiting on a connection or a sleep. However, we
don't actually require to wait for the peers to finish at all. So just
let them finish.
Secondly, make peer.disconnnect and server.shutdown atomic varaibles so
that checking them from multiple goroutines isn't race, and clean up
their usage.
Use this information so that we do not request a block per peer we got
an inv for it, makes multi peer much quieter and rather more bandwidth
efficient.
In order to remove a number of possible races we combine blockhandling
an synchandler and use one channel for all messages. This ensures that
all messages from a single peer will be recieved in order. It also
removes the need for a lot of locking between the peer removal code and
the block/inv handlers.
Implement the bucketing by source group and group using essentially the
same algorithm as the address maanger in bitcoind.
Fix up the saving of peer.json to do so in a json format that keeps bucket
metadata.
If we fail to load the some of the data we asssume that we have
incomplete information, so we nuke the existing file and reinitialise so
we have a clean slate.
PayToPubKeyHashScript generates a new pay to pubkey hash script to use
as the pkScript when creating new transactions. If the passed pubkey
hash is an invalid size, StackErrInvalidOpcode will be returned as an
error.
SignatureScript returns the signature script necessary to validate a
single input of a transaction.
This also adds sanity checking for serializing scripts into byte
slices. If the length of a serialized opcode does not equal the
expected length, StackErrInvalidOpcode will be returned when unparsing
a []parsedOpcode.
New internal tests were added to verify checks for valid and invalid
parsed opcodes.
This commit adds an exported function, IsPushOnlyScript, which can be used
to tell whether or not a script only pushes data (only contains opcodes
that push data).
Several of the functions require a map of contextual transaction data to
use as a source for referenced transactions. This commit exports the
underlying TxData type and creates a new type TxStore, which is a map of
points to the under TxData. In addition, this commit exposes a new
function, FetchTransactionStore, which returns a transaction store
(TxStore) containing all of the transactions referenced by the passed
transaction, as well as the existing transaction if it already exists.
This paves the way for subsequent commits which will expose some of the
functions which depend on this transaction store.
Previously a new goroutine was launched for each notification in order to
avoid blocking chain from continuing while the notification is being
processed. This approach had a couple of issues.
First, since goroutines are not guaranteed to execute in any given order,
the notifications were no longer handled in the same order as they were
sent. For the current code, this is not a problem, but upcoming code that
handles a transaction memory pool, the order needs to be correct.
Second, goroutines are relatively cheap, but it's still quite a bit of
overhead to launch 3-4 goroutines per block.
This commit modifies the handling code to have a single sink executing in
a separate goroutine. The main handler then adds the notifications to a
queue which is processed by the sink. This approach retains the
non-blocking behavior of the previous approach, but also keeps the order
correct and, as an additional benefit, is also more efficient.
This removes a horrible case of reach-around from per into the guts of
the blockmaanger to frob the chain. Soon, when we try to deduplicate the
fetching of blocks from multiple peers this will need decisions made in
a central point.
Discussed at length with davec.
This commit modifies the code to choose sane defaults for the backing
arrays for slices that involve a lot of appends such block locators, hash
processing, and needed transactions. This is an optimization to avoid
the overhead of growing the backing arrays and copying the data multiple
times in the most common case. This also prevents a leak in Go GC which
will likely ultimatley be fixed, but the efficiecy gains alone are worth
the change.
Removing from the bottom of a stack (nipN(depth)) would leak the first
entry in the array by slicing [1:], leaving array[0] dangling an
inaccessible (but unable to be freed until the whole slice is gone). We
left it like this for a while, but best not to leak the memory. Happens
rarely so the performance hit shouldn't matter that much. Do the same
thing for condstack.
Several of the messages store the parts that have a variable number of
elements as slices. This commit modifies the code to choose sane defaults
for the backing arrays for the slices so when the entries are actually
appended, a lot of the overhead of growing the backing arrays and copying
the data multiple times is avoided.
Along the same lines, when decoding messages, the actual size is known and
now is pre-allocated instead of dynamically growing the backing array
thereby avoiding some overhead.
When we do append loops, make an educated guess as to the size and make an array
with that capacity to avoid extra copying.
Doesn't affect the speed of the tests, over 4 runs the difference was lost in
the noise.
After discussion with others and thinking about the notification channel
some more, we've decided to leave it up to the caller to quickly handle
notifications. While it is true that notification should be handled
quickly to not block the chain processing code unnecessarily, launching a
goroutine in chain means the notifications are no longer necessarily in
order. Also, if the caller is not properly handling the notifications,
the goroutines end up sicking around forever. By leaving it up to the
caller to quickly handle the notification or launch a goroutine as
necessary for the caller, it provides the flexibility to ensure proper
notification ordering as well as control over other things such as
how to handle backpressure.
This commit adds detection and filtering for back-to-back duplicate
getblocks requests. This is needed because the trigger for requesting
more blocks is receiving an orphan. When the peer is further behind than
the number of blocks advertised via a single inventory message, the same
orphan block will be sent multiple times. When the peer receives the
final inventory message, it too contains the orphan that was previously
sent. This leads to a duplicate getblocks request that must be filtered
to prevent requesting the final series of blocks again.
This commit adds environment variable expansion and path cleaning to the
data directory. This allows the user to specify data paths in the config
file such as datadir=~/.btcd/data and datadir=$SOMEVAR/btcd. It also
adds usage instructions and an example to the sample btcd.conf file.
