This commit provides a new interface, MedianTimeSource, along with a
concrete implementation which allows improved accuracy of time by making
use of the median network time as calculated from multiple time samples.
The time samples are to be provided by callers and are intended to come
from remote clients.
The calculations performed in this implementation exactly mirror those in
Bitcoin Core because time calculations are part of the consensus rules and
hence need to match exactly.
Since the underlying database driver can now return an error when looking
up if blocks and transactions exist, the HaveBlock function now includes
an error return to allow any underlying errors to be exposed.
This commit adds a new behavior flag, BFDryRun which allows the caller
to indicate all checks should be performed against the block as normal
except it will not modify any state. This is useful to test that a block
is valid without actually modifying the current chain or memory state.
This commit also adds a few additional checks which were elided before
since they are implicitly handled by btcwire. However, with the ability
to propose blocks which didn't necessarily come through the btcwire path,
these checks need to be enforced in the chain code as well.
As a part of adding the checks, three new error codes named
ErrBlockTooBig, ErrTooManyTransactions, and ErrTxTooBig have been
introduced.
Closes#5.
This commit change the ProcessBlock function to accept a new type named
BehaviorFlags in place of the current fastAdd parameter.
This has been done to pave the way for adding more control over the checks
that are performed such as avoiding the proof of work checks which will be
in an upcoming commit. A bitmask was chosen because it will allow the
ProcessBlock API to remain a little more stable since new flag additions
that change the behavior are only likely to be used by new code because
adding flags does not change the existing behavior.
ok @jrick
This commit refactors the code to make use of the btcnet package for
network-specific parameters instead of switching on the specific network.
For example, the percentage of the network that needs to run version 2
blocks is different between testnet and mainnet. Previously the code was
using a switch to choose these values. With this refactor, those
parameters are part of the network parameters provided when creating a new
chain instance.
Another example is checkpoints, which have been moved to btcnet so they
can be specified by the caller instead of hard coded into this package.
As a result, the checkpoints for the standard networks are now specified
in the btcnet package.
This makes it easier to add new networks such as a testnet4 should it
become needed. It also allows callers to define their own custom network
parameters without having to modify the code of the package to add new
switch cases for the custom network.
This commit makes a slight variant of the existing calcPastMedianTime
function available to external callers. The new exported version
calculates the past median time from the end of the current main chain
versus an arbitrary block node.
This commit makes a slight variant of the existing checkConnectBlock
function available to external callers. The new exported version checks
if the passed block will connect to the end of the current main chain.
In order to support this, a few other small modifications have been made
to the initial index generation and the existing checkConnectBlock since
it previously made some assumptions about the state of genesis block which
can no longer be assumed due accepting blocks from callers directly.
Also, add a quick test to ensure the new function fails when checking if
the genesis block will connect again when it's already inserted.
This commit resolves an issue where the block node index was forcing
entire blocks to be kept in memory thereby forcing excessive memory usage.
For example, prior to this change, the memory usage could consume upwards
of 1.5GB while importing bootstrap.dat via the addblock utility. With
this change the entire import takes <150MB. This also has the same memory
reduction to btcd since it uses the same code path.
Previously the code was only adding a new block node as a child in the
inernal node index for the cases it extended the main chain or a side
chain and not for a node which caused a reorg. This resulted in the block
node pruning code not clearing the parent link of reorged nodes which
ultimately led to a sanity check error accordingly.
This commit resolves the issue by ensuring new block nodes are added as
children of their respective parents in all cases.
Closes#4.
This commit changes the node index creation to use block headers instead
of full blocks. This speeds up the initial node index generation since it
doesn't require loading a bunch of full blocks at startup.
Previously the code performed a database query for every checkpoint (going
backwards) to find the latest known checkpoint on every block. This was
particularly noticabled near the beginning of the block chain when there
are still several checkpoints that haven't been reached yet.
This commit changes the logic to cache the latest known checkpoint while
keeping track of when it needs to be updated once a new later known
checkpoint has been reached.
While here, also add a log message when a checkpoint has been reached and
verified.
It is not necessary to do all of the transaction validation on
blocks if they have been confirmed to be in the block chain leading
up to the final checkpoint in a given blockschain.
This algorithm fetches block headers from the peer, then once it has
established the full blockchain connection, it requests blocks.
Any blocks before the final checkpoint pass true for fastAdd on
btcchain operation, which causes it to do less valiation on the block.
Rather than fetching the hash of each block individually 2k+ times, make
use of the FetchHeightRange function so all of the most recent hashes can
be fetched at once.
The original thought was that chain would also house the transaction
memory pool, but that ultimately was decided against. As a result,
it only makes sense to query chain for blocks rather than generic
inventory.
Rather than using a channel for notifictions, use a callback instead.
There are several issues that arise with async notifications via a
channel, so use a callback instead. The caller can always make the
notifications async by using channels in the provided callback if
needed.
This commit modifies the GenerateInitialIndex function to update the best
chain with each node as it is loaded. This allows the best chain to be
set immediately upon generating the initial index which is a slight
performance optimization.
This commit provides a new exported function, HaveInventory, which can
be used to determine if the already has the item referenced by passed
inventory vector. Currently it only provides support for blocks and
cursory support for transactions in the main chain. Types that are
unrecognized are returned as if they unknown as expected.
This commit provides a new exported function, IsKnownOrphan, which can
be used to determine if the passed hash is currently already known to the
chain as an orphan.
This commit exposes GetOrphanRoot to callers. It also modifies the code
to allow the function to be called in a concurrent safe manner. This
allows the information to be safely obtained from multiple goroutines.
It is not safe to remove elements from a slice while iterating them with
the range statement since it does not reevaluate the slice on each
iteration nor does it adjust the index for the modified slice. This
commit modifies the code to use a for loop with an index (which does
reevaluate on every iteration) and manually adjusts the index when
elements are removed from the slice.
This commit adds a new optional function named GenerateInitialIndex. It
generates the required number of initial block nodes in an optimized
fashion. Since the memory block index is sparse and previous nodes are
dynamically loaded as needed, this function is not stricty needed.
However, during initial startup (when there are no nodes in memory yet),
dynamically loading all of the required nodes on the fly in the usual way
is much slower than preloading them.
The recent pruning code made the parent hash for a node available directly
as a field in the node. Make use of this in the getPrevNodeFromNode
function to avoid having to load the full block data associated with the
node to get the parent hash.
This commit implements pruning for the block nodes that form the memory
chain which are no longer needed. The validation of a block only requires
information from a set number of previous nodes. The initial code did not
prune old nodes as they were no longer needed, but the vast majority of
the infrastructure to support it was already in place as that was always
the goal. This commit finishes implementing the missing bits to make it
a reality.
Previously, the code was using big rational numbers for work values which
resulted in carrying way too much precision around (and ultimately a lot
of extra memory and computation to carry that precision). This commit
converts the work values to big integers and calculates them with integer
division. This is acceptable because the numerator is multiplied by 2^256
which is higher than the maximum possible proof of work. Therefore
anything after the decimal is superfluous precision for the purposes of
chain selection.
Also, add a check for negative difficulty values when calculating the work
value. Negative values won't occur in practice with valid blocks, but
it's possible an invalid block could trigger the code path, so be safe and
check for it.
This commit adds info level log statements when a block causes a chain
fork, extends a side chain (fork), or causes a reorganize. When a reorg
happens, the new and old chain heads along with the fork point are logged.
In addition to returning errors to the caller, log the error with a prefix
in a few key places that helps identify the origin for errors. In some
cases, the underlying error comes from a different subsystem such as the
SQL database driver and the error messages can be fairly generic.
This commit modifies the code to use params based on the active network
which paves the way for supporting the special rules and different genesis
blocks used by the test networks.
