The recent addition of the fast add path to support headers first was not
running the block node index pruning code which removes unneeded block
nodes from memory. This resulted in higher memory usage than needed in
fast add mode.
This commit changes the node index creation to use block headers instead
of full blocks. This speeds up the initial node index generation since it
doesn't require loading a bunch of full blocks at startup.
This commit modifies local variables that are used for more convenient
access to a block's header to use pointers. This avoids copying the
header multiple times.
It is not necessary to do all of the transaction validation on
blocks if they have been confirmed to be in the block chain leading
up to the final checkpoint in a given blockschain.
This algorithm fetches block headers from the peer, then once it has
established the full blockchain connection, it requests blocks.
Any blocks before the final checkpoint pass true for fastAdd on
btcchain operation, which causes it to do less valiation on the block.
This commit implements pruning for the block nodes that form the memory
chain which are no longer needed. The validation of a block only requires
information from a set number of previous nodes. The initial code did not
prune old nodes as they were no longer needed, but the vast majority of
the infrastructure to support it was already in place as that was always
the goal. This commit finishes implementing the missing bits to make it
a reality.
Even though the code currently makes heavy use of nodes which will have
the appropriate height, blocks which did not come from the database will
not have a height set. As a result, this could possibly result in a
height of 0 being used when unintended. By setting the block height in
the block as soon as its known (regardless of the source), we can future
proof against bugs that would likely be hard to track down.
In addition to returning errors to the caller, log the error with a prefix
in a few key places that helps identify the origin for errors. In some
cases, the underlying error comes from a different subsystem such as the
SQL database driver and the error messages can be fairly generic.
This commit modifies the code to use params based on the active network
which paves the way for supporting the special rules and different genesis
blocks used by the test networks.