The block index now tracks the set of dirty block nodes with status
changes that haven't been persisted and flushes the changes to the DB
at the appropriate times.
Now that all headers are stored in the in-memory index, the database
bucket managed by blockchain, and in the flat files, it makes sense to
drop the redundant data from the block index bucket in ffldb.
To avoid modifying the database interface, this reimplements
FetchBlockHeader(s) to use header data stored in flat files. This can
be trivially implemented by delegating to FetchBlockRegion.
Currently only the blocks in the active chain are loaded into the
block index on initialization. This instead iterates over the entire
block index bucket in LevelDB and loads all nodes.
The bucket contains block headers keyed by the block height encoded as
big-endian concatenated with the block hash. This allows block headers
to be fetched from the DB in height order with a cursor.
These method allows safe concurrent access to reading and modifying
block node statuses. When block statuses get persisted in a later
change, the setter methods can be used to mark block nodes as dirty.
Each node in the block index records some flags about its validation
state. This is just stored in memory for now, but can save effort if
attempting to reconnect a block that failed validation or was
disconnected.
Previously, rpctest would start a btcd node using the btcd executable
in the environment PATH. This caused difficult-to-find issues where
the code would be tested against an older version of btcd, or another
fork entirely. Now it compiles btcd the first time it is needed and
uses that fresh version when launching nodes.
This was only used to test block proposals, which has been changed to
instead use CheckConnectBlockTemplate. The flag complicated the
implementation of some chain processing routines and would be
difficult to implement with headers-first syncing.
This renames CheckConnectBlock to CheckConnectBlockTemplate and
modifies it to be easily consumable by the getblocktemplate RPC
handler. Performs full block validation now instead of partial
validation.
This propagates the interrupt channel through to blockchain and the
indexers so that it is possible to interrupt long-running operations
such as catching up indexes.
The helper function parseListeners has been changed to return a
slice of net.Addrs with Network() returning tcp4 or tcp6 instead of
returning two slices of IPv4 and IPv6 addresses to simplify calling
code. Also improves how local addresses are added to the address
manager when listening on wildcard addresses.
Also splits some newServer logic into new method initListeners.
The modifies the encoding of witness stacks in JSON responses to use a
slice of strings instead of a single space-separated string for
compatibility with Core.
This updates the README to remove the mailing lists which no longer
exist and to link the GPG public key used to sign the release tags
directly from the repository instead of the no longer active website.
This update adds additional callsite logging options via btclog and
fixes an error with the rotator package that caused it to stop running
when creating any log messages larger than 4096 bytes.
While here, switch to the new Write method of the Rotator object as
this is more efficient than using the Reader interface with a pipe.
Changes from @jrick.
This refactors the code that locates blocks (inventory discovery) out of
server and into blockchain where it can make use of the new much more
efficient chain view and more easily be tested. As an aside, it really
belongs in blockchain anyways since it's purely dealing with the block
index and best chain.
Since the majority of the network has moved to header-based semantics,
this also provides an additional optimization to allow headers to be
located directly versus needing to first discover the hashes and then
fetch the headers.
The new functions are named LocateBlocks and LocateHeaders. The former
returns a slice of located hashes and the latter returns a slice of
located headers.
Finally, it also updates the RPC server getheaders call and related
plumbing to use the new LocateHeaders function.
A comprehensive suite of tests is provided to ensure both functions
behave correctly for both correct and incorrect block locators.
This modifies calcSignatureHash to use a shallow copy of the transaction
versus a deep copy since the actual scripts themselves are not modified
and therefore don't need to be copied.
This is being done because profiling the most overall allocated space
shows that the deep copy performed in calcSignatureHash accounts for
nearly 20% of all allocations on a synced running instance. Also,
copying all of the additional data makes it more time consuming as well.
With this change, that figure drops from ~20% to ~5% of all allocations.
The following benchmark shows the relative speedups and allocation
reduction as a result of the optimization on my system. In particular,
the changes result in approximately a 15% speedup and a whopping 99.89%
reduction in allocations when using a large transaction with thousands
of inputs which was the worst case scenario.
benchmark old allocs new allocs delta
--------------------------------------------------------------------
BenchmarkCalcSignatureHash 11151 12 -99.89%
benchmark old ns/op new ns/op delta
--------------------------------------------------------------------
BenchmarkCalcSignatureHash 3599845 3056359 -15.10%
- Remove inMainChain from block nodes since that can now be efficiently
determined by using the chain view
- Track the best chain via a chain view instead of a single block node
- Use the tip of the best chain view everywhere bestNode was used
- Update chain view tip instead of updating best node
- Change reorg logic to use more efficient chain view fork finding logic
- Change block locator code over to use more efficient chain view logic
- Remove now unused block-index-based block locator code
- Move BlockLocator definition to chain.go
- Move BlockLocatorFromHash and LatestBlockLocator to chain.go
- Update both to use more efficient chain view logic
- Rework IsCheckpointCandidate to use block index and chain view
- Optimize MainChainHasBlock to use chain view instead of hitting db
- Move to chain.go since it no longer involves database I/O
- Removed error return since it can no longer fail
- Optimize BlockHeightByHash to use chain view instead of hitting db
- Move to chain.go since it no longer involves database I/O
- Removed error return since it can no longer fail
- Optimize BlockHashByHeight to use chain view instead of hitting db
- Move to chain.go since it no longer involves database I/O
- Removed error return since it can no longer fail
- Optimize HeightRange to use chain view instead of hitting db
- Move to chain.go since it no longer involves database I/O
- Optimize BlockByHeight to use chain view for main chain check
- Optimize BlockByHash to use chain view for main chain check
