This commit adds a quick check to the transaction store fetch code which
simply returns an empty store if no hashes were requested rather than
bothering the db with an empty list.
This commit modifies the transaction lookup code to use a set instead of a
slice (list). This allows the lookup to automatically prevent duplicate
requests to the database.
Previously, the code simply added every referenced transaction to a list
without checking for duplicates, which led to multiple requests against
the database for the same transaction. It also meant the request list
could grow quite large with all of the duplicates using far more memory
than required.
While the end result was accurate, operating that way is not as efficient
as only requesting unique transactions.
This commit implents a basic infrastructure to be used throughout the
tests for creating a new chain instance that is ready to have tests run
against it. It also returns a teardown function the caller can use to
clean up after it is done testing. This paves the way for adding more
tests.
The original thought was that chain would also house the transaction
memory pool, but that ultimately was decided against. As a result,
it only makes sense to query chain for blocks rather than generic
inventory.
This commit corrects the reading of the serialized height in coinbase
transactions for block height of version 2 or greater. On mainnet, the
serialized height is always 3 bytes and will continue to be so for
something like another ~159 years, so there was no issue with mainnet.
However on testnet, there are some version 2 blocks which are low enough
in the chain to only take 2 bytes to serialize.
In addition, this commit adds a full tests for the relavant function
including negative tests and variable length serialized lengths for block
heights.
Closes#1.
btcdb was changed a while back to not insert the genesis block by default.
This commit modifies the reorg test to insert it as required so not all
blocks are orphans.
Rather than using a channel for notifictions, use a callback instead.
There are several issues that arise with async notifications via a
channel, so use a callback instead. The caller can always make the
notifications async by using channels in the provided callback if
needed.
This commit modifies CheckInputTransactions to ensure that not only must a
transaction exist in the transaction store, it must also not have any
errors associated with it or be nil.
Several of the functions require a map of contextual transaction data to
use as a source for referenced transactions. This commit exports the
underlying TxData type and creates a new type TxStore, which is a map of
points to the under TxData. In addition, this commit exposes a new
function, FetchTransactionStore, which returns a transaction store
(TxStore) containing all of the transactions referenced by the passed
transaction, as well as the existing transaction if it already exists.
This paves the way for subsequent commits which will expose some of the
functions which depend on this transaction store.
This commit modifies the code to choose sane defaults for the backing
arrays for slices that involve a lot of appends such block locators, hash
processing, and needed transactions. This is an optimization to avoid
the overhead of growing the backing arrays and copying the data multiple
times in the most common case. This also prevents a leak in Go GC which
will likely ultimatley be fixed, but the efficiecy gains alone are worth
the change.
After discussion with others and thinking about the notification channel
some more, we've decided to leave it up to the caller to quickly handle
notifications. While it is true that notification should be handled
quickly to not block the chain processing code unnecessarily, launching a
goroutine in chain means the notifications are no longer necessarily in
order. Also, if the caller is not properly handling the notifications,
the goroutines end up sicking around forever. By leaving it up to the
caller to quickly handle the notification or launch a goroutine as
necessary for the caller, it provides the flexibility to ensure proper
notification ordering as well as control over other things such as
how to handle backpressure.
Rather than sending the root of the an orphan chain when sending an orphan
notification, send the hash of the orphan block itself. The caller can
then call the GetOrphanRoot function with the hash to get the root of the
orphan chain as needed. This is being changed since it's cleaner and more
detministic sending the hash for the orphan block that was just processed
rather than sending a possibly different hash depending on whether there
is an orphan chain or not.
Since the notification channel is provided by the caller and it may or may
not be buffered, send notifications in their own goroutine so the chain
processing code does not have to wait around on the caller to process the
notification before continuing.
Rather than removing disconnected transactions from the node viewpoint
transaction store, clear the entry instead. This is needed for the
connect code to update transactions that were removed on the other side of
a chain fork.
This commit adds a new type, BlockLocator, and supporting infrastructure
to support generating block locators from a provided hash and getting
a full block locator for the latest known block. This will likely be
expanded in the future to provide the opposite functionality as well.
That is to say the ability to lookup a block using a block locator.
This commit modifies the GenerateInitialIndex function to update the best
chain with each node as it is loaded. This allows the best chain to be
set immediately upon generating the initial index which is a slight
performance optimization.
This commit provides a new exported function, HaveInventory, which can
be used to determine if the already has the item referenced by passed
inventory vector. Currently it only provides support for blocks and
cursory support for transactions in the main chain. Types that are
unrecognized are returned as if they unknown as expected.
This commit provides a new exported function, IsKnownOrphan, which can
be used to determine if the passed hash is currently already known to the
chain as an orphan.
This commit exposes GetOrphanRoot to callers. It also modifies the code
to allow the function to be called in a concurrent safe manner. This
allows the information to be safely obtained from multiple goroutines.
It is not safe to remove elements from a slice while iterating them with
the range statement since it does not reevaluate the slice on each
iteration nor does it adjust the index for the modified slice. This
commit modifies the code to use a for loop with an index (which does
reevaluate on every iteration) and manually adjusts the index when
elements are removed from the slice.
This commit updates the calls into btcutil, btcscript, and btcwire for the
latest API changes which remove the need for the protocol version for
serialization and deserialization of blocks and transactions.
This commit adds a new optional function named GenerateInitialIndex. It
generates the required number of initial block nodes in an optimized
fashion. Since the memory block index is sparse and previous nodes are
dynamically loaded as needed, this function is not stricty needed.
However, during initial startup (when there are no nodes in memory yet),
dynamically loading all of the required nodes on the fly in the usual way
is much slower than preloading them.
The recent pruning code made the parent hash for a node available directly
as a field in the node. Make use of this in the getPrevNodeFromNode
function to avoid having to load the full block data associated with the
node to get the parent hash.
This commit implements pruning for the block nodes that form the memory
chain which are no longer needed. The validation of a block only requires
information from a set number of previous nodes. The initial code did not
prune old nodes as they were no longer needed, but the vast majority of
the infrastructure to support it was already in place as that was always
the goal. This commit finishes implementing the missing bits to make it
a reality.
Previously, the code was using big rational numbers for work values which
resulted in carrying way too much precision around (and ultimately a lot
of extra memory and computation to carry that precision). This commit
converts the work values to big integers and calculates them with integer
division. This is acceptable because the numerator is multiplied by 2^256
which is higher than the maximum possible proof of work. Therefore
anything after the decimal is superfluous precision for the purposes of
chain selection.
Also, add a check for negative difficulty values when calculating the work
value. Negative values won't occur in practice with valid blocks, but
it's possible an invalid block could trigger the code path, so be safe and
check for it.
