Commit graph

216 commits

Author SHA1 Message Date
Jim Posen
04444c1d0e blockchain: Use CheckConnectBlockTemplate for RPC block proposals.
This renames CheckConnectBlock to CheckConnectBlockTemplate and
modifies it to be easily consumable by the getblocktemplate RPC
handler. Performs full block validation now instead of partial
validation.
2017-10-12 06:07:46 -05:00
Dave Collins
8c883d1fca
blockchain/indexers: Allow interrupts.
This propagates the interrupt channel through to blockchain and the
indexers so that it is possible to interrupt long-running operations
such as catching up indexes.
2017-09-05 11:02:46 -05:00
Dave Collins
42b969a827
blockchain: Remove unused min memory nodes field.
This removes the minMemoryNodes field from the BlockChain struct since
it is no longer used anywhere.
2017-08-30 17:08:34 -05:00
Dave Collins
02a06a2cd8
blockchain: Update some comments to match reality. 2017-08-30 17:07:46 -05:00
Jim Posen
0a9fb53548 blockchain: Add node to block index in maybeAcceptBlock.
This has the same effect but makes it clearer that all blocks written to
the database end up in the block index.
2017-08-30 11:29:52 -07:00
Dave Collins
34b1373a68
blockchain: Refactor inv discovery for chain view.
This refactors the code that locates blocks (inventory discovery) out of
server and into blockchain where it can make use of the new much more
efficient chain view and more easily be tested.  As an aside, it really
belongs in blockchain anyways since it's purely dealing with the block
index and best chain.

Since the majority of the network has moved to header-based semantics,
this also provides an additional optimization to allow headers to be
located directly versus needing to first discover the hashes and then
fetch the headers.

The new functions are named LocateBlocks and LocateHeaders.  The former
returns a slice of located hashes and the latter returns a slice of
located headers.

Finally, it also updates the RPC server getheaders call and related
plumbing to use the new LocateHeaders function.

A comprehensive suite of tests is provided to ensure both functions
behave correctly for both correct and incorrect block locators.
2017-08-24 14:15:34 -05:00
Dave Collins
20910511e9
blockchain: Refactor to use new chain view.
- Remove inMainChain from block nodes since that can now be efficiently
  determined by using the chain view
- Track the best chain via a chain view instead of a single block node
  - Use the tip of the best chain view everywhere bestNode was used
  - Update chain view tip instead of updating best node
- Change reorg logic to use more efficient chain view fork finding logic
- Change block locator code over to use more efficient chain view logic
  - Remove now unused block-index-based block locator code
  - Move BlockLocator definition to chain.go
  - Move BlockLocatorFromHash and LatestBlockLocator to chain.go
    - Update both to use more efficient chain view logic
- Rework IsCheckpointCandidate to use block index and chain view
- Optimize MainChainHasBlock to use chain view instead of hitting db
  - Move to chain.go since it no longer involves database I/O
  - Removed error return since it can no longer fail
- Optimize BlockHeightByHash to use chain view instead of hitting db
  - Move to chain.go since it no longer involves database I/O
  - Removed error return since it can no longer fail
- Optimize BlockHashByHeight to use chain view instead of hitting db
  - Move to chain.go since it no longer involves database I/O
  - Removed error return since it can no longer fail
- Optimize HeightRange to use chain view instead of hitting db
  - Move to chain.go since it no longer involves database I/O
- Optimize BlockByHeight to use chain view for main chain check
- Optimize BlockByHash to use chain view for main chain check
2017-08-23 23:43:37 -05:00
Dave Collins
2d84a74d28
blockchain: Correct chainview locator comment. 2017-08-21 14:16:31 -05:00
Dave Collins
991a72e34e
blockchain: Remove unused verify disable code.
This removes the DisableVerify function and related state since nothing
uses it anymore since the command line option was removed.  It is a
remnant of initial development.
2017-08-21 04:10:23 -05:00
Dave Collins
1a947c46b2
blockchain: Faster chain view block locator.
This exposes the ability to more efficiently create a block locator from
a chain view for a given block node by using their ability to do O(1)
lookups.

It also adds tests to ensure the behavior is correct.
2017-08-19 23:58:04 -05:00
Dave Collins
93d8dfc760
blockchain: Optimize block locator generation.
This significantly optimizes and simplifies the generation of block
locators by making use of the fact that all block nodes are now in
memory and therefore it is no longer necessary to consult the database
for the hashes or worry about issues related to dynamic loading of nodes.

Also, it slightly modifies the algorithm so that the doubling doesn't
start for one additional iteration in order to mirror other prominent
clients on the network.  Due to the way block locators are used, this
does not change any semantics in terms of requesting and locating
blocks.

Finally, the semantics of BlockLocatorFromHash have been changed to
return a locator for the current tip in the case the hash is unknown.
This is far preferable since only including the passed block hash, when
it isn't known, could end up leading to causing a redownload of the
entire chain under certain circumstances.
2017-08-19 23:19:58 -05:00
Dave Collins
e02fbcf5a1
blockchain: Consolidate tests into package.
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.

While here, remove the reorganization test since it is much better
handled by the full block tests and is no longer needed and do some
light cleanup on a few other tests.

The full block tests had to remain in the separate test package since it
is a circular dependency otherwise.  This did require duplicating some
of the chain setup code, but given the other benefits this is
acceptable.
2017-08-19 23:05:17 -05:00
Dave Collins
f4fe6c373e
blockchain: Convert seq lock tests to synthetic.
This introduces the concept of a synthetic block chain that can be used
in the tests to avoid needing setup a full blown chain instance with a
database and generate valid blocks and converts the sequence lock tests
in TestCalcSequenceLock to use it.

Not only does this speed up the test execution time, but it allows the
dependency on rpctest to be removed which will allow the sequence locks
tests to be consolidated into the main package without creating a
circular dependency.
2017-08-19 22:29:22 -05:00
Dave Collins
2a4be16b6c
blockchain: Improve and correct chainview set tip.
This modifies the function to set the tip in the new chainview code to
bulk copy existing nodes when it needs to expand the cap rather than
simply creating a new empty slice and allowing the walk code below it to
repopulate it.  This is a nice optimization since, in practice, most of
the time expanding the cap is only required when the active chain is
being extended after having run for a while which means the end result
is that it will be able to bulk copy all the nodes and just add the most
recent one versus having to walk them all and add them back.

Also, while here expand the tests for setting the tip to ensure the
nodes contained in the resulting view are correct after forcing the
resizes and correct a bug they exposed where changing between a
longer-shorter-longer chain where the longer chain is the same chain
could result in not populating the view correctly.

Finally, update the fake nodes generated by the tests to use a
nonce generated by a deterministic prng in order to ensure the hashes of
all fake nodes are unique, but reproducible.
2017-08-19 22:17:30 -05:00
Dave Collins
1d77a611e9
blockchain: Implement new chain view.
This implements a new type in the blockchain package that takes
advantage of the fact that all block nodes are now in memory to provide
a flat view of a specific chain of blocks (a specific branch of the
overall block tree) from a given tip all the way back to the genesis
block along with several convenience functions such as efficiently
comparing two views, quickly finding the fork point (if any) between two
views, and O(1) lookup of the node at a specific height.

The view is not currently used, but the intent is that the code will be
refactored to make use of these views to simplify and optimize several
areas such as best chain selection and reorg logic and finding successor
nodes.  They will also greatly simplify the process of disconnecting the
download logic from the connection logic.

A comprehensive suite of tests is provided to ensure the chain views
behave correctly.
2017-08-17 17:04:39 -05:00
Dave Collins
7e5e6b4610
blockchain: Cleanup a few sequence lock tests.
This cleans up the test in TestCalcSequenceLock in the following ways:
- Use calculated values instead of magic value so it is easier to update
  the tests as needed
- Make tests match the comments
- Change comments to be more consistent and fix some grammar errors
- Set mempool flag for unconfirmed tx tests since they are intended to
  mimic transactions in the mempool
2017-08-16 15:54:03 -05:00
Dave Collins
72f2a3fe49
blockchain: Optimize checkpoint handling.
This modifies the code that determines the most recently known
checkpoint to take advantage of recent changes which make the entire
block index available in memory by only storing a reference to the
specific node in the index that represents the latest known checkpoint.

Previously, the entire block was stored and new checkpoints required
loading it from the database.
2017-08-15 17:06:16 -05:00
Dave Collins
349450379f
blockchain: Remove threshold state db cache.
This completely removes the threshold state database caching code since
it can very quickly be calculated at startup now that the entire block
index is loaded first.
2017-08-15 16:55:40 -05:00
Dave Collins
296fa0a5a0
blockchain: Convert to full block index in mem.
This reworks the block index code such that it loads all of the headers
in the main chain at startup and constructs the full block index
accordingly.

Since the full index from the current best tip all the way back to the
genesis block is now guaranteed to be in memory, this also removes all
code related to dynamically loading the nodes and updates some of the
logic to take advantage of the fact traversing the block index can
longer potentially fail.  There are also more optimizations and
simplifications that can be made in the future as a result of this.

Due to removing all of the extra overhead of tracking the dynamic state,
and ensuring the block node structs are aligned to eliminate extra
padding, the end result of a fully populated block index now takes quite
a bit less memory than the previous dynamically loaded version.

The main downside is that it now takes a while to start whereas it was
nearly instant before, however, it is much better to provide more
efficient runtime operation since that is its ultimate purpose and the
benefits far outweigh this downside.

Some benefits are:

- Since every block node is in memory, the recent code which
  reconstructs headers from block nodes means that all headers can
  always be served from memory which is important since the majority of
  the network has moved to header-based semantics
- Several of the error paths can be removed since they are no longer
  necessary
- It is no longer expensive to calculate CSV sequence locks or median
  times of blocks way in the past
- It will be possible to create much more efficient iteration and
  simplified views of the overall index
- The entire threshold state database cache can be removed since it is
  cheap to construct it from the full block index as needed

An overview of the logic changes are as follows:

- Move AncestorNode from blockIndex to blockNode and greatly simplify
  since it no longer has to deal with the possibility of dynamically
  loading nodes and related failures
- Rename RelativeNode to RelativeAncestor, move to blockNode, and
  redefine in terms of AncestorNode
- Move CalcPastMedianTime from blockIndex to blockNode and remove no
  longer necessary test for nil
- Change calcSequenceLock to use Ancestor instead of RelativeAncestor
  since it reads more clearly
2017-08-15 15:42:34 -05:00
Dave Collins
3b5c2baa2e
blockchain: Consistency pass on subscribe code.
This takes care of a few minor nits on the recently merged subscribe
code.  In particular:

- Avoid extra unnecessary allocation on notifications slice
- Avoid defer overhead on notification mutex in sendNotifications
- Make test function comment start with the name of the function per
  Effective Go guidelines
- Use constant for number of subscribers in test
- Don't exceed column 80 in test print
2017-08-15 11:12:56 -05:00
Jim Posen
ca9baf77ec blockchain: Fix race condition in notifications_test 2017-08-14 22:48:32 -05:00
Jim Posen
19b42c3c9b Allow multiple subscribers to blockchain notifications.
The BlockChain struct emits notifications for various events, but
it is only possible to register one listener. This changes the
interface and implementations to allow multiple listeners.
2017-08-14 22:03:24 -05:00
Dave Collins
19eada0b4b
blockchain: Combine ErrDoubleSpend & ErrMissingTx.
This replaces the ErrDoubleSpend and ErrMissingTx error codes with a
single error code named ErrMissingTxOut and updates the relevant errors
and expected test results accordingly.

Once upon a time, the code relied on a transaction index, so it was able
to definitively differentiate between a transaction output that
legitimately did not exist and one that had already been spent.

However, since the code now uses a pruned utxoset, it is no longer
possible to reliably differentiate since once all outputs of a
transaction are spent, it is removed from the utxoset completely.
Consequently, a missing transaction could be either because the
transaction never existed or because it is fully spent.
2017-08-14 11:40:39 -05:00
Olaoluwa Osuntokun
fad0d1d56c blockchain: only populate hashcache if tx is segwitty 2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
728c0a4398 blockchain: guard enforcement of segwit rules by a BIP0009 state check
This commit updates the new segwit validation logic within block
validation to be guarded by an initial check to the version bits state
before conditionally enforcing the logic based off of the state.
2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
137aabd631 blockchain/indexers: add addrindex support for p2wkh and p2wsh 2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
d0768abcc4 BIP0141+blockchain: implement segwit block validation rules
This commit implements the new block validation rules as defined by
BIP0141. The new rules include the constraints that if a block has
transactions with witness data, then there MUST be a commitment within
the conies transaction to the root of a new merkle tree which commits
to the wtxid of all transactions. Additionally, rather than limiting
the size of a block by size in bytes, blocks are now limited by their
total weight unit. Similarly, a newly define “sig op cost” is now used
to limit the signature validation cost of transactions found within
blocks.
2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
1b80b334bc BIP0141+blockchain: add functions for extracting and validating witness commitment 2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
d38ae9ca0b BIP0141+blockchain: implement tx/block weight calculation funcitons
This commit implements the new “weight” metric introduced as part of
the segwit soft-fork. Post-fork activation, rather than limiting the
size of blocks and transactions based purely on serialized size, a new
metric “weight” will instead be used as a way to more accurately
reflect the costs of a tx/block on the system. With blocks constrained
by weight, the maximum block-size increases to ~4MB.
2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
9054ef8354 BIP 147: enforce NULLDUMMY w/ segwit verification
This commit implements the flag activation portion of BIP 0147. The
verification behavior triggered by the NULLDUMMY script verification
flag has been present within btcd for some time, however it wasn’t
activated by default.

With this commit, once segwit has activated, the ScriptStrictMultiSig
will also be activated within the Script VM. Additionally, the
ScriptStrictMultiSig is now a standard script verification flag which
is used unconditionally within the mempool.
2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
0db14c740b BIP0144: properly fetch witness data from witness-enabled peers
This commit modifies the logic within the block manager and service to
preferentially fetch transactions and blocks which include witness data
from fully upgraded peers.

Once the initial version handshake has completed, the server now tracks
which of the connected peers are witness enabled (they advertise
SFNodeWitness). From then on, if a peer is witness enabled, then btcd
will always request full witness data when fetching
transactions/blocks.
2017-08-13 23:17:40 -05:00
Gregory Trubetskoy
1e331153b4 Fix the unknown new rules activation warning 2017-08-09 19:10:28 -04:00
Dave Collins
9f0e66ee3f
blockchain: Correct invalid assertion.
This corrects the assertion in the decodeSpentTxOut function so it does
not improperly cause a panic when unwinding transactions during a reorg
under certain circumstances.  In particular, the provided transaction
version that is passed when a stxo entry does not exist is now -1 in
order to properly distinguish it from the zero value.

It also updates the tests accordingly.

This was discovered by the reorg on testnet from block
00000000000018c58c2d2816f03dac327d975a18af6edf1a369df67ecddaf816 to
0000000000001c1161a367156465cc6226e9f862d9c585f94db5779fdf5455ff.
2017-07-01 02:25:41 -05:00
Dave Collins
01cb59c67d
multi: Simplify code per gosimple linter.
This simplifies the code based on the recommendations of the gosimple
lint tool.
2017-07-01 02:25:10 -05:00
Josh Rickmar
a6965d493f all: Remove seelog logger.
The btclog package has been changed to defining its own logging
interface (rather than seelog's) and provides a default implementation
for callers to use.

There are two primary advantages to the new logger implementation.

First, all log messages are created before the call returns.  Compared
to seelog, this prevents data races when mutable variables are logged.

Second, the new logger does not implement any kind of artifical rate
limiting (what seelog refers to as "adaptive logging").  Log messages
are outputted as soon as possible and the application will appear to
perform much better when watching standard output.

Because log rotation is not a feature of the btclog logging
implementation, it is handled by the main package by importing a file
rotation package that provides an io.Reader interface for creating
output to a rotating file output.  The rotator has been configured
with the same defaults that btcd previously used in the seelog config
(10MB file limits with maximum of 3 rolls) but now compresses newly
created roll files.  Due to the high compressibility of log text, the
compressed files typically reduce to around 15-30% of the original
10MB file.
2017-06-19 16:46:50 -04:00
Steven Roose
3d0dfed40b Fix a ton of typos accumulated over time 2017-05-30 16:59:51 +02:00
Dave Collins
9918e2a561
multi: Update markdown files for GFM changes.
The github markdown interpreter has been changed such that it no longer
allows spaces in between the brackets and parenthesis of links and now
requires a newline in between anchors and other formatting.  This
updates all of the markdown files accordingly.

While here, it also corrects a couple of inconsistencies in some of the
README.md files.
2017-05-25 12:06:16 -05:00
Dave Collins
1ae306021e
rpctest: Make subpackage of integration.
This makes the rpctest package a subpackage of the integration package
since its primary purpose is for integration testing.
2017-05-11 15:17:29 -05:00
Olaoluwa Osuntokun
75569f599f
blockchain: enforce CSV soft-fork validation based on versionbits state
This commit modifies the existing block validation logic to examine the
current version bits state of the CSV soft-fork, enforcing the new
validation rules (BIPs 68, 112, and 113) accordingly based on the
current `ThesholdState`.
2017-05-10 15:37:11 -07:00
Olaoluwa Osuntokun
d767af1509
blockchain: update calcSequenceLocks test to activate CSV first 2017-05-10 15:37:04 -07:00
Dave Collins
efa50e6abc
multi: Simplify code per gosimple linter.
This simplifies the code based on the recommendations of the gosimple
lint tool.

Also, it increases the deadline for the linters to run to 10 minutes and
reduces the number of threads that is uses. This is being done because
the Travis environment has become increasingly slower and it also seems
to be hampered by too many threads running concurrently.
2017-03-22 15:34:13 -05:00
Dave Collins
d06c0bb181
blockchain: Use hash values in structs.
This modifies the blockNode and BestState structs in the blockchain
package to store hashes directly instead of pointers to them and updates
callers to deal with the API change in the exported BestState struct.

In general, the preferred approach for hashes moving forward is to store
hash values in complex data structures, particularly those that will be
used for cache entries, and accept pointers to hashes in arguments to
functions.

Some of the reasoning behind making this change is:

- It is generally preferred to avoid storing pointers to data in cache
  objects since doing so can easily lead to storing interior pointers
  into other structs that then can't be GC'd
- Keeping the hash values directly in the block node provides better
  cache locality
2017-02-03 11:36:33 -06:00
Dave Collins
05c7c14023
blockchain: Ignore side chains in exists check.
Since the code base is currently in the process of changing over to
decouple download and connection logic, but not all of the necessary
parts are updated yet, ensure blocks that are in the database, but do
not have an associated main chain block index entry, are treated as if
they do not exist for the purposes of chain connection and selection
logic.
2017-02-03 11:02:35 -06:00
Dave Collins
1ecfea4928
blockchain: Refactor main block index logic.
This refactors the block index logic into a separate struct and
introduces an individual lock for it so it can be queried independent of
the chain lock.
2017-02-01 13:14:41 -06:00
Dave Collins
59169540c3
blockchain: Reconstruct headers from block nodes.
This modifies the block node structure to include a couple of extra
fields needed to be able to reconstruct the block header from a node,
and exposes a new function from chain to fetch the block headers which
takes advantage of the new functionality to reconstruct the headers from
memory when possible.  Finally, it updates both the p2p and RPC servers
to make use of the new function.

This is useful since many of the block header fields need to be kept in
order to form the block index anyways and storing the extra fields means
the database does not have to be consulted when headers are requested if
the associated node is still in memory.

The following timings show representative performance gains as measured
from one system:

new: Time to fetch 100000 headers:   59ms
old: Time to fetch 100000 headers: 4783ms
2017-02-01 12:27:56 -06:00
Dave Collins
d552176e1d
blockchain: Remove unused block node removal func.
This removes the removeBlockNode function since it is no longer used
after the addition of CLTV removed block node pruning.
2017-02-01 11:25:29 -06:00
Dave Collins
0d0f50c464
blockchain: Remove unused median time calc func.
This removes the CalcPastMedianTime since it is now exposed much more
efficiently via the MedianTime field of the BestState snapshot returned
from the BestSnapshot function.
2017-02-01 10:50:27 -06:00
Dave Collins
5ffd552214
blockchain: Use int64 timestamps in block nodes.
This modifies the block nodes used in the blockchain package for keeping
track of the block index to use int64 for the timestamps instead of
time.Time.

This is being done because a time.Time takes 24 bytes while an int64
only takes 8 and the plan is to eventually move the entire block index
into memory instead of the current dynamically-loaded version, so
cutting the number of bytes used for the timestamp by a third is highly
desirable.

Also, the consensus code requires working with unix-style timestamps
anyways, so switching over to them in the block node does not seem
unreasonable.

Finally, this does not go so far as to change all of the time.Time
references, particularly those that are in the public API, so it is
purely an internal change.
2017-01-31 17:51:29 -06:00
Dave Collins
6a54323258
blockchain: Store side chain blocks in database.
This modifies the blockchain code to store all blocks that have passed
proof-of-work and contextual validity tests in the database even if they
may ultimately fail to connect.

This eliminates the need to store those blocks in memory, allows them to
be available as orphans later even if they were never part of the main
chain, and helps pave the way toward being able to separate the download
logic from the connection logic.
2017-01-31 10:25:43 -06:00
Dave Collins
c065733c31
multi: Improvements to configurable checkpoints.
This contains a bit of cleanup and additional logic to improve the
recently-added ability to specify additional checkpoints via the
--addcheckpoint option.

In particular:
- Improve error messages in the checkpoint parsing
- Correct the mergeCheckpoints function to weed out duplicate height
  checkpoints while using the most-recently provided one as described by
  its comment
- Add an assertion to blockchain.New that the provided checkpoints are
  sorted as required
- Keep comments to 80 columns and use two spaces after periods in them to
  be consistent with the rest of the code base
- Make the entry in doc.go match the actual btcd -h output
2017-01-23 12:07:54 -06:00
Steven Roose
c2af640c95 blockchain: Allow adding additional checkpoints
Introduces a `--checkpoint` flag that allows the user to specify
additional checkpoints or override the default ones provided in the
chain params.
2017-01-18 23:56:43 +01:00
David Hill
406ddd17d3 blockchain: TimeSource is required. 2017-01-12 11:19:51 -05:00
Alex Bosworth
2dfda48b94 Correct typo 2017-01-11 13:21:14 -08:00
Dave Collins
712399c0db
blockchain: Correct startup threshold state warns.
The thresholdState and deploymentState functions expect the block node
for the block prior to which the threshold state is calculated, however
the startup code which checked the threshold states was using the
current best node instead of its parent.

While here, also update the comments and rename a couple of variables to
help make this fact more clear.
2016-12-03 13:42:26 -06:00
Dave Collins
318c4760c0
blockchain: Only enforce BIP0030 prior to BIP0034.
This modifies the code to only enforce the fairly expensive BIP0030
(duplicate transcactions) checks when the chain has not yet reached the
BIP0034 activation height (and is not one of the 2 special historical
blocks that break the rule and prompted BIP0034 to being with) since
that BIP made it impossible to create duplicate coinbases and thus
removed the possibility of creating transactions that overwrite older
ones.

This is a rather large optimization because the check is expensive due
to involving a ton of cache misses in the utxoset.  For example, the
following are times it took to perform the BIP0030 check on blocks
425490 - 425502 and a system with a relatively old Hitachi spinner HDD:

block 425490: 674.5857ms
block 425491: 726.5923ms
block 425492: 827.6051ms
block 425493: 680.0863ms
block 425494: 722.0917ms
block 425495: 700.0889ms
block 425496: 647.5823ms
block 425497: 445.0565ms
block 425498: 602.5765ms
block 425499: 375.0476ms
block 425500: 771.0979ms
block 425501: 461.5586ms
block 425502: 603.0766ms

As can be seen from these numbers, this reduces the block validation
time by an average of just over half a second for the given
representative data set and hardware.

Signed-off-by: Dave Collins <davec@conformal.com>
2016-12-02 15:18:07 -06:00
Dave Collins
9cdc1b8afd
blockchain: Remove isMajorityVersion code.
Now that all softforking is done via BIP0009 versionbits, replace the
old isMajorityVersion deployment mechanism with hard coded historical
block heights at which they became active.

Since the activation heights vary per network, this adds new parameters
to the chaincfg.Params struct for them and sets the correct heights at
which each softfork became active on each chain.

It should be noted that this is a technically hard fork since the
behavior of alternate chain history is different with these hard-coded
activation heights as opposed to the old isMajorityVersion code.  In
particular, an alternate chain history could activate one of the soft
forks earlier than these hard-coded heights which means the old code
would reject blocks which violate the new soft fork rules whereas this
new code would not.

However, all of the soft forks this refers to were activated so far in
the chain history there is there is no way a reorg that long could
happen and checkpoints reject alternate chains before the most recent
checkpoint anyways.  Furthermore, the same change was made in Bitcoin
Core so this needs to be changed to be consistent anyways.
2016-11-30 14:26:13 -06:00
Dave Collins
c440584efc
Implement infrastructure for BIP0009.
This commit adds all of the infrastructure needed to support BIP0009
soft forks.

The following is an overview of the changes:

- Add new configuration options to the chaincfg package which allows the
  rule deployments to be defined per chain
- Implement code to calculate the threshold state as required by BIP0009
  - Use threshold state caches that are stored to the database in order
    to accelerate startup time
  - Remove caches that are invalid due to definition changes in the
    params including additions, deletions, and changes to existing
    entries
- Detect and warn when a new unknown rule is about to activate or has
  been activated in the block connection code
- Detect and warn when 50% of the last 100 blocks have unexpected
  versions.
- Remove the latest block version from wire since it no longer applies
- Add a version parameter to the wire.NewBlockHeader function since the
  default is no longer available
- Update the miner block template generation code to use the calculated
  block version based on the currently defined rule deployments and
  their threshold states as of the previous block
- Add tests for new error type
- Add tests for threshold state cache
2016-11-30 13:51:59 -06:00
Dave Collins
915fa6639b
multi: Simplify code per gosimple linter.
This simplifies the code based on the recommendations of the gosimple
lint tool.
2016-11-03 13:00:35 -05:00
Dave Collins
af524fb3e7
multi: Remove unnecessary convs found by unconvert.
This removes all unnecessary typecast conversions as found by the
unconvert linter.
2016-11-03 11:59:38 -05:00
Dave Collins
e320330d29
multi: Remove unused code found by deadcode. 2016-11-02 17:37:31 -05:00
Dave Collins
14b51fc5f8
multi: Correct misspellings detected by misspell. 2016-10-28 09:43:38 -05:00
Dave Collins
51a6309cdd
blockchain: Run gofmt -s.
Also compress some definitions a bit while simplifying.
2016-10-27 23:20:25 -05:00
Dave Collins
f6ad7eb2c9
wire: Make NewMsgTx accept the tx version.
This modifies the NewMsgTx function to accept the transaction version as
a parameter and updates all callers.

The reason for this change is so the transaction version can be bumped
in wire without breaking existing tests and to provide the caller with
the flexibility to create the specific transaction version they desire.
2016-10-27 14:09:29 -05:00
Olaoluwa Osuntokun
ecdaf9726c
blockchain: add basic tests for SequenceLockActive 2016-10-26 21:49:00 -07:00
Olaoluwa Osuntokun
e855c0dd82
blockchain: add table driven unit tests for CalcSequenceLock 2016-10-26 21:48:57 -07:00
Olaoluwa Osuntokun
de709f28f6
blockchain: add new LockTimeToSequence function
This commit adds a new function to the package: `LockTimeToSequence`.
The function is a simple utility function which aides the caller to
mapping a targeted time or block based relative lock-time to the
appropriate sequence number.
2016-10-26 21:48:54 -07:00
Olaoluwa Osuntokun
1914200080
blockchain: introduce SequenceLocks for relative lock-time calcs
This commit introduces the concept of “sequence locks” borrowed from
Bitcoin Core for converting an input’s relative time-locks to an
absolute value based on a particular block for input maturity
evaluation.

A sequence lock is computed as the most distant maturity height/time
amongst all the referenced outputs within a particular transaction.

A transaction with sequence locks activated within any of its inputs
can *only* be included within a block if from the point-of-view of that
block either the time-based or height-based maturity for all referenced
inputs has been met.

A transaction with sequence locks can only be accepted to the mempool
iff from the point-of-view of the *next* (yet to be found block) all
referenced inputs within the transaction are mature.
2016-10-26 21:48:44 -07:00
Dave Collins
9634a8cb0d
fullblocktests: Add missing doc.go file. 2016-10-23 13:31:26 -05:00
Dave Collins
49cbaf23dd
blockchain: Support small coinbase block size
This modifies the ExtractCoinbaseHeight function to recognize small
canonically serialized block heights in coinbase scripts of blocks
higher than version 2.

This allows regression test chains in which blocks encode the serialized
height in the coinbase starting from block 1.
2016-10-20 12:09:09 -05:00
David Hill
b1621332cc Optimize by removing defers
defer's are nice for readability but they do add overhead.  This
gets rid of defer's where it is just as easy as not to use one.
2016-10-18 17:56:51 -04:00
Dave Collins
f21410e47c
blockchain: Add block validation infrastructure.
This adds a full-blown testing infrastructure in order to test consensus
validation rules.  It is built around the idea of dynamically generating
full blocks that target specific rules linked together to form a block
chain.  In order to properly test the rules, each test instance starts
with a valid block that is then modified in the specific way needed to
test a specific rule.

Blocks which exercise following rules have been added for this initial
version.  These tests were largely ported from the original Java-based
'official' block acceptance tests as well as some additional tests
available in the Core python port.  It is expected that further tests
can be added over time as consensus rules change.

* Enough valid blocks to have a stable base of mature coinbases to spend
  for futher tests
* Basic forking and chain reorganization
* Double spends on forks
* Too much proof-of-work coinbase (extending main chain, in block that
  forces a reorg, and in a valid fork)
* Max and too many signature operations via various combinations of
  OP_CHECKSIG, OP_MULTISIG, OP_CHECKSIGVERIFY, and OP_MULTISIGVERIFY
* Too many and max signature operations with offending sigop after
  invalid data push
* Max and too many signature operations via pay-to-script-hash redeem
  scripts
* Attempt to spend tx created on a different fork
* Attempt to spend immature coinbase (on main chain and fork)
* Max size block and block that exceeds the max size
* Children of rejected blocks are either orphans or rejected
* Coinbase script too small and too large
* Max length coinbase script
* Attempt to spend tx in blocks that failed to connect
* Valid non-coinbase tx in place of coinbase
* Block with no transactions
* Invalid proof-of-work
* Block with a timestamp too far in the future
* Invalid merkle root
* Invalid proof-of-work limit (bits header field)
* Negative proof-of-work limit (bits header field)
* Two coinbase transactions
* Duplicate transactions
* Spend from transaction that does not exist
* Timestamp exactly at and one second after the median time
* Blocks with same hash via merkle root tricks
* Spend from transaction index that is out of range
* Transaction that spends more that its inputs provide
* Transaction with same hash as an existing tx that has not been
  fully spent (BIP0030)
* Non-final coinbase and non-coinbase txns
* Max size block with canonical encoding which exceeds max size with
  non-canonical encoding
* Spend from transaction earlier in same block
* Spend from transaction later in same block
* Double spend transaction from earlier in same block
* Coinbase that pays more than subsidy + fees
* Coinbase that includes subsidy + fees
* Invalid opcode in dead execution path
* Reorganization of txns with OP_RETURN outputs
* Spend of an OP_RETURN output
* Transaction with multiple OP_RETURN outputs
* Large max-sized block reorganization test (disabled by default since
  it takes a long time and a lot of memory to run)

Finally, the README.md files in the main and docs directories have been
updated to reflect the use of the new testing framework.
2016-10-17 12:16:53 -05:00
Dave Collins
1cba5c8fc0
blockchain: Remove exported CalcPastTimeMedian func.
This removes the exported CalcPastTimeMedian function from the
blockchain package as it is no longer needed since the information is
now available via the BestState snapshot.

Also, update the only known caller of this, which is the chain state in
block manager, to use the snapshot instead.  In reality, now that
everything the block manager chain state provides is available via the
blockchain BestState snapshot, the entire thing can be removed, however
that will be done in a separate to commit to keep the changes targeted.
2016-10-17 10:41:25 -05:00
Dave Collins
77913ad2e8
blockchain: Expose main chain flag on ProcessBlock.
This modifies the blockchain.ProcessBlock function to return an
additional boolean as the first parameter which indicates whether or not
the block ended up on the main chain.

This is primarily useful for upcoming test code that needs to be able to
tell the difference between a block accepted to a side chain and a block
that either extends the main chain or causes a reorganize that causes it
to become the main chain.  However, it is also useful for the addblock
utility since it allows a better error in the case a file with out of
order blocks is provided.
2016-10-13 16:47:50 -05:00
Chris Martin
c20db1cf14 blockchain: make scriptval_test fail more nicely
Return early when failing on len(blocks) > 1.

Check for len(blocks) == 0 so we can fail on that condition with an
error message instead of a panic.
2016-09-16 15:44:27 -04:00
Dave Collins
15bace88dc
mempool: Add basic test harness infrastructure.
This adds a basic test harness infrastructure for the mempool package
which aims to make writing tests for it much easier.

The harness provides functionality for creating and signing transactions
as well as a fake chain that provides utxos for use in generating valid
transactions and allows an arbitrary chain height to be set.  In order
to simplify transaction creation, a single signing key and payment
address is used throughout and a convenience type for spendable outputs
is provided.

The harness is initialized with a spendable coinbase output by default
and the fake chain height set to the maturity height needed to ensure
the provided output is in fact spendable as well as a policy that is
suitable for testing.

Since tests are in the same package and each harness provides a unique
pool and fake chain instance, the tests can safely reach into the pool
policy, or any other state, and change it for a given harness without
affecting the others.

In order to be able to make use of the existing blockchain.Viewpoint
type, a Clone method has been to the UtxoEntry type which allows the
fake chain instance to keep a single view with the actual available
unspent utxos while the mempool ends up fetching a subset of the view
with the specifically requested entries cloned.

To demo the harness, this also contains a couple of tests which make use
of it:

- TestSimpleOrphanChain -- Ensures an entire chain of orphans is
  properly accepted and connects up when the missing parent transaction
  is added
- TestOrphanRejects -- Ensure orphans are actually rejected when the
  flag on ProcessTransactions is set to reject them
2016-08-24 10:04:40 -05:00
Dave Collins
c57c18c8e2
blockchain: Add median time to state snapshot.
This adds a new field to the best chain state snapshot for the
calculated past median time as returned by the calcPastMedianTime
function.  This is useful since it provides fast access to it without
having to acquire the chain lock which is needed to recalculate it.

This will ultimately allow the associated exported function to be
removed since it only exists to be able to calculate this exact value,
however this commit only introduces the new field in order to keep the
changes minimal.
2016-08-23 01:29:11 -05:00
Dave Collins
a7b35d9f9e chaincfg/blockchain: Parameterize more chain consts. (#732)
This moves several of the chain constants to the Params struct in the
chaincfg package which is intended for that purpose.  This is mostly a
backport of the same modifications made in Decred along with a few
additional things cleaned up.

The following is an overview of the changes:

- Comment all fields in the Params struct definition
- Add locals to BlockChain instance for the calculated values based on
  the provided chain params
- Rename the following param fields:
  - SubsidyHalvingInterval -> SubsidyReductionInterval
  - ResetMinDifficulty -> ReduceMinDifficulty
- Add new Param fields:
  - CoinbaseMaturity
  - TargetTimePerBlock
  - TargetTimespan
  - BlocksPerRetarget
  - RetargetAdjustmentFactor
  - MinDiffReductionTime
2016-08-10 16:02:23 -05:00
Dave Collins
bd4e64d1d4 chainhash: Abstract hash logic to new package. (#729)
This is mostly a backport of some of the same modifications made in
Decred along with a few additional things cleaned up.  In particular,
this updates the code to make use of the new chainhash package.

Also, since this required API changes anyways and the hash algorithm is
no longer tied specifically to SHA, all other functions throughout the
code base which had "Sha" in their name have been changed to Hash so
they are not incorrectly implying the hash algorithm.

The following is an overview of the changes:

- Remove the wire.ShaHash type
- Update all references to wire.ShaHash to the new chainhash.Hash type
- Rename the following functions and update all references:
  - wire.BlockHeader.BlockSha -> BlockHash
  - wire.MsgBlock.BlockSha -> BlockHash
  - wire.MsgBlock.TxShas -> TxHashes
  - wire.MsgTx.TxSha -> TxHash
  - blockchain.ShaHashToBig -> HashToBig
  - peer.ShaFunc -> peer.HashFunc
- Rename all variables that included sha in their name to include hash
  instead
- Update for function name changes in other dependent packages such as
  btcutil
- Update copyright dates on all modified files
- Update glide.lock file to use the required version of btcutil
2016-08-08 14:04:33 -05:00
David Hill
61a15f6f1b blockchain: optimize HaveBlock (#720)
If a block is known to exist in the memory chain or database then
there is no need to check the orphan pool.
2016-07-25 10:16:57 -05:00
Dave Collins
00ebb9d14d blockchain: Associate time src with chain instance.
Rather than making the caller to pass in the median time source on
ProcessBlock and IsCurrent, modify the Config struct to include the
median time source and associate it with the chain instance when it is
created.

This is being done because both the ProcessBlock and IsCurrent functions
require access to the blockchain state already, it is a little bit safer
to ensure the time source matches the chain instance state, it
simplifies the caller logic, and it also allows its use within the logic
of the blockchain package itself which will be required by upcoming
rule change warning logic that is part of BIP9.
2016-07-14 13:10:47 -05:00
Dave Collins
de4fb24389 blockchain: Remove unused root field. (#680)
This removes the root field and all references to it from the BlockChain
since it is no longer required.

It was previously required because the chain state was not initialized
when the instance was created.  However, that is no longer the case, so
there is no reason to keep it around any longer.
2016-04-25 16:17:29 -05:00
Dave Collins
5a1e77bd2d blockchain: Remove unneeded unspentness size check. (#665)
The current code is needlessly checking the number of bytes needed to
serialize the unspentness bitmap in the utxo against a maximum value
that could never be returned because the function takes a uint32 output
index which is treated as a bit offset, and converts it bytes, which
will necessarily be less than a max uint32.

This check also causes a compile error on arm where native integers are
32 bits.

This simply removes the unneeded check.
2016-04-13 21:43:16 -05:00
Dave Collins
b580cdb7d3 database: Replace with new version.
This commit removes the old database package, moves the new package into
its place, and updates all imports accordingly.
2016-04-12 14:55:15 -05:00
Dave Collins
7c174620f7 indexers: Implement optional tx/address indexes.
This introduces a new indexing infrastructure for supporting optional
indexes using the new database and blockchain infrastructure along with
two concrete indexer implementations which provide both a
transaction-by-hash and a transaction-by-address index.

The new infrastructure is mostly separated into a package named indexers
which is housed under the blockchain package.  In order to support this,
a new interface named IndexManager has been introduced in the blockchain
package which provides methods to be notified when the chain has been
initialized and when blocks are connected and disconnected from the main
chain.  A concrete implementation of an index manager is provided by the
new indexers package.

The new indexers package also provides a new interface named Indexer
which allows the index manager to manage concrete index implementations
which conform to the interface.

The following is high level overview of the main index infrastructure
changes:

- Define a new IndexManager interface in the blockchain package and
  modify the package to make use of the interface when specified
- Create a new indexers package
  - Provides an Index interface which allows concrete indexes to plugin
    to an index manager
  - Provides a concrete IndexManager implementation
    - Handles the lifecycle of all indexes it manages
    - Tracks the index tips
    - Handles catching up disabled indexes that have been reenabled
    - Handles reorgs while the index was disabled
    - Invokes the appropriate methods for all managed indexes to allow
      them to index and deindex the blocks and transactions
  - Implement a transaction-by-hash index
    - Makes use of internal block IDs to save a significant amount of
      space and indexing costs over the old transaction index format
  - Implement a transaction-by-address index
    - Makes use of a leveling scheme in order to provide a good tradeoff
      between space required and indexing costs
- Supports enabling and disabling indexes at will
- Support the ability to drop indexes if they are no longer desired

The following is an overview of the btcd changes:

- Add a new index logging subsystem
- Add new options --txindex and --addrindex in order to enable the
  optional indexes
  - NOTE: The transaction index will automatically be enabled when the
    address index is enabled because it depends on it
- Add new options --droptxindex and --dropaddrindex to allow the indexes
  to be removed
  - NOTE: The address index will also be removed when the transaction
    index is dropped because it depends on it
- Update getrawtransactions RPC to make use of the transaction index
- Reimplement the searchrawtransaction RPC that makes use of the address
  index
- Update sample-btcd.conf to include sample usage for the new optional
  index flags
2016-04-11 17:16:42 -05:00
Dave Collins
491acd4ca6 blockchain: Rework to use new db interface.
This commit is the first stage of several that are planned to convert
the blockchain package into a concurrent safe package that will
ultimately allow support for multi-peer download and concurrent chain
processing.  The goal is to update btcd proper after each step so it can
take advantage of the enhancements as they are developed.

In addition to the aforementioned benefit, this staged approach has been
chosen since it is absolutely critical to maintain consensus.
Separating the changes into several stages makes it easier for reviewers
to logically follow what is happening and therefore helps prevent
consensus bugs.  Naturally there are significant automated tests to help
prevent consensus issues as well.

The main focus of this stage is to convert the blockchain package to use
the new database interface and implement the chain-related functionality
which it no longer handles.  It also aims to improve efficiency in
various areas by making use of the new database and chain capabilities.

The following is an overview of the chain changes:

- Update to use the new database interface
- Add chain-related functionality that the old database used to handle
  - Main chain structure and state
  - Transaction spend tracking
- Implement a new pruned unspent transaction output (utxo) set
  - Provides efficient direct access to the unspent transaction outputs
  - Uses a domain specific compression algorithm that understands the
    standard transaction scripts in order to significantly compress them
  - Removes reliance on the transaction index and paves the way toward
    eventually enabling block pruning
- Modify the New function to accept a Config struct instead of
  inidividual parameters
- Replace the old TxStore type with a new UtxoViewpoint type that makes
  use of the new pruned utxo set
- Convert code to treat the new UtxoViewpoint as a rolling view that is
  used between connects and disconnects to improve efficiency
- Make best chain state always set when the chain instance is created
  - Remove now unnecessary logic for dealing with unset best state
- Make all exported functions concurrent safe
  - Currently using a single chain state lock as it provides a straight
    forward and easy to review path forward however this can be improved
    with more fine grained locking
- Optimize various cases where full blocks were being loaded when only
  the header is needed to help reduce the I/O load
- Add the ability for callers to get a snapshot of the current best
  chain stats in a concurrent safe fashion
  - Does not block callers while new blocks are being processed
- Make error messages that reference transaction outputs consistently
  use <transaction hash>:<output index>
- Introduce a new AssertError type an convert internal consistency
  checks to use it
- Update tests and examples to reflect the changes
- Add a full suite of tests to ensure correct functionality of the new
  code

The following is an overview of the btcd changes:

- Update to use the new database and chain interfaces
- Temporarily remove all code related to the transaction index
- Temporarily remove all code related to the address index
- Convert all code that uses transaction stores to use the new utxo
  view
- Rework several calls that required the block manager for safe
  concurrency to use the chain package directly now that it is
  concurrent safe
- Change all calls to obtain the best hash to use the new best state
  snapshot capability from the chain package
- Remove workaround for limits on fetching height ranges since the new
  database interface no longer imposes them
- Correct the gettxout RPC handler to return the best chain hash as
  opposed the hash the txout was found in
- Optimize various RPC handlers:
  - Change several of the RPC handlers to use the new chain snapshot
    capability to avoid needlessly loading data
  - Update several handlers to use new functionality to avoid accessing
    the block manager so they are able to return the data without
    blocking when the server is busy processing blocks
  - Update non-verbose getblock to avoid deserialization and
    serialization overhead
  - Update getblockheader to request the block height directly from
    chain and only load the header
  - Update getdifficulty to use the new cached data from chain
  - Update getmininginfo to use the new cached data from chain
  - Update non-verbose getrawtransaction to avoid deserialization and
    serialization overhead
  - Update gettxout to use the new utxo store versus loading
    full transactions using the transaction index

The following is an overview of the utility changes:
- Update addblock to use the new database and chain interfaces
- Update findcheckpoint to use the new database and chain interfaces
- Remove the dropafter utility which is no longer supported

NOTE: The transaction index and address index will be reimplemented in
another commit.
2016-04-11 16:47:27 -05:00
Dave Collins
f389742b39 multi: Update with result of gofmt -s.
This commit updates the code to make use of the most recent simplified
output from gofmt.
2016-02-25 13:02:54 -06:00
Dave Collins
eb882f39f8 multi: Fix several misspellings in the comments.
This commit corrects several typos in the comments found by misspell.
2016-02-25 11:17:12 -06:00
Dario Nieuwenhuis
87182a2ddf blockchain: Correct serialized coinbase height error message. 2015-12-21 20:59:12 +01:00
Josh Rickmar
23bcb367b2 blockchain: Fix bogus error message.
In the presence of overflow, the actual output sum is still not
negative and should not be reported as so.
2015-11-09 11:33:37 -05:00
David Hill
7811770d31 Implement BIP0065 changeover logic for v4 blocks.
This commit implements the changeover logic for version 4 blocks as
described by BIP0065.
2015-10-28 13:28:50 -04:00
Dave Collins
3942a116e4 docs: Make various README.md files consistent.
First, it removes the documentation section from all the README.md files
and instead puts a web-based godoc badge and link at the top with the
other badges.  This is being done since the local godoc tool no longer
ships with Go by default, so the instructions no longer work without
first installing godoc. Due to this, pretty much everyone uses the
web-based godoc these days anyways.  Anyone who has manually installed
godoc won't need instructions.

Second, it makes sure the ISC license badge is at the top with the other
badges and removes the textual reference in the overview section.

Finally, it's modifies the Installation section to Installation and
Updating and adds a '-u' to the 'go get' command since it works for both
and thus is simpler.
2015-10-23 14:51:36 -05:00
Olaoluwa Osuntokun
0029905d43 Integrate a valid ECDSA signature cache into btcd
Introduce an ECDSA signature verification into btcd in order to
mitigate a certain DoS attack and as a performance optimization.

The benefits of SigCache are two fold. Firstly, usage of SigCache
mitigates a DoS attack wherein an attacker causes a victim's client to
hang due to worst-case behavior triggered while processing attacker
crafted invalid transactions. A detailed description of the mitigated
DoS attack can be found here: https://bitslog.wordpress.com/2013/01/23/fixed-bitcoin-vulnerability-explanation-why-the-signature-cache-is-a-dos-protection/
Secondly, usage of the SigCache introduces a signature verification
optimization which speeds up the validation of transactions within a
block, if they've already been seen and verified within the mempool.

The server itself manages the sigCache instance. The blockManager and
txMempool respectively now receive pointers to the created sigCache
instance. All read (sig triplet existence) operations on the sigCache
will not block unless a separate goroutine is adding an entry (writing)
to the sigCache. GetBlockTemplate generation now also utilizes the
sigCache in order to avoid unnecessarily double checking signatures
when generating a template after previously accepting a txn to the
mempool. Consequently, the CPU miner now also employs the same
optimization.

The maximum number of entries for the sigCache has been introduced as a
config parameter in order to allow users to configure the amount of
memory consumed by this new additional caching.
2015-10-08 17:31:42 -07:00
danda
43774fe6bb adds optional prevOut section to vin for searchrawtransactions api. See https://github.com/btcsuite/btcd/issues/485 2015-08-23 09:58:03 -07:00
Dave Collins
0280fa0264 Convert block heights to int32.
This commit converts all block height references to int32 instead of
int64.  The current target block production rate is 10 mins per block
which means it will take roughly 40,800 years to reach the maximum
height an int32 affords.  Even if the target rate were lowered to one
block per minute, it would still take roughly another 4,080 years to
reach the maximum.

In the mean time, there is no reason to use a larger type which results
in higher memory and disk space usage.  However, for now, in order to
avoid having to reserialize a bunch of database information, the heights
are still serialized to the database as 8-byte uint64s.

This is being mainly being done in preparation for further upcoming
infrastructure changes which will use the smaller and more efficient
4-byte serialization in the database as well.
2015-08-11 11:13:17 -05:00
David Hill
9ffd96bf51 Revert "Move IsFinalizedTransaction to txscript."
This reverts commit 17da2ba7fa.

This was done prematurely.  This will be revisited when a code
restructure is more urgent.
2015-06-29 11:12:35 -04:00
David Hill
17da2ba7fa Move IsFinalizedTransaction to txscript.
This change moves IsFinalizedTransaction to txscript and also changes
the first argument to take a wire.MsgTx instead of btcutil.Tx.  This
is needed for an upcoming diff in which txscript will require
IsFinalizedTransaction and we do not want to import the btcd/blockchain.
2015-06-28 09:43:14 -04:00
David Hill
527f585463 txscript: Move lockTimeThreshold to txscript
Move lockTimeThreshold to txscript and export it.  This is a
consensus value which txscript will need in an upcoming diff.
2015-06-26 10:55:22 -04:00
Dave Collins
19eae8d8a1 blockchain: Split block and header validation.
This commit refactors the consensus rule checks for block headers and
blocks in the blockchain package into separate functions.  These changes
contain no modifications to consensus rules and the code still passes all
block consensus tests.  It is only a refactoring.

This is being done to help pave the way toward supporting concurrent
downloads.  While the package already supports headers-first mode up
through the latest checkpoint through the use of the BFFastAdd flag and
hard-coded checkpoints, it currently only works when downloading from a
single peer.  In order to support concurrent downloads from multiple
peers, the ability for the caller to do things such as independently
checking a block header (both context-free and full-context checks) will
be needed.

There are several more changes that will be necessary to support
concurrent downloads as well, such as making the package concurrent safe,
modifying it to make use of the new database API, etc.  Those changes are
planned for future commits.
2015-05-12 16:04:42 -05:00
Dave Collins
6e402deb35 Relicense to the btcsuite developers.
This commit relicenses all code in this repository to the btcsuite
developers.
2015-05-01 12:00:56 -05:00
Dave Collins
8dd7412a84 txscript: Rename Script to Engine.
This commit renames the Script type to Engine to better reflect its
purpose.  It also renames the NewScript function to NewEngine to match.

This is being done because name Script for the engine is confusing since
it implies it is an actual script rather than the execution environment
for the script.  It also paves the way for eventually supplying a
ParsedScript type which will be less likely to be confused with the
execution environment.

While moving the code, some additional variable names and comments have
been updated to better match the style used throughout the rest of the
code base.  In addition, an attempt has been made to use consistent naming
of the engine as 'vm' instead of using different variables names as it was
previously.

Finally, the relevant engine code has been moved into a new file named
engine.go and related tests moved to engine_test.go.
2015-04-20 15:31:23 -05:00
Dave Collins
bec90e253c txscript: Remove unneeded param from NewScript.
This commit removes the unnecessary sigScript parameter from the
txscript.NewScript function.  This has bothered me for a while because it
can and really should be obtained from the provided transaction and input
index.  The way it was, the passed script could technically be different
than what is in the transaction.  Obviously that would be an improper use
of the API, but it's safer and more convenient to simply pull it from the
provided transaction and index.

Also, since the function signature is changing anyways, make the input
index parameter come after the transaction which it references.
2015-04-20 13:46:11 -05:00
Dave Collins
a4a52ae24f wire: Remove errs from BlockHeader/MsgBlock/MsgTx.
This commit removes the error returns from the BlockHeader.BlockSha,
MsgBlock.BlockSha, and MsgTx.TxSha functions since they can never fail and
end up causing a lot of unneeded error checking throughout the code base.

It also updates all call sites for the change.
2015-04-17 01:27:12 -05:00
Dave Collins
750d657666 Update for recent btcutil Block.Sha API change. 2015-04-17 00:44:15 -05:00
Dave Collins
6211eef7ee wire: Add new DoubleSha256SH function.
This commit adds a new function which is similar to the DoubleSha256
function except it returns a ShaHash copy instead of a byte slice.  It
also adds a new benchmark for it.

This can be a slight optimization in certain cases where the caller
ultimately wants a ShaHash since it can avoid a heap allocation and
additional copy to convert the result to a ShaHash (the function simply
performs a type cast against the returned array which is not possible
against a []byte).

existing: DoubleSha256     500000   3081 ns/op   32 B/op   1 allocs/op
     new: DoubleSha256SH   500000   2939 ns/op    0 B/op   0 allocs/op

The hashing functions for blocks and transactions have also been updated
to make use of the new function since they directly return the ShaHash.
The transaction change in particular is quite useful since transactions
are frequently hashed and this change allows all of those hashes to avoid
an additional heap allocation.
2015-04-06 11:37:43 -05:00
Dave Collins
f5cdf2d6a8 Minor hashing-related optimizations.
This commit contains three classes of optimizations:
 - Reducing the number of unnecessary hash copies
 - Improve the performance of the DoubleSha256 function
 - A couple of minor optimizations of the ShaHash functions

The first class is a result of the Bytes function on a ShaHash making a
copy of the bytes before returning them.  It really should have been named
CloneBytes, but that would break the API now.

To address this, a comment has been added to the function which explicitly
calls out the copy behavior.  In addition, all call sites of .Bytes on a
ShaHash in the code base have been updated to simply slice the array when
a copy is not needed.  This saves a significant amount of data copying.

The second optimization modifies the DoubleSha256 function to directly use
fastsha256.Sum256 instead of the hasher interface.  This reduces the
number of allocations needed.  A benchmark for the function has been added
as well.

old: BenchmarkDoubleSha256  500000   3691 ns/op   192 B/op   3 allocs/op
new: BenchmarkDoubleSha256  500000   3081 ns/op    32 B/op   1 allocs/op

The final optimizations are for the ShaHash IsEqual and SetBytes functions
which have been modified to make use of the fact the type is an array and
remove an unneeded subslice.
2015-04-06 11:33:58 -05:00
Olaoluwa Osuntokun
1bf564d963 Fix #138 by dynamically updating heights of peers
In order to avoid prior situations of stalled syncs due to
outdated peer height data, we now update block heights up peers in
real-time as we learn of their announced
blocks.

Updates happen when:
   * A peer sends us an orphan block. We update based on
     the height embedded in the scriptSig for the coinbase tx
   * When a peer sends us an inv for a block we already know
     of
   * When peers announce new blocks. Subsequent
     announcements that lost the announcement race are
     recognized and peer heights are updated accordingly

Additionally, the `getpeerinfo` command has been modified
to include both the starting height, and current height of
connected peers.

Docs have been updated with `getpeerinfo` extension.
2015-04-01 17:22:45 -07:00
Dave Collins
279308288c blockchain: Provide new IsCoinBaseTx function.
This commit adds a new function to the blockchain package named
IsCoinBaseTx which performs the same function as IsCoinBase except it
takes raw wire transactions as opposed to the higher level util
transactions.

While here, it also adds a file for benchmarks along with a couple of
benchmarks for the IsCoinBase and IsCoinBaseTx functions.

Finally, the function was very slightly optimized:

BenchmarkIsCoinBaseOld  100000000  10.7 ns/op  0 B/op  0 allocs/op
BenchmarkIsCoinBaseNew  200000000  6.05 ns/op  0 B/op  0 allocs/op
2015-03-10 13:55:24 -05:00
Dave Collins
3ed8f363e7 Implement BIP0066 changeover logic for v3 blocks.
This commit implements the changeover logic for version 3 blocks as
described by BIP0066.
2015-02-26 09:54:29 -06:00
Dave Collins
65eb8020d2 blockchain: Determine script flags sooner.
This commit moves the definition of the flags which are needed to check
transaction scripts higher up the call stack to pave the way for adding
support for v3 blocks.  While here, also spruce up a couple of sections.

There are no functional changes in this commit.
2015-02-25 16:56:34 -06:00
David Hill
3318a24a88 Simplify chain configuration. 2015-02-18 20:33:33 -05:00
David Hill
b40a6f86ef blockchain: Just fetch the height
We only care about the height, so just fetch the height instead of
the entire block.
2015-02-12 16:10:08 -05:00
Josh Rickmar
fc8dae49e4 Make profiling easier to follow with static dispatch.
For example, when performing a heap profile, this:

         .          .    125:   fetchFunc := db.FetchUnSpentTxByShaList
         .          .    126:   if includeSpent {
         .          .    127:           fetchFunc = db.FetchTxByShaList
         .          .    128:   }
         .     9.11MB    129:   txReplyList := fetchFunc(txList)

Now becomes this:

         .          .    125:   var txReplyList []*database.TxListReply
         .          .    126:   if includeSpent {
         .          .    127:           txReplyList = db.FetchTxByShaList(txList)
         .          .    128:   } else {
         .     8.75MB    129:           txReplyList = db.FetchUnSpentTxByShaList(txList)
         .          .    130:   }

And it's clear where the majority of our allocations are coming from.
2015-02-06 22:14:13 -06:00
Dave Collins
c6bc8ac1eb Update btcnet path import paths to new location. 2015-02-05 23:24:53 -06:00
Dave Collins
2e5bb96eea Correct a few instances of missed pkg name changes. 2015-02-05 17:54:04 -06:00
Dave Collins
03433dad6a Update btcwire path import paths to new location. 2015-02-05 15:16:39 -06:00
Dave Collins
b69a849114 Import btcchain repo into blockchain directory.
This commit contains the entire btcchain repository along with several
changes needed to move all of the files into the blockchain directory in
order to prepare it for merging.  This does NOT update btcd or any of the
other packages to use the new location as that will be done separately.

- All import paths in the old btcchain test files have been changed to
  the new location
- All references to btcchain as the package name have been changed to
  blockchain
2015-01-30 15:49:59 -06:00