blockchain: Add block validation infrastructure.

This adds a full-blown testing infrastructure in order to test consensus
validation rules.  It is built around the idea of dynamically generating
full blocks that target specific rules linked together to form a block
chain.  In order to properly test the rules, each test instance starts
with a valid block that is then modified in the specific way needed to
test a specific rule.

Blocks which exercise following rules have been added for this initial
version.  These tests were largely ported from the original Java-based
'official' block acceptance tests as well as some additional tests
available in the Core python port.  It is expected that further tests
can be added over time as consensus rules change.

* Enough valid blocks to have a stable base of mature coinbases to spend
  for futher tests
* Basic forking and chain reorganization
* Double spends on forks
* Too much proof-of-work coinbase (extending main chain, in block that
  forces a reorg, and in a valid fork)
* Max and too many signature operations via various combinations of
  OP_CHECKSIG, OP_MULTISIG, OP_CHECKSIGVERIFY, and OP_MULTISIGVERIFY
* Too many and max signature operations with offending sigop after
  invalid data push
* Max and too many signature operations via pay-to-script-hash redeem
  scripts
* Attempt to spend tx created on a different fork
* Attempt to spend immature coinbase (on main chain and fork)
* Max size block and block that exceeds the max size
* Children of rejected blocks are either orphans or rejected
* Coinbase script too small and too large
* Max length coinbase script
* Attempt to spend tx in blocks that failed to connect
* Valid non-coinbase tx in place of coinbase
* Block with no transactions
* Invalid proof-of-work
* Block with a timestamp too far in the future
* Invalid merkle root
* Invalid proof-of-work limit (bits header field)
* Negative proof-of-work limit (bits header field)
* Two coinbase transactions
* Duplicate transactions
* Spend from transaction that does not exist
* Timestamp exactly at and one second after the median time
* Blocks with same hash via merkle root tricks
* Spend from transaction index that is out of range
* Transaction that spends more that its inputs provide
* Transaction with same hash as an existing tx that has not been
  fully spent (BIP0030)
* Non-final coinbase and non-coinbase txns
* Max size block with canonical encoding which exceeds max size with
  non-canonical encoding
* Spend from transaction earlier in same block
* Spend from transaction later in same block
* Double spend transaction from earlier in same block
* Coinbase that pays more than subsidy + fees
* Coinbase that includes subsidy + fees
* Invalid opcode in dead execution path
* Reorganization of txns with OP_RETURN outputs
* Spend of an OP_RETURN output
* Transaction with multiple OP_RETURN outputs
* Large max-sized block reorganization test (disabled by default since
  it takes a long time and a lot of memory to run)

Finally, the README.md files in the main and docs directories have been
updated to reflect the use of the new testing framework.

README.md

@@ -10,17 +10,18 @@ This project is currently under active development and is in a Beta state.  It
 is extremely stable and has been in production use since October 2013.
 
 It properly downloads, validates, and serves the block chain using the exact
-rules (including bugs) for block acceptance as Bitcoin Core.  We have taken
+rules (including consensus bugs) for block acceptance as Bitcoin Core.  We have
-great care to avoid btcd causing a fork to the block chain.  It passes all of
+taken great care to avoid btcd causing a fork to the block chain.  It includes a
-the 'official' block acceptance tests
+full block validation testing framework which contains all of the 'official'
-(https://github.com/TheBlueMatt/test-scripts) as well as all of the JSON test
+block acceptance tests (and some additional ones) that is run on every pull
-data in the Bitcoin Core code.
+request to help ensure it properly follows consensus.  Also, it passes all of
+the JSON test data in the Bitcoin Core code.
 
-It also relays newly mined blocks, maintains a transaction pool, and relays
+It also properly relays newly mined blocks, maintains a transaction pool, and
-individual transactions that have not yet made it into a block.  It ensures all
+relays individual transactions that have not yet made it into a block.  It
-transactions admitted to the pool follow the rules required by the block chain
+ensures all individual transactions admitted to the pool follow the rules
-and also includes the same checks which filter transactions based on
+required by the block chain and also includes more strict checks which filter
-miner requirements ("standard" transactions) as Bitcoin Core.
+transactions based on miner requirements ("standard" transactions).
 
 One key difference between btcd and Bitcoin Core is that btcd does *NOT* include
 wallet functionality and this was a very intentional design decision.  See the

blockchain/chain_test.go

@@ -36,7 +36,8 @@ func TestHaveBlock(t *testing.T) {
 	}
 
 	// Create a new database and chain instance to run tests against.
-	chain, teardownFunc, err := chainSetup("haveblock")
+	chain, teardownFunc, err := chainSetup("haveblock",
+		&chaincfg.MainNetParams)
 	if err != nil {
 		t.Errorf("Failed to setup chain instance: %v", err)
 		return

blockchain/common_test.go

@@ -18,6 +18,7 @@ import (
 	"github.com/btcsuite/btcd/chaincfg/chainhash"
 	"github.com/btcsuite/btcd/database"
 	_ "github.com/btcsuite/btcd/database/ffldb"
+	"github.com/btcsuite/btcd/txscript"
 	"github.com/btcsuite/btcd/wire"
 )
 
@@ -56,9 +57,9 @@ func isSupportedDbType(dbType string) bool {
 }
 
 // chainSetup is used to create a new db and chain instance with the genesis
-// block already inserted.  In addition to the new chain instnce, it returns
+// block already inserted.  In addition to the new chain instance, it returns
 // a teardown function the caller should invoke when done testing to clean up.
-func chainSetup(dbName string) (*blockchain.BlockChain, func(), error) {
+func chainSetup(dbName string, params *chaincfg.Params) (*blockchain.BlockChain, func(), error) {
 	if !isSupportedDbType(testDbType) {
 		return nil, nil, fmt.Errorf("unsupported db type %v", testDbType)
 	}
@@ -109,13 +110,14 @@ func chainSetup(dbName string) (*blockchain.BlockChain, func(), error) {
 
 	// Copy the chain params to ensure any modifications the tests do to
 	// the chain parameters do not affect the global instance.
-	mainNetParams := chaincfg.MainNetParams
+	paramsCopy := *params
 
 	// Create the main chain instance.
 	chain, err := blockchain.New(&blockchain.Config{
 		DB:          db,
-		ChainParams: &mainNetParams,
+		ChainParams: &paramsCopy,
 		TimeSource:  blockchain.NewMedianTime(),
+		SigCache:    txscript.NewSigCache(1000),
 	})
 	if err != nil {
 		teardown()

blockchain/fullblocks_test.go

@@ -0,0 +1,198 @@
+// Copyright (c) 2016 The Decred developers
+// Copyright (c) 2016 The btcsuite developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package blockchain_test
+
+import (
+	"bytes"
+	"testing"
+
+	"github.com/btcsuite/btcd/blockchain"
+	"github.com/btcsuite/btcd/blockchain/fullblocktests"
+	"github.com/btcsuite/btcd/chaincfg"
+	"github.com/btcsuite/btcd/chaincfg/chainhash"
+	"github.com/btcsuite/btcd/wire"
+	"github.com/btcsuite/btcutil"
+)
+
+// TestFullBlocks ensures all tests generated by the fullblocktests package
+// have the expected result when processed via ProcessBlock.
+func TestFullBlocks(t *testing.T) {
+	tests, err := fullblocktests.Generate(false)
+	if err != nil {
+		t.Fatalf("failed to generate tests: %v", err)
+	}
+
+	// Create a new database and chain instance to run tests against.
+	chain, teardownFunc, err := chainSetup("fullblocktest",
+		&chaincfg.RegressionNetParams)
+	if err != nil {
+		t.Errorf("Failed to setup chain instance: %v", err)
+		return
+	}
+	defer teardownFunc()
+
+	// testAcceptedBlock attempts to process the block in the provided test
+	// instance and ensures that it was accepted according to the flags
+	// specified in the test.
+	testAcceptedBlock := func(item fullblocktests.AcceptedBlock) {
+		blockHeight := item.Height
+		block := btcutil.NewBlock(item.Block)
+		block.SetHeight(blockHeight)
+		t.Logf("Testing block %s (hash %s, height %d)",
+			item.Name, block.Hash(), blockHeight)
+
+		isMainChain, isOrphan, err := chain.ProcessBlock(block,
+			blockchain.BFNone)
+		if err != nil {
+			t.Fatalf("block %q (hash %s, height %d) should "+
+				"have been accepted: %v", item.Name,
+				block.Hash(), blockHeight, err)
+		}
+
+		// Ensure the main chain and orphan flags match the values
+		// specified in the test.
+		if isMainChain != item.IsMainChain {
+			t.Fatalf("block %q (hash %s, height %d) unexpected main "+
+				"chain flag -- got %v, want %v", item.Name,
+				block.Hash(), blockHeight, isMainChain,
+				item.IsMainChain)
+		}
+		if isOrphan != item.IsOrphan {
+			t.Fatalf("block %q (hash %s, height %d) unexpected "+
+				"orphan flag -- got %v, want %v", item.Name,
+				block.Hash(), blockHeight, isOrphan,
+				item.IsOrphan)
+		}
+	}
+
+	// testRejectedBlock attempts to process the block in the provided test
+	// instance and ensures that it was rejected with the reject code
+	// specified in the test.
+	testRejectedBlock := func(item fullblocktests.RejectedBlock) {
+		blockHeight := item.Height
+		block := btcutil.NewBlock(item.Block)
+		block.SetHeight(blockHeight)
+		t.Logf("Testing block %s (hash %s, height %d)",
+			item.Name, block.Hash(), blockHeight)
+
+		_, _, err := chain.ProcessBlock(block, blockchain.BFNone)
+		if err == nil {
+			t.Fatalf("block %q (hash %s, height %d) should not "+
+				"have been accepted", item.Name, block.Hash(),
+				blockHeight)
+		}
+
+		// Ensure the error code is of the expected type and the reject
+		// code matches the value specified in the test instance.
+		rerr, ok := err.(blockchain.RuleError)
+		if !ok {
+			t.Fatalf("block %q (hash %s, height %d) returned "+
+				"unexpected error type -- got %T, want "+
+				"blockchain.RuleError", item.Name, block.Hash(),
+				blockHeight, err)
+		}
+		if rerr.ErrorCode != item.RejectCode {
+			t.Fatalf("block %q (hash %s, height %d) does not have "+
+				"expected reject code -- got %v, want %v",
+				item.Name, block.Hash(), blockHeight,
+				rerr.ErrorCode, item.RejectCode)
+		}
+	}
+
+	// testRejectedNonCanonicalBlock attempts to decode the block in the
+	// provided test instance and ensures that it failed to decode with a
+	// message error.
+	testRejectedNonCanonicalBlock := func(item fullblocktests.RejectedNonCanonicalBlock) {
+		headerLen := len(item.RawBlock)
+		if headerLen > 80 {
+			headerLen = 80
+		}
+		blockHash := chainhash.DoubleHashH(item.RawBlock[0:headerLen])
+		blockHeight := item.Height
+		t.Logf("Testing block %s (hash %s, height %d)", item.Name,
+			blockHash, blockHeight)
+
+		// Ensure there is an error due to deserializing the block.
+		var msgBlock wire.MsgBlock
+		err := msgBlock.BtcDecode(bytes.NewReader(item.RawBlock), 0)
+		if _, ok := err.(*wire.MessageError); !ok {
+			t.Fatalf("block %q (hash %s, height %d) should have "+
+				"failed to decode", item.Name, blockHash,
+				blockHeight)
+		}
+	}
+
+	// testOrphanOrRejectedBlock attempts to process the block in the
+	// provided test instance and ensures that it was either accepted as an
+	// orphan or rejected with a rule violation.
+	testOrphanOrRejectedBlock := func(item fullblocktests.OrphanOrRejectedBlock) {
+		blockHeight := item.Height
+		block := btcutil.NewBlock(item.Block)
+		block.SetHeight(blockHeight)
+		t.Logf("Testing block %s (hash %s, height %d)",
+			item.Name, block.Hash(), blockHeight)
+
+		_, isOrphan, err := chain.ProcessBlock(block, blockchain.BFNone)
+		if err != nil {
+			// Ensure the error code is of the expected type.
+			if _, ok := err.(blockchain.RuleError); !ok {
+				t.Fatalf("block %q (hash %s, height %d) "+
+					"returned unexpected error type -- "+
+					"got %T, want blockchain.RuleError",
+					item.Name, block.Hash(), blockHeight,
+					err)
+			}
+		}
+
+		if !isOrphan {
+			t.Fatalf("block %q (hash %s, height %d) was accepted, "+
+				"but is not considered an orphan", item.Name,
+				block.Hash(), blockHeight)
+		}
+	}
+
+	// testExpectedTip ensures the current tip of the blockchain is the
+	// block specified in the provided test instance.
+	testExpectedTip := func(item fullblocktests.ExpectedTip) {
+		blockHeight := item.Height
+		block := btcutil.NewBlock(item.Block)
+		block.SetHeight(blockHeight)
+		t.Logf("Testing tip for block %s (hash %s, height %d)",
+			item.Name, block.Hash(), blockHeight)
+
+		// Ensure hash and height match.
+		best := chain.BestSnapshot()
+		if *best.Hash != item.Block.BlockHash() ||
+			best.Height != blockHeight {
+
+			t.Fatalf("block %q (hash %s, height %d) should be "+
+				"the current tip -- got (hash %s, height %d)",
+				item.Name, block.Hash(), blockHeight, best.Hash,
+				best.Height)
+		}
+	}
+
+	for testNum, test := range tests {
+		for itemNum, item := range test {
+			switch item := item.(type) {
+			case fullblocktests.AcceptedBlock:
+				testAcceptedBlock(item)
+			case fullblocktests.RejectedBlock:
+				testRejectedBlock(item)
+			case fullblocktests.RejectedNonCanonicalBlock:
+				testRejectedNonCanonicalBlock(item)
+			case fullblocktests.OrphanOrRejectedBlock:
+				testOrphanOrRejectedBlock(item)
+			case fullblocktests.ExpectedTip:
+				testExpectedTip(item)
+			default:
+				t.Fatalf("test #%d, item #%d is not one of "+
+					"the supported test instance types -- "+
+					"got type: %T", testNum, itemNum, item)
+			}
+		}
+	}
+}

blockchain/fullblocktests/README.md

@@ -0,0 +1,31 @@
+fullblocktests
+==============
+
+[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)]
+(https://travis-ci.org/btcsuite/btcd) [![ISC License]
+(http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org)
+[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)]
+(http://godoc.org/github.com/btcsuite/btcd/blockchain/fullblocktests)
+
+Package fullblocktests provides a set of full block tests to be used for testing
+the consensus validation rules.  The tests are intended to be flexible enough to
+allow both unit-style tests directly against the blockchain code as well as
+integration style tests over the peer-to-peer network.  To achieve that goal,
+each test contains additional information about the expected result, however
+that information can be ignored when doing comparison tests between two
+independent versions over the peer-to-peer network.
+
+This package has intentionally been designed so it can be used as a standalone
+package for any projects needing to test their implementation against a full set
+of blocks that excerise the consensus validation rules.
+
+## Installation and Updating
+
+```bash
+$ go get -u github.com/btcsuite/btcd/blockchain/fullblocktests
+```
+
+## License
+
+Package fullblocktests is licensed under the [copyfree](http://copyfree.org) ISC
+License.

blockchain/fullblocktests/params.go

@@ -0,0 +1,142 @@
+// Copyright (c) 2016 The btcsuite developers
+// Use of this source code is governed by an ISC
+// license that can be found in the LICENSE file.
+
+package fullblocktests
+
+import (
+	"encoding/hex"
+	"math/big"
+	"time"
+
+	"github.com/btcsuite/btcd/chaincfg"
+	"github.com/btcsuite/btcd/chaincfg/chainhash"
+	"github.com/btcsuite/btcd/wire"
+)
+
+// newHashFromStr converts the passed big-endian hex string into a
+// wire.Hash.  It only differs from the one available in chainhash in that
+// it panics on an error since it will only (and must only) be called with
+// hard-coded, and therefore known good, hashes.
+func newHashFromStr(hexStr string) *chainhash.Hash {
+	hash, err := chainhash.NewHashFromStr(hexStr)
+	if err != nil {
+		panic(err)
+	}
+	return hash
+}
+
+// fromHex converts the passed hex string into a byte slice and will panic if
+// there is an error.  This is only provided for the hard-coded constants so
+// errors in the source code can be detected. It will only (and must only) be
+// called for initialization purposes.
+func fromHex(s string) []byte {
+	r, err := hex.DecodeString(s)
+	if err != nil {
+		panic("invalid hex in source file: " + s)
+	}
+	return r
+}
+
+var (
+	// bigOne is 1 represented as a big.Int.  It is defined here to avoid
+	// the overhead of creating it multiple times.
+	bigOne = big.NewInt(1)
+
+	// regressionPowLimit is the highest proof of work value a Bitcoin block
+	// can have for the regression test network.  It is the value 2^255 - 1.
+	regressionPowLimit = new(big.Int).Sub(new(big.Int).Lsh(bigOne, 255), bigOne)
+
+	// regTestGenesisBlock defines the genesis block of the block chain which serves
+	// as the public transaction ledger for the regression test network.
+	regTestGenesisBlock = wire.MsgBlock{
+		Header: wire.BlockHeader{
+			Version:    1,
+			PrevBlock:  *newHashFromStr("0000000000000000000000000000000000000000000000000000000000000000"),
+			MerkleRoot: *newHashFromStr("4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b"),
+			Timestamp:  time.Unix(1296688602, 0), // 2011-02-02 23:16:42 +0000 UTC
+			Bits:       0x207fffff,               // 545259519 [7fffff0000000000000000000000000000000000000000000000000000000000]
+			Nonce:      2,
+		},
+		Transactions: []*wire.MsgTx{{
+			Version: 1,
+			TxIn: []*wire.TxIn{{
+				PreviousOutPoint: wire.OutPoint{
+					Hash:  chainhash.Hash{},
+					Index: 0xffffffff,
+				},
+				SignatureScript: fromHex("04ffff001d010445" +
+					"5468652054696d65732030332f4a616e2f" +
+					"32303039204368616e63656c6c6f72206f" +
+					"6e206272696e6b206f66207365636f6e64" +
+					"206261696c6f757420666f72206261686b73"),
+				Sequence: 0xffffffff,
+			}},
+			TxOut: []*wire.TxOut{{
+				Value: 0,
+				PkScript: fromHex("4104678afdb0fe5548271967f1" +
+					"a67130b7105cd6a828e03909a67962e0ea1f" +
+					"61deb649f6bc3f4cef38c4f35504e51ec138" +
+					"c4f35504e51ec112de5c384df7ba0b8d578a" +
+					"4c702b6bf11d5fac"),
+			}},
+			LockTime: 0,
+		}},
+	}
+)
+
+// regressionNetParams defines the network parameters for the regression test
+// network.
+//
+// NOTE: The test generator intentionally does not use the existing definitions
+// in the chaincfg package since the intent is to be able to generate known
+// good tests which exercise that code.  Using the chaincfg parameters would
+// allow them to change out from under the tests potentially invalidating them.
+var regressionNetParams = &chaincfg.Params{
+	Name:        "regtest",
+	Net:         wire.TestNet,
+	DefaultPort: "18444",
+
+	// Chain parameters
+	GenesisBlock:             &regTestGenesisBlock,
+	GenesisHash:              newHashFromStr("5bec7567af40504e0994db3b573c186fffcc4edefe096ff2e58d00523bd7e8a6"),
+	PowLimit:                 regressionPowLimit,
+	PowLimitBits:             0x207fffff,
+	CoinbaseMaturity:         100,
+	SubsidyReductionInterval: 150,
+	TargetTimespan:           time.Hour * 24 * 14, // 14 days
+	TargetTimePerBlock:       time.Minute * 10,    // 10 minutes
+	RetargetAdjustmentFactor: 4,                   // 25% less, 400% more
+	ReduceMinDifficulty:      true,
+	MinDiffReductionTime:     time.Minute * 20, // TargetTimePerBlock * 2
+	GenerateSupported:        true,
+
+	// Checkpoints ordered from oldest to newest.
+	Checkpoints: nil,
+
+	// Enforce current block version once majority of the network has
+	// upgraded.
+	// 75% (750 / 1000)
+	// Reject previous block versions once a majority of the network has
+	// upgraded.
+	// 95% (950 / 1000)
+	BlockEnforceNumRequired: 750,
+	BlockRejectNumRequired:  950,
+	BlockUpgradeNumToCheck:  1000,
+
+	// Mempool parameters
+	RelayNonStdTxs: true,
+
+	// Address encoding magics
+	PubKeyHashAddrID: 0x6f, // starts with m or n
+	ScriptHashAddrID: 0xc4, // starts with 2
+	PrivateKeyID:     0xef, // starts with 9 (uncompressed) or c (compressed)
+
+	// BIP32 hierarchical deterministic extended key magics
+	HDPrivateKeyID: [4]byte{0x04, 0x35, 0x83, 0x94}, // starts with tprv
+	HDPublicKeyID:  [4]byte{0x04, 0x35, 0x87, 0xcf}, // starts with tpub
+
+	// BIP44 coin type used in the hierarchical deterministic path for
+	// address generation.
+	HDCoinType: 1,
+}

blockchain/reorganization_test.go

@@ -14,6 +14,7 @@ import (
 	"testing"
 
 	"github.com/btcsuite/btcd/blockchain"
+	"github.com/btcsuite/btcd/chaincfg"
 	"github.com/btcsuite/btcd/wire"
 	"github.com/btcsuite/btcutil"
 )
@@ -46,7 +47,7 @@ func TestReorganization(t *testing.T) {
 	t.Logf("Number of blocks: %v\n", len(blocks))
 
 	// Create a new database and chain instance to run tests against.
-	chain, teardownFunc, err := chainSetup("reorg")
+	chain, teardownFunc, err := chainSetup("reorg", &chaincfg.MainNetParams)
 	if err != nil {
 		t.Errorf("Failed to setup chain instance: %v", err)
 		return

blockchain/validate_test.go

@@ -21,7 +21,8 @@ import (
 // fails.
 func TestCheckConnectBlock(t *testing.T) {
 	// Create a new database and chain instance to run tests against.
-	chain, teardownFunc, err := chainSetup("checkconnectblock")
+	chain, teardownFunc, err := chainSetup("checkconnectblock",
+		&chaincfg.MainNetParams)
 	if err != nil {
 		t.Errorf("Failed to setup chain instance: %v", err)
 		return

docs/README.md

@@ -29,19 +29,19 @@ licensed under the [copyfree](http://www.copyfree.org) ISC License.
 This project is currently under active development and is in a Beta state.  It
 is extremely stable and has been in production use since October 2013.
 
-It currently properly downloads, validates, and serves the block chain using the
+It properly downloads, validates, and serves the block chain using the exact
-exact rules (including bugs) for block acceptance as the reference
+rules (including consensus bugs) for block acceptance as Bitcoin Core.  We have
-implementation, [bitcoind](https://github.com/bitcoin/bitcoin).  We have taken
+taken great care to avoid btcd causing a fork to the block chain.  It includes a
-great care to avoid btcd causing a fork to the block chain. It passes all of
+full block validation testing framework which contains all of the 'official'
-the '[official](https://github.com/TheBlueMatt/test-scripts/)' block acceptance
+block acceptance tests (and some additional ones) that is run on every pull
-tests.
+request to help ensure it properly follows consensus.  Also, it passes all of
+the JSON test data in the Bitcoin Core code.
 
 It also properly relays newly mined blocks, maintains a transaction pool, and
-relays individual transactions that have not yet made it into a block. It
+relays individual transactions that have not yet made it into a block.  It
 ensures all individual transactions admitted to the pool follow the rules
-required into the block chain and also includes the vast majority of the more
+required by the block chain and also includes more strict checks which filter
-strict checks which filter transactions based on miner requirements ("standard"
+transactions based on miner requirements ("standard" transactions).
-transactions).
 
 One key difference between btcd and Bitcoin Core is that btcd does *NOT* include
 wallet functionality and this was a very intentional design decision.  See the