LBRYCommunity/lbcd
1
0
Fork 0
Code Issues 17 Pull requests 4 Projects Releases 28 Packages Wiki Activity

blockchain: Add block validation infrastructure.

Browse source 
This adds a full-blown testing infrastructure in order to test consensus
validation rules.  It is built around the idea of dynamically generating
full blocks that target specific rules linked together to form a block
chain.  In order to properly test the rules, each test instance starts
with a valid block that is then modified in the specific way needed to
test a specific rule.

Blocks which exercise following rules have been added for this initial
version.  These tests were largely ported from the original Java-based
'official' block acceptance tests as well as some additional tests
available in the Core python port.  It is expected that further tests
can be added over time as consensus rules change.

* Enough valid blocks to have a stable base of mature coinbases to spend
  for futher tests
* Basic forking and chain reorganization
* Double spends on forks
* Too much proof-of-work coinbase (extending main chain, in block that
  forces a reorg, and in a valid fork)
* Max and too many signature operations via various combinations of
  OP_CHECKSIG, OP_MULTISIG, OP_CHECKSIGVERIFY, and OP_MULTISIGVERIFY
* Too many and max signature operations with offending sigop after
  invalid data push
* Max and too many signature operations via pay-to-script-hash redeem
  scripts
* Attempt to spend tx created on a different fork
* Attempt to spend immature coinbase (on main chain and fork)
* Max size block and block that exceeds the max size
* Children of rejected blocks are either orphans or rejected
* Coinbase script too small and too large
* Max length coinbase script
* Attempt to spend tx in blocks that failed to connect
* Valid non-coinbase tx in place of coinbase
* Block with no transactions
* Invalid proof-of-work
* Block with a timestamp too far in the future
* Invalid merkle root
* Invalid proof-of-work limit (bits header field)
* Negative proof-of-work limit (bits header field)
* Two coinbase transactions
* Duplicate transactions
* Spend from transaction that does not exist
* Timestamp exactly at and one second after the median time
* Blocks with same hash via merkle root tricks
* Spend from transaction index that is out of range
* Transaction that spends more that its inputs provide
* Transaction with same hash as an existing tx that has not been
  fully spent (BIP0030)
* Non-final coinbase and non-coinbase txns
* Max size block with canonical encoding which exceeds max size with
  non-canonical encoding
* Spend from transaction earlier in same block
* Spend from transaction later in same block
* Double spend transaction from earlier in same block
* Coinbase that pays more than subsidy + fees
* Coinbase that includes subsidy + fees
* Invalid opcode in dead execution path
* Reorganization of txns with OP_RETURN outputs
* Spend of an OP_RETURN output
* Transaction with multiple OP_RETURN outputs
* Large max-sized block reorganization test (disabled by default since
  it takes a long time and a lot of memory to run)

Finally, the README.md files in the main and docs directories have been
updated to reflect the use of the new testing framework.
This commit is contained in:
Dave Collins 2016-09-13 18:11:12 -05:00
parent 1cba5c8fc0
commit f21410e47c
No known key found for this signature in database
GPG key ID: B8904D9D9C93D1F2
10 changed files with 2539 additions and 27 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

21
README.md
View file

@ -10,17 +10,18 @@ This project is currently under active development and is in a Beta state. It
is extremely stable and has been in production use since October 2013.
It properly downloads, validates, and serves the block chain using the exact
rules (including bugs) for block acceptance as Bitcoin Core. We have taken
great care to avoid btcd causing a fork to the block chain. It passes all of
the 'official' block acceptance tests
(https://github.com/TheBlueMatt/test-scripts) as well as all of the JSON test
data in the Bitcoin Core code.
rules (including consensus bugs) for block acceptance as Bitcoin Core. We have
taken great care to avoid btcd causing a fork to the block chain. It includes a
full block validation testing framework which contains all of the 'official'
block acceptance tests (and some additional ones) that is run on every pull
request to help ensure it properly follows consensus. Also, it passes all of
the JSON test data in the Bitcoin Core code.
It also relays newly mined blocks, maintains a transaction pool, and relays
individual transactions that have not yet made it into a block. It ensures all
transactions admitted to the pool follow the rules required by the block chain
and also includes the same checks which filter transactions based on
miner requirements ("standard" transactions) as Bitcoin Core.
It also properly relays newly mined blocks, maintains a transaction pool, and
relays individual transactions that have not yet made it into a block. It
ensures all individual transactions admitted to the pool follow the rules
required by the block chain and also includes more strict checks which filter
transactions based on miner requirements ("standard" transactions).
One key difference between btcd and Bitcoin Core is that btcd does *NOT* include
wallet functionality and this was a very intentional design decision. See the

3
blockchain/chain_test.go
View file

@ -36,7 +36,8 @@ func TestHaveBlock(t *testing.T) {
}
// Create a new database and chain instance to run tests against.
chain, teardownFunc, err := chainSetup("haveblock")
chain, teardownFunc, err := chainSetup("haveblock",
&chaincfg.MainNetParams)
if err != nil {
t.Errorf("Failed to setup chain instance: %v", err)
return

10
blockchain/common_test.go
View file

@ -18,6 +18,7 @@ import (
"github.com/btcsuite/btcd/chaincfg/chainhash"
"github.com/btcsuite/btcd/database"
_ "github.com/btcsuite/btcd/database/ffldb"
"github.com/btcsuite/btcd/txscript"
"github.com/btcsuite/btcd/wire"
)
@ -56,9 +57,9 @@ func isSupportedDbType(dbType string) bool {
}
// chainSetup is used to create a new db and chain instance with the genesis
// block already inserted. In addition to the new chain instnce, it returns
// block already inserted. In addition to the new chain instance, it returns
// a teardown function the caller should invoke when done testing to clean up.
func chainSetup(dbName string) (*blockchain.BlockChain, func(), error) {
func chainSetup(dbName string, params *chaincfg.Params) (*blockchain.BlockChain, func(), error) {
if !isSupportedDbType(testDbType) {
return nil, nil, fmt.Errorf("unsupported db type %v", testDbType)
}
@ -109,13 +110,14 @@ func chainSetup(dbName string) (*blockchain.BlockChain, func(), error) {
// Copy the chain params to ensure any modifications the tests do to
// the chain parameters do not affect the global instance.
mainNetParams := chaincfg.MainNetParams
paramsCopy := *params
// Create the main chain instance.
chain, err := blockchain.New(&blockchain.Config{
DB: db,
ChainParams: &mainNetParams,
ChainParams: &paramsCopy,
TimeSource: blockchain.NewMedianTime(),
SigCache: txscript.NewSigCache(1000),
})
if err != nil {
teardown()

198
blockchain/fullblocks_test.go Normal file
View file

@ -0,0 +1,198 @@
// Copyright (c) 2016 The Decred developers
// Copyright (c) 2016 The btcsuite developers
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.
package blockchain_test
import (
"bytes"
"testing"
"github.com/btcsuite/btcd/blockchain"
"github.com/btcsuite/btcd/blockchain/fullblocktests"
"github.com/btcsuite/btcd/chaincfg"
"github.com/btcsuite/btcd/chaincfg/chainhash"
"github.com/btcsuite/btcd/wire"
"github.com/btcsuite/btcutil"
)
// TestFullBlocks ensures all tests generated by the fullblocktests package
// have the expected result when processed via ProcessBlock.
func TestFullBlocks(t *testing.T) {
tests, err := fullblocktests.Generate(false)
if err != nil {
t.Fatalf("failed to generate tests: %v", err)
}
// Create a new database and chain instance to run tests against.
chain, teardownFunc, err := chainSetup("fullblocktest",
&chaincfg.RegressionNetParams)
if err != nil {
t.Errorf("Failed to setup chain instance: %v", err)
return
}
defer teardownFunc()
// testAcceptedBlock attempts to process the block in the provided test
// instance and ensures that it was accepted according to the flags
// specified in the test.
testAcceptedBlock := func(item fullblocktests.AcceptedBlock) {
blockHeight := item.Height
block := btcutil.NewBlock(item.Block)
block.SetHeight(blockHeight)
t.Logf("Testing block %s (hash %s, height %d)",
item.Name, block.Hash(), blockHeight)
isMainChain, isOrphan, err := chain.ProcessBlock(block,
blockchain.BFNone)
if err != nil {
t.Fatalf("block %q (hash %s, height %d) should "+
"have been accepted: %v", item.Name,
block.Hash(), blockHeight, err)
}
// Ensure the main chain and orphan flags match the values
// specified in the test.
if isMainChain != item.IsMainChain {
t.Fatalf("block %q (hash %s, height %d) unexpected main "+
"chain flag -- got %v, want %v", item.Name,
block.Hash(), blockHeight, isMainChain,
item.IsMainChain)
}
if isOrphan != item.IsOrphan {
t.Fatalf("block %q (hash %s, height %d) unexpected "+
"orphan flag -- got %v, want %v", item.Name,
block.Hash(), blockHeight, isOrphan,
item.IsOrphan)
}
}
// testRejectedBlock attempts to process the block in the provided test
// instance and ensures that it was rejected with the reject code
// specified in the test.
testRejectedBlock := func(item fullblocktests.RejectedBlock) {
blockHeight := item.Height
block := btcutil.NewBlock(item.Block)
block.SetHeight(blockHeight)
t.Logf("Testing block %s (hash %s, height %d)",
item.Name, block.Hash(), blockHeight)
_, _, err := chain.ProcessBlock(block, blockchain.BFNone)
if err == nil {
t.Fatalf("block %q (hash %s, height %d) should not "+
"have been accepted", item.Name, block.Hash(),
blockHeight)
}
// Ensure the error code is of the expected type and the reject
// code matches the value specified in the test instance.
rerr, ok := err.(blockchain.RuleError)
if !ok {
t.Fatalf("block %q (hash %s, height %d) returned "+
"unexpected error type -- got %T, want "+
"blockchain.RuleError", item.Name, block.Hash(),
blockHeight, err)
}
if rerr.ErrorCode != item.RejectCode {
t.Fatalf("block %q (hash %s, height %d) does not have "+
"expected reject code -- got %v, want %v",
item.Name, block.Hash(), blockHeight,
rerr.ErrorCode, item.RejectCode)
}
}
// testRejectedNonCanonicalBlock attempts to decode the block in the
// provided test instance and ensures that it failed to decode with a
// message error.
testRejectedNonCanonicalBlock := func(item fullblocktests.RejectedNonCanonicalBlock) {
headerLen := len(item.RawBlock)
if headerLen > 80 {
headerLen = 80
}
blockHash := chainhash.DoubleHashH(item.RawBlock[0:headerLen])
blockHeight := item.Height
t.Logf("Testing block %s (hash %s, height %d)", item.Name,
blockHash, blockHeight)
// Ensure there is an error due to deserializing the block.
var msgBlock wire.MsgBlock
err := msgBlock.BtcDecode(bytes.NewReader(item.RawBlock), 0)
if _, ok := err.(*wire.MessageError); !ok {
t.Fatalf("block %q (hash %s, height %d) should have "+
"failed to decode", item.Name, blockHash,
blockHeight)
}
}
// testOrphanOrRejectedBlock attempts to process the block in the
// provided test instance and ensures that it was either accepted as an
// orphan or rejected with a rule violation.
testOrphanOrRejectedBlock := func(item fullblocktests.OrphanOrRejectedBlock) {
blockHeight := item.Height
block := btcutil.NewBlock(item.Block)
block.SetHeight(blockHeight)
t.Logf("Testing block %s (hash %s, height %d)",
item.Name, block.Hash(), blockHeight)
_, isOrphan, err := chain.ProcessBlock(block, blockchain.BFNone)
if err != nil {
// Ensure the error code is of the expected type.
if _, ok := err.(blockchain.RuleError); !ok {
t.Fatalf("block %q (hash %s, height %d) "+
"returned unexpected error type -- "+
"got %T, want blockchain.RuleError",
item.Name, block.Hash(), blockHeight,
err)
}
}
if !isOrphan {
t.Fatalf("block %q (hash %s, height %d) was accepted, "+
"but is not considered an orphan", item.Name,
block.Hash(), blockHeight)
}
}
// testExpectedTip ensures the current tip of the blockchain is the
// block specified in the provided test instance.
testExpectedTip := func(item fullblocktests.ExpectedTip) {
blockHeight := item.Height
block := btcutil.NewBlock(item.Block)
block.SetHeight(blockHeight)
t.Logf("Testing tip for block %s (hash %s, height %d)",
item.Name, block.Hash(), blockHeight)
// Ensure hash and height match.
best := chain.BestSnapshot()
if *best.Hash != item.Block.BlockHash() ||
best.Height != blockHeight {
t.Fatalf("block %q (hash %s, height %d) should be "+
"the current tip -- got (hash %s, height %d)",
item.Name, block.Hash(), blockHeight, best.Hash,
best.Height)
}
}
for testNum, test := range tests {
for itemNum, item := range test {
switch item := item.(type) {
case fullblocktests.AcceptedBlock:
testAcceptedBlock(item)
case fullblocktests.RejectedBlock:
testRejectedBlock(item)
case fullblocktests.RejectedNonCanonicalBlock:
testRejectedNonCanonicalBlock(item)
case fullblocktests.OrphanOrRejectedBlock:
testOrphanOrRejectedBlock(item)
case fullblocktests.ExpectedTip:
testExpectedTip(item)
default:
t.Fatalf("test #%d, item #%d is not one of "+
"the supported test instance types -- "+
"got type: %T", testNum, itemNum, item)
}
}
}
}

31
blockchain/fullblocktests/README.md Normal file
View file

@ -0,0 +1,31 @@
fullblocktests
==============
[![Build Status](http://img.shields.io/travis/btcsuite/btcd.svg)]
(https://travis-ci.org/btcsuite/btcd) [![ISC License]
(http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org)
[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)]
(http://godoc.org/github.com/btcsuite/btcd/blockchain/fullblocktests)
Package fullblocktests provides a set of full block tests to be used for testing
the consensus validation rules. The tests are intended to be flexible enough to
allow both unit-style tests directly against the blockchain code as well as
integration style tests over the peer-to-peer network. To achieve that goal,
each test contains additional information about the expected result, however
that information can be ignored when doing comparison tests between two
independent versions over the peer-to-peer network.
This package has intentionally been designed so it can be used as a standalone
package for any projects needing to test their implementation against a full set
of blocks that excerise the consensus validation rules.
## Installation and Updating
```bash
$ go get -u github.com/btcsuite/btcd/blockchain/fullblocktests
```
## License
Package fullblocktests is licensed under the [copyfree](http://copyfree.org) ISC
License.

2135
blockchain/fullblocktests/generate.go Normal file
View file

File diff suppressed because it is too large Load diff

142
blockchain/fullblocktests/params.go Normal file
View file

@ -0,0 +1,142 @@
// Copyright (c) 2016 The btcsuite developers
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.
package fullblocktests
import (
"encoding/hex"
"math/big"
"time"
"github.com/btcsuite/btcd/chaincfg"
"github.com/btcsuite/btcd/chaincfg/chainhash"
"github.com/btcsuite/btcd/wire"
)
// newHashFromStr converts the passed big-endian hex string into a
// wire.Hash. It only differs from the one available in chainhash in that
// it panics on an error since it will only (and must only) be called with
// hard-coded, and therefore known good, hashes.
func newHashFromStr(hexStr string) *chainhash.Hash {
hash, err := chainhash.NewHashFromStr(hexStr)
if err != nil {
panic(err)
}
return hash
}
// fromHex converts the passed hex string into a byte slice and will panic if
// there is an error. This is only provided for the hard-coded constants so
// errors in the source code can be detected. It will only (and must only) be
// called for initialization purposes.
func fromHex(s string) []byte {
r, err := hex.DecodeString(s)
if err != nil {
panic("invalid hex in source file: " + s)
}
return r
}
var (
// bigOne is 1 represented as a big.Int. It is defined here to avoid
// the overhead of creating it multiple times.
bigOne = big.NewInt(1)
// regressionPowLimit is the highest proof of work value a Bitcoin block
// can have for the regression test network. It is the value 2^255 - 1.
regressionPowLimit = new(big.Int).Sub(new(big.Int).Lsh(bigOne, 255), bigOne)
// regTestGenesisBlock defines the genesis block of the block chain which serves
// as the public transaction ledger for the regression test network.
regTestGenesisBlock = wire.MsgBlock{
Header: wire.BlockHeader{
Version: 1,
PrevBlock: *newHashFromStr("0000000000000000000000000000000000000000000000000000000000000000"),
MerkleRoot: *newHashFromStr("4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b"),
Timestamp: time.Unix(1296688602, 0), // 2011-02-02 23:16:42 +0000 UTC
Bits: 0x207fffff, // 545259519 [7fffff0000000000000000000000000000000000000000000000000000000000]
Nonce: 2,
},
Transactions: []*wire.MsgTx{{
Version: 1,
TxIn: []*wire.TxIn{{
PreviousOutPoint: wire.OutPoint{
Hash: chainhash.Hash{},
Index: 0xffffffff,
},
SignatureScript: fromHex("04ffff001d010445" +
"5468652054696d65732030332f4a616e2f" +
"32303039204368616e63656c6c6f72206f" +
"6e206272696e6b206f66207365636f6e64" +
"206261696c6f757420666f72206261686b73"),
Sequence: 0xffffffff,
}},
TxOut: []*wire.TxOut{{
Value: 0,
PkScript: fromHex("4104678afdb0fe5548271967f1" +
"a67130b7105cd6a828e03909a67962e0ea1f" +
"61deb649f6bc3f4cef38c4f35504e51ec138" +
"c4f35504e51ec112de5c384df7ba0b8d578a" +
"4c702b6bf11d5fac"),
}},
LockTime: 0,
}},
}
)
// regressionNetParams defines the network parameters for the regression test
// network.
//
// NOTE: The test generator intentionally does not use the existing definitions
// in the chaincfg package since the intent is to be able to generate known
// good tests which exercise that code. Using the chaincfg parameters would
// allow them to change out from under the tests potentially invalidating them.
var regressionNetParams = &chaincfg.Params{
Name: "regtest",
Net: wire.TestNet,
DefaultPort: "18444",
// Chain parameters
GenesisBlock: &regTestGenesisBlock,
GenesisHash: newHashFromStr("5bec7567af40504e0994db3b573c186fffcc4edefe096ff2e58d00523bd7e8a6"),
PowLimit: regressionPowLimit,
PowLimitBits: 0x207fffff,
CoinbaseMaturity: 100,
SubsidyReductionInterval: 150,
TargetTimespan: time.Hour * 24 * 14, // 14 days
TargetTimePerBlock: time.Minute * 10, // 10 minutes
RetargetAdjustmentFactor: 4, // 25% less, 400% more
ReduceMinDifficulty: true,
MinDiffReductionTime: time.Minute * 20, // TargetTimePerBlock * 2
GenerateSupported: true,
// Checkpoints ordered from oldest to newest.
Checkpoints: nil,
// Enforce current block version once majority of the network has
// upgraded.
// 75% (750 / 1000)
// Reject previous block versions once a majority of the network has
// upgraded.
// 95% (950 / 1000)
BlockEnforceNumRequired: 750,
BlockRejectNumRequired: 950,
BlockUpgradeNumToCheck: 1000,
// Mempool parameters
RelayNonStdTxs: true,
// Address encoding magics
PubKeyHashAddrID: 0x6f, // starts with m or n
ScriptHashAddrID: 0xc4, // starts with 2
PrivateKeyID: 0xef, // starts with 9 (uncompressed) or c (compressed)
// BIP32 hierarchical deterministic extended key magics
HDPrivateKeyID: [4]byte{0x04, 0x35, 0x83, 0x94}, // starts with tprv
HDPublicKeyID: [4]byte{0x04, 0x35, 0x87, 0xcf}, // starts with tpub
// BIP44 coin type used in the hierarchical deterministic path for
// address generation.
HDCoinType: 1,
}

3
blockchain/reorganization_test.go
View file

@ -14,6 +14,7 @@ import (
"testing"
"github.com/btcsuite/btcd/blockchain"
"github.com/btcsuite/btcd/chaincfg"
"github.com/btcsuite/btcd/wire"
"github.com/btcsuite/btcutil"
)
@ -46,7 +47,7 @@ func TestReorganization(t *testing.T) {
t.Logf("Number of blocks: %v\n", len(blocks))
// Create a new database and chain instance to run tests against.
chain, teardownFunc, err := chainSetup("reorg")
chain, teardownFunc, err := chainSetup("reorg", &chaincfg.MainNetParams)
if err != nil {
t.Errorf("Failed to setup chain instance: %v", err)
return

3
blockchain/validate_test.go
View file

@ -21,7 +21,8 @@ import (
// fails.
func TestCheckConnectBlock(t *testing.T) {
// Create a new database and chain instance to run tests against.
chain, teardownFunc, err := chainSetup("checkconnectblock")
chain, teardownFunc, err := chainSetup("checkconnectblock",
&chaincfg.MainNetParams)
if err != nil {
t.Errorf("Failed to setup chain instance: %v", err)
return

20
docs/README.md
View file

@ -29,19 +29,19 @@ licensed under the [copyfree](http://www.copyfree.org) ISC License.
This project is currently under active development and is in a Beta state. It
is extremely stable and has been in production use since October 2013.
It currently properly downloads, validates, and serves the block chain using the
exact rules (including bugs) for block acceptance as the reference
implementation, [bitcoind](https://github.com/bitcoin/bitcoin). We have taken
great care to avoid btcd causing a fork to the block chain. It passes all of
the '[official](https://github.com/TheBlueMatt/test-scripts/)' block acceptance
tests.
It properly downloads, validates, and serves the block chain using the exact
rules (including consensus bugs) for block acceptance as Bitcoin Core. We have
taken great care to avoid btcd causing a fork to the block chain. It includes a
full block validation testing framework which contains all of the 'official'
block acceptance tests (and some additional ones) that is run on every pull
request to help ensure it properly follows consensus. Also, it passes all of
the JSON test data in the Bitcoin Core code.
It also properly relays newly mined blocks, maintains a transaction pool, and
relays individual transactions that have not yet made it into a block. It
relays individual transactions that have not yet made it into a block. It
ensures all individual transactions admitted to the pool follow the rules
required into the block chain and also includes the vast majority of the more
strict checks which filter transactions based on miner requirements ("standard"
transactions).
required by the block chain and also includes more strict checks which filter
transactions based on miner requirements ("standard" transactions).
One key difference between btcd and Bitcoin Core is that btcd does *NOT* include
wallet functionality and this was a very intentional design decision. See the