lbcd/mempool/mempool_test.go
Dave Collins a59ac5b18f
multi: Rework utxoset/view to use outpoints.
This modifies the utxoset in the database and related UtxoViewpoint to
store and work with unspent transaction outputs on a per-output basis
instead of at a transaction level.  This was inspired by similar recent
changes in Bitcoin Core.

The primary motivation is to simplify the code, pave the way for a
utxo cache, and generally focus on optimizing runtime performance.

The tradeoff is that this approach does somewhat increase the size of
the serialized utxoset since it means that the transaction hash is
duplicated for each output as a part of the key and some additional
details such as whether the containing transaction is a coinbase and the
block height it was a part of are duplicated in each output.

However, in practice, the size difference isn't all that large, disk
space is relatively cheap, certainly cheaper than memory, and it is much
more important to provide more efficient runtime operation since that is
the ultimate purpose of the daemon.

While performing this conversion, it also simplifies the code to remove
the transaction version information from the utxoset as well as the
spend journal.  The logic for only serializing it under certain
circumstances is complicated and it isn't actually used anywhere aside
from the gettxout RPC where it also isn't used by anything important
either.  Consequently, this also removes the version field of the
gettxout RPC result.

The utxos in the database are automatically migrated to the new format
with this commit and it is possible to interrupt and resume the
migration process.

Finally, it also updates the tests for the new format and adds a new
function to the tests to convert the old test data to the new format for
convenience.  The data has already been converted and updated in the
commit.

An overview of the changes are as follows:

- Remove transaction version from both spent and unspent output entries
  - Update utxo serialization format to exclude the version
  - Modify the spend journal serialization format
    - The old version field is now reserved and always stores zero and
      ignores it when reading
    - This allows old entries to be used by new code without having to
      migrate the entire spend journal
  - Remove version field from gettxout RPC result
- Convert UtxoEntry to represent a specific utxo instead of a
  transaction with all remaining utxos
  - Optimize for memory usage with an eye towards a utxo cache
    - Combine details such as whether the txout was contained in a
      coinbase, is spent, and is modified into a single packed field of
      bit flags
    - Align entry fields to eliminate extra padding since ultimately
      there will be a lot of these in memory
    - Introduce a free list for serializing an outpoint to the database
      key format to significantly reduce pressure on the GC
  - Update all related functions that previously dealt with transaction
    hashes to accept outpoints instead
  - Update all callers accordingly
  - Only add individually requested outputs from the mempool when
    constructing a mempool view
- Modify the spend journal to always store the block height and coinbase
  information with every spent txout
  - Introduce code to handle fetching the missing information from
    another utxo from the same transaction in the event an old style
    entry is encountered
    - Make use of a database cursor with seek to do this much more
      efficiently than testing every possible output
- Always decompress data loaded from the database now that a utxo entry
  only consists of a specific output
- Introduce upgrade code to migrate the utxo set to the new format
  - Store versions of the utxoset and spend journal buckets
  - Allow migration process to be interrupted and resumed
- Update all tests to expect the correct encodings, remove tests that no
  longer apply, and add new ones for the new expected behavior
  - Convert old tests for the legacy utxo format deserialization code to
    test the new function that is used during upgrade
  - Update the utxostore test data and add function that was used to
    convert it
- Introduce a few new functions on UtxoViewpoint
  - AddTxOut for adding an individual txout versus all of them
  - addTxOut to handle the common code between the new AddTxOut and
    existing AddTxOuts
  - RemoveEntry for removing an individual txout
  - fetchEntryByHash for fetching any remaining utxo for a given
    transaction hash
2018-05-27 03:07:41 -05:00

868 lines
29 KiB
Go

// Copyright (c) 2016 The btcsuite developers
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.
package mempool
import (
"encoding/hex"
"reflect"
"runtime"
"sync"
"testing"
"time"
"github.com/btcsuite/btcd/blockchain"
"github.com/btcsuite/btcd/btcec"
"github.com/btcsuite/btcd/chaincfg"
"github.com/btcsuite/btcd/chaincfg/chainhash"
"github.com/btcsuite/btcd/txscript"
"github.com/btcsuite/btcd/wire"
"github.com/btcsuite/btcutil"
)
// fakeChain is used by the pool harness to provide generated test utxos and
// a current faked chain height to the pool callbacks. This, in turn, allows
// transactions to appear as though they are spending completely valid utxos.
type fakeChain struct {
sync.RWMutex
utxos *blockchain.UtxoViewpoint
currentHeight int32
medianTimePast time.Time
}
// FetchUtxoView loads utxo details about the inputs referenced by the passed
// transaction from the point of view of the fake chain. It also attempts to
// fetch the utxos for the outputs of the transaction itself so the returned
// view can be examined for duplicate transactions.
//
// This function is safe for concurrent access however the returned view is NOT.
func (s *fakeChain) FetchUtxoView(tx *btcutil.Tx) (*blockchain.UtxoViewpoint, error) {
s.RLock()
defer s.RUnlock()
// All entries are cloned to ensure modifications to the returned view
// do not affect the fake chain's view.
// Add an entry for the tx itself to the new view.
viewpoint := blockchain.NewUtxoViewpoint()
prevOut := wire.OutPoint{Hash: *tx.Hash()}
for txOutIdx := range tx.MsgTx().TxOut {
prevOut.Index = uint32(txOutIdx)
entry := s.utxos.LookupEntry(prevOut)
viewpoint.Entries()[prevOut] = entry.Clone()
}
// Add entries for all of the inputs to the tx to the new view.
for _, txIn := range tx.MsgTx().TxIn {
entry := s.utxos.LookupEntry(txIn.PreviousOutPoint)
viewpoint.Entries()[txIn.PreviousOutPoint] = entry.Clone()
}
return viewpoint, nil
}
// BestHeight returns the current height associated with the fake chain
// instance.
func (s *fakeChain) BestHeight() int32 {
s.RLock()
height := s.currentHeight
s.RUnlock()
return height
}
// SetHeight sets the current height associated with the fake chain instance.
func (s *fakeChain) SetHeight(height int32) {
s.Lock()
s.currentHeight = height
s.Unlock()
}
// MedianTimePast returns the current median time past associated with the fake
// chain instance.
func (s *fakeChain) MedianTimePast() time.Time {
s.RLock()
mtp := s.medianTimePast
s.RUnlock()
return mtp
}
// SetMedianTimePast sets the current median time past associated with the fake
// chain instance.
func (s *fakeChain) SetMedianTimePast(mtp time.Time) {
s.Lock()
s.medianTimePast = mtp
s.Unlock()
}
// CalcSequenceLock returns the current sequence lock for the passed
// transaction associated with the fake chain instance.
func (s *fakeChain) CalcSequenceLock(tx *btcutil.Tx,
view *blockchain.UtxoViewpoint) (*blockchain.SequenceLock, error) {
return &blockchain.SequenceLock{
Seconds: -1,
BlockHeight: -1,
}, nil
}
// spendableOutput is a convenience type that houses a particular utxo and the
// amount associated with it.
type spendableOutput struct {
outPoint wire.OutPoint
amount btcutil.Amount
}
// txOutToSpendableOut returns a spendable output given a transaction and index
// of the output to use. This is useful as a convenience when creating test
// transactions.
func txOutToSpendableOut(tx *btcutil.Tx, outputNum uint32) spendableOutput {
return spendableOutput{
outPoint: wire.OutPoint{Hash: *tx.Hash(), Index: outputNum},
amount: btcutil.Amount(tx.MsgTx().TxOut[outputNum].Value),
}
}
// poolHarness provides a harness that includes functionality for creating and
// signing transactions as well as a fake chain that provides utxos for use in
// generating valid transactions.
type poolHarness struct {
// signKey is the signing key used for creating transactions throughout
// the tests.
//
// payAddr is the p2sh address for the signing key and is used for the
// payment address throughout the tests.
signKey *btcec.PrivateKey
payAddr btcutil.Address
payScript []byte
chainParams *chaincfg.Params
chain *fakeChain
txPool *TxPool
}
// CreateCoinbaseTx returns a coinbase transaction with the requested number of
// outputs paying an appropriate subsidy based on the passed block height to the
// address associated with the harness. It automatically uses a standard
// signature script that starts with the block height that is required by
// version 2 blocks.
func (p *poolHarness) CreateCoinbaseTx(blockHeight int32, numOutputs uint32) (*btcutil.Tx, error) {
// Create standard coinbase script.
extraNonce := int64(0)
coinbaseScript, err := txscript.NewScriptBuilder().
AddInt64(int64(blockHeight)).AddInt64(extraNonce).Script()
if err != nil {
return nil, err
}
tx := wire.NewMsgTx(wire.TxVersion)
tx.AddTxIn(&wire.TxIn{
// Coinbase transactions have no inputs, so previous outpoint is
// zero hash and max index.
PreviousOutPoint: *wire.NewOutPoint(&chainhash.Hash{},
wire.MaxPrevOutIndex),
SignatureScript: coinbaseScript,
Sequence: wire.MaxTxInSequenceNum,
})
totalInput := blockchain.CalcBlockSubsidy(blockHeight, p.chainParams)
amountPerOutput := totalInput / int64(numOutputs)
remainder := totalInput - amountPerOutput*int64(numOutputs)
for i := uint32(0); i < numOutputs; i++ {
// Ensure the final output accounts for any remainder that might
// be left from splitting the input amount.
amount := amountPerOutput
if i == numOutputs-1 {
amount = amountPerOutput + remainder
}
tx.AddTxOut(&wire.TxOut{
PkScript: p.payScript,
Value: amount,
})
}
return btcutil.NewTx(tx), nil
}
// CreateSignedTx creates a new signed transaction that consumes the provided
// inputs and generates the provided number of outputs by evenly splitting the
// total input amount. All outputs will be to the payment script associated
// with the harness and all inputs are assumed to do the same.
func (p *poolHarness) CreateSignedTx(inputs []spendableOutput, numOutputs uint32) (*btcutil.Tx, error) {
// Calculate the total input amount and split it amongst the requested
// number of outputs.
var totalInput btcutil.Amount
for _, input := range inputs {
totalInput += input.amount
}
amountPerOutput := int64(totalInput) / int64(numOutputs)
remainder := int64(totalInput) - amountPerOutput*int64(numOutputs)
tx := wire.NewMsgTx(wire.TxVersion)
for _, input := range inputs {
tx.AddTxIn(&wire.TxIn{
PreviousOutPoint: input.outPoint,
SignatureScript: nil,
Sequence: wire.MaxTxInSequenceNum,
})
}
for i := uint32(0); i < numOutputs; i++ {
// Ensure the final output accounts for any remainder that might
// be left from splitting the input amount.
amount := amountPerOutput
if i == numOutputs-1 {
amount = amountPerOutput + remainder
}
tx.AddTxOut(&wire.TxOut{
PkScript: p.payScript,
Value: amount,
})
}
// Sign the new transaction.
for i := range tx.TxIn {
sigScript, err := txscript.SignatureScript(tx, i, p.payScript,
txscript.SigHashAll, p.signKey, true)
if err != nil {
return nil, err
}
tx.TxIn[i].SignatureScript = sigScript
}
return btcutil.NewTx(tx), nil
}
// CreateTxChain creates a chain of zero-fee transactions (each subsequent
// transaction spends the entire amount from the previous one) with the first
// one spending the provided outpoint. Each transaction spends the entire
// amount of the previous one and as such does not include any fees.
func (p *poolHarness) CreateTxChain(firstOutput spendableOutput, numTxns uint32) ([]*btcutil.Tx, error) {
txChain := make([]*btcutil.Tx, 0, numTxns)
prevOutPoint := firstOutput.outPoint
spendableAmount := firstOutput.amount
for i := uint32(0); i < numTxns; i++ {
// Create the transaction using the previous transaction output
// and paying the full amount to the payment address associated
// with the harness.
tx := wire.NewMsgTx(wire.TxVersion)
tx.AddTxIn(&wire.TxIn{
PreviousOutPoint: prevOutPoint,
SignatureScript: nil,
Sequence: wire.MaxTxInSequenceNum,
})
tx.AddTxOut(&wire.TxOut{
PkScript: p.payScript,
Value: int64(spendableAmount),
})
// Sign the new transaction.
sigScript, err := txscript.SignatureScript(tx, 0, p.payScript,
txscript.SigHashAll, p.signKey, true)
if err != nil {
return nil, err
}
tx.TxIn[0].SignatureScript = sigScript
txChain = append(txChain, btcutil.NewTx(tx))
// Next transaction uses outputs from this one.
prevOutPoint = wire.OutPoint{Hash: tx.TxHash(), Index: 0}
}
return txChain, nil
}
// newPoolHarness returns a new instance of a pool harness initialized with a
// fake chain and a TxPool bound to it that is configured with a policy suitable
// for testing. Also, the fake chain is populated with the returned spendable
// outputs so the caller can easily create new valid transactions which build
// off of it.
func newPoolHarness(chainParams *chaincfg.Params) (*poolHarness, []spendableOutput, error) {
// Use a hard coded key pair for deterministic results.
keyBytes, err := hex.DecodeString("700868df1838811ffbdf918fb482c1f7e" +
"ad62db4b97bd7012c23e726485e577d")
if err != nil {
return nil, nil, err
}
signKey, signPub := btcec.PrivKeyFromBytes(btcec.S256(), keyBytes)
// Generate associated pay-to-script-hash address and resulting payment
// script.
pubKeyBytes := signPub.SerializeCompressed()
payPubKeyAddr, err := btcutil.NewAddressPubKey(pubKeyBytes, chainParams)
if err != nil {
return nil, nil, err
}
payAddr := payPubKeyAddr.AddressPubKeyHash()
pkScript, err := txscript.PayToAddrScript(payAddr)
if err != nil {
return nil, nil, err
}
// Create a new fake chain and harness bound to it.
chain := &fakeChain{utxos: blockchain.NewUtxoViewpoint()}
harness := poolHarness{
signKey: signKey,
payAddr: payAddr,
payScript: pkScript,
chainParams: chainParams,
chain: chain,
txPool: New(&Config{
Policy: Policy{
DisableRelayPriority: true,
FreeTxRelayLimit: 15.0,
MaxOrphanTxs: 5,
MaxOrphanTxSize: 1000,
MaxSigOpCostPerTx: blockchain.MaxBlockSigOpsCost / 4,
MinRelayTxFee: 1000, // 1 Satoshi per byte
MaxTxVersion: 1,
},
ChainParams: chainParams,
FetchUtxoView: chain.FetchUtxoView,
BestHeight: chain.BestHeight,
MedianTimePast: chain.MedianTimePast,
CalcSequenceLock: chain.CalcSequenceLock,
SigCache: nil,
AddrIndex: nil,
}),
}
// Create a single coinbase transaction and add it to the harness
// chain's utxo set and set the harness chain height such that the
// coinbase will mature in the next block. This ensures the txpool
// accepts transactions which spend immature coinbases that will become
// mature in the next block.
numOutputs := uint32(1)
outputs := make([]spendableOutput, 0, numOutputs)
curHeight := harness.chain.BestHeight()
coinbase, err := harness.CreateCoinbaseTx(curHeight+1, numOutputs)
if err != nil {
return nil, nil, err
}
harness.chain.utxos.AddTxOuts(coinbase, curHeight+1)
for i := uint32(0); i < numOutputs; i++ {
outputs = append(outputs, txOutToSpendableOut(coinbase, i))
}
harness.chain.SetHeight(int32(chainParams.CoinbaseMaturity) + curHeight)
harness.chain.SetMedianTimePast(time.Now())
return &harness, outputs, nil
}
// testContext houses a test-related state that is useful to pass to helper
// functions as a single argument.
type testContext struct {
t *testing.T
harness *poolHarness
}
// testPoolMembership tests the transaction pool associated with the provided
// test context to determine if the passed transaction matches the provided
// orphan pool and transaction pool status. It also further determines if it
// should be reported as available by the HaveTransaction function based upon
// the two flags and tests that condition as well.
func testPoolMembership(tc *testContext, tx *btcutil.Tx, inOrphanPool, inTxPool bool) {
txHash := tx.Hash()
gotOrphanPool := tc.harness.txPool.IsOrphanInPool(txHash)
if inOrphanPool != gotOrphanPool {
_, file, line, _ := runtime.Caller(1)
tc.t.Fatalf("%s:%d -- IsOrphanInPool: want %v, got %v", file,
line, inOrphanPool, gotOrphanPool)
}
gotTxPool := tc.harness.txPool.IsTransactionInPool(txHash)
if inTxPool != gotTxPool {
_, file, line, _ := runtime.Caller(1)
tc.t.Fatalf("%s:%d -- IsTransactionInPool: want %v, got %v",
file, line, inTxPool, gotTxPool)
}
gotHaveTx := tc.harness.txPool.HaveTransaction(txHash)
wantHaveTx := inOrphanPool || inTxPool
if wantHaveTx != gotHaveTx {
_, file, line, _ := runtime.Caller(1)
tc.t.Fatalf("%s:%d -- HaveTransaction: want %v, got %v", file,
line, wantHaveTx, gotHaveTx)
}
}
// TestSimpleOrphanChain ensures that a simple chain of orphans is handled
// properly. In particular, it generates a chain of single input, single output
// transactions and inserts them while skipping the first linking transaction so
// they are all orphans. Finally, it adds the linking transaction and ensures
// the entire orphan chain is moved to the transaction pool.
func TestSimpleOrphanChain(t *testing.T) {
t.Parallel()
harness, spendableOuts, err := newPoolHarness(&chaincfg.MainNetParams)
if err != nil {
t.Fatalf("unable to create test pool: %v", err)
}
tc := &testContext{t, harness}
// Create a chain of transactions rooted with the first spendable output
// provided by the harness.
maxOrphans := uint32(harness.txPool.cfg.Policy.MaxOrphanTxs)
chainedTxns, err := harness.CreateTxChain(spendableOuts[0], maxOrphans+1)
if err != nil {
t.Fatalf("unable to create transaction chain: %v", err)
}
// Ensure the orphans are accepted (only up to the maximum allowed so
// none are evicted).
for _, tx := range chainedTxns[1 : maxOrphans+1] {
acceptedTxns, err := harness.txPool.ProcessTransaction(tx, true,
false, 0)
if err != nil {
t.Fatalf("ProcessTransaction: failed to accept valid "+
"orphan %v", err)
}
// Ensure no transactions were reported as accepted.
if len(acceptedTxns) != 0 {
t.Fatalf("ProcessTransaction: reported %d accepted "+
"transactions from what should be an orphan",
len(acceptedTxns))
}
// Ensure the transaction is in the orphan pool, is not in the
// transaction pool, and is reported as available.
testPoolMembership(tc, tx, true, false)
}
// Add the transaction which completes the orphan chain and ensure they
// all get accepted. Notice the accept orphans flag is also false here
// to ensure it has no bearing on whether or not already existing
// orphans in the pool are linked.
acceptedTxns, err := harness.txPool.ProcessTransaction(chainedTxns[0],
false, false, 0)
if err != nil {
t.Fatalf("ProcessTransaction: failed to accept valid "+
"orphan %v", err)
}
if len(acceptedTxns) != len(chainedTxns) {
t.Fatalf("ProcessTransaction: reported accepted transactions "+
"length does not match expected -- got %d, want %d",
len(acceptedTxns), len(chainedTxns))
}
for _, txD := range acceptedTxns {
// Ensure the transaction is no longer in the orphan pool, is
// now in the transaction pool, and is reported as available.
testPoolMembership(tc, txD.Tx, false, true)
}
}
// TestOrphanReject ensures that orphans are properly rejected when the allow
// orphans flag is not set on ProcessTransaction.
func TestOrphanReject(t *testing.T) {
t.Parallel()
harness, outputs, err := newPoolHarness(&chaincfg.MainNetParams)
if err != nil {
t.Fatalf("unable to create test pool: %v", err)
}
tc := &testContext{t, harness}
// Create a chain of transactions rooted with the first spendable output
// provided by the harness.
maxOrphans := uint32(harness.txPool.cfg.Policy.MaxOrphanTxs)
chainedTxns, err := harness.CreateTxChain(outputs[0], maxOrphans+1)
if err != nil {
t.Fatalf("unable to create transaction chain: %v", err)
}
// Ensure orphans are rejected when the allow orphans flag is not set.
for _, tx := range chainedTxns[1:] {
acceptedTxns, err := harness.txPool.ProcessTransaction(tx, false,
false, 0)
if err == nil {
t.Fatalf("ProcessTransaction: did not fail on orphan "+
"%v when allow orphans flag is false", tx.Hash())
}
expectedErr := RuleError{}
if reflect.TypeOf(err) != reflect.TypeOf(expectedErr) {
t.Fatalf("ProcessTransaction: wrong error got: <%T> %v, "+
"want: <%T>", err, err, expectedErr)
}
code, extracted := extractRejectCode(err)
if !extracted {
t.Fatalf("ProcessTransaction: failed to extract reject "+
"code from error %q", err)
}
if code != wire.RejectDuplicate {
t.Fatalf("ProcessTransaction: unexpected reject code "+
"-- got %v, want %v", code, wire.RejectDuplicate)
}
// Ensure no transactions were reported as accepted.
if len(acceptedTxns) != 0 {
t.Fatal("ProcessTransaction: reported %d accepted "+
"transactions from failed orphan attempt",
len(acceptedTxns))
}
// Ensure the transaction is not in the orphan pool, not in the
// transaction pool, and not reported as available
testPoolMembership(tc, tx, false, false)
}
}
// TestOrphanEviction ensures that exceeding the maximum number of orphans
// evicts entries to make room for the new ones.
func TestOrphanEviction(t *testing.T) {
t.Parallel()
harness, outputs, err := newPoolHarness(&chaincfg.MainNetParams)
if err != nil {
t.Fatalf("unable to create test pool: %v", err)
}
tc := &testContext{t, harness}
// Create a chain of transactions rooted with the first spendable output
// provided by the harness that is long enough to be able to force
// several orphan evictions.
maxOrphans := uint32(harness.txPool.cfg.Policy.MaxOrphanTxs)
chainedTxns, err := harness.CreateTxChain(outputs[0], maxOrphans+5)
if err != nil {
t.Fatalf("unable to create transaction chain: %v", err)
}
// Add enough orphans to exceed the max allowed while ensuring they are
// all accepted. This will cause an eviction.
for _, tx := range chainedTxns[1:] {
acceptedTxns, err := harness.txPool.ProcessTransaction(tx, true,
false, 0)
if err != nil {
t.Fatalf("ProcessTransaction: failed to accept valid "+
"orphan %v", err)
}
// Ensure no transactions were reported as accepted.
if len(acceptedTxns) != 0 {
t.Fatalf("ProcessTransaction: reported %d accepted "+
"transactions from what should be an orphan",
len(acceptedTxns))
}
// Ensure the transaction is in the orphan pool, is not in the
// transaction pool, and is reported as available.
testPoolMembership(tc, tx, true, false)
}
// Figure out which transactions were evicted and make sure the number
// evicted matches the expected number.
var evictedTxns []*btcutil.Tx
for _, tx := range chainedTxns[1:] {
if !harness.txPool.IsOrphanInPool(tx.Hash()) {
evictedTxns = append(evictedTxns, tx)
}
}
expectedEvictions := len(chainedTxns) - 1 - int(maxOrphans)
if len(evictedTxns) != expectedEvictions {
t.Fatalf("unexpected number of evictions -- got %d, want %d",
len(evictedTxns), expectedEvictions)
}
// Ensure none of the evicted transactions ended up in the transaction
// pool.
for _, tx := range evictedTxns {
testPoolMembership(tc, tx, false, false)
}
}
// TestBasicOrphanRemoval ensure that orphan removal works as expected when an
// orphan that doesn't exist is removed both when there is another orphan that
// redeems it and when there is not.
func TestBasicOrphanRemoval(t *testing.T) {
t.Parallel()
const maxOrphans = 4
harness, spendableOuts, err := newPoolHarness(&chaincfg.MainNetParams)
if err != nil {
t.Fatalf("unable to create test pool: %v", err)
}
harness.txPool.cfg.Policy.MaxOrphanTxs = maxOrphans
tc := &testContext{t, harness}
// Create a chain of transactions rooted with the first spendable output
// provided by the harness.
chainedTxns, err := harness.CreateTxChain(spendableOuts[0], maxOrphans+1)
if err != nil {
t.Fatalf("unable to create transaction chain: %v", err)
}
// Ensure the orphans are accepted (only up to the maximum allowed so
// none are evicted).
for _, tx := range chainedTxns[1 : maxOrphans+1] {
acceptedTxns, err := harness.txPool.ProcessTransaction(tx, true,
false, 0)
if err != nil {
t.Fatalf("ProcessTransaction: failed to accept valid "+
"orphan %v", err)
}
// Ensure no transactions were reported as accepted.
if len(acceptedTxns) != 0 {
t.Fatalf("ProcessTransaction: reported %d accepted "+
"transactions from what should be an orphan",
len(acceptedTxns))
}
// Ensure the transaction is in the orphan pool, not in the
// transaction pool, and reported as available.
testPoolMembership(tc, tx, true, false)
}
// Attempt to remove an orphan that has no redeemers and is not present,
// and ensure the state of all other orphans are unaffected.
nonChainedOrphanTx, err := harness.CreateSignedTx([]spendableOutput{{
amount: btcutil.Amount(5000000000),
outPoint: wire.OutPoint{Hash: chainhash.Hash{}, Index: 0},
}}, 1)
if err != nil {
t.Fatalf("unable to create signed tx: %v", err)
}
harness.txPool.RemoveOrphan(nonChainedOrphanTx)
testPoolMembership(tc, nonChainedOrphanTx, false, false)
for _, tx := range chainedTxns[1 : maxOrphans+1] {
testPoolMembership(tc, tx, true, false)
}
// Attempt to remove an orphan that has a existing redeemer but itself
// is not present and ensure the state of all other orphans (including
// the one that redeems it) are unaffected.
harness.txPool.RemoveOrphan(chainedTxns[0])
testPoolMembership(tc, chainedTxns[0], false, false)
for _, tx := range chainedTxns[1 : maxOrphans+1] {
testPoolMembership(tc, tx, true, false)
}
// Remove each orphan one-by-one and ensure they are removed as
// expected.
for _, tx := range chainedTxns[1 : maxOrphans+1] {
harness.txPool.RemoveOrphan(tx)
testPoolMembership(tc, tx, false, false)
}
}
// TestOrphanChainRemoval ensure that orphan chains (orphans that spend outputs
// from other orphans) are removed as expected.
func TestOrphanChainRemoval(t *testing.T) {
t.Parallel()
const maxOrphans = 10
harness, spendableOuts, err := newPoolHarness(&chaincfg.MainNetParams)
if err != nil {
t.Fatalf("unable to create test pool: %v", err)
}
harness.txPool.cfg.Policy.MaxOrphanTxs = maxOrphans
tc := &testContext{t, harness}
// Create a chain of transactions rooted with the first spendable output
// provided by the harness.
chainedTxns, err := harness.CreateTxChain(spendableOuts[0], maxOrphans+1)
if err != nil {
t.Fatalf("unable to create transaction chain: %v", err)
}
// Ensure the orphans are accepted (only up to the maximum allowed so
// none are evicted).
for _, tx := range chainedTxns[1 : maxOrphans+1] {
acceptedTxns, err := harness.txPool.ProcessTransaction(tx, true,
false, 0)
if err != nil {
t.Fatalf("ProcessTransaction: failed to accept valid "+
"orphan %v", err)
}
// Ensure no transactions were reported as accepted.
if len(acceptedTxns) != 0 {
t.Fatalf("ProcessTransaction: reported %d accepted "+
"transactions from what should be an orphan",
len(acceptedTxns))
}
// Ensure the transaction is in the orphan pool, not in the
// transaction pool, and reported as available.
testPoolMembership(tc, tx, true, false)
}
// Remove the first orphan that starts the orphan chain without the
// remove redeemer flag set and ensure that only the first orphan was
// removed.
harness.txPool.mtx.Lock()
harness.txPool.removeOrphan(chainedTxns[1], false)
harness.txPool.mtx.Unlock()
testPoolMembership(tc, chainedTxns[1], false, false)
for _, tx := range chainedTxns[2 : maxOrphans+1] {
testPoolMembership(tc, tx, true, false)
}
// Remove the first remaining orphan that starts the orphan chain with
// the remove redeemer flag set and ensure they are all removed.
harness.txPool.mtx.Lock()
harness.txPool.removeOrphan(chainedTxns[2], true)
harness.txPool.mtx.Unlock()
for _, tx := range chainedTxns[2 : maxOrphans+1] {
testPoolMembership(tc, tx, false, false)
}
}
// TestMultiInputOrphanDoubleSpend ensures that orphans that spend from an
// output that is spend by another transaction entering the pool are removed.
func TestMultiInputOrphanDoubleSpend(t *testing.T) {
t.Parallel()
const maxOrphans = 4
harness, outputs, err := newPoolHarness(&chaincfg.MainNetParams)
if err != nil {
t.Fatalf("unable to create test pool: %v", err)
}
harness.txPool.cfg.Policy.MaxOrphanTxs = maxOrphans
tc := &testContext{t, harness}
// Create a chain of transactions rooted with the first spendable output
// provided by the harness.
chainedTxns, err := harness.CreateTxChain(outputs[0], maxOrphans+1)
if err != nil {
t.Fatalf("unable to create transaction chain: %v", err)
}
// Start by adding the orphan transactions from the generated chain
// except the final one.
for _, tx := range chainedTxns[1:maxOrphans] {
acceptedTxns, err := harness.txPool.ProcessTransaction(tx, true,
false, 0)
if err != nil {
t.Fatalf("ProcessTransaction: failed to accept valid "+
"orphan %v", err)
}
if len(acceptedTxns) != 0 {
t.Fatalf("ProcessTransaction: reported %d accepted transactions "+
"from what should be an orphan", len(acceptedTxns))
}
testPoolMembership(tc, tx, true, false)
}
// Ensure a transaction that contains a double spend of the same output
// as the second orphan that was just added as well as a valid spend
// from that last orphan in the chain generated above (and is not in the
// orphan pool) is accepted to the orphan pool. This must be allowed
// since it would otherwise be possible for a malicious actor to disrupt
// tx chains.
doubleSpendTx, err := harness.CreateSignedTx([]spendableOutput{
txOutToSpendableOut(chainedTxns[1], 0),
txOutToSpendableOut(chainedTxns[maxOrphans], 0),
}, 1)
if err != nil {
t.Fatalf("unable to create signed tx: %v", err)
}
acceptedTxns, err := harness.txPool.ProcessTransaction(doubleSpendTx,
true, false, 0)
if err != nil {
t.Fatalf("ProcessTransaction: failed to accept valid orphan %v",
err)
}
if len(acceptedTxns) != 0 {
t.Fatalf("ProcessTransaction: reported %d accepted transactions "+
"from what should be an orphan", len(acceptedTxns))
}
testPoolMembership(tc, doubleSpendTx, true, false)
// Add the transaction which completes the orphan chain and ensure the
// chain gets accepted. Notice the accept orphans flag is also false
// here to ensure it has no bearing on whether or not already existing
// orphans in the pool are linked.
//
// This will cause the shared output to become a concrete spend which
// will in turn must cause the double spending orphan to be removed.
acceptedTxns, err = harness.txPool.ProcessTransaction(chainedTxns[0],
false, false, 0)
if err != nil {
t.Fatalf("ProcessTransaction: failed to accept valid tx %v", err)
}
if len(acceptedTxns) != maxOrphans {
t.Fatalf("ProcessTransaction: reported accepted transactions "+
"length does not match expected -- got %d, want %d",
len(acceptedTxns), maxOrphans)
}
for _, txD := range acceptedTxns {
// Ensure the transaction is no longer in the orphan pool, is
// in the transaction pool, and is reported as available.
testPoolMembership(tc, txD.Tx, false, true)
}
// Ensure the double spending orphan is no longer in the orphan pool and
// was not moved to the transaction pool.
testPoolMembership(tc, doubleSpendTx, false, false)
}
// TestCheckSpend tests that CheckSpend returns the expected spends found in
// the mempool.
func TestCheckSpend(t *testing.T) {
t.Parallel()
harness, outputs, err := newPoolHarness(&chaincfg.MainNetParams)
if err != nil {
t.Fatalf("unable to create test pool: %v", err)
}
// The mempool is empty, so none of the spendable outputs should have a
// spend there.
for _, op := range outputs {
spend := harness.txPool.CheckSpend(op.outPoint)
if spend != nil {
t.Fatalf("Unexpeced spend found in pool: %v", spend)
}
}
// Create a chain of transactions rooted with the first spendable
// output provided by the harness.
const txChainLength = 5
chainedTxns, err := harness.CreateTxChain(outputs[0], txChainLength)
if err != nil {
t.Fatalf("unable to create transaction chain: %v", err)
}
for _, tx := range chainedTxns {
_, err := harness.txPool.ProcessTransaction(tx, true,
false, 0)
if err != nil {
t.Fatalf("ProcessTransaction: failed to accept "+
"tx: %v", err)
}
}
// The first tx in the chain should be the spend of the spendable
// output.
op := outputs[0].outPoint
spend := harness.txPool.CheckSpend(op)
if spend != chainedTxns[0] {
t.Fatalf("expected %v to be spent by %v, instead "+
"got %v", op, chainedTxns[0], spend)
}
// Now all but the last tx should be spent by the next.
for i := 0; i < len(chainedTxns)-1; i++ {
op = wire.OutPoint{
Hash: *chainedTxns[i].Hash(),
Index: 0,
}
expSpend := chainedTxns[i+1]
spend = harness.txPool.CheckSpend(op)
if spend != expSpend {
t.Fatalf("expected %v to be spent by %v, instead "+
"got %v", op, expSpend, spend)
}
}
// The last tx should have no spend.
op = wire.OutPoint{
Hash: *chainedTxns[txChainLength-1].Hash(),
Index: 0,
}
spend = harness.txPool.CheckSpend(op)
if spend != nil {
t.Fatalf("Unexpeced spend found in pool: %v", spend)
}
}