LBRYCommunity/lbcwallet
1
0
Fork 0
Code Issues 5 Pull requests Packages Projects Releases 14 Wiki Activity

Another day, another tx store implementation.

Browse source 
The last transaction store was a great example of how not to write
scalable software.  For a variety of reasons, it was very slow at
processing transaction inserts.  Among them:

1) Every single transaction record being saved in a linked list
   (container/list), and inserting into this list would be an O(n)
   operation so that records could be ordered by receive date.

2) Every single transaction in the above mentioned list was iterated
   over in order to find double spends which must be removed.  It is
   silly to do this check for mined transactions, which already have
   been checked for this by btcd.  Worse yet, if double spends were
   found, the list would be iterated a second (or third, or fourth)
   time for each removed transaction.

3) All spend tracking for signed-by-wallet transactions was found on
   each transaction insert, even if the now spent previous transaction
   outputs were known by the caller.

This list could keep going on, but you get the idea.  It was bad.

To resolve these issues a new transaction store had to be implemented.
The new implementation:

1) Tracks mined and unmined transactions in different data structures.
   Mined transactions are cheap to track because the required double
   spend checks have already been performed by the chain server, and
   double spend checks are only required to be performed on
   newly-inserted mined transactions which may conflict with previous
   unmined transactions.

2) Saves mined transactions grouped by block first, and then by their
   transaction index.  Lookup keys for mined transactions are simply
   the block height (in the best chain, that's all we save) and index
   of the transaction in the block.  This makes looking up any
   arbitrary transaction almost an O(1) operation (almost, because
   block height and block indexes are mapped to their slice indexes
   with a Go map).

3) Saves records in each transaction for whether the outputs are
   wallet credits (spendable by wallet) and for whether inputs debit
   from previous credits.  Both structures point back to the source
   or spender (credits point to the transaction that spends them, or
   nil for unspent credits, and debits include keys to lookup the
   transaction credits they spent.  While complicated to keep track
   of, this greatly simplifies the spent tracking for transactions
   across rollbacks and transaction removals.

4) Implements double spend checking as an almost O(1) operation.  A
   Go map is used to map each previous outpoint for all unconfirmed
   transactions to the unconfirmed tx record itself.  Checking for
   double spends on confirmed transaction inserts only involves
   looking up each previous outpoint of the inserted tx in this map.
   If a double spend is found, removal is simplified by only
   removing the transaction and its spend chain from store maps,
   rather than iterating a linked list several times over to remove
   each dead transaction in the spend chain.

5) Allows the caller to specify the previous credits which are spent
   by a debiting transaction.  When a transaction is created by
   wallet, the previous outputs are already known, and by passing
   their record types to the AddDebits method, lookups for each
   previously unspent credit are omitted.

6) Bookkeeps all blocks with transactions with unspent credits, and
   bookkeeps the transaction indexes of all transactions with unspent
   outputs for a single block.  For the case where the caller adding a
   debit record does not know what credits a transaction debits from,
   these bookkeeping structures allow the store to only consider known
   unspent transactions, rather than searching through both spent and
   unspents.

7) Saves amount deltas for the entire balance as a result of each
   block, due to transactions within that block.  This improves the
   performance of calculating the full balance by not needing to
   iterate over every transaction, and then every credit, to determine
   if a credit is spent or unspent.  When transactions are moved from
   unconfirmed to a block structure, the amount deltas are incremented
   by the amount of all transaction credits (both spent and unspent)
   and debited by the total amount the transaction spends from
   previous wallet credits.  For the common case of calculating a
   balance with just one confirmation, the only involves iterating
   over each block structure and adding the (possibly negative)
   amount delta.  Coinbase rewards are saved similarly, but with a
   different amount variable so they can be seperatly included or
   excluded.

Due to all of the changes in how the store internally works, the
serialization format has changed.  To simplify the serialization
logic, support for reading the last store file version has been
removed.  Past this change, a rescan (run automatically) will be
required to rebuild the transaction history.
This commit is contained in:
Josh Rickmar 2014-05-05 16:12:05 -05:00
parent e956d0b290
commit e9bdf2a094
10 changed files with 2997 additions and 1546 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

182
account.go
View file

@ -79,21 +79,19 @@ func (a *Account) AddressUsed(addr btcutil.Address) bool {
pkHash := addr.ScriptAddress()
for _, record := range a.TxStore.SortedRecords() {
txout, ok := record.(*tx.RecvTxOut)
if !ok {
continue
}
for _, r := range a.TxStore.Records() {
credits := r.Credits()
for _, c := range credits {
// Extract addresses from this output's pkScript.
_, addrs, _, err := c.Addresses(cfg.Net())
if err != nil {
continue
}
// Extract addresses from this output's pkScript.
_, addrs, _, err := txout.Addresses(cfg.Net())
if err != nil {
continue
}
for _, a := range addrs {
if bytes.Equal(a.ScriptAddress(), pkHash) {
return true
for _, a := range addrs {
if bytes.Equal(a.ScriptAddress(), pkHash) {
return true
}
}
}
}
@ -115,8 +113,12 @@ func (a *Account) CalculateBalance(confirms int) float64 {
return 0.
}
bal := a.TxStore.Balance(confirms, bs.Height)
return float64(bal) / float64(btcutil.SatoshiPerBitcoin)
bal, err := a.TxStore.Balance(confirms, bs.Height)
if err != nil {
log.Errorf("Cannot calculate balance: %v", err)
return 0
}
return bal.ToUnit(btcutil.AmountBTC)
}
// CalculateAddressBalance sums the amounts of all unspent transaction
@ -134,23 +136,25 @@ func (a *Account) CalculateAddressBalance(addr btcutil.Address, confirms int) fl
return 0.
}
var bal int64 // Measured in satoshi
for _, txout := range a.TxStore.UnspentOutputs() {
// Utxos not yet in blocks (height -1) should only be
// added if confirmations is 0.
if confirmed(confirms, txout.Height(), bs.Height) {
var bal btcutil.Amount
unspent, err := a.TxStore.UnspentOutputs()
if err != nil {
return 0.
}
for _, credit := range unspent {
if confirmed(confirms, credit.BlockHeight, bs.Height) {
// We only care about the case where len(addrs) == 1, and err
// will never be non-nil in that case
_, addrs, _, _ := txout.Addresses(cfg.Net())
_, addrs, _, _ := credit.Addresses(cfg.Net())
if len(addrs) != 1 {
continue
}
if addrs[0].EncodeAddress() == addr.EncodeAddress() {
bal += txout.Value()
bal += credit.Amount()
}
}
}
return float64(bal) / float64(btcutil.SatoshiPerBitcoin)
return bal.ToUnit(btcutil.AmountBTC)
}
// CurrentAddress gets the most recently requested Bitcoin payment address
@ -171,23 +175,28 @@ func (a *Account) CurrentAddress() (btcutil.Address, error) {
// ListSinceBlock returns a slice of objects with details about transactions
// since the given block. If the block is -1 then all transactions are included.
// This is intended to be used for listsinceblock RPC replies.
func (a *Account) ListSinceBlock(since, curBlockHeight int32, minconf int) ([]btcjson.ListTransactionsResult, error) {
func (a *Account) ListSinceBlock(since, curBlockHeight int32,
minconf int) ([]btcjson.ListTransactionsResult, error) {
var txList []btcjson.ListTransactionsResult
for _, txRecord := range a.TxStore.SortedRecords() {
for _, txRecord := range a.TxStore.Records() {
// Transaction records must only be considered if they occur
// after the block height since.
if since != -1 && txRecord.Height() <= since {
if since != -1 && txRecord.BlockHeight <= since {
continue
}
// Transactions that have not met minconf confirmations are to
// be ignored.
if !confirmed(minconf, txRecord.Height(), curBlockHeight) {
if !confirmed(minconf, txRecord.BlockHeight, curBlockHeight) {
continue
}
txList = append(txList,
txRecord.TxInfo(a.name, curBlockHeight, a.Net())...)
jsonResults, err := txRecord.ToJSON(a.name, curBlockHeight, a.Net())
if err != nil {
return nil, err
}
txList = append(txList, jsonResults...)
}
return txList, nil
@ -206,12 +215,15 @@ func (a *Account) ListTransactions(from, count int) ([]btcjson.ListTransactionsR
var txList []btcjson.ListTransactionsResult
records := a.TxStore.SortedRecords()
records := a.TxStore.Records()
lastLookupIdx := len(records) - count
// Search in reverse order: lookup most recently-added first.
for i := len(records) - 1; i >= from && i >= lastLookupIdx; i-- {
txList = append(txList,
records[i].TxInfo(a.name, bs.Height, a.Net())...)
jsonResults, err := records[i].ToJSON(a.name, bs.Height, a.Net())
if err != nil {
return nil, err
}
txList = append(txList, jsonResults...)
}
return txList, nil
@ -231,25 +243,27 @@ func (a *Account) ListAddressTransactions(pkHashes map[string]struct{}) (
}
var txList []btcjson.ListTransactionsResult
for _, txRecord := range a.TxStore.SortedRecords() {
txout, ok := txRecord.(*tx.RecvTxOut)
if !ok {
continue
}
// We only care about the case where len(addrs) == 1, and err
// will never be non-nil in that case
_, addrs, _, _ := txout.Addresses(cfg.Net())
if len(addrs) != 1 {
continue
}
apkh, ok := addrs[0].(*btcutil.AddressPubKeyHash)
if !ok {
continue
}
for _, r := range a.TxStore.Records() {
for _, c := range r.Credits() {
// We only care about the case where len(addrs) == 1,
// and err will never be non-nil in that case.
_, addrs, _, _ := c.Addresses(cfg.Net())
if len(addrs) != 1 {
continue
}
apkh, ok := addrs[0].(*btcutil.AddressPubKeyHash)
if !ok {
continue
}
if _, ok := pkHashes[string(apkh.ScriptAddress())]; ok {
info := txout.TxInfo(a.name, bs.Height, a.Net())
txList = append(txList, info...)
if _, ok := pkHashes[string(apkh.ScriptAddress())]; !ok {
continue
}
jsonResult, err := c.ToJSON(a.name, bs.Height, a.Net())
if err != nil {
return nil, err
}
txList = append(txList, jsonResult)
}
}
@ -268,11 +282,14 @@ func (a *Account) ListAllTransactions() ([]btcjson.ListTransactionsResult, error
}
// Search in reverse order: lookup most recently-added first.
records := a.TxStore.SortedRecords()
records := a.TxStore.Records()
var txList []btcjson.ListTransactionsResult
for i := len(records) - 1; i >= 0; i-- {
info := records[i].TxInfo(a.name, bs.Height, a.Net())
txList = append(txList, info...)
jsonResults, err := records[i].ToJSON(a.name, bs.Height, a.Net())
if err != nil {
return nil, err
}
txList = append(txList, jsonResults...)
}
return txList, nil
@ -429,12 +446,16 @@ func (a *Account) Track() {
i++
}
err := NotifyReceived(CurrentServerConn(), addrstrs)
if err != nil {
jsonErr := NotifyReceived(CurrentServerConn(), addrstrs)
if jsonErr != nil {
log.Error("Unable to request transaction updates for address.")
}
for _, txout := range a.TxStore.UnspentOutputs() {
unspent, err := a.TxStore.UnspentOutputs()
if err != nil {
log.Errorf("Unable to access unspent outputs: %v", err)
}
for _, txout := range unspent {
ReqSpentUtxoNtfn(txout)
}
}
@ -443,7 +464,7 @@ func (a *Account) Track() {
// account. This is needed for catching btcwallet up to a long-running
// btcd process, as otherwise it would have missed notifications as
// blocks are attached to the main chain.
func (a *Account) RescanActiveJob() *RescanJob {
func (a *Account) RescanActiveJob() (*RescanJob, error) {
// Determine the block necesary to start the rescan for all active
// addresses.
height := int32(0)
@ -463,21 +484,25 @@ func (a *Account) RescanActiveJob() *RescanJob {
addrs = append(addrs, actives[i].Address())
}
unspents := a.TxStore.UnspentOutputs()
unspents, err := a.TxStore.UnspentOutputs()
if err != nil {
return nil, err
}
outpoints := make([]*btcwire.OutPoint, 0, len(unspents))
for i := range unspents {
outpoints = append(outpoints, unspents[i].OutPoint())
for _, c := range unspents {
outpoints = append(outpoints, c.OutPoint())
}
return &RescanJob{
job := &RescanJob{
Addresses: map[*Account][]btcutil.Address{a: addrs},
OutPoints: outpoints,
StartHeight: height,
}
return job, nil
}
func (a *Account) ResendUnminedTxs() {
txs := a.TxStore.UnminedSignedTxs()
txs := a.TxStore.UnminedDebitTxs()
txbuf := new(bytes.Buffer)
for _, tx_ := range txs {
tx_.MsgTx().Serialize(txbuf)
@ -628,8 +653,8 @@ func (a *Account) ReqNewTxsForAddress(addr btcutil.Address) {
// ReqSpentUtxoNtfn sends a message to btcd to request updates for when
// a stored UTXO has been spent.
func ReqSpentUtxoNtfn(t *tx.RecvTxOut) {
op := t.OutPoint()
func ReqSpentUtxoNtfn(c *tx.Credit) {
op := c.OutPoint()
log.Debugf("Requesting spent UTXO notifications for Outpoint hash %s index %d",
op.Hash, op.Index)
@ -645,25 +670,22 @@ func (a *Account) TotalReceived(confirms int) (float64, error) {
return 0, err
}
var totalSatoshis int64
for _, record := range a.TxStore.SortedRecords() {
txout, ok := record.(*tx.RecvTxOut)
if !ok {
continue
}
var amount btcutil.Amount
for _, r := range a.TxStore.Records() {
for _, c := range r.Credits() {
// Ignore change.
if c.Change() {
continue
}
// Ignore change.
if txout.Change() {
continue
}
// Tally if the appropiate number of block confirmations have passed.
if confirmed(confirms, txout.Height(), bs.Height) {
totalSatoshis += txout.Value()
// Tally if the appropiate number of block confirmations have passed.
if confirmed(confirms, c.BlockHeight, bs.Height) {
amount += c.Amount()
}
}
}
return float64(totalSatoshis) / float64(btcutil.SatoshiPerBitcoin), nil
return amount.ToUnit(btcutil.AmountBTC), nil
}
// confirmed checks whether a transaction at height txHeight has met

70
acctmgr.go
View file

@ -27,7 +27,6 @@ import (
"github.com/conformal/btcwire"
"os"
"strings"
"time"
)
// Errors relating to accounts.
@ -563,22 +562,22 @@ func (am *AccountManager) BlockNotify(bs *wallet.BlockStamp) {
// the transaction IDs match, the record in the TxStore is updated with
// the full information about the newly-mined tx, and the TxStore is
// scheduled to be written to disk..
func (am *AccountManager) RecordSpendingTx(tx_ *btcutil.Tx, block *tx.BlockDetails) {
now := time.Now()
var created time.Time
if block != nil && now.After(block.Time) {
created = block.Time
} else {
created = now
}
func (am *AccountManager) RecordSpendingTx(tx_ *btcutil.Tx, block *tx.Block) error {
for _, a := range am.AllAccounts() {
// TODO(jrick): This needs to iterate through each txout's
// addresses and find whether this account's keystore contains
// any of the addresses this tx sends to.
a.TxStore.InsertSignedTx(tx_, created, block)
txr, err := a.TxStore.InsertTx(tx_, block)
if err != nil {
return err
}
// When received as a notification, we don't know what the inputs are.
if _, err := txr.AddDebits(nil); err != nil {
return err
}
am.ds.ScheduleTxStoreWrite(a)
}
return nil
}
// CalculateBalance returns the balance, calculated using minconf block
@ -744,7 +743,9 @@ func (am *AccountManager) ListAccounts(minconf int) map[string]float64 {
// ListSinceBlock returns a slice of objects representing all transactions in
// the wallets since the given block.
// To be used for the listsinceblock command.
func (am *AccountManager) ListSinceBlock(since, curBlockHeight int32, minconf int) ([]btcjson.ListTransactionsResult, error) {
func (am *AccountManager) ListSinceBlock(since, curBlockHeight int32,
minconf int) ([]btcjson.ListTransactionsResult, error) {
// Create and fill a map of account names and their balances.
var txList []btcjson.ListTransactionsResult
for _, a := range am.AllAccounts() {
@ -761,18 +762,18 @@ func (am *AccountManager) ListSinceBlock(since, curBlockHeight int32, minconf in
// GetTransaction.
type accountTx struct {
Account string
Tx tx.Record
Tx *tx.TxRecord
}
// GetTransaction returns an array of accountTx to fully represent the effect of
// a transaction on locally known wallets. If we know nothing about a
// transaction an empty array will be returned.
func (am *AccountManager) GetTransaction(txsha *btcwire.ShaHash) []accountTx {
func (am *AccountManager) GetTransaction(txSha *btcwire.ShaHash) []accountTx {
accumulatedTxen := []accountTx{}
for _, a := range am.AllAccounts() {
for _, record := range a.TxStore.SortedRecords() {
if *record.TxSha() != *txsha {
for _, record := range a.TxStore.Records() {
if *record.Tx().Sha() != *txSha {
continue
}
@ -794,6 +795,7 @@ func (am *AccountManager) GetTransaction(txsha *btcwire.ShaHash) []accountTx {
// transaction an empty array will be returned.
func (am *AccountManager) ListUnspent(minconf, maxconf int,
addresses map[string]bool) ([]*btcjson.ListUnSpentResult, error) {
bs, err := GetCurBlock()
if err != nil {
return nil, err
@ -803,14 +805,18 @@ func (am *AccountManager) ListUnspent(minconf, maxconf int,
var results []*btcjson.ListUnSpentResult
for _, a := range am.AllAccounts() {
for _, rtx := range a.TxStore.UnspentOutputs() {
confs := confirms(rtx.Height(), bs.Height)
unspent, err := a.TxStore.UnspentOutputs()
if err != nil {
return nil, err
}
for _, credit := range unspent {
confs := confirms(credit.BlockHeight, bs.Height)
switch {
case int(confs) < minconf, int(confs) > maxconf:
continue
}
_, addrs, _, _ := rtx.Addresses(cfg.Net())
_, addrs, _, _ := credit.Addresses(cfg.Net())
if filter {
for _, addr := range addrs {
_, ok := addresses[addr.EncodeAddress()]
@ -821,13 +827,12 @@ func (am *AccountManager) ListUnspent(minconf, maxconf int,
continue
}
include:
outpoint := rtx.OutPoint()
result := &btcjson.ListUnSpentResult{
TxId: outpoint.Hash.String(),
Vout: float64(outpoint.Index),
TxId: credit.Tx().Sha().String(),
Vout: float64(credit.OutputIndex),
Account: a.Name(),
ScriptPubKey: hex.EncodeToString(rtx.PkScript()),
Amount: float64(rtx.Value()) / 1e8,
ScriptPubKey: hex.EncodeToString(credit.TxOut().PkScript),
Amount: credit.Amount().ToUnit(btcutil.AmountBTC),
Confirmations: float64(confs),
}
@ -847,23 +852,26 @@ func (am *AccountManager) ListUnspent(minconf, maxconf int,
// RescanActiveAddresses begins a rescan for all active addresses for
// each account.
func (am *AccountManager) RescanActiveAddresses() {
func (am *AccountManager) RescanActiveAddresses() error {
var job *RescanJob
for _, a := range am.AllAccounts() {
acctJob := a.RescanActiveJob()
acctJob, err := a.RescanActiveJob()
if err != nil {
return err
}
if job == nil {
job = acctJob
} else {
job.Merge(acctJob)
}
}
if job == nil {
return
if job != nil {
// Submit merged job and block until rescan completes.
jobFinished := am.rm.SubmitJob(job)
<-jobFinished
}
// Submit merged job and block until rescan completes.
jobFinished := am.rm.SubmitJob(job)
<-jobFinished
return nil
}
func (am *AccountManager) ResendUnminedTxs() {

69
createtx.go
View file

@ -56,27 +56,27 @@ const minTxFee = 10000
// miner. i is measured in satoshis.
var TxFeeIncrement = struct {
sync.Mutex
i int64
i btcutil.Amount
}{
i: minTxFee,
}
type CreatedTx struct {
tx *btcutil.Tx
time time.Time
inputs []*tx.Credit
changeAddr btcutil.Address
}
// ByAmount defines the methods needed to satisify sort.Interface to
// sort a slice of Utxos by their amount.
type ByAmount []*tx.RecvTxOut
type ByAmount []*tx.Credit
func (u ByAmount) Len() int {
return len(u)
}
func (u ByAmount) Less(i, j int) bool {
return u[i].Value() < u[j].Value()
return u[i].Amount() < u[j].Amount()
}
func (u ByAmount) Swap(i, j int) {
@ -89,8 +89,8 @@ func (u ByAmount) Swap(i, j int) {
// is the total number of satoshis which would be spent by the combination
// of all selected previous outputs. err will equal ErrInsufficientFunds if there
// are not enough unspent outputs to spend amt.
func selectInputs(utxos []*tx.RecvTxOut, amt int64,
minconf int) (selected []*tx.RecvTxOut, btcout int64, err error) {
func selectInputs(utxos []*tx.Credit, amt btcutil.Amount,
minconf int) (selected []*tx.Credit, out btcutil.Amount, err error) {
bs, err := GetCurBlock()
if err != nil {
@ -100,13 +100,13 @@ func selectInputs(utxos []*tx.RecvTxOut, amt int64,
// Create list of eligible unspent previous outputs to use as tx
// inputs, and sort by the amount in reverse order so a minimum number
// of inputs is needed.
eligible := make([]*tx.RecvTxOut, 0, len(utxos))
eligible := make([]*tx.Credit, 0, len(utxos))
for _, utxo := range utxos {
if confirmed(minconf, utxo.Height(), bs.Height) {
if confirmed(minconf, utxo.BlockHeight, bs.Height) {
// Coinbase transactions must have have reached maturity
// before their outputs may be spent.
if utxo.IsCoinbase() {
confs := confirms(utxo.Height(), bs.Height)
confs := confirms(utxo.BlockHeight, bs.Height)
if confs < btcchain.CoinbaseMaturity {
continue
}
@ -117,20 +117,20 @@ func selectInputs(utxos []*tx.RecvTxOut, amt int64,
sort.Sort(sort.Reverse(ByAmount(eligible)))
// Iterate throguh eligible transactions, appending to outputs and
// increasing btcout. This is finished when btcout is greater than the
// increasing out. This is finished when out is greater than the
// requested amt to spend.
for _, e := range eligible {
selected = append(selected, e)
btcout += e.Value()
if btcout >= amt {
return selected, btcout, nil
out += e.Amount()
if out >= amt {
return selected, out, nil
}
}
if btcout < amt {
if out < amt {
return nil, 0, ErrInsufficientFunds
}
return selected, btcout, nil
return selected, out, nil
}
// txToPairs creates a raw transaction sending the amounts for each
@ -142,7 +142,9 @@ func selectInputs(utxos []*tx.RecvTxOut, amt int64,
// address, changeUtxo will point to a unconfirmed (height = -1, zeroed
// block hash) Utxo. ErrInsufficientFunds is returned if there are not
// enough eligible unspent outputs to create the transaction.
func (a *Account) txToPairs(pairs map[string]int64, minconf int) (*CreatedTx, error) {
func (a *Account) txToPairs(pairs map[string]btcutil.Amount,
minconf int) (*CreatedTx, error) {
// Wallet must be unlocked to compose transaction.
if a.IsLocked() {
return nil, wallet.ErrWalletLocked
@ -152,7 +154,7 @@ func (a *Account) txToPairs(pairs map[string]int64, minconf int) (*CreatedTx, er
msgtx := btcwire.NewMsgTx()
// Calculate minimum amount needed for inputs.
var amt int64
var amt btcutil.Amount
for _, v := range pairs {
// Error out if any amount is negative.
if v <= 0 {
@ -188,21 +190,25 @@ func (a *Account) txToPairs(pairs map[string]int64, minconf int) (*CreatedTx, er
// a higher fee if not enough was originally chosen.
txNoInputs := msgtx.Copy()
var selectedInputs []*tx.RecvTxOut
unspent, err := a.TxStore.UnspentOutputs()
if err != nil {
return nil, err
}
var selectedInputs []*tx.Credit
// These are nil/zeroed until a change address is needed, and reused
// again in case a change utxo has already been chosen.
var changeAddr btcutil.Address
// Get the number of satoshis to increment fee by when searching for
// the minimum tx fee needed.
fee := int64(0)
fee := btcutil.Amount(0)
for {
msgtx = txNoInputs.Copy()
// Select unspent outputs to be used in transaction based on the amount
// neededing to sent, and the current fee estimation.
inputs, btcin, err := selectInputs(a.TxStore.UnspentOutputs(),
amt+fee, minconf)
inputs, btcin, err := selectInputs(unspent, amt+fee, minconf)
if err != nil {
return nil, err
}
@ -262,7 +268,7 @@ func (a *Account) txToPairs(pairs map[string]int64, minconf int) (*CreatedTx, er
}
sigscript, err := btcscript.SignatureScript(msgtx, i,
input.PkScript(), btcscript.SigHashAll, privkey,
input.TxOut().PkScript, btcscript.SigHashAll, privkey,
ai.Compressed())
if err != nil {
return nil, fmt.Errorf("cannot create sigscript: %s", err)
@ -290,7 +296,7 @@ func (a *Account) txToPairs(pairs map[string]int64, minconf int) (*CreatedTx, er
}
for i, txin := range msgtx.TxIn {
engine, err := btcscript.NewScript(txin.SignatureScript,
selectedInputs[i].PkScript(), i, msgtx, flags)
selectedInputs[i].TxOut().PkScript, i, msgtx, flags)
if err != nil {
return nil, fmt.Errorf("cannot create script engine: %s", err)
}
@ -304,7 +310,7 @@ func (a *Account) txToPairs(pairs map[string]int64, minconf int) (*CreatedTx, er
msgtx.BtcEncode(buf, btcwire.ProtocolVersion)
info := &CreatedTx{
tx: btcutil.NewTx(msgtx),
time: time.Now(),
inputs: selectedInputs,
changeAddr: changeAddr,
}
return info, nil
@ -316,12 +322,12 @@ func (a *Account) txToPairs(pairs map[string]int64, minconf int) (*CreatedTx, er
// and none of the outputs contain a value less than 1 bitcent.
// Otherwise, the fee will be calculated using TxFeeIncrement,
// incrementing the fee for each kilobyte of transaction.
func minimumFee(tx *btcwire.MsgTx, allowFree bool) int64 {
func minimumFee(tx *btcwire.MsgTx, allowFree bool) btcutil.Amount {
txLen := tx.SerializeSize()
TxFeeIncrement.Lock()
incr := TxFeeIncrement.i
TxFeeIncrement.Unlock()
fee := int64(1+txLen/1000) * incr
fee := btcutil.Amount(int64(1+txLen/1000) * int64(incr))
if allowFree && txLen < 1000 {
fee = 0
@ -335,8 +341,9 @@ func minimumFee(tx *btcwire.MsgTx, allowFree bool) int64 {
}
}
if fee < 0 || fee > btcutil.MaxSatoshi {
fee = btcutil.MaxSatoshi
max := btcutil.Amount(btcutil.MaxSatoshi)
if fee < 0 || fee > max {
fee = max
}
return fee
@ -345,14 +352,14 @@ func minimumFee(tx *btcwire.MsgTx, allowFree bool) int64 {
// allowFree calculates the transaction priority and checks that the
// priority reaches a certain threshhold. If the threshhold is
// reached, a free transaction fee is allowed.
func allowFree(curHeight int32, txouts []*tx.RecvTxOut, txSize int) bool {
func allowFree(curHeight int32, txouts []*tx.Credit, txSize int) bool {
const blocksPerDayEstimate = 144
const txSizeEstimate = 250
var weightedSum int64
for _, txout := range txouts {
depth := chainDepth(txout.Height(), curHeight)
weightedSum += txout.Value() * int64(depth)
depth := chainDepth(txout.BlockHeight, curHeight)
weightedSum += int64(txout.Amount()) * int64(depth)
}
priority := float64(weightedSum) / float64(txSize)
return priority > float64(btcutil.SatoshiPerBitcoin)*blocksPerDayEstimate/txSizeEstimate

59
ntfns.go
View file

@ -33,20 +33,20 @@ import (
"github.com/conformal/btcws"
)
func parseBlock(block *btcws.BlockDetails) (*tx.BlockDetails, error) {
func parseBlock(block *btcws.BlockDetails) (*tx.Block, int, error) {
if block == nil {
return nil, nil
return nil, btcutil.TxIndexUnknown, nil
}
blksha, err := btcwire.NewShaHashFromStr(block.Hash)
if err != nil {
return nil, err
return nil, btcutil.TxIndexUnknown, err
}
return &tx.BlockDetails{
b := &tx.Block{
Height: block.Height,
Hash: *blksha,
Index: int32(block.Index),
Time: time.Unix(block.Time, 0),
}, nil
}
return b, block.Index, nil
}
type notificationHandler func(btcjson.Cmd) error
@ -80,13 +80,11 @@ func NtfnRecvTx(n btcjson.Cmd) error {
return fmt.Errorf("%v handler: bad transaction bytes: %v", n.Method(), err)
}
var block *tx.BlockDetails
if rtx.Block != nil {
block, err = parseBlock(rtx.Block)
if err != nil {
return fmt.Errorf("%v handler: bad block: %v", n.Method(), err)
}
block, txIdx, err := parseBlock(rtx.Block)
if err != nil {
return fmt.Errorf("%v handler: bad block: %v", n.Method(), err)
}
tx_.SetIndex(txIdx)
// For transactions originating from this wallet, the sent tx history should
// be recorded before the received history. If wallet created this tx, wait
@ -106,14 +104,6 @@ func NtfnRecvTx(n btcjson.Cmd) error {
SendTxHistSyncChans.remove <- *tx_.Sha()
}
now := time.Now()
var received time.Time
if block != nil && now.After(block.Time) {
received = block.Time
} else {
received = now
}
// For every output, find all accounts handling that output address (if any)
// and record the received txout.
for outIdx, txout := range tx_.MsgTx().TxOut {
@ -128,7 +118,11 @@ func NtfnRecvTx(n btcjson.Cmd) error {
}
for _, a := range accounts {
record, err := a.TxStore.InsertRecvTxOut(tx_, uint32(outIdx), false, received, block)
txr, err := a.TxStore.InsertTx(tx_, block)
if err != nil {
return err
}
cred, err := txr.AddCredit(uint32(outIdx), false)
if err != nil {
return err
}
@ -139,21 +133,23 @@ func NtfnRecvTx(n btcjson.Cmd) error {
// has already been notified and is now in a block, a txmined notifiction
// should be sent once to let frontends that all previous send/recvs
// for this unconfirmed tx are now confirmed.
recvTxOP := btcwire.NewOutPoint(tx_.Sha(), uint32(outIdx))
op := *cred.OutPoint()
previouslyNotifiedReq := NotifiedRecvTxRequest{
op: *recvTxOP,
op: op,
response: make(chan NotifiedRecvTxResponse),
}
NotifiedRecvTxChans.access <- previouslyNotifiedReq
if <-previouslyNotifiedReq.response {
NotifiedRecvTxChans.remove <- *recvTxOP
NotifiedRecvTxChans.remove <- op
} else {
// Notify frontends of new recv tx and mark as notified.
NotifiedRecvTxChans.add <- *recvTxOP
NotifiedRecvTxChans.add <- op
// need access to the RecvTxOut to get the json info object
NotifyNewTxDetails(allClients, a.Name(),
record.TxInfo(a.Name(), bs.Height, a.Wallet.Net())[0])
ltr, err := cred.ToJSON(a.Name(), bs.Height, a.Wallet.Net())
if err != nil {
return err
}
NotifyNewTxDetails(allClients, a.Name(), ltr)
}
// Notify frontends of new account balance.
@ -255,13 +251,12 @@ func NtfnRedeemingTx(n btcjson.Cmd) error {
return fmt.Errorf("%v handler: bad transaction bytes: %v", n.Method(), err)
}
block, err := parseBlock(cn.Block)
block, txIdx, err := parseBlock(cn.Block)
if err != nil {
return fmt.Errorf("%v handler: bad block: %v", n.Method(), err)
}
AcctMgr.RecordSpendingTx(tx_, block)
return nil
tx_.SetIndex(txIdx)
return AcctMgr.RecordSpendingTx(tx_, block)
}
// NtfnRescanProgress handles btcd rescanprogress notifications resulting

108
rpcserver.go
View file

@ -223,7 +223,7 @@ func WalletRequestProcessor() {
err := f(n)
AcctMgr.Release()
switch err {
case tx.ErrInconsistantStore:
case tx.ErrInconsistentStore:
// Assume this is a broken btcd reordered
// notifications. Restart the connection
// to reload accounts files from their last
@ -949,9 +949,9 @@ func GetTransaction(icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
return nil, &btcjson.ErrNoTxInfo
}
var sr *tx.SignedTx
var srAccount string
var amountReceived int64
received := btcutil.Amount(0)
var debitTx *tx.TxRecord
var debitAccount string
ret := btcjson.GetTransactionResult{
Details: []btcjson.GetTransactionDetailsResult{},
@ -959,19 +959,20 @@ func GetTransaction(icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
}
details := []btcjson.GetTransactionDetailsResult{}
for _, e := range accumulatedTxen {
switch record := e.Tx.(type) {
case *tx.RecvTxOut:
if record.Change() {
for _, cred := range e.Tx.Credits() {
// Change is ignored.
if cred.Change() {
continue
}
amountReceived += record.Value()
_, addrs, _, _ := record.Addresses(cfg.Net())
received += cred.Amount()
var addr string
_, addrs, _, _ := cred.Addresses(cfg.Net())
if len(addrs) == 1 {
addr = addrs[0].EncodeAddress()
}
details = append(details, btcjson.GetTransactionDetailsResult{
Account: e.Account,
// TODO(oga) We don't mine for now so there
@ -980,28 +981,31 @@ func GetTransaction(icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
// specially with the category depending on
// whether it is an orphan or in the blockchain.
Category: "receive",
Amount: float64(record.Value()) / float64(btcutil.SatoshiPerBitcoin),
Amount: cred.Amount().ToUnit(btcutil.AmountBTC),
Address: addr,
})
case *tx.SignedTx:
// there should only be a single SignedTx record, if any.
// If found, it will be added to the beginning.
sr = record
srAccount = e.Account
}
if e.Tx.Debits() != nil {
// There should only be a single debits record for any
// of the account's transaction records.
debitTx = e.Tx
debitAccount = e.Account
}
}
totalAmount := amountReceived
if sr != nil {
totalAmount -= sr.TotalSent()
totalAmount := received
if debitTx != nil {
debits := debitTx.Debits()
totalAmount -= debits.InputAmount()
info := btcjson.GetTransactionDetailsResult{
Account: srAccount,
Account: debitAccount,
Category: "send",
// negative since it is a send
Amount: float64(-(sr.TotalSent() - amountReceived)) / float64(btcutil.SatoshiPerBitcoin),
Fee: float64(sr.Fee()) / float64(btcutil.SatoshiPerBitcoin),
Amount: (-debits.OutputAmount(true)).ToUnit(btcutil.AmountBTC),
Fee: debits.Fee().ToUnit(btcutil.AmountBTC),
}
_, addrs, _, _ := btcscript.ExtractPkScriptAddrs(sr.Tx().MsgTx().TxOut[0].PkScript, cfg.Net())
_, addrs, _, _ := debitTx.Credits()[0].Addresses(cfg.Net())
if len(addrs) == 1 {
info.Address = addrs[0].EncodeAddress()
}
@ -1012,10 +1016,11 @@ func GetTransaction(icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
}
ret.Details = append(ret.Details, details...)
ret.Amount = totalAmount.ToUnit(btcutil.AmountBTC)
// Generic information should be the same, so just use the first one.
first := accumulatedTxen[0]
ret.Amount = float64(totalAmount) / float64(btcutil.SatoshiPerBitcoin)
ret.TxID = first.Tx.TxSha().String()
ret.TxID = first.Tx.Tx().Sha().String()
buf := bytes.NewBuffer(nil)
buf.Grow(first.Tx.Tx().MsgTx().SerializeSize())
@ -1032,12 +1037,16 @@ func GetTransaction(icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
// timereceived depending on if a transaction was send or
// receive. We ideally should provide the correct numbers for
// both. Right now they will always be the same
ret.Time = first.Tx.Time().Unix()
ret.TimeReceived = first.Tx.Time().Unix()
if details := first.Tx.Block(); details != nil {
ret.BlockIndex = int64(details.Index)
ret.BlockHash = details.Hash.String()
ret.BlockTime = details.Time.Unix()
ret.Time = first.Tx.Received().Unix()
ret.TimeReceived = first.Tx.Received().Unix()
if txr := first.Tx; txr.BlockHeight != -1 {
txBlock, err := txr.Block()
if err != nil {
return nil, &btcjson.Error{
Code: btcjson.ErrWallet.Code,
Message: err.Error(),
}
}
bs, err := GetCurBlock()
if err != nil {
return nil, &btcjson.Error{
@ -1045,7 +1054,10 @@ func GetTransaction(icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
Message: err.Error(),
}
}
ret.Confirmations = int64(bs.Height - details.Height + 1)
ret.BlockIndex = int64(first.Tx.Tx().Index())
ret.BlockHash = txBlock.Hash.String()
ret.BlockTime = txBlock.Time.Unix()
ret.Confirmations = int64(confirms(txr.BlockHeight, bs.Height))
}
// TODO(oga) if the tx is a coinbase we should set "generated" to true.
// Since we do not mine this currently is never the case.
@ -1312,7 +1324,7 @@ func ListUnspent(icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
// sendPairs is a helper routine to reduce duplicated code when creating and
// sending payment transactions.
func sendPairs(icmd btcjson.Cmd, account string, amounts map[string]int64,
func sendPairs(icmd btcjson.Cmd, account string, amounts map[string]btcutil.Amount,
minconf int) (interface{}, *btcjson.Error) {
// Check that the account specified in the request exists.
a, err := AcctMgr.Account(account)
@ -1401,8 +1413,8 @@ func SendFrom(icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
return nil, &e
}
// Create map of address and amount pairs.
pairs := map[string]int64{
cmd.ToAddress: cmd.Amount,
pairs := map[string]btcutil.Amount{
cmd.ToAddress: btcutil.Amount(cmd.Amount),
}
return sendPairs(cmd, cmd.FromAccount, pairs, cmd.MinConf)
@ -1429,7 +1441,13 @@ func SendMany(icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
return nil, &e
}
return sendPairs(cmd, cmd.FromAccount, cmd.Amounts, cmd.MinConf)
// Recreate address/amount pairs, using btcutil.Amount.
pairs := make(map[string]btcutil.Amount, len(cmd.Amounts))
for k, v := range cmd.Amounts {
pairs[k] = btcutil.Amount(v)
}
return sendPairs(cmd, cmd.FromAccount, pairs, cmd.MinConf)
}
// SendToAddress handles a sendtoaddress RPC request by creating a new
@ -1454,8 +1472,8 @@ func SendToAddress(icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
}
// Mock up map of address and amount pairs.
pairs := map[string]int64{
cmd.Address: cmd.Amount,
pairs := map[string]btcutil.Amount{
cmd.Address: btcutil.Amount(cmd.Amount),
}
return sendPairs(cmd, "", pairs, 1)
@ -1518,7 +1536,12 @@ func SendBeforeReceiveHistorySync(add, done, remove chan btcwire.ShaHash,
func handleSendRawTxReply(icmd btcjson.Cmd, txIDStr string, a *Account, txInfo *CreatedTx) (interface{}, *btcjson.Error) {
// Add to transaction store.
stx, err := a.TxStore.InsertSignedTx(txInfo.tx, time.Now(), nil)
txr, err := a.TxStore.InsertTx(txInfo.tx, nil)
if err != nil {
log.Warnf("Error adding sent tx history: %v", err)
return nil, &btcjson.ErrInternal
}
debits, err := txr.AddDebits(txInfo.inputs)
if err != nil {
log.Warnf("Error adding sent tx history: %v", err)
return nil, &btcjson.ErrInternal
@ -1528,7 +1551,12 @@ func handleSendRawTxReply(icmd btcjson.Cmd, txIDStr string, a *Account, txInfo *
// Notify frontends of new SendTx.
bs, err := GetCurBlock()
if err == nil {
for _, details := range stx.TxInfo(a.Name(), bs.Height, a.Net()) {
ltr, err := debits.ToJSON(a.Name(), bs.Height, a.Net())
if err != nil {
log.Warnf("Error adding sent tx history: %v", err)
return nil, &btcjson.ErrInternal
}
for _, details := range ltr {
NotifyNewTxDetails(allClients, a.Name(), details)
}
}
@ -1589,7 +1617,7 @@ func SetTxFee(icmd btcjson.Cmd) (interface{}, *btcjson.Error) {
// Set global tx fee.
TxFeeIncrement.Lock()
TxFeeIncrement.i = cmd.Amount
TxFeeIncrement.i = btcutil.Amount(cmd.Amount)
TxFeeIncrement.Unlock()
// A boolean true result is returned upon success.

16
sockets.go
View file

@ -740,8 +740,8 @@ func Handshake(rpc ServerConn) error {
log.Debugf("Checking for previous saved block with height %v hash %v",
bs.Height, bs.Hash)
_, err := GetBlock(rpc, bs.Hash.String())
if err != nil {
_, jsonErr := GetBlock(rpc, bs.Hash.String())
if jsonErr != nil {
continue
}
@ -763,7 +763,14 @@ func Handshake(rpc ServerConn) error {
// Begin tracking wallets against this btcd instance.
AcctMgr.Track()
AcctMgr.RescanActiveAddresses()
if err := AcctMgr.RescanActiveAddresses(); err != nil {
return err
}
// TODO: Only begin tracking new unspent outputs as a result
// of the rescan. This is also pretty racy, as a new block
// could arrive between rescan and by the time the new outpoint
// is added to btcd's websocket's unspent output set.
AcctMgr.Track()
// (Re)send any unmined transactions to btcd in case of a btcd restart.
AcctMgr.ResendUnminedTxs()
@ -782,6 +789,9 @@ func Handshake(rpc ServerConn) error {
a.fullRescan = true
AcctMgr.Track()
AcctMgr.RescanActiveAddresses()
// TODO: only begin tracking new unspent outputs as a result of the
// rescan. This is also racy (see comment for second Track above).
AcctMgr.Track()
AcctMgr.ResendUnminedTxs()
return nil
}

129
tx/json.go Normal file
View file

@ -0,0 +1,129 @@
/*
* Copyright (c) 2013, 2014 Conformal Systems LLC <info@conformal.com>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
package tx
import (
"github.com/conformal/btcjson"
"github.com/conformal/btcscript"
"github.com/conformal/btcutil"
"github.com/conformal/btcwire"
)
// ToJSON returns a slice of btcjson listtransaction result types for all credits
// and debits of this transaction.
func (t *TxRecord) ToJSON(account string, chainHeight int32,
net btcwire.BitcoinNet) ([]btcjson.ListTransactionsResult, error) {
var results []btcjson.ListTransactionsResult
if d := t.Debits(); d != nil {
r, err := d.ToJSON(account, chainHeight, net)
if err != nil {
return nil, err
}
results = r
}
for _, c := range t.Credits() {
r, err := c.ToJSON(account, chainHeight, net)
if err != nil {
return nil, err
}
results = append(results, r)
}
return results, nil
}
// ToJSON returns a slice of objects that may be marshaled as a JSON array
// of JSON objects for a listtransactions RPC reply.
func (d *Debits) ToJSON(account string, chainHeight int32,
net btcwire.BitcoinNet) ([]btcjson.ListTransactionsResult, error) {
msgTx := d.Tx().MsgTx()
reply := make([]btcjson.ListTransactionsResult, 0, len(msgTx.TxOut))
for _, txOut := range msgTx.TxOut {
address := ""
_, addrs, _, _ := btcscript.ExtractPkScriptAddrs(txOut.PkScript, net)
if len(addrs) == 1 {
address = addrs[0].EncodeAddress()
}
result := btcjson.ListTransactionsResult{
Account: account,
Address: address,
Category: "send",
Amount: btcutil.Amount(-txOut.Value).ToUnit(btcutil.AmountBTC),
Fee: d.Fee().ToUnit(btcutil.AmountBTC),
TxID: d.Tx().Sha().String(),
Time: d.txRecord.received.Unix(),
TimeReceived: d.txRecord.received.Unix(),
WalletConflicts: []string{},
}
if d.BlockHeight != -1 {
b, err := d.s.lookupBlock(d.BlockHeight)
if err != nil {
return nil, err
}
result.BlockHash = b.Hash.String()
result.BlockIndex = int64(d.Tx().Index())
result.BlockTime = b.Time.Unix()
result.Confirmations = int64(confirms(b.Height, chainHeight))
}
reply = append(reply, result)
}
return reply, nil
}
// ToJSON returns a slice of objects that may be marshaled as a JSON array
// of JSON objects for a listtransactions RPC reply.
func (c *Credit) ToJSON(account string, chainHeight int32,
net btcwire.BitcoinNet) (btcjson.ListTransactionsResult, error) {
msgTx := c.Tx().MsgTx()
txout := msgTx.TxOut[c.OutputIndex]
var address string
_, addrs, _, _ := btcscript.ExtractPkScriptAddrs(txout.PkScript, net)
if len(addrs) == 1 {
address = addrs[0].EncodeAddress()
}
result := btcjson.ListTransactionsResult{
Account: account,
Category: "receive",
Address: address,
Amount: btcutil.Amount(txout.Value).ToUnit(btcutil.AmountBTC),
TxID: c.Tx().Sha().String(),
Time: c.received.Unix(),
TimeReceived: c.received.Unix(),
WalletConflicts: []string{},
}
if c.BlockHeight != -1 {
b, err := c.s.lookupBlock(c.BlockHeight)
if err != nil {
return btcjson.ListTransactionsResult{}, err
}
result.BlockHash = b.Hash.String()
result.BlockIndex = int64(c.Tx().Index())
result.BlockTime = b.Time.Unix()
result.Confirmations = int64(confirms(b.Height, chainHeight))
}
return result, nil
}

1121
tx/serialization.go Normal file
View file

File diff suppressed because it is too large Load diff

2421
tx/tx.go
View file

File diff suppressed because it is too large Load diff

368
tx/tx_test.go
View file

@ -1,18 +1,16 @@
/*
* Copyright (c) 2013, 2014 Conformal Systems LLC <info@conformal.com>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
// Copyright (c) 2013, 2014 Conformal Systems LLC <info@conformal.com>
//
// Permission to use, copy, modify, and distribute this software for any
// purpose with or without fee is hereby granted, provided that the above
// copyright notice and this permission notice appear in all copies.
//
// THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
// WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
// MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
// ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
// WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
// ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
// OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
package tx_test
@ -34,27 +32,29 @@ var (
TstRecvTx, _ = btcutil.NewTxFromBytes(TstRecvSerializedTx)
TstRecvTxSpendingTxBlockHash, _ = btcwire.NewShaHashFromStr("00000000000000017188b968a371bab95aa43522665353b646e41865abae02a4")
TstRecvAmt = int64(10000000)
TstRecvTxBlockDetails = &BlockDetails{
TstRecvIndex = 684
TstRecvTxBlockDetails = &Block{
Height: 276425,
Hash: *TstRecvTxSpendingTxBlockHash,
Index: 684,
Time: time.Unix(1387737310, 0),
}
TstRecvCurrentHeight = int32(284498) // mainnet blockchain height at time of writing
TstRecvTxOutConfirms = 8074 // hardcoded number of confirmations given the above block height
TstSpendingTxBlockHeight = int32(279143)
TstSignedTxBlockHash, _ = btcwire.NewShaHashFromStr("00000000000000017188b968a371bab95aa43522665353b646e41865abae02a4")
TstSignedTxBlockDetails = &BlockDetails{
TstSpendingSerializedTx, _ = hex.DecodeString("0100000003ad3fba7ebd67c09baa9538898e10d6726dcb8eadb006be0c7388c8e46d69d361000000006b4830450220702c4fbde5532575fed44f8d6e8c3432a2a9bd8cff2f966c3a79b2245a7c88db02210095d6505a57e350720cb52b89a9b56243c15ddfcea0596aedc1ba55d9fb7d5aa0012103cccb5c48a699d3efcca6dae277fee6b82e0229ed754b742659c3acdfed2651f9ffffffffdbd36173f5610e34de5c00ed092174603761595d90190f790e79cda3e5b45bc2010000006b483045022000fa20735e5875e64d05bed43d81b867f3bd8745008d3ff4331ef1617eac7c44022100ad82261fc57faac67fc482a37b6bf18158da0971e300abf5fe2f9fd39e107f58012102d4e1caf3e022757512c204bf09ff56a9981df483aba3c74bb60d3612077c9206ffffffff65536c9d964b6f89b8ef17e83c6666641bc495cb27bab60052f76cd4556ccd0d040000006a473044022068e3886e0299ffa69a1c3ee40f8b6700f5f6d463a9cf9dbf22c055a131fc4abc02202b58957fe19ff1be7a84c458d08016c53fbddec7184ac5e633f2b282ae3420ae012103b4e411b81d32a69fb81178a8ea1abaa12f613336923ee920ffbb1b313af1f4d2ffffffff02ab233200000000001976a91418808b2fbd8d2c6d022aed5cd61f0ce6c0a4cbb688ac4741f011000000001976a914f081088a300c80ce36b717a9914ab5ec8a7d283988ac00000000")
TstSpendingTx, _ = btcutil.NewTxFromBytes(TstSpendingSerializedTx)
TstSpendingTxBlockHeight = int32(279143)
TstSignedTxBlockHash, _ = btcwire.NewShaHashFromStr("00000000000000017188b968a371bab95aa43522665353b646e41865abae02a4")
TstSignedTxIndex = 123
TstSignedTxBlockDetails = &Block{
Height: TstSpendingTxBlockHeight,
Hash: *TstSignedTxBlockHash,
Index: 123,
Time: time.Unix(1389114091, 0),
}
)
func TestTxStore(t *testing.T) {
func TestInsertsCreditsDebitsRollbacks(t *testing.T) {
// Create a double spend of the received blockchain transaction.
dupRecvTx, _ := btcutil.NewTxFromBytes(TstRecvSerializedTx)
// Switch txout amount to 1 BTC. Transaction store doesn't
@ -75,12 +75,12 @@ func TestTxStore(t *testing.T) {
spendingTx.AddTxOut(spendingTxOut1)
spendingTx.AddTxOut(spendingTxOut2)
TstSpendingTx := btcutil.NewTx(spendingTx)
var _ = TstSpendingTx
tests := []struct {
name string
f func(*Store) (*Store, error)
err error
bal, unc int64
bal, unc btcutil.Amount
unspents map[btcwire.OutPoint]struct{}
unmined map[btcwire.ShaHash]struct{}
}{
@ -89,7 +89,6 @@ func TestTxStore(t *testing.T) {
f: func(_ *Store) (*Store, error) {
return NewStore(), nil
},
err: nil,
bal: 0,
unc: 0,
unspents: map[btcwire.OutPoint]struct{}{},
@ -98,19 +97,25 @@ func TestTxStore(t *testing.T) {
{
name: "txout insert",
f: func(s *Store) (*Store, error) {
r, err := s.InsertRecvTxOut(TstRecvTx, 0, false, time.Now(), nil)
r, err := s.InsertTx(TstRecvTx, nil)
if err != nil {
return nil, err
}
_, err = r.AddCredit(0, false)
if err != nil {
return nil, err
}
// Verify that we can create the JSON output without any
// errors.
_, err = r.ToJSON("", 100, btcwire.MainNet)
if err != nil {
return nil, err
}
// If the above succeeded, try using the record. This will
// dereference the tx and panic if the above didn't catch
// an inconsistant insert.
_ = r.TxInfo("", 100, btcwire.MainNet)
return s, nil
},
err: nil,
bal: 0,
unc: TstRecvTx.MsgTx().TxOut[0].Value,
unc: btcutil.Amount(TstRecvTx.MsgTx().TxOut[0].Value),
unspents: map[btcwire.OutPoint]struct{}{
*btcwire.NewOutPoint(TstRecvTx.Sha(), 0): {},
},
@ -119,16 +124,24 @@ func TestTxStore(t *testing.T) {
{
name: "insert duplicate unconfirmed",
f: func(s *Store) (*Store, error) {
r, err := s.InsertRecvTxOut(TstRecvTx, 0, false, time.Now(), nil)
r, err := s.InsertTx(TstRecvTx, nil)
if err != nil {
return nil, err
}
_, err = r.AddCredit(0, false)
if err != nil {
return nil, err
}
_, err = r.ToJSON("", 100, btcwire.MainNet)
if err != nil {
return nil, err
}
_ = r.TxInfo("", 100, btcwire.MainNet)
return s, nil
},
err: nil,
bal: 0,
unc: TstRecvTx.MsgTx().TxOut[0].Value,
unc: btcutil.Amount(TstRecvTx.MsgTx().TxOut[0].Value),
unspents: map[btcwire.OutPoint]struct{}{
*btcwire.NewOutPoint(TstRecvTx.Sha(), 0): {},
},
@ -137,15 +150,24 @@ func TestTxStore(t *testing.T) {
{
name: "confirmed txout insert",
f: func(s *Store) (*Store, error) {
r, err := s.InsertRecvTxOut(TstRecvTx, 0, false, TstRecvTxBlockDetails.Time, TstRecvTxBlockDetails)
TstRecvTx.SetIndex(TstRecvIndex)
r, err := s.InsertTx(TstRecvTx, TstRecvTxBlockDetails)
if err != nil {
return nil, err
}
_, err = r.AddCredit(0, false)
if err != nil {
return nil, err
}
_, err = r.ToJSON("", 100, btcwire.MainNet)
if err != nil {
return nil, err
}
_ = r.TxInfo("", 100, btcwire.MainNet)
return s, nil
},
err: nil,
bal: TstRecvTx.MsgTx().TxOut[0].Value,
bal: btcutil.Amount(TstRecvTx.MsgTx().TxOut[0].Value),
unc: 0,
unspents: map[btcwire.OutPoint]struct{}{
*btcwire.NewOutPoint(TstRecvTx.Sha(), 0): {},
@ -155,15 +177,24 @@ func TestTxStore(t *testing.T) {
{
name: "insert duplicate confirmed",
f: func(s *Store) (*Store, error) {
r, err := s.InsertRecvTxOut(TstRecvTx, 0, false, TstRecvTxBlockDetails.Time, TstRecvTxBlockDetails)
TstRecvTx.SetIndex(TstRecvIndex)
r, err := s.InsertTx(TstRecvTx, TstRecvTxBlockDetails)
if err != nil {
return nil, err
}
_, err = r.AddCredit(0, false)
if err != nil {
return nil, err
}
_, err = r.ToJSON("", 100, btcwire.MainNet)
if err != nil {
return nil, err
}
_ = r.TxInfo("", 100, btcwire.MainNet)
return s, nil
},
err: nil,
bal: TstRecvTx.MsgTx().TxOut[0].Value,
bal: btcutil.Amount(TstRecvTx.MsgTx().TxOut[0].Value),
unc: 0,
unspents: map[btcwire.OutPoint]struct{}{
*btcwire.NewOutPoint(TstRecvTx.Sha(), 0): {},
@ -171,29 +202,43 @@ func TestTxStore(t *testing.T) {
unmined: map[btcwire.ShaHash]struct{}{},
},
{
name: "insert duplicate unconfirmed",
name: "rollback confirmed credit",
f: func(s *Store) (*Store, error) {
r, err := s.InsertRecvTxOut(TstRecvTx, 0, false, time.Now(), nil)
err := s.Rollback(TstRecvTxBlockDetails.Height)
if err != nil {
return nil, err
}
_ = r.TxInfo("", 100, btcwire.MainNet)
return s, nil
},
err: ErrInconsistantStore,
bal: 0,
unc: btcutil.Amount(TstRecvTx.MsgTx().TxOut[0].Value),
unspents: map[btcwire.OutPoint]struct{}{
*btcwire.NewOutPoint(TstRecvTx.Sha(), 0): {},
},
unmined: map[btcwire.ShaHash]struct{}{},
},
{
name: "insert double spend with new txout value",
name: "insert confirmed double spend",
f: func(s *Store) (*Store, error) {
r, err := s.InsertRecvTxOut(TstDoubleSpendTx, 0, false, TstRecvTxBlockDetails.Time, TstRecvTxBlockDetails)
TstDoubleSpendTx.SetIndex(TstRecvIndex)
r, err := s.InsertTx(TstDoubleSpendTx, TstRecvTxBlockDetails)
if err != nil {
return nil, err
}
_, err = r.AddCredit(0, false)
if err != nil {
return nil, err
}
_, err = r.ToJSON("", 100, btcwire.MainNet)
if err != nil {
return nil, err
}
_ = r.TxInfo("", 100, btcwire.MainNet)
return s, nil
},
err: nil,
bal: TstDoubleSpendTx.MsgTx().TxOut[0].Value,
bal: btcutil.Amount(TstDoubleSpendTx.MsgTx().TxOut[0].Value),
unc: 0,
unspents: map[btcwire.OutPoint]struct{}{
*btcwire.NewOutPoint(TstDoubleSpendTx.Sha(), 0): {},
@ -201,16 +246,29 @@ func TestTxStore(t *testing.T) {
unmined: map[btcwire.ShaHash]struct{}{},
},
{
name: "insert unconfirmed signed tx",
name: "insert unconfirmed debit",
f: func(s *Store) (*Store, error) {
r, err := s.InsertSignedTx(TstSpendingTx, time.Now(), nil)
prev, err := s.InsertTx(TstDoubleSpendTx, TstRecvTxBlockDetails)
if err != nil {
return nil, err
}
r, err := s.InsertTx(TstSpendingTx, nil)
if err != nil {
return nil, err
}
_, err = r.AddDebits(prev.Credits())
if err != nil {
return nil, err
}
_, err = r.ToJSON("", 100, btcwire.MainNet)
if err != nil {
return nil, err
}
_ = r.TxInfo("", 100, btcwire.MainNet)
return s, nil
},
err: nil,
bal: 0,
unc: 0,
unspents: map[btcwire.OutPoint]struct{}{},
@ -219,16 +277,29 @@ func TestTxStore(t *testing.T) {
},
},
{
name: "insert unconfirmed signed tx again",
name: "insert unconfirmed debit again",
f: func(s *Store) (*Store, error) {
r, err := s.InsertSignedTx(TstSpendingTx, time.Now(), nil)
prev, err := s.InsertTx(TstDoubleSpendTx, TstRecvTxBlockDetails)
if err != nil {
return nil, err
}
r, err := s.InsertTx(TstSpendingTx, nil)
if err != nil {
return nil, err
}
_, err = r.AddDebits(prev.Credits())
if err != nil {
return nil, err
}
_, err = r.ToJSON("", 100, btcwire.MainNet)
if err != nil {
return nil, err
}
_ = r.TxInfo("", 100, btcwire.MainNet)
return s, nil
},
err: nil,
bal: 0,
unc: 0,
unspents: map[btcwire.OutPoint]struct{}{},
@ -239,16 +310,24 @@ func TestTxStore(t *testing.T) {
{
name: "insert change (index 0)",
f: func(s *Store) (*Store, error) {
r, err := s.InsertRecvTxOut(TstSpendingTx, 0, true, time.Now(), nil)
r, err := s.InsertTx(TstSpendingTx, nil)
if err != nil {
return nil, err
}
_, err = r.AddCredit(0, true)
if err != nil {
return nil, err
}
_, err = r.ToJSON("", 100, btcwire.MainNet)
if err != nil {
return nil, err
}
_ = r.TxInfo("", 100, btcwire.MainNet)
return s, nil
},
err: nil,
bal: 0,
unc: TstSpendingTx.MsgTx().TxOut[0].Value,
unc: btcutil.Amount(TstSpendingTx.MsgTx().TxOut[0].Value),
unspents: map[btcwire.OutPoint]struct{}{
*btcwire.NewOutPoint(TstSpendingTx.Sha(), 0): {},
},
@ -259,16 +338,24 @@ func TestTxStore(t *testing.T) {
{
name: "insert output back to this own wallet (index 1)",
f: func(s *Store) (*Store, error) {
r, err := s.InsertRecvTxOut(TstSpendingTx, 1, true, time.Now(), nil)
r, err := s.InsertTx(TstSpendingTx, nil)
if err != nil {
return nil, err
}
_, err = r.AddCredit(1, true)
if err != nil {
return nil, err
}
_, err = r.ToJSON("", 100, btcwire.MainNet)
if err != nil {
return nil, err
}
_ = r.TxInfo("", 100, btcwire.MainNet)
return s, nil
},
err: nil,
bal: 0,
unc: TstSpendingTx.MsgTx().TxOut[0].Value + TstSpendingTx.MsgTx().TxOut[1].Value,
unc: btcutil.Amount(TstSpendingTx.MsgTx().TxOut[0].Value + TstSpendingTx.MsgTx().TxOut[1].Value),
unspents: map[btcwire.OutPoint]struct{}{
*btcwire.NewOutPoint(TstSpendingTx.Sha(), 0): {},
*btcwire.NewOutPoint(TstSpendingTx.Sha(), 1): {},
@ -278,17 +365,21 @@ func TestTxStore(t *testing.T) {
},
},
{
name: "confirmed signed tx",
name: "confirm signed tx",
f: func(s *Store) (*Store, error) {
r, err := s.InsertSignedTx(TstSpendingTx, TstSignedTxBlockDetails.Time, TstSignedTxBlockDetails)
TstSpendingTx.SetIndex(TstSignedTxIndex)
r, err := s.InsertTx(TstSpendingTx, TstSignedTxBlockDetails)
if err != nil {
return nil, err
}
_, err = r.ToJSON("", 100, btcwire.MainNet)
if err != nil {
return nil, err
}
_ = r.TxInfo("", 100, btcwire.MainNet)
return s, nil
},
err: nil,
bal: TstSpendingTx.MsgTx().TxOut[0].Value + TstSpendingTx.MsgTx().TxOut[1].Value,
bal: btcutil.Amount(TstSpendingTx.MsgTx().TxOut[0].Value + TstSpendingTx.MsgTx().TxOut[1].Value),
unc: 0,
unspents: map[btcwire.OutPoint]struct{}{
*btcwire.NewOutPoint(TstSpendingTx.Sha(), 0): {},
@ -299,11 +390,13 @@ func TestTxStore(t *testing.T) {
{
name: "rollback after spending tx",
f: func(s *Store) (*Store, error) {
s.Rollback(TstSignedTxBlockDetails.Height + 1)
err := s.Rollback(TstSignedTxBlockDetails.Height + 1)
if err != nil {
return nil, err
}
return s, nil
},
err: nil,
bal: TstSpendingTx.MsgTx().TxOut[0].Value + TstSpendingTx.MsgTx().TxOut[1].Value,
bal: btcutil.Amount(TstSpendingTx.MsgTx().TxOut[0].Value + TstSpendingTx.MsgTx().TxOut[1].Value),
unc: 0,
unspents: map[btcwire.OutPoint]struct{}{
*btcwire.NewOutPoint(TstSpendingTx.Sha(), 0): {},
@ -314,12 +407,14 @@ func TestTxStore(t *testing.T) {
{
name: "rollback spending tx block",
f: func(s *Store) (*Store, error) {
s.Rollback(TstSignedTxBlockDetails.Height)
err := s.Rollback(TstSignedTxBlockDetails.Height)
if err != nil {
return nil, err
}
return s, nil
},
err: nil,
bal: 0,
unc: TstSpendingTx.MsgTx().TxOut[0].Value + TstSpendingTx.MsgTx().TxOut[1].Value,
unc: btcutil.Amount(TstSpendingTx.MsgTx().TxOut[0].Value + TstSpendingTx.MsgTx().TxOut[1].Value),
unspents: map[btcwire.OutPoint]struct{}{
*btcwire.NewOutPoint(TstSpendingTx.Sha(), 0): {},
*btcwire.NewOutPoint(TstSpendingTx.Sha(), 1): {},
@ -331,12 +426,14 @@ func TestTxStore(t *testing.T) {
{
name: "rollback double spend tx block",
f: func(s *Store) (*Store, error) {
s.Rollback(TstRecvTxBlockDetails.Height)
err := s.Rollback(TstRecvTxBlockDetails.Height)
if err != nil {
return nil, err
}
return s, nil
},
err: nil,
bal: 0,
unc: TstSpendingTx.MsgTx().TxOut[0].Value + TstSpendingTx.MsgTx().TxOut[1].Value,
unc: btcutil.Amount(TstSpendingTx.MsgTx().TxOut[0].Value + TstSpendingTx.MsgTx().TxOut[1].Value),
unspents: map[btcwire.OutPoint]struct{}{
*btcwire.NewOutPoint(TstSpendingTx.Sha(), 0): {},
*btcwire.NewOutPoint(TstSpendingTx.Sha(), 1): {},
@ -348,15 +445,24 @@ func TestTxStore(t *testing.T) {
{
name: "insert original recv txout",
f: func(s *Store) (*Store, error) {
r, err := s.InsertRecvTxOut(TstRecvTx, 0, false, TstRecvTxBlockDetails.Time, TstRecvTxBlockDetails)
TstRecvTx.SetIndex(TstRecvIndex)
r, err := s.InsertTx(TstRecvTx, TstRecvTxBlockDetails)
if err != nil {
return nil, err
}
_, err = r.AddCredit(0, false)
if err != nil {
return nil, err
}
_, err = r.ToJSON("", 100, btcwire.MainNet)
if err != nil {
return nil, err
}
_ = r.TxInfo("", 100, btcwire.MainNet)
return s, nil
},
err: nil,
bal: TstRecvTx.MsgTx().TxOut[0].Value,
bal: btcutil.Amount(TstRecvTx.MsgTx().TxOut[0].Value),
unc: 0,
unspents: map[btcwire.OutPoint]struct{}{
*btcwire.NewOutPoint(TstRecvTx.Sha(), 0): {},
@ -368,29 +474,37 @@ func TestTxStore(t *testing.T) {
var s *Store
for _, test := range tests {
tmpStore, err := test.f(s)
if err != test.err {
t.Fatalf("%s: error mismatch: expected: %v, got: %v", test.name, test.err, err)
}
if test.err != nil {
continue
if err != nil {
t.Fatalf("%s: got error: %v", test.name, err)
}
s = tmpStore
bal := s.Balance(1, TstRecvCurrentHeight)
if bal != test.bal {
t.Errorf("%s: balance mismatch: expected: %d, got: %d", test.name, test.bal, bal)
bal, err := s.Balance(1, TstRecvCurrentHeight)
if err != nil {
t.Fatalf("%s: Confirmed Balance() failed: %v", test.name, err)
}
unc := s.Balance(0, TstRecvCurrentHeight) - bal
if bal != test.bal {
t.Fatalf("%s: balance mismatch: expected: %d, got: %d", test.name, test.bal, bal)
}
unc, err := s.Balance(0, TstRecvCurrentHeight)
if err != nil {
t.Fatalf("%s: Unconfirmed Balance() failed: %v", test.name, err)
}
unc -= bal
if unc != test.unc {
t.Errorf("%s: unconfimred balance mismatch: expected %d, got %d", test.name, test.unc, unc)
t.Errorf("%s: unconfirmed balance mismatch: expected %d, got %d", test.name, test.unc, unc)
}
// Check that unspent outputs match expected.
for _, record := range s.UnspentOutputs() {
if record.Spent() {
unspent, err := s.UnspentOutputs()
if err != nil {
t.Fatal(err)
}
for _, r := range unspent {
if r.Spent() {
t.Errorf("%s: unspent record marked as spent", test.name)
}
op := *record.OutPoint()
op := *r.OutPoint()
if _, ok := test.unspents[op]; !ok {
t.Errorf("%s: unexpected unspent output: %v", test.name, op)
}
@ -400,10 +514,10 @@ func TestTxStore(t *testing.T) {
t.Errorf("%s: missing expected unspent output(s)", test.name)
}
// Check that unmined signed txs match expected.
for _, tx := range s.UnminedSignedTxs() {
// Check that unmined sent txs match expected.
for _, tx := range s.UnminedDebitTxs() {
if _, ok := test.unmined[*tx.Sha()]; !ok {
t.Errorf("%s: unexpected unmined signed tx: %v", test.name, *tx.Sha())
t.Fatalf("%s: unexpected unmined signed tx: %v", test.name, *tx.Sha())
}
delete(test.unmined, *tx.Sha())
}
@ -431,3 +545,51 @@ func TestTxStore(t *testing.T) {
}
}
}
func TestFindingSpentCredits(t *testing.T) {
s := NewStore()
// Insert transaction and credit which will be spent.
r, err := s.InsertTx(TstRecvTx, TstRecvTxBlockDetails)
if err != nil {
t.Fatal(err)
}
_, err = r.AddCredit(0, false)
if err != nil {
t.Fatal(err)
}
// Insert confirmed transaction which spends the above credit.
TstSpendingTx.SetIndex(TstSignedTxIndex)
r2, err := s.InsertTx(TstSpendingTx, TstSignedTxBlockDetails)
if err != nil {
t.Fatal(err)
}
_, err = r2.AddCredit(0, false)
if err != nil {
t.Fatal(err)
}
_, err = r2.AddDebits(nil)
if err != nil {
t.Fatal(err)
}
bal, err := s.Balance(1, TstSignedTxBlockDetails.Height)
if err != nil {
t.Fatal(err)
}
if bal != btcutil.Amount(TstSpendingTx.MsgTx().TxOut[0].Value) {
t.Fatal("bad balance")
}
unspents, err := s.UnspentOutputs()
if err != nil {
t.Fatal(err)
}
op := btcwire.NewOutPoint(TstSpendingTx.Sha(), 0)
if *unspents[0].OutPoint() != *op {
t.Fatal("unspent outpoint doesn't match expected")
}
if len(unspents) > 1 {
t.Fatal("has more than one unspent credit")
}
}