lbcd/blockchain/scriptval.go

260 lines
7.3 KiB
Go
Raw Normal View History

// Copyright (c) 2013-2014 The btcsuite developers
2013-07-18 16:49:28 +02:00
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.
package blockchain
2013-07-18 16:49:28 +02:00
import (
"fmt"
2014-07-02 18:04:59 +02:00
"math"
"runtime"
"github.com/btcsuite/btcd/txscript"
"github.com/btcsuite/btcd/wire"
"github.com/btcsuite/btcutil"
2013-07-18 16:49:28 +02:00
)
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
// txValidateItem holds a transaction along with which input to validate.
type txValidateItem struct {
txInIndex int
txIn *wire.TxIn
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
tx *btcutil.Tx
2013-07-18 16:49:28 +02:00
}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
// txValidator provides a type which asynchronously validates transaction
// inputs. It provides several channels for communication and a processing
// function that is intended to be in run multiple goroutines.
type txValidator struct {
validateChan chan *txValidateItem
quitChan chan struct{}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
resultChan chan error
txStore TxStore
flags txscript.ScriptFlags
Integrate a valid ECDSA signature cache into btcd Introduce an ECDSA signature verification into btcd in order to mitigate a certain DoS attack and as a performance optimization. The benefits of SigCache are two fold. Firstly, usage of SigCache mitigates a DoS attack wherein an attacker causes a victim's client to hang due to worst-case behavior triggered while processing attacker crafted invalid transactions. A detailed description of the mitigated DoS attack can be found here: https://bitslog.wordpress.com/2013/01/23/fixed-bitcoin-vulnerability-explanation-why-the-signature-cache-is-a-dos-protection/ Secondly, usage of the SigCache introduces a signature verification optimization which speeds up the validation of transactions within a block, if they've already been seen and verified within the mempool. The server itself manages the sigCache instance. The blockManager and txMempool respectively now receive pointers to the created sigCache instance. All read (sig triplet existence) operations on the sigCache will not block unless a separate goroutine is adding an entry (writing) to the sigCache. GetBlockTemplate generation now also utilizes the sigCache in order to avoid unnecessarily double checking signatures when generating a template after previously accepting a txn to the mempool. Consequently, the CPU miner now also employs the same optimization. The maximum number of entries for the sigCache has been introduced as a config parameter in order to allow users to configure the amount of memory consumed by this new additional caching.
2015-09-25 01:22:00 +02:00
sigCache *txscript.SigCache
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
}
// sendResult sends the result of a script pair validation on the internal
// result channel while respecting the quit channel. The allows orderly
// shutdown when the validation process is aborted early due to a validation
// error in one of the other goroutines.
func (v *txValidator) sendResult(result error) {
select {
case v.resultChan <- result:
case <-v.quitChan:
}
}
// validateHandler consumes items to validate from the internal validate channel
// and returns the result of the validation on the internal result channel. It
// must be run as a goroutine.
func (v *txValidator) validateHandler() {
out:
for {
select {
case txVI := <-v.validateChan:
// Ensure the referenced input transaction is available.
txIn := txVI.txIn
originTxHash := &txIn.PreviousOutPoint.Hash
originTx, exists := v.txStore[*originTxHash]
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
if !exists || originTx.Err != nil || originTx.Tx == nil {
str := fmt.Sprintf("unable to find input "+
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
"transaction %v referenced from "+
"transaction %v", originTxHash,
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
txVI.tx.Sha())
err := ruleError(ErrMissingTx, str)
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
v.sendResult(err)
break out
}
originMsgTx := originTx.Tx.MsgTx()
// Ensure the output index in the referenced transaction
// is available.
originTxIndex := txIn.PreviousOutPoint.Index
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
if originTxIndex >= uint32(len(originMsgTx.TxOut)) {
str := fmt.Sprintf("out of bounds "+
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
"input index %d in transaction %v "+
"referenced from transaction %v",
originTxIndex, originTxHash,
txVI.tx.Sha())
err := ruleError(ErrBadTxInput, str)
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
v.sendResult(err)
break out
}
// Create a new script engine for the script pair.
sigScript := txIn.SignatureScript
pkScript := originMsgTx.TxOut[originTxIndex].PkScript
vm, err := txscript.NewEngine(pkScript, txVI.tx.MsgTx(),
Integrate a valid ECDSA signature cache into btcd Introduce an ECDSA signature verification into btcd in order to mitigate a certain DoS attack and as a performance optimization. The benefits of SigCache are two fold. Firstly, usage of SigCache mitigates a DoS attack wherein an attacker causes a victim's client to hang due to worst-case behavior triggered while processing attacker crafted invalid transactions. A detailed description of the mitigated DoS attack can be found here: https://bitslog.wordpress.com/2013/01/23/fixed-bitcoin-vulnerability-explanation-why-the-signature-cache-is-a-dos-protection/ Secondly, usage of the SigCache introduces a signature verification optimization which speeds up the validation of transactions within a block, if they've already been seen and verified within the mempool. The server itself manages the sigCache instance. The blockManager and txMempool respectively now receive pointers to the created sigCache instance. All read (sig triplet existence) operations on the sigCache will not block unless a separate goroutine is adding an entry (writing) to the sigCache. GetBlockTemplate generation now also utilizes the sigCache in order to avoid unnecessarily double checking signatures when generating a template after previously accepting a txn to the mempool. Consequently, the CPU miner now also employs the same optimization. The maximum number of entries for the sigCache has been introduced as a config parameter in order to allow users to configure the amount of memory consumed by this new additional caching.
2015-09-25 01:22:00 +02:00
txVI.txInIndex, v.flags, v.sigCache)
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
if err != nil {
str := fmt.Sprintf("failed to parse input "+
"%s:%d which references output %s:%d - "+
"%v (input script bytes %x, prev output "+
"script bytes %x)", txVI.tx.Sha(),
txVI.txInIndex, originTxHash,
originTxIndex, err, sigScript, pkScript)
err := ruleError(ErrScriptMalformed, str)
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
v.sendResult(err)
break out
}
// Execute the script pair.
if err := vm.Execute(); err != nil {
str := fmt.Sprintf("failed to validate input "+
"%s:%d which references output %s:%d - "+
"%v (input script bytes %x, prev output "+
"script bytes %x)", txVI.tx.Sha(),
txVI.txInIndex, originTxHash,
originTxIndex, err, sigScript, pkScript)
err := ruleError(ErrScriptValidation, str)
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
v.sendResult(err)
break out
}
// Validation succeeded.
v.sendResult(nil)
case <-v.quitChan:
break out
}
}
}
// Validate validates the scripts for all of the passed transaction inputs using
// multiple goroutines.
func (v *txValidator) Validate(items []*txValidateItem) error {
if len(items) == 0 {
2013-07-18 16:49:28 +02:00
return nil
}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
// Limit the number of goroutines to do script validation based on the
// number of processor cores. This help ensure the system stays
// reasonably responsive under heavy load.
maxGoRoutines := runtime.NumCPU() * 3
if maxGoRoutines <= 0 {
maxGoRoutines = 1
}
if maxGoRoutines > len(items) {
maxGoRoutines = len(items)
2013-07-18 16:49:28 +02:00
}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
// Start up validation handlers that are used to asynchronously
// validate each transaction input.
for i := 0; i < maxGoRoutines; i++ {
go v.validateHandler()
2013-07-18 16:49:28 +02:00
}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
// Validate each of the inputs. The quit channel is closed when any
// errors occur so all processing goroutines exit regardless of which
// input had the validation error.
numInputs := len(items)
currentItem := 0
processedItems := 0
for processedItems < numInputs {
// Only send items while there are still items that need to
// be processed. The select statement will never select a nil
// channel.
var validateChan chan *txValidateItem
var item *txValidateItem
if currentItem < numInputs {
validateChan = v.validateChan
item = items[currentItem]
}
select {
case validateChan <- item:
currentItem++
case err := <-v.resultChan:
processedItems++
if err != nil {
close(v.quitChan)
return err
}
}
2013-07-18 16:49:28 +02:00
}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
close(v.quitChan)
2013-07-18 16:49:28 +02:00
return nil
}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
// newTxValidator returns a new instance of txValidator to be used for
// validating transaction scripts asynchronously.
Integrate a valid ECDSA signature cache into btcd Introduce an ECDSA signature verification into btcd in order to mitigate a certain DoS attack and as a performance optimization. The benefits of SigCache are two fold. Firstly, usage of SigCache mitigates a DoS attack wherein an attacker causes a victim's client to hang due to worst-case behavior triggered while processing attacker crafted invalid transactions. A detailed description of the mitigated DoS attack can be found here: https://bitslog.wordpress.com/2013/01/23/fixed-bitcoin-vulnerability-explanation-why-the-signature-cache-is-a-dos-protection/ Secondly, usage of the SigCache introduces a signature verification optimization which speeds up the validation of transactions within a block, if they've already been seen and verified within the mempool. The server itself manages the sigCache instance. The blockManager and txMempool respectively now receive pointers to the created sigCache instance. All read (sig triplet existence) operations on the sigCache will not block unless a separate goroutine is adding an entry (writing) to the sigCache. GetBlockTemplate generation now also utilizes the sigCache in order to avoid unnecessarily double checking signatures when generating a template after previously accepting a txn to the mempool. Consequently, the CPU miner now also employs the same optimization. The maximum number of entries for the sigCache has been introduced as a config parameter in order to allow users to configure the amount of memory consumed by this new additional caching.
2015-09-25 01:22:00 +02:00
func newTxValidator(txStore TxStore, flags txscript.ScriptFlags, sigCache *txscript.SigCache) *txValidator {
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
return &txValidator{
validateChan: make(chan *txValidateItem),
quitChan: make(chan struct{}),
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
resultChan: make(chan error),
txStore: txStore,
Integrate a valid ECDSA signature cache into btcd Introduce an ECDSA signature verification into btcd in order to mitigate a certain DoS attack and as a performance optimization. The benefits of SigCache are two fold. Firstly, usage of SigCache mitigates a DoS attack wherein an attacker causes a victim's client to hang due to worst-case behavior triggered while processing attacker crafted invalid transactions. A detailed description of the mitigated DoS attack can be found here: https://bitslog.wordpress.com/2013/01/23/fixed-bitcoin-vulnerability-explanation-why-the-signature-cache-is-a-dos-protection/ Secondly, usage of the SigCache introduces a signature verification optimization which speeds up the validation of transactions within a block, if they've already been seen and verified within the mempool. The server itself manages the sigCache instance. The blockManager and txMempool respectively now receive pointers to the created sigCache instance. All read (sig triplet existence) operations on the sigCache will not block unless a separate goroutine is adding an entry (writing) to the sigCache. GetBlockTemplate generation now also utilizes the sigCache in order to avoid unnecessarily double checking signatures when generating a template after previously accepting a txn to the mempool. Consequently, the CPU miner now also employs the same optimization. The maximum number of entries for the sigCache has been introduced as a config parameter in order to allow users to configure the amount of memory consumed by this new additional caching.
2015-09-25 01:22:00 +02:00
sigCache: sigCache,
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
flags: flags,
}
}
// ValidateTransactionScripts validates the scripts for the passed transaction
// using multiple goroutines.
Integrate a valid ECDSA signature cache into btcd Introduce an ECDSA signature verification into btcd in order to mitigate a certain DoS attack and as a performance optimization. The benefits of SigCache are two fold. Firstly, usage of SigCache mitigates a DoS attack wherein an attacker causes a victim's client to hang due to worst-case behavior triggered while processing attacker crafted invalid transactions. A detailed description of the mitigated DoS attack can be found here: https://bitslog.wordpress.com/2013/01/23/fixed-bitcoin-vulnerability-explanation-why-the-signature-cache-is-a-dos-protection/ Secondly, usage of the SigCache introduces a signature verification optimization which speeds up the validation of transactions within a block, if they've already been seen and verified within the mempool. The server itself manages the sigCache instance. The blockManager and txMempool respectively now receive pointers to the created sigCache instance. All read (sig triplet existence) operations on the sigCache will not block unless a separate goroutine is adding an entry (writing) to the sigCache. GetBlockTemplate generation now also utilizes the sigCache in order to avoid unnecessarily double checking signatures when generating a template after previously accepting a txn to the mempool. Consequently, the CPU miner now also employs the same optimization. The maximum number of entries for the sigCache has been introduced as a config parameter in order to allow users to configure the amount of memory consumed by this new additional caching.
2015-09-25 01:22:00 +02:00
func ValidateTransactionScripts(tx *btcutil.Tx, txStore TxStore, flags txscript.ScriptFlags, sigCache *txscript.SigCache) error {
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
// Collect all of the transaction inputs and required information for
// validation.
txIns := tx.MsgTx().TxIn
txValItems := make([]*txValidateItem, 0, len(txIns))
for txInIdx, txIn := range txIns {
// Skip coinbases.
if txIn.PreviousOutPoint.Index == math.MaxUint32 {
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
continue
2013-07-18 16:49:28 +02:00
}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
txVI := &txValidateItem{
txInIndex: txInIdx,
txIn: txIn,
tx: tx,
2013-07-18 16:49:28 +02:00
}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
txValItems = append(txValItems, txVI)
2013-07-18 16:49:28 +02:00
}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
// Validate all of the inputs.
Integrate a valid ECDSA signature cache into btcd Introduce an ECDSA signature verification into btcd in order to mitigate a certain DoS attack and as a performance optimization. The benefits of SigCache are two fold. Firstly, usage of SigCache mitigates a DoS attack wherein an attacker causes a victim's client to hang due to worst-case behavior triggered while processing attacker crafted invalid transactions. A detailed description of the mitigated DoS attack can be found here: https://bitslog.wordpress.com/2013/01/23/fixed-bitcoin-vulnerability-explanation-why-the-signature-cache-is-a-dos-protection/ Secondly, usage of the SigCache introduces a signature verification optimization which speeds up the validation of transactions within a block, if they've already been seen and verified within the mempool. The server itself manages the sigCache instance. The blockManager and txMempool respectively now receive pointers to the created sigCache instance. All read (sig triplet existence) operations on the sigCache will not block unless a separate goroutine is adding an entry (writing) to the sigCache. GetBlockTemplate generation now also utilizes the sigCache in order to avoid unnecessarily double checking signatures when generating a template after previously accepting a txn to the mempool. Consequently, the CPU miner now also employs the same optimization. The maximum number of entries for the sigCache has been introduced as a config parameter in order to allow users to configure the amount of memory consumed by this new additional caching.
2015-09-25 01:22:00 +02:00
validator := newTxValidator(txStore, flags, sigCache)
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
if err := validator.Validate(txValItems); err != nil {
return err
2013-07-18 16:49:28 +02:00
}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
return nil
2013-07-18 16:49:28 +02:00
}
// checkBlockScripts executes and validates the scripts for all transactions in
// the passed block.
func checkBlockScripts(block *btcutil.Block, txStore TxStore,
Integrate a valid ECDSA signature cache into btcd Introduce an ECDSA signature verification into btcd in order to mitigate a certain DoS attack and as a performance optimization. The benefits of SigCache are two fold. Firstly, usage of SigCache mitigates a DoS attack wherein an attacker causes a victim's client to hang due to worst-case behavior triggered while processing attacker crafted invalid transactions. A detailed description of the mitigated DoS attack can be found here: https://bitslog.wordpress.com/2013/01/23/fixed-bitcoin-vulnerability-explanation-why-the-signature-cache-is-a-dos-protection/ Secondly, usage of the SigCache introduces a signature verification optimization which speeds up the validation of transactions within a block, if they've already been seen and verified within the mempool. The server itself manages the sigCache instance. The blockManager and txMempool respectively now receive pointers to the created sigCache instance. All read (sig triplet existence) operations on the sigCache will not block unless a separate goroutine is adding an entry (writing) to the sigCache. GetBlockTemplate generation now also utilizes the sigCache in order to avoid unnecessarily double checking signatures when generating a template after previously accepting a txn to the mempool. Consequently, the CPU miner now also employs the same optimization. The maximum number of entries for the sigCache has been introduced as a config parameter in order to allow users to configure the amount of memory consumed by this new additional caching.
2015-09-25 01:22:00 +02:00
scriptFlags txscript.ScriptFlags, sigCache *txscript.SigCache) error {
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
// Collect all of the transaction inputs and required information for
// validation for all transactions in the block into a single slice.
numInputs := 0
for _, tx := range block.Transactions() {
numInputs += len(tx.MsgTx().TxIn)
}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
txValItems := make([]*txValidateItem, 0, numInputs)
for _, tx := range block.Transactions() {
for txInIdx, txIn := range tx.MsgTx().TxIn {
// Skip coinbases.
if txIn.PreviousOutPoint.Index == math.MaxUint32 {
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
continue
}
txVI := &txValidateItem{
txInIndex: txInIdx,
txIn: txIn,
tx: tx,
}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
txValItems = append(txValItems, txVI)
}
}
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
// Validate all of the inputs.
Integrate a valid ECDSA signature cache into btcd Introduce an ECDSA signature verification into btcd in order to mitigate a certain DoS attack and as a performance optimization. The benefits of SigCache are two fold. Firstly, usage of SigCache mitigates a DoS attack wherein an attacker causes a victim's client to hang due to worst-case behavior triggered while processing attacker crafted invalid transactions. A detailed description of the mitigated DoS attack can be found here: https://bitslog.wordpress.com/2013/01/23/fixed-bitcoin-vulnerability-explanation-why-the-signature-cache-is-a-dos-protection/ Secondly, usage of the SigCache introduces a signature verification optimization which speeds up the validation of transactions within a block, if they've already been seen and verified within the mempool. The server itself manages the sigCache instance. The blockManager and txMempool respectively now receive pointers to the created sigCache instance. All read (sig triplet existence) operations on the sigCache will not block unless a separate goroutine is adding an entry (writing) to the sigCache. GetBlockTemplate generation now also utilizes the sigCache in order to avoid unnecessarily double checking signatures when generating a template after previously accepting a txn to the mempool. Consequently, the CPU miner now also employs the same optimization. The maximum number of entries for the sigCache has been introduced as a config parameter in order to allow users to configure the amount of memory consumed by this new additional caching.
2015-09-25 01:22:00 +02:00
validator := newTxValidator(txStore, scriptFlags, sigCache)
Rework and improve async script validation logic. The previous script validation logic entailed starting up a hard-coded number of goroutines to process the transaction scripts in parallel. In particular, one goroutine (up to 8 max) was started per transaction in a block and another one was started for each input script pair in the each transaction. This resulted in 64 goroutines simultaneously running scripts and verifying cryptographic signatures. This could easily lead to the overall system feeling sluggish. Further the previous design could also result in bursty behavior since the number of inputs to a transaction as well as its complexity can vary widely between transactions. For example, starting 2 goroutines (one to process the transaction and one for actual script pair validation) to verify a transaction with a single input was not desirable. Finally, the previous design validated all transactions and inputs regardless of a failure in one of the other scripts. This really didn't have a big impact since it's quite rare that blocks with invalid verifications are being processed, but it was a potential way DoS vector. This commit changes the logic in a few ways to improve things: - The max number of validation goroutines is now based on the number of cores in the system - All transaction inputs from all transactions in the block are collated into a single list which is fed through the aforementioned validation goroutines - The validation CPU usage is much more consistent due to the collation of inputs - A validation error in any goroutine immediately stops validation of all remaining inputs - The errors have been improved to include context about what tx script pair failed as opposed to showing the information as a warning This closes conformal/btcd#59.
2014-01-16 19:48:37 +01:00
if err := validator.Validate(txValItems); err != nil {
return err
2013-07-18 16:49:28 +02:00
}
return nil
}