This begins the process of converting the ExtractPkScriptAddrs function
to use the optimized extraction functions recently introduced as part of
the typeOfScript conversion.
In order to ease the review process, the detection of each script type
will be converted in a separate commit such that the script is only
parsed as a fallback for the cases that are not already converted to
more efficient variants.
In particular, this converts the detection for pay-to-script-hash
scripts.
This converts the ExtractAtomicSwapDataPushes function to make use of
the new tokenizer instead of the far less efficient parseScript thereby
significantly optimizing the function.
The new implementation is designed such that it should be fairly easy to
move the function into the atomic swap tools where it more naturally
belongs now that the tokenizer makes it possible to analyze scripts
outside of the txscript module. Consequently, this also deprecates the
function.
The following is a before and after comparison of attempting to extract
from both a typical atomic swap script and a very large non-atomic swap
script:
benchmark old ns/op new ns/op delta
BenchmarkExtractAtomicSwapDataPushesLarge-8 61332 44.4 -99.93%
BenchmarkExtractAtomicSwapDataPushes-8 990 260 -73.74%
benchmark old allocs new allocs delta
BenchmarkExtractAtomicSwapDataPushesLarge-8 1 0 -100.00%
BenchmarkExtractAtomicSwapDataPushes-8 2 1 -50.00%
benchmark old bytes new bytes delta
BenchmarkExtractAtomicSwapDataPushesLarge-8 311299 0 -100.00%
BenchmarkExtractAtomicSwapDataPushes-8 3168 96 -96.97%
This renames the canonicalPush function to isCanonicalPush and converts
it to accept an opcode as a byte and the associate data as a byte slice
instead of the internal parse opcode data struct in order to make it
more flexible for raw script analysis.
It also updates all callers and tests accordingly.
This converts the PushedData function to make use of the new tokenizer
instead of the far less efficient parseScript thereby significantly
optimizing the function.
Also, the comment is modified to explicitly call out the script version
semantics.
The following is a before and after comparison of extracting the data
from a very large script:
benchmark old ns/op new ns/op delta
BenchmarkPushedData-8 64837 1790 -97.24%
benchmark old allocs new allocs delta
BenchmarkPushedData-8 7 6 -14.29%
benchmark old bytes new bytes delta
BenchmarkPushedData-8 312816 1520 -99.51%
This converts the CalcMultiSigStats function to make use of the new
extractMultisigScriptDetails function instead of the far less efficient
parseScript thereby significantly optimizing the function.
The tests are also updated accordingly.
The following is a before and after comparison of analyzing a standard
multisig script:
benchmark old ns/op new ns/op delta
---------------------------------------------------------------
BenchmarkCalcMultiSigStats 972 79.5 -91.82%
benchmark old allocs new allocs delta
---------------------------------------------------------------
BenchmarkCalcMultiSigStats 1 0 -100.00%
benchmark old bytes new bytes delta
---------------------------------------------------------------
BenchmarkCalcMultiSigStats 2304 0 -100.00%
This converts CalcScriptInfo and dependent expectedInputs to make use of
the new script tokenizer as well as several of the other recently added
raw script analysis functions in order to remove the reliance on parsed
opcodes as a step towards utlimately removing them altogether.
It is worth noting that this has the side effect of significantly
optimizing the function as well, however, since it is deprecated, no
benchmarks are provided.
This concludes the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the far less efficient parsed opcodes.
In particular, it converts the detection of witness script hash scripts
to use raw script analysis and the new tokenizer.
With all of the limbs now useing optimized variants, the following is a
before an after comparison of calling GetScriptClass on a large script:
benchmark old ns/op new ns/op delta
BenchmarkGetScriptClass-8 61515 15.3 -99.98%
benchmark old allocs new allocs delta
BenchmarkGetScriptClass-8 1 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkGetScriptClass-8 311299 0 -100.00%
This continues the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the far less efficient parsed opcodes.
In particular, it converts the detection of witness pubkey hash scripts
to use raw script analysis and the new tokenizer.
The following is a before and after comparison of analyzing a large
script:
benchmark old ns/op new ns/op delta
BenchmarkIsWitnessPubKeyHash-8 61688 62839 +1.87%
benchmark old allocs new allocs delta
BenchmarkIsWitnessPubKeyHash-8 1 1 +0.00%
benchmark old bytes new bytes delta
BenchmarkIsWitnessPubKeyHash-8 311299 311299 +0.00%
This continues the process of converting the typeOfScript function to
use a combination of raw script analysize and the tokenizer instead of
parsed opcode, with the intent of significanty optimizing the function.
In particular, it converts the detection of null data scripts to use raw
script analysis.
This continues the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the far less efficient parsed opcodes.
In particular, it converts the detection of pay-to-pubkey-hash scripts
to use raw script analysis.
This continues the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the face less efficient parsed opcodes, with the intent of
significantly optimizing the function.
In particular, it converts the detection of pay-to-pubkey scripts to use
raw script analysis.
This continues the process of converting the typeOfScript function to
use a combination of raw script analysis and the new tokenizer instead
of the far less efficient parsed opcodes.
In particular, for this commit, since the ability to detect multisig
scripts via the new tokenizer is now available, the function is simply
updated to make use of it.
This begins the process of converting the typeOfScript function to use a
combination of raw script analysis and the new tokenizer instead of the
far less efficient parsed opcodes with the intent of significantly
optimizing the function.
In order to ease the review process, each script type will be converted
in a separate commit and the typeOfScript function will be updated such
that the script is only parsed as a fallback for the cases that are not
already converted to more efficient raw script variants.
In particular, for this commit, since the ability to detect
pay-to-script-hash via raw script analysis is now available, the
function is simply updated to make use of it.
This converts the typeOfScript function to accept a script version and
raw script instead of an array of internal parsed opcodes in order to
make it more flexible for raw script analysis.
Also, this adds a comment to CalcScriptInfo to call out the specific
version semantics and deprecates the function since nothing currently
uses it, and the relevant information can now be obtained by callers
more directly through the use of the new script tokenizer.
All other callers are updated accordingly.
This converts the IsNullData function to analyze the raw script instead
of using the far less efficient parseScript, thereby significantly
optimizing the function.
The following is a before and after comparison of analyzing a large
script:
benchmark old ns/op new ns/op delta
BenchmarkIsNullDataScript-8 62495 2.65 -100.00%
benchmark old allocs new allocs delta
BenchmarkIsNullDataScript-8 1 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkIsNullDataScript-8 311299 0 -100.00%
This converts the IsPayToWitnessScriptHash function to analyze the raw
script instead of using the far less efficient parseScript, thereby
significantly optimizing the function.
In order to accomplish this, it introduces two new functions. The first
one is named extractWitnessScriptHash and works with the raw script byte
to simultaneously deteremine if the script is a p2wsh script, and in the
case that is is, extract and return the hash. The second new function is
named isWitnessScriptHashScript and is defined in terms of the former.
The extract function approach was chosed because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type. Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result, so long as the extraction
does not require allocations.
Finally, this also deprecates the isWitnessScriptHash function that
requires opcodes in favor of the new functions and modifies the comment
on IsPayToWitnessScriptHash to call out the script version semantics.
The following is a before and after comparison of executing
IsPayToWitnessScriptHash on a large script:
benchmark old ns/op new ns/op delta
BenchmarkIsWitnessScriptHash-8 62774 0.63 -100.00%
benchmark old allocs new allocs delta
BenchmarkIsWitnessScriptHash-8 1 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkIsWitnessScriptHash-8 311299 0 -100.00%
This converts the IsPayToWitnessPubKeyHash function to analyze the raw
script instead of the far less efficient parseScript, thereby
significantly optimizing the function.
In order to accomplish this, it introduces two new functions. The first
one is named extractWitnessPubKeyHash and works with the raw script
bytes to simultaneously deteremine if the script is a p2wkh, and in case
it is, extract and return the hash. The second new function is name
isWitnessPubKeyHashScript which is defined in terms of the former.
The extract function is approach was chosen because it is common for
callers to want to only extract relevant details from the script if the
script is of the specific type. Extracting those details requires the
exact same checks to ensure the script is of the correct type, so it is
more efficient to combine the two and define the type determination in
terms of the result so long as the extraction does not require
allocations.
Finally, this deprecates the isWitnessPubKeyHash function that requires
opcodes in favor of the new functions and modifies the comment on
IsPayToWitnessPubKeyHash to explicitly call out the script version
semantics.
The following is a before and after comparison of executing
IsPayToWitnessPubKeyHash on a large script:
benchmark old ns/op new ns/op delta
BenchmarkIsWitnessPubKeyHash-8 68927 0.53 -100.00%
benchmark old allocs new allocs delta
BenchmarkIsWitnessPubKeyHash-8 1 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkIsWitnessPubKeyHash-8 311299 0 -100.00%
This converts the IsMultisigSigScript function to analyze the raw script
and make use of the new tokenizer instead of the far less efficient
parseScript thereby significantly optimizing the function.
In order to accomplish this, it first rejects scripts that can't
possibly fit the bill due to the final byte of what would be the redeem
script not being the appropriate opcode or the overall script not having
enough bytes. Then, it uses a new function that is introduced named
finalOpcodeData that uses the tokenizer to return any data associated
with the final opcode in the signature script (which will be nil for
non-push opcodes or if the script fails to parse) and analyzes it as if
it were a redeem script when it is non nil.
It is also worth noting that this new implementation intentionally has
the same semantic difference from the existing implementation as the
updated IsMultisigScript function in regards to allowing zero pubkeys
whereas previously it incorrectly required at least one pubkey.
Finally, the comment is modified to explicitly call out the script
version semantics.
The following is a before and after comparison of analyzing a large
script that is not a multisig script and both a 1-of-2 multisig public
key script (which should be false) and a signature script comprised of a
pay-to-script-hash 1-of-2 multisig redeem script (which should be true):
benchmark old ns/op new ns/op delta
BenchmarkIsMultisigSigScriptLarge-8 69328 2.93 -100.00%
BenchmarkIsMultisigSigScript-8 2375 146 -93.85%
benchmark old allocs new allocs delta
BenchmarkIsMultisigSigScriptLarge-8 5 0 -100.00%
BenchmarkIsMultisigSigScript-8 3 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkIsMultisigSigScriptLarge-8 330035 0 -100.00%
BenchmarkIsMultisigSigScript-8 9472 0 -100.00%
This converts the IsMultisigScript function to make use of the new
tokenizer instead of the far less efficient parseScript thereby
significantly optimizing the function.
In order to accomplish this, it introduces two new functions. The first
one is named extractMultisigScriptDetails and works with the raw script
bytes to simultaneously determine if the script is a multisignature
script, and in the case it is, extract and return the relevant details.
The second new function is named isMultisigScript and is defined in
terms of the former.
The extract function accepts the script version, raw script bytes, and a
flag to determine whether or not the public keys should also be
extracted. The flag is provided because extracting pubkeys results in
an allocation that the caller might wish to avoid.
The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type. Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.
It is important to note that this new implementation intentionally has a
semantic difference from the existing implementation in that it will now
correctly identify a multisig script with zero pubkeys whereas
previously it incorrectly required at least one pubkey. This change is
acceptable because the function only deals with standardness rather than
consensus rules.
Finally, this also deprecates the isMultiSig function that requires
opcodes in favor of the new functions and deprecates the error return on
the export IsMultisigScript function since it really does not make sense
given the purpose of the function.
The following is a before and after comparison of analyzing both a large
script that is not a multisig script and a 1-of-2 multisig public key
script:
benchmark old ns/op new ns/op delta
BenchmarkIsMultisigScriptLarge-8 64166 5.52 -99.99%
BenchmarkIsMultisigScript-8 630 59.4 -90.57%
benchmark old allocs new allocs delta
BenchmarkIsMultisigScriptLarge-8 1 0 -100.00%
BenchmarkIsMultisigScript-8 1 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkIsMultisigScriptLarge-8 311299 0 -100.00%
BenchmarkIsMultisigScript-8 2304 0 -100.00%
This converts the IsPayToScriptHash function to analyze the raw script
instead of using the far less efficient parseScript thereby
significantly optimizing the function.
In order to accomplish this, it introduces two new functions. The first
one is named extractScriptHash and works with the raw script bytes to
simultaneously determine if the script is a p2sh script, and in the case
it is, extract and return the hash. The second new function is named
isScriptHashScript and is defined in terms of the former.
The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type. Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.
Finally, this also deprecates the isScriptHash function that requires
opcodes in favor of the new functions and modifies the comment on
IsPayToScriptHash to explicitly call out the script version semantics.
The following is a before and after comparison of analyzing a large
script that is not a p2sh script:
benchmark old ns/op new ns/op delta
BenchmarkIsPayToScriptHash-8 62393 0.60 -100.00%
benchmark old allocs new allocs delta
BenchmarkIsPayToScriptHash-8 1 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkIsPayToScriptHash-8 311299 0 -100.00%
This converts the IsPayToPubKeyHash function to analyze the raw script
instead of using the far less efficient parseScript, thereby
significantly optimization the function.
In order to accomplish this, it introduces two new functions. The first
one is named extractPubKeyHash and works with the raw script bytes
to simultaneously determine if the script is a pay-to-pubkey-hash script,
and in the case it is, extract and return the hash. The second new
function is named isPubKeyHashScript and is defined in terms of the
former.
The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type. Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.
The following is a before and after comparison of analyzing a large
script:
benchmark old ns/op new ns/op delta
BenchmarkIsPubKeyHashScript-8 62228 0.45 -100.00%
benchmark old allocs new allocs delta
BenchmarkIsPubKeyHashScript-8 1 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkIsPubKeyHashScript-8 311299 0 -100.00%
This converts the IsPayToScriptHash function to analyze the raw script
instead of using the far less efficient parseScript, thereby
significantly optimizing the function.
In order to accomplish this, it introduces four new functions:
extractCompressedPubKey, extractUncompressedPubKey, extractPubKey, and
isPubKeyScript. The extractPubKey function makes use of
extractCompressedPubKey and extractUncompressedPubKey to combine their
functionality as a convenience and isPubKeyScript is defined in terms of
extractPubKey.
The extractCompressedPubKey works with the raw script bytes to
simultaneously determine if the script is a pay-to-compressed-pubkey
script, and in the case it is, extract and return the raw compressed
pubkey bytes.
Similarly, the extractUncompressedPubKey works in the same way except it
determines if the script is a pay-to-uncompressed-pubkey script and
returns the raw uncompressed pubkey bytes in the case it is.
The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type. Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.
The following is a before and after comparison of analyzing a large
script:
benchmark old ns/op new ns/op delta
BenchmarkIsPubKeyScript-8 62323 2.97 -100.00%
benchmark old allocs new allocs delta
BenchmarkIsPubKeyScript-8 1 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkIsPubKeyScript-8 311299 0 -100.00%
This converts the asSmallInt function to accept an opcode as a byte
instead of the internal opcode data struct in order to make it more
flexible for raw script analysis.
It also updates all callers accordingly.
This converts the isSmallInt function to accept an opcode as a byte
instead of the internal opcode data struct in order to make it more
flexible for raw script analysis.
The comment is modified to explicitly call out the script version
semantics.
Finally, it updates all callers accordingly.
This commit implements the flag activation portion of BIP 0147. The
verification behavior triggered by the NULLDUMMY script verification
flag has been present within btcd for some time, however it wasn’t
activated by default.
With this commit, once segwit has activated, the ScriptStrictMultiSig
will also be activated within the Script VM. Additionally, the
ScriptStrictMultiSig is now a standard script verification flag which
is used unconditionally within the mempool.
This commit implements full witness program validation for the
currently defined version 0 witness programs. This includes validation
logic for nested p2sh, p2wsh, and p2wkh. Additionally, when in witness
validation mode, an additional set of constrains are enforced such as
using the new sighash digest algorithm and enforcing clean stack
behavior within witness programs.
This commit introduces a series of internal and external helper
functions which enable the txscript package to be aware of the new
standard script templates introduced as part of BIP0141. The two new
standard script templates recognized are pay-to-witness-key-hash
(P2WKH) and pay-to-witness-script-hash (P2WSH).
ScriptVerifyNullFail defines that signatures must be empty if a
CHECKSIG or CHECKMULTISIG operation fails.
This commit also enables ScriptVerifyNullFail at the mempool policy
level.
This converts the majority of script errors from generic errors created
via errors.New and fmt.Errorf to use a concrete type that implements the
error interface with an error code and description.
This allows callers to programmatically detect the type of error via
type assertions and an error code while still allowing the errors to
provide more context.
For example, instead of just having an error the reads "disabled opcode"
as would happen prior to these changes when a disabled opcode is
encountered, the error will now read "attempt to execute disabled opcode
OP_FOO".
While it was previously possible to programmatically detect many errors
due to them being exported, they provided no additional context and
there were also various instances that were just returning errors
created on the spot which callers could not reliably detect without
resorting to looking at the actual error message, which is nearly always
bad practice.
Also, while here, export the MaxStackSize and MaxScriptSize constants
since they can be useful for consumers of the package and perform some
minor cleanup of some of the tests.
This modifies the recently-added NullDataScript function in several
ways in an effort to make them more consistent with the tests in the
rest of the code base and improve/correct the logic:
- Use the hexToBytes and mustParseShortForm functions
- Consistently format the test errors
- Replace the valid bool flag with an expected error and test against it
- Ensure the returned script type is the expected type in all cases
This adds a new function named NullDataScript to the txscript package that returns a provably-pruneable OP_RETURN script with the provided data. The function will return an error if the provided data is larger than the maximum allowed length for a nulldata script to be be considered standard.
This corrects the isNullData standard transaction type test to work
properly with canonically-encoded data pushes. In particular, single
byte data pushes that are small integers (0-16) are converted to the
equivalent numeric opcodes when canonically encoded and the code failed
to detect them properly.
It also adds several tests to ensure that both canonical and
non-canonical nulldata scripts are recognized properly and modifies the
test failure print to include the script that failed.
This does not affect consensus since it is just a standardness check.
isMultiSig was not verifying the number of pubkeys specified matched
the number of pubkeys provided. This caused certain non-standard
scripts to be considered multisig scripts.
However, the script still would have failed during execution.
NOTE: This only affects whether or not the script is considered
standard and does NOT affect consensus.
Also, add a test for this check.
See https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki for
more information.
This commit mimics Bitcoin Core commit bc60b2b4b401f0adff5b8b9678903ff8feb5867b
and includes additional tests from Bitcoin Core commit
cb54d17355864fa08826d6511a0d7692b21ef2c9
We've already been generating lowS sigs for quite a while. This removes
the malleability vector.
This mimics Bitcoin Core commit 49dd5c629df0a08cf3b1ea8085c03312d1a81696
- Move reference tests to test package since they are intended to
exercise the engine as callers would
- Improve the short form script parsing to allow additional opcodes:
DATA_#, OP_#, FALSE, TRUE
- Make use of a function to decode hex strings rather than manually
defining byte slices
- Update the tests to make use of the short form script parsing logic
rather than manually defining byte slices
- Consistently replace all []byte{} and [][]byte{} with nil
- Define tests only used in a specific function inside that func
- Move invalid flag combination test to engine_test since that is what
it is testing
- Remove all redundant script tests in favor of the JSON-based tests in
the data directory.
- Move several functions from internal_test.go to the test files
associated with what the tests are checking
