Commit graph

204 commits

Author SHA1 Message Date
Dave Collins
2b5edd2b5e txscript: Add benchmark for IsPushOnlyScript. 2022-05-23 21:46:21 -07:00
Dave Collins
6be04a8e43 txscript: Optimize IsMultisigSigScript.
This converts the IsMultisigSigScript function to analyze the raw script
and make use of the new tokenizer instead of the far less efficient
parseScript thereby significantly optimizing the function.

In order to accomplish this, it first rejects scripts that can't
possibly fit the bill due to the final byte of what would be the redeem
script not being the appropriate opcode or the overall script not having
enough bytes.  Then, it uses a new function that is introduced named
finalOpcodeData that uses the tokenizer to return any data associated
with the final opcode in the signature script (which will be nil for
non-push opcodes or if the script fails to parse) and analyzes it as if
it were a redeem script when it is non nil.

It is also worth noting that this new implementation intentionally has
the same semantic difference from the existing implementation as the
updated IsMultisigScript function in regards to allowing zero pubkeys
whereas previously it incorrectly required at least one pubkey.

Finally, the comment is modified to explicitly call out the script
version semantics.

The following is a before and after comparison of analyzing a large
script that is not a multisig script and both a 1-of-2 multisig public
key script (which should be false) and a signature script comprised of a
pay-to-script-hash 1-of-2 multisig redeem script (which should be true):

benchmark                               old ns/op     new ns/op     delta
BenchmarkIsMultisigSigScriptLarge-8     69328         2.93          -100.00%
BenchmarkIsMultisigSigScript-8          2375          146           -93.85%

benchmark                               old allocs     new allocs     delta
BenchmarkIsMultisigSigScriptLarge-8     5              0              -100.00%
BenchmarkIsMultisigSigScript-8          3              0              -100.00%

benchmark                               old bytes     new bytes     delta
BenchmarkIsMultisigSigScriptLarge-8     330035        0             -100.00%
BenchmarkIsMultisigSigScript-8          9472          0             -100.00%
2022-05-23 21:46:21 -07:00
Dave Collins
3bdeaa46bf txscript: Add benchmarks for IsMutlsigSigScript. 2022-05-23 21:46:21 -07:00
Dave Collins
67e2cbe374 txscript: Optimize IsMultisigScript.
This converts the IsMultisigScript function to make use of the new
tokenizer instead of the far less efficient parseScript thereby
significantly optimizing the function.

In order to accomplish this, it introduces two new functions.  The first
one is named extractMultisigScriptDetails and works with the raw script
bytes to simultaneously determine if the script is a multisignature
script, and in the case it is, extract and return the relevant details.
The second new function is named isMultisigScript and is defined in
terms of the former.

The extract function accepts the script version, raw script bytes, and a
flag to determine whether or not the public keys should also be
extracted.  The flag is provided because extracting pubkeys results in
an allocation that the caller might wish to avoid.

The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type.  Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.

It is important to note that this new implementation intentionally has a
semantic difference from the existing implementation in that it will now
correctly identify a multisig script with zero pubkeys whereas
previously it incorrectly required at least one pubkey.  This change is
acceptable because the function only deals with standardness rather than
consensus rules.

Finally, this also deprecates the isMultiSig function that requires
opcodes in favor of the new functions and deprecates the error return on
the export IsMultisigScript function since it really does not make sense
given the purpose of the function.

The following is a before and after comparison of analyzing both a large
script that is not a multisig script and a 1-of-2 multisig public key
script:

benchmark                            old ns/op     new ns/op     delta
BenchmarkIsMultisigScriptLarge-8     64166         5.52          -99.99%
BenchmarkIsMultisigScript-8          630           59.4          -90.57%

benchmark                            old allocs     new allocs     delta
BenchmarkIsMultisigScriptLarge-8     1              0              -100.00%
BenchmarkIsMultisigScript-8          1              0              -100.00%

benchmark                            old bytes     new bytes     delta
BenchmarkIsMultisigScriptLarge-8     311299        0             -100.00%
BenchmarkIsMultisigScript-8          2304          0             -100.00%
2022-05-23 21:46:21 -07:00
Dave Collins
a6244b516d txscript: Add benchmarks for IsMutlsigScript. 2022-05-23 21:46:21 -07:00
Dave Collins
69aefa65e6 txscript: Optimize IsPayToScriptHash.
This converts the IsPayToScriptHash function to analyze the raw script
instead of using the far less efficient parseScript thereby
significantly optimizing the function.

In order to accomplish this, it introduces two new functions.  The first
one is named extractScriptHash and works with the raw script bytes to
simultaneously determine if the script is a p2sh script, and in the case
it is, extract and return the hash.  The second new function is named
isScriptHashScript and is defined in terms of the former.

The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type.  Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.

Finally, this also deprecates the isScriptHash function that requires
opcodes in favor of the new functions and modifies the comment on
IsPayToScriptHash to explicitly call out the script version semantics.

The following is a before and after comparison of analyzing a large
script that is not a p2sh script:

benchmark                        old ns/op     new ns/op     delta
BenchmarkIsPayToScriptHash-8     62393         0.60          -100.00%

benchmark                        old allocs     new allocs     delta
BenchmarkIsPayToScriptHash-8     1              0              -100.00%

benchmark                        old bytes     new bytes     delta
BenchmarkIsPayToScriptHash-8     311299        0             -100.00%
2022-05-23 21:46:21 -07:00
Dave Collins
e7228a2e5f txscript: Add benchmark for IsPayToScriptHash. 2022-05-23 21:46:21 -07:00
Conner Fromknecht
eb03d84098 txscript: Optimize IsPayToPubKeyHash
This converts the IsPayToPubKeyHash function to analyze the raw script
instead of using the far less efficient parseScript, thereby
significantly optimization the function.

In order to accomplish this, it introduces two new functions.  The first
one is named extractPubKeyHash and works with the raw script bytes
to simultaneously determine if the script is a pay-to-pubkey-hash script,
and in the case it is, extract and return the hash.  The second new
function is named isPubKeyHashScript and is defined in terms of the
former.

The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type.  Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.

The following is a before and after comparison of analyzing a large
script:

benchmark                         old ns/op     new ns/op     delta
BenchmarkIsPubKeyHashScript-8     62228         0.45          -100.00%

benchmark                         old allocs     new allocs     delta
BenchmarkIsPubKeyHashScript-8     1              0              -100.00%

benchmark                         old bytes     new bytes     delta
BenchmarkIsPubKeyHashScript-8     311299        0             -100.00%
2022-05-23 21:46:21 -07:00
Conner Fromknecht
c4da180d4f txscript: Add benchmark for IsPayToPubKeyHash 2022-05-23 21:46:21 -07:00
Conner Fromknecht
5d18558bc6 txscript: Optimize IsPayToPubKey
This converts the IsPayToScriptHash function to analyze the raw script
instead of using the far less efficient parseScript, thereby
significantly optimizing the function.

In order to accomplish this, it introduces four new functions:
extractCompressedPubKey, extractUncompressedPubKey, extractPubKey, and
isPubKeyScript.  The extractPubKey function makes use of
extractCompressedPubKey and extractUncompressedPubKey to combine their
functionality as a convenience and isPubKeyScript is defined in terms of
extractPubKey.

The extractCompressedPubKey works with the raw script bytes to
simultaneously determine if the script is a pay-to-compressed-pubkey
script, and in the case it is, extract and return the raw compressed
pubkey bytes.

Similarly, the extractUncompressedPubKey works in the same way except it
determines if the script is a pay-to-uncompressed-pubkey script and
returns the raw uncompressed pubkey bytes in the case it is.

The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type.  Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.

The following is a before and after comparison of analyzing a large
script:

benchmark                     old ns/op     new ns/op     delta
BenchmarkIsPubKeyScript-8     62323         2.97          -100.00%

benchmark                     old allocs     new allocs     delta
BenchmarkIsPubKeyScript-8     1              0              -100.00%

benchmark                     old bytes     new bytes     delta
BenchmarkIsPubKeyScript-8     311299        0             -100.00%
2022-05-23 21:46:21 -07:00
Dave Collins
28eaf3492d txscript: Add benchmark for IsPayToPubKey 2022-05-23 21:46:21 -07:00
Dave Collins
8c68575331 txscript: Make asSmallInt accept raw opcode.
This converts the asSmallInt function to accept an opcode as a byte
instead of the internal opcode data struct in order to make it more
flexible for raw script analysis.

It also updates all callers accordingly.
2022-05-23 21:46:21 -07:00
Dave Collins
45de22d457 txscript: Make isSmallInt accept raw opcode.
This converts the isSmallInt function to accept an opcode as a byte
instead of the internal opcode data struct in order to make it more
flexible for raw script analysis.

The comment is modified to explicitly call out the script version
semantics.

Finally, it updates all callers accordingly.
2022-05-23 21:46:21 -07:00
Conner Fromknecht
a02b71bcf9 txscript/reference_test: Convert sighash calc test
This converts the tests for calculating signature hashes to use the
exported function which handles the raw script versus the now deprecated
variant requiring parsed opcodes.

Backport of 06f769ef72e6042e7f2b5ff1c512ef1371d615e5
2022-05-23 21:46:21 -07:00
Dave Collins
18aa1a59df txscript: Optimize CalcSignatureHash.
This modifies the CalcSignatureHash function to make use of the new
signature hash calculation function that accepts raw scripts without
needing to first parse them.  Consequently, it also doubles as a slight
optimization to the execution time and a significant reduction in the
number of allocations.

In order to convert the CalcScriptHash function and keep the same
semantics, a new function named checkScriptParses is introduced which
will quickly determine if a script can be fully parsed without failure
and return the parse failure in the case it can't.

The following is a before and after comparison of analyzing a large
multiple input transaction:

benchmark                  old ns/op     new ns/op     delta
BenchmarkCalcSigHash-8     3627895       3619477       -0.23%

benchmark                  old allocs     new allocs     delta
BenchmarkCalcSigHash-8     1335           801            -40.00%

benchmark                  old bytes     new bytes     delta
BenchmarkCalcSigHash-8     1373812       1293354       -5.86%
2022-05-23 21:46:21 -07:00
Conner Fromknecht
07c1a9343d txscript: Introduce raw script sighash calc func.
This introduces a new function named calcSignatureHashRaw which accepts
the raw script bytes to calculate the script hash versus requiring the
parsed opcode only to unparse them later in order to make it more
flexible for working with raw scripts.

Since there are several places in the rest of the code that currently
only have access to the parsed opcodes, this modifies the existing
calcSignatureHash to first unparse the script before calling the new
function.

Backport of decred/dcrd:f306a72a16eaabfb7054a26f9d9f850b87b00279
2022-05-23 21:46:21 -07:00
Dave Collins
ce08988514 txscript: Optimize script disasm.
This converts the DisasmString function to make use of the new
zero-allocation script tokenizer instead of the far less efficient
parseScript thereby significantly optimizing the function.

In order to facilitate this, the opcode disassembly functionality is
split into a separate function called disasmOpcode that accepts the
opcode struct and data independently as opposed to requiring a parsed
opcode.  The new function also accepts a pointer to a string builder so
the disassembly can be more efficiently be built.

While here, the comment is modified to explicitly call out the script
version semantics.

The following is a before and after comparison of a large script:

benchmark                   old ns/op     new ns/op     delta
BenchmarkDisasmString-8     102902        40124         -61.01%

benchmark                   old allocs     new allocs     delta
BenchmarkDisasmString-8     46             51             +10.87%

benchmark                   old bytes     new bytes     delta
BenchmarkDisasmString-8     389324        130552        -66.47%
2022-05-23 21:46:20 -07:00
Dave Collins
94bb41664b txscript: Add benchmark for DisasmString. 2022-05-23 21:46:20 -07:00
Dave Collins
ac002d6422 txscript: Introduce zero-alloc script tokenizer.
This implements an efficient and zero-allocation script tokenizer that
is exported to both provide a new capability to tokenize scripts to
external consumers of the API as well as to serve as a base for
refactoring the existing highly inefficient internal code.

It is important to note that this tokenizer is intended to be used in
consensus critical code in the future, so it must exactly follow the
existing semantics.

The current script parsing mechanism used throughout the txscript module
is to fully tokenize the scripts into an array of internal parsed
opcodes which are then examined and passed around in order to implement
virtually everything related to scripts.

While that approach does simplify the analysis of certain scripts and
thus provide some nice properties in that regard, it is both extremely
inefficient in many cases, and makes it impossible for external
consumers of the API to implement any form of custom script analysis
without manually implementing a bunch of error prone tokenizing code or,
alternatively, the script engine exposing internal structures.

For example, as shown by profiling the total memory allocations of an
initial sync, the existing script parsing code allocates a total of
around 295.12GB, which equates to around 50% of all allocations
performed.  The zero-alloc tokenizer this introduces will allow that to
be reduced to virtually zero.

The following is a before and after comparison of tokenizing a large
script with a high opcode count using the existing code versus the
tokenizer this introduces for both speed and memory allocations:

benchmark                    old ns/op     new ns/op     delta
BenchmarkScriptParsing-8     63464         677           -98.93%

benchmark                    old allocs     new allocs     delta
BenchmarkScriptParsing-8     1              0              -100.00%

benchmark                    old bytes     new bytes     delta
BenchmarkScriptParsing-8     311299        0             -100.00%

The following is an overview of the changes:

- Introduce new error code ErrUnsupportedScriptVersion
- Implement zero-allocation script tokenizer
- Add a full suite of tests to ensure the tokenizer works as intended
  and follows the required consensus semantics
- Add an example of using the new tokenizer to count the number of
  opcodes in a script
- Update README.md to include the new example
- Update script parsing benchmark to use the new tokenizer
2022-05-23 21:46:20 -07:00
Dave Collins
fc5b1a817c txscript: Add benchmark for script parsing. 2022-05-23 21:46:20 -07:00
Conner Fromknecht
b2784102f4 txscript: Add benchmark for CalcWitnessSigHash 2022-05-23 21:46:20 -07:00
Dave Collins
42f4b4025c txscript: Add benchmark for CalcSignatureHash 2022-05-23 21:46:20 -07:00
Jake Sylvestre
d08785547a docs: update shields 2021-03-05 07:45:19 -05:00
Conner Fromknecht
5300a19d06
txscript/hashcache_test: call rand.Seed once in init
This resolves the more fundamental flake in the unit tests noted in the
prior commit.

Because multiple unit tests call rand.Seed in parallel, it's possible
they can be executed with the same unix timestamp (in seconds). If the
second call happens between generating the hash cache and checking that
the cache doesn't contain a random txn, the random transaction is in
fact a duplicate of one generated earlier since the RNG state was reset.

To remedy, we initialize rand.Seed once in the init function.
2021-02-02 13:31:47 -08:00
Conner Fromknecht
1dd693480c
txscript/hashcache_test: always add inputs during getTxn
TestHashCacheAddContainsHashes flakes fairly regularly when rebasing
PR #1684 with:
    txid <txid> wasn't inserted into cache but was found.

With probabilty 1/10^2 there will be no inputs on the transaction. This
reduces the entropy in the txid, and I belive is the primary cause of
the flake.
2021-02-02 12:44:22 -08:00
Dan Cline
77fd96753c txscript: add benchmark for IsUnspendable
- create benchmarks to measure allocations
 - add test for benchmark input
 - create a low alloc parseScriptTemplate
 - refactor parsing logic for a single opcode
2021-02-02 09:20:31 -05:00
Anirudha Bose
ac3f235eb9 rpcclient: implement getaddressinfo command
Fields such as label, and labelspurpose are not included, since they
are deprecated, and will be removed in Bitcoin Core 0.21.
2020-09-21 09:47:58 -04:00
David Hill
f7399e6157 build: clean linter warnings 2020-05-13 08:58:39 -04:00
David Hill
a505b99ba3 build: replace travis-ci with github actions.
test go 1.14
use golangci-lint
2020-05-13 08:52:05 -04:00
Tyler Chambers
1d0bfca5b0 fix error message 2020-03-05 16:51:41 -05:00
Wilmer Paulino
5328af0b63
txscript: refactor ComputePkScript 2019-06-03 13:55:28 -07:00
Wilmer Paulino
545bc5d474
txscript: handle variable length P2PKH signatures in ComputePkScript
Since P2PKH signatures have variable lengths, we would attempt to parse
P2PKH scripts as P2SH if they didn't fit the previous length
constraints.
2019-06-03 13:55:24 -07:00
Wilmer Paulino
0cec774a75
txscript: add support to re-derive output's PkScript from input
In this commit, we extend the txscript package to support re-deriving
the PkScript of an output by looking at the input's signature
script/witness attempting to spend it. As of this commit, the only
supported types are P2SH, v0 P2WSH, and v0 P2WPKH.

This will serve useful to detect when a particular script has been spent
on-chain.

A set of test vectors has also been added for the supported script types
to ensure its correctness.
2019-01-11 18:30:42 -08:00
Dave Collins
07edce81b0
txscript: Cleanup strict signature enforcement.
This cleans up the code for handling the checksig and checkmultisig
opcode strict signatures to explicitly call out any semantics that are
likely not obvious and improve readability.

It also introduce new distinct errors for each condition which can
result in a signature being rejected due to not following the strict
encoding requirements and updates reference test adaptor accordingly.
2018-08-23 11:56:28 -05:00
Olaoluwa Osuntokun
b72e16f0d6 multi: correct all import paths 2018-05-23 16:46:15 -07:00
Olaoluwa Osuntokun
3d1caa2f83 multi: update to point to roasbeef forks 2018-05-23 16:46:15 -07:00
Olaoluwa Osuntokun
4a6dc67067 txscript: add exported sighash calc func 2018-05-23 16:46:15 -07:00
Josh Rickmar
1cd648d784 Require atomic swap contracts to specify the secret size.
This allows redeeming parties to audit the secret size and ensure it
will be usable once revealed.
2018-02-16 16:18:43 -05:00
Nicola 'tekNico' Larosa
11fcd83963 btcd/multi: fix a number of typos in comments. 2018-01-25 23:23:59 -06:00
Marko Bencun
16dbb2602a txscript: export calcSignatureHash
This is a useful function for users of this library, and deserves to
be public.
2018-01-25 23:14:55 -06:00
Shuai Qi
b6afec5e51 txscript: Fix comment typo. 2018-01-25 22:37:50 -06:00
Josh Rickmar
2e60448ffc txscript: Require SHA256 secret hashes for atomic swaps 2017-11-28 10:07:13 -05:00
Janus Troelsen
8cea3866d0 Update script_test.go
Typo fix
2017-10-26 14:26:30 +02:00
Josh Rickmar
4803a8291c txscript: Add API to parse atomic swap contracts. 2017-09-20 12:44:35 -05:00
Alex Bosworth
63d1550d42 txscript: Trivial typo fixes. 2017-09-08 12:56:38 -05:00
Dave Collins
6b802379ec
txscript: Shallow tx copy for signature hash calc.
This modifies calcSignatureHash to use a shallow copy of the transaction
versus a deep copy since the actual scripts themselves are not modified
and therefore don't need to be copied.

This is being done because profiling the most overall allocated space
shows that the deep copy performed in calcSignatureHash accounts for
nearly 20% of all allocations on a synced running instance.  Also,
copying all of the additional data makes it more time consuming as well.

With this change, that figure drops from ~20% to ~5% of all allocations.

The following benchmark shows the relative speedups and allocation
reduction as a result of the optimization on my system.  In particular,
the changes result in approximately a 15% speedup and a whopping 99.89%
reduction in allocations when using a large transaction with thousands
of inputs which was the worst case scenario.

benchmark                        old allocs    new allocs    delta
--------------------------------------------------------------------
BenchmarkCalcSignatureHash       11151         12            -99.89%

benchmark                        old ns/op     new ns/op     delta
--------------------------------------------------------------------
BenchmarkCalcSignatureHash       3599845       3056359       -15.10%
2017-08-24 12:48:11 -05:00
Olaoluwa Osuntokun
3b4b4e7942 txscript: update reference tests to include new segwit related cases 2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
ff6cb25e89 txscript: convert all new segwit error types to ErrorCode's 2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
80dd96ac5d txscript: add new post segwit policies to standard script flags 2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
0a7bbda6dd txscript: add verification of the post-segwit pub key type constraint
This commit adds verification of the post-segwit standardness
requirement that all pubkeys involved in checks operations MUST be
serialized as compressed public keys. A new ScriptFlag has been added
to guard this behavior when executing scripts.
2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
9367aedfd7 txscript: add verification of the post-segwit minimal if policy
This commit modifies the op-code execution for OP_IF and OP_NOTIF to
enforce the additional “minimal if” constraints which require the
top-stack item when the op codes are encountered to be either an empty
vector, or exactly [0x01].
2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
9054ef8354 BIP 147: enforce NULLDUMMY w/ segwit verification
This commit implements the flag activation portion of BIP 0147. The
verification behavior triggered by the NULLDUMMY script verification
flag has been present within btcd for some time, however it wasn’t
activated by default.

With this commit, once segwit has activated, the ScriptStrictMultiSig
will also be activated within the Script VM. Additionally, the
ScriptStrictMultiSig is now a standard script verification flag which
is used unconditionally within the mempool.
2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
aaf187427e BIP0141+txscript: implement witness program validation
This commit implements full witness program validation for the
currently defined version 0 witness programs. This includes validation
logic for nested p2sh, p2wsh, and p2wkh. Additionally, when in witness
validation mode, an additional set of constrains are enforced such as
using the new sighash digest algorithm and enforcing clean stack
behavior within witness programs.
2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
b564111aff txscript: fix off-by-one error due to new OP_CODESEPARATOR behavior in segwit
This commit fixes an off-by-one error which is only manifested by the
new behavior of OP_CODESEPARATOR within sig hashes triggered by the
segwit behavior. The current behavior within the Script VM
(txscript.Engine) is known to be fully correct to the extent that it has
been verified. However, once segwit activates a consensus divergence
would emerge due to *when* the program counter was incremented in the
previous code (pre-this-commit).

Currently (pre-segwit) when calculating the pre-image to a transaction
sighash for signature verification, *all* instances of OP_CODESEPARATOR
are removed from the subScript being signed before generating the final
sighash. SegWit has additional nerfed the behavior of OP_CODESEPARATOR
by no longer removing them (and starting after the last instance), but
instead simply starting the subScript to be directly *after* the last
instance of an OP_CODESEPARATOR within the pkScript.

Due to this new behavior, without this commit, an off-by-one error
(which only matters post-segwit), would cause txscript to generate an
incorrect subScript since the instance of OP_CODESEPARATOR would remain
as part of the subScript instead of being sliced off as the new behavior
dictates. The off-by-one error itself is manifested due to a slight
divergence in txscript.Engine’s logic compared to Bitcoin Core.  In
Bitcoin Core script verification is as follows: first the next op-code
is fetched, then program counter is incremented, and finally the op-code
itself is executed. Before this commit, btcd flipped the order
of the last two steps, executing the op-code *before* the program
counter was incremented.

This commit fixes the post-segwit consensus divergence by incrementing
the program-counter *before* the next op-code is executed. It is
important to note that this divergence is only significant post-segwit,
meaning that txscript.Engine is still consensus compliant independent of
this commit.
2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
653459c810 BIP0141+txscript: implement signature operation cost calculations 2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
469e53ca27 BIP0141+txscript: awareness of new standard script templates, add helper funcs
This commit introduces a series of internal and external helper
functions which enable the txscript package to be aware of the new
standard script templates introduced as part of BIP0141. The two new
standard script templates recognized are pay-to-witness-key-hash
(P2WKH) and pay-to-witness-script-hash (P2WSH).
2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
98cae74275 BIP0143+txscript: add segwit sighash, signing, and HashCache integration
This commit implements most of BIP0143 by adding logic to implement the
new sighash calculation, signing, and additionally introduces the
HashCache optimization which eliminates the O(N^2) computational
complexity for the SIGHASH_ALL sighash type.

The HashCache struct is the equivalent to the existing SigCache struct,
but for caching the reusable midstate for transactions which are
spending segwitty outputs.
2017-08-13 23:17:40 -05:00
Josh Rickmar
a6965d493f all: Remove seelog logger.
The btclog package has been changed to defining its own logging
interface (rather than seelog's) and provides a default implementation
for callers to use.

There are two primary advantages to the new logger implementation.

First, all log messages are created before the call returns.  Compared
to seelog, this prevents data races when mutable variables are logged.

Second, the new logger does not implement any kind of artifical rate
limiting (what seelog refers to as "adaptive logging").  Log messages
are outputted as soon as possible and the application will appear to
perform much better when watching standard output.

Because log rotation is not a feature of the btclog logging
implementation, it is handled by the main package by importing a file
rotation package that provides an io.Reader interface for creating
output to a rotating file output.  The rotator has been configured
with the same defaults that btcd previously used in the seelog config
(10MB file limits with maximum of 3 rolls) but now compresses newly
created roll files.  Due to the high compressibility of log text, the
compressed files typically reduce to around 15-30% of the original
10MB file.
2017-06-19 16:46:50 -04:00
Steven Roose
bf43e56f2f Fix warnings from ineffassign
I left one at the end of fullblocktest, since I suspected the unused
variable assignments there were set for the possibility of extending the
tests.
2017-06-07 17:59:33 -05:00
Dave Collins
9918e2a561
multi: Update markdown files for GFM changes.
The github markdown interpreter has been changed such that it no longer
allows spaces in between the brackets and parenthesis of links and now
requires a newline in between anchors and other formatting.  This
updates all of the markdown files accordingly.

While here, it also corrects a couple of inconsistencies in some of the
README.md files.
2017-05-25 12:06:16 -05:00
Dave Collins
0ea4a6ebd4
multi: Switch to upstream golang.org/x/crypto.
Now that glide is used for version management and a specific commit of
the upstream repository can be locked it is no longer necessary to
maintain a fork of the package specifically to keep a stable dependency.

While here, update the glide dependency for btcutil as well since it was
switched to use the upstream path as well.
2017-05-09 11:35:01 -05:00
Dave Collins
efa50e6abc
multi: Simplify code per gosimple linter.
This simplifies the code based on the recommendations of the gosimple
lint tool.

Also, it increases the deadline for the linters to run to 10 minutes and
reduces the number of threads that is uses. This is being done because
the Travis environment has become increasingly slower and it also seems
to be hampered by too many threads running concurrently.
2017-03-22 15:34:13 -05:00
David Hill
9f71f090e6 txscript: Drop the mutex before doing crypto 2017-01-31 13:47:41 -05:00
David Hill
0efea24aa6 txscript: Implement ScriptVerifyNullFail
ScriptVerifyNullFail defines that signatures must be empty if a
CHECKSIG or CHECKMULTISIG operation fails.

This commit also enables ScriptVerifyNullFail at the mempool policy
level.
2017-01-13 14:19:11 -05:00
Dave Collins
153dca5c1e
txscript: Convert reference tests to new format.
This updates the data driven transaction script tests to use the most
recent format and test data as implemented by Core so the test data can
more easily be updated and help prove cross-compatibility correctness.

In particular, the new format combines the previously separate valid and
invalid test data files into a single file and adds a field for the
expected result.  This is a nice improvement since it means tests can
now ensure script failures are due to a specific expected reason as
opposed to only generically detecting failure as the previous format
required.

The btcd script engine typically returns more fine grained errors than
the test data expects, so the test adapter handles this by allowing
expected errors in the test data to be mapped to multiple txscript
errors.

It should also be noted that the tests related to segwit have been
stripped from the data since the segwit PR has not landed in master yet,
however the test adapter does recognize the new ability for optional
segwit data to be supplied, though it will need to properly construct
the transaction using that data when the time comes.
2017-01-12 13:13:21 -06:00
Dave Collins
fdc2bc867b
txscript: Significantly improve errors.
This converts the majority of script errors from generic errors created
via errors.New and fmt.Errorf to use a concrete type that implements the
error interface with an error code and description.

This allows callers to programmatically detect the type of error via
type assertions and an error code while still allowing the errors to
provide more context.

For example, instead of just having an error the reads "disabled opcode"
as would happen prior to these changes when a disabled opcode is
encountered, the error will now read "attempt to execute disabled opcode
OP_FOO".

While it was previously possible to programmatically detect many errors
due to them being exported, they provided no additional context and
there were also various instances that were just returning errors
created on the spot which callers could not reliably detect without
resorting to looking at the actual error message, which is nearly always
bad practice.

Also, while here, export the MaxStackSize and MaxScriptSize constants
since they can be useful for consumers of the package and perform some
minor cleanup of some of the tests.
2017-01-12 13:12:39 -06:00
David Hill
ab0f30c00d mining: drop getwork support.
Since the Midstate is no longer needed, switch to using
crypto/sha256.
2017-01-11 13:51:57 -05:00
Dave Collins
bc576b13b4
txscript: Only do CSV txver check if enabled.
The CSV consensus rules dictate that the opcode fails when the
transaction version is not at least version 2, however that only applies
if the disable flag is not set in the sequence.

This is not an issue at the current time because we do not yet enforce
CSV at a consensus level, however, I noticed this discrepancy when doing
a thorough audit of the CSV paths due to the ongoing work to add full
consensus-enforced CSV support.

As a result, this must be merged prior to enabling consensus enforcement
for CSV or it would open up the potential for a hard fork.
2016-12-03 12:33:57 -06:00
David Hill
b134beb3b7 txscript: reduce allocs in calcSignatureHash 2016-11-16 12:28:22 -06:00
David Hill
807d344fe9 Unassign some TODO's 2016-11-15 17:47:33 -06:00
Dave Collins
915fa6639b
multi: Simplify code per gosimple linter.
This simplifies the code based on the recommendations of the gosimple
lint tool.
2016-11-03 13:00:35 -05:00
Dave Collins
af524fb3e7
multi: Remove unnecessary convs found by unconvert.
This removes all unnecessary typecast conversions as found by the
unconvert linter.
2016-11-03 11:59:38 -05:00
Dave Collins
14b51fc5f8
multi: Correct misspellings detected by misspell. 2016-10-28 09:43:38 -05:00
Dave Collins
f6ad7eb2c9
wire: Make NewMsgTx accept the tx version.
This modifies the NewMsgTx function to accept the transaction version as
a parameter and updates all callers.

The reason for this change is so the transaction version can be bumped
in wire without breaking existing tests and to provide the caller with
the flexibility to create the specific transaction version they desire.
2016-10-27 14:09:29 -05:00
David Hill
4494f0f852 txscript: Remove OP_SMALLDATA 2016-10-21 15:18:42 -04:00
Dave Collins
0731f2ddc9 txscript: Cleanup and improve NullDataScript tests.
This modifies the recently-added NullDataScript function in several
ways in an effort to make them more consistent with the tests in the
rest of the code base and improve/correct the logic:

- Use the hexToBytes and mustParseShortForm functions
- Consistently format the test errors
- Replace the valid bool flag with an expected error and test against it
- Ensure the returned script type is the expected type in all cases
2016-10-21 10:26:09 -05:00
DanielKrawisz
b77654f8d4 txscript: Add null data script creator
This adds a new function named NullDataScript to the txscript package that returns a provably-pruneable  OP_RETURN script with the provided data.  The function will return an error if the provided data is larger than the maximum allowed length for a nulldata script to be be considered standard.
2016-10-21 09:37:48 -05:00
Dave Collins
59a3fc2f66 txscript: Consolidate tests into txscript package.
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.

Also, do some light cleanup on a few tests while here.
2016-10-20 09:28:33 -05:00
Dave Collins
b60e3547d2 txscript: Correct nulldata standardness check.
This corrects the isNullData standard transaction type test to work
properly with canonically-encoded data pushes.  In particular, single
byte data pushes that are small integers (0-16) are converted to the
equivalent numeric opcodes when canonically encoded and the code failed
to detect them properly.

It also adds several tests to ensure that both canonical and
non-canonical nulldata scripts are recognized properly and modifies the
test failure print to include the script that failed.

This does not affect consensus since it is just a standardness check.
2016-10-20 01:44:58 -05:00
David Hill
a6bf1d9850 txscript: Implement CheckSequenceVerify (BIP0112) 2016-10-19 12:06:44 -04:00
Dave Collins
cee207c64c txscript: Expose AddOps on ScriptBuilder. (#734)
This exposes a new function on the ScriptBuilder type named AddOps that
allows multiple opcodes to be added via a single call and adds tests to
exercise the new function.

Finally, it updates a couple of places in the signing code that were
abusing the interface by setting its private script directly to use the
new public function instead.
2016-08-12 19:29:28 -05:00
Dave Collins
bd4e64d1d4 chainhash: Abstract hash logic to new package. (#729)
This is mostly a backport of some of the same modifications made in
Decred along with a few additional things cleaned up.  In particular,
this updates the code to make use of the new chainhash package.

Also, since this required API changes anyways and the hash algorithm is
no longer tied specifically to SHA, all other functions throughout the
code base which had "Sha" in their name have been changed to Hash so
they are not incorrectly implying the hash algorithm.

The following is an overview of the changes:

- Remove the wire.ShaHash type
- Update all references to wire.ShaHash to the new chainhash.Hash type
- Rename the following functions and update all references:
  - wire.BlockHeader.BlockSha -> BlockHash
  - wire.MsgBlock.BlockSha -> BlockHash
  - wire.MsgBlock.TxShas -> TxHashes
  - wire.MsgTx.TxSha -> TxHash
  - blockchain.ShaHashToBig -> HashToBig
  - peer.ShaFunc -> peer.HashFunc
- Rename all variables that included sha in their name to include hash
  instead
- Update for function name changes in other dependent packages such as
  btcutil
- Update copyright dates on all modified files
- Update glide.lock file to use the required version of btcutil
2016-08-08 14:04:33 -05:00
Mawueli Kofi Adzoe
7f07fb1093 txscript: Fix typo. (#700)
* Fix tiny typo. Bump copyright year.
* Clarify documentation.
2016-05-22 23:23:20 -05:00
Dave Collins
644570487f txscript: Reduce script parse opcode allocs. (#677)
This changes the script template parsing function to use a pointer into
the constant global opcode array for parsed opcodes as opposed to making
a copy of the opcode entries which causes unnecessary allocations.

Profiling showed that after roughly 48 hours of operation, this
copy was the culprit of 207 million unnecessary allocations.
2016-04-25 16:17:07 -05:00
Olaoluwa Osuntokun
3b39edcaa1 txscript: optimize sigcache lookup (#598)
Profiles discovered that lookups into the signature cache included an
expensive comparison to the stored `sigInfo` struct. This lookup had the
potential to be more expensive than directly verifying the signature
itself!

In addition, evictions were rather expensive because they involved
reading from /dev/urandom, or equivalent, for each eviction once the
signature cache was full as well as potentially iterating over every
item in the cache in the worst-case.

To remedy this poor performance several changes have been made:
* Change the lookup key to the fixed sized 32-byte signature hash
* Perform a full equality check only if there is a cache hit which
    results in a significant  speed up for both insertions and existence
checks
* Override entries in the case of a colliding hash on insert Add an
* .IsEqual() method to the Signature and PublicKey types in the
  btcec package to facilitate easy equivalence testing
* Allocate the signature cache map with the max number of entries in
  order to avoid unnecessary map re-sizes/allocations
* Optimize evictions from the signature cache Delete the first entry
* seen which is safe from manipulation due to
    the pre image resistance of the hash function
* Double the default maximum number of entries within the signature
  cache due to the reduction in the size of a cache entry
  * With this eviction scheme, removals are effectively O(1)

Fixes #575.
2016-04-13 21:56:10 -05:00
Dave Collins
5ff5fc5fa2 txscript: Correct comments on alt stack methods. (#657) 2016-04-11 14:22:25 -05:00
Dave Collins
f389742b39 multi: Update with result of gofmt -s.
This commit updates the code to make use of the most recent simplified
output from gofmt.
2016-02-25 13:02:54 -06:00
Dave Collins
eb882f39f8 multi: Fix several misspellings in the comments.
This commit corrects several typos in the comments found by misspell.
2016-02-25 11:17:12 -06:00
Dave Collins
9abc2c0e19 txscript: Comment improvements and fixes
This commit improves and corrects a few comments in txscript to ensure
they match reality.
2016-02-11 21:43:32 -06:00
Kefkius
d272bfebb7 Fix documentation for opcodeInvalid
Change 'opcodeReserved' to 'opcodeInvalid'
2016-02-11 20:42:41 -06:00
Kefkius
d759d1d3df Remove duplicate stack tests. 2016-02-09 11:17:04 -06:00
Dave Collins
c7e6c1e88f txscript: Correct JSON float conversions in tests.
This modifies the conversion of the output index from the JSON-based
test data for valid and invalid transactions as well as the signature
hash type for signature hash tests to first convert to a signed int and
then to an unsigned int.  This is necessary because the result of a
direct conversion of a float to an unsigned int is implementation
dependent and doesn't result in the expected value on all platforms.

Also, while here, change the function names in the error prints to match
the actual names.

Fixes #600.
2016-02-03 13:38:35 -06:00
Chris Shepherd
528ddaf23e txscript: Fix typo in README 2016-01-29 12:39:11 -08:00
Mawuli Adzoe
14ccab80e7 Review and fix typos in SigCache code. 2015-12-30 11:57:15 -07:00
David Hill
34a94b7d0b txscript: sync Bitcoin Core script tests 2015-12-30 09:38:16 -05:00
Mawuli Adzoe
6e133b58da txscript: Fix docs to match function.
Changed the order of return values described in the docs to be
consistent with the function’s actual return value signature.
2015-12-29 11:42:03 -07:00
Rune T. Aune
b691a222d5 Add signature hash calculation tests from Bitcoin Core.
500 tests with various transactions and scripts, verifying that
calcSignatureHash generates the expected hash in each case.

This requires changing SigHashType to uint32; that won't affect the
standard use-cases, but will make calcSignatureHash behave more like the
Core counterpart for non-standard SigHashType settings, like those in
some of these tests.
2015-11-15 16:39:00 -05:00
Dave Collins
3942a116e4 docs: Make various README.md files consistent.
First, it removes the documentation section from all the README.md files
and instead puts a web-based godoc badge and link at the top with the
other badges.  This is being done since the local godoc tool no longer
ships with Go by default, so the instructions no longer work without
first installing godoc. Due to this, pretty much everyone uses the
web-based godoc these days anyways.  Anyone who has manually installed
godoc won't need instructions.

Second, it makes sure the ISC license badge is at the top with the other
badges and removes the textual reference in the overview section.

Finally, it's modifies the Installation section to Installation and
Updating and adds a '-u' to the 'go get' command since it works for both
and thus is simpler.
2015-10-23 14:51:36 -05:00
David Hill
2e6e896aa6 txscript: Sync Bitcoin Core tests. 2015-10-22 16:10:29 -04:00
David Hill
3fa416a7ef txscript: fix isMultiSig bug.
isMultiSig was not verifying the number of pubkeys specified matched
the number of pubkeys provided.  This caused certain non-standard
scripts to be considered multisig scripts.

However, the script still would have failed during execution.

NOTE: This only affects whether or not the script is considered
standard and does NOT affect consensus.

Also, add a test for this check.
2015-10-22 15:55:34 -04:00