Commit graph

3535 commits

Author SHA1 Message Date
Dave Collins
02dab1695f
txscript: Add benchmarks for IsMutlsigSigScript. 2021-11-16 18:46:01 -08:00
Dave Collins
0eaae2663b
txscript: Optimize IsMultisigScript.
This converts the IsMultisigScript function to make use of the new
tokenizer instead of the far less efficient parseScript thereby
significantly optimizing the function.

In order to accomplish this, it introduces two new functions.  The first
one is named extractMultisigScriptDetails and works with the raw script
bytes to simultaneously determine if the script is a multisignature
script, and in the case it is, extract and return the relevant details.
The second new function is named isMultisigScript and is defined in
terms of the former.

The extract function accepts the script version, raw script bytes, and a
flag to determine whether or not the public keys should also be
extracted.  The flag is provided because extracting pubkeys results in
an allocation that the caller might wish to avoid.

The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type.  Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.

It is important to note that this new implementation intentionally has a
semantic difference from the existing implementation in that it will now
correctly identify a multisig script with zero pubkeys whereas
previously it incorrectly required at least one pubkey.  This change is
acceptable because the function only deals with standardness rather than
consensus rules.

Finally, this also deprecates the isMultiSig function that requires
opcodes in favor of the new functions and deprecates the error return on
the export IsMultisigScript function since it really does not make sense
given the purpose of the function.

The following is a before and after comparison of analyzing both a large
script that is not a multisig script and a 1-of-2 multisig public key
script:

benchmark                            old ns/op     new ns/op     delta
BenchmarkIsMultisigScriptLarge-8     64166         5.52          -99.99%
BenchmarkIsMultisigScript-8          630           59.4          -90.57%

benchmark                            old allocs     new allocs     delta
BenchmarkIsMultisigScriptLarge-8     1              0              -100.00%
BenchmarkIsMultisigScript-8          1              0              -100.00%

benchmark                            old bytes     new bytes     delta
BenchmarkIsMultisigScriptLarge-8     311299        0             -100.00%
BenchmarkIsMultisigScript-8          2304          0             -100.00%
2021-11-16 18:45:59 -08:00
Dave Collins
4d31d1599d
txscript: Add benchmarks for IsMutlsigScript. 2021-11-16 18:45:57 -08:00
Dave Collins
215af7ff54
txscript: Optimize IsPayToScriptHash.
This converts the IsPayToScriptHash function to analyze the raw script
instead of using the far less efficient parseScript thereby
significantly optimizing the function.

In order to accomplish this, it introduces two new functions.  The first
one is named extractScriptHash and works with the raw script bytes to
simultaneously determine if the script is a p2sh script, and in the case
it is, extract and return the hash.  The second new function is named
isScriptHashScript and is defined in terms of the former.

The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type.  Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.

Finally, this also deprecates the isScriptHash function that requires
opcodes in favor of the new functions and modifies the comment on
IsPayToScriptHash to explicitly call out the script version semantics.

The following is a before and after comparison of analyzing a large
script that is not a p2sh script:

benchmark                        old ns/op     new ns/op     delta
BenchmarkIsPayToScriptHash-8     62393         0.60          -100.00%

benchmark                        old allocs     new allocs     delta
BenchmarkIsPayToScriptHash-8     1              0              -100.00%

benchmark                        old bytes     new bytes     delta
BenchmarkIsPayToScriptHash-8     311299        0             -100.00%
2021-11-16 18:45:55 -08:00
Dave Collins
665c29802e
txscript: Add benchmark for IsPayToScriptHash. 2021-11-16 18:45:52 -08:00
Conner Fromknecht
c771f4fb38
txscript: Optimize IsPayToPubKeyHash
This converts the IsPayToPubKeyHash function to analyze the raw script
instead of using the far less efficient parseScript, thereby
significantly optimization the function.

In order to accomplish this, it introduces two new functions.  The first
one is named extractPubKeyHash and works with the raw script bytes
to simultaneously determine if the script is a pay-to-pubkey-hash script,
and in the case it is, extract and return the hash.  The second new
function is named isPubKeyHashScript and is defined in terms of the
former.

The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type.  Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.

The following is a before and after comparison of analyzing a large
script:

benchmark                         old ns/op     new ns/op     delta
BenchmarkIsPubKeyHashScript-8     62228         0.45          -100.00%

benchmark                         old allocs     new allocs     delta
BenchmarkIsPubKeyHashScript-8     1              0              -100.00%

benchmark                         old bytes     new bytes     delta
BenchmarkIsPubKeyHashScript-8     311299        0             -100.00%
2021-11-16 18:45:50 -08:00
Conner Fromknecht
2d2608c34e
txscript: Add benchmark for IsPayToPubKeyHash 2021-11-16 18:45:47 -08:00
Conner Fromknecht
99cb679b6f
txscript: Optimize IsPayToPubKey
This converts the IsPayToScriptHash function to analyze the raw script
instead of using the far less efficient parseScript, thereby
significantly optimizing the function.

In order to accomplish this, it introduces four new functions:
extractCompressedPubKey, extractUncompressedPubKey, extractPubKey, and
isPubKeyScript.  The extractPubKey function makes use of
extractCompressedPubKey and extractUncompressedPubKey to combine their
functionality as a convenience and isPubKeyScript is defined in terms of
extractPubKey.

The extractCompressedPubKey works with the raw script bytes to
simultaneously determine if the script is a pay-to-compressed-pubkey
script, and in the case it is, extract and return the raw compressed
pubkey bytes.

Similarly, the extractUncompressedPubKey works in the same way except it
determines if the script is a pay-to-uncompressed-pubkey script and
returns the raw uncompressed pubkey bytes in the case it is.

The extract function approach was chosen because it is common for
callers to want to only extract relevant details from a script if the
script is of the specific type.  Extracting those details requires
performing the exact same checks to ensure the script is of the correct
type, so it is more efficient to combine the two into one and define the
type determination in terms of the result so long as the extraction does
not require allocations.

The following is a before and after comparison of analyzing a large
script:

benchmark                     old ns/op     new ns/op     delta
BenchmarkIsPubKeyScript-8     62323         2.97          -100.00%

benchmark                     old allocs     new allocs     delta
BenchmarkIsPubKeyScript-8     1              0              -100.00%

benchmark                     old bytes     new bytes     delta
BenchmarkIsPubKeyScript-8     311299        0             -100.00%
2021-11-16 18:45:45 -08:00
Dave Collins
05aa488a87
txscript: Add benchmark for IsPayToPubKey 2021-11-16 18:45:43 -08:00
Dave Collins
dfb1a6797b
txscript: Make asSmallInt accept raw opcode.
This converts the asSmallInt function to accept an opcode as a byte
instead of the internal opcode data struct in order to make it more
flexible for raw script analysis.

It also updates all callers accordingly.
2021-11-16 18:45:40 -08:00
Dave Collins
583b74040d
txscript: Make isSmallInt accept raw opcode.
This converts the isSmallInt function to accept an opcode as a byte
instead of the internal opcode data struct in order to make it more
flexible for raw script analysis.

The comment is modified to explicitly call out the script version
semantics.

Finally, it updates all callers accordingly.
2021-11-16 18:45:38 -08:00
Conner Fromknecht
c6f4cafe57
txscript/reference_test: Convert sighash calc test
This converts the tests for calculating signature hashes to use the
exported function which handles the raw script versus the now deprecated
variant requiring parsed opcodes.

Backport of 06f769ef72e6042e7f2b5ff1c512ef1371d615e5
2021-11-16 18:45:35 -08:00
Dave Collins
c19535b145
txscript: Optimize CalcSignatureHash.
This modifies the CalcSignatureHash function to make use of the new
signature hash calculation function that accepts raw scripts without
needing to first parse them.  Consequently, it also doubles as a slight
optimization to the execution time and a significant reduction in the
number of allocations.

In order to convert the CalcScriptHash function and keep the same
semantics, a new function named checkScriptParses is introduced which
will quickly determine if a script can be fully parsed without failure
and return the parse failure in the case it can't.

The following is a before and after comparison of analyzing a large
multiple input transaction:

benchmark                  old ns/op     new ns/op     delta
BenchmarkCalcSigHash-8     3627895       3619477       -0.23%

benchmark                  old allocs     new allocs     delta
BenchmarkCalcSigHash-8     1335           801            -40.00%

benchmark                  old bytes     new bytes     delta
BenchmarkCalcSigHash-8     1373812       1293354       -5.86%
2021-11-16 18:45:32 -08:00
Conner Fromknecht
af757d3d0d
txscript: Introduce raw script sighash calc func.
This introduces a new function named calcSignatureHashRaw which accepts
the raw script bytes to calculate the script hash versus requiring the
parsed opcode only to unparse them later in order to make it more
flexible for working with raw scripts.

Since there are several places in the rest of the code that currently
only have access to the parsed opcodes, this modifies the existing
calcSignatureHash to first unparse the script before calling the new
function.

Backport of decred/dcrd:f306a72a16eaabfb7054a26f9d9f850b87b00279
2021-11-16 18:45:29 -08:00
Dave Collins
f980c9a28d
txscript: Optimize script disasm.
This converts the DisasmString function to make use of the new
zero-allocation script tokenizer instead of the far less efficient
parseScript thereby significantly optimizing the function.

In order to facilitate this, the opcode disassembly functionality is
split into a separate function called disasmOpcode that accepts the
opcode struct and data independently as opposed to requiring a parsed
opcode.  The new function also accepts a pointer to a string builder so
the disassembly can be more efficiently be built.

While here, the comment is modified to explicitly call out the script
version semantics.

The following is a before and after comparison of a large script:

benchmark                   old ns/op     new ns/op     delta
BenchmarkDisasmString-8     102902        40124         -61.01%

benchmark                   old allocs     new allocs     delta
BenchmarkDisasmString-8     46             51             +10.87%

benchmark                   old bytes     new bytes     delta
BenchmarkDisasmString-8     389324        130552        -66.47%
2021-11-16 18:45:27 -08:00
Dave Collins
099784267e
txscript: Add benchmark for DisasmString. 2021-11-16 18:45:24 -08:00
Dave Collins
c997417978
txscript: Introduce zero-alloc script tokenizer.
This implements an efficient and zero-allocation script tokenizer that
is exported to both provide a new capability to tokenize scripts to
external consumers of the API as well as to serve as a base for
refactoring the existing highly inefficient internal code.

It is important to note that this tokenizer is intended to be used in
consensus critical code in the future, so it must exactly follow the
existing semantics.

The current script parsing mechanism used throughout the txscript module
is to fully tokenize the scripts into an array of internal parsed
opcodes which are then examined and passed around in order to implement
virtually everything related to scripts.

While that approach does simplify the analysis of certain scripts and
thus provide some nice properties in that regard, it is both extremely
inefficient in many cases, and makes it impossible for external
consumers of the API to implement any form of custom script analysis
without manually implementing a bunch of error prone tokenizing code or,
alternatively, the script engine exposing internal structures.

For example, as shown by profiling the total memory allocations of an
initial sync, the existing script parsing code allocates a total of
around 295.12GB, which equates to around 50% of all allocations
performed.  The zero-alloc tokenizer this introduces will allow that to
be reduced to virtually zero.

The following is a before and after comparison of tokenizing a large
script with a high opcode count using the existing code versus the
tokenizer this introduces for both speed and memory allocations:

benchmark                    old ns/op     new ns/op     delta
BenchmarkScriptParsing-8     63464         677           -98.93%

benchmark                    old allocs     new allocs     delta
BenchmarkScriptParsing-8     1              0              -100.00%

benchmark                    old bytes     new bytes     delta
BenchmarkScriptParsing-8     311299        0             -100.00%

The following is an overview of the changes:

- Introduce new error code ErrUnsupportedScriptVersion
- Implement zero-allocation script tokenizer
- Add a full suite of tests to ensure the tokenizer works as intended
  and follows the required consensus semantics
- Add an example of using the new tokenizer to count the number of
  opcodes in a script
- Update README.md to include the new example
- Update script parsing benchmark to use the new tokenizer
2021-11-16 18:45:22 -08:00
Dave Collins
bcb9643d39
txscript: Add benchmark for script parsing. 2021-11-16 18:45:19 -08:00
Conner Fromknecht
47806df63d
txscript: Add benchmark for CalcWitnessSigHash 2021-11-16 18:45:16 -08:00
Dave Collins
843d7607ef
txscript: Add benchmark for CalcSignatureHash 2021-11-16 18:45:14 -08:00
3nprob
cc7327c194 rpcclient: Add retry with backoffs to HTTP POST requests
Adds behavior similar to the retries of persistent RPC connections
to HTTP request.

* Initial backoff: 500ms
* Linear increase
* Max retries: 10

Room for future improvement:
* Make configurable
* Add jitter
* Tests for retry behavior
2021-11-16 09:08:07 -05:00
Aarush Bhat
65e986844e Update connmanager_test.go 2021-11-10 07:59:03 -05:00
naveen
31791ba4dc Included permissions for GitHub action
The default GitHub Action is write which is not required for this
action.
2021-10-26 10:00:04 -04:00
pengyonghui
c56a053fdf fix typos 2021-10-26 09:56:57 -04:00
pengyonghui
d590f3f77d fix typo 2021-10-26 09:56:57 -04:00
Jonathan Chappelow
a148fa797a addrmgr: make KnownAddress methods thread-safe
This gives KnownAddress a sync.RWMutex so the exported methods may
safely access the na (*wire.NetAddress) and lastattempt fields.
The AddrManager is updated to lock the new KnownAddress mutex before
assigning to na or lastattempt.
The other KnownAddress fields are only accessed by AddrManager, using
its own Mutex for synchronization.
2021-10-26 09:55:49 -04:00
Olaoluwa Osuntokun
e3449998be
Merge pull request #1752 from Roasbeef/config-disable-stall-handler
peer+server: add new config option to optionally disable stall detection
2021-10-05 11:44:31 -07:00
Olaoluwa Osuntokun
e98a1a1b4c
peer+server: add new config option to optionally disable stall detection
In this commit, we add a new config options that allows one to start
`btcd` in an operating mode that disables the stall detection. This can
be useful in simnet/regtest integration tests settings where it's
important that `btcd` holds on to its possibly sole connection to the
only other node in the test harness.

A new config flag has been added to gate this behavior, which is off by
default.
2021-10-01 14:55:50 -07:00
naveen
4caf037c52 Upgraded the docker version to 1.16
With this changes https://github.com/btcsuite/btcd/pull/1753/ merged in
the docker image also has to be upgraded.
2021-09-17 11:17:29 -04:00
Olaoluwa Osuntokun
bca4298ada
Merge pull request #1753 from Roasbeef/bump-go-version
build: bump min Go version to 1.16.8 add Go 1.17.1
2021-09-16 14:32:45 -07:00
eugene
f8e6854197 mempool: introduce GetDustThreshold to export dust limit calculation
This commit modifies no behavior and would allow other projects to
retrieve the dust limit for a particular output type before the
amount of the output is known. This is particularly useful in the
Lightning Network for channel negotiation.
2021-09-16 15:17:17 -04:00
Olaoluwa Osuntokun
7ae5b74dee
build: bump min Go version to 1.16.8 add Go 1.17.1 2021-09-15 18:19:34 -07:00
Marius van der Wijden
5e6736aad5 btcec: added testcase for point at infinity 2021-09-13 15:59:28 -04:00
Marius van der Wijden
73f7eac903 btcec: check if recovered pk is at point of infinity 2021-09-13 15:59:28 -04:00
JeremyRand
3e2d8464f1
rpcclient: Export symbols needed for custom commands (#1457)
* rpcclient: Export sendCmd and response

This facilitates using custom commands with rpcclient.

See https://github.com/btcsuite/btcd/issues/1083

* rpcclient: Export receiveFuture

This facilitates using custom commands with rpcclient.

See https://github.com/btcsuite/btcd/issues/1083

* rpcclient: Add customcommand example

* rpcclient: remove "Namecoin" from customcommand readme heading
2021-09-02 08:39:55 +02:00
John C. Vernaleo
f9d72f05a4 Switch irc to libera.chat 2021-08-31 07:50:29 -04:00
eugene
f5a1fb9965 mempool: export isDust for use in other projects
This changes isDust to IsDust so other golang projects (btcwallet
or lnd) can use the precise dust calculation used by btcd.
2021-08-03 09:34:49 -04:00
Calvin Kim
b3e6bd6161 rpcserverhelp: Remove extra period for gettxout--synopsis 2021-07-27 10:27:50 -04:00
Anirudha Bose
86a17263b0
Merge pull request #1729 from gnasr/fix-psbtopts-feerate-type
btcjson: Update WalletCreateFundedPsbtOpts.FeeRate type
2021-06-25 21:49:46 +02:00
Gabriel Nasr
505915dc3f btcjson: Update WalletCreateFundedPsbtOpts.FeeRate from *int64 to *float64 2021-06-25 15:23:44 -03:00
Anirudha Bose
63438c6d36 Update release date for v0.22.0-beta in CHANGES file 2021-06-01 13:16:51 -04:00
John C. Vernaleo
aaf19b26f3 btcd: bump version to v0.22.0-beta 2021-06-01 09:36:33 -04:00
Anirudha Bose
418f9204f4 Update CHANGES file for 0.22.0 release 2021-05-26 09:54:22 -04:00
Olaoluwa Osuntokun
ee5896bad5 mempool: add additional test case for inherited RBF replacement
In this commit, we add an additional test case for inherited RBF
replacement. This test case asserts that if a parent is marked as being
replaceable, but the child isn't, then the child can still be replaced
as according to BIP 125 it shoudl _inhreit_ the replaceability of its
parent.

The addition of this test case was prompted by the recently discovered
Bitcoin Core "CVE" [1]. It turns out that bitcoind doesn't properly
implement BIP 125. Namely it fails to allow a child to "inherit"
replaceability if its parent is also replaceable. Our implementation
makes this trait rather explicit due to its recursive implementation.
Kudos to the original implementer @wpaulino for getting this correct.

[1]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-May/018893.html.
2021-05-13 10:15:27 -04:00
Oliver Gugger
7b6c2b3423 chaincfg: fix deployment bit numbers
On signet all previous soft forks and also taproot are always activated,
meaning the version is always 0x20000000 for all blocks. To make sure
they activate properly in `btcd` we therefore need to use the correct
bit to mask the version.
This means that on any custom signet there would need to be 2016 blocks
mined before SegWit or Taproot can be used.
2021-05-11 15:55:06 -04:00
John C. Vernaleo
0ec4bdc1b8 Don't reference the readme that we don't produce 2021-05-06 18:51:45 -04:00
Olaoluwa Osuntokun
ce697fe7e8
Merge pull request #1716 from halseth/witness-commitment-rpctest
rpctest: add witness commitment when calling CreateBlock
2021-04-29 15:55:35 -07:00
Olaoluwa Osuntokun
7eba688b65
Merge pull request #1692 from guggero/signet
wire+chaincfg: add signet params
2021-04-26 11:01:13 -07:00
Johan T. Halseth
37a6e8485b
rpctest: add witness commitment when calling CreateBlock
If we tried to include transactions having witnesses, the block would be
invalid since the witness commitment was not added.
2021-04-26 13:53:53 +02:00
Johan T. Halseth
f0f4784c1c
mining: extract witness commitment add into method 2021-04-26 13:53:22 +02:00