5df6f089b5 More tests of signer checks (Andrew Chow)
7c8bffdc24 Test that a non-witness script as witness utxo is not signed (Andrew Chow)
8254e9950f Additional sanity checks in SignPSBTInput (Pieter Wuille)
c05712cb59 Only wipe wrong UTXO type data if overwritten by wallet (Pieter Wuille)
Pull request description:
The current PSBT signing code can end up producing a non-segwit signature, while only the UTXO being spent is provided in the PSBT (as opposed to the entire transaction being spent). This may be used to trick a user to incorrectly decide a transaction has the semantics he intends to sign.
Fix this by refusing to sign if there is any mismatch between the provided data and what is being signed.
Tree-SHA512: b55790d79d8166e05513fc4c603a982a33710e79dc3c045060cddac6b48a1be3a28ebf8db63f988b6567b15dd27fd09bbaf48846e323c8635376ac20178956f4
bd19cc78cf Serialize non-witness utxo as a non-witness tx but always deserialize as witness (Andrew Chow)
43811e6338 Fix PSBT deserialization of 0-input transactions (Andrew Chow)
Pull request description:
0-input transactions can be ambiguously deserialized as being witness transactions. Since the unsigned transaction is never serialized as a witness transaction as it has no witnesses, we should always deserialize it as a non-witness transaction and set the serialization flags as such.
When a transaction is serliazed for the non-witness-utxo, it is always a valid network transaction and thus it should be always be deserialized as a witness transaction and the deserialzation flags are set as such.
Fixes#13958
Tree-SHA512: 1937b3cb2618534478d4f533541fb9efce3cb5badb5d1964bfe19400f4aacc6c8ecedaf1f20d26b20baf94f81fd07dfb15b3b08089ecbd63aeecbc18c7c48086
0-input transactions can be ambiguously deserialized as being witness
transactions. Since the unsigned transaction is never serialized as
a witness transaction as it has no witnesses, we should always
deserialize it as a non-witness transaction and set the serialization
flags as such.
Also always serialize the unsigned transaction as a non-witness transaction.
68400d8b96 tests: Use explicit imports (practicalswift)
Pull request description:
Enable automatic detection of undefined names in Python tests scripts. Remove wildcard imports.
Wildcard imports make it unclear which names are present in the namespace, confusing both readers and many automated tools.
An additional benefit of not using wildcard imports in tests scripts is that readers of a test script then can infer the rough testing scope just by looking at the imports.
Before this commit:
```
$ contrib/devtools/lint-python.sh | head -10
./test/functional/feature_rbf.py:8:1: F403 'from test_framework.util import *' used; unable to detect undefined names
./test/functional/feature_rbf.py:9:1: F403 'from test_framework.script import *' used; unable to detect undefined names
./test/functional/feature_rbf.py:10:1: F403 'from test_framework.mininode import *' used; unable to detect undefined names
./test/functional/feature_rbf.py:15:12: F405 bytes_to_hex_str may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:17:58: F405 CScript may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:25:13: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:26:31: F405 satoshi_round may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:26:60: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:30:41: F405 satoshi_round may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:30:68: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
$
```
After this commit:
```
$ contrib/devtools/lint-python.sh | head -10
$
```
Tree-SHA512: 3f826d39cffb6438388e5efcb20a9622ff8238247e882d68f7b38609877421b2a8e10e9229575f8eb6a8fa42dec4256986692e92922c86171f750a0e887438d9
To avoid:
$ make check
{…omissis…}
Running test/util/bitcoin-util-test.py...
/usr/local/bin/python3.7 ../test/util/bitcoin-util-test.py
../test/util/bitcoin-util-test.py:31: DeprecationWarning: This method will be removed in future versions. Use 'parser.read_file()' instead.
config.readfp(open(os.path.join(os.path.dirname(__file__), "../config.ini"), encoding="utf8"))
$ python3 --version
Python 3.7.0
cf9ed307e6 qa: blocktools enforce named args for amount (MarcoFalke)
Pull request description:
Since #13669 changed some signatures, I think it might be worthwhile to enforce named args for primitive types such as amounts.
Tree-SHA512: 2733e7b6a20590b54bd54e81a09e3f5e2fadf4390bed594916b70729bcf485b048266012c1203369e0968032a2c6a2719107ac17ee925d8939af3df916eab1a6
4b7091a842 Replace median fee rate with feerate percentiles (Marcin Jachymiak)
Pull request description:
Currently, the `medianfeerate` statistic is calculated from the feerate of the middle transaction of a list of transactions sorted by feerate.
This PR instead uses the value of the 50th percentile weight unit in the block, and also calculates the feerate at the 10th, 25th, 75th, and 90th percentiles. This more accurately corresponds with what is generally meant by median feerate.
Tree-SHA512: 59255e243df90d7afbe69839408c58c9723884b8ab82c66dc24a769e89c6d539db1905374a3f025ff28272fb25a0b90e92d8101103e39a6d9c0d60423a596714
4441ad677a Make format string linter understand basic template parameter syntax (practicalswift)
Pull request description:
Make format string linter understand basic template parameter syntax.
Fixes issue described in https://github.com/bitcoin/bitcoin/pull/13705#issuecomment-412046126.
Thanks to @ken2812221 for reporting!
Tree-SHA512: 8fb995bc6b29d8b9926ef5969e02cf71c494e829434fcdeb4ed5fabad5ab96e86e5b8eea705e8a416927757b4fa4e58abc0fd4f483daa58c94e2c6fdcb8ee822
e306be7429 Use 72 byte dummy signatures when watching only inputs may be used (Andrew Chow)
48b1473c89 Use 71 byte signature for DUMMY_SIGNATURE_CREATOR (Andrew Chow)
18dfea0dd0 Always create 70 byte signatures with low R values (Andrew Chow)
Pull request description:
When creating signatures for transactions, always make one which has a 32 byte or smaller R and 32 byte or smaller S value. This results in signatures that are always less than 71 bytes (32 byte R + 32 byte S + 6 bytes DER + 1 byte sighash) with low R values. In most cases, the signature will be 71 bytes.
Because R is not mutable in the same way that S is, a low R value can only be found by trying different nonces. RFC 6979 for deterministic nonce generation has the option to specify additional entropy, so we simply use that and add a uin32_t counter which we increment in order to try different nonces. Nonces are sill deterministically generated as the nonce used will the be the first one where the counter results in a nonce that results in a low R value. Because different nonces need to be tried, time to produce a signature does increase. On average, it takes twice as long to make a signature as two signatures need to be created, on average, to find one with a low R.
Having a fixed size signature makes size calculations easier and also saves half a byte of transaction size, on average.
DUMMY_SIGNATURE_CREATOR has been modified to produce 71 byte dummy signatures instead of 72 byte signatures.
Tree-SHA512: 3cd791505126ce92da7c631856a97ba0b59e87d9c132feff6e0eef1dc47768e81fbb38bfbe970371bedf9714b7f61a13a5fe9f30f962c81734092a4d19a4ef33
Removes medianfeerate result from getblockstats.
Adds feerate_percentiles which give the feerate of the 10th, 25th, 50th,
75th, and 90th percentile weight unit in the block.
fafe73a626 qa: Raise feature_help timeout to 5s (MarcoFalke)
faabd7bc47 qa: Use files for stdout/stderr to support Windows (MarcoFalke)
facb56ffaf qa: Run gen_rpcauth with sys.executable (MarcoFalke)
fada8966c5 qa: Close stdout and stderr file when node stops (MarcoFalke)
Pull request description:
### qa: Close stdout and stderr file when node stops
Since these files are potentially deleted by the test framework for cleanup, they should be closed first. Otherwise this will lead to errors on Windows when the tests finish successfully.
Side note: After the patch, it is no longer possible to reopen the file on Windows (see https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile)
### qa: Run gen_rpcauth with sys.executable
Similar to `test_runner.py`, the `sys.executable` needs to be passed down into subprocesses to pass on native Windows. (Should have no effect on Linux)
### qa: Use files for stdout/stderr to support Windows
It seems that using PIPE is not supported on Windows. Also, it is easier to just use the files that capture the stdout and stderr within the test node class.
Tree-SHA512: ec675012b10705978606b7fcbdb287c39a8e6e3732aae2fa4041d963a3c6993c6eac6a9a3cbd5479514e7d8017fe74c12235d1ed6fed2e8af8f3c71981e91864
fa85c985ed qa: Add p2p_invalid_locator test (MarcoFalke)
Pull request description:
Should not be merged *before* #13907
Tree-SHA512: a67ca407854c421ed20a184d0b0dc90085aed3e3431d9652a107fa3022244767e67f67e50449b7e95721f56906836b134615875f28a21e8a012eb22cfe6a66a5
1f87c372b5 Simplify comparison in rpc_blockchain.py. (Daniel Kraft)
Pull request description:
The test for `gettxoutsetinfo` in `rpc_blockchain.py` verifies that the result is the same as before after invalidating and reconsidering a block. The comparison has to exclude the `disk_size` field, though, as it is not deterministic.
Instead of comparing all the other fields for equality, this change explicitly removes the `disk_size` field and then compares the full objects. This makes the intent more explicit (compare everything except for `disk_size`, not compare just a given list of fields) and also the code simpler.
Tree-SHA512: 3c376a8836b62988fb2f0117c9ca65de64a33bf3cd4980a123de30bf5e7b7a48eda477b25e03d672ff076e205c698e83432469156caa0f0f3ebbb0480f0dd77d
When extra entropy is not specified by the caller, CKey::Sign will
now always create a signature that has a low R value and is at most
70 bytes. The resulting signature on the stack will be 71 bytes when
the sighash byte is included.
Using low R signatures means that the resulting DER encoded signature
will never need to have additional padding to account for high R
values.
The test for gettxoutsetinfo in rpc_blockchain.py verifies that the
result is the same as before after invalidating and reconsidering a
block. The comparison has to exclude the 'disk_size' field, though, as
it is not deterministic.
Instead of comparing all the other fields for equality, this change
explicitly removes the 'disk_size' field and then compares the full
objects. This makes the intent more explicit (compare everything except
for disk_size, not compare just a given list of fields) and also the
code simpler.
fa5587fe71 qa: wait_for_verack by default (MarcoFalke)
Pull request description:
This removes the need to do so manually every time a connection is added.
Tree-SHA512: a46c92cb4df41e30778b42b9fd3dcbd8d2d82aa7503d1213cb1c1165034f648d8caee01c292e2d87d05b0f71696996eef5be8a753f35ab49e5f66b0e3bf29f21
3fc20632a3 qt: Set BLOCK_CHAIN_SIZE = 220 (DrahtBot)
2b6a2f4a28 Regenerate manpages (DrahtBot)
eb7daf4d60 Update copyright headers to 2018 (DrahtBot)
Pull request description:
Some trivial maintenance to avoid having to do it again after the 0.17 branch off.
(The scripts to do this are in `./contrib/`)
Tree-SHA512: 16b2af45e0351b1c691c5311d48025dc6828079e98c2aa2e600dc5910ee8aa01858ca6c356538150dc46fe14c8819ed8ec8e4ec9a0f682b9950dd41bc50518fa
faa24441ec policy: Remove promiscuousmempoolflags (MarcoFalke)
Pull request description:
It seems odd to clutter validation code with features that can only ever be used for testing (testnet or regtest). Removing that test-only code makes the mempool logic less painful to understand and easier to reason about when changed or refactored in the future.
Tree-SHA512: 3b897aa9604ac8d82ebe9573c6efd468c93ddaa08d378ebc902e247b7aa6c68fcde71e5b449c08f17a067146cdc66dc50a67ce06d07607c27e5189a49c3fba3f
This linter checks that the number of arguments passed to each variadic format
string function matches the number of format specifiers in the format string.
a1a998cf24 wallet: Fix backupwallet for multiwallets (Daniel Kraft)
Pull request description:
`backupwallet` was broken for multiwallets in their own directories (i.e. something like `DATADIR/wallets/mywallet/wallet.dat`). In this case, the backup would use `DATADIR/wallets/wallet.dat` as source file and not take the specific wallet's directory into account.
This led to either an error during the backup (if the wrong source file was not present) or would silently back up the wrong wallet; especially the latter behaviour can be quite bad for users.
Tree-SHA512: 7efe2450ca047e40719fcc7cc211ed94699056020ac737cada7b59e8240298675960570c45079add424d0aab520437d5050d956acd695a9c2452dd4317b4d2c4
fa67505e1e qa: Quote wallet name for rpc path (MarcoFalke)
Pull request description:
When using external multiwallets they are specified by their full path which might contain non-ascii characters (e.g. umlauts or emojis).
Fix this by url-quoting the path.
Tree-SHA512: 7cc66514579d9f602f88a6817c5ab43a44c6d3711df452dc904173f0bc34e2c0b2c34877497f06b61f6720c532fa183053f54622dc454e316c89cee7eaa72463
fa5b440971 qa: Extract rpc_timewait as test param (MarcoFalke)
Pull request description:
Also increase it for wallet_dump and wallet_groups
Tree-SHA512: 7367bc584228bda3010c453713a1505c54a8ef3d116be47dab9934d30594089dfeb27ffa862f7517fd0ec8b5dc07f4904d67ef2a53dd284cbe2a58982e410e2b
f6b7fc349c Support h instead of ' in hardened descriptor paths (Pieter Wuille)
fddea672eb Add experimental warning to scantxoutset (Jonas Schnelli)
6495849bfd [QA] Extend tests to more combinations (Pieter Wuille)
1af237faef [QA] Add xpub range tests in scantxoutset tests (Jonas Schnelli)
151600bb49 Swap in descriptors support into scantxoutset (Pieter Wuille)
0652c3284f Descriptor tests (Pieter Wuille)
fe8a7dcd78 Output descriptors module (Pieter Wuille)
e54d76044b Add simple FlatSigningProvider (Pieter Wuille)
29943a904a Add more methods to Span class (Pieter Wuille)
Pull request description:
As promised, here is an implementation of my output descriptor concept (https://gist.github.com/sipa/e3d23d498c430bb601c5bca83523fa82) and integration within the `scantxoutset` RPC that was just added through #12196.
It changes the RPC to use descriptors for everything; I hope the interface is simple enough to encompass all use cases. It includes support for P2PK, P2PKH, P2WPKH, P2SH, P2WSH, multisig, xpubs, xprvs, and chains of keys - combined in every possible way.
Tree-SHA512: 63b54a96e7a72f5b04a8d645b8517d43ecd6a65a41f9f4e593931ce725a8845ab0baa1e9db6a7243190d8ac841f6e7e2f520d98c539312d78f7fd687d2c7b88f
a13647b8bd [qa] Add test for too-large wallet output groups (Suhas Daftuar)
57ec1c97b2 [wallet] correctly limit output group size (Suhas Daftuar)
Pull request description:
Also add a test to ensure that output groups are being limited, even if a wallet has many outputs corresponding to the same scriptPubKey (the test fails without the first commit).
Tree-SHA512: 2aaa82005b0910488f5cbf40690d4c5e2f46949e299ef70b4cb6e440713811443d411dcbc6d71b1701fd82423073125e21747787d70830cd021c841afb732d51
