685d1d8115 [tests] Check signrawtransaction* errors on missing prevtx info (Anthony Towns)
a3b065b51f Error on missing amount in signrawtransaction* (Anthony Towns)
Pull request description:
Signatures using segregated witness commit to the amount being spent, so that value must be passed into signrawtransactionwithkey and signrawtransactionwithwallet. This ensures an error is issued if that doesn't happen, rather than just assuming the value is 0 and producing a signature that is almost certainly invalid.
Based on Ben Woosley's #12458, Fixes: #12429.
Tree-SHA512: 8e2ff89d5bcf79548e569210af0d850028bc98d86c149b92207c9300ab1d63664a7e2b222c1be403a15941aa5cf36ccc3c0d570ee1c1466f3496b4fe06c17e11
d280617bf5 [qa] Add a test for merkle proof malleation (Suhas Daftuar)
ed82f17000 have verifytxoutproof check the number of txns in proof structure (Gregory Sanders)
Pull request description:
Recent publication of a weakness in Bitcoin's merkle tree construction demonstrates many SPV applications vulnerable to an expensive to pull off yet still plausible attack: https://bitslog.wordpress.com/2018/06/09/leaf-node-weakness-in-bitcoin-merkle-tree-design/
This change would at least allow `verifytxoutproof` to properly validate that the proof matches a known block, with known number of transactions any time after the full block is processed. This should neuter the attack entirely.
The negative is that a header-only processed block/future syncing mode would cause this to fail until the node has imported the data required.
related: #13451
`importprunedfunds` needs this check as well. Can expand it to cover this if people like the idea.
Tree-SHA512: 0682ec2b622a38b29f3f635323e0a8b6fc071e8a6fd134c954579926ee7b516e642966bafa667016744ce49c16e19b24dbc8801f982a36ad0a6a4aff6d93f82b
57889e688d bitcoin-tx: Stricter check for valid integers (Daniel Kraft)
Pull request description:
Just calling `atoi` to convert strings to integers does not check for valid integers very thoroughly; in particular, it just ignores everything starting from the first non-numeral character. Even a string like "foo" is fine and silently returns 0.
This meant that `bitcoin-tx` would not fail if such a string was passed in various places where an integer is expected (like the `locktime` or an input/output index); this means that it would, for instance, silently accept a typo and interpret it in an unexpected way.
In this change, we use `ParseInt64` for parsing strings to integers, which actually verifies that the full string is valid as number. New tests in the `bitcoin-util-test` cover the new error paths.
This fixes#13599.
Tree-SHA512: 146a0af275e9f57784e5d0582d3defbac35551b54b6b7232f8a0b20db04aa611125e52aa4512ef2f8ed2cafc2a12fe586f9d10ed66d641cff090288f279b1988
161e8d40a4 RPC: Add new getzmqnotifications method. (Daniel Kraft)
caac39b0ac Make ZMQ notification interface instance global. (Daniel Kraft)
Pull request description:
This adds a new RPC method `getzmqnotifications`, which returns information about all active ZMQ notification endpoints. This is useful for software that layers on top of bitcoind, so it can verify that ZeroMQ is enabled and also figure out where it should listen.
See #13526.
Tree-SHA512: edce722925741c84ddbf7b3a879fc9db1907e5269d0d97138fe724035d93ee541c2118c24fa92f4197403f380d0e25c2fda5ca6c62d526792ea749cf527a99a0
ea65182f03 [wallet] loadwallet shouldn't create new wallets. (John Newbery)
Pull request description:
A bug in the initial implementation of loadwallet meant that if the
arguement was a directory that didn't contain a wallet.dat file, a new
wallet would be created in that directory. Fix that so that if a
directory is passed in, it must contain a wallet.dat file.
Bug reported by promag (João Barbosa).
Tree-SHA512: 0a59fa8a33fde51a88544ad288b00e4995284fe16424f643076aaba42b8244fff362145217650ee53d518dfab7efbed4237632c34cdd3dcbbecaa9ecaab5fd7b
Just calling atoi to convert strings to integers does not check for
valid integers very thoroughly; in particular, it just ignores
everything starting from the first non-numeral character. Even a string
like "foo" is fine and silently returns 0.
This meant that bitcoin-tx would not fail if such a string was passed in
various places where an integer is expected (like the locktime or an
input/output index); this means that it would, for instance, silently
accept a typo and interpret it in an unexpected way.
In this change, we use ParseInt64 for parsing strings to integers,
which actually verifies that the full string is valid as number.
New tests in the bitcoin-util-test cover the new error paths.
e3aab295e [tests] p2p_segwit: sync_blocks in subtest wrapper. (John Newbery)
55e805085 [tests] p2p_segwit: remove unnecessary arguments from subtests. (John Newbery)
25711c269 [tests] p2p_segwit: log and assert segwit status in subtest wrapper. (John Newbery)
6839863d5 [tests] p2p_segwit: Make sure each subtest leaves utxos for the next. (John Newbery)
bfe32734d [tests] p2p_segwit: wrap subtests with subtest wrapper. (John Newbery)
2af4e398d [tests] p2p_segwit: re-order function definitions. (John Newbery)
94a0134a4 [tests] p2p_segwit: standardise comments/docstrings. (John Newbery)
f7c7f8ecf [tests] p2p_segwit: Fix flake8 warnings. (John Newbery)
Pull request description:
`p2p_segwit.py` is a very long test, composed of multiple subtests. When it fails it's difficult to debug for a couple of reasons:
- Control flow jumps between different methods in the test class, so it's a little difficult to follow the code.
- state may be carried forward unintentionally from one subtest to the next.
Improve that by wrapping the subtests with a `@subtest` decorator which:
- logs progress
- asserts state after each subtest
As usual, I've also included a few commits which generally tidy up the test and improve style.
Tree-SHA512: 3650602b3ce9823dc968cc5f2e716757feadc3dbedb3605eb79bb3df91a6db8ae53431f253b440da690e3a8e9d76de84fad4368a2663aeb40e6b9427cf948870
This adds a new RPC method "getzmqnotifications", which returns
information about all active ZMQ notification endpoints. This is useful
for software that layers on top of bitcoind, so it can verify that
ZeroMQ is enabled and also figure out where it should listen.
See https://github.com/bitcoin/bitcoin/issues/13526.
75848bcf40 [tests] Fix p2p_sendheaders race (John Newbery)
Pull request description:
p2p_sendheaders has a race in part 1.3.
part 1.2 sends a block to the node over the 'test_node' connection, but
doesn't wait for an inv to be received on the 'inv_node' connection. If
we get to part 1.3 before that inv has been received, then the
subsequent call to check_last_inv_announcement could fail.
Tree-SHA512: ba9baffb3a9c0d379259190c737a7a4ad2e1133005a5b026af4f6b67a2978e24db39289551ad29134151879593ef5472be7e569a3557c0740fb51f5c56263d9a
fa87da2f17 qa: Avoid start/stop of the network thread mid-test (MarcoFalke)
Pull request description:
This simplifies test writing by removing the need to handle the network thread in tests. E.g. start thread, join thread, restart thread mid-test, adding p2p connections at the "right" time, ...
Tree-SHA512: 533642f12fef5496f1933855edcdab1a7ed901d088d34911749cd0f9e044c8a6cb1f89985ac3a7f41a512943663e4e270a61978f6f072143ae050cd102d4eab8
A bug in the initial implementation of loadwallet meant that if the
arguement was a directory that didn't contain a wallet.dat file, a new
wallet would be created in that directory. Fix that so that if a
directory is passed in, it must contain a wallet.dat file.
Bug reported by promag (João Barbosa).
fa103a5d5e [qa] wallet_basic: Specify minimum required amount for listunspent (MarcoFalke)
Pull request description:
A value less than that would fail the tests later on anyway:
```
File "./test/functional/wallet_basic.py", line 250, in run_test
self.nodes[1].sendrawtransaction(signed_raw_tx['hex'])
test_framework.authproxy.JSONRPCException: bad-txns-in-belowout, value in (1.00) < value out (49.998) (code 16) (-26)
Tree-SHA512: 7e72ad02b5623bc078610da06c34721836822a920a4e85b12a1e0f339e3205cdc11d39763197770e649fb73376f922ff91a8f244b465195e50a6798658e04f80
df10f07db1 [wallet] Don't use accounts when checking balance in sendmany (John Newbery)
e209184101 [wallet] deprecate sendfrom RPC method. (John Newbery)
Pull request description:
A couple of fixups from the accounts API deprecation PR (#12953):
- properly deprecate `sendfrom`
- don't use accounts when calculating balance in `sendmany` (unless the `-deprecatedrpc=accounts` flag is being used)
Tree-SHA512: 1befde055067438c4c3391bbff1aaed0e6249efd708c567db3f1faad40a0f28e64f95e5bad0679ae826d24a0239e4bc8a1c392dc93e2e7502343a7f6b1d1845c
927e1150bc Test: Harden lint-filenames.sh (wodry)
Pull request description:
- This fixes that only files with lower case file name suffix where found before, which contradicted the Regex to find uppercase characters in file names (including suffixes I guess).
- `--full-name` switch was added to git ls-files, to define that the found file always includes it's full path in the git project.
- since we know now that the file name includes the full path, we can harden the Regex to exclude the secp256k1 and univalue sub folders.
- use backslash line break to make code easier to read and avoid too long line.
Tree-SHA512: 9b55fe4965ae2084112b9f8a81bf9c657756c2cb5004986e7b6102a76adaf62c7d7a53257d9f13c5d8a1c75870b52c744d13830e3edd454a099e810357c914e5
000000035b Obsolete #!/bin/bash shebang (DesWurstes)
Pull request description:
> `#!/bin/bash` assumes it is always installed to `/bin/` which can cause issues
> `#!/usr/bin/env bash` searches the user's `PATH` to find the `bash` binary
Details: https://github.com/dylanaraps/pure-bash-bible#obsolete-syntax
I'm open to comments: Should I also fix `#!/bin/sh`?
Tree-SHA512: b47bb4828116aa119f1899c68fee081270d51a898535490b9c616bf0f3660ad953f29c361eafc759bc64cdd54ee6eeecb2d79e9fdb5291a996a515c719805476
fa1eac9cdb [qa] mininode: Expose connection state through is_connected (MarcoFalke)
Pull request description:
This gets rid of some non-type safe string comparisons and access to members that are implementation details of `class P2PConnection(asyncore.dispatcher)`. Such refactoring is required to replace the deprecated asyncore with something more sane.
Changes:
* Get rid of non-enum member `state` and replace is with bool `connected`
* Get rid of confusing argument `pushbuf` and literally just push to the buffer at the call site
Tree-SHA512: 09074c7e5ed251a2e0509ef205ab82f89887c1e1fa1cc6efc1db60d196eb2403788a4987df8809fd06d80ef652e614c5d3c3fdef70096fc5815102243388288d
p2p_sendheaders has a race in part 1.3.
part 1.2 sends a block to the node over the 'test_node' connection, but
doesn't wait for an inv to be received on the 'inv_node' connection. If
we get to part 1.3 before that inv has been received, then the
subsequent call to check_last_inv_announcement could fail.
47776a958b Add linter: Make sure all shell scripts opt out of locale dependence using "export LC_ALL=C" (practicalswift)
3352da8da1 Add "export LC_ALL=C" to all shell scripts (practicalswift)
Pull request description:
~~Make sure `LC_ALL=C` is set when using `grep` range expressions.~~
Make sure `LC_ALL=C` is set in all shell scripts.
From the `grep(1)` documentation:
> Within a bracket expression, a range expression consists of two characters separated by a hyphen. It matches any single character that sorts between the two characters, inclusive, using the locale's collating sequence and character set. For example, in the default C locale, `[a-d]` is equivalent to `[abcd]`. Many locales sort characters in dictionary order, and in these locales `[a-d]` is typically not equivalent to `[abcd]`; it might be equivalent to `[aBbCcDd]`, for example. To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the `LC_ALL` environment variable to the value C.
Context: [Locale issue found when reviewing #13450](https://github.com/bitcoin/bitcoin/pull/13450/files#r194877736)
Tree-SHA512: fd74d2612998f9b49ef9be24410e505d8c842716f84d085157fc7f9799d40e8a7b4969de783afcf99b7fae4f91bbb4559651f7dd6578a6a081a50bdea29f0909
c8176b3cc7 Add linter: Make sure we explicitly open all text files using UTF-8 or ASCII encoding in Python (practicalswift)
634bd97001 Explicitly specify encoding when opening text files in Python code (practicalswift)
Pull request description:
Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python.
As requested by @laanwj in #13440.
Tree-SHA512: 1651c00fe220ceb273324abd6703aee504029b96c7ef0e3029145901762c733c9b9d24927da281394fd4681a5bff774336c04eed01fafea997bb32192c334c06
86edf4a2a5 expose CBlockIndex::nTx in getblock(header) (Gregory Sanders)
Pull request description:
Recent publication of a weakness in Bitcoin's merkle tree construction demonstrates many SPV applications vulnerable to an expensive to pull off yet still plausible attack: https://bitslog.wordpress.com/2018/06/09/leaf-node-weakness-in-bitcoin-merkle-tree-design/
Including the coinbase in the txoutproof seems the most effective fix, however results in a significant efficiency downgrade. Transactors will not even know a priori what the size of their proof will be within a couple orders of magnitude, unless they use the mid-state of SHA2 as detailed in the blog post.
Some applications, like Elements blockchain platform that take SPV-style proofs have optional access to a bitcoind to verify these proofs of inclusion and check depth in the chain. Returning `CBlockIndex::nTx` would allow an extremely easy and compact way of checking the depth of the tree, with no additional overhead to the codebase, and works with pruned nodes.
`getblockheader` is arguably not the place for it, but as mentioned before, is a natural workflow for us checking depth of a block in a possibly pruned node.
We should also ensure that `verifytxoutproof` ends up validating this depth fact as well, but left this for another PR.
Tree-SHA512: af4cf48e704c6088f8da06a477fda1aaa6f8770cee9b876c4465d1075966d6a95831a88817673fe5a0d6bbcdc1ffcbc1892e2be0d838c60fc6958d33eacdcc14
fa7a6cf1b3 policy: Treat segwit as always active (MarcoFalke)
Pull request description:
Now that segwit is active for a long time, there is no need to reject transactions with the reason that segwit hasn't activated.
Strictly speaking, this is a bug fix, because with the release of 0.16, we create segwit transactions in our wallet by default without checking if they are allowed by local policy.
More broadly, this simplifies the code as if "premature witness" was always set to true with the corresponding command line args.
Tree-SHA512: 484c26aa3a66faba6b41e8554a91a29bfc15fbf6caae3d5363a3966283143189c4bd5333a610b0669c1238f75620691264e73f6b9f1161cdacf7574d946436da
fa8071a098 qa: Log as utf-8 (MarcoFalke)
Pull request description:
Explicitly read and write the log files with utf-8 as encoding
Tree-SHA512: ca28f37f34a09845c736ff6c4c21733c3c39584f52c81e48ff25e5e35979c317d0989862b2b93acc7e359fbcc20b99533365455830b2ddb41eb4d8c17314534e
16e3cd380a Clarify include recommendation (practicalswift)
6d10f43738 Enforce the use of bracket syntax includes ("#include <foo.h>") (practicalswift)
906bee8e5f Use bracket syntax includes ("#include <foo.h>") (practicalswift)
Pull request description:
When analysing includes in the project it is often assumed that the preferred bracket include syntax (`#include <foo.h>`) mentioned in `developer-docs.md` is used consistently. @sipa:s excellent circular dependencies script [`circular-dependencies.py`](50c69b7801/contrib/devtools/circular-dependencies.py) (#13228) is an example of a script making this reasonable assumption.
This PR enables automatic Travis checking of the include syntax making sure that the bracket syntax includes (`#include <foo.h>`) is used consistently.
Tree-SHA512: a414921aabe8e487ebed42f3f1cbd02fecd1add385065c1f2244cd602c31889e61fea5a801507ec501ef9bd309b05d3c999f915cec1c2b44f085bb0d2835c182
67e0e04140 [wallet] [docs] Update release notes for removing `getlabeladdress` (John Newbery)
81608178cf [wallet] [rpc] Remove getlabeladdress RPC (John Newbery)
Pull request description:
labels are associated with addresses (rather than addresses being
associated with labels, as was the case with accounts). The
getlabeladdress does not make sense in this model, so remove it.
getaccountaddress is still supported for one release as the accounts
API is deprecated.
Tree-SHA512: 7f45d0456248ebcc4e54dd34e2578a09a8ea8e4fceda75238ccea9d731dc99a3f3c0519b18a9739de17d2e6e59c9c2259ba67c9ae2e3cb2a40ddb14b9193fe29
698cfd0811 docs: Mention lint-locale-dependence.sh in developer-notes.md (practicalswift)
0a4ea2f458 build: Add linter for checking accidental locale dependence (practicalswift)
Pull request description:
This linter will check for code accidentally introducing locale dependencies.
Unnecessary locale dependence can cause bugs that are very tricky to isolate and fix. We should avoid using locale dependent functions if possible.
Context: https://github.com/bitcoin/bitcoin/pull/12881#issuecomment-378564722
Example output:
```
$ contrib/devtools/lint-locale-dependence.sh
The locale dependent function tolower(...) appears to be used:
src/init.cpp: if (s[0] == '0' && std::tolower(s[1]) == 'x') {
Unnecessary locale dependence can cause bugs that are very
tricky to isolate and fix. Please avoid using locale dependent
functions if possible.
Advice not applicable in this specific case? Add an exception
by updating the ignore list in contrib/devtools/lint-locale-dependence.sh
```
**Note to reviewers:** What is the most appropriate `LOCALE_DEPENDENT_FUNCTIONS` function list? What should be added or removed?
Tree-SHA512: 14e448828804bb02bf59070647e38b52fce120c700c903a4a8472769a2cee5dd529bd3fc182386993cb8720482cf4250b63a0a477db61b941ae4babe5c65025f
fa4760fbb3 qa: Increase includeconf test coverage (MarcoFalke)
Pull request description:
This adds some missing `return false` for error conditions and adds test coverage [1] for those.
Also, extend recursion warning when the chain was set in one of the includeconfs.
[1] See the red lines in https://marcofalke.github.io/btc_cov/total.coverage/src/util.cpp.gcov.html for missing coverage.
Tree-SHA512: d32563c9bb277879895a173e699034db5ecdb4061a1ec8890c566d61e36a09efa5eda19a029baf952ff6d568f8b9684a13a0bb90827850075470975e2088fee4
faac7a2db4 qa: Avoid checking reject code for now (MarcoFalke)
Pull request description:
The node will often disconnect before sending a reject code. A more
robust solution would be to read from the debug log. See #13006
Tree-SHA512: 1dabf8a43dabbc722f4ffe4fbc1f870090253a66290b2d1a95e7a24e14c6442b493c314480c0314587164eb65e5d468aa9eb5e107ad90bb3ca821a97ea4d373c
fa26cf0156 qa: Fixup setting of PATH env var (MarcoFalke)
Pull request description:
This was an oversight of mine in #13188
Can be trivially tested with `BITCOIND=bitcoin-qt ./test/functional/wallet_disable.py` before and after this fix.
Tree-SHA512: 06c7b2f12158855eb2b6392861943821bd7ad3152cf0dd49ac4abd878e5b937ebee55e256ce5bdc1c2a9c775a452112c34533366c934ff5f0f412b3a7e1c8118
fa3c910bfe test: Move linters to test/lint, add readme (MarcoFalke)
Pull request description:
This moves the checks and linters from `devtools` to a subfolder in `test`. (Motivated by my opinion that the dev tools are mostly for generating code and updating the repo whereas the linters are read-only checks.)
Also, adds a readme to clarify that checks and linters are only meant to prevent bugs and user facing issues, not merely stylistic preference or inconsistencies. (This is motivated by the diversity in developers and work flows as well as existing code styles. It would be too disruptive to change all existing code to a single style or too burdensome to force all developers to adhere to a single style. Also note that our style guide is changing, so locking in at the wrong style "too early" would only waste resources.)
Tree-SHA512: 9b10e89f2aeaf0c8a9ae248aa891d74e0abf0569f8e5dfd266446efa8bfaf19f0ea0980abf0b0b22f0d8416ee90d7435d21a9f9285b66df43f370b7979173406
87fe292d89 doc: Mention disabling BIP61 in bips.md (Wladimir J. van der Laan)
fe16dd8226 net: Add option `-enablebip61` to configure sending of BIP61 notifications (Wladimir J. van der Laan)
Pull request description:
This commit adds a boolean option `-peersendreject`, defaulting to `1`, that can be used to disable the sending of [BIP61](https://github.com/bitcoin/bips/blob/master/bip-0061.mediawiki) `reject` messages. This functionality has been requested for various reasons:
- security (DoS): reject messages can reveal internal state that can be used to target certain resources such as the mempool more easily.
- bandwidth: a typical node sends lots of reject messages; this counts against upstream bandwidth. Also the reject messages tend to be larger than the message that was rejected.
On the other hand, reject messages can be useful while developing client software (I found them indispensable while creating bitcoin-submittx), as well as for our own test cases, so whatever the default becomes on the long run, IMO the functionality should be retained as option. But that's a discussion for later, for now it's simply a node operator decision.
Also adds a RPC test that checks the functionality.
Tree-SHA512: 9488cc53e13cd8e5c6f8eb472a44309572673405c1d1438c3488f627fae622c95e2198bde5ed7d29e56b948e2918bf1920239e9f865889f4c37c097c37a4d7a9
fa865efa4a qa: Fix wallet_listreceivedby race (MarcoFalke)
Pull request description:
Generating a block on node 0 will only get node 0 out of IBD and not node 1. So the inv for the `txid` is dropped by node 1 and the call to `sync_all` fails.
Solve it by a call to `sync_blocks` after `generate`.
Tree-SHA512: e21b01a9e8c90bd6a3aad290c97cc4866ab384e22797b318eed55ae2767512203597d3a184b23ad5a3fe76bdbb8a3d5c51e097d56b160232851164434059ff23
41d0476f62 Tests: Add data file (Anthony Towns)
4cbfb6aad9 Tests: Test new getblockstats RPC (Jorge Timón)
35e77a0288 RPC: Introduce getblockstats (Jorge Timón)
cda8e36f01 Refactor: RPC: Separate GetBlockChecked() from getblock() (Jorge Timón)
Pull request description:
It returns per block statistics about several things. It should be easy to add more if people think of other things to add or remove some if I went too far (but once written, why not keep it? EDIT: answer: not to test or maintain them).
The currently available options are: minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs (EDIT: see updated list in the rpc call documentation)
For the x axis, one can use height or block.nTime (I guess I could add mediantime if there's interest [EDIT: nobody showed interest but I implemented mediantime nonetheless, in fact there's no distinction between x or y axis anymore, that's for the caller to judge]).
To calculate fees, -txindex is required.
Tree-SHA512: 2b2787a3c7dc4a11df1fce62c8a4c748f5347d7f7104205d5f0962ffec1e0370c825b49fd4d58ce8ce86bf39d8453f698bcd46206eea505f077541ca7d59b18c
labels are associated with addresses (rather than addresses being
associated with labels, as was the case with accounts). The
getlabeladdress does not make sense in this model, so remove it.
getaccountaddress is still supported for one release as the accounts
API is deprecated.
2352aa9 test: Ensure that recursive -includeconf produces appropriate warnings (Karl-Johan Alm)
c5bcc7d util: warn about recursive -includeconf arguments in configuration files (Karl-Johan Alm)
Pull request description:
This is a follow-up PR to #10267, and addresses https://github.com/bitcoin/bitcoin/pull/10267#issuecomment-387546144.
~~I am adding extra work for @jnewbery in #12755 here -- maybe I should just rebase on top of that, but not sure what the appropriate approach is here.~~
Tree-SHA512: 87f0c32436b70424e33616ffb88d7cb699f90d6a583a10237e224b28fc936d6a9df95536c8c52ee8546b3942da92b2a357e61bf87e00d1462bc10d46d3bee352
7384a35 [tests] Remove spurious error log in p2p_segwit.py (John Newbery)
Pull request description:
Since 265d7c44b1, when wait_until() fails,
an error message is logged to the test framework log. This means that if
wait_until() is called inside a try-except with the expectation that it
will fail, a spurious error message is logged.
wait_until() shouldn't be called with the expectation of failure. Fix
that in p2p_segwit.py.
Tree-SHA512: 0a43790b58fee7d2d6bef36e736b0b9ffdde6de5f12d33d15e8e07323597e2be4cd98f17e7fc3a135e06bdafe36613466c0a57e81134e59a251383c62b91918f
a8da482 Bump wallet version for pre split keypool (Andrew Chow)
dfcd9f3 Use a keypool of presplit keys after upgrading to hd chain split (Andrew Chow)
5c50e93 Allow -upgradewallet to upgradewallets to HD (Andrew Chow)
2bcf2b5 Test sethdseed (Andrew Chow)
b5ba01a Add 'sethdseed' RPC to initialize or replace HD seed (Chris Moore)
dd3c07a Separate HaveKey function that checks whether a key is in a keystore (Andrew Chow)
Pull request description:
Revival/rebase of #11085
Adds a new command `sethdseed` which allows you to either set or generate a new HD seed to be used. A new keypool can be generated or the original one kept and new keys added to the keypool will come from the new HD seed.
Wallets that are not HD will be upgraded to be version FEATURE_HD_SPLIT when the `sethdseed` RPC command is used.
I have also add some tests for this.
Additionally `-upgradewallet` can now be used to upgrade a wallet from non-HD to HD. When it is used for such an upgrade, the keypool will be regenerated.
Tree-SHA512: e56c792e150590429ac4a1061e8d6f7b20cca06366e184eb9bbade4cd6ae82699a28fe84f87031eadba97ad2c1606517a105f00fb7b45779c979243020071adb
This commit adds a boolean option `-enablebip61`, defaulting to `1`, that
can be used to disable the sending of BIP61 `reject` messages. This
functionality has been requested for various reasons:
- security (DoS): reject messages can reveal internal state that can be
used to target certain resources such as the mempool more easily.
- bandwidth: a typical node sends lots of reject messages; this counts
against upstream bandwidth. Also the reject messages tend to be larger
than the message that was rejected.
On the other hand, reject messages can be useful while developing client
software (I found them indispensable while creating bitcoin-submittx),
as well as for our own test cases, so whatever the default becomes on the
long run, IMO the functionality should be retained as option. But that's
a discussion for later.
364bae5 qa: Pad scriptPubKeys to get minimum sized txs (MarcoFalke)
7485488 Policy to reject extremely small transactions (Johnson Lau)
0f8719b Add transaction tests for constant scriptCode (Johnson Lau)
9dabfe4 Add constant scriptCode policy in non-segwit scripts (Johnson Lau)
Pull request description:
This disables `OP_CODESEPARATOR` in non-segwit scripts (even in an unexecuted branch), and makes a positive `FindAndDelete` result invalid. This ensures that the `scriptCode` serialized in `SignatureHash` is always the same as the script passing to the `EvalScript`.
Tree-SHA512: a0552cb920294d130251c48053fa2ff1fbdd26332e62b52147d918837852750f0ce35ce2cd1cbdb86588943312f8154ccb4925e850dbb7c2254bc353070cd5f8
5d536619ab [tests] Remove 'account' API from wallet functional tests (John Newbery)
Pull request description:
Next step in #12952. Removes all usage of the 'account' API from the wallet functional tests, except:
- rpc_deprecated.py (which specifically tests the `-deprecatedrpc=accounts` command line argument is working properly).
- `wallet_labels.py` (which tests that both the 'label' and 'account' APIs work in V0.17).
'account' API usage for both of those tests can be removed once V0.17 has been branched.
Also excluded is:
- `wallet_importprunedfunds.py` (which fails due to a bitcoind OOM error)
Tree-SHA512: 6701b32f83d2d47597ba093ded665d7aa630f7a9c759ff15e3e33a3e3bc7600e8d29cf4e72aed5f8f9f6769cc9b614c681951720eab1ed2473f5f8dec57e7a6f
12d1b77f7e [tests] Fixed intermittent failure in p2p_sendheaders.py. (lmanners)
Pull request description:
Added handling for the case where headers are announced over more than one message.
refs #12453
Tree-SHA512: 2c5b48ff019089b86e358181ba170d3aac09d4ae41ec79c2718e0ee83705860501bbcb8fd94d0f5c4f86c0d54a96781a967716621bb8c5ecc991b39af3cec506
09c6699900 [qa] Handle disconnect_node race (Suhas Daftuar)
Pull request description:
Several tests call disconnect_nodes() on each node-pair in rapid
succession, resulting in a race condition if a node disconnects a peer
in-between the calculation of the nodeid's to disconnect and the
invocation of the disconnectnode rpc call. Handle this.
Tree-SHA512: 3078cea0006fcb507c812004a777c505eb1e9dda7c6df12dbbe72395a73ff6f6760f597b6492054f5487b34534417ddef5fbad30553c135c288c4b7cfce79223
Since 265d7c44b1, when wait_until() fails,
an error message is logged to the test framework log. This means that if
wait_until() is called inside a try-except with the expectation that it
will fail, a spurious error message is logged.
wait_until() shouldn't be called with the expectation of failure. Fix
that in p2p_segwit.py.
Several tests call disconnect_nodes() on each node-pair in rapid
succession, resulting in a race condition if a node disconnects a peer
in-between the calculation of the nodeid's to disconnect and the
invocation of the disconnectnode rpc call. Handle this.
By default, libc will print fatal errors to /dev/tty instead of stderr.
Adding the LIBC_FATAL_STDERR_ to the environment variables allows
us to catch libc errors in stderr and test for them.
fac1e1f qa: Remove unused option --srcdir (MarcoFalke)
Pull request description:
The `srcdir` option was both unused and misleading; It should have been called `builddir`. So remove it.
Tree-SHA512: 2c24dcf2aa82219158b8cbbf03dd3f0f51f805f1f5f670faa1fd59e5a8d60fda120ffddadeccb058d8d3f20583b4952be7afd2df6bbefb9367d35c0f0a9fda3c
891beb0 [test] fundrawtransaction: lock watch-only shared address (Karl-Johan Alm)
Pull request description:
`self.nodes[0]` creates an address which is watch-only-shared with `self.nodes[3]`. If `nodes[0]` spends the associated UTXO during any of its sends later, the watchonly test will fail, as `nodes[3]` now has insufficient funds.
I ran into this in #12257 and this commit is in that PR as well, but I figured I'd split it out (and remove from there once/if merged).
Tree-SHA512: d04a04b1ecebe82127cccd47c1b3de311bf07f4b51dff80db20ea2f142e1d5c4a85ed6180c5c0b081d550e238c742e119b953f60f487deac5a3f3536e1a8d9fe
25b7ab9 doc: Add release notes for -includeconf (Karl-Johan Alm)
0f0badd test: Test includeconf parameter. (Karl-Johan Alm)
629ff8c -includeconf=<path> support in config handler, for including external configuration files (Karl-Johan Alm)
Pull request description:
Fixes: #10071.
Done:
- adds `-includeconf=<path>`, where `<path>` is relative to `datadir` or to the path of the file being read, if in a file
- protects against circular includes
- updates help docs
~~~Thoughts:~~~
- ~~~I am not sure how to test this in a neat manner. Feedback on this would be nice. Will dig/think though.~~~
Tree-SHA512: cb31f1b2f69fbc0890d264948eb2e501ac05cf12f5e06a5942f9c1539eb15ea8dc3cae817f4073aecb2fcc21d0386747f14f89d990772003a76e2a6d25642553
1e46d8a Get rid of ambiguous OutputType::NONE value (Russell Yanofsky)
Pull request description:
Based on suggestion by @sipa https://github.com/bitcoin/bitcoin/pull/12119#issuecomment-357982763
After #12119, the NONE output type was overloaded to refer to either an output type that couldn't be parsed, or to an automatic change output mode. This change drops the NONE enum and uses a simple bool to indicate parse failure, and a new CHANGE_AUTO enum to refer the change output type.
This change is almost a pure refactoring except it makes RPCs reject empty string ("") address types instead of treating them like they were unset. This simplifies the parsing code a little bit and could prevent RPC usage mistakes. It's noted in the release notes.
Follows up #12408 by @MarcoFalke
Followups for future PRs:
- [ ] Add explicit support for specifying "auto" in `ParseOutputType` as suggested by promag and sipa: https://github.com/bitcoin/bitcoin/pull/12729#issuecomment-374799567 and https://github.com/bitcoin/bitcoin/pull/12729#discussion_r175969481
- [ ] Add wallet `AddressChangeType` method to complement `TransactionChangeType`: https://github.com/bitcoin/bitcoin/pull/12729#discussion_r175969618.
Tree-SHA512: 8b08b272bcb177a0a9e556dcd965840a7fe601ef83ca97938b879c9b1a33b5b3f96939e1bceef11ba7c644ac21bfd6c1dbc6ca715cd1da4ace50475240e4ee48
d8e9a2a Remove "rpc" category from GetWarnings (Wladimir J. van der Laan)
7da3b0a rpc: Move RPC_FORBIDDEN_BY_SAFE_MODE code to reserved section (Wladimir J. van der Laan)
2ae705d Remove Safe mode (Andrew Chow)
Pull request description:
Rebase of #10563. Safe mode was [disabled by default and deprecated in 0.16](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.16.0.md#safe-mode-disabled-by-default), so probably should be removed for 0.17.
> Rationale:
>
> Safe mode is useless. It only disables some RPC commands when large work forks are detected. Nothing else is affected by safe mode. It seems that very few people would be affected by safe mode. The people who use Core as a wallet are primarily using it through the GUI, which safe mode does not effect. In the GUI, transactions will still be made as normal; only a warning is displayed.
>
> I also don't think that we should be disabling RPC commands or any functionality in general. If we do, it should be done consistently, which safe mode is not. If we want to keep the idea of a safe mode around, I think that the current system needs to go first before a new system can be implemented.
Tree-SHA512: 067938f47ca6e879fb6c3c4e21f9946fd7c5da3cde67ef436f1666798c78d049225b9111dc97064f42b3bc549d3915229fa19ad5a634588f381e34fc65d64044
7d0f80b Use anonymous namespace instead of static functions (Pieter Wuille)
b61fb71 Mention removal of bare multisig IsMine in release notes (Pieter Wuille)
9c2a8b8 Do not treat bare multisig as IsMine (Pieter Wuille)
08f3228 Optimization: only test for witness scripts at top level (Pieter Wuille)
3619735 Track difference between scriptPubKey and P2SH execution in IsMine (Pieter Wuille)
ac6ec62 Switch to a private version of SigVersion inside IsMine (Pieter Wuille)
19fc973 Do not expose SigVersion argument to IsMine (Pieter Wuille)
fb1dfbb Remove unused IsMine overload (Pieter Wuille)
952d821 Make CScript -> CScriptID conversion explicit (Pieter Wuille)
Pull request description:
Currently our wallet code will treat bare multisig outputs (meaning scriptPubKeys with multiple public keys + `OP_CHECKMULTISIG` operator in it) as ours without the user asking for it, as long as all private keys in it are in our wallet.
This is a pointless feature. As it only works when all private keys are in one place, it's useless compared to single key outputs (P2PK, P2PKH, P2WPKH, P2SH-P2WPKH), and worse in terms of space, cost, UTXO size, and ability to test (due to lack of address format for them).
Furthermore, they are problematic in that producing a list of all `scriptPubKeys` we accept is not tractable (it involves all combinations of all public keys that are ours). In further wallet changes I'd like to move to a model where all scriptPubKeys that are treated as ours are explicit, rather than defined by whatever keys we have. The current behavior of the wallet is very hard to model in such a design, so I'd like to get rid of it.
I think there are two options:
* Remove it entirely (do not ever accept bare multisig outputs as ours, unless watched)
* Only accept bare multisig outputs in situations where the P2SH version of that output would also be acceptable
This PR implements the first option. The second option was explored in #12874.
Tree-SHA512: 917ed45b3cac864cee53e27f9a3e900390c576277fbd6751b1250becea04d692b3b426fa09065a3399931013bd579c4f3dbeeb29d51d19ed0c64da75d430ad9a
41ff967 list the types of scripts we should consider for a witness program (fivepiece)
4f933b3 p2wpkh, p2wsh and p2sh-nested scripts in decodescript (fivepiece)
Pull request description:
Attempts to address #12244 . `p2wsh` addresses are returned only for scripts that are neither `p2sh` nor any witness program.
Tree-SHA512: eb47f094c1a4c2ad2bcf27a8032307e43cf787d50bf739281aeb4101d97316a2f307b05118bf138298c937fa34e15f91436443a9b313f809fad2c43e94cd1831
7de1de7 Add new fee structure with all sub-fields denominated in BTC (mryandao)
Pull request description:
the denomination for `fee` is current in btc while the other such as `decendentFee` and `ancestorFee` are in satoshis.
Tree-SHA512: e428f6dca1d339f89ab73e38ce5903f5465c46b159069d9bcc3f8b1140fe6657fa49a11abe0088e9f7ba9999f64af72a349a4735bf5eaa61b8e4a185b23543f3
fa811b0 qa: Normalize executable location (MarcoFalke)
Pull request description:
This removes the need to override the executable locations by just reading them from the config file. Beside making the code easier to read, running individual test on Windows is now possible by default (without providing further command line arguments).
Note: Of course, it is still possible to manually specify the location through the `BITCOIND` environment variable, e.g. `bitcoin-qt`.
Tree-SHA512: bee6d22246796242d747120ca18aaab089f73067de213c9111182561985c5912228a0b0f7f9eec025ecfdb44db031f15652f30d67c489d481c995bb3232a7ac7
fac0db0 wallet: Make fee settings non-static members (MarcoFalke)
Pull request description:
The wallet header defined some globals (they were called "settings"), that should be class members instead.
This commit is hopefully only refactoring, apart from a multiwallet bugfix: Calling the rpc `settxfee` for one wallet, would set (and change) the fee rate for all loaded wallets. (See added test case)
Tree-SHA512: 4ab6ec2f5c714742396ded5e451ec3b1ceb771e3696492de29889d866de4365b3fbe4a2784d085c8b8bd11b1ebb8a1fec99ab2c62eee716791cfc67c0cf29e1b
cead28b [docs] Add release notes for deprecated 'account' API (John Newbery)
72c9575 [wallet] [tests] Add tests for accounts/labels APIs (John Newbery)
109e05d [wallet] [rpc] Deprecate wallet 'account' API (John Newbery)
3576ab1 [wallet] [rpc] Deprecate account RPC methods (John Newbery)
3db1ba0 [tests] Set -deprecatedrpc=accounts in tests (John Newbery)
4e671f0 [tests] Rename rpc_listtransactions.py to wallet_listtransactions.py (John Newbery)
a28b907 [wallet] [rpc] Remove duplicate entries in rpcwallet.cpp's CRPCCommand table (John Newbery)
Pull request description:
Deprecate all accounts functionality and make it only accessible by using `-deprecatedrpc=accounts`.
Accounts specific RPCs, account arguments, and account related results all require the `-deprecatedrpc=accunts` startup option now in order to see account things.
Several wallet functional tests use the accounts system. Those tests are unchanged, except to start the nodes with `-deprecatedrpc=accounts`. We can slowly migrate those tests to use the 'label' API instead of the 'account' API before accounts are fully removed.
Tree-SHA512: 89f4ae2fe6de4a1422f1817b0997ae22d63ab5a1a558362ce923a3871f3e42963405d6573c69c27f1764679cdee5b51bf52202cc407f1361bfd8066d652f3f37
8b8032e test: Add rpcauth pair that generated by rpcauth (Chun Kuan Lee)
Pull request description:
This PR adds a rpcauth pair that is randomly generated. Also checks that rpcauth.py works fine.
Resolve#12995
Tree-SHA512: d9661f40e306bcf528dc25919c874ebcdbdd21101319985dc12ce133c80fd0021cfee5e4bfe8ee7970eccc2e24c97e596263b270fe0b79f3613ae573a825ed63
09b30db Asserts that the tx version number is a signed 32-bit integer. (251)
Pull request description:
This PR attempts to resolve#11561 by addressing the feedback from @MarcoFalke; and @gmaxwell in #12430.
Commit 30e9d24 adds a functional test to `rpc_rawtransaction.py` to assert that the transaction version number in the RPC output is a signed 32-bit integer.
The functional test uses the raw transaction data from Mainnet transaction `c659729a7fea5071361c2c1a68551ca2bf77679b27086cc415adeeb03852e369`.
Tree-SHA512: d78f3120b9aa04537561ab5584769a838b25e162c5caa6e1543256fb27538aa4c708c939fb5ba93ccb3fa676c2d92ce8eb9cc78869f80ac96be64a7bec7bebd0
1accfbc Output values for "min relay fee not met" error (Kristaps Kaupe)
Pull request description:
It is already done this way for "mempool min fee not met" error.
Tree-SHA512: 829db78ecc066cf93b8e93ff1aeb4e7b98883cf45f341d5be6e6b4dff4135f3f54fa49b3a6f12eb43f676a9ba54f981143c9887f786881e584370434a9566cfd
80a5e59 [qa] Attach node index to test_node AssertionError and print messages (James O'Beirne)
Pull request description:
In the midst of fighting with https://github.com/bitcoin/bitcoin/pull/12873 it became apparent that there're a number of assertions and print statements which are emitted by test nodes but don't identify the node in question. This change makes debugging a bit easier by adding identifying information to non-logger test_node-related error messages.
Tree-SHA512: 7cc86f2c81f4b3fdba15ec9a2d21a84c4b083629e845e82288087c3affbbdc5c68e74067621856cc97fe84fbc8cb4f5ca4977a51ef381e5d74515df8eb001239
1f83839 [wallet] [tests] Test disallowed multiwallet params (John Newbery)
3476e3c [wallet] Fix zapwallettxes/multiwallet interaction. (John Newbery)
Pull request description:
`-zapwallettxes` should be disallowed when starting bitcoin in multiwallet mode.
There's code in `WalletInit::ParameterInteraction()` to disallow `-zapwallettxes` when running in multiwallet mode. This code functioned as expected when passing the parameter `-zapwallettxes=1`, but not when passing the parameter `-zapwallettxes` (ie without the value specified). Fix that and add a test.
The new test in the
_[wallet] [tests] Test disallowed multiwallet params_ commit reproduces the bug and should fail against master.
Fixes#12505
Tree-SHA512: 6cd921717e9c7d2773ca84c946c310c2adec8430e37cbecdb33a620f510db3058a72bd328411812ba415111bc52a3367b332c9d15a37a92ccfd7ae785d2f32ab
e87fefc test: Add rpc_bind test to default-run tests (Wladimir J. van der Laan)
Pull request description:
Skip the parts that cannot be run on the host due to lack of IPv6 support or a second interface to bind on, and warn appropriately.
Without no strong requirements (besides being Linux only, which will skip the test) left, add this test to the default in test_runner.
~~(the non-IPv6 parts of the two dual-IPv4/6 tests could also be enabled, but first going to look what Travis does here to see if there wasn't another reason it was disabled)~~ done, it only makes sense for the first
Tree-SHA512: 724259b14f59dccc7e61ef071359336adb0f76a63db392b6ce6940e21c8ee0470c35374e82970681261685ef299cd70b0c1372598cea85d341f64c2c40ea28ee
c1d742025c [tests] Fix feature_block flakiness (John Newbery)
Pull request description:
feature_block.py occasionally fails on Travis. I believe this is due to
a a race condition when reconnecting to bitcoind after a subtest that
expects disconnection. If the test runs ahead and sends the INV for the
subsequent test before we've received the initial sync getheaders, then
we may end up sending two headers messages - one as a response to the
initial sync getheaders and one in response to the INV getheaders. If
both of those headers fail validation with a DoS score of 50 or higher,
then we'll unexpectedly be disconnected.
There is only one validation failure that has a DoS score bewteen 50 and
100, which is high-hash. That's why the test is failing immediately
after the "Reject a block with invalid work" subtest.
Fix is to wait for the initial getheaders from the peer before we
start populating our blockstore. That way we won't have any invalid
headers to respond to it with.
Tree-SHA512: dc17d795fcfaf0f8c0bf1e9732b5e11fbc8febbfafba4c231b7c13a5404a2c297dcd703a7a75bc7f353c893e12efc87f424f2201abd47ba5268af32d4d2e841f
feature_block.py occasionally fails on Travis. I believe this is due to
a a race condition when reconnecting to bitcoind after a subtest that
expects disconnection. If the test runs ahead and sends the INV for the
subsequent test before we've received the initial sync getheaders, then
we may end up sending two headers messages - one as a response to the
initial sync getheaders and one in response to the INV getheaders. If
both of those headers fail validation with a DoS score of 50 or higher,
then we'll unexpectedly be disconnected.
There is only one validation failure that has a DoS score bewteen 50 and
100, which is high-hash. That's why the test is failing immediately
after the "Reject a block with invalid work" subtest.
Fix is to wait for the initial getheaders from the peer before we
start populating our blockstore. That way we won't have any invalid
headers to respond to it with.
