341f7c7b0e macOS fix: Check for correct version of flake8 to avoid spurious warnings. The brew installed flake8 version is Python 2 based and does not work. (practicalswift)
908a559f33 macOS fix: Add excludes for checks added in the newer shellcheck version installed by brew (practicalswift)
ec4d57bbb3 macOS fix: Work around empty (sub)expression error when using BSD grep (practicalswift)
b57d7d92fe macOS fix: Avoid mapfile due to ancient version of bash shipped with macOS (practicalswift)
Pull request description:
The linters are thoroughly tested under Ubuntu which is what we use in Travis. When reading #14041 I understood that some developers were experiencing problems when running the linters on their local machines.
Assuming these local machines were running macOS I installed a fresh macOS VM, followed the instructions in `build-osx.md` and ran the linters.
This PR contains the changes needed to make `lint-all.sh` run as expected.
Ideally the linters would continuously run also under a Travis macOS environment to make sure we catch these kind of issues before merge.
Tree-SHA512: b39c9a970d14d27db1fb592539923c0bc676b5217f415d02fda3f17bf54d46faa172376e8a3ecab07ca68a3acba9aebe00b2b1b2161b2a36b85fbb672e7efb5c
5d62dcf9cf lint: Make sure we read the command line inputs using utf-8 decoding in python (Chun Kuan Lee)
Pull request description:
Make sure we read the command line inputs using utf-8 decoding in python
occurred from travis cron job:
contrib/verify-commits/verify-commits.py should run with utf-8, otherwise it would raise UnicodeDecodeError
`UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 744: ordinal not in range(128)`
Tree-SHA512: 90e4ad57fdbbbecb0a21fc2d2b03a04f5ef125e54124719ef36e5a85326930b732b47534757a7c3a8730096f3947b009ec898191928b5c2d38f9f4b3e37db48d
ca1a093127 Add regression test: Don't assert(...) with side effects (practicalswift)
4c3c9c3869 Don't assert(...) with side effects (practicalswift)
Pull request description:
Don't `assert(...)` with side effects.
From the developer notes:
> **Assertions should not have side-effects**
>
> Rationale: Even though the source code is set to refuse to compile with assertions disabled, having side-effects in assertions is unexpected and makes the code harder to understand
These assertions were introduced quite recently (in #14069 which was merged two days ago) and since this is a recurring thing (see #13534 – "Don't assert(foo()) where foo() has side effects" from May) I added a simple regression test for the most obvious common side effect.
Tree-SHA512: be65db9d8d5d0f5752152ba73fe3fbb0531880f156d3cd7dfdf1752709979b63214e46ae64b1adbe1e09fa121278f4087f4ae49bff16cf8f5aec16ea6bde3650
6af6d9b23d test: Add tests for RPC help (João Barbosa)
Pull request description:
At the moment the new test checks for:
- invalid usages
- expected output for unknown command
- current RPC command titles (derived from command categories) — this prevents adding wrong RPC categories and new categories must be added to the test
Tree-SHA512: f987535d001b1cd300656588602b1634099ea68a1dd2282180c30fa56caf7f990be9e2dc86c7431dfcf7fd686d0299a8d4935df178a2c9f0fb6fbebcba748eb5
b193d5a443 Removes the Boost case_conv.hpp dependency. (251)
7a208d9fad Implements custom tolower and toupper functions. (251)
e2ba043b8d Implements ParseNetwork unit test. (251)
Pull request description:
This pull request removes the `boost/algorithm/string/case_conv.hpp` dependency from the project.
`boost/algorithm/string/case_conv.hpp` is included for the `boost::to_lower` and `boost::to_upper` template functions.
We can replace the calls to these functions with straightforward alternative implementations that use the C++ Standard Library, because the functions are called with `std::string` objects that use standard 7-bit ASCII characters as argument.
The refactored implementation should work without the explicit `static_cast<unsigned char>` cast and `unsigned char` lambda return type. Both have been added defensively and to be explicit. Especially in case of the former, behaviour is undefined (potentially result in a crash) if the `std::toupper` argument is not an `unsigned char`.
A potential alternative, maybe even preferred, implementation to address the `boost::to_lower` function call in `ParseNetwork(std::string)` could have been:
```c++
if (net == "ipv4" || net == "IPv4") return NET_IPV4;
if (net == "ipv6" || net == "IPv6") return NET_IPV6;
```
This alternative implementation would however change the external behaviour of `ParseNetwork(std::string)`.
This pull requests includes a unit test to validate the implementation of `ParseNetwork(std::string)` prior and after the removal of the `case_conv.hpp` dependency.
`boost/algorithm/string/case_conv.hpp` has been removed from the `EXPECTED_BOOST_INCLUDES` in `test/lint/lint-includes.sh` because it is no longer required.
Tree-SHA512: d803ae709f2368a3efb223097384a722436955bce0c44a1a5cffd0abb3164be0cce85ba0e9ebd9408166df3f1a95ea0c0d29e3a2534af2fae206c0419d67fde9
1661a472b8 add unicode compatible file_lock for Windows (Chun Kuan Lee)
Pull request description:
boost::interprocess::file_lock cannot open the files that contain characters which cannot be parsed by the user's code page on Windows.
This PR is seperated from #13426 for easier review.
Tree-SHA512: e240479cda65958bf6e1319840b83928b2b50da81d99f4f002fb3b62621370bcd4bcfacd2b8c0678c443a650d6ba53d9d12618b591e5bfd67ac14388a18fd822
fae040010d qa: Add some actual witness in rpc_rawtransaction (MarcoFalke)
Pull request description:
The transaction was serialized with the witness flag but didn't include any witness, so add some dummy witness...
Tree-SHA512: fe71167c16e9b0053110be7c544e1ea08868f04ffee8d4c74887c9bcdcd5b59d5e8dd53c67e104a1bdbbf606202bc3fbef6017f402f2c75bdb2ebd9f7aabb2b1
This commit removes the `boost/algorithm/string/case_conv.hpp` dependency from the project. It replaces the `boost::to_lower` and `boost::to_upper` functions with custom functions that are locale independent and ASCII deterministic.
16e288acdd test padding non micro timestamps (John Newbery)
995dd89d88 [Tests] Make combine_logs.py handle multi-line logs (John Newbery)
Pull request description:
combine_logs.py currently inserts additional newlines into multi-line
log messages, and doesn't color them properly. Fix both of those.
Tree-SHA512: dbe2f3ecc7cfbc95ee4350e648d127538c79cb6555257d4aeec12fe3d159366742b68e90e620c8ed7219a44b973395c7e5929ba374fae115fbee25560db645f6
boost::interprocess::file_lock cannot open the files that contain characters which cannot be parsed by the user's code page on Windows.
This commit add a new class to handle those specific file for Windows.
bb08423d5c [doc] Add release notes for 'account' API removal (John Newbery)
1f4b865e57 [wallet] Re-sort wallet RPC commands (John Newbery)
f0dc850bf6 [wallet] Remove wallet account RPCs (John Newbery)
c410f41575 [tests] Remove wallet accounts test (John Newbery)
Pull request description:
This is the first part of #13825. It simply removes the RPC methods and tests.
#13825 touches lots of files and will require frequent rebasing.
Breaking it down for easier reviewing and fewer rebases.
Tree-SHA512: d29af8e7a035e4484e6b9bb56cb86592be0ec112d8ba4ce19c15d15366ff3086e89e99fca26b90c9d66f6d3e06894486d0f29948df0bb7dcb1e2c49c6887a85a
414326952c use export LC_ALL=C.UTF-8 (Julian Fleischer)
728c82d029 make script exit if a command fails (Julian Fleischer)
506890b24d move remaining travis build steps into individual files (Julian Fleischer)
272306ea57 number .travis/ script according to build lifecycle and add README to explain (Julian Fleischer)
519e2739cf move lint stage up to resemble travis build ui (Julian Fleischer)
86d34f0e65 abort script in END_FOLD on non-zero exit code (Julian Fleischer)
4f2f88c7b0 move script sections info individual files and comply with shellcheck (Julian Fleischer)
Pull request description:
This PR is extracted from https://github.com/bitcoin/bitcoin/pull/13816 to make that one easier to review. It follows on https://github.com/bitcoin/bitcoin/pull/13849 and https://github.com/bitcoin/bitcoin/pull/13851
In here the shell script parts from `travis.yml` are extracted into `.travis/before_install.sh`, `.travis/install.sh`, `.travis/before_script.sh`, `.travis/script.sh`, and `.travis/lint.sh`.
This has the benefit that `test/lint/lint-shell.sh` will also shellcheck these parts. Also it makes the individual script parts more readable.
Tree-SHA512: c497e1687ceb1c1d795de177d3fc35af908bc8e3f781a871afabdecf031e581d4db229290627249e35ef7c09952bc34884e4734ea91d40f57b4a9efb85bba2e3
5778bf95d9 Report minfeefilter value in getpeerinfo rpc (Anthony Towns)
Pull request description:
Lowering the minimum relay fee is only useful when many nodes in the p2p network also lower the fee, so to make it easier to understand progress on that front, this includes the value of the minfeefilter in getpeerinfo, so you at least have visibility to what fees your neighbours are currently accepting.
Tree-SHA512: 059f01bf2a32c98fce1648a13b7898701203b354d0209ee34e6683994b720eb594cf24968e66b699caae5e17e53d351e73281f042dd094decde14d3a318e9fb3
Lowering the minimum relay fee is only useful when many nodes in the
p2p network also lower the fee, so to make it easier to understand
progress on that front, this includes the value of the minfeefilter in
getpeerinfo, so you at least have visibility to what fees your neighbours
are currently accepting.
870bd4c73d Update functional RBF test to check replaceable flag (dexX7)
820d31f95f Add "bip125-replaceable" flag to mempool RPCs (dexX7)
Pull request description:
This pull request adds a flag "bip125-replaceable" to the mempool RPCs getrawmempool, getmempoolentry, getmempoolancestors and getmempooldescendants, which indicates whether an unconfirmed transaction might be replaced.
Initially the flag was added to the raw transaction RPCs, but thanks to @conscott, it was moved to the mempool RPCs, which actually have access to the mempool.
~~This pull request adds a flag "bip125-replaceable" to the RPCs "getrawtransaction" and "decoderawtransaction", which indicates, whether a transaction signals BIP 125 replaceability.~~
There was some discussion in #7817, whether showing replaceability in the UI could lead to the false assumption that transactions that don't signal BIP 125 are truely non-replaceable, but given that this PR tackles the raw transaction interface, which is a rather low level tool, I believe having this extra piece of information isn't bad.
Tree-SHA512: 1f5511957af2c20a9a6c79d80a335c3be37a2402dbf829c40cceaa01a24868eab81a9c1cdb0b3d77198fa3bb82799e3540a5c0ce7f35bbac80d73f7133ff7cbc
317f2cb3f4 test: Check RPC settxfee errors (João Barbosa)
48618daf26 Add checks for settxfee reasonableness (Anthony Towns)
Pull request description:
When using the `settxfee` RPC, the value is silently ignored if it is less than either than minrelaytxfee or the wallet's mintxfee. This adds an error response if that's going to happen, but still allows "settxfee 0" to deliberately default to the minimum value.
Tree-SHA512: ce685584cf8d6b9ca2cc97196d494220e3892b6a804a458086e04b3a23df281da432ad0a3053106a064c90c541ddb6f6b96a27cf8376d45af1e44449baf88456
faaac5caaa RPCTypeCheck bip32derivs arg in walletcreatefunded (Gregory Sanders)
1f0c4282e9 QA: add basic walletcreatefunded optional arg test (Gregory Sanders)
1f18d7b591 walletcreatefundedpsbt: remove duplicate replaceable arg (Gregory Sanders)
2252ec5008 Allow ConstructTransaction to not throw error with 0-input txn (Gregory Sanders)
Pull request description:
1) Previously an empty input argument transaction that is marked for replaceability fails to pass the `SignalsOptInRBF` check right before funding it. Explicitly check for that condition before throwing an error.
2) The rpc call had two separate `replaceable` arguments, each of which being used in mutually exclusive places. I preserved the `options` version to retain compatability with `fundtransaction`.
Tree-SHA512: 26eb0c9e2d38ea51d11f741d61100223253271a084adadeb7e78c6d4e9004636f089e4273c5bf64a41bd7e9ff795317acf30531cb36aeb0d8db9304b3c8270c3
fa091b0016 qa: Add tests for submitheader (MarcoFalke)
36b1b63f20 rpc: Expose ProcessNewBlockHeaders (MarcoFalke)
Pull request description:
This exposes `ProcessNewBlockHeaders` as an rpc called `submitheader`. This can be used to check for invalid block headers and submission of valid block headers via the rpc.
Tree-SHA512: a61e850470f15465f88e450609116df0a98d5d9afadf36b2033d820933d8b6a4012f9f2b3246319c08a0e511bef517f5d808cd0f44ffca91d10895a938004f0b
5df6f089b5 More tests of signer checks (Andrew Chow)
7c8bffdc24 Test that a non-witness script as witness utxo is not signed (Andrew Chow)
8254e9950f Additional sanity checks in SignPSBTInput (Pieter Wuille)
c05712cb59 Only wipe wrong UTXO type data if overwritten by wallet (Pieter Wuille)
Pull request description:
The current PSBT signing code can end up producing a non-segwit signature, while only the UTXO being spent is provided in the PSBT (as opposed to the entire transaction being spent). This may be used to trick a user to incorrectly decide a transaction has the semantics he intends to sign.
Fix this by refusing to sign if there is any mismatch between the provided data and what is being signed.
Tree-SHA512: b55790d79d8166e05513fc4c603a982a33710e79dc3c045060cddac6b48a1be3a28ebf8db63f988b6567b15dd27fd09bbaf48846e323c8635376ac20178956f4
bd19cc78cf Serialize non-witness utxo as a non-witness tx but always deserialize as witness (Andrew Chow)
43811e6338 Fix PSBT deserialization of 0-input transactions (Andrew Chow)
Pull request description:
0-input transactions can be ambiguously deserialized as being witness transactions. Since the unsigned transaction is never serialized as a witness transaction as it has no witnesses, we should always deserialize it as a non-witness transaction and set the serialization flags as such.
When a transaction is serliazed for the non-witness-utxo, it is always a valid network transaction and thus it should be always be deserialized as a witness transaction and the deserialzation flags are set as such.
Fixes#13958
Tree-SHA512: 1937b3cb2618534478d4f533541fb9efce3cb5badb5d1964bfe19400f4aacc6c8ecedaf1f20d26b20baf94f81fd07dfb15b3b08089ecbd63aeecbc18c7c48086
0-input transactions can be ambiguously deserialized as being witness
transactions. Since the unsigned transaction is never serialized as
a witness transaction as it has no witnesses, we should always
deserialize it as a non-witness transaction and set the serialization
flags as such.
Also always serialize the unsigned transaction as a non-witness transaction.
68400d8b96 tests: Use explicit imports (practicalswift)
Pull request description:
Enable automatic detection of undefined names in Python tests scripts. Remove wildcard imports.
Wildcard imports make it unclear which names are present in the namespace, confusing both readers and many automated tools.
An additional benefit of not using wildcard imports in tests scripts is that readers of a test script then can infer the rough testing scope just by looking at the imports.
Before this commit:
```
$ contrib/devtools/lint-python.sh | head -10
./test/functional/feature_rbf.py:8:1: F403 'from test_framework.util import *' used; unable to detect undefined names
./test/functional/feature_rbf.py:9:1: F403 'from test_framework.script import *' used; unable to detect undefined names
./test/functional/feature_rbf.py:10:1: F403 'from test_framework.mininode import *' used; unable to detect undefined names
./test/functional/feature_rbf.py:15:12: F405 bytes_to_hex_str may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:17:58: F405 CScript may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:25:13: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:26:31: F405 satoshi_round may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:26:60: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:30:41: F405 satoshi_round may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:30:68: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
$
```
After this commit:
```
$ contrib/devtools/lint-python.sh | head -10
$
```
Tree-SHA512: 3f826d39cffb6438388e5efcb20a9622ff8238247e882d68f7b38609877421b2a8e10e9229575f8eb6a8fa42dec4256986692e92922c86171f750a0e887438d9
To avoid:
$ make check
{…omissis…}
Running test/util/bitcoin-util-test.py...
/usr/local/bin/python3.7 ../test/util/bitcoin-util-test.py
../test/util/bitcoin-util-test.py:31: DeprecationWarning: This method will be removed in future versions. Use 'parser.read_file()' instead.
config.readfp(open(os.path.join(os.path.dirname(__file__), "../config.ini"), encoding="utf8"))
$ python3 --version
Python 3.7.0
cf9ed307e6 qa: blocktools enforce named args for amount (MarcoFalke)
Pull request description:
Since #13669 changed some signatures, I think it might be worthwhile to enforce named args for primitive types such as amounts.
Tree-SHA512: 2733e7b6a20590b54bd54e81a09e3f5e2fadf4390bed594916b70729bcf485b048266012c1203369e0968032a2c6a2719107ac17ee925d8939af3df916eab1a6
4b7091a842 Replace median fee rate with feerate percentiles (Marcin Jachymiak)
Pull request description:
Currently, the `medianfeerate` statistic is calculated from the feerate of the middle transaction of a list of transactions sorted by feerate.
This PR instead uses the value of the 50th percentile weight unit in the block, and also calculates the feerate at the 10th, 25th, 75th, and 90th percentiles. This more accurately corresponds with what is generally meant by median feerate.
Tree-SHA512: 59255e243df90d7afbe69839408c58c9723884b8ab82c66dc24a769e89c6d539db1905374a3f025ff28272fb25a0b90e92d8101103e39a6d9c0d60423a596714
4441ad677a Make format string linter understand basic template parameter syntax (practicalswift)
Pull request description:
Make format string linter understand basic template parameter syntax.
Fixes issue described in https://github.com/bitcoin/bitcoin/pull/13705#issuecomment-412046126.
Thanks to @ken2812221 for reporting!
Tree-SHA512: 8fb995bc6b29d8b9926ef5969e02cf71c494e829434fcdeb4ed5fabad5ab96e86e5b8eea705e8a416927757b4fa4e58abc0fd4f483daa58c94e2c6fdcb8ee822
e306be7429 Use 72 byte dummy signatures when watching only inputs may be used (Andrew Chow)
48b1473c89 Use 71 byte signature for DUMMY_SIGNATURE_CREATOR (Andrew Chow)
18dfea0dd0 Always create 70 byte signatures with low R values (Andrew Chow)
Pull request description:
When creating signatures for transactions, always make one which has a 32 byte or smaller R and 32 byte or smaller S value. This results in signatures that are always less than 71 bytes (32 byte R + 32 byte S + 6 bytes DER + 1 byte sighash) with low R values. In most cases, the signature will be 71 bytes.
Because R is not mutable in the same way that S is, a low R value can only be found by trying different nonces. RFC 6979 for deterministic nonce generation has the option to specify additional entropy, so we simply use that and add a uin32_t counter which we increment in order to try different nonces. Nonces are sill deterministically generated as the nonce used will the be the first one where the counter results in a nonce that results in a low R value. Because different nonces need to be tried, time to produce a signature does increase. On average, it takes twice as long to make a signature as two signatures need to be created, on average, to find one with a low R.
Having a fixed size signature makes size calculations easier and also saves half a byte of transaction size, on average.
DUMMY_SIGNATURE_CREATOR has been modified to produce 71 byte dummy signatures instead of 72 byte signatures.
Tree-SHA512: 3cd791505126ce92da7c631856a97ba0b59e87d9c132feff6e0eef1dc47768e81fbb38bfbe970371bedf9714b7f61a13a5fe9f30f962c81734092a4d19a4ef33
Removes medianfeerate result from getblockstats.
Adds feerate_percentiles which give the feerate of the 10th, 25th, 50th,
75th, and 90th percentile weight unit in the block.
fafe73a626 qa: Raise feature_help timeout to 5s (MarcoFalke)
faabd7bc47 qa: Use files for stdout/stderr to support Windows (MarcoFalke)
facb56ffaf qa: Run gen_rpcauth with sys.executable (MarcoFalke)
fada8966c5 qa: Close stdout and stderr file when node stops (MarcoFalke)
Pull request description:
### qa: Close stdout and stderr file when node stops
Since these files are potentially deleted by the test framework for cleanup, they should be closed first. Otherwise this will lead to errors on Windows when the tests finish successfully.
Side note: After the patch, it is no longer possible to reopen the file on Windows (see https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile)
### qa: Run gen_rpcauth with sys.executable
Similar to `test_runner.py`, the `sys.executable` needs to be passed down into subprocesses to pass on native Windows. (Should have no effect on Linux)
### qa: Use files for stdout/stderr to support Windows
It seems that using PIPE is not supported on Windows. Also, it is easier to just use the files that capture the stdout and stderr within the test node class.
Tree-SHA512: ec675012b10705978606b7fcbdb287c39a8e6e3732aae2fa4041d963a3c6993c6eac6a9a3cbd5479514e7d8017fe74c12235d1ed6fed2e8af8f3c71981e91864
fa85c985ed qa: Add p2p_invalid_locator test (MarcoFalke)
Pull request description:
Should not be merged *before* #13907
Tree-SHA512: a67ca407854c421ed20a184d0b0dc90085aed3e3431d9652a107fa3022244767e67f67e50449b7e95721f56906836b134615875f28a21e8a012eb22cfe6a66a5
1f87c372b5 Simplify comparison in rpc_blockchain.py. (Daniel Kraft)
Pull request description:
The test for `gettxoutsetinfo` in `rpc_blockchain.py` verifies that the result is the same as before after invalidating and reconsidering a block. The comparison has to exclude the `disk_size` field, though, as it is not deterministic.
Instead of comparing all the other fields for equality, this change explicitly removes the `disk_size` field and then compares the full objects. This makes the intent more explicit (compare everything except for `disk_size`, not compare just a given list of fields) and also the code simpler.
Tree-SHA512: 3c376a8836b62988fb2f0117c9ca65de64a33bf3cd4980a123de30bf5e7b7a48eda477b25e03d672ff076e205c698e83432469156caa0f0f3ebbb0480f0dd77d
When extra entropy is not specified by the caller, CKey::Sign will
now always create a signature that has a low R value and is at most
70 bytes. The resulting signature on the stack will be 71 bytes when
the sighash byte is included.
Using low R signatures means that the resulting DER encoded signature
will never need to have additional padding to account for high R
values.
The test for gettxoutsetinfo in rpc_blockchain.py verifies that the
result is the same as before after invalidating and reconsidering a
block. The comparison has to exclude the 'disk_size' field, though, as
it is not deterministic.
Instead of comparing all the other fields for equality, this change
explicitly removes the 'disk_size' field and then compares the full
objects. This makes the intent more explicit (compare everything except
for disk_size, not compare just a given list of fields) and also the
code simpler.
fa5587fe71 qa: wait_for_verack by default (MarcoFalke)
Pull request description:
This removes the need to do so manually every time a connection is added.
Tree-SHA512: a46c92cb4df41e30778b42b9fd3dcbd8d2d82aa7503d1213cb1c1165034f648d8caee01c292e2d87d05b0f71696996eef5be8a753f35ab49e5f66b0e3bf29f21
3fc20632a3 qt: Set BLOCK_CHAIN_SIZE = 220 (DrahtBot)
2b6a2f4a28 Regenerate manpages (DrahtBot)
eb7daf4d60 Update copyright headers to 2018 (DrahtBot)
Pull request description:
Some trivial maintenance to avoid having to do it again after the 0.17 branch off.
(The scripts to do this are in `./contrib/`)
Tree-SHA512: 16b2af45e0351b1c691c5311d48025dc6828079e98c2aa2e600dc5910ee8aa01858ca6c356538150dc46fe14c8819ed8ec8e4ec9a0f682b9950dd41bc50518fa
ParseHashV validates the length and encoding of the string and throws
an informative RPC error on failure, which is as good or better than
these alternative calls.
Note I switched ParseHashV to check string length first, because
IsHex tests that the length is even, and an error like:
"must be of length 64 (not 63, for X)" is much more informative than
"must be hexadecimal string (not X)"
faa24441ec policy: Remove promiscuousmempoolflags (MarcoFalke)
Pull request description:
It seems odd to clutter validation code with features that can only ever be used for testing (testnet or regtest). Removing that test-only code makes the mempool logic less painful to understand and easier to reason about when changed or refactored in the future.
Tree-SHA512: 3b897aa9604ac8d82ebe9573c6efd468c93ddaa08d378ebc902e247b7aa6c68fcde71e5b449c08f17a067146cdc66dc50a67ce06d07607c27e5189a49c3fba3f
This linter checks that the number of arguments passed to each variadic format
string function matches the number of format specifiers in the format string.
a1a998cf24 wallet: Fix backupwallet for multiwallets (Daniel Kraft)
Pull request description:
`backupwallet` was broken for multiwallets in their own directories (i.e. something like `DATADIR/wallets/mywallet/wallet.dat`). In this case, the backup would use `DATADIR/wallets/wallet.dat` as source file and not take the specific wallet's directory into account.
This led to either an error during the backup (if the wrong source file was not present) or would silently back up the wrong wallet; especially the latter behaviour can be quite bad for users.
Tree-SHA512: 7efe2450ca047e40719fcc7cc211ed94699056020ac737cada7b59e8240298675960570c45079add424d0aab520437d5050d956acd695a9c2452dd4317b4d2c4
fa67505e1e qa: Quote wallet name for rpc path (MarcoFalke)
Pull request description:
When using external multiwallets they are specified by their full path which might contain non-ascii characters (e.g. umlauts or emojis).
Fix this by url-quoting the path.
Tree-SHA512: 7cc66514579d9f602f88a6817c5ab43a44c6d3711df452dc904173f0bc34e2c0b2c34877497f06b61f6720c532fa183053f54622dc454e316c89cee7eaa72463
fa5b440971 qa: Extract rpc_timewait as test param (MarcoFalke)
Pull request description:
Also increase it for wallet_dump and wallet_groups
Tree-SHA512: 7367bc584228bda3010c453713a1505c54a8ef3d116be47dab9934d30594089dfeb27ffa862f7517fd0ec8b5dc07f4904d67ef2a53dd284cbe2a58982e410e2b
f6b7fc349c Support h instead of ' in hardened descriptor paths (Pieter Wuille)
fddea672eb Add experimental warning to scantxoutset (Jonas Schnelli)
6495849bfd [QA] Extend tests to more combinations (Pieter Wuille)
1af237faef [QA] Add xpub range tests in scantxoutset tests (Jonas Schnelli)
151600bb49 Swap in descriptors support into scantxoutset (Pieter Wuille)
0652c3284f Descriptor tests (Pieter Wuille)
fe8a7dcd78 Output descriptors module (Pieter Wuille)
e54d76044b Add simple FlatSigningProvider (Pieter Wuille)
29943a904a Add more methods to Span class (Pieter Wuille)
Pull request description:
As promised, here is an implementation of my output descriptor concept (https://gist.github.com/sipa/e3d23d498c430bb601c5bca83523fa82) and integration within the `scantxoutset` RPC that was just added through #12196.
It changes the RPC to use descriptors for everything; I hope the interface is simple enough to encompass all use cases. It includes support for P2PK, P2PKH, P2WPKH, P2SH, P2WSH, multisig, xpubs, xprvs, and chains of keys - combined in every possible way.
Tree-SHA512: 63b54a96e7a72f5b04a8d645b8517d43ecd6a65a41f9f4e593931ce725a8845ab0baa1e9db6a7243190d8ac841f6e7e2f520d98c539312d78f7fd687d2c7b88f
a13647b8bd [qa] Add test for too-large wallet output groups (Suhas Daftuar)
57ec1c97b2 [wallet] correctly limit output group size (Suhas Daftuar)
Pull request description:
Also add a test to ensure that output groups are being limited, even if a wallet has many outputs corresponding to the same scriptPubKey (the test fails without the first commit).
Tree-SHA512: 2aaa82005b0910488f5cbf40690d4c5e2f46949e299ef70b4cb6e440713811443d411dcbc6d71b1701fd82423073125e21747787d70830cd021c841afb732d51
fad231ad41 Fix merging of global unknown data in PSBTs (Andrew Chow)
41df035ee1 Check that PSBT keys are the correct length (Andrew Chow)
Pull request description:
This PR fixes a few bugs that were found and adds tests checking for these errors.
Specifically:
- Single byte keys are checked to actually be one byte.
- Unknown global data must be merged when combining two PSBTs.
Tree-SHA512: c0e7b4bc607d510cc005aaa7c0813ee58c5467ab7ce4adce485522dfeee92b1af3d29fe89df778b0ea812bb3827e085b30e04d4f4ebcefd8364d809573991332
12dd101345 scripted-diff: Remove trailing whitespaces (João Barbosa)
Pull request description:
The script test/lint/lint-whitespace.sh should prevent new cases.
This happens in some pulls where the code editor and the author 'git add's them, so this would fix it all.
Tree-SHA512: bcdd3472fcd01a2754e52212c7db1de2fdc422728b06785481954a27162fb72001cb73708329cc56e95bcc5e45c1348ebc4eacc2ccfa6aa12413c7ec450b6a33
e3245f2e7b Removes Boost predicate.hpp dependency (251)
Pull request description:
This pull request removes the `boost/algorithm/string/predicate.hpp` dependency from the project.
To replace the the `predicate.hpp` dependency from the project the function calls to `boost::algorithm::starts_with` and `boost::algorithm::ends_with` have been replaced with respectively C++11's `std::basic_string::front` and `std::basic_string::back` function calls.
Refactors that were not required, but have been done anyways:
- The Boost function `all` was implicitly made available via the `predicate.hpp` header. Instead of including the appropriate header, function calls to `all` have been replaced with function calls to `std::all_of`.
- The `boost::algorithm::is_digit` predicate has been replaced with a custom `IsDigit` function that is locale independent and ASCII deterministic.
Tree-SHA512: 22dda6adfb4d7ac0cabac8cc33e8fb8330c899805acc1ae4ede402c4b11ea75a399414b389dfaa3650d23b47f41351b4650077af9005d598fbe48d5277bdc320
232f96f5c8 doc: Add release notes for -avoidpartialspends (Karl-Johan Alm)
e00b4699cc clean-up: Remove no longer used ivars from CInputCoin (Karl-Johan Alm)
43e04d13b1 wallet: Remove deprecated OutputEligibleForSpending (Karl-Johan Alm)
0128121101 test: Add basic testing for wallet groups (Karl-Johan Alm)
59d6f7b4e2 wallet: Switch to using output groups instead of coins in coin selection (Karl-Johan Alm)
87ebce25d6 wallet: Add output grouping (Karl-Johan Alm)
bb629cb9dc Add -avoidpartialspends and m_avoid_partial_spends (Karl-Johan Alm)
65b3eda458 wallet: Add input bytes to CInputCoin (Karl-Johan Alm)
a443d7a0ca moveonly: CoinElegibilityFilter into coinselection.h (Karl-Johan Alm)
173e18a289 utils: Add insert() convenience templates (Karl-Johan Alm)
Pull request description:
This PR adds an optional (off by default) `-avoidpartialspends` flag, which changes coin select to use output groups rather than outputs, where each output group corresponds to all outputs with the same destination.
It is a privacy improvement, as each time you spend some output, any other output that is publicly associated with the destination (address) will also be spent at the same time, at the cost of fee increase for cases where coin select without group restriction would find a more optimal set of coins (see example below).
For regular use without address reuse, this PR should have no effect on the user experience whatsoever; it only affects users who, for some reason, have multiple outputs with the same destination (i.e. address reuse).
Nodes with this turned off will still try to avoid partial spending, if the fee of the resulting transaction is not greater than the fee of the original transaction.
Example: a node has four outputs linked to two addresses `A` and `B`:
* 1.0 btc to `A`
* 0.5 btc to `A`
* 1.0 btc to `B`
* 0.5 btc to `B`
The node sends 0.2 btc to `C`. Without `-avoidpartialspends`, the following coin selection will occur:
* 0.5 btc to `A` or `B` is picked
* 0.2 btc is output to `C`
* 0.3 - fee is output to (unique change address)
With `-avoidpartialspends`, the following will instead happen:
* Both of (0.5, 1.0) btc to `A` or `B` is picked (one or the other pair)
* 0.2 btc is output to `C`
* 1.3 - fee is output to (unique change address)
As noted, the pro here is that, assuming nobody sends to the address after you spend from it, you will only ever use one address once. The con is that the transaction becomes slightly larger in this case, because it is overpicking outputs to adhere to the no partial spending rule.
This complements #10386, in particular it addresses @luke-jr and @gmaxwell's concerns in https://github.com/bitcoin/bitcoin/pull/10386#issuecomment-300667926 and https://github.com/bitcoin/bitcoin/pull/10386#issuecomment-302361381.
Together with `-avoidreuse`, this fully addresses the concerns in #10065 I believe.
Tree-SHA512: 24687a4490ba59cf4198ed90052944ff4996653a4257833bb52ed24d058b3e924800c9b3790aeb6be6385b653b49e304453e5d7ff960e64c682fc23bfc447621
https://github.com/bitcoin/bitcoin/pull/13715 introduced a new check
for _transport.is_closing() in mininode's P2PConnection's. This function
is only available from Python 3.4.4, though, while Bitcoin is supposed
to support all Python 3.4 versions.
In this change, we make the check conditional on is_closing() being
available. If it is not, then we revert to the behaviour before the
check was introduced; this means that
https://github.com/bitcoin/bitcoin/issues/13579 is not fixed for old
systems, but at least the tests work as they used to do before.
This includes a small refactoring from a one-line lambda to an
inline function, because this makes the code easier to read with more
and more conditions being added.
Fixes https://github.com/bitcoin/bitcoin/issues/13745.
This is a squashed commit that squashes the following commits:
This commit removes the `boost/algorithm/string/predicate.hpp` dependenc
from the project by replacing the function calls to `boost::algorithm::starts_with`
`boost::algorithm::ends_with` and `all` with respectively C++11'
`std::basic_string::front`, `std::basic_string::back`, `std::all_of` function calls
This commit replaces `boost::algorithm::is_digit` with a locale independent isdigi
function, because the use of the standard library's `isdigit` and `std::isdigit
functions is discoraged in the developer notes
5f019d5354 Removes the boost/algorithm/string/join dependency (251)
Pull request description:
This commit removes the `boost/algorithm/string/join` dependency from the project by replacing `boost::algorithm::join` with the helper function proposed by @MarcoFalke in https://github.com/bitcoin/bitcoin/pull/13726#discussion_r204159967
Tree-SHA512: d4ba3e7621b76bd5210aec9b8d6c320f7ee963d7f902e6d2d3fc0eadbee1cd77799e5c09be9c11452d2825f25740fc436cdec3a6b6c66ced674d771e4ed306ae
f447a0a707 Remove program options from build system (Chun Kuan Lee)
11588c639e Replace boost program_options (Chun Kuan Lee)
Pull request description:
Concept from #12744, but without parsing negated options.
Tree-SHA512: 7f418744bb8934e313d77a5f162633746ef5d043de802b9c9cd9f7c1842e7e566eb5f171cd9e2cc13317281b2449c6fbd553fa4f09b837e6af2f5d2b2aabdca2
5c613aadd6 lint: Add linter for circular dependencies (Ben Woosley)
Pull request description:
Protects against added circular depencies, makes it explicit in the
code when circular dependencies have been removed.
Modeled after EXPECTED_BOOST_INCLUDES in lint-includes.sh
Example output:
```
$ test/lint/lint-circular-dependencies.sh
A new circular dependency in the form of "qt/paymentserver -> qt/walletmodel -> qt/paymentserver" appears to have been introduced.
$ echo $?
1
$ test/lint/lint-circular-dependencies.sh
Good job! The circular dependency "Fake" is no longer present.
Please remove it from EXPECTED_CIRCULAR_DEPENDENCIES in test/lint/lint-circular-dependencies.sh
to make sure this circular dependency is not accidentally reintroduced.
$ echo $?
1
$ test/lint/lint-circular-dependencies.sh
$ echo $?
0
```
Tree-SHA512: 4519434de29f6d50859daed1480e531c01c1cdbc3f0a5f093251daf62ae2b5b9073fb274b86f541a985e06837aa1165b76558c5f35fb51a759d72e83f1b61e44
fa4bf92be9 Remove dead service bits code (MarcoFalke)
Pull request description:
Seems fine to remove for the upcoming 0.17 release
Fixes#10993
Tree-SHA512: 3a4664b787e3da399bcaaba693619bd384826df14f469dbdfbbfffc540d9da3f2b322cda262b43388376785f77907c2540541c239ab0fca82bd7eb69d02b6b7a
a3fa4d6a6a QA: Fix bug in -usecli logic that converts booleans to non-lowercase strings (Jonas Schnelli)
4704e5f074 [QA] add createwallet disableprivatekey test (Jonas Schnelli)
c7b8f343e9 [Qt] Disable creating receive addresses when private keys are disabled (Jonas Schnelli)
2f15c2bc20 Add disable privatekeys option to createwallet (Jonas Schnelli)
cebefba085 Add option to disable private keys during internal wallet creation (Jonas Schnelli)
9995a602a6 Add facility to store wallet flags (64 bits) (Jonas Schnelli)
Pull request description:
This mode ('createwallet {"disableprivatekeys": true}') is intended for a sane pure watch-only mode, ideal for a use-case where one likes to use Bitcoin-Core in conjunction with a hardware-wallet or another solutions for cold-storage.
Since we have support for custom change addresses in `fundrawtransaction`, pure watch-only wallets including coin-selection are possible and do make sense for some use cases.
This new mode disables all forms of private key generation and ensure that no mix between hot and cold keys are possible.
Tree-SHA512: 3ebe7e8d54c4d4e5f790c348d4c292d456f573960a5b04d69ca5ef43a9217c7e7671761c6968cdc56f9a8bc235f3badd358576651af9f10855a0eb731f3fc508
Checks that all of the one byte type keys are actually one byte and
throw an error if they are not.
Add tests for each type to check for this behavior.
020628e3a4 Tests for PSBT (Andrew Chow)
a4b06fb42e Create wallet RPCs for PSBT (Andrew Chow)
c27fe419ef Create utility RPCs for PSBT (Andrew Chow)
8b5ef27937 SignPSBTInput wrapper function (Andrew Chow)
58a8e28918 Refactor transaction creation and transaction funding logic (Andrew Chow)
e9d86a43ad Methods for interacting with PSBT structs (Andrew Chow)
12bcc64f27 Add pubkeys and whether input was witness to SignatureData (Andrew Chow)
41c607f09b Implement PSBT Structures and un/serialization methods per BIP 174 (Andrew Chow)
Pull request description:
This Pull Request fully implements the [updated](https://github.com/bitcoin/bips/pull/694) BIP 174 specification. It is based upon #13425 which implements the majority of the signing logic.
BIP 174 specifies a binary transaction format which contains the information necessary for a signer to produce signatures for the transaction and holds the signatures for an input while the input does not have a complete set of signatures.
This PR contains structs for PSBT, serialization, and deserialzation code. Some changes to `SignatureData` have been made to support detection of UTXO type and storing public keys.
***
Many RPCs have been added to handle PSBTs.
`walletprocesspsbt` takes a PSBT format transaction, updates the PSBT with any inputs related to this wallet, signs, and finalizes the transaction. There is also an option to not sign and just update.
`walletcreatefundedpsbt` creates a PSBT from user provided data in the same form as createrawtransaction. It also funds the transaction and takes an options argument in the same form as `fundrawtransaction`. The resulting PSBT is blank with no input or output data filled in. It is analogous to a combination of `createrawtransaction` and `fundrawtransaction`
`decodepsbt` takes a PSBT and decodes it to JSON. It is analogous to `decoderawtransaction`
`combinepsbt` takes multiple PSBTs for the same tx and combines them. It is analogous to `combinerawtransaction`
`finalizepsbt` takes a PSBT and finalizes the inputs. If all inputs are final, it extracts the network serialized transaction and returns that instead of a PSBT unless instructed otherwise.
`createpsbt` is like `createrawtransaction` but for PSBTs instead of raw transactions.
`convertpsbt` takes a network serialized transaction and converts it into a psbt. The resulting psbt will lose all signature data and an explicit flag must be set to allow transactions with signature data to be converted.
***
This supersedes #12136
Tree-SHA512: 1ac7a79e5bc669933f0a6fcc93ded55263fdde9e8c144a30266b13ef9f62aacf43edd4cbca1ffbe003090b067e9643c9298c79be69d7c1b10231b32acafb6338
fabe28a0cd qa: Temporarily disable test that reads the default datadir location (MarcoFalke)
41a8c8dfaf travis: Check that ~/.bitcoin is never created (MarcoFalke)
Pull request description:
Tree-SHA512: d114db29a18f684d207caa0d7c947b13c945e2dd8b6d7fdeacdf7aa194f8123579d1139331b9d308df69a1132796e805a9ab63580aebde9b719860c0ff4b5652
Protects against added circular depencies, makes it explicit in the
code when circular dependencies have been removed.
Modeled after EXPECTED_BOOST_INCLUDES in lint-includes.sh
b9f4b211df tests: Use MAX_SCRIPT_ELEMENT_SIZE from script.py (Daniel Kraft)
Pull request description:
`p2p_segwit.py` and `test_framework/script.py` both define a constant for `MAX_SCRIPT_ELEMENT_SIZE` (=520 bytes), which is redundant. This change uses the constant defined in the `script.py` module for `p2p_segwit.py`.
Tree-SHA512: 2bc295ff26d9b052d4e05b85c27e748175884d6689a92c19337fc4db8bf439e3abe3edc91af1aaf46d8dc42ed96a85ad17110546d2274a0d9cda3abd6b878a31
be98b2d9a8 [QA] Add scantxoutset test (Jonas Schnelli)
eec7cf7b33 scantxoutset: mention that scanning by address will miss P2PK txouts (Jonas Schnelli)
94d73d32ab scantxoutset: support legacy P2PK script type (Jonas Schnelli)
892de1dfea scantxoutset: add support for scripts (Jonas Schnelli)
78304941f7 Blockchain/RPC: Add scantxoutset method to scan UTXO set (Jonas Schnelli)
9048575511 Add FindScriptPubKey() to search the UTXO set (Jonas Schnelli)
Pull request description:
Alternative to #9152.
This takes `<n>` pubkeys and optionally `<n>` xpubs (together with a definable lookup windows where the default is 0-1000) and looks up common scripts in the UTXO set of all given or derived keys.
The output will be an array similar to `listunspent`. That array is compatible with `createrawtransaction` as well as with `signrawtransaction`.
This makes it possible to prepare sweeps and have them signed in a secure (cold) space.
Tree-SHA512: a2b22a117cf6e27febeb97e5d6fe30184926d50c0c7cbc77bb4121f490fed65560c52f8eac67a9720d7bf8f420efa42459768685c7e7cc03722859f51a5e1e3b
Added functional tests for PSBT that test the RPCs. Also added all
of the BIP 174 test vectors (except for the updater tests) in the
functional tests.
Added a Unit test for the BIP 174 updater test vector.
89e70f9d7f Fix that CWallet::AbandonTransaction would only traverse one level (Ben Woosley)
Pull request description:
Prior to this change, it would mark only the first layer of
child transactions abandoned, due to always following the input `hashTx`
rather than the current `now` tx.
Tree-SHA512: df068b49637d299ad73237c7244005fe5aa966d6beae57aff12e6948f173d9381e1b5d08533f7e3a1416991ed57f9f1f7b834057141d85c07dc60bb1f0872cea
Commit 3fdb29778a renamed share/rpcuser to share/rpcauth but left references to the old path in code; this commit fixes the old references.
Performed update using https://github.com/facebook/codemod with command: `codemod --extensions cpp,py,md 'share/rpcuser' 'share/rpcauth'`
-BEGIN VERIFY SCRIPT-
git grep --files-with-matches 'share/rpcuser' src/*.cpp | xargs sed -i -E 's:share/rpcuser:share/rpcauth:g'
git grep --files-with-matches 'share/rpcuser' test/functional/*.py | xargs sed -i -E 's:share/rpcuser:share/rpcauth:g'
-END VERIFY SCRIPT-
backupwallet was broken for multiwallets in their own directories
(i.e. something like DATADIR/wallets/mywallet/wallet.dat). In this
case, the backup would use DATADIR/wallets/wallet.dat as source file
and not take the specific wallet's directory into account.
This led to either an error during the backup (if the wrong source
file was not present) or would silently back up the wrong wallet;
especially the latter behaviour can be quite bad for users.
38040c34e1 [tests] Remove accounts from wallet_importprunedfunds.py (John Newbery)
Pull request description:
This was split from #13075 to not block review/merge of that PR.
Tree-SHA512: 631d7139ed2bda5222ec395cc75720261e2e1f741dba04723d09fe04ef6cf92222a3679d886026ec33e2db2d1e2fa1a0f36c2451581d0f733a9939a98c7118ab
f40b3b82df [tests] functional test for createmultisig RPC (Anthony Towns)
b9024fdda3 segwit support for createmultisig RPC (Anthony Towns)
d58055d25f Move AddAndGetDestinationForScript from wallet to outputype module (Anthony Towns)
9a44db2e46 Add outputtype module (Anthony Towns)
Pull request description:
Adds an "address_type" parameter that accepts "legacy", "p2sh-segwit", and "bech32" to choose the type of address created. Defaults to "legacy" rather than the value of the `-address-type` option for backwards compatibility.
As part of implementing this, OutputType is moved from wallet into its own module, and `AddAndGetDestinationForScript` is changed to apply to a `CKeyStore` rather than a wallet, and to invoke `keystore.AddCScript(script)` itself rather than expecting the caller to have done that.
Fixes#12502
Tree-SHA512: a08c1cfa89976e4fd7d29caa90919ebd34a446354d17abb862e99f2ee60ed9bc19d8a21a18547c51dc3812cb9fbed86af0bef2f1e971f62bf95cade4a7d86237
702ae1e21a [RPC] [wallet] allow getbalance to use min_conf and watch_only without accounts. (John Newbery)
cf15761f6d [wallet] GetBalance can take a min_depth argument. (John Newbery)
0f3d6e9ab7 [wallet] factor out GetAvailableWatchOnlyBalance() (John Newbery)
7110c830f8 [wallet] deduplicate GetAvailableCredit logic (John Newbery)
ef7bc8893c [wallet] Factor out GetWatchOnlyBalance() (John Newbery)
4279da4785 [wallet] GetBalance can take an isminefilter filter. (John Newbery)
Pull request description:
#12953 inadvertently removed the functionality to call `getbalance "*" <int> <bool>` to get the wallet's balance with either minconfs or include_watchonly.
This restores that functionality (when `-deprecatedrpc=accounts`), and also makes it possible to call ``getbalance minconf=<int> include_watchonly=<bool>` when accounts are not being used.
Tree-SHA512: 67e84de9291ed6d34b23c626f4dc5988ba0ae6c99708d02b87dd3aaad3f4b6baa6202a66cc2dadd30dd993a39de8036ee920fcaa8cbb1c5dfe606e6fac183344
eeeef80fb6 qa: Fix some TODOs in p2p_segwit (MarcoFalke)
Pull request description:
* I believe we don't need to redundantly test versionbits logic in every functional tests that tests a softfork deployment that is being done with versionbits. Thus, remove two `TODO`s that ask for that.
* Replace another `TODO` with `wait_until`.
* Some style fixups after #13467
Tree-SHA512: c7120404d50579d6f3b9092f1e259959190eeafe520231e3479c8c256a50bf7260ccc93f8301ac0e100c54037053f6849433ebb1c55607e01d94b9812e525083
Prior to this change, it would mark only the first layer of
child transactions abandoned, due to always following the input hashTx
rather than the current now tx.
p2p_segwit.py and test_framework/script.py both define a constant for
MAX_SCRIPT_ELEMENT_SIZE (=520 bytes), which is redundant. This change
uses the constant defined in the script.py module for p2p_segwit.py.
685d1d8115 [tests] Check signrawtransaction* errors on missing prevtx info (Anthony Towns)
a3b065b51f Error on missing amount in signrawtransaction* (Anthony Towns)
Pull request description:
Signatures using segregated witness commit to the amount being spent, so that value must be passed into signrawtransactionwithkey and signrawtransactionwithwallet. This ensures an error is issued if that doesn't happen, rather than just assuming the value is 0 and producing a signature that is almost certainly invalid.
Based on Ben Woosley's #12458, Fixes: #12429.
Tree-SHA512: 8e2ff89d5bcf79548e569210af0d850028bc98d86c149b92207c9300ab1d63664a7e2b222c1be403a15941aa5cf36ccc3c0d570ee1c1466f3496b4fe06c17e11
d280617bf5 [qa] Add a test for merkle proof malleation (Suhas Daftuar)
ed82f17000 have verifytxoutproof check the number of txns in proof structure (Gregory Sanders)
Pull request description:
Recent publication of a weakness in Bitcoin's merkle tree construction demonstrates many SPV applications vulnerable to an expensive to pull off yet still plausible attack: https://bitslog.wordpress.com/2018/06/09/leaf-node-weakness-in-bitcoin-merkle-tree-design/
This change would at least allow `verifytxoutproof` to properly validate that the proof matches a known block, with known number of transactions any time after the full block is processed. This should neuter the attack entirely.
The negative is that a header-only processed block/future syncing mode would cause this to fail until the node has imported the data required.
related: #13451
`importprunedfunds` needs this check as well. Can expand it to cover this if people like the idea.
Tree-SHA512: 0682ec2b622a38b29f3f635323e0a8b6fc071e8a6fd134c954579926ee7b516e642966bafa667016744ce49c16e19b24dbc8801f982a36ad0a6a4aff6d93f82b
57889e688d bitcoin-tx: Stricter check for valid integers (Daniel Kraft)
Pull request description:
Just calling `atoi` to convert strings to integers does not check for valid integers very thoroughly; in particular, it just ignores everything starting from the first non-numeral character. Even a string like "foo" is fine and silently returns 0.
This meant that `bitcoin-tx` would not fail if such a string was passed in various places where an integer is expected (like the `locktime` or an input/output index); this means that it would, for instance, silently accept a typo and interpret it in an unexpected way.
In this change, we use `ParseInt64` for parsing strings to integers, which actually verifies that the full string is valid as number. New tests in the `bitcoin-util-test` cover the new error paths.
This fixes#13599.
Tree-SHA512: 146a0af275e9f57784e5d0582d3defbac35551b54b6b7232f8a0b20db04aa611125e52aa4512ef2f8ed2cafc2a12fe586f9d10ed66d641cff090288f279b1988
161e8d40a4 RPC: Add new getzmqnotifications method. (Daniel Kraft)
caac39b0ac Make ZMQ notification interface instance global. (Daniel Kraft)
Pull request description:
This adds a new RPC method `getzmqnotifications`, which returns information about all active ZMQ notification endpoints. This is useful for software that layers on top of bitcoind, so it can verify that ZeroMQ is enabled and also figure out where it should listen.
See #13526.
Tree-SHA512: edce722925741c84ddbf7b3a879fc9db1907e5269d0d97138fe724035d93ee541c2118c24fa92f4197403f380d0e25c2fda5ca6c62d526792ea749cf527a99a0
ea65182f03 [wallet] loadwallet shouldn't create new wallets. (John Newbery)
Pull request description:
A bug in the initial implementation of loadwallet meant that if the
arguement was a directory that didn't contain a wallet.dat file, a new
wallet would be created in that directory. Fix that so that if a
directory is passed in, it must contain a wallet.dat file.
Bug reported by promag (João Barbosa).
Tree-SHA512: 0a59fa8a33fde51a88544ad288b00e4995284fe16424f643076aaba42b8244fff362145217650ee53d518dfab7efbed4237632c34cdd3dcbbecaa9ecaab5fd7b
Just calling atoi to convert strings to integers does not check for
valid integers very thoroughly; in particular, it just ignores
everything starting from the first non-numeral character. Even a string
like "foo" is fine and silently returns 0.
This meant that bitcoin-tx would not fail if such a string was passed in
various places where an integer is expected (like the locktime or an
input/output index); this means that it would, for instance, silently
accept a typo and interpret it in an unexpected way.
In this change, we use ParseInt64 for parsing strings to integers,
which actually verifies that the full string is valid as number.
New tests in the bitcoin-util-test cover the new error paths.
e3aab295e [tests] p2p_segwit: sync_blocks in subtest wrapper. (John Newbery)
55e805085 [tests] p2p_segwit: remove unnecessary arguments from subtests. (John Newbery)
25711c269 [tests] p2p_segwit: log and assert segwit status in subtest wrapper. (John Newbery)
6839863d5 [tests] p2p_segwit: Make sure each subtest leaves utxos for the next. (John Newbery)
bfe32734d [tests] p2p_segwit: wrap subtests with subtest wrapper. (John Newbery)
2af4e398d [tests] p2p_segwit: re-order function definitions. (John Newbery)
94a0134a4 [tests] p2p_segwit: standardise comments/docstrings. (John Newbery)
f7c7f8ecf [tests] p2p_segwit: Fix flake8 warnings. (John Newbery)
Pull request description:
`p2p_segwit.py` is a very long test, composed of multiple subtests. When it fails it's difficult to debug for a couple of reasons:
- Control flow jumps between different methods in the test class, so it's a little difficult to follow the code.
- state may be carried forward unintentionally from one subtest to the next.
Improve that by wrapping the subtests with a `@subtest` decorator which:
- logs progress
- asserts state after each subtest
As usual, I've also included a few commits which generally tidy up the test and improve style.
Tree-SHA512: 3650602b3ce9823dc968cc5f2e716757feadc3dbedb3605eb79bb3df91a6db8ae53431f253b440da690e3a8e9d76de84fad4368a2663aeb40e6b9427cf948870
This adds a new RPC method "getzmqnotifications", which returns
information about all active ZMQ notification endpoints. This is useful
for software that layers on top of bitcoind, so it can verify that
ZeroMQ is enabled and also figure out where it should listen.
See https://github.com/bitcoin/bitcoin/issues/13526.
75848bcf40 [tests] Fix p2p_sendheaders race (John Newbery)
Pull request description:
p2p_sendheaders has a race in part 1.3.
part 1.2 sends a block to the node over the 'test_node' connection, but
doesn't wait for an inv to be received on the 'inv_node' connection. If
we get to part 1.3 before that inv has been received, then the
subsequent call to check_last_inv_announcement could fail.
Tree-SHA512: ba9baffb3a9c0d379259190c737a7a4ad2e1133005a5b026af4f6b67a2978e24db39289551ad29134151879593ef5472be7e569a3557c0740fb51f5c56263d9a
fa87da2f17 qa: Avoid start/stop of the network thread mid-test (MarcoFalke)
Pull request description:
This simplifies test writing by removing the need to handle the network thread in tests. E.g. start thread, join thread, restart thread mid-test, adding p2p connections at the "right" time, ...
Tree-SHA512: 533642f12fef5496f1933855edcdab1a7ed901d088d34911749cd0f9e044c8a6cb1f89985ac3a7f41a512943663e4e270a61978f6f072143ae050cd102d4eab8
A bug in the initial implementation of loadwallet meant that if the
arguement was a directory that didn't contain a wallet.dat file, a new
wallet would be created in that directory. Fix that so that if a
directory is passed in, it must contain a wallet.dat file.
Bug reported by promag (João Barbosa).
fa103a5d5e [qa] wallet_basic: Specify minimum required amount for listunspent (MarcoFalke)
Pull request description:
A value less than that would fail the tests later on anyway:
```
File "./test/functional/wallet_basic.py", line 250, in run_test
self.nodes[1].sendrawtransaction(signed_raw_tx['hex'])
test_framework.authproxy.JSONRPCException: bad-txns-in-belowout, value in (1.00) < value out (49.998) (code 16) (-26)
Tree-SHA512: 7e72ad02b5623bc078610da06c34721836822a920a4e85b12a1e0f339e3205cdc11d39763197770e649fb73376f922ff91a8f244b465195e50a6798658e04f80
df10f07db1 [wallet] Don't use accounts when checking balance in sendmany (John Newbery)
e209184101 [wallet] deprecate sendfrom RPC method. (John Newbery)
Pull request description:
A couple of fixups from the accounts API deprecation PR (#12953):
- properly deprecate `sendfrom`
- don't use accounts when calculating balance in `sendmany` (unless the `-deprecatedrpc=accounts` flag is being used)
Tree-SHA512: 1befde055067438c4c3391bbff1aaed0e6249efd708c567db3f1faad40a0f28e64f95e5bad0679ae826d24a0239e4bc8a1c392dc93e2e7502343a7f6b1d1845c
927e1150bc Test: Harden lint-filenames.sh (wodry)
Pull request description:
- This fixes that only files with lower case file name suffix where found before, which contradicted the Regex to find uppercase characters in file names (including suffixes I guess).
- `--full-name` switch was added to git ls-files, to define that the found file always includes it's full path in the git project.
- since we know now that the file name includes the full path, we can harden the Regex to exclude the secp256k1 and univalue sub folders.
- use backslash line break to make code easier to read and avoid too long line.
Tree-SHA512: 9b55fe4965ae2084112b9f8a81bf9c657756c2cb5004986e7b6102a76adaf62c7d7a53257d9f13c5d8a1c75870b52c744d13830e3edd454a099e810357c914e5
000000035b Obsolete #!/bin/bash shebang (DesWurstes)
Pull request description:
> `#!/bin/bash` assumes it is always installed to `/bin/` which can cause issues
> `#!/usr/bin/env bash` searches the user's `PATH` to find the `bash` binary
Details: https://github.com/dylanaraps/pure-bash-bible#obsolete-syntax
I'm open to comments: Should I also fix `#!/bin/sh`?
Tree-SHA512: b47bb4828116aa119f1899c68fee081270d51a898535490b9c616bf0f3660ad953f29c361eafc759bc64cdd54ee6eeecb2d79e9fdb5291a996a515c719805476
fa1eac9cdb [qa] mininode: Expose connection state through is_connected (MarcoFalke)
Pull request description:
This gets rid of some non-type safe string comparisons and access to members that are implementation details of `class P2PConnection(asyncore.dispatcher)`. Such refactoring is required to replace the deprecated asyncore with something more sane.
Changes:
* Get rid of non-enum member `state` and replace is with bool `connected`
* Get rid of confusing argument `pushbuf` and literally just push to the buffer at the call site
Tree-SHA512: 09074c7e5ed251a2e0509ef205ab82f89887c1e1fa1cc6efc1db60d196eb2403788a4987df8809fd06d80ef652e614c5d3c3fdef70096fc5815102243388288d
p2p_sendheaders has a race in part 1.3.
part 1.2 sends a block to the node over the 'test_node' connection, but
doesn't wait for an inv to be received on the 'inv_node' connection. If
we get to part 1.3 before that inv has been received, then the
subsequent call to check_last_inv_announcement could fail.
47776a958b Add linter: Make sure all shell scripts opt out of locale dependence using "export LC_ALL=C" (practicalswift)
3352da8da1 Add "export LC_ALL=C" to all shell scripts (practicalswift)
Pull request description:
~~Make sure `LC_ALL=C` is set when using `grep` range expressions.~~
Make sure `LC_ALL=C` is set in all shell scripts.
From the `grep(1)` documentation:
> Within a bracket expression, a range expression consists of two characters separated by a hyphen. It matches any single character that sorts between the two characters, inclusive, using the locale's collating sequence and character set. For example, in the default C locale, `[a-d]` is equivalent to `[abcd]`. Many locales sort characters in dictionary order, and in these locales `[a-d]` is typically not equivalent to `[abcd]`; it might be equivalent to `[aBbCcDd]`, for example. To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the `LC_ALL` environment variable to the value C.
Context: [Locale issue found when reviewing #13450](https://github.com/bitcoin/bitcoin/pull/13450/files#r194877736)
Tree-SHA512: fd74d2612998f9b49ef9be24410e505d8c842716f84d085157fc7f9799d40e8a7b4969de783afcf99b7fae4f91bbb4559651f7dd6578a6a081a50bdea29f0909
c8176b3cc7 Add linter: Make sure we explicitly open all text files using UTF-8 or ASCII encoding in Python (practicalswift)
634bd97001 Explicitly specify encoding when opening text files in Python code (practicalswift)
Pull request description:
Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python.
As requested by @laanwj in #13440.
Tree-SHA512: 1651c00fe220ceb273324abd6703aee504029b96c7ef0e3029145901762c733c9b9d24927da281394fd4681a5bff774336c04eed01fafea997bb32192c334c06
86edf4a2a5 expose CBlockIndex::nTx in getblock(header) (Gregory Sanders)
Pull request description:
Recent publication of a weakness in Bitcoin's merkle tree construction demonstrates many SPV applications vulnerable to an expensive to pull off yet still plausible attack: https://bitslog.wordpress.com/2018/06/09/leaf-node-weakness-in-bitcoin-merkle-tree-design/
Including the coinbase in the txoutproof seems the most effective fix, however results in a significant efficiency downgrade. Transactors will not even know a priori what the size of their proof will be within a couple orders of magnitude, unless they use the mid-state of SHA2 as detailed in the blog post.
Some applications, like Elements blockchain platform that take SPV-style proofs have optional access to a bitcoind to verify these proofs of inclusion and check depth in the chain. Returning `CBlockIndex::nTx` would allow an extremely easy and compact way of checking the depth of the tree, with no additional overhead to the codebase, and works with pruned nodes.
`getblockheader` is arguably not the place for it, but as mentioned before, is a natural workflow for us checking depth of a block in a possibly pruned node.
We should also ensure that `verifytxoutproof` ends up validating this depth fact as well, but left this for another PR.
Tree-SHA512: af4cf48e704c6088f8da06a477fda1aaa6f8770cee9b876c4465d1075966d6a95831a88817673fe5a0d6bbcdc1ffcbc1892e2be0d838c60fc6958d33eacdcc14
fa7a6cf1b3 policy: Treat segwit as always active (MarcoFalke)
Pull request description:
Now that segwit is active for a long time, there is no need to reject transactions with the reason that segwit hasn't activated.
Strictly speaking, this is a bug fix, because with the release of 0.16, we create segwit transactions in our wallet by default without checking if they are allowed by local policy.
More broadly, this simplifies the code as if "premature witness" was always set to true with the corresponding command line args.
Tree-SHA512: 484c26aa3a66faba6b41e8554a91a29bfc15fbf6caae3d5363a3966283143189c4bd5333a610b0669c1238f75620691264e73f6b9f1161cdacf7574d946436da
fa8071a098 qa: Log as utf-8 (MarcoFalke)
Pull request description:
Explicitly read and write the log files with utf-8 as encoding
Tree-SHA512: ca28f37f34a09845c736ff6c4c21733c3c39584f52c81e48ff25e5e35979c317d0989862b2b93acc7e359fbcc20b99533365455830b2ddb41eb4d8c17314534e
16e3cd380a Clarify include recommendation (practicalswift)
6d10f43738 Enforce the use of bracket syntax includes ("#include <foo.h>") (practicalswift)
906bee8e5f Use bracket syntax includes ("#include <foo.h>") (practicalswift)
Pull request description:
When analysing includes in the project it is often assumed that the preferred bracket include syntax (`#include <foo.h>`) mentioned in `developer-docs.md` is used consistently. @sipa:s excellent circular dependencies script [`circular-dependencies.py`](50c69b7801/contrib/devtools/circular-dependencies.py) (#13228) is an example of a script making this reasonable assumption.
This PR enables automatic Travis checking of the include syntax making sure that the bracket syntax includes (`#include <foo.h>`) is used consistently.
Tree-SHA512: a414921aabe8e487ebed42f3f1cbd02fecd1add385065c1f2244cd602c31889e61fea5a801507ec501ef9bd309b05d3c999f915cec1c2b44f085bb0d2835c182
67e0e04140 [wallet] [docs] Update release notes for removing `getlabeladdress` (John Newbery)
81608178cf [wallet] [rpc] Remove getlabeladdress RPC (John Newbery)
Pull request description:
labels are associated with addresses (rather than addresses being
associated with labels, as was the case with accounts). The
getlabeladdress does not make sense in this model, so remove it.
getaccountaddress is still supported for one release as the accounts
API is deprecated.
Tree-SHA512: 7f45d0456248ebcc4e54dd34e2578a09a8ea8e4fceda75238ccea9d731dc99a3f3c0519b18a9739de17d2e6e59c9c2259ba67c9ae2e3cb2a40ddb14b9193fe29
698cfd0811 docs: Mention lint-locale-dependence.sh in developer-notes.md (practicalswift)
0a4ea2f458 build: Add linter for checking accidental locale dependence (practicalswift)
Pull request description:
This linter will check for code accidentally introducing locale dependencies.
Unnecessary locale dependence can cause bugs that are very tricky to isolate and fix. We should avoid using locale dependent functions if possible.
Context: https://github.com/bitcoin/bitcoin/pull/12881#issuecomment-378564722
Example output:
```
$ contrib/devtools/lint-locale-dependence.sh
The locale dependent function tolower(...) appears to be used:
src/init.cpp: if (s[0] == '0' && std::tolower(s[1]) == 'x') {
Unnecessary locale dependence can cause bugs that are very
tricky to isolate and fix. Please avoid using locale dependent
functions if possible.
Advice not applicable in this specific case? Add an exception
by updating the ignore list in contrib/devtools/lint-locale-dependence.sh
```
**Note to reviewers:** What is the most appropriate `LOCALE_DEPENDENT_FUNCTIONS` function list? What should be added or removed?
Tree-SHA512: 14e448828804bb02bf59070647e38b52fce120c700c903a4a8472769a2cee5dd529bd3fc182386993cb8720482cf4250b63a0a477db61b941ae4babe5c65025f
fa4760fbb3 qa: Increase includeconf test coverage (MarcoFalke)
Pull request description:
This adds some missing `return false` for error conditions and adds test coverage [1] for those.
Also, extend recursion warning when the chain was set in one of the includeconfs.
[1] See the red lines in https://marcofalke.github.io/btc_cov/total.coverage/src/util.cpp.gcov.html for missing coverage.
Tree-SHA512: d32563c9bb277879895a173e699034db5ecdb4061a1ec8890c566d61e36a09efa5eda19a029baf952ff6d568f8b9684a13a0bb90827850075470975e2088fee4
faac7a2db4 qa: Avoid checking reject code for now (MarcoFalke)
Pull request description:
The node will often disconnect before sending a reject code. A more
robust solution would be to read from the debug log. See #13006
Tree-SHA512: 1dabf8a43dabbc722f4ffe4fbc1f870090253a66290b2d1a95e7a24e14c6442b493c314480c0314587164eb65e5d468aa9eb5e107ad90bb3ca821a97ea4d373c
fa26cf0156 qa: Fixup setting of PATH env var (MarcoFalke)
Pull request description:
This was an oversight of mine in #13188
Can be trivially tested with `BITCOIND=bitcoin-qt ./test/functional/wallet_disable.py` before and after this fix.
Tree-SHA512: 06c7b2f12158855eb2b6392861943821bd7ad3152cf0dd49ac4abd878e5b937ebee55e256ce5bdc1c2a9c775a452112c34533366c934ff5f0f412b3a7e1c8118
fa3c910bfe test: Move linters to test/lint, add readme (MarcoFalke)
Pull request description:
This moves the checks and linters from `devtools` to a subfolder in `test`. (Motivated by my opinion that the dev tools are mostly for generating code and updating the repo whereas the linters are read-only checks.)
Also, adds a readme to clarify that checks and linters are only meant to prevent bugs and user facing issues, not merely stylistic preference or inconsistencies. (This is motivated by the diversity in developers and work flows as well as existing code styles. It would be too disruptive to change all existing code to a single style or too burdensome to force all developers to adhere to a single style. Also note that our style guide is changing, so locking in at the wrong style "too early" would only waste resources.)
Tree-SHA512: 9b10e89f2aeaf0c8a9ae248aa891d74e0abf0569f8e5dfd266446efa8bfaf19f0ea0980abf0b0b22f0d8416ee90d7435d21a9f9285b66df43f370b7979173406
87fe292d89 doc: Mention disabling BIP61 in bips.md (Wladimir J. van der Laan)
fe16dd8226 net: Add option `-enablebip61` to configure sending of BIP61 notifications (Wladimir J. van der Laan)
Pull request description:
This commit adds a boolean option `-peersendreject`, defaulting to `1`, that can be used to disable the sending of [BIP61](https://github.com/bitcoin/bips/blob/master/bip-0061.mediawiki) `reject` messages. This functionality has been requested for various reasons:
- security (DoS): reject messages can reveal internal state that can be used to target certain resources such as the mempool more easily.
- bandwidth: a typical node sends lots of reject messages; this counts against upstream bandwidth. Also the reject messages tend to be larger than the message that was rejected.
On the other hand, reject messages can be useful while developing client software (I found them indispensable while creating bitcoin-submittx), as well as for our own test cases, so whatever the default becomes on the long run, IMO the functionality should be retained as option. But that's a discussion for later, for now it's simply a node operator decision.
Also adds a RPC test that checks the functionality.
Tree-SHA512: 9488cc53e13cd8e5c6f8eb472a44309572673405c1d1438c3488f627fae622c95e2198bde5ed7d29e56b948e2918bf1920239e9f865889f4c37c097c37a4d7a9
fa865efa4a qa: Fix wallet_listreceivedby race (MarcoFalke)
Pull request description:
Generating a block on node 0 will only get node 0 out of IBD and not node 1. So the inv for the `txid` is dropped by node 1 and the call to `sync_all` fails.
Solve it by a call to `sync_blocks` after `generate`.
Tree-SHA512: e21b01a9e8c90bd6a3aad290c97cc4866ab384e22797b318eed55ae2767512203597d3a184b23ad5a3fe76bdbb8a3d5c51e097d56b160232851164434059ff23
41d0476f62 Tests: Add data file (Anthony Towns)
4cbfb6aad9 Tests: Test new getblockstats RPC (Jorge Timón)
35e77a0288 RPC: Introduce getblockstats (Jorge Timón)
cda8e36f01 Refactor: RPC: Separate GetBlockChecked() from getblock() (Jorge Timón)
Pull request description:
It returns per block statistics about several things. It should be easy to add more if people think of other things to add or remove some if I went too far (but once written, why not keep it? EDIT: answer: not to test or maintain them).
The currently available options are: minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs (EDIT: see updated list in the rpc call documentation)
For the x axis, one can use height or block.nTime (I guess I could add mediantime if there's interest [EDIT: nobody showed interest but I implemented mediantime nonetheless, in fact there's no distinction between x or y axis anymore, that's for the caller to judge]).
To calculate fees, -txindex is required.
Tree-SHA512: 2b2787a3c7dc4a11df1fce62c8a4c748f5347d7f7104205d5f0962ffec1e0370c825b49fd4d58ce8ce86bf39d8453f698bcd46206eea505f077541ca7d59b18c
labels are associated with addresses (rather than addresses being
associated with labels, as was the case with accounts). The
getlabeladdress does not make sense in this model, so remove it.
getaccountaddress is still supported for one release as the accounts
API is deprecated.
2352aa9 test: Ensure that recursive -includeconf produces appropriate warnings (Karl-Johan Alm)
c5bcc7d util: warn about recursive -includeconf arguments in configuration files (Karl-Johan Alm)
Pull request description:
This is a follow-up PR to #10267, and addresses https://github.com/bitcoin/bitcoin/pull/10267#issuecomment-387546144.
~~I am adding extra work for @jnewbery in #12755 here -- maybe I should just rebase on top of that, but not sure what the appropriate approach is here.~~
Tree-SHA512: 87f0c32436b70424e33616ffb88d7cb699f90d6a583a10237e224b28fc936d6a9df95536c8c52ee8546b3942da92b2a357e61bf87e00d1462bc10d46d3bee352
7384a35 [tests] Remove spurious error log in p2p_segwit.py (John Newbery)
Pull request description:
Since 265d7c44b1, when wait_until() fails,
an error message is logged to the test framework log. This means that if
wait_until() is called inside a try-except with the expectation that it
will fail, a spurious error message is logged.
wait_until() shouldn't be called with the expectation of failure. Fix
that in p2p_segwit.py.
Tree-SHA512: 0a43790b58fee7d2d6bef36e736b0b9ffdde6de5f12d33d15e8e07323597e2be4cd98f17e7fc3a135e06bdafe36613466c0a57e81134e59a251383c62b91918f
a8da482 Bump wallet version for pre split keypool (Andrew Chow)
dfcd9f3 Use a keypool of presplit keys after upgrading to hd chain split (Andrew Chow)
5c50e93 Allow -upgradewallet to upgradewallets to HD (Andrew Chow)
2bcf2b5 Test sethdseed (Andrew Chow)
b5ba01a Add 'sethdseed' RPC to initialize or replace HD seed (Chris Moore)
dd3c07a Separate HaveKey function that checks whether a key is in a keystore (Andrew Chow)
Pull request description:
Revival/rebase of #11085
Adds a new command `sethdseed` which allows you to either set or generate a new HD seed to be used. A new keypool can be generated or the original one kept and new keys added to the keypool will come from the new HD seed.
Wallets that are not HD will be upgraded to be version FEATURE_HD_SPLIT when the `sethdseed` RPC command is used.
I have also add some tests for this.
Additionally `-upgradewallet` can now be used to upgrade a wallet from non-HD to HD. When it is used for such an upgrade, the keypool will be regenerated.
Tree-SHA512: e56c792e150590429ac4a1061e8d6f7b20cca06366e184eb9bbade4cd6ae82699a28fe84f87031eadba97ad2c1606517a105f00fb7b45779c979243020071adb
This commit adds a boolean option `-enablebip61`, defaulting to `1`, that
can be used to disable the sending of BIP61 `reject` messages. This
functionality has been requested for various reasons:
- security (DoS): reject messages can reveal internal state that can be
used to target certain resources such as the mempool more easily.
- bandwidth: a typical node sends lots of reject messages; this counts
against upstream bandwidth. Also the reject messages tend to be larger
than the message that was rejected.
On the other hand, reject messages can be useful while developing client
software (I found them indispensable while creating bitcoin-submittx),
as well as for our own test cases, so whatever the default becomes on the
long run, IMO the functionality should be retained as option. But that's
a discussion for later.
364bae5 qa: Pad scriptPubKeys to get minimum sized txs (MarcoFalke)
7485488 Policy to reject extremely small transactions (Johnson Lau)
0f8719b Add transaction tests for constant scriptCode (Johnson Lau)
9dabfe4 Add constant scriptCode policy in non-segwit scripts (Johnson Lau)
Pull request description:
This disables `OP_CODESEPARATOR` in non-segwit scripts (even in an unexecuted branch), and makes a positive `FindAndDelete` result invalid. This ensures that the `scriptCode` serialized in `SignatureHash` is always the same as the script passing to the `EvalScript`.
Tree-SHA512: a0552cb920294d130251c48053fa2ff1fbdd26332e62b52147d918837852750f0ce35ce2cd1cbdb86588943312f8154ccb4925e850dbb7c2254bc353070cd5f8
5d536619ab [tests] Remove 'account' API from wallet functional tests (John Newbery)
Pull request description:
Next step in #12952. Removes all usage of the 'account' API from the wallet functional tests, except:
- rpc_deprecated.py (which specifically tests the `-deprecatedrpc=accounts` command line argument is working properly).
- `wallet_labels.py` (which tests that both the 'label' and 'account' APIs work in V0.17).
'account' API usage for both of those tests can be removed once V0.17 has been branched.
Also excluded is:
- `wallet_importprunedfunds.py` (which fails due to a bitcoind OOM error)
Tree-SHA512: 6701b32f83d2d47597ba093ded665d7aa630f7a9c759ff15e3e33a3e3bc7600e8d29cf4e72aed5f8f9f6769cc9b614c681951720eab1ed2473f5f8dec57e7a6f
12d1b77f7e [tests] Fixed intermittent failure in p2p_sendheaders.py. (lmanners)
Pull request description:
Added handling for the case where headers are announced over more than one message.
refs #12453
Tree-SHA512: 2c5b48ff019089b86e358181ba170d3aac09d4ae41ec79c2718e0ee83705860501bbcb8fd94d0f5c4f86c0d54a96781a967716621bb8c5ecc991b39af3cec506
09c6699900 [qa] Handle disconnect_node race (Suhas Daftuar)
Pull request description:
Several tests call disconnect_nodes() on each node-pair in rapid
succession, resulting in a race condition if a node disconnects a peer
in-between the calculation of the nodeid's to disconnect and the
invocation of the disconnectnode rpc call. Handle this.
Tree-SHA512: 3078cea0006fcb507c812004a777c505eb1e9dda7c6df12dbbe72395a73ff6f6760f597b6492054f5487b34534417ddef5fbad30553c135c288c4b7cfce79223
Since 265d7c44b1, when wait_until() fails,
an error message is logged to the test framework log. This means that if
wait_until() is called inside a try-except with the expectation that it
will fail, a spurious error message is logged.
wait_until() shouldn't be called with the expectation of failure. Fix
that in p2p_segwit.py.
Several tests call disconnect_nodes() on each node-pair in rapid
succession, resulting in a race condition if a node disconnects a peer
in-between the calculation of the nodeid's to disconnect and the
invocation of the disconnectnode rpc call. Handle this.
By default, libc will print fatal errors to /dev/tty instead of stderr.
Adding the LIBC_FATAL_STDERR_ to the environment variables allows
us to catch libc errors in stderr and test for them.
fac1e1f qa: Remove unused option --srcdir (MarcoFalke)
Pull request description:
The `srcdir` option was both unused and misleading; It should have been called `builddir`. So remove it.
Tree-SHA512: 2c24dcf2aa82219158b8cbbf03dd3f0f51f805f1f5f670faa1fd59e5a8d60fda120ffddadeccb058d8d3f20583b4952be7afd2df6bbefb9367d35c0f0a9fda3c
891beb0 [test] fundrawtransaction: lock watch-only shared address (Karl-Johan Alm)
Pull request description:
`self.nodes[0]` creates an address which is watch-only-shared with `self.nodes[3]`. If `nodes[0]` spends the associated UTXO during any of its sends later, the watchonly test will fail, as `nodes[3]` now has insufficient funds.
I ran into this in #12257 and this commit is in that PR as well, but I figured I'd split it out (and remove from there once/if merged).
Tree-SHA512: d04a04b1ecebe82127cccd47c1b3de311bf07f4b51dff80db20ea2f142e1d5c4a85ed6180c5c0b081d550e238c742e119b953f60f487deac5a3f3536e1a8d9fe
25b7ab9 doc: Add release notes for -includeconf (Karl-Johan Alm)
0f0badd test: Test includeconf parameter. (Karl-Johan Alm)
629ff8c -includeconf=<path> support in config handler, for including external configuration files (Karl-Johan Alm)
Pull request description:
Fixes: #10071.
Done:
- adds `-includeconf=<path>`, where `<path>` is relative to `datadir` or to the path of the file being read, if in a file
- protects against circular includes
- updates help docs
~~~Thoughts:~~~
- ~~~I am not sure how to test this in a neat manner. Feedback on this would be nice. Will dig/think though.~~~
Tree-SHA512: cb31f1b2f69fbc0890d264948eb2e501ac05cf12f5e06a5942f9c1539eb15ea8dc3cae817f4073aecb2fcc21d0386747f14f89d990772003a76e2a6d25642553
1e46d8a Get rid of ambiguous OutputType::NONE value (Russell Yanofsky)
Pull request description:
Based on suggestion by @sipa https://github.com/bitcoin/bitcoin/pull/12119#issuecomment-357982763
After #12119, the NONE output type was overloaded to refer to either an output type that couldn't be parsed, or to an automatic change output mode. This change drops the NONE enum and uses a simple bool to indicate parse failure, and a new CHANGE_AUTO enum to refer the change output type.
This change is almost a pure refactoring except it makes RPCs reject empty string ("") address types instead of treating them like they were unset. This simplifies the parsing code a little bit and could prevent RPC usage mistakes. It's noted in the release notes.
Follows up #12408 by @MarcoFalke
Followups for future PRs:
- [ ] Add explicit support for specifying "auto" in `ParseOutputType` as suggested by promag and sipa: https://github.com/bitcoin/bitcoin/pull/12729#issuecomment-374799567 and https://github.com/bitcoin/bitcoin/pull/12729#discussion_r175969481
- [ ] Add wallet `AddressChangeType` method to complement `TransactionChangeType`: https://github.com/bitcoin/bitcoin/pull/12729#discussion_r175969618.
Tree-SHA512: 8b08b272bcb177a0a9e556dcd965840a7fe601ef83ca97938b879c9b1a33b5b3f96939e1bceef11ba7c644ac21bfd6c1dbc6ca715cd1da4ace50475240e4ee48
d8e9a2a Remove "rpc" category from GetWarnings (Wladimir J. van der Laan)
7da3b0a rpc: Move RPC_FORBIDDEN_BY_SAFE_MODE code to reserved section (Wladimir J. van der Laan)
2ae705d Remove Safe mode (Andrew Chow)
Pull request description:
Rebase of #10563. Safe mode was [disabled by default and deprecated in 0.16](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.16.0.md#safe-mode-disabled-by-default), so probably should be removed for 0.17.
> Rationale:
>
> Safe mode is useless. It only disables some RPC commands when large work forks are detected. Nothing else is affected by safe mode. It seems that very few people would be affected by safe mode. The people who use Core as a wallet are primarily using it through the GUI, which safe mode does not effect. In the GUI, transactions will still be made as normal; only a warning is displayed.
>
> I also don't think that we should be disabling RPC commands or any functionality in general. If we do, it should be done consistently, which safe mode is not. If we want to keep the idea of a safe mode around, I think that the current system needs to go first before a new system can be implemented.
Tree-SHA512: 067938f47ca6e879fb6c3c4e21f9946fd7c5da3cde67ef436f1666798c78d049225b9111dc97064f42b3bc549d3915229fa19ad5a634588f381e34fc65d64044
7d0f80b Use anonymous namespace instead of static functions (Pieter Wuille)
b61fb71 Mention removal of bare multisig IsMine in release notes (Pieter Wuille)
9c2a8b8 Do not treat bare multisig as IsMine (Pieter Wuille)
08f3228 Optimization: only test for witness scripts at top level (Pieter Wuille)
3619735 Track difference between scriptPubKey and P2SH execution in IsMine (Pieter Wuille)
ac6ec62 Switch to a private version of SigVersion inside IsMine (Pieter Wuille)
19fc973 Do not expose SigVersion argument to IsMine (Pieter Wuille)
fb1dfbb Remove unused IsMine overload (Pieter Wuille)
952d821 Make CScript -> CScriptID conversion explicit (Pieter Wuille)
Pull request description:
Currently our wallet code will treat bare multisig outputs (meaning scriptPubKeys with multiple public keys + `OP_CHECKMULTISIG` operator in it) as ours without the user asking for it, as long as all private keys in it are in our wallet.
This is a pointless feature. As it only works when all private keys are in one place, it's useless compared to single key outputs (P2PK, P2PKH, P2WPKH, P2SH-P2WPKH), and worse in terms of space, cost, UTXO size, and ability to test (due to lack of address format for them).
Furthermore, they are problematic in that producing a list of all `scriptPubKeys` we accept is not tractable (it involves all combinations of all public keys that are ours). In further wallet changes I'd like to move to a model where all scriptPubKeys that are treated as ours are explicit, rather than defined by whatever keys we have. The current behavior of the wallet is very hard to model in such a design, so I'd like to get rid of it.
I think there are two options:
* Remove it entirely (do not ever accept bare multisig outputs as ours, unless watched)
* Only accept bare multisig outputs in situations where the P2SH version of that output would also be acceptable
This PR implements the first option. The second option was explored in #12874.
Tree-SHA512: 917ed45b3cac864cee53e27f9a3e900390c576277fbd6751b1250becea04d692b3b426fa09065a3399931013bd579c4f3dbeeb29d51d19ed0c64da75d430ad9a
41ff967 list the types of scripts we should consider for a witness program (fivepiece)
4f933b3 p2wpkh, p2wsh and p2sh-nested scripts in decodescript (fivepiece)
Pull request description:
Attempts to address #12244 . `p2wsh` addresses are returned only for scripts that are neither `p2sh` nor any witness program.
Tree-SHA512: eb47f094c1a4c2ad2bcf27a8032307e43cf787d50bf739281aeb4101d97316a2f307b05118bf138298c937fa34e15f91436443a9b313f809fad2c43e94cd1831
7de1de7 Add new fee structure with all sub-fields denominated in BTC (mryandao)
Pull request description:
the denomination for `fee` is current in btc while the other such as `decendentFee` and `ancestorFee` are in satoshis.
Tree-SHA512: e428f6dca1d339f89ab73e38ce5903f5465c46b159069d9bcc3f8b1140fe6657fa49a11abe0088e9f7ba9999f64af72a349a4735bf5eaa61b8e4a185b23543f3
fa811b0 qa: Normalize executable location (MarcoFalke)
Pull request description:
This removes the need to override the executable locations by just reading them from the config file. Beside making the code easier to read, running individual test on Windows is now possible by default (without providing further command line arguments).
Note: Of course, it is still possible to manually specify the location through the `BITCOIND` environment variable, e.g. `bitcoin-qt`.
Tree-SHA512: bee6d22246796242d747120ca18aaab089f73067de213c9111182561985c5912228a0b0f7f9eec025ecfdb44db031f15652f30d67c489d481c995bb3232a7ac7
fac0db0 wallet: Make fee settings non-static members (MarcoFalke)
Pull request description:
The wallet header defined some globals (they were called "settings"), that should be class members instead.
This commit is hopefully only refactoring, apart from a multiwallet bugfix: Calling the rpc `settxfee` for one wallet, would set (and change) the fee rate for all loaded wallets. (See added test case)
Tree-SHA512: 4ab6ec2f5c714742396ded5e451ec3b1ceb771e3696492de29889d866de4365b3fbe4a2784d085c8b8bd11b1ebb8a1fec99ab2c62eee716791cfc67c0cf29e1b
cead28b [docs] Add release notes for deprecated 'account' API (John Newbery)
72c9575 [wallet] [tests] Add tests for accounts/labels APIs (John Newbery)
109e05d [wallet] [rpc] Deprecate wallet 'account' API (John Newbery)
3576ab1 [wallet] [rpc] Deprecate account RPC methods (John Newbery)
3db1ba0 [tests] Set -deprecatedrpc=accounts in tests (John Newbery)
4e671f0 [tests] Rename rpc_listtransactions.py to wallet_listtransactions.py (John Newbery)
a28b907 [wallet] [rpc] Remove duplicate entries in rpcwallet.cpp's CRPCCommand table (John Newbery)
Pull request description:
Deprecate all accounts functionality and make it only accessible by using `-deprecatedrpc=accounts`.
Accounts specific RPCs, account arguments, and account related results all require the `-deprecatedrpc=accunts` startup option now in order to see account things.
Several wallet functional tests use the accounts system. Those tests are unchanged, except to start the nodes with `-deprecatedrpc=accounts`. We can slowly migrate those tests to use the 'label' API instead of the 'account' API before accounts are fully removed.
Tree-SHA512: 89f4ae2fe6de4a1422f1817b0997ae22d63ab5a1a558362ce923a3871f3e42963405d6573c69c27f1764679cdee5b51bf52202cc407f1361bfd8066d652f3f37
8b8032e test: Add rpcauth pair that generated by rpcauth (Chun Kuan Lee)
Pull request description:
This PR adds a rpcauth pair that is randomly generated. Also checks that rpcauth.py works fine.
Resolve#12995
Tree-SHA512: d9661f40e306bcf528dc25919c874ebcdbdd21101319985dc12ce133c80fd0021cfee5e4bfe8ee7970eccc2e24c97e596263b270fe0b79f3613ae573a825ed63
09b30db Asserts that the tx version number is a signed 32-bit integer. (251)
Pull request description:
This PR attempts to resolve#11561 by addressing the feedback from @MarcoFalke; and @gmaxwell in #12430.
Commit 30e9d24 adds a functional test to `rpc_rawtransaction.py` to assert that the transaction version number in the RPC output is a signed 32-bit integer.
The functional test uses the raw transaction data from Mainnet transaction `c659729a7fea5071361c2c1a68551ca2bf77679b27086cc415adeeb03852e369`.
Tree-SHA512: d78f3120b9aa04537561ab5584769a838b25e162c5caa6e1543256fb27538aa4c708c939fb5ba93ccb3fa676c2d92ce8eb9cc78869f80ac96be64a7bec7bebd0
1accfbc Output values for "min relay fee not met" error (Kristaps Kaupe)
Pull request description:
It is already done this way for "mempool min fee not met" error.
Tree-SHA512: 829db78ecc066cf93b8e93ff1aeb4e7b98883cf45f341d5be6e6b4dff4135f3f54fa49b3a6f12eb43f676a9ba54f981143c9887f786881e584370434a9566cfd
80a5e59 [qa] Attach node index to test_node AssertionError and print messages (James O'Beirne)
Pull request description:
In the midst of fighting with https://github.com/bitcoin/bitcoin/pull/12873 it became apparent that there're a number of assertions and print statements which are emitted by test nodes but don't identify the node in question. This change makes debugging a bit easier by adding identifying information to non-logger test_node-related error messages.
Tree-SHA512: 7cc86f2c81f4b3fdba15ec9a2d21a84c4b083629e845e82288087c3affbbdc5c68e74067621856cc97fe84fbc8cb4f5ca4977a51ef381e5d74515df8eb001239
1f83839 [wallet] [tests] Test disallowed multiwallet params (John Newbery)
3476e3c [wallet] Fix zapwallettxes/multiwallet interaction. (John Newbery)
Pull request description:
`-zapwallettxes` should be disallowed when starting bitcoin in multiwallet mode.
There's code in `WalletInit::ParameterInteraction()` to disallow `-zapwallettxes` when running in multiwallet mode. This code functioned as expected when passing the parameter `-zapwallettxes=1`, but not when passing the parameter `-zapwallettxes` (ie without the value specified). Fix that and add a test.
The new test in the
_[wallet] [tests] Test disallowed multiwallet params_ commit reproduces the bug and should fail against master.
Fixes#12505
Tree-SHA512: 6cd921717e9c7d2773ca84c946c310c2adec8430e37cbecdb33a620f510db3058a72bd328411812ba415111bc52a3367b332c9d15a37a92ccfd7ae785d2f32ab
e87fefc test: Add rpc_bind test to default-run tests (Wladimir J. van der Laan)
Pull request description:
Skip the parts that cannot be run on the host due to lack of IPv6 support or a second interface to bind on, and warn appropriately.
Without no strong requirements (besides being Linux only, which will skip the test) left, add this test to the default in test_runner.
~~(the non-IPv6 parts of the two dual-IPv4/6 tests could also be enabled, but first going to look what Travis does here to see if there wasn't another reason it was disabled)~~ done, it only makes sense for the first
Tree-SHA512: 724259b14f59dccc7e61ef071359336adb0f76a63db392b6ce6940e21c8ee0470c35374e82970681261685ef299cd70b0c1372598cea85d341f64c2c40ea28ee
c1d742025c [tests] Fix feature_block flakiness (John Newbery)
Pull request description:
feature_block.py occasionally fails on Travis. I believe this is due to
a a race condition when reconnecting to bitcoind after a subtest that
expects disconnection. If the test runs ahead and sends the INV for the
subsequent test before we've received the initial sync getheaders, then
we may end up sending two headers messages - one as a response to the
initial sync getheaders and one in response to the INV getheaders. If
both of those headers fail validation with a DoS score of 50 or higher,
then we'll unexpectedly be disconnected.
There is only one validation failure that has a DoS score bewteen 50 and
100, which is high-hash. That's why the test is failing immediately
after the "Reject a block with invalid work" subtest.
Fix is to wait for the initial getheaders from the peer before we
start populating our blockstore. That way we won't have any invalid
headers to respond to it with.
Tree-SHA512: dc17d795fcfaf0f8c0bf1e9732b5e11fbc8febbfafba4c231b7c13a5404a2c297dcd703a7a75bc7f353c893e12efc87f424f2201abd47ba5268af32d4d2e841f
feature_block.py occasionally fails on Travis. I believe this is due to
a a race condition when reconnecting to bitcoind after a subtest that
expects disconnection. If the test runs ahead and sends the INV for the
subsequent test before we've received the initial sync getheaders, then
we may end up sending two headers messages - one as a response to the
initial sync getheaders and one in response to the INV getheaders. If
both of those headers fail validation with a DoS score of 50 or higher,
then we'll unexpectedly be disconnected.
There is only one validation failure that has a DoS score bewteen 50 and
100, which is high-hash. That's why the test is failing immediately
after the "Reject a block with invalid work" subtest.
Fix is to wait for the initial getheaders from the peer before we
start populating our blockstore. That way we won't have any invalid
headers to respond to it with.
Such outputs can still be watched, and signed for, but they aren't treated as valid payments.
That means they won't cause transactions to appear in listtransactions, their outputs to be
shown under listunspent, or affect balances.
8b56fc0b91 [qa] Test that v0 segwit outputs can't be spent pre-activation (Suhas Daftuar)
ccb8ca42a4 Always enforce SCRIPT_VERIFY_WITNESS with P2SH (Suhas Daftuar)
5c31b20a35 [qa] Remove some pre-activation segwit tests (Suhas Daftuar)
95749a5836 Separate NULLDUMMY enforcement from SEGWIT enforcement (Suhas Daftuar)
ce650182f4 Use P2SH consensus rules for all blocks (Suhas Daftuar)
Pull request description:
As discussed at the IRC meeting back in October (https://botbot.me/freenode/bitcoin-core-dev/2017-10-12/?msg=92231929&page=2), I had looked into the feasibility of enforcing P2SH and SCRIPT_VERIFY_WITNESS back to the genesis block.
The P2SH change is pretty straightforward -- there was only one historical block on mainnet that violated the rule, so I carved out an exception to it, similar to the way we have exceptions for the BIP30 violators.
The segwit change is not entirely as clear. The code changes themselves are relatively straightforward: we can just always turn on SCRIPT_VERIFY_WITNESS whenever P2SH is active. However conceptually, this amounts to splitting up BIP141 into two parts, the part that implements new script rules, and the part that handles witness commitments in blocks.
Arguably though the script rules are really defined in BIP 143 anyway, and so this really amounts to backdating BIP 143 -- script rules for v0 segwit outputs -- back to genesis. So maybe conceptually this isn't so bad...
I don't feel strongly about this change in either direction; I started working on it because I was searching for a way to simplify the way we understand and implement the consensus rules around segwit, but I'm not yet sure whether I think this achieves anything toward that goal.
ping @TheBlueMatt
Tree-SHA512: 73551d4a983eb9792c7ac67f56005822528ac4d1fd52c27cee6d305ebee953f69687ef4ddee8bdc0fec77f77e6b5a9d669750793efee54c076533a095e233042
9e50c337c Note new weight field in release-notes. (Matt Corallo)
d0d9112b7 Test new weight field in p2p_segwit (Matt Corallo)
2874709a9 Expose a transaction's weight via RPC (Matt Corallo)
Pull request description:
This seems like an obvious oversight.
Tree-SHA512: defd047de34fb06a31f589e1a4eef68fcae85095cc67b7c8fb434237bb40300d7f3f97e852d3e7226330e26b96943846b7baf6da0cfc79db8d56e9c1f7848ad9
All account RPC methods are now deprecated and can only be called if
bitcoind has been started with the -deprecatedrpc=accounts switch.
Affected RPC methods are:
- getaccount
- getaccountaddress
- getaddressesbyaccount
- getreceivedbyaccount
- listaccouts
- listreceivedbyaccount
- move
- setaccount
c25321f Add config changes to release notes (Anthony Towns)
5e3cbe0 [tests] Unit tests for -testnet/-regtest in [test]/[regtest] sections (Anthony Towns)
005ad26 ArgsManager: special handling for -regtest and -testnet (Anthony Towns)
608415d [tests] Unit tests for network-specific config entries (Anthony Towns)
68797e2 ArgsManager: Warn when ignoring network-specific config setting (Anthony Towns)
d1fc4d9 ArgsManager: limit some options to only apply on mainnet when in default section (Anthony Towns)
8a9817d [tests] Use regtest section in functional tests configs (Anthony Towns)
30f9407 [tests] Unit tests for config file sections (Anthony Towns)
95eb66d ArgsManager: support config file sections (Anthony Towns)
4d34fcc ArgsManager: drop m_negated_args (Anthony Towns)
3673ca3 ArgsManager: keep command line and config file arguments separate (Anthony Towns)
Pull request description:
The weekly meeting on [2017-12-07](http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-12-07-19.00.log.html) discussed allowing options to bitcoin to have some sensitivity to what network is in use. @theuni suggested having sections in the config file:
<cfields> an alternative to that would be sections in a config file. and on the
cmdline they'd look like namespaces. so, [testnet] port=5. or -testnet::port=5.
This approach is (more or less) supported by `boost::program_options::detail::config_file_iterator` -- when it sees a `[testnet]` section with `port=5`, it will treat that the same as "testnet.port=5". So `[testnet] port=5` (or `testnet.port=5` without the section header) in bitcoin.conf and `-testnet.port=5` on the command line.
The other aspect to this question is possibly limiting some options so that there is no possibility of accidental cross-contamination across networks. For example, if you're using a particular wallet.dat on mainnet, you may not want to accidentally use the same wallet on testnet and risk reusing keys.
I've set this up so that the `-addnode` and `-wallet` options are `NETWORK_ONLY`, so that if you have a bitcoin.conf:
wallet=/secret/wallet.dat
upnp=1
and you run `bitcoind -testnet` or `bitcoind -regtest`, then the `wallet=` setting will be ignored, and should behave as if your bitcoin.conf had specified:
upnp=1
[main]
wallet=/secret/wallet.dat
For any `NETWORK_ONLY` options, if you're using `-testnet` or `-regtest`, you'll have to add the prefix to any command line options. This was necessary for `multiwallet.py` for instance.
I've left the "default" options as taking precedence over network specific ones, which might be backwards. So if you have:
maxmempool=200
[regtest]
maxmempool=100
your maxmempool will still be 200 on regtest. The advantage of doing it this way is that if you have `[regtest] maxmempool=100` in bitcoin.conf, and then say `bitcoind -regtest -maxmempool=200`, the same result is probably in line with what you expect...
The other thing to note is that I'm using the chain names from `chainparamsbase.cpp` / `ChainNameFromCommandLine`, so the sections are `[main]`, `[test]` and `[regtest]`; not `[mainnet]` or `[testnet]` as might be expected.
Thoughts? Ping @MeshCollider @laanwj @jonasschnelli @morcos
Tree-SHA512: f00b5eb75f006189987e5c15e154a42b66ee251777768c1e185d764279070fcb7c41947d8794092b912a03d985843c82e5189871416995436a6260520fb7a4db
Future commits will deprecate the accounts RPC methods, arguments and
return objects. Set the -deprecatedrpc=accounts switch now so tests
don't break in intermediate commits.
9db48c5634 tests: Remove redundant bytes² (practicalswift)
Pull request description:
This is a follow-up to #12993. As @jnewbery noted `bytes()` is idempotent.
Tree-SHA512: 0eb25e0c2c46f9abaac30f964c5eb422bece1414c840a717d86794424294cb19d995a6db7c8df2a2f4ec84776b05274a637f2c111738f397051f510e57184752
Skip the parts that cannot be run on the host due to lack
of IPv6 support or a second interface to bind on, and warn
appropriately.
Without no strong requirements (besides being Linux only, in which case
the test is skipped) left, just add this test to the default in
test_runner.
Includes suggested changes by John Newbery.
41ba061 [docs] Add release notes for wallet 'label' API. (John Newbery)
189e0ef [wallet] [rpc] introduce 'label' API for wallet (Wladimir J. van der Laan)
Pull request description:
Add label API to wallet RPC.
This is one step towards #3816 ("Remove bolt-on account system") although it doesn't
actually remove anything yet.
These initially mirror the account functions, with the following differences:
- These functions aren't DEPRECATED in the help
- Help mentions 'label' instead of accounts. In the language used, labels are
associated with addresses, instead of addresses associated with labels. (unlike
with accounts.)
- Labels have no balance
- No balances in `listlabels`
- `listlabels` has no minconf or watchonly argument
- Like in the GUI, labels can be set on any address, not just receiving addreses
- Unlike accounts, labels can be deleted.
Being unable to delete them is a common annoyance (see #1231).
Currently only by reassigning all addresses using `setlabel`, but an explicit
call `deletelabel` which assigns all address to the default label may make
sense.
Tree-SHA512: 45cc313c68ad529ce3a15c02181d2ab0083a7e14fe824e2cde34972713fecce512e3d4b9aa46db5355f2baa857c44b234d4fe9709225bc23c7ebbc0e03febbf5
Add label API to wallet RPC.
This is one step towards #3816 ("Remove bolt-on account system") although it doesn't
actually remove anything yet.
These initially mirror the account functions, with the following differences:
- These functions aren't DEPRECATED in the help
- Help mentions 'label' instead of accounts. In the language used, labels are
associated with addresses, instead of addresses associated with labels. (unlike
with accounts.)
- Labels have no balance
- No balances in `listlabels`
- `listlabels` has no minconf or watchonly argument
- Like in the GUI, labels can be set on any address, not just receiving addreses
- Unlike accounts, labels can be deleted.
Being unable to delete them is a common annoyance (see #1231).
Currently only by reassigning all addresses using `setlabel`, but an explicit
call `deletelabel` which assigns all address to the default label may make
sense.
Thanks to Pierre Rochard for test fixes.
f526046 adapt bumpfee change discard test to be more strict and add note on p2sh discrep (Gregory Sanders)
5805d6f feebumper: discard change outputs below discard rate (Gregory Sanders)
Pull request description:
The "discard rate" is the concept we use to ensure the wallet isnt creating not so useful just-above-relay dust.
Outside of bumpfee previous to this PR, and manually creating such an output, the wallet will never make change outputs of that size, preferring to send them to fees instead.
"Worst case" for the user is that users pay a slightly higher feerate than they were expecting, which is already a possibility with relay dust.
Tree-SHA512: dd69351810dc1709437602e7db1be46e4e905ccd8e16d03952de8b4c1fdbf9cb7e6c99968930896baf6b5c7cb005a03ec0506a2669d22e21e32982e60329606b
75d0e4c544 [qa] Delete cookie file before starting node (Suhas Daftuar)
Pull request description:
When a node is restarted during a test after an unclean shutdown (such
as with -dbcrashratio), it's possible an old cookie file was left
behind. This can cause a race condition when restarting the node, where
the test framework might try to connect using credentials from the
old cookie file, just as the node will generate new credentials and
overwrite the old file.
Delete any such cookie file if present prior to startup.
Tree-SHA512: ae1e8bf8fd20e07c32b0715025693bb28b0e3dd34f328cae4346abf579b0c97b5db1c02782e1c46b7a3b6058d268b6d46b668e847658a6eed0be857ffb0d65dc
fab9095d40 qa: Windows fixups for functional tests (MarcoFalke)
Pull request description:
Just two minor fixups to have less errors when the tests run on native windows.
* Strip whitespace from lines when reading from a notification file
* Instead of clumsily creating a file with weird permissions, just create a folder for the same effect in `mempool_persist.py`
Tree-SHA512: 48a8b439f14ab9b44c5cd228cd03105e8613e703e3c2951cdf724931bc95172a9ad9bfe69fc23e73dd91b058c1352263c0ac6e8de2ceb0ebf804c8ff52bba394
If a cookie file exists in a datadir prior to node startup, it must have
been leftover from a prior unclean shutdown. As bitcoind will overwrite
it anyway, delete it before starting up to prevent the test framework
from inadvertently trying to connect using stale credentials.
c9cce0a Tests: Add Metaclass for BitcoinTestFramework (Will Ayd)
Pull request description:
BitcoinTestFramework instructs developers in its docstring to override
`set_test_params` and `run_test` in subclasses while being sure NOT to
override `__init__` and `main` . This change adds a metaclass to ensure
that developers adhere to that protocol, raising a ``TypeError`` in
instances where they have not.
closes#12835
Tree-SHA512: 5a47a7ead1f18361138cad4374747c4a8f29d25506f7b2c2a8c1c966a0b65e5ccf7317f9a078df8680fdab5d3fb71fee46a159c9f381878a3683c1e9f874abbe
2b2b96cd45 Use std::bind instead of boost::bind to re-lock the wallet (Suhas Daftuar)
662d19ff72 [rpcwallet] Clamp walletpassphrase value at 100M seconds (Suhas Daftuar)
Pull request description:
Larger values seem to trigger a bug on macos+libevent (resulting in the rpc server stopping).
Tree-SHA512: 890f3b641f6c586e2f8f629a9d23bca6ceb8b237b285561aad488cb7adf941a21177d3129d0c2b8293c0a673cd8e401957dbe2b6b3b7c8c4e991bb411d260102
e36a0c0 [qa] Ensure bitcoind processes are cleaned up when tests end (Suhas Daftuar)
Pull request description:
When tests fail (such as due to a bug in the test, race condition, etc), it's possible that we could follow code paths that bypass our normal node shutdown that occurs in `TestNode.stop_node`. Add a destructor to `TestNode` that cleans this up.
Tree-SHA512: 72e04bc21462ebd0cb346fd1fe0540da454acfbad41923a0b06ea2317e9045b68e58f9adb02d8200891aca89a9d03a022eb72282aeb31a3b3afe7c6843a4b450
8dd547d82b Adding logging for loop iteration level in p2p_sendheaders.py (ccdle12)
Pull request description:
PR for #12453
New contributor looking at mainly the 'good first issues'.
From my understanding of the issue:
* Track the iteration level of loop when test fails in Travis CI
Further clarification on the issue would be greatly appreciated to narrow down what can be fixed or updated.
Tree-SHA512: 228524aad54c09979d990a0fc6818e13a11e1ab5e78b606b892e897bba7b1e09bf25447feb631049e45ac017eeb61fbf1c1f11e8bd5103648f976a05099ba9f9
55efc1f [tests] simplify binary and hex response parsing in interface_rest.py (Roman Zeyde)
ade5964 [tests] only use 2 nodes in interface_rest.py (John Newbery)
ad00fbe [tests] refactor interface_rest.py to avoid code repetition (John Newbery)
7a3181a [tests] Make json request building more consistent in interface_rest.py (John Newbery)
3fd4490 [tests] improve logging and documentation in interface_rest.py (John Newbery)
abf190e [tests] fix flake8 warnings in interface_rest.py test (John Newbery)
Pull request description:
Following the comment at https://github.com/bitcoin/bitcoin/pull/12717#pullrequestreview-106189117.
Tree-SHA512: b55560f0d8f3069584f5a2398285483a0a23514b2b2bd2c1ced2db1cb30dc24f60f720d0fa4de30259f7918d3178d94680ae9321649544d1d04d687a2e672559
self.nodes[0] creates an address which is watch-only-shared with self.nodes[3]. If nodes[0] spends the associated UTXO during any of its sends later, the watchonly test will fail, as nodes[3] now has insufficient funds.
Note that this also adds a new find_vout_for_address function to the test framework.
Based on suggestion by Pieter Wuille <pieter.wuille@gmail.com> at
https://github.com/bitcoin/bitcoin/pull/12119#issuecomment-357982763
After #12119, the NONE output type was overloaded to refer to either an output
type that couldn't be parsed, or to an automatic change output mode. This
change drops the NONE enum and uses a simple bool indicate parse failure, and a
new CHANGE_AUTO enum to refer the change output type.
This change is almost a pure refactoring except it makes RPCs reject empty
string ("") address types instead of treating them like they were unset. This
simplifies the parsing code a little bit and could prevent RPC usage mistakes.
It's noted in the release notes.
BitcoinTestFramework instructs developers in its docstring to override
`set_test_params` and `run_test` in subclasses while being sure NOT to
override `__init__` and `main` . This change adds a metaclass to ensure
that developers adhere to that protocol, raising a ``TypeError`` in
instances where they have not.
closes#12835
9c92c8c827 [tests] Remove Comparison Test Framework (John Newbery)
e80c640d78 [tests] Remove bip9-softforks.py (John Newbery)
Pull request description:
Builds on #11772, #11773 and #11817. Please review those PRs first.
Final step in #10603.
- First commit removes bip9-softforks.py. bip9-sofforks.py was intended to be a generic test for versionbits deployments. However, it only tests CSV activation and was not updated to test segwit activation. CSV activation is tested by bip68-112-113-p2p.py, so this test is duplicated effort. Rather than try to update it to use the BitcoinTestFramework, just remove it. (see https://github.com/btcdrak/bitcoin/pull/8 for previous discussion around the redundancy of bip9-softforks.py)
- Second commit removes the now unused BitcoinComparisonFramework class and the comptool and blockstore modules.
Tree-SHA512: 4bb7196d521048b3b8ba95c87dde73005a1ac73d29ccbb869f11ce9a71089686e7eacd7335337853041dfbd3a5b110172b105adbada58779814d4db22b1376f5
faace13868 qa: Match full plain text by default (MarcoFalke)
Pull request description:
Instead of escaping all full plain text error strings, just compare their strings by default.
Tree-SHA512: 42e28f55105eb947ac6af6ce4056f0ec0f701d85f1c2a38b35ab777bbdf2296bdb79639c345621b8adc03a98b28c7630ded9a67b8b04a48e2c3a49d598ecdcd7
Changing logs to debug level and format of statment to j == ...
Updating debug statements to include comments when checking for iteration level of i and j
Removing comments above log prints in Part 3
Removing unwanted file
Added log prints in loops in p2p_sendheaders
The new P2PDataStore class was sending full blocks in headers messages,
which meant that calls to send_blocks_and_test() would blow up memory if
called with a large number of blocks. Fix that by only sending headers
in headers messages.
bip9-sofforks.py was intended to be a generic test for versionbits
deployments. However, it only tests CSV activation and was not updated
to test segwit activation. CSV activation is tested by
bip68-112-113-p2p.py, so this test is duplicated effort.
Rather than try to update it to use the BitcoinTestFramework, just
remove it.
b55555d rpc: Add testmempoolaccept (MarcoFalke)
Pull request description:
To check if a single raw transaction makes it into the current transaction pool, one had to call `sendrawtransaction`. However, on success, this adds the transaction to the mempool with no easy way to undo.
The call `testmempoolaccept` is introduced to provide a way to solely check the result without changing the mempool state.
Tree-SHA512: 5afd9311190135cee8fc1f229c7d39bf893f1028f29e28d34f70df820198ff97b4bf86b41cbbd6e6c36a5c30073cefa92d541c74a4939c7a2a6fa283dfd41b63
ea04bf7862 Enable flake8 warning F841 ("local variable 'foo' is assigned to but never used") (practicalswift)
169f3e8637 Remove assigned but never used local variables (practicalswift)
Pull request description:
Remove assigned but never used local variables. Enable Travis checking for unused local variables.
Tree-SHA512: d6052ec9044c5d1f03d874ea3c8addd5a156779213ef9200f89d3ae53230f2fd1691aff405c3dae14178e5ef09912c4432e92f606ef4a5220ed9daa140cdee81
63048ec73d [tests] Test starting bitcoind with -h and -version (John Newbery)
Pull request description:
Test that starting bitcoind/bitcoin-qt with `-h` and `-version` works as expected.
Prompted by https://github.com/bitcoin/bitcoin/pull/10762#commitcomment-28345993, which is a nullpointer dereference triggered by starting bitcoin-qt with `-h`.
On master, this test passes when run over bitcoind, but fails when running over bitcoin-qt. I used xvfb as a virtual frame buffer to test:
```
BITCOIND=/home/ubuntu/bitcoin/src/qt/bitcoin-qt xvfb-run ./feature_help.py --nocleanup
2018-03-30T17:09:37.767000Z TestFramework (INFO): Initializing test directory /tmp/user/1000/testdi4dre13
2018-03-30T17:09:37.767000Z TestFramework (INFO): Start bitcoin with -h for help text
2018-03-30T17:09:37.841000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 126, in main
self.run_test()
File "./feature_help.py", line 25, in run_test
assert_equal(ret_code, 0)
File "/home/ubuntu/bitcoin/test/functional/test_framework/util.py", line 39, in assert_equal
raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
AssertionError: not(-11 == 0)
2018-03-30T17:09:37.842000Z TestFramework (INFO): Stopping nodes
Traceback (most recent call last):
File "./feature_help.py", line 42, in <module>
HelpTest().main()
File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 149, in main
self.stop_nodes()
File "/home/ubuntu/bitcoin/test/functional/test_framework/test_framework.py", line 273, in stop_nodes
node.stop_node()
File "/home/ubuntu/bitcoin/test/functional/test_framework/test_node.py", line 141, in stop_node
self.stop()
File "/home/ubuntu/bitcoin/test/functional/test_framework/test_node.py", line 87, in __getattr__
assert self.rpc_connected and self.rpc is not None, "Error: no RPC connection"
AssertionError: Error: no RPC connection
```
Passes for bitcoind and bitcoin-qt when run on #12836.
Longer term, we should consider running functional tests over bitcoin-qt in one of the Travis jobs.
Tree-SHA512: 0c2f40f3d5f0e78c3a1b07dbee8fd383eebab27ed0bf2a98a5b9cc66613dbd7b70e363c56163a37e02f68ae7ff7b3ae1769705d0e110ca68a00f8693315730a4
12982682a6 [tests] Change feature_csv_activation.py to use BitcoinTestFramework (John Newbery)
db7ffb9d1b [tests] Move utility functions in feature_csv_activation.py out of class. (John Newbery)
0842edf9ee [tests] Remove nested loops from feature_csv_activation.py (John Newbery)
2e511d5424 [tests] improve logging in feature_csv_activation.py (John Newbery)
6f7f5bc002 [tests] fix flake8 nits in feature_csv_activation.py (John Newbery)
Pull request description:
Next step in #10603.
- first four commits tidy up bip68-112-113-p2p.py
- fifth commit removes usage of ComparisonTestFramework
Tree-SHA512: 34466be12280096ad92ac842f9c594ae40c19be9b4edc73c1e37964d03d55f4e75b80cea50c9940404096effc23705671503a883a7b7773b5866a29f653ba710
5a67c0524e [tests] Fix intermittent rpc_net.py failure. (John Newbery)
Pull request description:
rpc_net.py would intermittently fail on Travis, probably
due to assuming that two consecutive RPC calls were atomic.
Fix this by trying the test up to three times and only failing
if all three attempts fail.
fixes#11778
Tree-SHA512: f2f5047e05114ad2110377e6221ce057680c240952971fb33863834587d2250896c6b697ba27eef739cd0ab23faf47dfae8cafadc9833cbfab5a6f7e36dae5e2
8394300859 [Tests] fix a typo in TestNode.assert_start_raises_init_error() (Roman Zeyde)
Pull request description:
`self.wait_util_stopped()` should be `self.wait_until_stopped()`.
Also, use a specific Exception subclass for indicating node failure to start (instead of using `AssetionError` and an `except Exception` clause).
Following https://travis-ci.org/bitcoin/bitcoin/jobs/359066226#L2726 and depending on #12806 (which fixes the root cause of the Travis test failure).
Tree-SHA512: 7bd5a95586a412472ef9dffdb086789d7275ddaf862724e21cebb3418d0c97e6d89b4d1a58375e42114060d028403d6eab89e3a1e9a833ffe8dadf3439ab1fe2
265d7c4 [tests] Improve assert message when wait_until() fails (John Newbery)
ebf053a [tests] Change feature_block.py to use BitcoinTestFramework (John Newbery)
fc02c12 [tests] Add logging to feature_block.py (John Newbery)
3898c4f [tests] Tidy up feature_block.py (John Newbery)
5cd01d2 [tests] Fix flake8 warnings in feature_block.py (John Newbery)
Pull request description:
Next step in #10603.
- first three commits tidy up feature_block.py
- fourth commit removes usage of ComparisonTestFramework
Longer term, it would be better to separate net_processing testing from validation testing, but I think this is still a useful PR, since it moves us away from the comparison test framework.
Tree-SHA512: d0bb3ad22ad0aa1222877f4212bff075f9ce358e99c69c26d9913e4b346d931b8380e744434a9f6f37812c352cdaa75791691565bfeb18afcb619c06c6ca32a3
b466f6b [Tests] Use blockmaxweight where tests previously had blockmaxsize (Conor Scott)
Pull request description:
Fix for #12768: `-blockmaxsize` has been removed, but some tests were using this feature, so update with `-blockmaxweight`
Tree-SHA512: 5f6d643daee43366c6e61f4154a3920efb4cef1455e483575cf19b0f95bc6e56668360c1562fa18f85b336e48f64e482bd29b1ecb227d7c78c4344d7f5d32ed3
rpc_net.py would intermittently fail on Travis, probably
due to assuming that two consecutive RPC calls were atomic.
Fix this by only testing that amounts are bounded above and
below rather than equal.
adc2586 doc: Refer to witness reserved value as spec. in the BIP (MarcoFalke)
Pull request description:
BIP141 refers to the coinbase's input's witness that consists of a single 32-byte array as "witness reserved value".
This updates the code to follow the BIP
Tree-SHA512: 49c9463519bd11b9ff322eeecd638f7627aa8efdfb869f8549f9a160ff34281e1b5a0b9d83545a692de6f5ff795055292c423403b0f3ce7597e3f32273cf1deb
d71bedb qa: Fix function names in feature_blocksdir (MarcoFalke)
Pull request description:
This fixes the test failure on master:
```
AttributeError: 'BlocksdirTest' object has no attribute 'assert_start_raises_init_error'
```
Tree-SHA512: d96a9b707a9b4fb8752b15f28dae02c60c25cbec21dca5f3ee62e2717c6a49951533c24b52ed0d6e99c5a964ef2c3e90fdc58a9104122714ae9874e121955df6
a192636 -blocksdir: keep blockindex leveldb database in datadir (Jonas Schnelli)
f38e4fd QA: Add -blocksdir test (Jonas Schnelli)
386a6b6 Allow to optional specify the directory for the blocks storage (Jonas Schnelli)
Pull request description:
Since the actual block files taking up more and more space, it may be desirable to have them stored in a different location then the data directory (use case: SSD for chainstate, etc., HD for blocks).
This PR adds a `-blocksdir` option that allows one to keep the blockfiles and the blockindex external from the data directory (instead of creating symlinks).
I fist had an option to keep the blockindex within the datadir, but seems to make no sense since accessing the index will (always) lead to access (r/w) the block files.
Tree-SHA512: f8b9e1a681679eac25076dc30e45e6e12d4b2d9ac4be907cbea928a75af081dbcb0f1dd3e97169ab975f73d0bd15824c00c2a34638f3b284b39017171fce2409
4757c04 [config] Remove blockmaxsize option (John Newbery)
Pull request description:
The blockmaxsize option was marked as deprecated in V0.15.1, and code
was added to convert provided blockmaxsize into blockmaxweight. However,
this code was incorrectly implemented, and blockmaxsize was silently
ignored.
No users have complained about blockmaxsize being ignored, so just
remove it in V0.17.
Fixes#12640
cc @ajtowns
Tree-SHA512: 968d71d37bf175c5a02539ddec289a12586f886e1dfe64c1d9aa5e39db48d06d21665153824fac3b11503a55f0812d2f1115a2d726aafd37b76ed629ec0aa671
d2527bd Rename wallet_accounts.py test (Russell Yanofsky)
045eeb8 Rename account to label where appropriate (Russell Yanofsky)
Pull request description:
Rename account to label where appropriate
This change only updates strings and adds RPC aliases, but should simplify the implementation of address labels in https://github.com/bitcoin/bitcoin/pull/7729, by getting renaming out of the way and letting that change focus on semantics.
The difference between accounts and labels is that labels apply only to addresses, while accounts apply to both addresses and transactions (transactions have "from" and "to" accounts). The code associating accounts with transactions is clumsy and unreliable so we would like get rid of it.
---
There is a rebased version of #7729 atop this PR at https://github.com/ryanofsky/bitcoin/commits/pr/label, see https://github.com/bitcoin/bitcoin/pull/7729#issuecomment-338417139.
Tree-SHA512: b3f934e612922d6290f50137f8ba71ddfaea4485713c7d97e89400a8b73b09b254f9186dffa462c77f5847721f5af9852b5572ade5443d8ee95dd150b3edb7ff
The blockmaxsize option was marked as deprecated in V0.15.1, and code
was added to convert provided blockmaxsize into blockmaxweight. However,
this code was incorrectly implemented, and blockmaxsize was silently
ignored.
No users have complained about blockmaxsize being ignored, so just
remove it in V0.17.
