2c3eade704 Make fs::path::string() always return utf-8 string (Chun Kuan Lee)
Pull request description:
Imbue `fs::path` with `std::codecvt_utf8_utf16` at `SetupEnvironment()`, so that default string encoding will be utf-8 inside `fs::path`.
Tree-SHA512: 0cb59464d777278decbf24771fc5ff0cb2caa7bc2fe8ee5cd36c97a2324873a3caad131f08f050393b488316ee7f4ab0b28b7fa4699e41839f8e51b9867d5118
This change:
* adds a length check to ParseHashStr, appropriate given its use to populate
a 256-bit number from a hex str.
* allows the caller to handle the failure, which allows for the more
appropriate JSONRPCError on failure in prioritisetransaction rpc
661ac15a4a appveyor: Run functional tests on appveyor (Chun Kuan Lee)
2148c36b6e tests: Make it possible to run functional tests on Windows (Chun Kuan Lee)
Pull request description:
This PR do the following things:
- Make functional tests compatible with Windows
- Print color output in functional tests for Windows 10
- Run util and functional tests on appveyor
- Do not run symlink tests on Windows
Note:
- The wallet_multiwallet.py fail is unrelated to the test framework, it's a bug related to c++ code or maybe dependencies. `bitcoind` would exit with 0xC0000005(Access violation) during shutdown occasionally. Disable this for now.
- Not using `--failfast` because this is still in experimental. We should track if there is any other error.
- Disable ZMQ tests because the python zmq library could cause access violation sometimes.
- Disable `feature_notifications` because Bitcoin Core handles the command in different thread, whicha can cause a race condition.
Tree-SHA512: b76db137d264e62a5c130e1cbca7a2ca002a7a0f4153fa0b92c1ea6c9c09ef0533e11c49bdbd566c472d8ff59f245758feb5e5a6ec6cb6bb66a1c67bab5fa48a
5eb20f81d9 Consistently use ParseHashV to validate hash inputs in rpc (Ben Woosley)
Pull request description:
ParseHashV validates the length and encoding of the string and throws
an informative RPC error on failure, which is as good or better than
these alternative calls.
Note I switched ParseHashV to check string length first, because
IsHex tests that the length is even, and an error like:
"must be of length 64 (not 63, for X)" is much more informative than
"must be hexadecimal string (not X)" in that case.
Split from #13420
Tree-SHA512: f0786b41c0d7793ff76e4b2bb35547873070bbf7561d510029e8edb93f59176277efcd4d183b3185532ea69fc0bbbf3dbe9e19362e8017007ae9d51266cd78ae
addwitnessaddress is deprecated. Remove the call to that RPC from
wallet_dump.py and improve testing of all types of address (legacy,
p2sh-segwit and bech32)
a2eb6f5405 [rpc] Add getnodeaddresses RPC command (chris-belcher)
Pull request description:
Implements issue https://github.com/bitcoin/bitcoin/issues/9463
New getnodeaddresses call gives access via RPC to the peers known by the node. It may be useful for bitcoin wallets to broadcast their transactions over tor for improved privacy without using the centralized DNS seeds. getnodeaddresses is very similar to the getaddr p2p method.
Please advise me on the best approach for writing an automated test. By my reading the getaddr p2p method also isn't really tested.
Tree-SHA512: ad03abf518847476495b76a2f5394b8030aa86654429167fa618e21460abb505c10ef9817ec1b80472320d41d0aff5dc94a8efce023aaaaf5e81386aa92b852b
New getnodeaddresses call gives access via RPC to the peers known by
the node. It may be useful for bitcoin wallets to broadcast their
transactions over tor for improved privacy without using the
centralized DNS seeds. getnodeaddresses is very similar to the getaddr
p2p method.
Tests the new rpc call by feeding IP address to a test node via the p2p
protocol, then obtaining someone of those addresses with
getnodeaddresses and checking that they are a subset.
c1dde3a949 No longer shutdown after encrypting the wallet (Andrew Chow)
d7637c5a3f After encrypting the wallet, reload the database environment (Andrew Chow)
5d296ac810 Add function to close all Db's and reload the databae environment (Andrew Chow)
a769461d5e Move BerkeleyEnvironment deletion from internal method to callsite (Andrew Chow)
Pull request description:
This is the replacement for #11678 which implements @ryanofsky's [suggestion](https://github.com/bitcoin/bitcoin/pull/11678#pullrequestreview-76464511).
Shutting down the software was to prevent the BDB environment from writing unencrypted private keys to disk in the database log files, as was noted [here](https://bitcointalk.org/index.php?topic=51474.msg616068#msg616068). This PR replaces the shutdown behavior with a CDBEnv flush, close, and reopen which achieves the same effect: everything is cleanly flushed and closed, the log files are removed, and then the environment reopened to continue normal operation.
To ensure that no unencrypted private keys are in the log files after encrypting the wallet, I wrote [this script](https://gist.github.com/achow101/7f7143e6c3d3fdc034d3470e72823e9d) to pull private keys from the original wallet file and searches for these keys in the log files (note that you will have to change your file paths to make it work on your own machine).
As for concerns about private keys being written to slack space or being kept in memory, these behaviors no longer exist after the original wallet encryption PR and the shutting down solution from 2011.
cc @ryanofsky
Tree-SHA512: 34b894283b0677a873d06dee46dff8424dec85a2973009ac9b84bcf3d22d05f227c494168c395219d9aee3178e420cf70d4b3eeacc9785aa86b6015d25758e75
9dcb6763fb [qa] Use correct python index slices in example test (Suhas Daftuar)
Pull request description:
There's an off-by-one in the list indices used in example_test.py.
Tree-SHA512: d75b77c1e0b3931d02dfa043da4cb6fe8e62864a73717ce5c184d9dbeb25579342c6365cc7bbcc7c4382d76a320a528bf3c69107854dfc6fa704133d0ba11012
fa8433e379 qa: Remove unneded import_deterministic_coinbase_privkeys overwrite, add comments (MarcoFalke)
e413c2ddd1 qa: Fix codespell error and have lint-spelling error instead of warn (MarcoFalke)
Pull request description:
Currently the functional tests require the wallet module to be compiled into the Bitcoin Core executable. For example the premine (or datadir cache) to speed up tests when run in parallel would mine a bunch of blocks and store the private keys to sign the coinbase tx outputs in a wallet. There is no need to have the overhead of the whole wallet module by using keys that are deterministic for all runs.
Note that this change most likely requires the `./test/cache/` to be cleared.
Tree-SHA512: 9ce26036b0e10f0f888f66a1e50be6a357343f9ffb302ae24a7bb3df2f083a31702ef308b738a03b08a1b623aeddac5d6563dc1b15078c0357b7dafad7808ec3
fa6ab8ada1 rpc: Return more specific reject reason for submitblock (MarcoFalke)
Pull request description:
The second commit in #13439 made the `TODO` in the first commit impossible to solve.
The meaning of `fNewBlock` changed from "This is the first time we process this block" to "We are about to write the new *valid* block".
So whenever `fNewBlock` is true, the block was valid. And whenever the `fNewBlock` is false, the block is either valid or invalid. If it was valid and not new, we know it is a `"duplicate"`. In all other cases, the `BIP22ValidationResult()` will return the reason why it is invalid.
Tree-SHA512: 4b6edf7a912339c3acb0fccfabbdd6d812a0321fb1639c244c2714e58dc119aa2b8c6bf8f7d61ea609a1b861bbc23f920370fcf989c48452721e259a8ce93d24
b6a253337f Remove redundant BIP174 test from rpc_psbt.json (araspitzu)
Pull request description:
There was a duplicate test for SIGNER role inside 'test/functional/data/rpc_psbt.json', namely test number 2 was equal to test number 3 in the array of data for 'signer'. This pull request removes the 3rd (redundant) test.
Tree-SHA512: e2128c93183f2e0acf5247274397c77a962accf95dee3bb6f785494cf3080a3f28ea47d8209e36b3064490c821690d1742c22e0d76370cb1688dcb2ab91d8f57
fac9539836 qa: Run all tests even if wallet is not compiled (MarcoFalke)
faa669cbcd qa: Premine to deterministic address with -disablewallet (MarcoFalke)
Pull request description:
Currently the test_runner would exit if the wallet was not compiled into the Bitcoin Core executable. However, a lot of the tests run without the wallet just fine and there is no need to globally require the wallet to run the tests.
Tree-SHA512: 63177260aa29126fd20f0be217a82b10b62288ab846f96f1cbcc3bd2c52702437703475d91eae3f8d821a3149fc62b725a4c5b2a7b3657b67ffcbc81532a03bb
bb6ca65f98 gui: get special folder in unicode (Chun Kuan Lee)
1c5d225853 Drop boost::scoped_array (Chun Kuan Lee)
Pull request description:
Drop boost::scoped_array and simplify the code.
`TCHAR` should be defined as `wchar_t` if `UNICODE` is defined. So we can use `.toStdWString().c_str()` to get wchar_t C-style string.
Fix#13819
Tree-SHA512: 3fd4aa784129c9d1576b01e6ee27faa42d793e152d132f2dde504d917dad3a8e95e065fcbc54a3895d74fb6b2a9ed4f5ec67d893395552f585e225486a84a454
98ea64cf23 Let wallet importmulti RPC accept labels for standard scriptPubKeys (Russell Yanofsky)
Pull request description:
Allow importmulti RPC to apply address labels when importing standard scriptPubKeys. This makes the importmulti RPC less finnicky about import formats and also simpler internally.
Tree-SHA512: 102426b21239f1fa5f38162dc3f4145572caef76e63906afd786b7aff1670d6cd93456f8d85f737588eedc49c11bef2e1e8019b8b2cbf6097c77b3501b0cab1f
fac3e22b18 qa: Read reject reasons from debug log, not p2p messages (MarcoFalke)
Pull request description:
For local testing we don't need to rely on p2p messages just to assert a reject reason.
Replace reading p2p messages with reading from the debug log file.
Tree-SHA512: fa59598ecf5e00cfb420ef1892d90aa415501fd882e1c608894dc577b0d00e93a442326d3a9167fef77d26aafbe345b730b49109982ccad68a5942384564a90b
f7e9e70468 [rpc] Remove deprecated sigrawtransaction rpc method. (John Newbery)
90c834089a [RPC] Remove warning about wallet addresses in createmultisig() (John Newbery)
df905e390e [rpc] Remove deprecated validateaddress usage. (John Newbery)
Pull request description:
The following rpc features were deprecated in V0.17:
- `validateaddress` returning wallet information about an address
- `signrawtransaction`
This PR fully removes those features. It can be merged once V0.17 has been branched from master.
Tree-SHA512: 28293d218cf7e348632081e362f8775f243d091f49aed54c354f017d4a12ae92b87b99f81ee592a1bbf4aebd5d8cd5119278141edde7a0399ff82917ed68b9f6
ed2332aeff test: Add test for config file parsing errors (MarcoFalke)
a66c0f78a9 util: Report parse errors in configuration file (Wladimir J. van der Laan)
Pull request description:
Report errors while parsing the configuration file, instead of silently ignoring them.
$ src/bitcoind -regtest
Error reading configuration file: parse error on line 22: nodebuglogfile, if you intended to specify a negated option, use nodebuglogfile=1 instead
$ src/bitcoind -regtest
Error reading configuration file: parse error on line 22: sdafsdfafs
$ src/bitcoind -regtest
Error reading configuration file: parse error on line 24: -nodebuglogfile=1, options in the configuration file must be specified without leading -
(inspired by https://github.com/bitcoin/bitcoin/pull/14100#issuecomment-417264823)
Tree-SHA512: d516342b65db2969edf200390994bbbda23654c648f85dcc99f9f2d217d3d59a72e0f58227be7b4746529dcfa54ba26d8188ba9f14a57c9ab00015d7283fade2
341f7c7b0e macOS fix: Check for correct version of flake8 to avoid spurious warnings. The brew installed flake8 version is Python 2 based and does not work. (practicalswift)
908a559f33 macOS fix: Add excludes for checks added in the newer shellcheck version installed by brew (practicalswift)
ec4d57bbb3 macOS fix: Work around empty (sub)expression error when using BSD grep (practicalswift)
b57d7d92fe macOS fix: Avoid mapfile due to ancient version of bash shipped with macOS (practicalswift)
Pull request description:
The linters are thoroughly tested under Ubuntu which is what we use in Travis. When reading #14041 I understood that some developers were experiencing problems when running the linters on their local machines.
Assuming these local machines were running macOS I installed a fresh macOS VM, followed the instructions in `build-osx.md` and ran the linters.
This PR contains the changes needed to make `lint-all.sh` run as expected.
Ideally the linters would continuously run also under a Travis macOS environment to make sure we catch these kind of issues before merge.
Tree-SHA512: b39c9a970d14d27db1fb592539923c0bc676b5217f415d02fda3f17bf54d46faa172376e8a3ecab07ca68a3acba9aebe00b2b1b2161b2a36b85fbb672e7efb5c
5d62dcf9cf lint: Make sure we read the command line inputs using utf-8 decoding in python (Chun Kuan Lee)
Pull request description:
Make sure we read the command line inputs using utf-8 decoding in python
occurred from travis cron job:
contrib/verify-commits/verify-commits.py should run with utf-8, otherwise it would raise UnicodeDecodeError
`UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 744: ordinal not in range(128)`
Tree-SHA512: 90e4ad57fdbbbecb0a21fc2d2b03a04f5ef125e54124719ef36e5a85326930b732b47534757a7c3a8730096f3947b009ec898191928b5c2d38f9f4b3e37db48d
ca1a093127 Add regression test: Don't assert(...) with side effects (practicalswift)
4c3c9c3869 Don't assert(...) with side effects (practicalswift)
Pull request description:
Don't `assert(...)` with side effects.
From the developer notes:
> **Assertions should not have side-effects**
>
> Rationale: Even though the source code is set to refuse to compile with assertions disabled, having side-effects in assertions is unexpected and makes the code harder to understand
These assertions were introduced quite recently (in #14069 which was merged two days ago) and since this is a recurring thing (see #13534 – "Don't assert(foo()) where foo() has side effects" from May) I added a simple regression test for the most obvious common side effect.
Tree-SHA512: be65db9d8d5d0f5752152ba73fe3fbb0531880f156d3cd7dfdf1752709979b63214e46ae64b1adbe1e09fa121278f4087f4ae49bff16cf8f5aec16ea6bde3650
6af6d9b23d test: Add tests for RPC help (João Barbosa)
Pull request description:
At the moment the new test checks for:
- invalid usages
- expected output for unknown command
- current RPC command titles (derived from command categories) — this prevents adding wrong RPC categories and new categories must be added to the test
Tree-SHA512: f987535d001b1cd300656588602b1634099ea68a1dd2282180c30fa56caf7f990be9e2dc86c7431dfcf7fd686d0299a8d4935df178a2c9f0fb6fbebcba748eb5
b193d5a443 Removes the Boost case_conv.hpp dependency. (251)
7a208d9fad Implements custom tolower and toupper functions. (251)
e2ba043b8d Implements ParseNetwork unit test. (251)
Pull request description:
This pull request removes the `boost/algorithm/string/case_conv.hpp` dependency from the project.
`boost/algorithm/string/case_conv.hpp` is included for the `boost::to_lower` and `boost::to_upper` template functions.
We can replace the calls to these functions with straightforward alternative implementations that use the C++ Standard Library, because the functions are called with `std::string` objects that use standard 7-bit ASCII characters as argument.
The refactored implementation should work without the explicit `static_cast<unsigned char>` cast and `unsigned char` lambda return type. Both have been added defensively and to be explicit. Especially in case of the former, behaviour is undefined (potentially result in a crash) if the `std::toupper` argument is not an `unsigned char`.
A potential alternative, maybe even preferred, implementation to address the `boost::to_lower` function call in `ParseNetwork(std::string)` could have been:
```c++
if (net == "ipv4" || net == "IPv4") return NET_IPV4;
if (net == "ipv6" || net == "IPv6") return NET_IPV6;
```
This alternative implementation would however change the external behaviour of `ParseNetwork(std::string)`.
This pull requests includes a unit test to validate the implementation of `ParseNetwork(std::string)` prior and after the removal of the `case_conv.hpp` dependency.
`boost/algorithm/string/case_conv.hpp` has been removed from the `EXPECTED_BOOST_INCLUDES` in `test/lint/lint-includes.sh` because it is no longer required.
Tree-SHA512: d803ae709f2368a3efb223097384a722436955bce0c44a1a5cffd0abb3164be0cce85ba0e9ebd9408166df3f1a95ea0c0d29e3a2534af2fae206c0419d67fde9
1661a472b8 add unicode compatible file_lock for Windows (Chun Kuan Lee)
Pull request description:
boost::interprocess::file_lock cannot open the files that contain characters which cannot be parsed by the user's code page on Windows.
This PR is seperated from #13426 for easier review.
Tree-SHA512: e240479cda65958bf6e1319840b83928b2b50da81d99f4f002fb3b62621370bcd4bcfacd2b8c0678c443a650d6ba53d9d12618b591e5bfd67ac14388a18fd822
fae040010d qa: Add some actual witness in rpc_rawtransaction (MarcoFalke)
Pull request description:
The transaction was serialized with the witness flag but didn't include any witness, so add some dummy witness...
Tree-SHA512: fe71167c16e9b0053110be7c544e1ea08868f04ffee8d4c74887c9bcdcd5b59d5e8dd53c67e104a1bdbbf606202bc3fbef6017f402f2c75bdb2ebd9f7aabb2b1
This commit removes the `boost/algorithm/string/case_conv.hpp` dependency from the project. It replaces the `boost::to_lower` and `boost::to_upper` functions with custom functions that are locale independent and ASCII deterministic.
16e288acdd test padding non micro timestamps (John Newbery)
995dd89d88 [Tests] Make combine_logs.py handle multi-line logs (John Newbery)
Pull request description:
combine_logs.py currently inserts additional newlines into multi-line
log messages, and doesn't color them properly. Fix both of those.
Tree-SHA512: dbe2f3ecc7cfbc95ee4350e648d127538c79cb6555257d4aeec12fe3d159366742b68e90e620c8ed7219a44b973395c7e5929ba374fae115fbee25560db645f6
boost::interprocess::file_lock cannot open the files that contain characters which cannot be parsed by the user's code page on Windows.
This commit add a new class to handle those specific file for Windows.
bb08423d5c [doc] Add release notes for 'account' API removal (John Newbery)
1f4b865e57 [wallet] Re-sort wallet RPC commands (John Newbery)
f0dc850bf6 [wallet] Remove wallet account RPCs (John Newbery)
c410f41575 [tests] Remove wallet accounts test (John Newbery)
Pull request description:
This is the first part of #13825. It simply removes the RPC methods and tests.
#13825 touches lots of files and will require frequent rebasing.
Breaking it down for easier reviewing and fewer rebases.
Tree-SHA512: d29af8e7a035e4484e6b9bb56cb86592be0ec112d8ba4ce19c15d15366ff3086e89e99fca26b90c9d66f6d3e06894486d0f29948df0bb7dcb1e2c49c6887a85a
414326952c use export LC_ALL=C.UTF-8 (Julian Fleischer)
728c82d029 make script exit if a command fails (Julian Fleischer)
506890b24d move remaining travis build steps into individual files (Julian Fleischer)
272306ea57 number .travis/ script according to build lifecycle and add README to explain (Julian Fleischer)
519e2739cf move lint stage up to resemble travis build ui (Julian Fleischer)
86d34f0e65 abort script in END_FOLD on non-zero exit code (Julian Fleischer)
4f2f88c7b0 move script sections info individual files and comply with shellcheck (Julian Fleischer)
Pull request description:
This PR is extracted from https://github.com/bitcoin/bitcoin/pull/13816 to make that one easier to review. It follows on https://github.com/bitcoin/bitcoin/pull/13849 and https://github.com/bitcoin/bitcoin/pull/13851
In here the shell script parts from `travis.yml` are extracted into `.travis/before_install.sh`, `.travis/install.sh`, `.travis/before_script.sh`, `.travis/script.sh`, and `.travis/lint.sh`.
This has the benefit that `test/lint/lint-shell.sh` will also shellcheck these parts. Also it makes the individual script parts more readable.
Tree-SHA512: c497e1687ceb1c1d795de177d3fc35af908bc8e3f781a871afabdecf031e581d4db229290627249e35ef7c09952bc34884e4734ea91d40f57b4a9efb85bba2e3
5778bf95d9 Report minfeefilter value in getpeerinfo rpc (Anthony Towns)
Pull request description:
Lowering the minimum relay fee is only useful when many nodes in the p2p network also lower the fee, so to make it easier to understand progress on that front, this includes the value of the minfeefilter in getpeerinfo, so you at least have visibility to what fees your neighbours are currently accepting.
Tree-SHA512: 059f01bf2a32c98fce1648a13b7898701203b354d0209ee34e6683994b720eb594cf24968e66b699caae5e17e53d351e73281f042dd094decde14d3a318e9fb3
Lowering the minimum relay fee is only useful when many nodes in the
p2p network also lower the fee, so to make it easier to understand
progress on that front, this includes the value of the minfeefilter in
getpeerinfo, so you at least have visibility to what fees your neighbours
are currently accepting.
870bd4c73d Update functional RBF test to check replaceable flag (dexX7)
820d31f95f Add "bip125-replaceable" flag to mempool RPCs (dexX7)
Pull request description:
This pull request adds a flag "bip125-replaceable" to the mempool RPCs getrawmempool, getmempoolentry, getmempoolancestors and getmempooldescendants, which indicates whether an unconfirmed transaction might be replaced.
Initially the flag was added to the raw transaction RPCs, but thanks to @conscott, it was moved to the mempool RPCs, which actually have access to the mempool.
~~This pull request adds a flag "bip125-replaceable" to the RPCs "getrawtransaction" and "decoderawtransaction", which indicates, whether a transaction signals BIP 125 replaceability.~~
There was some discussion in #7817, whether showing replaceability in the UI could lead to the false assumption that transactions that don't signal BIP 125 are truely non-replaceable, but given that this PR tackles the raw transaction interface, which is a rather low level tool, I believe having this extra piece of information isn't bad.
Tree-SHA512: 1f5511957af2c20a9a6c79d80a335c3be37a2402dbf829c40cceaa01a24868eab81a9c1cdb0b3d77198fa3bb82799e3540a5c0ce7f35bbac80d73f7133ff7cbc
317f2cb3f4 test: Check RPC settxfee errors (João Barbosa)
48618daf26 Add checks for settxfee reasonableness (Anthony Towns)
Pull request description:
When using the `settxfee` RPC, the value is silently ignored if it is less than either than minrelaytxfee or the wallet's mintxfee. This adds an error response if that's going to happen, but still allows "settxfee 0" to deliberately default to the minimum value.
Tree-SHA512: ce685584cf8d6b9ca2cc97196d494220e3892b6a804a458086e04b3a23df281da432ad0a3053106a064c90c541ddb6f6b96a27cf8376d45af1e44449baf88456
faaac5caaa RPCTypeCheck bip32derivs arg in walletcreatefunded (Gregory Sanders)
1f0c4282e9 QA: add basic walletcreatefunded optional arg test (Gregory Sanders)
1f18d7b591 walletcreatefundedpsbt: remove duplicate replaceable arg (Gregory Sanders)
2252ec5008 Allow ConstructTransaction to not throw error with 0-input txn (Gregory Sanders)
Pull request description:
1) Previously an empty input argument transaction that is marked for replaceability fails to pass the `SignalsOptInRBF` check right before funding it. Explicitly check for that condition before throwing an error.
2) The rpc call had two separate `replaceable` arguments, each of which being used in mutually exclusive places. I preserved the `options` version to retain compatability with `fundtransaction`.
Tree-SHA512: 26eb0c9e2d38ea51d11f741d61100223253271a084adadeb7e78c6d4e9004636f089e4273c5bf64a41bd7e9ff795317acf30531cb36aeb0d8db9304b3c8270c3
fa091b0016 qa: Add tests for submitheader (MarcoFalke)
36b1b63f20 rpc: Expose ProcessNewBlockHeaders (MarcoFalke)
Pull request description:
This exposes `ProcessNewBlockHeaders` as an rpc called `submitheader`. This can be used to check for invalid block headers and submission of valid block headers via the rpc.
Tree-SHA512: a61e850470f15465f88e450609116df0a98d5d9afadf36b2033d820933d8b6a4012f9f2b3246319c08a0e511bef517f5d808cd0f44ffca91d10895a938004f0b
5df6f089b5 More tests of signer checks (Andrew Chow)
7c8bffdc24 Test that a non-witness script as witness utxo is not signed (Andrew Chow)
8254e9950f Additional sanity checks in SignPSBTInput (Pieter Wuille)
c05712cb59 Only wipe wrong UTXO type data if overwritten by wallet (Pieter Wuille)
Pull request description:
The current PSBT signing code can end up producing a non-segwit signature, while only the UTXO being spent is provided in the PSBT (as opposed to the entire transaction being spent). This may be used to trick a user to incorrectly decide a transaction has the semantics he intends to sign.
Fix this by refusing to sign if there is any mismatch between the provided data and what is being signed.
Tree-SHA512: b55790d79d8166e05513fc4c603a982a33710e79dc3c045060cddac6b48a1be3a28ebf8db63f988b6567b15dd27fd09bbaf48846e323c8635376ac20178956f4
bd19cc78cf Serialize non-witness utxo as a non-witness tx but always deserialize as witness (Andrew Chow)
43811e6338 Fix PSBT deserialization of 0-input transactions (Andrew Chow)
Pull request description:
0-input transactions can be ambiguously deserialized as being witness transactions. Since the unsigned transaction is never serialized as a witness transaction as it has no witnesses, we should always deserialize it as a non-witness transaction and set the serialization flags as such.
When a transaction is serliazed for the non-witness-utxo, it is always a valid network transaction and thus it should be always be deserialized as a witness transaction and the deserialzation flags are set as such.
Fixes#13958
Tree-SHA512: 1937b3cb2618534478d4f533541fb9efce3cb5badb5d1964bfe19400f4aacc6c8ecedaf1f20d26b20baf94f81fd07dfb15b3b08089ecbd63aeecbc18c7c48086
0-input transactions can be ambiguously deserialized as being witness
transactions. Since the unsigned transaction is never serialized as
a witness transaction as it has no witnesses, we should always
deserialize it as a non-witness transaction and set the serialization
flags as such.
Also always serialize the unsigned transaction as a non-witness transaction.
68400d8b96 tests: Use explicit imports (practicalswift)
Pull request description:
Enable automatic detection of undefined names in Python tests scripts. Remove wildcard imports.
Wildcard imports make it unclear which names are present in the namespace, confusing both readers and many automated tools.
An additional benefit of not using wildcard imports in tests scripts is that readers of a test script then can infer the rough testing scope just by looking at the imports.
Before this commit:
```
$ contrib/devtools/lint-python.sh | head -10
./test/functional/feature_rbf.py:8:1: F403 'from test_framework.util import *' used; unable to detect undefined names
./test/functional/feature_rbf.py:9:1: F403 'from test_framework.script import *' used; unable to detect undefined names
./test/functional/feature_rbf.py:10:1: F403 'from test_framework.mininode import *' used; unable to detect undefined names
./test/functional/feature_rbf.py:15:12: F405 bytes_to_hex_str may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:17:58: F405 CScript may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:25:13: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:26:31: F405 satoshi_round may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:26:60: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:30:41: F405 satoshi_round may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
./test/functional/feature_rbf.py:30:68: F405 COIN may be undefined, or defined from star imports: test_framework.mininode, test_framework.script, test_framework.util
$
```
After this commit:
```
$ contrib/devtools/lint-python.sh | head -10
$
```
Tree-SHA512: 3f826d39cffb6438388e5efcb20a9622ff8238247e882d68f7b38609877421b2a8e10e9229575f8eb6a8fa42dec4256986692e92922c86171f750a0e887438d9
To avoid:
$ make check
{…omissis…}
Running test/util/bitcoin-util-test.py...
/usr/local/bin/python3.7 ../test/util/bitcoin-util-test.py
../test/util/bitcoin-util-test.py:31: DeprecationWarning: This method will be removed in future versions. Use 'parser.read_file()' instead.
config.readfp(open(os.path.join(os.path.dirname(__file__), "../config.ini"), encoding="utf8"))
$ python3 --version
Python 3.7.0
cf9ed307e6 qa: blocktools enforce named args for amount (MarcoFalke)
Pull request description:
Since #13669 changed some signatures, I think it might be worthwhile to enforce named args for primitive types such as amounts.
Tree-SHA512: 2733e7b6a20590b54bd54e81a09e3f5e2fadf4390bed594916b70729bcf485b048266012c1203369e0968032a2c6a2719107ac17ee925d8939af3df916eab1a6
4b7091a842 Replace median fee rate with feerate percentiles (Marcin Jachymiak)
Pull request description:
Currently, the `medianfeerate` statistic is calculated from the feerate of the middle transaction of a list of transactions sorted by feerate.
This PR instead uses the value of the 50th percentile weight unit in the block, and also calculates the feerate at the 10th, 25th, 75th, and 90th percentiles. This more accurately corresponds with what is generally meant by median feerate.
Tree-SHA512: 59255e243df90d7afbe69839408c58c9723884b8ab82c66dc24a769e89c6d539db1905374a3f025ff28272fb25a0b90e92d8101103e39a6d9c0d60423a596714
4441ad677a Make format string linter understand basic template parameter syntax (practicalswift)
Pull request description:
Make format string linter understand basic template parameter syntax.
Fixes issue described in https://github.com/bitcoin/bitcoin/pull/13705#issuecomment-412046126.
Thanks to @ken2812221 for reporting!
Tree-SHA512: 8fb995bc6b29d8b9926ef5969e02cf71c494e829434fcdeb4ed5fabad5ab96e86e5b8eea705e8a416927757b4fa4e58abc0fd4f483daa58c94e2c6fdcb8ee822
e306be7429 Use 72 byte dummy signatures when watching only inputs may be used (Andrew Chow)
48b1473c89 Use 71 byte signature for DUMMY_SIGNATURE_CREATOR (Andrew Chow)
18dfea0dd0 Always create 70 byte signatures with low R values (Andrew Chow)
Pull request description:
When creating signatures for transactions, always make one which has a 32 byte or smaller R and 32 byte or smaller S value. This results in signatures that are always less than 71 bytes (32 byte R + 32 byte S + 6 bytes DER + 1 byte sighash) with low R values. In most cases, the signature will be 71 bytes.
Because R is not mutable in the same way that S is, a low R value can only be found by trying different nonces. RFC 6979 for deterministic nonce generation has the option to specify additional entropy, so we simply use that and add a uin32_t counter which we increment in order to try different nonces. Nonces are sill deterministically generated as the nonce used will the be the first one where the counter results in a nonce that results in a low R value. Because different nonces need to be tried, time to produce a signature does increase. On average, it takes twice as long to make a signature as two signatures need to be created, on average, to find one with a low R.
Having a fixed size signature makes size calculations easier and also saves half a byte of transaction size, on average.
DUMMY_SIGNATURE_CREATOR has been modified to produce 71 byte dummy signatures instead of 72 byte signatures.
Tree-SHA512: 3cd791505126ce92da7c631856a97ba0b59e87d9c132feff6e0eef1dc47768e81fbb38bfbe970371bedf9714b7f61a13a5fe9f30f962c81734092a4d19a4ef33
Removes medianfeerate result from getblockstats.
Adds feerate_percentiles which give the feerate of the 10th, 25th, 50th,
75th, and 90th percentile weight unit in the block.
fafe73a626 qa: Raise feature_help timeout to 5s (MarcoFalke)
faabd7bc47 qa: Use files for stdout/stderr to support Windows (MarcoFalke)
facb56ffaf qa: Run gen_rpcauth with sys.executable (MarcoFalke)
fada8966c5 qa: Close stdout and stderr file when node stops (MarcoFalke)
Pull request description:
### qa: Close stdout and stderr file when node stops
Since these files are potentially deleted by the test framework for cleanup, they should be closed first. Otherwise this will lead to errors on Windows when the tests finish successfully.
Side note: After the patch, it is no longer possible to reopen the file on Windows (see https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile)
### qa: Run gen_rpcauth with sys.executable
Similar to `test_runner.py`, the `sys.executable` needs to be passed down into subprocesses to pass on native Windows. (Should have no effect on Linux)
### qa: Use files for stdout/stderr to support Windows
It seems that using PIPE is not supported on Windows. Also, it is easier to just use the files that capture the stdout and stderr within the test node class.
Tree-SHA512: ec675012b10705978606b7fcbdb287c39a8e6e3732aae2fa4041d963a3c6993c6eac6a9a3cbd5479514e7d8017fe74c12235d1ed6fed2e8af8f3c71981e91864
fa85c985ed qa: Add p2p_invalid_locator test (MarcoFalke)
Pull request description:
Should not be merged *before* #13907
Tree-SHA512: a67ca407854c421ed20a184d0b0dc90085aed3e3431d9652a107fa3022244767e67f67e50449b7e95721f56906836b134615875f28a21e8a012eb22cfe6a66a5
1f87c372b5 Simplify comparison in rpc_blockchain.py. (Daniel Kraft)
Pull request description:
The test for `gettxoutsetinfo` in `rpc_blockchain.py` verifies that the result is the same as before after invalidating and reconsidering a block. The comparison has to exclude the `disk_size` field, though, as it is not deterministic.
Instead of comparing all the other fields for equality, this change explicitly removes the `disk_size` field and then compares the full objects. This makes the intent more explicit (compare everything except for `disk_size`, not compare just a given list of fields) and also the code simpler.
Tree-SHA512: 3c376a8836b62988fb2f0117c9ca65de64a33bf3cd4980a123de30bf5e7b7a48eda477b25e03d672ff076e205c698e83432469156caa0f0f3ebbb0480f0dd77d
When extra entropy is not specified by the caller, CKey::Sign will
now always create a signature that has a low R value and is at most
70 bytes. The resulting signature on the stack will be 71 bytes when
the sighash byte is included.
Using low R signatures means that the resulting DER encoded signature
will never need to have additional padding to account for high R
values.
The test for gettxoutsetinfo in rpc_blockchain.py verifies that the
result is the same as before after invalidating and reconsidering a
block. The comparison has to exclude the 'disk_size' field, though, as
it is not deterministic.
Instead of comparing all the other fields for equality, this change
explicitly removes the 'disk_size' field and then compares the full
objects. This makes the intent more explicit (compare everything except
for disk_size, not compare just a given list of fields) and also the
code simpler.
fa5587fe71 qa: wait_for_verack by default (MarcoFalke)
Pull request description:
This removes the need to do so manually every time a connection is added.
Tree-SHA512: a46c92cb4df41e30778b42b9fd3dcbd8d2d82aa7503d1213cb1c1165034f648d8caee01c292e2d87d05b0f71696996eef5be8a753f35ab49e5f66b0e3bf29f21
3fc20632a3 qt: Set BLOCK_CHAIN_SIZE = 220 (DrahtBot)
2b6a2f4a28 Regenerate manpages (DrahtBot)
eb7daf4d60 Update copyright headers to 2018 (DrahtBot)
Pull request description:
Some trivial maintenance to avoid having to do it again after the 0.17 branch off.
(The scripts to do this are in `./contrib/`)
Tree-SHA512: 16b2af45e0351b1c691c5311d48025dc6828079e98c2aa2e600dc5910ee8aa01858ca6c356538150dc46fe14c8819ed8ec8e4ec9a0f682b9950dd41bc50518fa
ParseHashV validates the length and encoding of the string and throws
an informative RPC error on failure, which is as good or better than
these alternative calls.
Note I switched ParseHashV to check string length first, because
IsHex tests that the length is even, and an error like:
"must be of length 64 (not 63, for X)" is much more informative than
"must be hexadecimal string (not X)"
faa24441ec policy: Remove promiscuousmempoolflags (MarcoFalke)
Pull request description:
It seems odd to clutter validation code with features that can only ever be used for testing (testnet or regtest). Removing that test-only code makes the mempool logic less painful to understand and easier to reason about when changed or refactored in the future.
Tree-SHA512: 3b897aa9604ac8d82ebe9573c6efd468c93ddaa08d378ebc902e247b7aa6c68fcde71e5b449c08f17a067146cdc66dc50a67ce06d07607c27e5189a49c3fba3f
This linter checks that the number of arguments passed to each variadic format
string function matches the number of format specifiers in the format string.
a1a998cf24 wallet: Fix backupwallet for multiwallets (Daniel Kraft)
Pull request description:
`backupwallet` was broken for multiwallets in their own directories (i.e. something like `DATADIR/wallets/mywallet/wallet.dat`). In this case, the backup would use `DATADIR/wallets/wallet.dat` as source file and not take the specific wallet's directory into account.
This led to either an error during the backup (if the wrong source file was not present) or would silently back up the wrong wallet; especially the latter behaviour can be quite bad for users.
Tree-SHA512: 7efe2450ca047e40719fcc7cc211ed94699056020ac737cada7b59e8240298675960570c45079add424d0aab520437d5050d956acd695a9c2452dd4317b4d2c4
