Adds a utility to get resident set size memory usage for a test
node and a context manager that allows assertions based upon
maximum memory use increase.
4fb3388db9 check that a separator is found for psbt inputs, outputs, and global map (Andrew Chow)
Pull request description:
Currently it doesn't make sure that a separator was found so PSBTs missing a trailing separator would still pass. This fixes that and adds a test case for it.
It really only makes sense to check for the separator for the output maps as if an input or global map was missing a separator, the fields following it would be interpreted as belonging to the previous input or global map. However I have added the check for those two anyways to be consistent.
Tree-SHA512: 50c0c08e201ba02494b369a4d36ddb73e6634eb5a4e4e201c4ef38fd2dbeea2c642b8a04d50c91615da61ecbfade37309e47431368f4b1064539c42015766b50
This requires a small changes to a few tests, but means that
deterministic addresses will always be imported (unless setup_nodes
behaviour is explicitly overridden).
862d159d63 Add test for conversion from non-witness to witness UTXO (Pieter Wuille)
f8c1714634 Convert non-witness UTXOs to witness if witness sig created (Andrew Chow)
Pull request description:
If a witness signature was created when a non-witness UTXO is used, convert the non-witness UTXO to a witness one.
Port of #14196 to master.
Tree-SHA512: 2235eeb008ffa48e821628032d689e4a83bff6c29b93fa050ab2ee492b0e67b3a30f29a680d4a0e574e05c3a2f9edf0005e161fbe25b7aef2acd034a2424e2f2
4bd125fff0 tests: Print dots by default (Chun Kuan Lee)
Pull request description:
In cron job (https://travis-ci.org/bitcoin/bitcoin/builds/445823485), the functional tests would fail due to silent for 10 mins.
After applying this patch, we con't see any extra characters printed on screen but also avoid timeout (https://travis-ci.org/ken2812221/bitcoin/builds/445981698)
Tree-SHA512: c0412e171a451b27f9734311c7f063ad3fd7142087ed1e3786b4f303acaebc043f970523d6c2d4ef57ec5857040e2b6f7fd6345304353e7805d76044d317344d
c11875c590 Add segwit address tests for importmulti (MeshCollider)
201451b1ca Make getaddressinfo return solvability (MeshCollider)
1753d217ea Add release notes for importmulti segwit change (MeshCollider)
353c064596 Fix typo in test_framework/blocktools (MeshCollider)
f6ed748cf0 Add SegWit support to importmulti with some ProcessImport cleanup (MeshCollider)
Pull request description:
Add support for segwit to importmulti, supports P2WSH, P2WPKH, P2SH-P2WPKH, P2SH-P2WSH. Adds a new `witnessscript` parameter which must be used for the witness scripts in the relevant situations.
Also includes some tests for the various import types.
~Also makes the change in #14019 redundant, but cherry-picks the test from that PR to test the behavior (@achow101).~
Fixes#12253, also addresses the second point in #12703, and fixes#14407
Tree-SHA512: 775a755c524d1c387a99acddd772f677d2073876b72403dcfb92c59f9b405ae13ceedcf4dbd2ee1d7a8db91c494f67ca137161032ee3a2071282eeb411be090a
fa78a2fc67 [tests] Test that nodes respond to getdata with notfound (MarcoFalke)
Pull request description:
If a node has not announced a tx at all, then it should respond to
getdata messages for that tx with notfound, to avoid leaking tx
origination privacy.
In the future this could be adjusted such that a node responds with
notfound when a tx has not been announced to us, but that seems
to be a more involved change. See e.g.
https://github.com/jnewbery/bitcoin/commits/pr14220.1
Tree-SHA512: 6244afa5bd5d8fec9b89dfc02c9958bc370195145a0f3715f33200d6cf73a376c94193d44bf4523867196e6591c53ede8f9b6a77cb296b48c114a117b8c8b1fa
If a node has not announced a tx at all, then it should respond to
getdata messages for that tx with notfound, to avoid leaking tx
origination privacy.
ed2e18398b Remove fs::relative call and fix listwalletdir tests (João Barbosa)
Pull request description:
The implementation of `fs::relative` resolves symlinks which is not intended
in ListWalletDir. The replacement does what is required, and `listwalletdir` RPC
tests are fixed accordingly.
Also, `fs::recursive_directory_iterator` iteration is fixed to build with boost 1.47.
Based on #14559
Tree-SHA512: 1da516226073f195285d10d9d9648c90cce0158c5d1eb9c31217bb4abb575cd37f07c00787c5a850554d6120bbc5a3cbc5cb47d4488b32ac6bcb52bc1882d600
3be209d103 rpc: Always throw in getblockstats if -txindex is required (João Barbosa)
Pull request description:
Previously blocks with only the coinbase transaction didn't cause
the RPC error even if the requested stats required -txindex and
it wasn't enabled.
Fixes#14499.
Tree-SHA512: d3a6402889e3ce7199632e79eba66d7d471ff7de5c564d35312e2340cc6d84ef544a8172548fbc2eedf5e637b56dc57bbf7a9815ab798c7f226755f897fd8f3e
The implementation of fs::relative resolves symlinks which is not intended
in ListWalletDir. The replacement does what is required, and listwalletdir
tests are fixed accordingly.
Also, building with boost 1.47 required 2 changes:
- replace fs::relative with an alternative implementation;
- fix fs::recursive_directory_iterator iteration.
4ea77320c5 tests: add test case for loading copied wallet twice (Chun Kuan Lee)
2d796faf62 wallet: Fix duplicate fileid (Chun Kuan Lee)
Pull request description:
The implementation in current master can not detect if the file ID is duplicate with flushed `BerkeleyEnvironment`. This PR would store the file ID in a global variable `g_fileids` and release it when the `BerkeleyDatabase` close. So it won't have to rely on a `Db*`.
Fix#14304
Tree-SHA512: 0632254b696bb4c671b5e2e5781e9012df54ba3c6ab0f919d9f6d31f374d3b0f8bd968b90b537884ac8c3d2906afdd58c2ce258666263464c7dbd636960b0e8f
96c509e4d0 show the progress of functional test (Isidoro Ghezzi)
Pull request description:
example: (added the progress index `n/m`)
```
1/107 - wallet_hd.py passed, Duration: 27 s
.........................................................................................
2/107 - mining_getblocktemplate_longpoll.py passed, Duration: 72 s
..................................................................
3/107 - feature_maxuploadtarget.py passed, Duration: 78 s
```
Tree-SHA512: 17b840048222e2c3676a92041b491521fee3b86049b2f2467a225aece40717732341801872d9867fcb7260e904e322c7184b76fca16d2dc687aa75dd741484ad
8907df9e02 qa: Ensure wallet unload during walletpassphrase timeout (João Barbosa)
321decffa1 rpc: Fix wallet unload during walletpassphrase timeout (João Barbosa)
Pull request description:
Replaces the raw wallet pointer in the `RPCRunLater` callback with a `std::weak_ptr` to check if the wallet is not expired.
To test:
```
bitcoind -regtest
bitcoin-cli -regtest encryptwallet foobar
bitcoin-cli -regtest walletpassphrase foobar 5 && bitcoin-cli -regtest unloadwallet ""
```
Fixes#14452.
Tree-SHA512: 311e839234f5fb7955ab5412a2cfc1903ee7132ea56a8ab992ede3614586834886bd65192b76531ae0aa3a526b38e70ca2e1cdbabe52995906ff97b49d93c268
d4d70eda33 Fix listreceivedbyaddress not taking address as a string (Eric Scrivner)
Pull request description:
Fixes#14173. Add the patch in #14173 and include a regression test.
Tree-SHA512: 5a9794e0c43e90d18c899841afbaf15eb9129d7d2f6570fccf0a1793697fe170d224c3c3995b1a35c536fac19819042823d9e3bd23b019d0f03434499243d2f5
94e21c1501 test: forward timeouts properly in send_blocks_and_test (James O'Beirne)
Pull request description:
Small change motivated by frustrations while writing `feature_block` tests; when a timeout is passed to `send_blocks_and_test` it isn't forwarded onto constituent waiting calls - you can end up waiting 60 seconds when you articulated e.g. 5. Respect the given timeout all the way down.
Tree-SHA512: 3a964764fc5e3431ae3b17bd642a27a1bd4526541a799ef63696c9dab0289a005a13d645770be6e46ea262d22a58f79d2b407293a39397b036f616fe20c21241
369244f654 utils: Fix broken Windows filelock (Chun Kuan Lee)
Pull request description:
Fix broken filelock on Windows, also add a test for this. It's a regression introduced by #13862.
Tree-SHA512: 15665b1930cf39ec71f3ab07def8e2897659f6fd4d2de749d63a5a8ec920e4a04282f12bc262f242b1b3d14d2dd9fa191ddbcf16a46fb927b5b2b14d9f6b5d01
ca6d86c322 tests: Stop node before removing the notification file (Chun Kuan Lee)
Pull request description:
Stop node before removing the notification file to make sure the command has been terminated. After then we could removing those files safely and do not receive any permission error. (See #14446)
The permission error is Windows specific, documented in python doc:
>On Windows, attempting to remove a file that is in use causes an exception to be raised
See https://docs.python.org/3/library/os.html#os.remove
Tree-SHA512: fbdabf3a9a838bb59ba207dd9e9fbdd87c702a99ad66bee0b2b1537f80f8630d22d9d5e9c4ded23a82a66bfc10989227fb024b27393425abe0e5a2ad4e4cbb82
d56a068935 docs: Add release notes for listwalletdir RPC (João Barbosa)
0cb3cad166 qa: Add tests for listwalletdir RPC (João Barbosa)
cc3377360c rpc: Add listwalletdir RPC (João Barbosa)
d1b03b8e5f interfaces: Add getWalletDir and listWalletDir to Node (João Barbosa)
fc4db35bfd wallet: Add ListWalletDir utility (João Barbosa)
Pull request description:
`ListWalletDir` returns all available wallets in the current wallet directory.
Based on MeshCollider work in pull #11485.
Tree-SHA512: 5843e3dbd1e0449f55bb8ea7c241a536078ff6ffcaad88ce5fcf8963971d48c78600fbc4f44919523b8a92329d5d8a5f567a3e0ccb0270fdd27366e19603a716
Adds a generate() method to the TestNode class in the test framework.
This method intercepts calls to generate, imports a dewterministic
private key to the node and then calls generatetoaddress to generate the
block to that address.
Note that repeated calls to importprivkey for the same private keys are
no-ops, so it's fine to call the generate() method many times.
In advance of deprecating the generate RPC method, make some small
changes to a small number of inidividual test cases:
- make memory checking less prescriptive in wallet_basic.py
- replace calls to generate with generatetoaddress in wallet_keypool.py
- replace calls to generate with generatetoaddress and fixup label
issues in wallet_labels.py
- replace calls to generate with generatetoaddress in wallet_multiwallet.py
854c85ae90 test: allow arguments to be forwarded to flake8 in lint-python.sh (James O'Beirne)
Pull request description:
In order to use `lint-python.sh` from within various in-editor linting frameworks (e.g. [ALE](https://github.com/w0rp/ale), [flycheck](https://github.com/flycheck/flycheck)), we need to allow its arguments to be forwarded to the wrapped flake8 invocation.
For what it's worth, here's my bitcoin-specific ALE vim config for doing so (requires this changeset):
```vim
$ grep python /home/james/src/bitcoin/.exrc
let g:ale_python_flake8_executable="/home/james/src/bitcoin/test/lint/lint-python.sh"
```
Tree-SHA512: 0d5500238ea5fde26ee9c21f6518a3a3dc8409c77ad1271ff7e7a94ef45a8c8d2e1b8ad3df3075dd4062ee0fff534625b1bc79613f869cd3c2d9260814ffc7ee
faa4043c66 qa: Run more tests with wallet disabled (MarcoFalke)
Pull request description:
Instead of skipping the whole test, only skip the wallet specific section of a test if the wallet is not compiled in. This is mostly an indentation change, so can be reviewed with `--ignore-all-space`.
Tree-SHA512: 5941a8b6b00dca5cf9438c5f6f010ba812115188a69e427d7ade4c1ab8cfe7a57c73daf52c66235dbb24b1cd9ab7c7a17c49bc23d931e041b605d79116a71f66
62c304ea48 tests: Allow closed http server in assert_start_raises_init_error (Chun Kuan Lee)
Pull request description:
The rpc handler may be unregistered when http server haven't been closed yet. So it may be allowable to get -342 `non-JSON HTTP response with \'%i %s\' from server` (503 Service Unavailable)
See https://ci.appveyor.com/project/DrahtBot/bitcoin/build/master.2001. It shows "Rejecting request while shutting down" between "RPC stopped" and "Stopped HTTP server"
Tree-SHA512: e1f50ab9096cf23494ccc2850c01516c4a75f112c99108759157b22fce2011682a4b88216f206702f6a56e960182c00098053ad75f13aa0eafe27046adae63da
42a995ae48 [tests] Remove rpc_zmq.py (John Newbery)
Pull request description:
rpc_zmq.py is racy and fails intermittently. Remove that test file and
move the getzmqnotifications RPC test into interface_zmq.py.
Tree-SHA512: 666c8f252f8a392deda1bd531e84fdc04bdae4eab09407657ade2b5fc0aeffa247735e20314236f56e4e3402476673f3b7538d6e09f5af6976021ba2377ce63c
380c843217 utils: Convert Windows args to utf-8 string (Chun Kuan Lee)
Pull request description:
Create a new class `WinCmdLineArgs` when building for Windows. It converts all command line arguments to utf8 string.
Tree-SHA512: f098520fd123a8a452bc84a55dc8c0b88f0c475410efe57f2ccc393f86c396eed59ea1575ddc1b920323792e390fdb092061d80cdcd9b682f0ac79a22a22ff82
- changes importprivkey behavior to overwrite existent label if one
is passed and keep existing ones if no label is passed
- tests behavior of importprivkey on existing address labels and
different same key destination
c7b3e487f2 tests: exclude all tests with difference parameters (Chun Kuan Lee)
Pull request description:
Fix broken exclusion list in functional tests. See https://github.com/bitcoin/bitcoin/pull/14007#pullrequestreview-158309105
Tree-SHA512: b6c2b86fef13e3c00c695adaeeb3e47ee9b48877c71bc605d24201ce931b2ef3ae9f5f199071fa1ec5de2d7aadc478410094c380cc297922e683e9b2569cda03
9c5af58d51 Consolidate redundant implementations of ParseHashStr (Ben Woosley)
Pull request description:
This change:
* adds a length check to all calls to `ParseHashStr`, appropriate given its use to populate
a 256-bit number from a hex str
* allows the caller to handle the failure, which allows for the more
appropriate `JSONRPCError` on failure in `prioritisetransaction` rpc
Relative to #14288
Tree-SHA512: baa791147e5ceb3c30c70df3981aaf807bf7d4a90a0be3625540b59aa4b9a9d303a452bfef18bf167cbb833ef9591b4ef5948bf4a1ce67b421d804ae8d20ea53
a2a04a5abb Bugfix: Only run bitcoin-tx tests when bitcoin-tx is enabled (Luke Dashjr)
92af71cea9 configure: Make it possible to build only one of bitcoin-cli or bitcoin-tx (Luke Dashjr)
Pull request description:
Includes #5618 (which the reasons for rejecting no longer hold true)
Tree-SHA512: f30a8e4a2f70166b7cabef77c4674163b3a9da14c6a547d34f00d1056a19bf4d23e22851eea726fad2afc8735d5473ae91122c770b65ac3886663dc20e2c5b70
e460232876 Document fixed attribute behavior in critical test framework classes. (Justin Turner Arthur)
17b42f4122 Check for specific tx acceptance failures based on script signature (Justin Turner Arthur)
3a4449e9ad Strictly enforce instance attrs in critical functional test classes. (Justin Turner Arthur)
1d0ce94a54 Fix for incorrect version attr set on functional test segwit block. (Justin Turner Arthur)
ba923e32a0 test: Fix broken segwit test (practicalswift)
Pull request description:
No extra attributes will be able to be added to instances of the C++ class ports or of other critical classes without causing an exception. Helps prevent adding or depending on attributes that aren't in the intended object structure. It may prevent issues such as the one fixed in bitcoin/bitcoin#14300.
This request fixes the erroneous version attribute used in the p2p_segwit.py functional tests. This pull includes the commit from bitcoin/bitcoin#14300.
Tree-SHA512: 1b8c58e7aa0f71075ed5ff3e5be0a5182599108d8cd9bce682feac3b1842508124288e9335432b16a43f40f159c9710899e6d84af1b5868f48c947bc6f3e07ec
7ac911afe7 [docs] Add release notes for removing `-usehd` (John Newbery)
25548b2958 [wallet] Remove -usehd (John Newbery)
Pull request description:
`-usehd` is no longer used (except to tell the user that they've set it incorrectly for the wallet that they're loading). Remove it (in the same spirit as #14272)
Tree-SHA512: 5bdcd2bb9bb8504a01343595bcd1bd433d97b730255152c725103c1ac3fa3a9d9e5220a4c29d4c72307cf803e1c09d31080f83603c23dc77263846e17b1826f0
2c3eade704 Make fs::path::string() always return utf-8 string (Chun Kuan Lee)
Pull request description:
Imbue `fs::path` with `std::codecvt_utf8_utf16` at `SetupEnvironment()`, so that default string encoding will be utf-8 inside `fs::path`.
Tree-SHA512: 0cb59464d777278decbf24771fc5ff0cb2caa7bc2fe8ee5cd36c97a2324873a3caad131f08f050393b488316ee7f4ab0b28b7fa4699e41839f8e51b9867d5118
This change:
* adds a length check to ParseHashStr, appropriate given its use to populate
a 256-bit number from a hex str.
* allows the caller to handle the failure, which allows for the more
appropriate JSONRPCError on failure in prioritisetransaction rpc
661ac15a4a appveyor: Run functional tests on appveyor (Chun Kuan Lee)
2148c36b6e tests: Make it possible to run functional tests on Windows (Chun Kuan Lee)
Pull request description:
This PR do the following things:
- Make functional tests compatible with Windows
- Print color output in functional tests for Windows 10
- Run util and functional tests on appveyor
- Do not run symlink tests on Windows
Note:
- The wallet_multiwallet.py fail is unrelated to the test framework, it's a bug related to c++ code or maybe dependencies. `bitcoind` would exit with 0xC0000005(Access violation) during shutdown occasionally. Disable this for now.
- Not using `--failfast` because this is still in experimental. We should track if there is any other error.
- Disable ZMQ tests because the python zmq library could cause access violation sometimes.
- Disable `feature_notifications` because Bitcoin Core handles the command in different thread, whicha can cause a race condition.
Tree-SHA512: b76db137d264e62a5c130e1cbca7a2ca002a7a0f4153fa0b92c1ea6c9c09ef0533e11c49bdbd566c472d8ff59f245758feb5e5a6ec6cb6bb66a1c67bab5fa48a
5eb20f81d9 Consistently use ParseHashV to validate hash inputs in rpc (Ben Woosley)
Pull request description:
ParseHashV validates the length and encoding of the string and throws
an informative RPC error on failure, which is as good or better than
these alternative calls.
Note I switched ParseHashV to check string length first, because
IsHex tests that the length is even, and an error like:
"must be of length 64 (not 63, for X)" is much more informative than
"must be hexadecimal string (not X)" in that case.
Split from #13420
Tree-SHA512: f0786b41c0d7793ff76e4b2bb35547873070bbf7561d510029e8edb93f59176277efcd4d183b3185532ea69fc0bbbf3dbe9e19362e8017007ae9d51266cd78ae
addwitnessaddress is deprecated. Remove the call to that RPC from
wallet_dump.py and improve testing of all types of address (legacy,
p2sh-segwit and bech32)
a2eb6f5405 [rpc] Add getnodeaddresses RPC command (chris-belcher)
Pull request description:
Implements issue https://github.com/bitcoin/bitcoin/issues/9463
New getnodeaddresses call gives access via RPC to the peers known by the node. It may be useful for bitcoin wallets to broadcast their transactions over tor for improved privacy without using the centralized DNS seeds. getnodeaddresses is very similar to the getaddr p2p method.
Please advise me on the best approach for writing an automated test. By my reading the getaddr p2p method also isn't really tested.
Tree-SHA512: ad03abf518847476495b76a2f5394b8030aa86654429167fa618e21460abb505c10ef9817ec1b80472320d41d0aff5dc94a8efce023aaaaf5e81386aa92b852b
New getnodeaddresses call gives access via RPC to the peers known by
the node. It may be useful for bitcoin wallets to broadcast their
transactions over tor for improved privacy without using the
centralized DNS seeds. getnodeaddresses is very similar to the getaddr
p2p method.
Tests the new rpc call by feeding IP address to a test node via the p2p
protocol, then obtaining someone of those addresses with
getnodeaddresses and checking that they are a subset.
c1dde3a949 No longer shutdown after encrypting the wallet (Andrew Chow)
d7637c5a3f After encrypting the wallet, reload the database environment (Andrew Chow)
5d296ac810 Add function to close all Db's and reload the databae environment (Andrew Chow)
a769461d5e Move BerkeleyEnvironment deletion from internal method to callsite (Andrew Chow)
Pull request description:
This is the replacement for #11678 which implements @ryanofsky's [suggestion](https://github.com/bitcoin/bitcoin/pull/11678#pullrequestreview-76464511).
Shutting down the software was to prevent the BDB environment from writing unencrypted private keys to disk in the database log files, as was noted [here](https://bitcointalk.org/index.php?topic=51474.msg616068#msg616068). This PR replaces the shutdown behavior with a CDBEnv flush, close, and reopen which achieves the same effect: everything is cleanly flushed and closed, the log files are removed, and then the environment reopened to continue normal operation.
To ensure that no unencrypted private keys are in the log files after encrypting the wallet, I wrote [this script](https://gist.github.com/achow101/7f7143e6c3d3fdc034d3470e72823e9d) to pull private keys from the original wallet file and searches for these keys in the log files (note that you will have to change your file paths to make it work on your own machine).
As for concerns about private keys being written to slack space or being kept in memory, these behaviors no longer exist after the original wallet encryption PR and the shutting down solution from 2011.
cc @ryanofsky
Tree-SHA512: 34b894283b0677a873d06dee46dff8424dec85a2973009ac9b84bcf3d22d05f227c494168c395219d9aee3178e420cf70d4b3eeacc9785aa86b6015d25758e75
