fa05d52af4 test: Bump timeout to run tests in travis thread sanitizer (MarcoFalke)
Pull request description:
No need to exclude tests because their timeout was set too strict
Tree-SHA512: ea7032e674a66456bab40122adf4beebd703fd626a7567ca76d2a4e78041af115564d863dc7f742d39bd4adb249c29751439a1238d5e5c34e474c7221d4fd6e1
f3f6dde56e Test coinbase category in wallet rpcs (andrewtoth)
e982f0b682 Add all category options to wallet rpc help (andrewtoth)
Pull request description:
The current helptext for `listtransactions`, `listsinceblock` and `gettransaction` only list two of the five possible options for `category`. This incorrectly implies that these are the only two options, and can cause problems if the other three options aren't accounted for. Also, some of the documentation is incorrect when specifying which options are returned for which categories.
This PR updates the helptext for these RPCs and adds a functional regression test for the cases when the other three categories are returned.
Tree-SHA512: 67dd7ff6269a3b0f17f5d1a61b0ae1fb1f3778f05e1c440bfbb9b3a005c9c6d740abcace20f3d597cf2bd6779c494448690f13fab0bd2340f206213bc7890b51
eaf4070e3a Add suppression for InterruptRPC (fRPCRunning) data race (practicalswift)
5e5138a721 travis: Use trap and set -e errtrace (Chun Kuan Lee)
069752b726 build: Enable functional tests in the ThreadSanitizer (TSan) build job (practicalswift)
Pull request description:
Enable functional tests in the ThreadSanitizer (TSan) build job.
This is a follow-up to @MarcoFalke's #14764 which added TSan but for unit tests only.
Tree-SHA512: dcc24d311fa124772c3036b16c2bf94732ece36c3e22b4bb8fe941772e52157ab2b1a90b1880b81079c2eef2d344ca7e1da58324b75dbf82d16204d591ad49fb
faee59103d test: Fix race in mempool_accept (MarcoFalke)
Pull request description:
If we happen to pick the same random coin to spend, there would be mempool conflicts in some runs of the test. Fix that by popping from a static list of coins to spend from.
Tree-SHA512: f6fd37e43d919371aa8bc3a2c93b569f9169961fe702f3641bb63180c3a88f12ca1857e9ed4d3723d5f04ca8ab5ef009a90e679580f36246a10b987620a55bee
fa4b8c90d3 test: add_nodes can only be called once after set_test_params (MarcoFalke)
faa831102a Revert "tests: Support calling add_nodes more than once" (MarcoFalke)
Pull request description:
Writing tests should be straightforward and with little side-effects as possible.
I don't see how this is needed and can not be achieved with `self.num_nodes` (and `self.extra_args` et al.)
Tree-SHA512: 83a67f2cba9d97e21d80847ff405a4633fcb0d5674486efa57ee1813e46efe8709ae0fb462b8339a01ebeca5c4f2d29ecb1807d648b8fd9ee8ce336b08d580a8
4de11a3682 Remove Python 2 import workarounds (practicalswift)
Pull request description:
Remove Python 2 import workarounds.
As noted by @jnewbery in https://github.com/bitcoin/bitcoin/pull/14903#discussion_r241396925:
> This exception handling is a vestige from when github-merge.py supported Python 2 and Python 3. We only support Python 3 now so we should be able to remove it entirely and just import from urllib.request.
Tree-SHA512: e0d21e6299dd62fb669ad95cbd3d19f7c803195fd336621aac72fd10ddc7431d90443831072a2e1eb2fc880d1d88eb7c3e2ead3da59f545f6db07d349af98fb3
98a1846b00 tests: Support calling add_nodes more than once (Steven Roose)
Pull request description:
Ran into this while writing [a multi-chain test for Elements](https://github.com/ElementsProject/elements/pull/458) where I call this method more than once.
Tree-SHA512: f2d698fcb560552aa5d81a4c3fbf40b7269b228b34d85a118291649ef83f8c0a30cd82a28d418237b55893bcecd538046b704e64a4d8a41f2c0aef8033dc83e5
aaaa8eb1ed test: consensus: Check that final transactions are valid (MarcoFalke)
fae3617d79 test: Correctly deserialize without witness (MarcoFalke)
Pull request description:
There is no check that checks that final transactions are valid, i.e. the consensus rules could be changed (accidentally) with none of the tests failing.
Tree-SHA512: 48f4c24bfcc525ddbc1bfe8c37131953b464823428c1f7a278ba6d98b98827b6b84a8eb2b33396bfb5b8cc4012b7cc1cd771637f405ea20beddae001c22aa290
31926ee8cf [test] functional framework: add CScript hex() for Python 3.4 (Sjors Provoost)
74ce326831 [test] Travis: enforce Python 3.4 support in functional tests (Sjors Provoost)
Pull request description:
The minimum supported version of Python is 3.4 according to [dependencies.md](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md). This PR makes the Travis linter use this version in order to catch accidental use of modern syntax.
Tree-SHA512: 71b2c102be72b135a8ba049378d66875760f20a04a657102a399240c5c2b2ddbdfa7d5ab4c0c0242ecc3259e0ee8eb2273f331bc5eb824f4ae4c3cc58aea37ac
This introduces various changes to the importmulti logic:
* Instead of processing input and importing things at the same time, first
process all input data and verify it, so no changes are made in case of
an error.
* Verify that no superfluous information is provided (no keys or scripts
that don't contribute to solvability in particular).
* Add way more sanity checks, by means of descending into all involved
scripts.
Make lint/check-doc.py Python 3.4 compatible.
Also add .python-version for pyenv which will cause tests with too
modern syntax to fail on developer machine rather than on Travis.
ee3b21dccb [tests] Add docstring for wallet_importmulti.py (John Newbery)
fbdba40594 [tests] add test_address method to wallet_import.py (John Newbery)
fd3a02c381 [tests] add test_importmulti method to wallet_import.py (John Newbery)
08a4a0f70f [tests] add get_multisig function to wallet_importmulti.py (John Newbery)
7c99614b40 [tests] add get_key function to wallet_importmulti.py (John Newbery)
e5a8ea8f14 [tests] tidy up imports in wallet_importmulti.py (John Newbery)
cb41ade6b1 [tests] fix flake8 warnings in wallet_importmulti.py (John Newbery)
Pull request description:
https://github.com/bitcoin/bitcoin/pull/14565 needs test coverage. This PR refactors wallet_importmulti.py to the following pattern:
1. Add `get_key()` and `get_multisig()` methods, which generate keys on node0 and return the priv/pubkeys and all scriptPubKey and address variants.
2. Add `test_importmulti()` method, which takes an importmulti request, sends it to node1 and tests against success and error codes/messages.
3. Add `test_address()` method, which takes an address, sends it as a getaddressinfo request to node1 and tests the values returned.
This does not add any specific testing for #14565, but makes it very straightforward to add that testing: `test_importmulti()` can be easily updated to test for returned warnings, and `test_address()` can be called multiple times against the different address variants for a singlesig/multisig.
Tree-SHA512: e0ae9d3436f0b4eec4f6b9bdc0f02aef49c5a16bbac319fd47b2cfcaf01d01780d7b296280e8760686a57fac63275eec09e2959d8aaeceae1b406d8eff768435
0dcac51049 wallet_keypool_topup.py: Test for all keypool address types (Gregory Sanders)
Pull request description:
To protect against regressions if key scanning is changed.
Tree-SHA512: d1c4bb033bafd97203a3f68fb262a501442be947907d67902f0391fbdec39c095196403c7675e602806cc68d7e2d1f552ab339a58346162379978d06dad1c4bb
Calling getblocktemplate without the segwit rule specified is most
likely a client error, since it results in lower fees for the miner.
Prevent this client error by failing getblocktemplate if called without
the segwit rule specified.
d6b3790d1a tests: check readability of cookie file (Chun Kuan Lee)
Pull request description:
This PR would wait until the `.cookie` file is readable
Possible fix no. 5 `PermissionError` in #14446
Tree-SHA512: e7055c7ca26a6eadbbe19e4eef08ffee61cd17de79b30af2f0d090f0ad81ca24815e3c7e034e5e30d47c580bb0b221b3955e9ff2fcec2274fbf7b9232ab0cdc7
5c40e7b91a test: allows test_runner command line to receive parameters for each test (marcoagner)
Pull request description:
Fixes#14791.
Seems to address the asked behaviour in a simple way, but could address more if a more complex behaviour is found to be needed (e.g. call `rpc_bind --ipv4` without the ".py" and have it added).
Tree-SHA512: d7e21dd0ada36dd97dac71196bb97702a92986b181beb3753e37e3294b899fb65129aff5f9a45fe92b06fbf7c74e605ccb87a422f874f7a85d30401f4c2228c8
28479f926f qa: Test bitcond shutdown (João Barbosa)
8d3f46ec39 http: Remove timeout to exit event loop (João Barbosa)
e98a9eede2 http: Remove unnecessary event_base_loopexit call (João Barbosa)
6b13580f4e http: Unlisten sockets after all workers quit (João Barbosa)
18e9685816 http: Send "Connection: close" header if shutdown is requested (João Barbosa)
02e1e4eff6 rpc: Add wait argument to stop (João Barbosa)
Pull request description:
Fixes#11777. Reverts #11006. Replaces #13501.
With this change the HTTP server will exit gracefully, meaning that all requests will finish processing and sending the response, even if this means to wait more than 2 seconds (current time allowed to exit the event loop).
Another small change is that connections are accepted even when the server is stopping, but HTTP requests are rejected. This can be improved later, especially if chunked replies are implemented.
Briefly, before this PR, this is the order or events when a request arrives (RPC `stop`):
1. `bufferevent_disable(..., EV_READ)`
2. `StartShutdown()`
3. `evhttp_del_accept_socket(...)`
4. `ThreadHTTP` terminates (event loop exits) because there are no active or pending events thanks to 1. and 3.
5. client doesn't get the response thanks to 4.
This can be verified by applying
```diff
// Event loop will exit after current HTTP requests have been handled, so
// this reply will get back to the client.
StartShutdown();
+ MilliSleep(2000);
return "Bitcoin server stopping";
}
```
and checking the log output:
```
Received a POST request for / from 127.0.0.1:62443
ThreadRPCServer method=stop user=__cookie__
Interrupting HTTP server
** Exited http event loop
Interrupting HTTP RPC server
Interrupting RPC
tor: Thread interrupt
Shutdown: In progress...
torcontrol thread exit
Stopping HTTP RPC server
addcon thread exit
opencon thread exit
Unregistering HTTP handler for / (exactmatch 1)
Unregistering HTTP handler for /wallet/ (exactmatch 0)
Stopping RPC
RPC stopped.
Stopping HTTP server
Waiting for HTTP worker threads to exit
msghand thread exit
net thread exit
... sleep 2 seconds ...
Waiting for HTTP event thread to exit
Stopped HTTP server
```
For this reason point 3. is moved right after all HTTP workers quit. In that moment HTTP replies are queued in the event loop which keeps spinning util all connections are closed. In order to trigger the server side close with keep alive connections (implicit in HTTP/1.1) the header `Connection: close` is sent if shutdown was requested. This can be tested by
```
bitcoind -regtest
nc localhost 18443
POST / HTTP/1.1
Authorization: Basic ...
Content-Type: application/json
Content-Length: 44
{"jsonrpc": "2.0","method":"stop","id":123}
```
Summing up, this PR:
- removes explicit event loop exit — event loop exits once there are no active or pending events
- changes the moment the listening sockets are removed — explained above
- sends header `Connection: close` on active requests when shutdown was requested which is relevant when it's a persistent connection (default in HTTP 1.1) — libevent is aware of this header and closes the connection gracefully
- removes event loop explicit break after 2 seconds timeout
Tree-SHA512: 4dac1e86abe388697c1e2dedbf31fb36a394cfafe5e64eadbf6ed01d829542785a8c3b91d1ab680d3f03f912d14fc87176428041141441d25dcb6c98a1e069d8
688f665a5e Scripts and tools & Docs: Used #!/usr/bin/env bash instead of obsolete #!/bin/bash, added linting for .sh files shebang and updated the Developer Notes. (vim88)
Pull request description:
As it was discussed in [#13510](https://github.com/bitcoin/bitcoin/pull/13510), it is better to use `#!/usr/bin/env bash` instead of `#!/bin/bash`.
Tree-SHA512: 25f71eb9a6a0cdc91568b5c6863205c5fe095f77a69e633503a2ac7805bd9013af8538e538c0c666ce96a28e3f43ce7a8df5f08d4ff007723bb588d85674f2da
43f9099901 scripted-diff: Run scripted-diff in subshell (Carl Dong)
Pull request description:
scripted-diffs should be run in subshells so that their execution does not
affect the shell variables of commit-script-check. Shell variables are not
unset before evaluating the scripted-diff, so that they might be used in
the subshell. To this end, the variable previously named i is now more
descriptively named commit, this also allows scripted-diffs to use the
commonly used variable i without fear of losing a reference to the
commit.
Tree-SHA512: 0d86c069c2a978ca07d71bcd2b1b273e9bfabfe7e31a50c7b1b860e04f178b81c65814c3a38fb01e50b41a5065b646f0dab5b05d9be71138e72d4baba607e37b
48b37db50 make peertimeout a debug argument, remove error message translation (Zain Iqbal Allarakhia)
8042bbfbf p2p: allow p2ptimeout to be configurable, speed up slow test (Zain Iqbal Allarakhia)
Pull request description:
**Summary:**
1. _Primary_: Adds a `debug_only=true` flag for peertimeout, defaults to 60 sec., the current hard-coded setting.
2. _Secondary_: Drastically speeds up `p2p_timeout.py` test.
3. _Secondary_: Tests that the correct code path is being tested by adding log assertions to the test.
**Rationale:**
- P2P timeout was hard-coded: make it explicitly specified and configurable, instead of a magic number.
- Addresses #13518; `p2p_timeout.py` takes 4 sec. to run instead of 61 sec.
- Makes `p2p_timeout.py` more explicit. Previously, we relied on a comment to inform us of the timeout amount being tested. Now it is specified directly in the test via passing in the new arg; `-peertimeout=3`.
- Opens us up to testing more P2P scenarios; oftentimes slow tests are the reason we don't test.
**Locally verified changes:**
_With Proposed Change (4.7 sec.):_
```
$ time ./test/functional/p2p_timeouts.py
2018-11-19T00:04:19.077000Z TestFramework (INFO): Initializing test directory /tmp/testhja7g2n7
2018-11-19T00:04:23.479000Z TestFramework (INFO): Stopping nodes
2018-11-19T00:04:23.683000Z TestFramework (INFO): Cleaning up /tmp/testhja7g2n7 on exit
2018-11-19T00:04:23.683000Z TestFramework (INFO): Tests successful
real 0m4.743s
```
_Currently on master (62.8 sec.):_
```
$ time ./test/functional/p2p_timeouts.py
2018-11-19T00:06:10.948000Z TestFramework (INFO): Initializing test directory /tmp/test6mo6k21h
2018-11-19T00:07:13.376000Z TestFramework (INFO): Stopping nodes
2018-11-19T00:07:13.631000Z TestFramework (INFO): Cleaning up /tmp/test6mo6k21h on exit
2018-11-19T00:07:13.631000Z TestFramework (INFO): Tests successful
real 1m2.836s
```
_Error message demonstrated for new argument `-peertimeout`:_
```
$ ./bitcoind -peertimeout=-5
...
Error: peertimeout cannot be configured with a negative value.
```
Tree-SHA512: ff7a244ebea54c4059407bf4fb86465714e6a79cef5d2bcaa22cfe831a81761aaf597ba4d5172fc2ec12266f54712216fc41b5d24849e5d9dab39ba6f09e3a2a
2012d4df2 Add CScriptNum decode python implementation in functional suite (Gregory Sanders)
Pull request description:
I needed this for reasons and thought it'd be good to upsteam it.
Tree-SHA512: 6ea89fa2a5f5a7759ba722f2b4ed5cd6423ebfff4e83ac8b8b5c935e6aa479684e626c5f41fa020816d2a9079a99af5564e30808594d5c13e3b51ec9b474926d
c5ed6e73d Move CheckBlock() call to critical section (Hennadii Stepanov)
Pull request description:
This is an alternative to #14803.
Refs:
- #14058
- #14072
- https://github.com/bitcoin/bitcoin/pull/14803#issuecomment-442233211 by @gmaxwell
> It doesn't support multithreaded validation and there are lot of things that prevent that, which is why I was concerned. Why doesn't the lock on the block index or even cs main prevent concurrency here?
- https://github.com/bitcoin/bitcoin/pull/14803#issuecomment-442237566 by @MarcoFalke
Tree-SHA512: 2152e97106e11da5763b2748234ecd2982daadab13a0da04215f4db60af802a44ab5700f32249137d122eb13fc2a02e0f2d561d364607d727d8c6ab879339afb
c1825b9d39 [tests] Add wallet_balance.py (John Newbery)
Pull request description:
Adds a test specifically to test the wallet's getbalance and
getunconfirmedbalance RPCs.
`wallet_basic.py` is too large and should be broken down into more focused test cases.
I wrote `wallet_balance.py` to test the changes in #14602. Offering as a PR in case people think it's more generally useful.
Tree-SHA512: 573ae8faf377df3d87d5112870b40690efb285fc5578fff8acc2ac1a0e4625ae65d3dfa8abbac577c87bec015038f425833783fa09f014f87906e8d098ed30d7
0fb2e69815 CreateTransaction: Assume minimum p2sh-p2wpkh spend size for unknown change (Gregory Sanders)
b06483c96a Remove stale comment in CalculateMaximumSignedInputSize (Gregory Sanders)
Pull request description:
This is triggered anytime a fundraw type call(psbt or legacy) is used with a change output address that the wallet doesn't know how to sign for.
This regression was added in 6a34ff5335 since BnB coin selection actually cares about this.
The fix is to assume the smallest typical spend, a P2SH-P2WPKH, which is calculated using a "prototype" dummy signature flow. Future work could generalize this infrastructure to get estimated sizes of inputs for a variety of types.
I also removed a comment which I believe is stale and misleading.
Tree-SHA512: c7e2be189e524f81a7aa4454ad9370cefba715e3781f1e462c8bab77e4d27540191419029e3ebda11e3744c0703271e479dcd560d05e4d470048d9633e34da16
4aabadbf44 tests: have combine_logs default to most recent test dir (James O'Beirne)
Pull request description:
Have `combine_logs.py` default to the most recent test directory if no argument is provided. This allows you to avoid an annoying copy-paste when iterating on a failing test, since you can do something like
```sh
alias testlogs='./test/functional/combine_logs.py -c | less'
./test/functional/some_test.py # fails
testlogs
```
Tree-SHA512: 919642ab09c314888a23c9491963b35b9da87e60deb740d1d5e816444aa9bdda5e519dc8ca131669f2d563167ef5f5abb14e22f20f47bf8362915ed578181846
fa5cef0f78 bench: Destroy wallet txs instead of leaking their memory (MarcoFalke)
Pull request description:
This should destroy the wallet txs when the benchmark ends to avoid having to hold them when the following benchmarks run.
Tree-SHA512: e2510946e6a47fad3ec5fb28d298df8ddc2e017455fcff777fa7bbc12d801c08739db6a7a7289509aaa881ccdc59dfff9bcb6772b48db2c457d3787081a46c06
29aeed1734 Bugfix: test/functional/mempool_accept: Ensure oversize transaction is actually oversize (Luke Dashjr)
Pull request description:
Simply integer dividing results in an acceptable size if the limit isn't an exact multiple of the input size.
Use math.ceil to ensure the transaction is always oversize.
(This issue can be triggered by changing the address style used.)
Tree-SHA512: e45062b0e8a3e9cb08e9dac5275b68d86e4377b460f1b3b995944090a055b0542a6986826312ec0e223369838094e42e20d8614b5c2bab9975b9a6f749295b21
109699dd33 Add release notes (Pieter Wuille)
b65326b562 Add matching descriptors to scantxoutset output + tests (Pieter Wuille)
16203d5df7 Add descriptors to listunspent and getaddressinfo + tests (Pieter Wuille)
9b2a25b13f Add tests for InferDescriptor and Descriptor::IsSolvable (Pieter Wuille)
225bf3e3b0 Add Descriptor::IsSolvable() to distinguish addr/raw from others (Pieter Wuille)
4d78bd93b5 Add support for inferring descriptors from scripts (Pieter Wuille)
Pull request description:
This PR adds functionality to convert a script to a descriptor, given a `SigningProvider` with the relevant information about public keys and redeemscripts/witnessscripts.
The feature is exposed in `listunspent`, `getaddressinfo`, and `scantxoutset` whenever these calls are applied to solvable outputs/addresses.
This is not very useful on its own, though when we add RPCs to import descriptors, or sign PSBTs using descriptors, these strings become a compact and standalone way of conveying everything necessary to sign an output (excluding private keys).
Unit tests and rudimentary RPC tests are included (more relevant tests can be added once RPCs support descriptors).
Fixes#14503.
Tree-SHA512: cb36b84a3e0200375b7e06a98c7e750cfaf95cf5de132cad59f7ec3cbd201f739427de0dc108f515be7aca203652089fbf5f24ed283d4553bddf23a3224ab31f
fa739d4bd7 qa: Add wallet_encryption error tests (MarcoFalke)
Pull request description:
The errors for empty passphrases are the help text of the RPC call, which is not very specific. Replace that with proper RPC errors and test them.
Tree-SHA512: 3137e0f8f2e42a1f8ab1eeb57c99052557725f6f85139ff48c24acc8f3cf4087802de5216f3ce97375b291d21bddb7cd1379a6f280166136a306a0c9663bbd42
Simply integer dividing results in an acceptable size if the limit isn't an exact multiple of the input size.
Use math.ceil to ensure the transaction is always oversize.
fa7d36b8e7 test: Move UBSAN suppressions to test/sanitizer_suppressions/ubsan (MarcoFalke)
fa36d4e456 travis: --disable-hardening for xenial thread sanitizer (MarcoFalke)
89bf196c88 travis: Run thread sanitizer (MarcoFalke)
Pull request description:
On unit tests only for now. Disabled for the gui unit tests and all functional tests.
Tree-SHA512: 56f7d3b44e7cb68c76a2dc5abd85658955b1c2188932e988667c5a1cbcdd6be995d37bb949d62c6eb08a4aebfc43ff0370b7da1719d4e4f322a3495c1941a5e0
fa5e0452e8 rpc: Documentation fixups (MarcoFalke)
fa91e8eda5 Use RPCHelpMan for all RPCs (MarcoFalke)
fa520e72f7 lint: Must use RPCHelpMan to generate the RPC docs (MarcoFalke)
Pull request description:
The resulting documentation should not change unless the type in the oneline-summary was previously incorrect. (E.g. string vs bool)
Tree-SHA512: 4ff355b6a53178f02781e97a7aca7ee1d0d97ff348b6bf5a01caa1c96904ee33c704465fae54c2cd7445097427fd04c71ad3779bb7a7ed886055ef36c1b5a1d0
3d2c7d6f94 Add regtest for JSON-RPC batch calls. (Daniel Kraft)
Pull request description:
This adds a new regtest file `interface_rpc.py`, containing a test for batch JSON-RPC requests. Those were previously not tested at all. Tests for basic requests are not really necessary, as those are used anyway in lots of other regtests.
The existing `interface_http.py` file is more about the underlying HTTP connection, so adding a new interface file for the JSON-RPC specific things makes sense.
Tree-SHA512: 7c7576004c8474e23c98f4bf25fb655328ba6bb73ea06744ebee1c0ffbb26bc132e621ae52955d51dab0803b322f8d711667626a777ac9b26003339c2484502f
This adds a new regtest file 'interface_rpc.py', containing a test for
batch JSON-RPC requests. Those were previously not tested at all. Tests
for basic requests are not really necessary, as those are used anyway
in lots of other regtests.
The existing interface_http.py file is more about the underlying HTTP
connection, so adding a new interface file for the JSON-RPC specific
things makes sense.
3fb09b9889 Warn unrecognized sections in the config file (Akio Nakamura)
Pull request description:
This PR intends to resolve#14702.
In the config file, sections are specified by square bracket pair "[]"$,
or included in the option name itself which separated by a period"(.)".
Typicaly, [testnet] is not a correct section name and specified options
in that section are ignored but user cannot recognize what is happen.
So, add some log-warning messages if unrecognized section names are
present in the config file after checking section only args.
note: Currentry, followings are out of scope of this PR.
1) Empty section name or option name can describe.
e.g. [] , .a=b, =c
2) Multiple period characters can exist in the section name and option name.
e.g. [c.d.e], [..], f.g.h.i=j, ..=k
Tree-SHA512: 2cea02a0525feb40320613989a75cd7b7b1bd12158d5e6f3174ca77e6a25bb84425dd8812f62483df9fc482045c7b5402d69bc714430518b1847d055a2dc304b
fa7da0617c qa: Check specific reject reasons in feature_block (MarcoFalke)
Pull request description:
There are some consensus checks that are essentially turned off because we never send the block, but only the header. It happens that the header was sufficient to determine the invalidity of the block according to our consensus rules in those cases. Fix that by forcing the full block on the node unsolicited.
Tree-SHA512: a5534318370367ea8de07d853de7e845c8f5637cd6d5457e932a9555af26cc212625e443c00c93586d556cc770f301248e7cabd68131a37791ae91706e7e40b2
6be7d14d24 Properly generate salt in rpcauth.py, update tests (Carl Dong)
Pull request description:
Previously, when iterating over bytes of the generated salt to construct
a hex string, only one character would be outputted when the byte is
less than 0x10. Meaning that for a 16 byte salt, the hex string might be
less than 32 characters and collisions would occur.
Tree-SHA512: 7038ecbbac846cd1851112396acd8a04475685f5b6f786e4e7316acba4a56cc711c275b7f52f0f2b6bc6cfdc0c0d9d39c3afeb2c0aff3a30fde516bf642fdf9f
591203149f wallet: Create IsDatabaseLoaded function (Chun Kuan Lee)
15c93f075a wallet: Add trailing wallet.dat when detecting duplicate wallet if it's a directory. (Chun Kuan Lee)
c456fbd8df Refactor: Move m_db pointers into BerkeleyDatabase (Russell Yanofsky)
Pull request description:
Fix#14538
Fix crash attempting to load the same wallet with different path strings that resolve to the same absolute path. The primary check which prevents loading the same wallet twice is:
6b8d0a2164/src/wallet/db.cpp (L44)
But this check is skipped if both wallet paths resolve to the same absolute path, due to caching here:
6b8d0a2164/src/wallet/db.cpp (L467)
Meanwhile a secondary check for duplicate wallets is not reliable because it based on a literal comparison, instead of comparison using absolute paths:
6b8d0a2164/src/wallet/wallet.cpp (L3853)
This PR fixes the latter check to compare the absolute path of a new wallet being loaded to absolute paths of wallets already loaded, so there should no longer be any way to load the same wallet more than once.
Tree-SHA512: 2fa01811c160b57be3b76c6b4983556a04bbce71a3f8202429987ec020664a062e897deedcd9248bc04e9baaa2fc7b464e2595dcaeff2af0818387bf1fcdbf6f
In the config file, sections are specified by square bracket pair "[]"$,
or included in the option name itself which separated by a period"(.)".
Typicaly, [testnet] is not a correct section name and specified options
in that section are ignored but user cannot recognize what is happen.
So, add some log/stderr-warning messages if unrecognized section names
are present in the config file after checking section only args.
Previously, when iterating over bytes of the generated salt to construct
a hex string, only one character would be outputted when the byte is
less than 0x10. Meaning that for a 16 byte salt, the hex string might be
less than 32 characters and collisions would occur.
fa0815c300 rpc: Correctly name arguments (Jon Layton)
Pull request description:
Consistently use the same name to describe arguments in the documentation and add a test that uses the name.
By splitting it up, the changes are easier to potentially backport and also make review easier when we switch to `RPCHelpMan`.
The tests should pass with or without the changes in `src`.
Partly stolen from #14459 (More RPC help description fixes by ch4ot1c)
Tree-SHA512: 1072992b1e93ac41006613523e54a0a8004f529fcb101eb9d74d91474abb0945a5a7539f249905151b904b87448f9efc0cacbd9e052fbe2ea9111e62f3e7249c
7afddfa8ce importmulti: Don't add internal addresses to address book (Gregory Sanders)
Pull request description:
Currently anything imported with `internal` will not be treated as change since checking the address book is a primary test of this.
Added basic tests of all combinations of arguments and change identification.
Resolves https://github.com/bitcoin/bitcoin/issues/14662
Tree-SHA512: a1f08dc624a3fadee93cc5392d50c4796b0c5eedf38e295382f71570f2066d9e978ed6e3962084b902989863fe1273a8642d8fdb094a266d69de10622a4176b0
fa5a6ce102 qa: Raise ci test_runner timeout to 40 mins (MarcoFalke)
fa3df025e1 travis: Avoid timeout on verify-commits check (MarcoFalke)
Pull request description:
The verify-commits check is too expensive to run in full (calculate Tree-SHA512 and clean-merge for every single merge commit in history) every day (the cron job runs every ~24h). Since the cron job is running every day, it is also redundant to redo most of the work on the next day.
So, only check two days worth of commits and assume that travis checked the Tree-SHA512 and clean-merge for all other commits already. The script will still check all the signatures, since the check-result for them depends on external inputs such as current time or the public keys we got from the server.
[Note that travis is not meant to do the verification for anyone or is meant to be trusted in any way. This check only serves as a belt-and-suspender to notify maintainers in case of a technical issue or script malfunction. But since the script is timing out for months now, its purpose is diminished right now.]
Tree-SHA512: 336c5cbcc03cdf50be96cd61412471be9078d862da8ba2054f337441e062a6067c95fbbd03912e3de6a116f3caa75fd3f01a04864d34aae1489faa3154572815
fa21568208 qa: Avoid race in p2p_invalid_block by waiting for the block request (MarcoFalke)
6c787d340c tests: Make feature_block pass on centos (MarcoFalke)
Pull request description:
This hopefully fixes#14661, which I believe is caused by a race in `send_blocks_and_test`. By setting `request_block=False` we only effectively check `node.getbestblockhash() != blocks[-1].hash` before returning and checking the debug.log. By setting `request_block=True` (the default) we make sure that we send the block, then sync with a ping before asserting on the debug.log.
Even if this patch doesn't fix the issue, it is good cleanup: There is no reason to not wait for the blocks to be requested, since in all these cases the header gives no indication that the block is consensus invalid. So this patch makes the test also a bit stricter and more useful.
Unrelated to this, I also include a fix that makes the tests pass on latest CentOS.
Tree-SHA512: c7abee3b7dc790a8af6c289159a7751bd962f6fa16c1537e7e21a0a0ef05b9596d1f4eb75319614603c05cb803e021314fa3596508ba443edd03046b25527e0f
a6b5ec18f rpc: creates possibility to preserve labels on importprivkey (marcoagner)
Pull request description:
Closes#13087.
As discussed in the issue, this is a feature request instead of a bug report since the behaviour was as intended (i.e. label with default: `''`). With this, the old behaviour is kept while the possibility to achieve the preservation of labels, as expected in the open issue, is added.
Tree-SHA512: b33be50e1e7f62f7ddfae953177ba0926e2d848961f9fac7501c2b513322c0cb95787745d07d137488267bad1104ecfdbe800c6747f94162eb07c976835c1386
88a79cb436 fix converttopsbt permitsigdata arg, add basic test (Gregory Sanders)
Pull request description:
The final check for extraneous sigdata has a flipped boolean, resulting in incorrect behavior.
Resolves https://github.com/bitcoin/bitcoin/issues/14355
Tree-SHA512: 5157a74b8ddebd7d836fba96765c4d7ed15a73d4289817353d3566a0f6803bd4bbc3f936735c517c7a83a6cbdb4052b9c61d23f6cc4ad00a6077278cd51adbd4
0385109444 Add test for rpcpassword hash error (MeshCollider)
13fe258e91 Error if rpcpassword in conf contains a hash character (MeshCollider)
Pull request description:
Fixes#13143 now #13482 was merged
Tree-SHA512: e7d00c8df1657f6b8d0eee1e06b9ce2b1b0a2de487377699382c1b057836e1571dac313ca878b5877c862f0461ba789a50b239d2a9f34accd8a6321f126e3d2a
fa9ed38d57 test_node: get_mem_rss fixups (MarcoFalke)
Pull request description:
Follow up to #14522:
* Fix math (Memory usage increase relative to previous memory usage, not final memory usage)
* remove `shell=True`
* assert that the node is running
* Make it work on BSD-like systems
Tree-SHA512: fc1b4f88173914b6cb6373655cffd781044a0c146339e3fa90da03b197faa20954567a77335965b857d29d27f32661698b6a0340f0c616f643b8c4510cd360c2
6b8d86ddb8 Require a public key to be retrieved when signing a P2PKH input (Andrew Chow)
Pull request description:
If we do not have the public key for a P2PKH input, we should not continue to attempt to sign for it.
This fixes a problem where a PSBT with a P2PKH output would include invalid BIP 32 derivation paths that are missing the public key.
Tree-SHA512: 850d5e74c06833da937d5bf0348bd134180be7167b6f9b9cecbf09f75e3543fbad60d0abbc0b9afdfa51ce165aa36168849f24a7c5abf1e75f37ce8f9a13d127
e13fea975d Add regression test for PSBT signing bug #14473 (Glenn Willen)
565500508a Refactor PSBTInput signing to enforce invariant (Glenn Willen)
0f5bda2bd9 Simplify arguments to SignPSBTInput (Glenn Willen)
53e6fffb8f Add bool PSBTInputSigned (Glenn Willen)
65166d4cf8 New PartiallySignedTransaction constructor from CTransction (Glenn Willen)
4f3f5cb4b1 Remove redundant txConst parameter to FillPSBT (Glenn Willen)
fe5d22bc67 More concise conversion of CDataStream to string (Glenn Willen)
Pull request description:
As discussed in the comments on #14473, I think that bug was caused primarily by failure to adhere to the invariant that a PSBTInput always has exactly one of the two utxo fields present -- an invariant that is already enforced by PSBTInput::IsSane, but which we were temporarily suspending during signing.
This refactor repairs the invariant, also fixing the bug. It also simplifies some other code, and removes redundant parameters from some related functions.
fixes#14473
Tree-SHA512: cbad3428175e30f9b7bac3f600668dd1a8f9acde16b915d27a940a2fa6d5149d4fbe236d5808fd590fb20a032274c99e8cac34bef17f79a53fdf69a5948c0fd0
Builds on travis are failing because the test node isn't
able to drop all the bad messages sent within the given
timeout. Reduce the number of bad messages we're sending
and increase the timeout to avoid failures on travis.
5a05aa2db2 Add metavar to match var name in help text + Change wording for better readability (Martin Erlandsson)
Pull request description:
The help text given by `test/functional/test_runner.py -h` refers to the value `n`, which is defined as `COMBINEDLOGSLEN` in the list of commands.
To make the help text consistent, this PR changes the display name `COMBINEDLOGSLEN` to `n` by setting the argparse [`metavar`](https://docs.python.org/3/library/argparse.html#metavar) attribute. (`metavar` only changes the _displayed_ name)
Alternatively: Do the opposite and change the help text to use `COMBINEDLOGSLEN`.
---
Before PR:
```
➜ bitcoin > test/functional/test_runner.py -h | grep -A 1 combinedlogslen
--combinedlogslen COMBINEDLOGSLEN, -c COMBINEDLOGSLEN
print a combined log (of length n lines) from all test nodes and test framework to the console on failure.
```
After PR:
```
➜ bitcoin > test/functional/test_runner.py -h | grep -A 1 combinedlogslen
--combinedlogslen n, -c n
print a combined log (of length n lines) from all test nodes and test frameworks to the console on failure.
```
---
Also, fixed pluralization typo.
Tree-SHA512: a1124a4976d29fae1e8ecd7fa2ac523b7f05d541c611166532f44692995691a96faf797fa71582d78634f328b500cbee49c6ef296c8f1a898a57c050cc4e721d
bbbbb3f885 qa: Add test to ensure node can generate all help texts at runtime (MarcoFalke)
Pull request description:
This might increase coverage, but more importantly this checks that the node doesn't crash when generating the help. (Right now the help is a static string, but in the future it might be generated at runtime)
Tree-SHA512: 0226e7c65f8a1a6fdc96c07dcf491d90559bc2355c92e9da9b1f174b09733fc349269e71da6d792f954de563a1e57c848471813eabae1a40b849a0d989520a0d
d20a9fa13d tests: add tests for invalid P2P messages (James O'Beirne)
62f94d39f8 tests: add P2PConnection.send_raw_message (James O'Beirne)
5aa31f6ef2 tests: add utility to assert node memory usage hasn't increased (James O'Beirne)
Pull request description:
- Adds `p2p_invalid_messages.py`: tests based on behavior for dealing with invalid and malformed P2P messages. Includes a test verifying that we can't DoS a node by spamming it with large invalid messages.
- Adds `TestNode.assert_memory_usage_stable`: a context manager that allows us to ensure memory usage doesn't significantly increase on a node during some test.
- Adds `P2PConnection.send_raw_message`: which allows us to construct and send messages with tweaked headers.
Tree-SHA512: 720a4894c1e6d8f1551b2ae710e5b06c9e4f281524623957cb01599be9afea82671dc26d6152281de0acb87720f0c53b61e2b27d40434d30e525dd9e31fa671f
086fc83571 Tests: Fix a comment (fridokus)
Pull request description:
Fix a comment that was false
Tree-SHA512: 945aa38229545e026e18c3abf53a4fbe6ec36413ce690fff7a1dd89b6e102d2b574524092e0ddf06cace82f3c040c59221b9b942be1203525814d2fbd50aaa0b
a4edb168b6 ZMQ: add options to configure outbound message high water mark, aka SNDHWM (mruddy)
Pull request description:
ZMQ: add options to configure outbound message high water mark, aka SNDHWM
This is my attempt at https://github.com/bitcoin/bitcoin/pull/13315
Tree-SHA512: a4cc3bcf179776899261a97c8c4f31f35d1d8950fd71a09a79c5c064879b38e600b26824c89c4091d941502ed5b0255390882f7d44baf9e6dc49d685a86e8edb
65f3672f3b wallet: Refactor to use WalletLocation (João Barbosa)
01a4c095c8 wallet: Add WalletLocation utility class (João Barbosa)
Pull request description:
Advantages of this change:
- avoid resolving wallet absolute path and name repetitively and in multiple places;
- avoid calling `GetWalletDir` in multiple places;
- extract these details from the actual wallet implementation.
The `WalletLocation` class can be a way to represent a wallet not yet loaded that exists in the wallet directory.
Tree-SHA512: 71ec09786e038499710e7acafe92d66ab9883fc894964e267443ae9c10a6872a10995c3987a169c436a4e793dae96b28fb97bd7f78483c4b72ac930fa23f8686
2068f089c8 scripted-diff: Move util files to separate directory. (Jim Posen)
Pull request description:
As discussed [here](https://github.com/bitcoin/bitcoin/pull/14074#issuecomment-429090584), this establishes a `util/` directory to introduce more organizational structure and have a clear place for new util files. It's really not scary to review, it's just one big scripted diff.
Tree-SHA512: 39cf15480d7d35e987b6088d52a857a2d5b1802e36c6b815eb42718d80cd95e669757af9bcc7c04426cd8523662cb1050b8da1e2377d3730672820ed298b894b
14a06525b2 tests: add test for 'getaddressinfo' RPC result 'ischange' field (whythat)
93d1aa9abc rpcwallet: add 'ischange' field to 'getaddressinfo' response (whythat)
Pull request description:
Implementation of proposal in #14396.
This introduces `CWallet::IsChange(CScript&)` method and replaces original `CWallet::IsChange(CTxOut&)` method with overloaded version that delegates to the new method with *txout*'s `scriptPubKey`. In this way `TODO` note from the original method can still be addressed in a single place.
Tree-SHA512: ef5dbc82d76b4b9b2fa6a70abc3385a677c55021f79e187ee2f392ee32bc6b406191f4129acae5c17b0206e72b6712e7e0cad574a4bbd966871c2e656c45e041
fa43626611 test_runner: Remove travis specific code (MarcoFalke)
Pull request description:
The tests are no longer run on travis, but in a docker, developer machines or a windows vm.
The code was essentially dead for months now. Fix that by explicitly passing in `--ci` to the test runner on our docker and appveyor windows vm.
Tree-SHA512: 5d48693c03e8eb27536658ccf9ba738fe93a72abd4b72c80caac084b5b2cdffa77a1031a671eeefe70b71d63500f55917803d4be54d01849722afdccb700a9e6
3fd7e76f6d [tests] Move deterministic address import to setup_nodes (John Newbery)
Pull request description:
This requires a small changes to a few tests, but means that
deterministic addresses will always be imported (unless setup_nodes
behaviour is explicitly overridden).
Tidies up the way we import deterministic addresses, requested in review comment here: https://github.com/bitcoin/bitcoin/pull/14468#discussion_r225594586.
Tree-SHA512: 2b32edf500e286c463398487ab1153116a1dc90f64a53614716373311abdc83d8a251fdd8f42d1146b56e308664deaf62952113f66e98bc37f23968096d1a961
Adds a utility to get resident set size memory usage for a test
node and a context manager that allows assertions based upon
maximum memory use increase.
4fb3388db9 check that a separator is found for psbt inputs, outputs, and global map (Andrew Chow)
Pull request description:
Currently it doesn't make sure that a separator was found so PSBTs missing a trailing separator would still pass. This fixes that and adds a test case for it.
It really only makes sense to check for the separator for the output maps as if an input or global map was missing a separator, the fields following it would be interpreted as belonging to the previous input or global map. However I have added the check for those two anyways to be consistent.
Tree-SHA512: 50c0c08e201ba02494b369a4d36ddb73e6634eb5a4e4e201c4ef38fd2dbeea2c642b8a04d50c91615da61ecbfade37309e47431368f4b1064539c42015766b50
This requires a small changes to a few tests, but means that
deterministic addresses will always be imported (unless setup_nodes
behaviour is explicitly overridden).
