Commit graph

227 commits

Author SHA1 Message Date
Wladimir J. van der Laan
a88bd3186d
Merge #14670: http: Fix HTTP server shutdown
28479f926f qa: Test bitcond shutdown (João Barbosa)
8d3f46ec39 http: Remove timeout to exit event loop (João Barbosa)
e98a9eede2 http: Remove unnecessary event_base_loopexit call (João Barbosa)
6b13580f4e http: Unlisten sockets after all workers quit (João Barbosa)
18e9685816 http: Send "Connection: close" header if shutdown is requested (João Barbosa)
02e1e4eff6 rpc: Add wait argument to stop (João Barbosa)

Pull request description:

  Fixes #11777. Reverts #11006. Replaces #13501.

  With this change the HTTP server will exit gracefully, meaning that all requests will finish processing and sending the response, even if this means to wait more than 2 seconds (current time allowed to exit the event loop).

  Another small change is that connections are accepted even when the server is stopping, but HTTP requests are rejected. This can be improved later, especially if chunked replies are implemented.

  Briefly, before this PR, this is the order or events when a request arrives (RPC `stop`):
   1. `bufferevent_disable(..., EV_READ)`
   2. `StartShutdown()`
   3. `evhttp_del_accept_socket(...)`
   4. `ThreadHTTP` terminates (event loop exits) because there are no active or pending events thanks to 1. and 3.
   5. client doesn't get the response thanks to 4.

  This can be verified by applying
  ```diff
       // Event loop will exit after current HTTP requests have been handled, so
       // this reply will get back to the client.
       StartShutdown();
  +    MilliSleep(2000);
       return "Bitcoin server stopping";
   }
  ```
  and checking the log output:
  ```
      Received a POST request for / from 127.0.0.1:62443
      ThreadRPCServer method=stop user=__cookie__
      Interrupting HTTP server
  **  Exited http event loop
      Interrupting HTTP RPC server
      Interrupting RPC
      tor: Thread interrupt
      Shutdown: In progress...
      torcontrol thread exit
      Stopping HTTP RPC server
      addcon thread exit
      opencon thread exit
      Unregistering HTTP handler for / (exactmatch 1)
      Unregistering HTTP handler for /wallet/ (exactmatch 0)
      Stopping RPC
      RPC stopped.
      Stopping HTTP server
      Waiting for HTTP worker threads to exit
      msghand thread exit
      net thread exit

      ... sleep 2 seconds ...

      Waiting for HTTP event thread to exit
      Stopped HTTP server
  ```

  For this reason point 3. is moved right after all HTTP workers quit. In that moment HTTP replies are queued in the event loop which keeps spinning util all connections are closed. In order to trigger the server side close with keep alive connections (implicit in HTTP/1.1) the header `Connection: close` is sent if shutdown was requested. This can be tested by
  ```
  bitcoind -regtest
  nc localhost 18443
  POST / HTTP/1.1
  Authorization: Basic ...
  Content-Type: application/json
  Content-Length: 44

  {"jsonrpc": "2.0","method":"stop","id":123}
  ```

  Summing up, this PR:
   - removes explicit event loop exit — event loop exits once there are no active or pending events
   - changes the moment the listening sockets are removed — explained above
   - sends header `Connection: close` on active requests when shutdown was requested which is relevant when it's a persistent connection (default in HTTP 1.1) — libevent is aware of this header and closes the connection gracefully
   - removes event loop explicit break after 2 seconds timeout

Tree-SHA512: 4dac1e86abe388697c1e2dedbf31fb36a394cfafe5e64eadbf6ed01d829542785a8c3b91d1ab680d3f03f912d14fc87176428041141441d25dcb6c98a1e069d8
2018-12-06 17:43:07 +01:00
andrewtoth
f3f6dde56e Test coinbase category in wallet rpcs 2018-12-01 13:00:38 -05:00
John Newbery
c1825b9d39 [tests] Add wallet_balance.py
Adds a test specifically to test the wallet's getbalance and
getunconfirmedbalance RPCs.
2018-11-30 10:28:01 -05:00
marcoagner
5c40e7b91a
test: allows test_runner command line to receive parameters for each test 2018-11-23 17:02:24 +00:00
João Barbosa
28479f926f qa: Test bitcond shutdown 2018-11-23 16:54:22 +00:00
Daniel Kraft
3d2c7d6f94 Add regtest for JSON-RPC batch calls.
This adds a new regtest file 'interface_rpc.py', containing a test for
batch JSON-RPC requests.  Those were previously not tested at all.  Tests
for basic requests are not really necessary, as those are used anyway
in lots of other regtests.

The existing interface_http.py file is more about the underlying HTTP
connection, so adding a new interface file for the JSON-RPC specific
things makes sense.
2018-11-21 19:43:06 +01:00
MarcoFalke
e0c6bb7823
Merge #14705: travis: Avoid timeout on verify-commits check
fa5a6ce102 qa: Raise ci test_runner timeout to 40 mins (MarcoFalke)
fa3df025e1 travis: Avoid timeout on verify-commits check (MarcoFalke)

Pull request description:

  The verify-commits check is too expensive to run in full (calculate Tree-SHA512 and clean-merge for every single merge commit in history) every day (the cron job runs every ~24h). Since the cron job is running every day, it is also redundant to redo most of the work on the next day.

  So, only check two days worth of commits and assume that travis checked the Tree-SHA512 and clean-merge for all other commits already. The script will still check all the signatures, since the check-result for them depends on external inputs such as current time or the public keys we got from the server.

  [Note that travis is not meant to do the verification for anyone or is meant to be trusted in any way. This check only serves as a belt-and-suspender to notify maintainers in case of a technical issue or script malfunction. But since the script is timing out for months now, its purpose is diminished right now.]

Tree-SHA512: 336c5cbcc03cdf50be96cd61412471be9078d862da8ba2054f337441e062a6067c95fbbd03912e3de6a116f3caa75fd3f01a04864d34aae1489faa3154572815
2018-11-13 12:25:25 -05:00
Jonas Schnelli
b60f4e3f09
Merge #13381: RPC: creates possibility to preserve labels on importprivkey
a6b5ec18f rpc: creates possibility to preserve labels on importprivkey (marcoagner)

Pull request description:

  Closes #13087.

  As discussed in the issue, this is a feature request instead of a bug report since the behaviour was as intended (i.e. label with default: `''`). With this, the old behaviour is kept while the possibility to achieve the preservation of labels, as expected in the open issue, is added.

Tree-SHA512: b33be50e1e7f62f7ddfae953177ba0926e2d848961f9fac7501c2b513322c0cb95787745d07d137488267bad1104ecfdbe800c6747f94162eb07c976835c1386
2018-11-13 14:52:01 +07:00
MarcoFalke
fa5a6ce102
qa: Raise ci test_runner timeout to 40 mins 2018-11-10 13:12:40 -05:00
practicalswift
590a57fdec tests: Remove unused testing code 2018-11-07 18:07:49 +01:00
MarcoFalke
6af27b8157
Merge #14619: tests: Fix value display name in test_runner help text
5a05aa2db2 Add metavar to match var name in help text + Change wording for better readability (Martin Erlandsson)

Pull request description:

  The help text given by `test/functional/test_runner.py -h` refers to the value `n`, which is defined as `COMBINEDLOGSLEN` in the list of commands.

  To make the help text consistent, this PR changes the display name `COMBINEDLOGSLEN` to `n` by setting the argparse [`metavar`](https://docs.python.org/3/library/argparse.html#metavar) attribute. (`metavar` only changes the _displayed_ name)

  Alternatively: Do the opposite and change the help text to use `COMBINEDLOGSLEN`.

  ---

  Before PR:
  ```
  ➜  bitcoin > test/functional/test_runner.py -h | grep -A 1 combinedlogslen
    --combinedlogslen COMBINEDLOGSLEN, -c COMBINEDLOGSLEN
                          print a combined log (of length n lines) from all test nodes and test framework to the console on failure.
  ```

  After PR:
  ```
  ➜  bitcoin > test/functional/test_runner.py -h | grep -A 1 combinedlogslen
    --combinedlogslen n, -c n
                          print a combined log (of length n lines) from all test nodes and test frameworks to the console on failure.
  ```
  ---
  Also, fixed pluralization typo.

Tree-SHA512: a1124a4976d29fae1e8ecd7fa2ac523b7f05d541c611166532f44692995691a96faf797fa71582d78634f328b500cbee49c6ef296c8f1a898a57c050cc4e721d
2018-11-06 10:44:40 -05:00
Wladimir J. van der Laan
024816d6cf
Merge #14522: tests: add invalid P2P message tests
d20a9fa13d tests: add tests for invalid P2P messages (James O'Beirne)
62f94d39f8 tests: add P2PConnection.send_raw_message (James O'Beirne)
5aa31f6ef2 tests: add utility to assert node memory usage hasn't increased (James O'Beirne)

Pull request description:

  - Adds `p2p_invalid_messages.py`: tests based on behavior for dealing with invalid and malformed P2P messages. Includes a test verifying that we can't DoS a node by spamming it with large invalid messages.
  - Adds `TestNode.assert_memory_usage_stable`: a context manager that allows us to ensure memory usage doesn't significantly increase on a node during some test.
  - Adds `P2PConnection.send_raw_message`: which allows us to construct and send messages with tweaked headers.

Tree-SHA512: 720a4894c1e6d8f1551b2ae710e5b06c9e4f281524623957cb01599be9afea82671dc26d6152281de0acb87720f0c53b61e2b27d40434d30e525dd9e31fa671f
2018-11-06 11:48:14 +01:00
Martin Erlandsson
5a05aa2db2 Add metavar to match var name in help text + Change wording for better readability 2018-11-06 07:54:52 +01:00
MarcoFalke
fa43626611
test_runner: Remove travis specific code 2018-11-01 18:03:26 -04:00
James O'Beirne
d20a9fa13d tests: add tests for invalid P2P messages
E.g., ensure that we can't DoS a node by sending it a bunch of large,
unrecognized messages.
2018-11-01 14:52:49 -04:00
Chun Kuan Lee
4bd125fff0 tests: Print dots by default 2018-10-30 21:19:43 +08:00
MarcoFalke
fa78a2fc67
[tests] Test that nodes respond to getdata with notfound
If a node has not announced a tx at all, then it should respond to
getdata messages for that tx with notfound, to avoid leaking tx
origination privacy.
2018-10-26 07:45:12 -04:00
MarcoFalke
6241eb3224
Merge #14504: tests: show the progress of functional tests
96c509e4d0 show the progress of functional test (Isidoro Ghezzi)

Pull request description:

  example: (added the progress index `n/m`)
  ```
  1/107 - wallet_hd.py passed, Duration: 27 s
  .........................................................................................
  2/107 - mining_getblocktemplate_longpoll.py passed, Duration: 72 s
  ..................................................................
  3/107 - feature_maxuploadtarget.py passed, Duration: 78 s
  ```

Tree-SHA512: 17b840048222e2c3676a92041b491521fee3b86049b2f2467a225aece40717732341801872d9867fcb7260e904e322c7184b76fca16d2dc687aa75dd741484ad
2018-10-24 11:14:09 -04:00
Isidoro Ghezzi
96c509e4d0
show the progress of functional test
example (added the progress index `n/m`)
```
1/107 - wallet_hd.py passed, Duration: 27 s
.........................................................................................
2/107 - mining_getblocktemplate_longpoll.py passed, Duration: 72 s
..................................................................
3/107 - feature_maxuploadtarget.py passed, Duration: 78 s
```

- clear dots line
```
$ test/functional/test_runner.py -t can_trash
Temporary test directory at can_trash/test_runner_₿_🏃_20181018_220600
1/105 - wallet_hd.py passed, Duration: 21 s
2/105 - mining_getblocktemplate_longpoll.py passed, Duration: 71 s
3/105 - feature_maxuploadtarget.py passed, Duration: 68 s
..................
```

- don't print the `dot` progressive if `--quiet`

- done_str
- nothing commit to check again travis tests
2018-10-22 01:12:42 +02:00
Chun Kuan Lee
369244f654 utils: Fix broken Windows filelock 2018-10-19 02:29:25 +08:00
John Newbery
42a995ae48 [tests] Remove rpc_zmq.py
rpc_zmq.py is racy and fails intermittently. Remove that test file and
move the getzmqnotifications RPC test into interface_zmq.py
2018-10-07 09:32:50 +09:00
marcoagner
a6b5ec18ff
rpc: creates possibility to preserve labels on importprivkey
- changes importprivkey behavior to overwrite existent label if one
is passed and keep existing ones if no label is passed

- tests behavior of importprivkey on existing address labels and
different same key destination
2018-10-05 13:33:21 +01:00
Chun Kuan Lee
c7b3e487f2 tests: exclude all tests with difference parameters 2018-09-26 04:25:31 +08:00
Chun Kuan Lee
2148c36b6e tests: Make it possible to run functional tests on Windows 2018-09-17 09:56:49 +08:00
Chun Kuan Lee
c0d947d725 tests: Reorder tests and move most of extended tests up to normal tests 2018-09-16 08:06:11 +08:00
MarcoFalke
fac9539836
qa: Run all tests even if wallet is not compiled 2018-09-10 17:53:21 -04:00
João Barbosa
6af6d9b23d test: Add tests for RPC help 2018-08-23 01:37:43 +01:00
MarcoFalke
a04888a075
Merge #13915: [qa] Add test for max number of entries in locator
fa85c985ed qa: Add p2p_invalid_locator test (MarcoFalke)

Pull request description:

  Should not be merged *before* #13907

Tree-SHA512: a67ca407854c421ed20a184d0b0dc90085aed3e3431d9652a107fa3022244767e67f67e50449b7e95721f56906836b134615875f28a21e8a012eb22cfe6a66a5
2018-08-11 06:59:14 -04:00
MarcoFalke
fa85c985ed
qa: Add p2p_invalid_locator test 2018-08-10 14:09:19 -04:00
Wladimir J. van der Laan
78dae8cacc
Merge #13780: 0.17: Pre-branch maintenance
3fc20632a3 qt: Set BLOCK_CHAIN_SIZE = 220 (DrahtBot)
2b6a2f4a28 Regenerate manpages (DrahtBot)
eb7daf4d60 Update copyright headers to 2018 (DrahtBot)

Pull request description:

  Some trivial maintenance to avoid having to do it again after the 0.17 branch off.

  (The scripts to do this are in `./contrib/`)

Tree-SHA512: 16b2af45e0351b1c691c5311d48025dc6828079e98c2aa2e600dc5910ee8aa01858ca6c356538150dc46fe14c8819ed8ec8e4ec9a0f682b9950dd41bc50518fa
2018-08-08 13:55:27 +02:00
MarcoFalke
5e17777777
qa: Create unicode tempdir in test_runner 2018-08-03 11:29:02 -04:00
DrahtBot
eb7daf4d60 Update copyright headers to 2018 2018-07-27 07:15:02 -04:00
Karl-Johan Alm
0128121101
test: Add basic testing for wallet groups 2018-07-24 15:05:37 +09:00
Wladimir J. van der Laan
6b6e854362
Merge #9662: Add createwallet "disableprivatekeys" option: a sane mode for watchonly-wallets
a3fa4d6a6a QA: Fix bug in -usecli logic that converts booleans to non-lowercase strings (Jonas Schnelli)
4704e5f074 [QA] add createwallet disableprivatekey test (Jonas Schnelli)
c7b8f343e9 [Qt] Disable creating receive addresses when private keys are disabled (Jonas Schnelli)
2f15c2bc20 Add disable privatekeys option to createwallet (Jonas Schnelli)
cebefba085 Add option to disable private keys during internal wallet creation (Jonas Schnelli)
9995a602a6 Add facility to store wallet flags (64 bits) (Jonas Schnelli)

Pull request description:

  This mode ('createwallet {"disableprivatekeys": true}') is intended for a sane pure watch-only mode, ideal for a use-case where one likes to use Bitcoin-Core in conjunction with a hardware-wallet or another solutions for cold-storage.

  Since we have support for custom change addresses in `fundrawtransaction`, pure watch-only wallets including coin-selection are possible and do make sense for some use cases.

  This new mode disables all forms of private key generation and ensure that no mix between hot and cold keys are possible.

Tree-SHA512: 3ebe7e8d54c4d4e5f790c348d4c292d456f573960a5b04d69ca5ef43a9217c7e7671761c6968cdc56f9a8bc235f3badd358576651af9f10855a0eb731f3fc508
2018-07-20 14:28:50 +02:00
Wladimir J. van der Laan
b654723461
Merge #13557: BIP 174 PSBT Serializations and RPCs
020628e3a4 Tests for PSBT (Andrew Chow)
a4b06fb42e Create wallet RPCs for PSBT (Andrew Chow)
c27fe419ef Create utility RPCs for PSBT (Andrew Chow)
8b5ef27937 SignPSBTInput wrapper function (Andrew Chow)
58a8e28918 Refactor transaction creation and transaction funding logic (Andrew Chow)
e9d86a43ad Methods for interacting with PSBT structs (Andrew Chow)
12bcc64f27 Add pubkeys and whether input was witness to SignatureData (Andrew Chow)
41c607f09b Implement PSBT Structures and un/serialization methods per BIP 174 (Andrew Chow)

Pull request description:

  This Pull Request fully implements the [updated](https://github.com/bitcoin/bips/pull/694) BIP 174 specification. It is based upon #13425 which implements the majority of the signing logic.

  BIP 174 specifies a binary transaction format which contains the information necessary for a signer to produce signatures for the transaction and holds the signatures for an input while the input does not have a complete set of signatures.

  This PR contains structs for PSBT, serialization, and deserialzation code. Some changes to `SignatureData` have been made to support detection of UTXO type and storing public keys.

  ***

  Many RPCs have been added to handle PSBTs.

  `walletprocesspsbt` takes a PSBT format transaction, updates the PSBT with any inputs related to this wallet, signs, and finalizes the transaction. There is also an option to not sign and just update.

  `walletcreatefundedpsbt` creates a PSBT from user provided data in the same form as createrawtransaction. It also funds the transaction and takes an options argument in the same form as `fundrawtransaction`. The resulting PSBT is blank with no input or output data filled in. It is analogous to a combination of `createrawtransaction` and `fundrawtransaction`

  `decodepsbt` takes a PSBT and decodes it to JSON. It is analogous to `decoderawtransaction`

  `combinepsbt` takes multiple PSBTs for the same tx and combines them. It is analogous to `combinerawtransaction`

  `finalizepsbt` takes a PSBT and finalizes the inputs. If all inputs are final, it extracts the network serialized transaction and returns that instead of a PSBT unless instructed otherwise.

  `createpsbt` is like `createrawtransaction` but for PSBTs instead of raw transactions.

  `convertpsbt` takes a network serialized transaction and converts it into a psbt. The resulting psbt will lose all signature data and an explicit flag must be set to allow transactions with signature data to be converted.

  ***

  This supersedes #12136

Tree-SHA512: 1ac7a79e5bc669933f0a6fcc93ded55263fdde9e8c144a30266b13ef9f62aacf43edd4cbca1ffbe003090b067e9643c9298c79be69d7c1b10231b32acafb6338
2018-07-18 20:25:44 +02:00
Wladimir J. van der Laan
8fceae0d6f
Merge #12196: Add scantxoutset RPC method
be98b2d9a8 [QA] Add scantxoutset test (Jonas Schnelli)
eec7cf7b33 scantxoutset: mention that scanning by address will miss P2PK txouts (Jonas Schnelli)
94d73d32ab scantxoutset: support legacy P2PK script type (Jonas Schnelli)
892de1dfea scantxoutset: add support for scripts (Jonas Schnelli)
78304941f7 Blockchain/RPC: Add scantxoutset method to scan UTXO set (Jonas Schnelli)
9048575511 Add FindScriptPubKey() to search the UTXO set (Jonas Schnelli)

Pull request description:

  Alternative to #9152.

  This takes `<n>` pubkeys and optionally  `<n>` xpubs (together with a definable lookup windows where the default is 0-1000) and looks up common scripts in the UTXO set of all given or derived keys.

  The output will be an array similar to `listunspent`. That array is compatible with `createrawtransaction` as well as with `signrawtransaction`.

  This makes it possible to prepare sweeps and have them signed in a secure (cold) space.

Tree-SHA512: a2b22a117cf6e27febeb97e5d6fe30184926d50c0c7cbc77bb4121f490fed65560c52f8eac67a9720d7bf8f420efa42459768685c7e7cc03722859f51a5e1e3b
2018-07-17 16:03:16 +02:00
Andrew Chow
020628e3a4 Tests for PSBT
Added functional tests for PSBT that test the RPCs. Also added all
of the BIP 174 test vectors (except for the updater tests) in the
functional tests.

Added a Unit test for the BIP 174 updater test vector.
2018-07-16 17:05:30 -07:00
Jonas Schnelli
be98b2d9a8
[QA] Add scantxoutset test 2018-07-15 21:18:11 +01:00
Pieter Wuille
b25a4c2284
Merge #13072: Update createmultisig RPC to support segwit
f40b3b82df [tests] functional test for createmultisig RPC (Anthony Towns)
b9024fdda3 segwit support for createmultisig RPC (Anthony Towns)
d58055d25f Move AddAndGetDestinationForScript from wallet to outputype module (Anthony Towns)
9a44db2e46 Add outputtype module (Anthony Towns)

Pull request description:

  Adds an "address_type" parameter that accepts "legacy", "p2sh-segwit", and "bech32" to choose the type of address created. Defaults to "legacy" rather than the value of the `-address-type` option for backwards compatibility.

  As part of implementing this, OutputType is moved from wallet into its own module, and `AddAndGetDestinationForScript` is changed to apply to a `CKeyStore` rather than a wallet, and to invoke `keystore.AddCScript(script)` itself rather than expecting the caller to have done that.

  Fixes #12502

Tree-SHA512: a08c1cfa89976e4fd7d29caa90919ebd34a446354d17abb862e99f2ee60ed9bc19d8a21a18547c51dc3812cb9fbed86af0bef2f1e971f62bf95cade4a7d86237
2018-07-13 20:31:13 -07:00
Jonas Schnelli
4704e5f074
[QA] add createwallet disableprivatekey test 2018-07-12 20:32:07 +01:00
Anthony Towns
f40b3b82df [tests] functional test for createmultisig RPC 2018-07-10 00:09:37 +10:00
Daniel Kraft
161e8d40a4 RPC: Add new getzmqnotifications method.
This adds a new RPC method "getzmqnotifications", which returns
information about all active ZMQ notification endpoints.  This is useful
for software that layers on top of bitcoind, so it can verify that
ZeroMQ is enabled and also figure out where it should listen.

See https://github.com/bitcoin/bitcoin/issues/13526.
2018-07-05 08:02:22 +02:00
practicalswift
634bd97001 Explicitly specify encoding when opening text files in Python code 2018-06-12 21:34:52 +02:00
MarcoFalke
fa6edfef35
qa: Remove portseed_offset from test runner 2018-06-08 13:31:27 -04:00
Wladimir J. van der Laan
b9551d3663
Merge #10757: RPC: Introduce getblockstats to plot things
41d0476f62 Tests: Add data file (Anthony Towns)
4cbfb6aad9 Tests: Test new getblockstats RPC (Jorge Timón)
35e77a0288 RPC: Introduce getblockstats (Jorge Timón)
cda8e36f01 Refactor: RPC: Separate GetBlockChecked() from getblock() (Jorge Timón)

Pull request description:

  It returns per block statistics about several things. It should be easy to add more if people think of other things to add or remove some if I went too far (but once written, why not keep it? EDIT: answer: not to test or maintain them).

  The currently available options are: minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs (EDIT: see updated list in the rpc call documentation)

  For the x axis, one can use height or block.nTime (I guess I could add mediantime if there's interest [EDIT: nobody showed interest but I implemented mediantime nonetheless, in fact there's no distinction between x or y axis anymore, that's for the caller to judge]).

  To calculate fees, -txindex is required.

Tree-SHA512: 2b2787a3c7dc4a11df1fce62c8a4c748f5347d7f7104205d5f0962ffec1e0370c825b49fd4d58ce8ce86bf39d8453f698bcd46206eea505f077541ca7d59b18c
2018-05-23 19:00:48 +02:00
Jorge Timón
4cbfb6aad9
Tests: Test new getblockstats RPC
Includes commit from Anthony Towns @ajtowns:

Tests: Save and load block and corresponding expected statistics
2018-05-22 23:26:33 +02:00
Wladimir J. van der Laan
4a50ec0efd
Merge #13188: qa: Remove unused option --srcdir
fac1e1f qa: Remove unused option --srcdir (MarcoFalke)

Pull request description:

  The `srcdir` option was both unused and misleading; It should have been called `builddir`. So remove it.

Tree-SHA512: 2c24dcf2aa82219158b8cbbf03dd3f0f51f805f1f5f670faa1fd59e5a8d60fda120ffddadeccb058d8d3f20583b4952be7afd2df6bbefb9367d35c0f0a9fda3c
2018-05-09 10:42:38 +02:00
Wladimir J. van der Laan
7b966d9e6e
Merge #10267: New -includeconf argument for including external configuration files
25b7ab9 doc: Add release notes for -includeconf (Karl-Johan Alm)
0f0badd test: Test includeconf parameter. (Karl-Johan Alm)
629ff8c -includeconf=<path> support in config handler, for including external configuration files (Karl-Johan Alm)

Pull request description:

  Fixes: #10071.

  Done:
  - adds `-includeconf=<path>`, where `<path>` is relative to `datadir` or to the path of the file being read, if in a file
  - protects against circular includes
  - updates help docs

  ~~~Thoughts:~~~
  - ~~~I am not sure how to test this in a neat manner. Feedback on this would be nice. Will dig/think though.~~~

Tree-SHA512: cb31f1b2f69fbc0890d264948eb2e501ac05cf12f5e06a5942f9c1539eb15ea8dc3cae817f4073aecb2fcc21d0386747f14f89d990772003a76e2a6d25642553
2018-05-09 06:36:54 +02:00
MarcoFalke
fac1e1f3b2
qa: Remove unused option --srcdir 2018-05-08 13:07:31 -04:00
Karl-Johan Alm
0f0badd5a3
test: Test includeconf parameter. 2018-05-02 14:26:45 +09:00
James O'Beirne
bf720c1460 Add --failfast option to functional test runner
Also cleans up run_test's arguments list (no more mutable default for `args`)
and call site.
2018-04-27 15:01:31 -04:00
Wladimir J. van der Laan
e074097c1b
Merge #13051: qa: Normalize executable location
fa811b0 qa: Normalize executable location (MarcoFalke)

Pull request description:

  This removes the need to override the executable locations by just reading them from the config file. Beside making the code easier to read, running individual test on Windows is now possible by default (without providing further command line arguments).

  Note: Of course, it is still possible to manually specify the location through the `BITCOIND` environment variable, e.g. `bitcoin-qt`.

Tree-SHA512: bee6d22246796242d747120ca18aaab089f73067de213c9111182561985c5912228a0b0f7f9eec025ecfdb44db031f15652f30d67c489d481c995bb3232a7ac7
2018-04-25 15:55:13 +02:00
Wladimir J. van der Laan
d1d54ae6a3
Merge #12953: Deprecate accounts
cead28b [docs] Add release notes for deprecated 'account' API (John Newbery)
72c9575 [wallet] [tests] Add tests for accounts/labels APIs (John Newbery)
109e05d [wallet] [rpc] Deprecate wallet 'account' API (John Newbery)
3576ab1 [wallet] [rpc] Deprecate account RPC methods (John Newbery)
3db1ba0 [tests] Set -deprecatedrpc=accounts in tests (John Newbery)
4e671f0 [tests] Rename rpc_listtransactions.py to wallet_listtransactions.py (John Newbery)
a28b907 [wallet] [rpc] Remove duplicate entries in rpcwallet.cpp's CRPCCommand table (John Newbery)

Pull request description:

  Deprecate all accounts functionality and make it only accessible by using `-deprecatedrpc=accounts`.

  Accounts specific RPCs, account arguments, and account related results all require the `-deprecatedrpc=accunts` startup option now in order to see account things.

  Several wallet functional tests use the accounts system. Those tests are unchanged, except to start the nodes with `-deprecatedrpc=accounts`. We can slowly migrate those tests to use the 'label' API instead of the 'account' API before accounts are fully removed.

Tree-SHA512: 89f4ae2fe6de4a1422f1817b0997ae22d63ab5a1a558362ce923a3871f3e42963405d6573c69c27f1764679cdee5b51bf52202cc407f1361bfd8066d652f3f37
2018-04-24 15:21:27 +02:00
MarcoFalke
fa811b0b1d
qa: Normalize executable location 2018-04-21 16:55:42 -04:00
John Newbery
4e671f0353 [tests] Rename rpc_listtransactions.py to wallet_listtransactions.py
listtransactions is a wallet RPC. The test name should indicate that
this is a wallet test.
2018-04-16 14:43:33 -04:00
Wladimir J. van der Laan
e87fefc60f test: Add rpc_bind test to default-run tests
Skip the parts that cannot be run on the host due to lack
of IPv6 support or a second interface to bind on, and warn
appropriately.

Without no strong requirements (besides being Linux only, in which case
the test is skipped) left, just add this test to the default in
test_runner.

Includes suggested changes by John Newbery.
2018-04-13 14:40:05 +02:00
John Newbery
e80c640d78 [tests] Remove bip9-softforks.py
bip9-sofforks.py was intended to be a generic test for versionbits
deployments. However, it only tests CSV activation and was not updated
to test segwit activation. CSV activation is tested by
bip68-112-113-p2p.py, so this test is duplicated effort.

Rather than try to update it to use the BitcoinTestFramework, just
remove it.
2018-04-02 13:45:58 -04:00
Wladimir J. van der Laan
18815b4bfb
Merge #11742: rpc: Add testmempoolaccept
b55555d rpc: Add testmempoolaccept (MarcoFalke)

Pull request description:

  To check if a single raw transaction makes it into the current transaction pool, one had to call `sendrawtransaction`. However, on success, this adds the transaction to the mempool with no easy way to undo.

  The call `testmempoolaccept` is introduced to provide a way to solely check the result without changing the mempool state.

Tree-SHA512: 5afd9311190135cee8fc1f229c7d39bf893f1028f29e28d34f70df820198ff97b4bf86b41cbbd6e6c36a5c30073cefa92d541c74a4939c7a2a6fa283dfd41b63
2018-04-02 16:02:33 +02:00
John Newbery
63048ec73d [tests] Test starting bitcoind with -h and -version 2018-03-30 16:35:09 -04:00
Anthony Towns
ffb033a6d5 test: List any failed tests at the end of test_runner output
Change sorting output to put failed tests at the end of test_runner
output.
2018-03-28 15:49:55 +02:00
Wladimir J. van der Laan
f92541f7ea test: Make summary row bold-red if any test failed
Make the summary row of the test runner bold red if any test fails.
This helps visibility if something fails.
2018-03-28 11:17:36 +02:00
Wladimir J. van der Laan
534b8fa560
Merge #12653: Allow to optional specify the directory for the blocks storage
a192636 -blocksdir: keep blockindex leveldb database in datadir (Jonas Schnelli)
f38e4fd QA: Add -blocksdir test (Jonas Schnelli)
386a6b6 Allow to optional specify the directory for the blocks storage (Jonas Schnelli)

Pull request description:

  Since the actual block files taking up more and more space, it may be desirable to have them stored in a different location then the data directory (use case: SSD for chainstate, etc., HD for blocks).

  This PR adds a `-blocksdir` option that allows one to keep the blockfiles and the blockindex external from the data directory (instead of creating symlinks).

  I fist had an option to keep the blockindex within the datadir, but seems to make no sense since accessing the index will (always) lead to access (r/w) the block files.

Tree-SHA512: f8b9e1a681679eac25076dc30e45e6e12d4b2d9ac4be907cbea928a75af081dbcb0f1dd3e97169ab975f73d0bd15824c00c2a34638f3b284b39017171fce2409
2018-03-27 21:22:36 +02:00
MarcoFalke
b55555da3e
rpc: Add testmempoolaccept 2018-03-24 11:17:08 -04:00
Russell Yanofsky
d2527bd54e Rename wallet_accounts.py test
This is a separate commit because changing the test at the same time as
renaming it breaks git (and github) rename detection.
2018-03-19 12:05:35 -04:00
Jeff Rade
0dbb32b2cb Avoiding 'file' function name from python2 with more descriptive variable naming 2018-03-18 17:55:41 -05:00
Jonas Schnelli
f38e4fdb06
QA: Add -blocksdir test 2018-03-18 12:24:10 +07:00
Jeff Rade
b0fec8d623 Append scripts to new test_list array to fix bad assignment 2018-03-17 13:42:31 -05:00
Jeff Rade
97bcd36811 [Trivial] Simplify if-else blocks and more descriptive variable naming 2018-03-16 13:52:00 -05:00
Wladimir J. van der Laan
987a80995a
Merge #11882: Disable default fallbackfee on mainnet
3f592b8 [QA] add wallet-rbf test (Jonas Schnelli)
8222e05 Disable wallet fallbackfee by default on mainnet (Jonas Schnelli)

Pull request description:

  Removes the default fallback fee on mainnet (but keeps it on testnet/regtest).

  Transactions using the fallbackfee in case the fallback fee has not been set are getting rejected.

Tree-SHA512: e54d2594b7f954e640cc513a18b0bfbe189f15e15bdeed4fe02b7677f939bca1731fef781b073127ffd4ce08a595fb118259b8826cdaa077ff7d5ae9495810db
2018-03-01 19:01:52 +01:00
Jonas Schnelli
3f592b81dc
[QA] add wallet-rbf test 2018-02-25 09:38:38 +08:00
MeshCollider
ada1af6d8f Fix typo in test_runner.py causing error 2018-02-15 14:54:41 +13:00
Wladimir J. van der Laan
b4d85490f0
Merge #11858: qa: Prepare tests for Windows
faefd29 qa: Prepare functional tests for Windows (MarcoFalke)

Pull request description:

  * Pass `sys.executable` when calling a python script via the subprocess
    module
  * Don't remove the log file while it is still open and written to
  * Properly use os.pathsep and os.path.sep when modifying the PATH
    environment variable
  * util-tests: Use os.path.join for Windows compatibility

  Ref:  #8227

Tree-SHA512: c507a536af104b3bde4366b6634099db826532bd3e7c35d694b5883c550920643b3eab79c76703ca67e1044ed09979e855088f7324321c8d52112514e334d614
2018-02-12 11:31:18 +01:00
Anthony Towns
125f4a4909 [tests] Require all tests to follow naming convention 2018-01-30 16:53:45 +10:00
Anthony Towns
6f881cc880 [tests] Remove EXPECTED_VIOLATION_COUNT 2018-01-25 09:44:30 +10:00
Anthony Towns
3150b3fea7 [tests] Rename misc functional tests. 2018-01-25 09:44:30 +10:00
Anthony Towns
81b79f2c39 [tests] Rename rpc_* functional tests. 2018-01-25 09:44:29 +10:00
Anthony Towns
61b8f7f273 [tests] Rename p2p_* functional tests. 2018-01-25 09:44:29 +10:00
Anthony Towns
90600bc7db [tests] Rename wallet_* functional tests. 2018-01-25 09:44:29 +10:00
Anthony Towns
ca6523d0c8 [tests] Rename feature_* functional tests. 2018-01-25 09:44:29 +10:00
MarcoFalke
faefd2923a
qa: Prepare functional tests for Windows
* Pass `sys.executable` when calling a python script via the subprocess
  module
* Don't remove the log file while it is still open and written to
* Properly use os.pathsep and os.path.sep when modifying the PATH
  environment variable
* util-tests: Use os.path.join for Windows compatibility
2018-01-24 15:52:07 -05:00
Russell Yanofsky
ac96e788fa test_runner: Readable output if create_cache.py fails
Without this change, create_cache.py process output is shown as a byte() object
with \n escapes in a single line that is hard to read.
2018-01-19 13:04:03 -05:00
Wladimir J. van der Laan
4db16ec827
Merge #11796: [tests] Functional test naming convention
5fecd84 [tests] Remove redundant import in blocktools.py test (Anthony Towns)
9b20bb4 [tests] Check tests conform to naming convention (Anthony Towns)
7250b4e [tests] README.md nit fixes (Anthony Towns)
82b2712 [tests] move witness util functions to blocktools.py (John Newbery)
1e10854 [tests] [docs] update README for new test naming scheme (John Newbery)

Pull request description:

  Splitting #11774 into two parts -- this part updates the README with the proposed naming convention, and adds some checks to test_runner.py that the number of tests violating the naming convention doesn't increase too much. Idea is this part of the change should not introduce merge conflicts or require much rebasing, so reviews of the complicated bits won't become invalidated too often; while the second part will just be file renames, which will require regular rebasing and will introduce merge conflicts with pending PRs, but can be merged later, and should also be much easier to review, since it will only include relatively trivial changes.

Tree-SHA512: b96557d41714addbbfe2aed62fb5a48639eaeb1eb3aba30ac1b3a86bb3cb8d796c6247f9c414c4695c4bf54c0ec9968ac88e2f88fb62483bc1a2f89368f7fc80
2018-01-15 10:38:01 +01:00
MarcoFalke
b7450cdbd8
Merge #11970: Add test coverage for bitcoin-cli multiwallet calls
a14dbff39e Allow multiwallet.py to be used with --usecli (Russell Yanofsky)
f6ade9ce1a [tests] allow tests to be run with --usecli (John Newbery)
ff9a363ff7 TestNodeCLI batch emulation (Russell Yanofsky)
ca9085afc5 Prevent TestNodeCLI.args mixups (Russell Yanofsky)
fcfb952bca Improve TestNodeCLI output parsing (Russell Yanofsky)

Pull request description:

  Lack of test coverage was pointed out by @jnewbery in https://github.com/bitcoin/bitcoin/pull/11687#discussion_r158133900

Tree-SHA512: 5f10e31abad11a5edab0da4e2515e39547adb6ab9e55e50427ab2eb7ec9a43d6b896b579b15863e5edc9beee7d8bf1c84d9dabd247be0760a1b9ae39e1e8ee02
2018-01-12 17:25:01 -05:00
Jonas Schnelli
d889c036cd
Merge #11403: SegWit wallet support
b224a47a1 Add address_types test (Pieter Wuille)
7ee54fd7c Support downgrading after recovered keypool witness keys (Pieter Wuille)
940a21932 SegWit wallet support (Pieter Wuille)
f37c64e47 Implicitly know about P2WPKH redeemscripts (Pieter Wuille)
57273f2b3 [test] Serialize CTransaction with witness by default (Pieter Wuille)
cf2c0b6f5 Support P2WPKH and P2SH-P2WPKH in dumpprivkey (Pieter Wuille)
37c03d3e0 Support P2WPKH addresses in create/addmultisig (Pieter Wuille)
3eaa003c8 Extend validateaddress information for P2SH-embedded witness (Pieter Wuille)
30a27dc5b Expose method to find key for a single-key destination (Pieter Wuille)
985c79552 Improve witness destination types and use them more (Pieter Wuille)
cbe197470 [refactor] GetAccount{PubKey,Address} -> GetAccountDestination (Pieter Wuille)
0c8ea6380 Abstract out IsSolvable from Witnessifier (Pieter Wuille)

Pull request description:

  This implements a minimum viable implementation of SegWit wallet support, based on top of #11389, and includes part of the functionality from #11089.

  Two new configuration options are added:
  * `-addresstype`, with options `legacy`, `p2sh`, and `bech32`. It controls what kind of addresses are produced by `getnewaddress`, `getaccountaddress`, and `createmultisigaddress`.
  * `-changetype`, with the same options, and by default equal to `-addresstype`, that controls what kind of change is used.

  All wallet private and public keys can be used for any type of address. Support for address types dependent on different derivation paths will need a major overhaul of how our internal detection of outputs work. I expect that that will happen for a next major version.

  The above also applies to imported keys, as having a distinction there but not for normal operations is a disaster for testing, and probably for comprehension of users. This has some ugly effects, like needing to associate the provided label to `importprivkey` with each style address for the corresponding key.

  To deal with witness outputs requiring a corresponding redeemscript in wallet, three approaches are used:
  * All SegWit addresses created through `getnewaddress` or multisig RPCs explicitly get their redeemscripts added to the wallet file. This means that downgrading after creating a witness address will work, as long as the wallet file is up to date.
  * All SegWit keys in the wallet get an _implicit_ redeemscript added, without it being written to the file. This means recovery of an old backup will work, as long as you use new software.
  * All keypool keys that are seen used in transactions explicitly get their redeemscripts added to the wallet files. This means that downgrading after recovering from a backup that includes a witness address will work.

  These approaches correspond to solutions 3a, 1a, and 5a respectively from https://gist.github.com/sipa/125cfa1615946d0c3f3eec2ad7f250a2. As argued there, there is no full solution for dealing with the case where you both downgrade and restore a backup, so that's also not implemented.

  `dumpwallet`, `importwallet`, `importmulti`, `signmessage` and `verifymessage` don't work with SegWit addresses yet. They're remaining TODOs, for this PR or a follow-up. Because of that, several tests unexpectedly run with `-addresstype=legacy` for now.

Tree-SHA512: d425dbe517c0422061ab8dacdc3a6ae47da071450932ed992c79559d922dff7b2574a31a8c94feccd3761c1dffb6422c50055e6dca8e3cf94a169bc95e39e959
2018-01-10 20:55:41 -10:00
Pieter Wuille
b224a47a1a Add address_types test
Improvements and cleanups by John Newbery
2018-01-09 15:35:31 -08:00
Pieter Wuille
940a21932b SegWit wallet support
This introduces two command line flags (-addresstype and -changetype) which control
the type of addresses/outputs created by the GUI and RPCs. Certain RPCs allow
overriding these (`getnewaddress` and `getrawchangeaddress`). Supported types
are "legacy" (P2PKH and P2SH-multisig), "p2sh-segwit" (P2SH-P2WPKH and P2SH-P2WSH-multisig),
and "bech32" (P2WPKH and P2WSH-multisig).

A few utility functions are added to the wallet to construct different address type
and to add the necessary entries to the wallet file to be compatible with earlier
versions (see `CWallet::LearnRelatedScripts`, `GetDestinationForKey`,
`GetAllDestinationsForKey`, `CWallet::AddAndGetDestinationForScript`).
2018-01-09 15:35:31 -08:00
Russell Yanofsky
a14dbff39e Allow multiwallet.py to be used with --usecli
Add test coverage for bitcoin-cli multiwallet calls.
2018-01-08 17:35:27 -05:00
Wladimir J. van der Laan
5961b23898
Merge #12062: Increment MIT Licence copyright header year on files modified in 2017
595a7ba Increment MIT Licence copyright header year on files modified in 2017 (Akira Takizawa)

Pull request description:

  Edited via:

  $ contrib/devtools/copyright_header.py update .

  ps) It is the same commit as #9450

Tree-SHA512: 274bfcd6cf2914315ed52f6db773a68800ce9d6bd225a3142654483f0bbc3fd865009e62f9d954f65765d038c626e55d2a64e37e16843809adc2f67abe659b6d
2018-01-03 14:33:28 +01:00
Akira Takizawa
595a7bab23 Increment MIT Licence copyright header year on files modified in 2017 2018-01-03 02:26:56 +09:00
MarcoFalke
fadf60e381 qa: Note on test order in test_runner 2017-12-21 08:56:40 -05:00
MeshCollider
be9a13c8a0 Add configuration/argument testing 2017-12-20 12:23:12 +13:00
Anthony Towns
9b20bb40fb [tests] Check tests conform to naming convention
Extra-Author: John Newbery <john@johnnewbery.com>
2017-12-08 01:50:02 +10:00
Jonas Schnelli
e054d0e532
[QA] Add node_network_limited test 2017-12-06 21:13:48 -10:00
Wladimir J. van der Laan
24df9af816
Merge #11781: Add -debuglogfile option
5a7c09a test: Add tests for `-debuglogfile` with subdirs (Anthony Towns)
4158734 doc: Update release notes for `-debuglogfile` (Wladimir J. van der Laan)
2323242 test: Add test for `-debuglogfile` (Wladimir J. van der Laan)
cf5f432 Add `-debuglogfile` option (Wladimir J. van der Laan)

Pull request description:

  This patch adds an option to configure the name and/or directory of the debug log file.

  The user can specify either a relative path, in which case the path is relative to the (network specific) data directory. They can also specify an absolute path to put the log anywhere else in the file system.

  Alternative to #11741 that gets rid of the concept of a "log directory" by specifying the path for the specific kind of log, the debug log. Which happens to be the only kind of log we have at this point*, but a hypothetical new kind of log (say, an audit log) would get a new option. This has more flexibility than specifying a directory which has to contain all of them.

  \* excluding `db.log` which is internally generated by the wallet database library, but that one moves along with `-walletdir`.

Tree-SHA512: 4434d0e598dc23504e5c9e67fdbaef56db4f0fd490f9f54fd503e69d4dda9b5b69c539e1794ed841e72161b7b1dc3374d2f1193dd431b057566750e56fd8f24b
2017-12-04 18:55:17 +01:00
Wladimir J. van der Laan
23232422e5 test: Add test for -debuglogfile 2017-12-01 11:28:13 +01:00
John Newbery
ff8a9b068a [tests] Add combinedlogslen argument to test_runner.py
Instead of calling combine_logs from test_runner when the environment variable
PYTHON_DEBUG is set, add a combinedlogslen argument. This makes it easier
to call when running locally, and allows a different length of combined logs
to be printed.
2017-11-29 15:10:51 -05:00
John Newbery
dba94ead0c [tests] [travis-ci] Move Travis functional test log post processing to test_runner 2017-11-29 15:10:05 -05:00
MeshCollider
c1e5d40e16 Make debugging test crash easier 2017-11-18 00:51:00 +13:00
Matt Corallo
f3d4adfa6f Make p2p-acceptablock not an extended test 2017-10-31 13:51:34 -04:00
Wladimir J. van der Laan
2ca518deff
Merge #11486: [tests] Add uacomment tests
c5dfa90 [tests] Add uacomment tests (Cristian Mircea Messel)

Pull request description:

  Checks for setting the value, max length and reserved characters

Tree-SHA512: a62e2cf8e455a3cd3987c0855f7bfc49de47504c01263e3573366e3cbff400c5678224773d4f1e4ac684fff34d987994e490a0978c4da05ff2a4bfa972c84723
2017-10-18 15:49:37 +02:00