Commit graph

1473 commits

Author SHA1 Message Date
Ben Woosley 510c6532ba
Extract ParseDescriptorRange
So as to be consistently informative when the checks fail, and
to protect against unintentional divergence among the checks.
2019-04-13 18:52:11 -07:00
MarcoFalke 0e9cb2d24d
Merge #15773: test: Add BitcoinTestFramework::sync_* methods
fafe5f0d09 test: Remove unused imports (MarcoFalke)
fa16a09215 scripted-diff: use self.sync_* methods (MarcoFalke)
faf77f9b90 test: Pass self to test_simple_bumpfee_succeeds (MarcoFalke)
fa6dc7c5c3 test: Add BitcoinTestFramework::sync_* methods (MarcoFalke)
fafe008cb4 test: Pass at most one node group to sync_all (MarcoFalke)
fa4680ed09 scripted-diff: Rename sync_blocks to send_blocks to avoid name collisions and confusion (MarcoFalke)

Pull request description:

  This adds methods to the test framework that can be called by just `self.sync_*()`.

  This avoids having to import the underlying util method. Also, in the default case, where all nodes are synced this avoid having to pass `self.nodes` explicitly.

  So the effective changes are:

  ```diff
  @@
  -from test_framework.util import sync_blocks, sync_mempools
  @@
  -        sync_blocks(self.nodes)
  +        self.sync_blocks()
  @@
  -        sync_mempools(self.nodes)
  +        self.sync_mempools()

ACKs for commit fafe5f:
  promag:
    utACK fafe5f0.
  jonatack:
    ACK fafe5f0d09, nice simplification.

Tree-SHA512: 5c81840edf9fb3c5de2d7bf95ca36a5a8d23567cd1479a0f4044547c2080e9a3c5cf375357bc8eebb5b68829be050a171ab2512cfd47b89feed51fe3bad2cd72
2019-04-11 13:23:05 -04:00
Gregory Sanders 184f8785f7 wallet_bumpfee.py: add test for change key preservation 2019-04-11 07:21:49 -04:00
Gregory Sanders d08becff85 add functional tests for feerate bumpfee with adding inputs 2019-04-11 07:21:49 -04:00
MarcoFalke 5392aee64f
Merge #15629: init: Throw error when network specific config is ignored
fae38c3dc6 doc: Fix all typos reported by codespell (MarcoFalke)
fa9058f0ed doc: Add release notes for 15629 (MarcoFalke)
fa4a922d78 qa: Add test for missing testnet section in conf file (MarcoFalke)
dddd6f0f58 init: Throw error when network specific config is ignored (MarcoFalke)

Pull request description:

  This should have no effect on mainnet users, but simplifies testing, where config settings are currently ignored with only a warning. Fix this by making it an error.

  Issues:
  *  bitcoin client 0.17.0 ignores wallet's name (file) #14523
  *  Can't set custom rpcport on testnet #13777
  * ...

ACKs for commit fae38c:

Tree-SHA512: 2e209526898eea6e444c803ec2666989cee4ca137492d32984998733c50a70056cb54657df8dc3027a6a0612738a8afce0bc35824b868c5f22281e00e0188530
2019-04-09 21:14:49 -04:00
MeshCollider 93de9abe6d
Merge #15632: Remove ResendWalletTransactions from the Validation Interface
833d98ae0 [wallet] Remove unnecessary Chain::Lock parameter from ResendWalletTransactions (John Newbery)
52b760fc6 [wallet] Schedule tx rebroadcasts in wallet (John Newbery)
f463cd107 [wallet] Keep track of the best block time in the wallet (John Newbery)

Pull request description:

  Remove the `Broadcast()`/`ResendWalletTransactions()` notification from the Validation interface.

  Closes #15619. See that issue for discussion.

ACKs for commit 833d98:
  ryanofsky:
    utACK 833d98ae07. No changes, just rebase.

Tree-SHA512: 7689f2083608ebad8c95ab6692f7842754e1ebe5508bc926a89cad7105cce41007648f37341ba5feb92b30a7aa87acd3abf264a4f1874e35a7161553f6ff3595
2019-04-10 09:53:08 +12:00
MarcoFalke fafe5f0d09
test: Remove unused imports 2019-04-09 12:10:35 -04:00
MarcoFalke fa16a09215
scripted-diff: use self.sync_* methods
-BEGIN VERIFY SCRIPT-
sed -i -e 's/sync_blocks(self.nodes)/self.sync_blocks()/g'     $(git grep -l 'sync_blocks(self.nodes)'   ./test/functional/*.py)
sed -i -e 's/sync_mempools(self.nodes)/self.sync_mempools()/g' $(git grep -l 'sync_mempools(self.nodes)' ./test/functional/*.py)

sed -i -e 's/  sync_blocks(/  self.sync_blocks(/g'     $(git grep -l sync_blocks   ./test/functional/*.py)
sed -i -e 's/  sync_mempools(/  self.sync_mempools(/g' $(git grep -l sync_mempools ./test/functional/*.py)
-END VERIFY SCRIPT-
2019-04-09 12:09:53 -04:00
MarcoFalke faf77f9b90
test: Pass self to test_simple_bumpfee_succeeds
Needed for a future scripted diff
2019-04-09 12:09:36 -04:00
MarcoFalke fa6dc7c5c3
test: Add BitcoinTestFramework::sync_* methods 2019-04-09 12:08:27 -04:00
MarcoFalke fafe008cb4
test: Pass at most one node group to sync_all 2019-04-09 12:08:26 -04:00
MarcoFalke f9f6c11a74
Merge #15771: qa: Prevent concurrency issues reading .cookie file
90bce24576 qa: Prevent concurrency issues reading .cookie file (João Barbosa)

Pull request description:

  Hopefully fixes #15733.

ACKs for commit 90bce2:

Tree-SHA512: 4db06a63bb57e8ae56a4eab9d352a9a8d66dd7425cf491ca5f9e1ec5e60e61cb5af9eedbd32a0a9f9bdd770d767adf499eed05dd03221686eb357f6417441b61
2019-04-09 10:46:44 -04:00
John Newbery 52b760fc6a [wallet] Schedule tx rebroadcasts in wallet
Removes the now-unused Broadcast/ResendWalletTransactions interface from
validationinterface.

The wallet_resendwallettransactions.py needs a sleep added at the start
to make sure that the rebroadcast scheduler is warmed up before the next
block is mined.
2019-04-09 10:38:13 -04:00
João Barbosa 90bce24576 qa: Prevent concurrency issues reading .cookie file 2019-04-09 14:28:07 +01:00
MarcoFalke f3ecf3025f
Merge #15772: test: Properly log named args in authproxy
fa078984c9 test: Properly log named args in authproxy (MarcoFalke)

Pull request description:

ACKs for commit fa0789:
  promag:
    ACK fa07898, for instance:
  practicalswift:
    utACK fa078984c9

Tree-SHA512: 3a2564c9b8392c2ef13657138fa0ba4a521015e2d53331156d2a07ccc9497fb268f21e8d93b065c5734d25e4aea8f5cf67f07e6ab93b0ec2987d66a136f94bb8
2019-04-09 08:57:59 -04:00
MeshCollider 54798c3a31
Merge #15749: Fix: importmulti only imports origin info for PKH outputs
b5d398772 Take non-importing keys into account for spendability warning in descriptor import (Pieter Wuille)
6e597001a Import all origin info in importmulti; even for non-importing pubkeys (Pieter Wuille)
9a93c91c8 Keep full pubkeys in FlatSigningProvider::origins (Pieter Wuille)

Pull request description:

  This fixes #15743 and #15742.

  Since #15263, pubkeys are no longer imported for non-PKH (or WPKH, or any wrapped form of those) outputs, as that would incorrectly mark outputs to single-key versions of multisig policies as watched.

  As a side effect, this change also caused origin info not to be imported anymore for multisig policies.

  Fix this by plumbing through the full pubkey information for origins in FlatSigningProvider, and then importing all origin info we have in `importmulti` (knowing more never hurts, and additional origin information has no negative consequences like importing the pubkeys themselves).

ACKs for commit b5d398:
  MeshCollider:
    utACK b5d3987724

Tree-SHA512: 37caa2be8d01b8baa12f70a58eaa7c583f5f0afbe012e02936dd8790dc5dc852f880b77258b34ddb68cae30c029585f2d1c4f5d00015380557a1e8b471e500f3
2019-04-10 00:33:35 +12:00
MarcoFalke fa4680ed09
scripted-diff: Rename sync_blocks to send_blocks to avoid name collisions and confusion
-BEGIN VERIFY SCRIPT-
FILE_LIST=$(git grep -l 'def sync_blocks' ./test/functional/*.py)
sed -i -e 's/def sync_blocks/def send_blocks/g'   $FILE_LIST
sed -i -e 's/self.sync_blocks/self.send_blocks/g' $FILE_LIST
-END VERIFY SCRIPT-
2019-04-08 19:40:00 -04:00
MarcoFalke fa078984c9
test: Properly log named args in authproxy 2019-04-08 18:38:54 -04:00
João Barbosa aa410c2b17 rpc: Validate maxfeerate with AmountFromValue 2019-04-08 16:12:54 +01:00
Jonas Schnelli 327d2746fb
Merge #15495: Add regtests for HTTP status codes
8f5d9431a Add regtests for HTTP status codes. (Daniel Kraft)

Pull request description:

  This adds explicit tests for the returned HTTP status codes to `interface_rpc.py` (for error cases) and the HTTP JSON-RPC client in general for success.

  #15381 brought up discussion about the HTTP status codes in general, and the general opinion was that the current choice may not be ideal but should not be changed to preserve compatibility with existing JSON-RPC clients.  Thus it makes sense to actually test the current status to ensure this desired compatibility is not broken accidentally.

ACKs for commit 8f5d94:
  laanwj:
    utACK 8f5d9431a3
  promag:
    utACK 8f5d943.
  jonasschnelli:
    utACK 8f5d9431a3

Tree-SHA512: 82503ccd134dd9145304e95cb6c61755f100bee27593d567cdd5c0c554d47e7b06d937456cab04107f46f4984930355db65d5e711008a0b05f2b8feec9f2950e
2019-04-08 09:06:42 +02:00
MarcoFalke efbc86733a
Merge #15660: [qa] Overhaul p2p_compactblocks.py
7813eb1db1 [qa] Overhaul p2p_compactblocks.py (Suhas Daftuar)

Pull request description:

  Remove tests of:
   - compactblock behavior in a simulated pre-segwit version of bitcoind
     This should have been removed a long time ago, as it is not generally
     necessary for us to test the behavior of old nodes (except perhaps if we
     want to test that upgrading from an old node to a new one behaves properly)

   - compactblock behavior during segwit upgrade (ie verifying that network
     behavior before and after activation was as expected)
     This is unnecessary to test now that segwit activation has already happened.

ACKs for commit 7813eb:
  jnewbery:
    utACK 7813eb1db1

Tree-SHA512: cadf035e6f822fa8cff974ed0c2e88a1d4d7da559b341e574e785fd3d309cc2c98c63bc05479265dc00550ae7b77fc3cbe815caae7f68bcff13a04367dca9b52
2019-04-06 18:26:07 -04:00
Jim Posen 19308c9e21 rpc: Add getblockfilter RPC method.
Retrieves and returns block filter and header from index.
2019-04-06 12:10:55 -07:00
Pieter Wuille b5d3987724 Take non-importing keys into account for spendability warning in descriptor import 2019-04-06 09:14:56 -07:00
MarcoFalke faca95effd
qa: Make swap_magic_bytes in p2p_invalid_messages atomic 2019-04-04 16:51:26 -04:00
MarcoFalke 8dbb2c5e67
Merge #15680: Remove resendwallettransactions RPC method
ea1a2d8794 [wallet] Remove ResendWalletTransactionsBefore (John Newbery)
f5162458cd [rpc] remove resendwallettransactions RPC (John Newbery)

Pull request description:

  Remove resendwallettransactions RPC method

  This RPC was added for testing wallet rebroadcasts. Since we now have a real test for wallet rebroadcasts, it's no longer needed.

  The call in wallet_basic.py can be removed because wallet_resendwallettransactions.py tests wallet rebroadcast.

ACKs for commit ea1a2d:
  MarcoFalke:
    re-utACK ea1a2d8794
  promag:
    utACK ea1a2d8.

Tree-SHA512: 48245d947be1a2d2b8c30d2946105818c454a03b70b63534ecadf2144da64dafe1c9527ea670a5f4d1acd05ccdfc6c9be43ca636ee2ba58a8b7a7b2fc7bc88fd
2019-04-02 10:30:57 -04:00
Gregory Sanders 124ea38e39 change default Python block serialization to witness 2019-04-02 10:18:12 -04:00
Suhas Daftuar 7813eb1db1 [qa] Overhaul p2p_compactblocks.py
Remove tests of:
 - compactblock behavior in a simulated pre-segwit version of bitcoind
   This should have been removed a long time ago, as it is not generally
   necessary for us to test the behavior of old nodes (except perhaps if we
   want to test that upgrading from an old node to a new one behaves properly)

 - compactblock behavior during segwit upgrade (ie verifying that network
   behavior before and after activation was as expected)
   This is unnecessary to test now that segwit activation has already happened.

Includes changes by John Newbery.
2019-04-01 17:08:54 -04:00
MarcoFalke 5a2a9b5b06
Merge #15652: wallet: Update transactions with current mempool after load
4bf1b1cefa qa: Check unconfirmed balance after loadwallet (João Barbosa)
2ebf650b2e wallet: Update transactions with current mempool after load (João Barbosa)
57908a739c interfaces: Add Chain::requestMempoolTransactions (João Barbosa)
0440481c6b wallet: Move CWallet::ReacceptWalletTransactions locks to callers (João Barbosa)

Pull request description:

  Fixes #15591.

ACKs for commit 4bf1b1:
  MarcoFalke:
    re-utACK 4bf1b1cefa
  jnewbery:
    utACK 4bf1b1cefa

Tree-SHA512: 604b1057c7f9fc3772084bf6914e52dd1a68a1cfd365f907e8ec78f6f5f726bc56a3cad9f2b665642714fbf3d51e37c1184ac396460bddeafd918e8f9f7af392
2019-04-01 15:27:02 -04:00
João Barbosa 4bf1b1cefa qa: Check unconfirmed balance after loadwallet 2019-03-31 11:37:41 +01:00
John Newbery f5162458cd [rpc] remove resendwallettransactions RPC
This RPC was added for testing wallet rebroadcasts. Since we now have a
real test for wallet rebroadcasts, it's no longer needed.

The call in wallet_basic.py can be removed because
wallet_resendwallettransactions.py tests wallet rebroadcast.
2019-03-29 15:06:59 -04:00
John Newbery 03d6d23810 [tests] make pruning test faster
This commit makes the pruning.py much faster.

Key insights to do this:

- pruning.py doesn't care what kind of transactions make up the big
blocks that are pruned in the test. Instead of making blocks with
several large, expensive to construct and validate transactions,
instead make the large blocks contain a single coinbase transaction with
a huge OP_RETURN txout.
- avoid stop-starting nodes where possible.

This test could probably be made even faster by using the P2P interface
for submitting blocks instead of the submitblock RPC.
2019-03-29 11:43:41 -04:00
John Newbery 1c29ac40fb [tests] style fixes in feature_pruning.py
Minor style fixups. No functional change.
2019-03-29 11:43:26 -04:00
MarcoFalke 3702e1c17b
Merge #15646: [tests] Add test for wallet rebroadcasts
529c1ae4a0 [tests] Add test for wallet rebroadcasts (John Newbery)

Pull request description:

  The existing wallet_resendwallettransactions.py test only tests the
  resendwallettransactions RPC. It does not test whether transactions are
  actually rebroadcast, or whether the rebroadcast logic is called on a
  timer.

  Update the test to not use the resendwallettransactions RPC and test
  that transactions are resent on a timer.

ACKs for commit 529c1a:
  MarcoFalke:
    re-utACK 529c1ae4a0

Tree-SHA512: 7341e7dd07cdc8ecbc08b1949121824148d2b58133a8e298ecdc5b7555713df3cecffb49854443cef9f033ef847cbf329e879a3bf57ab4e1fc733be432e9f718
2019-03-27 14:32:26 -04:00
John Newbery 529c1ae4a0 [tests] Add test for wallet rebroadcasts
The existing wallet_resendwallettransactions.py test only tests the
resendwallettransactions RPC. It does not test whether transactions are
actually rebroadcast, or whether the rebroadcast logic is called on a
timer.

This commit updates the test to not use the resendwallettransactions RPC and
test that transactions are rebroadcast on a timer.
2019-03-27 11:18:58 -04:00
Miguel Herranz e16b6a7188
rpc: Rename size to vsize in mempool related calls 2019-03-24 12:01:43 +08:00
Wladimir J. van der Laan 7b13c64645
Merge #15642: [rpc] Remove deprecated rpc warnings
947f73ceba [docs] remove reference to signrawtransaction in the developer docs. (John Newbery)
7b6616b78b [rpc] Remove deprecated functionality message from validateaddress help (John Newbery)
839c3f7c49 [rpc] Remove signrawtransaction warning (John Newbery)

Pull request description:

  Removes some deprecated code from the RPCs:

  - signrawtransaction was deprecated in 0.17 and removed in 0.18. A warning message was left in place to advise users to use signrawtransactionwithwallet and signrawtransactionwithkey. That warning can now be removed.
  - validateaddress had some functionality deprecated in 0.17 and removed in 0.18. The help text for that functionality was not removed in 0.18 and can be removed now.

Tree-SHA512: 981678a697954ff2c392752e5a183b4b12c4eb94f55766ee1aa97a70d300668237db8fc5748c2772869d0155ba4a93e38817887b98160ee972a6f6ee94e3f7d9
2019-03-23 09:23:09 +01:00
MarcoFalke fae38c3dc6
doc: Fix all typos reported by codespell 2019-03-22 13:19:44 -04:00
John Newbery 839c3f7c49 [rpc] Remove signrawtransaction warning
signrawtransaction was deprecated in 0.17 and removed in 0.18. A warning
was left in place to tell users to migrate to using
signrawtransactionswithwallet or signrawtransactionwithkey. Remove the
warning now that it's been two releases since the method was removed.
2019-03-22 10:30:25 -04:00
Ben Woosley bb8ae2c419
rpc: Expose g_is_mempool_loaded via getmempoolinfo and /rest/mempool/info.json
And use it to fix a race condition in mempool_persist.py:
https://travis-ci.org/Empact/bitcoin/jobs/487577243

Since e.g. getrawmempool returns errors based on this status, this
enables users to test it for readiness.
2019-03-22 02:31:20 -07:00
MarcoFalke fa4a922d78
qa: Add test for missing testnet section in conf file 2019-03-21 15:54:28 -04:00
MarcoFalke fab0a68aa2
qa: mininode: Clearer error message on invalid magic bytes 2019-03-19 17:14:12 -04:00
MarcoFalke c033c4b5ce
Merge #13541: wallet/rpc: sendrawtransaction maxfeerate
7abd2e697c wallet/rpc: add maxfeerate parameter to testmempoolaccept (Karl-Johan Alm)
6c0a6f73e3 wallet/rpc: add maxfeerate parameter to sendrawtransaction (Karl-Johan Alm)
e5efacb941 test: Refactor vout fetches in rpc_rawtransaction (Karl-Johan Alm)

Pull request description:

  This adds a new `maxfeerate` parameter to `sendrawtransaction` which forces the node to reject a transaction whose feerate is above the given fee rate.

  This is a safety harness from shooting yourself in the foot and accidentally overpaying fees.

  See also #12911.

Tree-SHA512: efa50134a7c17c9330cfdfd48ba400e095c0a419cc45e630618d8b44929c25d780d1bb2710c1fbbb6e687eca373505b0338cdaa7f2ff4ca22636d84c31557a2e
2019-03-18 13:22:15 -04:00
MarcoFalke 27781b6530
Merge #15466: tests: Print remaining jobs in test_runner.py
2e5d482659 tests: Print remaining jobs in test_runner.py (Steven Roose)

Pull request description:

  This helps finding out which tests fail to finish.

Tree-SHA512: d22beb82beecd33aaa50731c83075e49577842d29fd21aa63bcb859df5da99069eba9cc16eed5d91dbba8fb0fdc317fb88b3b370c4d3917e9da1cd13b0a622dc
2019-03-18 11:26:04 -04:00
Karl-Johan Alm 7abd2e697c
wallet/rpc: add maxfeerate parameter to testmempoolaccept 2019-03-14 08:48:46 +09:00
Karl-Johan Alm 6c0a6f73e3
wallet/rpc: add maxfeerate parameter to sendrawtransaction 2019-03-14 08:48:46 +09:00
fanquake 335931df4a
rpc: return a number for estimated_feerate in analyzepsbt 2019-03-12 07:17:42 +08:00
MarcoFalke c94852e791
Merge #15564: cli: remove duplicate wallet fields from -getinfo
3f6568d66b cli: remove duplicate wallet fields from -getinfo (fanquake)

Pull request description:

  `walletversion` and `balance` are both included below.

Tree-SHA512: cd9fe9739a2f492c8f7c0407b43a6fa95187f7e5318f05e080bac112f9f4333d2e9b84c505d098f8d66fa79439007d1c0b22e5a87d70bf5ea53ab647ee4c2046
2019-03-11 10:28:42 -04:00
fanquake 3f6568d66b
cli: remove duplicate wallet fields from -getinfo 2019-03-09 16:40:46 +08:00
fanquake 890396cbd5
cli: replace testnet with chain and return network name as per BIP70. 2019-03-09 16:35:26 +08:00
Karl-Johan Alm e5efacb941
test: Refactor vout fetches in rpc_rawtransaction 2019-03-06 09:47:40 +09:00
Steven Roose 2e5d482659
tests: Print remaining jobs in test_runner.py
This helps finding out which tests fail to finish.
2019-03-05 16:18:50 +00:00
MarcoFalke a74d588f21
Merge #14954: build: Require python 3.5
fa2797808e test: Remove python3.4 workaround in feature_dbcrash (MarcoFalke)
dddd1d05d3 .python-version: Specify full version 3.5.6 (MarcoFalke)
faa7cdf764 scripted-diff: Update copyright in ./test (MarcoFalke)
fa0e65b772 scripted-diff: test: Remove brackets after assert (MarcoFalke)
fab5a1e0f4 build: Require python 3.5 (MarcoFalke)
fa6bf21f5e scripted-diff: test: Use py3.5 bytes::hex() method (MarcoFalke)

Pull request description:

  Python 3.4 is EOL after March 2019, so switch to 3.5. See https://devguide.python.org/#status-of-python-branches

  This pull does the following in a bunch of commits:
  * scripted diff to use the `bytes::hex()` method in place of previous wrappers (`b2x`, `bytes_to_hex_str`, `hexlify`, ...)
  * Update the build system (gitian and travis) to remove python2.7 and replace it with python3.5
  * Another scripted-diff to remove brackets after `assert`. This is unrelated to the python3.5 switch, but a stylistic commit, so probably not worth to split up. The motivation behind it is to avoid asserting on data structures (such as tuples of length one), which never fails:
  ```py
  >>> assert(False,)   # with brackets
  >>> assert False,    # without brackets
  SyntaxError: invalid syntax
  >>> assert False     # proper assertion
  AssertionError
  ```
  * And then a final scripted diff to update the copyright headers in the `test` subfolder, since I touched most of the files anyway and it wouldn't make sense to split this commit out into a separate pull.

  For reference (contributed by luke-jr):

  Ubuntu LTS (bionic): 3.6.5
  Debian stable (stretch): 3.5.3
  RHEL 8 (expected before v0.19): 3.6.x
  Gentoo stable: 3.6.5
  Arch: 3.7.1

Tree-SHA512: 643c28cd2d5b9543ce4bf8ad2a8b282bc79b37dc5b25c9c8358e6ce201e2a67a546463e5f3430b16652eb2489d7c3ed4b0772cd2e2bf790fe68a5e3cc8a25029
2019-03-05 09:13:13 -05:00
MarcoFalke fa2797808e
test: Remove python3.4 workaround in feature_dbcrash 2019-03-04 18:17:29 -05:00
MarcoFalke faa7cdf764
scripted-diff: Update copyright in ./test
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./test/
-END VERIFY SCRIPT-
2019-03-02 10:58:35 -05:00
MarcoFalke fa0e65b772
scripted-diff: test: Remove brackets after assert
-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended -e 's/assert ?\((.+)\)(( )*)?(#.*)?$/assert \1\3\3\4/g' $(git grep -l --extended-regexp 'assert ?\(' test)
-END VERIFY SCRIPT-
2019-03-02 10:51:35 -05:00
MarcoFalke fab5a1e0f4
build: Require python 3.5 2019-03-02 10:40:23 -05:00
MarcoFalke fa6bf21f5e
scripted-diff: test: Use py3.5 bytes::hex() method
-BEGIN VERIFY SCRIPT-
sed -i -e "s/def bytes_to_hex_str/def b_2_x/g" $(git grep -l bytes_to_hex_str)

export RE_B_0="[^()]*"                          # match no bracket
export RE_B_1="${RE_B_0}\(${RE_B_0}\)${RE_B_0}" # match exactly one ()
export RE_B_2="${RE_B_0}\(${RE_B_1}\)${RE_B_0}" # match wrapped (())

export RE_M="(b2x|bytes_to_hex_str)\(((${RE_B_0}|${RE_B_1}|${RE_B_2})*)\)"

sed -i --regexp-extended -e "s/${RE_M}/\2.hex()/g"      $(git grep -l -E '(b2x|bytes_to_hex_str)')

sed -i --regexp-extended -e "/  +bytes_to_hex_str( as b2x)?,/d"    $(git grep -l bytes_to_hex_str)
sed -i --regexp-extended -e "s/ +bytes_to_hex_str( as b2x)?,//g"   $(git grep -l bytes_to_hex_str)
sed -i --regexp-extended -e "s/, bytes_to_hex_str( as b2x)?//g"    $(git grep -l bytes_to_hex_str)

export RE_M="(binascii\.)?hexlify\(((${RE_B_0}|${RE_B_1}|${RE_B_2})*)\).decode\(${RE_B_0}\)"

sed -i --regexp-extended -e "s/${RE_M}/\2.hex()/g" $(git grep -l hexlify -- ':(exclude)share')

sed -i --regexp-extended -e  "/from binascii import hexlify$/d" $(git grep -l hexlify -- ':(exclude)share')
sed -i --regexp-extended -e "s/(from binascii import) .*hexlify/\1 unhexlify/g" $(git grep -l hexlify -- ':(exclude)share')

sed -i -e 's/ignore-names "/ignore-names "b_2_x,/g' ./test/lint/lint-python-dead-code.sh
-END VERIFY SCRIPT-
2019-03-02 10:40:12 -05:00
MarcoFalke 789b0bbf2a
Merge #15335: Fix lack of warning of unrecognized section names
1a7ba84e11 Fix lack of warning of unrecognized section names (Akio Nakamura)

Pull request description:

  In #14708, It was introduced that to warn when unrecognized section names are exist in the config file.
  But ```m_config_sections.clear()```  in ```ArgsManager::ReadConfigStream()``` is called every time when reading each configuration file, so it can warn about only last reading file if ```includeconf``` exists.

  This PR fix lack of warning by collecting all section names by moving ```m_config_sections.clear()```  to ```ArgsManager::ReadConfigFiles()``` .
  Also add a test code to confirm this situation.

Tree-SHA512: 26aa0cbe3e4ae2e58cbe73d4492ee5cf465fd4c3e5df2c8ca7e282b627df9e637267af1e3816386b1dc6db2398b31936925ce0e432219fec3a9b3398f01e3e65
2019-03-02 09:59:18 -05:00
MarcoFalke 9e3122de05
Merge #15492: [rpc] remove deprecated generate method
07cae5287c [wallet] remove unused GetScriptForMining (Sjors Provoost)
8bb3e4c487 [rpc] remove deprecated generate method (Sjors Provoost)

Pull request description:

  As announced in v0.18, the wallet generate rpc method is deprecated and will be fully removed in v0.19.

  Clients should transition to using the node rpc method `generatetoaddress`.

Tree-SHA512: 9e5e913b59f3e18440b2b7b356124c7b87ad19f81a1ab6ada06a6c396b84e734895465f569296f1ba8c12abf74863bab5fd77765c9e806c239713aa83a59485f
2019-03-02 09:43:39 -05:00
MarcoFalke f9dbb319d2
Merge #15485: add rpc_misc.py, mv test getmemoryinfo, add test mallocinfo
f13ad1cae0 modify test for memory locked in case locking pages failed at some point (Adam Jonas)
2fa85ebd1c add rpc_misc.py, mv test getmemoryinfo, add test mallocinfo (Adam Jonas)

Pull request description:

  Creating the `rpc_misc.py` functional test file to add space for adding tests to a file that doesn't have a lot of coverage.
    - Removing the `getmemoryinfo()` smoke test from wallet basic rather than moving it to keep the wallet decoupled. Feel like testing for reasonable memory allocation values should suffice.
    - Adding coverage for `mallocinfo()`. Introduced standard lib XML parser since the function exports an XML string that describes the current state of the memory-allocation implementation in the caller.

Tree-SHA512: ced30115622916c88d1e729969ee331272ec9f2881eb36dee4bb7331bf633a6810a57fed63a0cfaf86de698edb5162e6a035efd07c89ece1df56b69d61288072
2019-03-01 13:54:20 -05:00
Adam Jonas f13ad1cae0 modify test for memory locked in case locking pages failed at some point 2019-03-01 10:20:57 -05:00
MarcoFalke a6d7026a45
Merge #15497: rpc: Consistent range arguments in scantxoutset/importmulti/deriveaddresses
ca253f6ebf Make deriveaddresses use stop/[start,stop] notation for ranges (Pieter Wuille)
1675b7ce55 Use stop/[start,stop] notation in importmulti desc range (Pieter Wuille)
4566011631 Add support for stop/[start,stop] ranges to scantxoutset (Pieter Wuille)
6b9f45e81b Support ranges arguments in RPC help (Pieter Wuille)
7aa6a8aefb Add ParseRange function to parse args of the form int/[int,int] (Pieter Wuille)

Pull request description:

  This introduces a consistent notation for RPC arguments in `scantxoutset`, `importmulti`, and `deriveaddresses`, either:
  * `"range" : int` to just specify the end of the range
  * `"range" : [int,int]` to specify both the begin and the end of the range.

  For `scantxoutset`, this is a backward compatible new feature. For the two other RPCs, it's an incompatible change, but neither of them has been in a release so far. Because of that non-released reason, this only makes sense in 0.18, in my opinion.

  I suggest this as an alternative to #15496, which only makes `deriveaddresses` compatible with `importmulti`, but not with the existing `scantxoutset` RPC. I also think `[int,int]` is more convenient than `{"start":int,"stop":int}`.

  I realize this is technically a feature added to `scantxoutset` after the feature freeze. If desired, I'll drop the `scantxoutset` changes.

Tree-SHA512: 1cbebb90cf34f106786dbcec7afbf3f43fb8b7e46cc7e6763faf1bc1babf12375a1b3c3cf86ee83c21ed2171d99b5a2f60331850bc613db25538c38b6a056676
2019-03-01 09:13:18 -05:00
Daniel Kraft 8f5d9431a3 Add regtests for HTTP status codes.
This adds explicit tests for the returned HTTP status codes to
interface_rpc.py (for error cases) and the HTTP JSON-RPC client in
general for success.

PR 15381 brought up discussion about the HTTP status codes in general,
and the general opinion was that the current choice may not be ideal
but should not be changed to preserve compatibility with existing
JSON-RPC clients.  Thus it makes sense to actually test the current
status to ensure this desired compatibility is not broken accidentally.
2019-03-01 08:27:14 +01:00
Pieter Wuille ca253f6ebf Make deriveaddresses use stop/[start,stop] notation for ranges 2019-02-28 21:40:09 -08:00
MarcoFalke fa852f0e8d
test: Bump timeout on tests that timeout on windows 2019-02-28 18:35:09 -05:00
Pieter Wuille 1675b7ce55 Use stop/[start,stop] notation in importmulti desc range 2019-02-28 13:05:10 -08:00
Pieter Wuille 4566011631 Add support for stop/[start,stop] ranges to scantxoutset 2019-02-28 13:01:44 -08:00
Sjors Provoost 8bb3e4c487
[rpc] remove deprecated generate method 2019-02-27 17:41:01 +01:00
Adam Jonas 2fa85ebd1c add rpc_misc.py, mv test getmemoryinfo, add test mallocinfo 2019-02-26 15:08:36 -05:00
MarcoFalke d88f7f8764
Merge #15471: rpc/gui: Remove 'Unknown block versions being mined' warning
ef362f2773 rpc/gui: Remove 'Unknown block versions being mined' warning (Wladimir J. van der Laan)

Pull request description:

  Due to miners inserting garbage into the version numbers causing false positives, the current version signalling has become completely useless. This removes the "unknown block versions" warning which has the tendency to scare users unnecessarily (and might get them to "update" to something bad).

  It preserves the warning in the logs. Whether this is desirable can be a point of discussion.

Tree-SHA512: 51407ccd24a571462465d9c7180f0f28307c50b82a03284abe783e181d8ab7e0638dbb710698d883f28de8a609db70763e39be2470d956e67c833da0768e43e9
2019-02-26 09:20:39 -05:00
MarcoFalke 8f470ecc53
Merge #15419: qa: Always refresh cache to be out of ibd
fa2cdc9ac2 test: Simplify create_cache (MarcoFalke)
fa25210d62 qa: Fix wallet_txn_doublespend issue (MarcoFalke)
1111aecbb5 qa: Always refresh stale cache to be out of ibd (MarcoFalke)
fab0d85802 qa: Remove mocktime unless required (MarcoFalke)

Pull request description:

  When starting a test, we are always in IBD because the timestamps on cached blocks are in the past. Usually, we solve that by generating a block at the beginning of the test.

  That is clumsy and might even lead to other problems such as #15360 and https://github.com/bitcoin/bitcoin/issues/14446#issuecomment-461926598

  So fix that by getting rid of mocktime and always refreshing the last block of the cache when starting the test framework.

  Should fix #14446

Tree-SHA512: 6af09800f9c86131349a103af617a54551f5f3f3260d38e14e3f30fdd3d91a0feb0100c56cbb12eae4aeac5571ae4b530b16345cbb831d2670237b53351a22c1
2019-02-25 11:44:21 -05:00
MarcoFalke fa2cdc9ac2
test: Simplify create_cache 2019-02-25 11:18:24 -05:00
Wladimir J. van der Laan ef362f2773 rpc/gui: Remove 'Unknown block versions being mined' warning
Due to miners inserting garbage into the version numbers, the current
version signalling has become completely useless. This removes the
"unknown block versions" warning which has the tendency to scare
users unnecessarily (and might get them to "update" to something
bad).

It preserves the warning in the logs. Whether this is desirable can
be a point of discussion.
2019-02-25 15:59:02 +01:00
MarcoFalke fa4ce7038d
rpc: Actually throw help when passed invalid number of params 2019-02-20 13:34:16 -05:00
MarcoFalke f9775a8655
Merge #15439: tests: remove byte.hex() to keep compatibility
1a062b85f0 tests: remove byte.hex() to keep compatibility (Akio Nakamura)

Pull request description:

  Use ```test_framework.util.bytes_to_hex_str()``` instead of ```bytes.hex()``` that new in Python 3.5 to support minimum version of Python(test).

  ```test/functional/test_framework/wallet_util.py``` is also reported to have '\.hex()' in #15397,
  but it does not matter because it calls CScript.hex() defined in wallet_util.py.

Tree-SHA512: 1019212e965f0848d235fab4da11959dffa42e8adfcd41216c10795cfc63c804b5deb5a3317f25d29940b9dcf088ab76fe3fa80d2679dc19f5f482dc5bde3283
2019-02-20 09:18:41 -05:00
MarcoFalke fa25210d62
qa: Fix wallet_txn_doublespend issue 2019-02-19 14:22:15 -05:00
MarcoFalke 1111aecbb5
qa: Always refresh stale cache to be out of ibd 2019-02-19 14:22:13 -05:00
MarcoFalke fab0d85802
qa: Remove mocktime unless required 2019-02-19 10:43:58 -05:00
Wladimir J. van der Laan 38429c4b62
Merge #15404: [test] Remove -txindex to start nodes
8e4b4f683a Address test todos by removing -txindex to nodes. Originally added when updating getrawtransaction to stop searching unspent utxos. (Amiti Uttarwar)

Pull request description:

  Original todos added when removing getrawtransaction default behavior of searching unspent utxos.

Tree-SHA512: d080953c3b0d2e5dca2265a15966dc25985a614c9cc86271ecd6276178ce428c85e262c24df92501695c32fed7beec0339b989f03cce91b57fb2efba201b7809
2019-02-19 16:31:20 +01:00
MarcoFalke 3e4fd40753
Merge #15415: [test] functional: allow custom cwd, use tmpdir as default
e3e1a5631e [test] functional: set cwd of nodes to tmpdir (Sjors Provoost)

Pull request description:

  Any process launched by bitcoind will have `self.datadir` as its `cwd`.

Tree-SHA512: 0b311643bb96c7dc2f693774620173243b3add40bf373284695af2f0071823b23485289fd2ffe152b7f63e0bfe989b16720077cfc2ce33905f9b8e7f2630f3c0
2019-02-19 10:12:28 -05:00
Sjors Provoost e3e1a5631e
[test] functional: set cwd of nodes to tmpdir 2019-02-19 08:53:51 +01:00
Akio Nakamura 1a062b85f0 tests: remove byte.hex() to keep compatibility
Use test_framework.util.bytes_to_hex_str() instead of bytes.hex() that
new in Python 3.5, to support minimum version of Python(test).
2019-02-19 16:38:44 +09:00
Akio Nakamura 1a7ba84e11 Fix lack of warning of unrecognized section names
1. Fix lack of warning by collecting all section names by moving
   m_config_sections.clear() to ArgsManager::ReadConfigFiles().
2. Add info(file name, line number) to warning message.
3. Add a test code to confirm this situation.
3. Do clear() in ReadConfigString().
2019-02-19 10:49:19 +09:00
MarcoFalke 6ba3f1fdfd
Merge #15397: Remove manual byte editing in wallet_tx_clone func test
6aaa0abc12 Remove manual byte editing in wallet_tx_clone func test (Gregory Sanders)

Pull request description:

  Adapted from @stevenroose

Tree-SHA512: 87f251579e347f870bd30fc57b0c130f00914a3dc78799826384eb049b91d49f2525d55899bf525997e23cc976ca7d10e6b56b23f7358acec307368d48a6f6f1
2019-02-18 09:16:35 -05:00
Amiti Uttarwar 8e4b4f683a Address test todos by removing -txindex to nodes.
Originally added when updating getrawtransaction to stop searching unspent utxos.
2019-02-17 21:42:08 -08:00
Wladimir J. van der Laan f60d029a2a
Merge #15368: Descriptor checksums
fd637be8d2 Add checksums to descriptors.md (Pieter Wuille)
be62903c41 Make descriptor checksums mandatory in deriveaddresses and importmulti (Pieter Wuille)
b52cb63688 Add getdescriptorinfo to compute checksum (Pieter Wuille)
3b40bff988 Descriptor checksum (Pieter Wuille)

Pull request description:

  This adds support for a descriptor-specific 8-character checksum.

  Descriptors may optionally be suffixed with a `#` plus these 8 checksum characters. Any descriptor that contains a `#` at the end must be followed by a valid checksum. If the `#` is missing entirely, it is valid without checksum.

  All RPCs are updated to report descriptors that include the checksum. On input, they are optional except in `deriveaddress` and `importmulti`, which require descriptors which include a checksum.

  A new RPC is also added to analyse descriptors (`getdescriptorinfo`), which can be used to compute the checksum for a descriptor without.

Tree-SHA512: a8294b09155eb6c67fbc178b5e2d3fbc0e9bec8b6de57a13f8835550d51c2cb32a428b3c9a188ded42b454d594e9305edbd4797906b755de77a8f33c79165f6b
2019-02-16 21:39:32 +01:00
Wladimir J. van der Laan d5b929c813
Merge #13932: Additional utility RPCs for PSBT
540729ef4b Implement analyzepsbt RPC and tests (Andrew Chow)
77542cf2a5 Move PSBT UTXO fetching to a separate method (Andrew Chow)
cb40b3abd4 Figure out what is missing during signing (Andrew Chow)
08f749c914 Implement joinpsbts RPC and tests (Andrew Chow)
7344a7b998 Implement utxoupdatepsbt RPC and tests (Andrew Chow)

Pull request description:

  This PR adds 3 new utility RPCs for interacting with PSBTs.

  `utxoupdatepsbt` updates a PSBT with UTXO information from the node. It only works with witness UTXOs because full transactions (as would be needed for non-witness UTXOs) are not available unless txindex is enabled.

  `joinpsbts` joins the inputs from multiple distinct PSBTs into one PSBT. e.g. if PSBT 1 has inputs 1 and 2, and PSBT 2 has inputs 3 and 4, `joinpsbts` would create a new PSBT with inputs 1, 2, 3, and 4.

  `analyzepsbt` analyzes a PSBT and determines the current state of it and all of its inputs, and the next step that needs to be done.

Tree-SHA512: 3c1fa302201abca76a8901d0c2be7b4ccbce334d989533c215f8b3e50e22f2f018ce6209544b26789f58f5980a253c0655111e1e20d47d5656e0414c64891a5c
2019-02-16 20:45:03 +01:00
Andrew Chow 540729ef4b Implement analyzepsbt RPC and tests 2019-02-16 11:51:02 -05:00
Pieter Wuille be62903c41 Make descriptor checksums mandatory in deriveaddresses and importmulti 2019-02-15 22:36:05 -08:00
Pieter Wuille b52cb63688 Add getdescriptorinfo to compute checksum 2019-02-15 22:36:05 -08:00
Pieter Wuille 3b40bff988 Descriptor checksum 2019-02-15 22:36:05 -08:00
Andrew Chow 08f749c914 Implement joinpsbts RPC and tests
Adds a joinpsbts RPC which combines multiple distinct PSBTs into
one PSBT.
2019-02-16 00:04:27 -05:00
Andrew Chow 7344a7b998 Implement utxoupdatepsbt RPC and tests 2019-02-16 00:04:27 -05:00
MarcoFalke eca1273c35
Merge #15383: [rpc] mining: Omit uninitialized currentblockweight, currentblocktx
fa178a6385 [rpc] mining: Omit uninitialized currentblockweight, currentblocktx (MarcoFalke)

Pull request description:

  Previously we'd report "0", which could be mistaken for a valid number. E.g. the number of transactions is 0 or the block weight is 0, whatever that means.

Tree-SHA512: ee94ab203a329e272211b726f4c23edec4b09c650ec363b77fd59ad9264165d73064f78ebb9e11b5c2c543b73c157752410a307655560531c7d5444d203aa0ea
2019-02-15 08:57:50 -05:00
Andrew Chow f4b00b70e8 Import public keys in order
Do public key imports in the order that they are specified in the import
or in the descriptor range.
2019-02-14 18:34:03 -05:00
Andrew Chow 9e1551b9ce Test pubkey import to keypool 2019-02-14 18:33:29 -05:00
Andrew Chow 9b81fd19ac Fetch keys from keypool when private keys are disabled
When private keys are disabled, still fetch keys from the keypool
if the keypool has keys. Those keys come from importing them and
adding them to the keypool.
2019-02-14 18:14:00 -05:00
MeshCollider 8d0ec74801
Merge #14021: Import key origin data through descriptors in importmulti
cb3511b9d Add release notes for importing key origin info change (Andrew Chow)
4c75a69f3 Test importing descriptors with key origin information (Andrew Chow)
02d6586d7 Import KeyOriginData when importing descriptors (Andrew Chow)
3d235dff5 Implement a function to add KeyOriginInfo to a wallet (Andrew Chow)
eab63bc26 Store key origin info in key metadata (Andrew Chow)
345bff601 Remove hdmasterkeyid (Andrew Chow)
bac8c676a Add a method to CWallet to write just CKeyMetadata (Andrew Chow)
e7652d3f6 Add WriteHDKeypath function and move *HDKeypath to util/bip32.{h,cpp} (Andrew Chow)
c45415f73 Refactor keymetadata writing to a separate method (Andrew Chow)

Pull request description:

  This PR allows for key origin data as defined by the descriptors document to be imported to the wallet when importing a descriptor using `importmulti`. This allows the `walletprocesspsbt` to include the BIP 32 derivation paths for keys that it is watching that are from a different HD wallet.

  In order to make this easier to use, a new field `hdmasterkeyfingerprint` has been added to `getaddressinfo`. Additionally I have removed `hdmasterkeyid` as was planned. I think that this API change is fine since it was going to be removed in 0.18 anyways. `CKeyMetadata` has also been extended to store key origin info to facilitate this.

Tree-SHA512: 9c7794f3c793da57e23c5abbdc3d58779ee9dea3d53168bb86c0643a4ad5a11a446264961e2f772f35eea645048cb60954ed58050002caee4e43cd9f51215097
2019-02-15 12:11:28 +13:00
Andrew Chow 4c75a69f36 Test importing descriptors with key origin information 2019-02-14 17:58:25 -05:00
Wladimir J. van der Laan 3facd9fdc4
Merge #14481: Add P2SH-P2WSH support to listunspent RPC
6ca836ab3a Add release note for listunspent P2WSH change (MeshCollider)
928beae007 Add test for P2SH-P2WSH in signrawtransactionwithkey and listunspent (MeshCollider)
314784a60f Make listunspent and signrawtransaction RPCs support witnessScript (MeshCollider)

Pull request description:

  This is a reworked version of #11708 after #12427 and the `signrawtransaction` split.

  For a P2WSH address, listunspent should return the witness script, and for a P2SH-P2WSH address, it should also return the inner witness script (because SignTransaction will automatically wrap it in P2SH if required).

  Includes a test which also tests the behaviour of #12427, and release note.

Tree-SHA512: a8e72cf16930312bf48ec47e44a68f8d7e26664043c1b4cc0983eb25aec4087e511188ff9a0f181cd7b8a0c068c60d7f1e7e3f226b79e8c48890039dcf57f7b7
2019-02-14 22:17:52 +01:00