Commit graph

1385 commits

Author SHA1 Message Date
Wladimir J. van der Laan
d5b929c813
Merge #13932: Additional utility RPCs for PSBT
540729ef4b Implement analyzepsbt RPC and tests (Andrew Chow)
77542cf2a5 Move PSBT UTXO fetching to a separate method (Andrew Chow)
cb40b3abd4 Figure out what is missing during signing (Andrew Chow)
08f749c914 Implement joinpsbts RPC and tests (Andrew Chow)
7344a7b998 Implement utxoupdatepsbt RPC and tests (Andrew Chow)

Pull request description:

  This PR adds 3 new utility RPCs for interacting with PSBTs.

  `utxoupdatepsbt` updates a PSBT with UTXO information from the node. It only works with witness UTXOs because full transactions (as would be needed for non-witness UTXOs) are not available unless txindex is enabled.

  `joinpsbts` joins the inputs from multiple distinct PSBTs into one PSBT. e.g. if PSBT 1 has inputs 1 and 2, and PSBT 2 has inputs 3 and 4, `joinpsbts` would create a new PSBT with inputs 1, 2, 3, and 4.

  `analyzepsbt` analyzes a PSBT and determines the current state of it and all of its inputs, and the next step that needs to be done.

Tree-SHA512: 3c1fa302201abca76a8901d0c2be7b4ccbce334d989533c215f8b3e50e22f2f018ce6209544b26789f58f5980a253c0655111e1e20d47d5656e0414c64891a5c
2019-02-16 20:45:03 +01:00
Andrew Chow
540729ef4b Implement analyzepsbt RPC and tests 2019-02-16 11:51:02 -05:00
Pieter Wuille
be62903c41 Make descriptor checksums mandatory in deriveaddresses and importmulti 2019-02-15 22:36:05 -08:00
Pieter Wuille
b52cb63688 Add getdescriptorinfo to compute checksum 2019-02-15 22:36:05 -08:00
Pieter Wuille
3b40bff988 Descriptor checksum 2019-02-15 22:36:05 -08:00
Andrew Chow
08f749c914 Implement joinpsbts RPC and tests
Adds a joinpsbts RPC which combines multiple distinct PSBTs into
one PSBT.
2019-02-16 00:04:27 -05:00
Andrew Chow
7344a7b998 Implement utxoupdatepsbt RPC and tests 2019-02-16 00:04:27 -05:00
MarcoFalke
eca1273c35
Merge #15383: [rpc] mining: Omit uninitialized currentblockweight, currentblocktx
fa178a6385 [rpc] mining: Omit uninitialized currentblockweight, currentblocktx (MarcoFalke)

Pull request description:

  Previously we'd report "0", which could be mistaken for a valid number. E.g. the number of transactions is 0 or the block weight is 0, whatever that means.

Tree-SHA512: ee94ab203a329e272211b726f4c23edec4b09c650ec363b77fd59ad9264165d73064f78ebb9e11b5c2c543b73c157752410a307655560531c7d5444d203aa0ea
2019-02-15 08:57:50 -05:00
Andrew Chow
f4b00b70e8 Import public keys in order
Do public key imports in the order that they are specified in the import
or in the descriptor range.
2019-02-14 18:34:03 -05:00
Andrew Chow
9e1551b9ce Test pubkey import to keypool 2019-02-14 18:33:29 -05:00
Andrew Chow
9b81fd19ac Fetch keys from keypool when private keys are disabled
When private keys are disabled, still fetch keys from the keypool
if the keypool has keys. Those keys come from importing them and
adding them to the keypool.
2019-02-14 18:14:00 -05:00
MeshCollider
8d0ec74801
Merge #14021: Import key origin data through descriptors in importmulti
cb3511b9d Add release notes for importing key origin info change (Andrew Chow)
4c75a69f3 Test importing descriptors with key origin information (Andrew Chow)
02d6586d7 Import KeyOriginData when importing descriptors (Andrew Chow)
3d235dff5 Implement a function to add KeyOriginInfo to a wallet (Andrew Chow)
eab63bc26 Store key origin info in key metadata (Andrew Chow)
345bff601 Remove hdmasterkeyid (Andrew Chow)
bac8c676a Add a method to CWallet to write just CKeyMetadata (Andrew Chow)
e7652d3f6 Add WriteHDKeypath function and move *HDKeypath to util/bip32.{h,cpp} (Andrew Chow)
c45415f73 Refactor keymetadata writing to a separate method (Andrew Chow)

Pull request description:

  This PR allows for key origin data as defined by the descriptors document to be imported to the wallet when importing a descriptor using `importmulti`. This allows the `walletprocesspsbt` to include the BIP 32 derivation paths for keys that it is watching that are from a different HD wallet.

  In order to make this easier to use, a new field `hdmasterkeyfingerprint` has been added to `getaddressinfo`. Additionally I have removed `hdmasterkeyid` as was planned. I think that this API change is fine since it was going to be removed in 0.18 anyways. `CKeyMetadata` has also been extended to store key origin info to facilitate this.

Tree-SHA512: 9c7794f3c793da57e23c5abbdc3d58779ee9dea3d53168bb86c0643a4ad5a11a446264961e2f772f35eea645048cb60954ed58050002caee4e43cd9f51215097
2019-02-15 12:11:28 +13:00
Andrew Chow
4c75a69f36 Test importing descriptors with key origin information 2019-02-14 17:58:25 -05:00
Wladimir J. van der Laan
3facd9fdc4
Merge #14481: Add P2SH-P2WSH support to listunspent RPC
6ca836ab3a Add release note for listunspent P2WSH change (MeshCollider)
928beae007 Add test for P2SH-P2WSH in signrawtransactionwithkey and listunspent (MeshCollider)
314784a60f Make listunspent and signrawtransaction RPCs support witnessScript (MeshCollider)

Pull request description:

  This is a reworked version of #11708 after #12427 and the `signrawtransaction` split.

  For a P2WSH address, listunspent should return the witness script, and for a P2SH-P2WSH address, it should also return the inner witness script (because SignTransaction will automatically wrap it in P2SH if required).

  Includes a test which also tests the behaviour of #12427, and release note.

Tree-SHA512: a8e72cf16930312bf48ec47e44a68f8d7e26664043c1b4cc0983eb25aec4087e511188ff9a0f181cd7b8a0c068c60d7f1e7e3f226b79e8c48890039dcf57f7b7
2019-02-14 22:17:52 +01:00
Andrew Chow
345bff6013 Remove hdmasterkeyid 2019-02-14 14:00:29 -05:00
MeshCollider
2452c6cc0a
Merge #14978: Factor out PSBT utilities from RPCs for use in GUI code; related refactoring.
102faad81 Factor out combine / finalize / extract PSBT helpers (Glenn Willen)
78b9893d0 Remove op== on PSBTs; check compatibility in Merge (Glenn Willen)
bd0dbe876 Switch away from exceptions in refactored tx code (Glenn Willen)
c6c3d42a7 Move PSBT definitions and code to separate files (Glenn Willen)
81cd95884 Factor BroadcastTransaction out of sendrawtransaction (Glenn Willen)
c734aaa15 Split DecodePSBT into Base64 and Raw versions (Glenn Willen)
162ffefd2 Add pf_invalid arg to std::string DecodeBase{32,64} (Glenn Willen)

Pull request description:

  * Move most PSBT definitions into psbt.h.
  * Move most PSBT RPC utilities into psbt.{h,cpp}.
  * Move wallet-touching PSBT RPC utilities (FillPSBT) into
      wallet/psbtwallet.{h,cpp}.
  * Switch exceptions from JSONRPCError() to new PSBTException class.
  * Split DecodePSBT into DecodeBase64PSBT (old behavior) and DecodeRawPSBT.
  * Add one new version of DecodeBase64 utility in strencodings.h (and
      corresponding DecodeBase32 for completeness).
  * Factor BroadcastTransaction utility function out of sendrawtransaction RPC
      handler in rpc/rawtransaction.cpp

  Note: For those keeping score at home wondering why refactor, this is in anticipation of (and developed in parallel with) a change to actually introduce GUI use of all this stuff, which is already under development and working-ish.

Tree-SHA512: 2197c448e657421f430943025357597e7b06c4c377d5d4b2622b9edea52a7193c48843dd731abb3a88ac4023a9c88d211991e0a9b740c22f2e1cbe72adefe390
2019-02-14 21:49:01 +13:00
Gregory Sanders
6aaa0abc12 Remove manual byte editing in wallet_tx_clone func test 2019-02-13 09:49:36 -05:00
MeshCollider
928beae007 Add test for P2SH-P2WSH in signrawtransactionwithkey and listunspent 2019-02-13 14:35:16 +13:00
MarcoFalke
029d28a7aa
Merge #15238: [QA] remove some magic mining constants in functional tests
b651ef7e1c submitheader: more directly test missing prev block header (Gregory Sanders)
1e7f741745 remove some magic mining constants in functional tests (Gregory Sanders)

Pull request description:

  The fewer magic numbers the better.

  Also more directly tested a `submitheader` case of bad previous blockhash.

Tree-SHA512: 52b01a6aa199fa909eea4e9e84409a901933e545724e33149cc4132c82168199fd678809b6d94d95c9ff6ad02238a9552363620d13b8beaa5d4b67ade9ef425c
2019-02-12 16:52:06 -05:00
MarcoFalke
ea022d9fd0
Merge #15378: tests: Added missing tests for RPC wallet errors
dc3b2ccb5f tests: Added missing tests for RPC wallet errors (Ben Carman)

Pull request description:

Tree-SHA512: b18dcd4f7547c974c93ae67dcd92a168bdb55951b164cf174cb1e59e0daa463187068aec43108309a75d65721a5c0bcdf10a16a9869620f160121e2287559926
2019-02-12 15:27:15 -05:00
MarcoFalke
fa178a6385
[rpc] mining: Omit uninitialized currentblockweight, currentblocktx 2019-02-12 11:34:57 -05:00
MarcoFalke
d8794a78a8
Merge #13787: Test for Windows encoding issue
15b31032a6 appveyor: Remove outdated libraries (Chun Kuan Lee)
ae892adceb tests: accept unicode characters on Windows (Chun Kuan Lee)

Pull request description:

  This PR removes unused Windows compatible path. Encoding issues has been solved.

Tree-SHA512: d24dce035fe3b576eaa002896f815a6691e88618ea371171d7e39883e1d63b3ed4d4631ab3b3a26bd67ae4e8ae13b1ff188942ab7ac6a93d704e0de301368cd0
2019-02-12 11:15:11 -05:00
MarcoFalke
03732f8644
Merge #14543: [QA] minor p2p_sendheaders fix of height in coinbase
1cdb9bb51f minor p2p_sendheaders fix of height in coinbase (Gregory Sanders)

Pull request description:

  > \# Now announce a header that forks the last two blocks

  Doesn't effect any behavior since BIP34 isn't active in regtest for many blocks.

Tree-SHA512: 3f214b956a94250bb640f63b6ff707930e1d4cb8df1bbf0fef4012d89a94bafbde0d7b42bbe7113ec33810169281c22c6e389445921d99decb74aa56e87a0f27
2019-02-12 11:10:02 -05:00
Ben Carman
dc3b2ccb5f
tests: Added missing tests for RPC wallet errors 2019-02-11 18:19:22 -06:00
Chun Kuan Lee
ae892adceb tests: accept unicode characters on Windows 2019-02-12 05:17:15 +08:00
Glenn Willen
c734aaa15d Split DecodePSBT into Base64 and Raw versions
Split up DecodePSBT, which both decodes base64 and then deserializes a
PartiallySignedTransaction, into two functions: DecodeBase64PSBT, which retains
the old behavior, and DecodeRawPSBT, which only performs the deserialization.

Add a test for base64 decoding failure.
2019-02-11 12:23:14 -08:00
MarcoFalke
e84e0d4b5e
Merge #15337: rpc: Fix for segfault if combinepsbt called with empty inputs
30d0f7be6e rpc: Fix for segfault if combinepsbt called with empty inputs (benthecarman)

Pull request description:

  Fixes #15300

Tree-SHA512: 25e7b4e6e48d8b0d197f0ab96df308fff33e2110f8929cb48914877fa7f4c4a84f173b1378fdb2dec5d03fe7d6d1aced4b577e55f9fe180d8147d9106ebf543f
2019-02-11 08:08:17 -05:00
MeshCollider
6f4e0d1542
Merge #15226: Allow creating blank (empty) wallets (alternative)
7687f7873 [wallet] Support creating a blank wallet (Andrew Chow)

Pull request description:

  Alternative (kind of) to #14938

  This PR adds a `blank` parameter to the `createwallet` RPC to create a wallet that has no private keys initially. `sethdseed` can then be used to make a clean wallet with a custom seed. `encryptwallet` can also be used to make a wallet that is born encrypted.

  Instead of changing the version number as done in #14938, a wallet flag is used to indicate that the wallet should be blank. This flag is set at creation, and then unset when the wallet is no longer blank. A wallet becomes non-blank when a HD seed is set or anything is imported. The main change to create a blank wallet is primarily taken from #14938.

  Also with this, the term "blank wallet" is used instead of "empty wallet" to avoid confusion with wallets that have balance which would also be referred to as "empty".

  This is built on top of #15225 in order to fix GUI issues.

Tree-SHA512: 824d685e11ac2259a26b5ece99c67a7bda94a570cd921472c464243ee356b7734595ad35cc439b34357135df041ed9cba951e6edac194935c3a55a1dc4fcbdea
2019-02-11 08:08:33 +13:00
Andrew Chow
7687f7873b [wallet] Support creating a blank wallet
A blank wallet is a wallet that has no keys, script or watch only things.
A new wallet flag indicating that it is blank will be set when the wallet
is blank. Once it is no longer blank (a seed has been generated, keys or
scripts imported, etc), the flag will be unset.
2019-02-10 12:24:53 -05:00
MarcoFalke
1477933f28
Merge #15370: test_runner: Remove unused --force option
fae8b8bb1a qa: Add tool-prefix to functional test readme (MarcoFalke)
faf3d22725 test_runner: Remove unused --force option (MarcoFalke)

Pull request description:

  When someone calls the script they already have all intention to call it, no need to specify a redundant `--force`.

  The functional tests are still disabled on the travis windows cross builds, where they'd run into issues when run under Wine.

Tree-SHA512: ada0dd9b3c0cd28c5832a12c5e04c029dc3bfe5ddf366fd0abc24fb7914d2e0f0a873fe756ade7ba780a561abe9bc731838c289accc421deda481269e08514cd
2019-02-08 10:40:34 -05:00
MarcoFalke
fae8b8bb1a
qa: Add tool-prefix to functional test readme 2019-02-08 08:33:07 -05:00
MarcoFalke
faf3d22725
test_runner: Remove unused --force option 2019-02-07 19:16:57 -05:00
Wladimir J. van der Laan
9127bd7aba
Merge #14491: Allow descriptor imports with importmulti
b985e9c850 Add release notes for importmulti descriptor support (MeshCollider)
fbb5e935ea Add test for importing via descriptor (MeshCollider)
9f48053d8f [wallet] Allow descriptor imports with importmulti (MeshCollider)
d2b381cc91 [wallet] Refactor ProcessImport() to call ProcessImportLegacy() (John Newbery)
4cac0ddd25 [wallet] Add ProcessImportLegacy() (John Newbery)
a1b25e12a5 [wallet] Refactor ProcessImport() (John Newbery)

Pull request description:

  ~~Based on #14454 #14565, last two commits only are for review.~~

  Best reviewed with `?w=1`

  Allows a descriptor to be imported into the wallet using `importmulti` RPC. Start and end of range can be specified for ranged descriptors. The descriptor is implicitly converted to old structures on import.

  Also adds a simple test of a P2SH-P2WPKH address being imported as a descriptor. More tests to come, as well as release notes.

Tree-SHA512: 160eb6fd574c4ae5b70e0109f7e5ccc95d9309138603408a1114ceb3c558065409c0d7afb66926bc8e1743c365a3b300c5f944ff18b2451acc0514fbeca1f2b3
2019-02-07 22:43:33 +01:00
MeshCollider
1933e38c1a
Merge #14667: Add deriveaddresses RPC util method
595283851 [rpc] util: add deriveaddresses method (Sjors Provoost)

Pull request description:

  Usage:

  ```sh
  bitcoin-cli deriveaddresses "wpkh([d34db33f/84h/0h/0h]xpub6DJ2dNUysrn5Vt36jH2KLBT2i1auw1tTSSomg8PhqNiUtx8QX2SvC9nrHu81fT41fvDUnhMjEzQgXnQjKEu3oaqMSzhSrHMxyyoEAmUHQbY/0/0)"
  [
    "bc1qg6ucjz7kgdedam7v5yarecy54uqw82yym06z3q"
  ] // part of the BIP32 test vector
  ```

  Avoids the need for external (BIP32) libraries to derive an address. Can be used in conjunction with `scantxoutset` as a poor mans wallet. Might be useful to test more complicated future descriptors.

  ~To keep it as simple as possible it only supports descriptors that result in a single address, so no `combo()` and ranges.~

  As discussed recently on IRC it might make sense to put this in a separate utility along with other descriptor and psbt utility functions which don't need a chain or wallet context. However I prefer to leave that to another PR.

Tree-SHA512: b8e53db11a8fd87638cc98766270cc3be9adc4b3e5085798a6a4e2e6ad252bf6d2189346bbb2da72d04d13f7f1e80b5cb88e8039653bea1f150602a876ef7f34
2019-02-08 08:21:52 +13:00
João Barbosa
6440e61375 qa: Drop RPC connection if --usecli 2019-02-06 00:04:28 +00:00
MarcoFalke
5029e94f85
Merge #14519: tests: add utility to easily profile node performance with perf
13782b8ba8 docs: add perf section to developer docs (James O'Beirne)
58180b5fd4 tests: add utility to easily profile node performance with perf (James O'Beirne)

Pull request description:

  Adds a context manager to easily (and selectively) profile node performance during functional test execution using `perf`.

  While writing some tests, I encountered some odd bitcoind slowness. I wrote up a utility (`TestNode.profile_with_perf`) that generates performance diagnostics for a node by running `perf` during the execution of a particular region of test code.

  `perf` usage is detailed in the excellent (and sadly unmerged) https://github.com/bitcoin/bitcoin/pull/12649; all due props to @eklitzke.

  ### Example

  ```python
  with node.profile_with_perf("large-msgs"):
      for i in range(200):
          node.p2p.send_message(some_large_msg)
      node.p2p.sync_with_ping()
  ```

  This generates a perf data file in the test node's datadir (`/tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data`).

  Running `perf report` generates nice output about where the node spent most of its time while running that part of the test:

  ```bash
  $ perf report -i /tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data --stdio \
    | c++filt \
    | less

  # To display the perf.data header info, please use --header/--header-only options.
  #
  #
  # Total Lost Samples: 0
  #
  # Samples: 135  of event 'cycles:pp'
  # Event count (approx.): 1458205679493582
  #
  # Children      Self  Command          Shared Object        Symbol
  # ........  ........  ...............  ...................  ........................................................................................................................................................................................................................................................................
  #
      70.14%     0.00%  bitcoin-net      bitcoind             [.] CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
                  |
                  ---CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

      70.14%     0.00%  bitcoin-net      bitcoind             [.] CNetMessage::readData(char const*, unsigned int)
                  |
                  ---CNetMessage::readData(char const*, unsigned int)
                     CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

      35.52%     0.00%  bitcoin-net      bitcoind             [.] std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
                  |
                  ---std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
                     CNetMessage::readData(char const*, unsigned int)
                     CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

  ...
  ```

Tree-SHA512: 9ac4ceaa88818d5eca00994e8e3c8ad42ae019550d6583972a0a4f7b0c4f61032e3d0c476b4ae58756bc5eb8f8015a19a7fc26c095bd588f31d49a37ed0c6b3e
2019-02-05 17:40:16 -05:00
MeshCollider
fbb5e935ea Add test for importing via descriptor 2019-02-05 19:42:05 +13:00
benthecarman
30d0f7be6e rpc: Fix for segfault if combinepsbt called with empty inputs 2019-02-04 21:26:52 -06:00
João Barbosa
d3bf3b930d qa: Test .walletlock file is closed 2019-02-04 18:50:21 +00:00
MarcoFalke
fac3a054cb
test: Fix race in p2p_invalid_messages 2019-02-02 17:45:20 -05:00
MarcoFalke
5c99bb0047
Merge #15247: qa: Use wallet to retrieve raw transactions
fa5278a419 qa: Use wallet to retrieve raw transactions (MarcoFalke)
fa2198328e qa: Style-only fixes in touched files (MarcoFalke)

Pull request description:

  Instead of asking the coin database and block storage about a transaction, pull it directly from the wallet in wallet related tests.

  This refactoring only makes sense in light of #15159.

  <sub>This product may contain minor stylistic cleanups

Tree-SHA512: ec34c7150d873da9f19fead3f7e3f758baba5ef10061942384c470a47a6f320690109be9c5160f0c8bc228272a729653d44c78471455337318f657d6c164ba23
2019-02-01 09:35:14 -05:00
Wladimir J. van der Laan
3e38d40873
Merge #15235: Do not import private keys to wallets with private keys disabled
e6c58d3b01 Do not import private keys to wallets with private keys disabled (Andrew Chow)
b5c5021b64 Refactor importwallet to extract data from the file and then import (Andrew Chow)
1f77f6754c tests: unify RPC argument to cli argument conversion and handle dicts and lists (Andrew Chow)

Pull request description:

  Fixes a bug where private keys could be imported to wallets with private keys disabled. Now every RPC which can import private keys checks for whether the wallet has private keys are disabled and errors if it is. Also added an belt-and-suspenders check to `AddKeyPubkeyWithDB` to have it assert that the wallet has private keys enabled.

Tree-SHA512: 5cd04febce9aa2bd9bfd02f312c6ff8705e37278cae59efd3895f6d6e2f1b477aefd297e2dd0860791bdd3d4f3cad8eb1a404f8f3d4e2035b91314ad2c1028ae
2019-02-01 14:01:32 +01:00
Andrew Chow
e6c58d3b01 Do not import private keys to wallets with private keys disabled 2019-01-31 14:29:28 -05:00
Andrew Chow
2e023419c5 tests: unify RPC argument to cli argument conversion and handle dicts and lists
When running tests with --usecli, unify the conversion from argument objects to
strings using a new function arg_to_cli(). This fixes boolean arguments when
using named arguments.

Also use json.dumps() to get the string values for arguments that are dicts and
lists so that bitcoind's JSON parser does not become confused.
2019-01-31 12:37:33 -05:00
MarcoFalke
252fd15add
Merge #13926: [Tools] bitcoin-wallet - a tool for creating and managing wallets offline
3c3e31c3a4 [tests] Add wallet-tool test (João Barbosa)
49d2374acf [tools] Add wallet inspection and modification tool (Jonas Schnelli)

Pull request description:

  Adds an offline tool `bitcoin-wallet-tool` for wallet creation and maintenance.

  Currently this tool can create a new wallet file, display information on an existing wallet, and run the salvage and zapwallettxes maintenance tasks on an existing wallet. It can later be extended to support other common wallet maintenance tasks.

  Doing wallet maintenance tasks in an offline tool makes much more sense (and is potentially safer) than having to spin up a full node.

Tree-SHA512: 75a28b8a58858d9d76c7532db40eacdefc5714ea5aab536fb1dc9756e2f7d750d69d68d59c50a68e633ce38fb5b8c3e3d4880db30fe01561e07ce58d42bceb2b
2019-01-31 11:07:51 -05:00
Wladimir J. van der Laan
36aeb43c01
Merge #15246: qa: Add tests for invalid message headers
fa3745bda8 qa: Add tests for invalid message headers (MarcoFalke)

Pull request description:

Tree-SHA512: b37e297cfd65a33a7af201f750a303cf437b438e40d38b1d2f562ccde67082616daa110ca1e5e3af6514ea4ca4b115362acf2ffa6263cea3c8e8189ce02dda67
2019-01-31 14:25:33 +01:00
João Barbosa
3c3e31c3a4 [tests] Add wallet-tool test
Original tests by João Barbosa <joao.paulo.barbosa@gmail.com>

Additional contribution by John Newbery <john@johnnewbery.com>
2019-01-30 16:26:56 -05:00
Wladimir J. van der Laan
77339e5c24
Merge #15163: Correct units for "-dbcache" and "-prune"
6f6514a080 Correct units for "-dbcache" and "-prune" (Hennadii Stepanov)

Pull request description:

  Actually, all `dbcache`-related values in the code are measured in MiB (not in megabytes, MB) or in bytes (e.g., `nTotalCache`).

  See: https://github.com/bitcoin/bitcoin/blob/master/src/txdb.h

  ba8c8b2227/src/init.cpp (L1405-L1424)

  Also, "-prune" is fixed:
  1. The GUI values in GB are translated to the node values in MiB correctly.
  2. The maximum of the "prune" `QSpinBox` is not limited by default value of 99 (GB).

  Fix: #15106

Tree-SHA512: 151ec43b31b1074db8b345fedb1dcc10bde225899a5296bfc183f57e1553d13ac27db8db100226646769ad03c9fcab29d88763065a471757c6c41ac51108459d
2019-01-30 20:22:57 +01:00
MarcoFalke
a47319dada
Merge #15159: [RPC] Remove lookup to UTXO set from GetTransaction
04da9f4834 [RPC] Update getrawtransaction interface (Amiti Uttarwar)

Pull request description:

  - stop checking unspent UTXOs for a transaction when txindex is not enabled, as per conversation here: https://github.com/bitcoin/bitcoin/issues/3220#issuecomment-377458383
  - code contributed by sipa

Tree-SHA512: aa07353bccc14b81b7803992a25d076d6bc06d15ec7c1b85828dc10aea7e0498d9b49f71783e352ab8a14b0bb2010cfb7835de3dfd1bc6f2323f460449348e66
2019-01-30 11:18:44 -05:00
Hennadii Stepanov
6f6514a080
Correct units for "-dbcache" and "-prune"
All dbcache-related values in the code are measured in MiB (not in
megabytes, MB) or in bytes.
The GUI "-prune" values in GB are translated to the node values in MiB
correctly. The maximum of the "-prune" QSpinBox is not limited by the
default value of 99 (GB).
Also, this improves log readability.
2019-01-30 07:17:22 +02:00
Sjors Provoost
595283851d
[rpc] util: add deriveaddresses method 2019-01-29 18:14:23 +01:00
MarcoFalke
fa5278a419
qa: Use wallet to retrieve raw transactions 2019-01-26 22:43:56 -05:00
Amiti Uttarwar
04da9f4834 [RPC] Update getrawtransaction interface 2019-01-26 18:36:53 -08:00
MarcoFalke
fa2198328e
qa: Style-only fixes in touched files 2019-01-25 11:29:10 -05:00
Ben Woosley
f618c58b75
Docs: Update python docs to reflect that wildcard imports are disallowed 2019-01-24 16:57:41 -08:00
MarcoFalke
fa3745bda8
qa: Add tests for invalid message headers 2019-01-24 17:08:22 -05:00
Andrew Chow
1f77f6754c tests: unify RPC argument to cli argument conversion and handle dicts and lists
When running tests with --usecli, unify the conversion from argument objects to
strings using a new function arg_to_cli(). This fixes boolean arguments when
using named arguments.

Also use json.dumps() to get the string values for arguments that are dicts and
lists so that bitcoind's JSON parser does not become confused.
2019-01-24 14:07:09 -05:00
Gregory Sanders
b651ef7e1c submitheader: more directly test missing prev block header 2019-01-24 09:48:34 -05:00
Gregory Sanders
1e7f741745 remove some magic mining constants in functional tests 2019-01-24 09:48:34 -05:00
Wladimir J. van der Laan
72bd4ab867
Merge #15193: Default -whitelistforcerelay to off
a36d97d866 Default -whitelistforcerelay to off (Suhas Daftuar)

Pull request description:

  No one seems to use this "feature", and at any rate the behavior of relaying transactions when they violate local policy is error-prone, if we ever consider changing the ban behavior of our software from one version to the next.

  Defaulting this to off means that users who use -whitelist won't be unexpectedly surprised by this interaction.  If anyone is still relying on this feature, it can still be explicitly turned on.

Tree-SHA512: 52650ad464a728d1648f496751e3f713077ea3a1de7278ed03531b2e8723e63cf2f6f41b56c98c0f73ffa22c36e01d9170b409ab452c737aca35b7ecd7a6b448
2019-01-24 15:25:17 +01:00
Jonas Schnelli
82cf6813a4
Merge #14353: REST: add blockhash call, fetch blockhash by height
42ff30ec6 [Docs] add short documentation for /rest/blockhashbyheight (Jonas Schnelli)
579d418f7 [QA] add rest tests for /rest/blockhashbyheight/<HEIGHT>.<FORMAT> (Jonas Schnelli)
eb9ef04c4 REST: add "blockhashbyheight" call, fetch blockhash by height (Jonas Schnelli)

Pull request description:

  Completes the REST interface for trivial block exploring by adding a call that allows to fetch the blockhash in the main chain by a given height.

Tree-SHA512: 94be9e56718f857279b11cc16dfa8d04f3b5a762e87ae54281b4d87247c71c844895f4944d5a47f09056bf851f4c4761ac4fbdbaaee957265d14de5c1c73e8d2
2019-01-22 19:59:02 -10:00
Suhas Daftuar
a36d97d866 Default -whitelistforcerelay to off 2019-01-22 12:18:45 -05:00
James O'Beirne
58180b5fd4 tests: add utility to easily profile node performance with perf
Introduces `TestNode.profile_with_perf()` context manager which
samples node execution to produce profiling data.

Also introduces a test framework flag, `--perf`, which will run
perf on all nodes for the duration of a given test.
2019-01-22 08:55:55 -05:00
Jonas Schnelli
579d418f74
[QA] add rest tests for /rest/blockhashbyheight/<HEIGHT>.<FORMAT> 2019-01-21 11:54:54 -10:00
Wladimir J. van der Laan
72506ed349
Merge #15177: rest: Improve tests and documention of /headers and /block
7cf994d5cf qa: Improve tests of /rest/headers and /rest/block (João Barbosa)
0825b86b28 doc: /rest/block responds with 404 if block does not exist (João Barbosa)
be625f7c55 doc: Explain empty result of /rest/headers (João Barbosa)

Pull request description:

  Follow up of #15107.

Tree-SHA512: a7fdeed05216e3eda9604664db529237c2d0ddf422cfac139d6345a22b6e00bfe870d4e3f177423db7d4efb295ac2dc0ca2eb20c9c27c0719b89fd5428860d03
2019-01-21 17:37:55 +01:00
João Barbosa
7cf994d5cf qa: Improve tests of /rest/headers and /rest/block 2019-01-16 14:46:09 +00:00
Wladimir J. van der Laan
64ee94356f
Merge #14409: utils and libraries: Make 'blocksdir' always net specific
e4a0c3547e Improve blocksdir functional test. (Hennadii Stepanov)
c3f1821ac7 Make blockdir always net specific (Hennadii Stepanov)

Pull request description:

  The blocks directory is net specific by definition.

  Also this prevents the side effect of calling `GetBlocksDir(false)` in the non-mainnet environment.
  Currently a new node creates an unused `blocks\` directory in the root of the data directory when `-testnet` or `-regtest` is specified.

  Refs:
  - #12653
  - https://github.com/bitcoin/bitcoin/pull/12653#discussion_r174784834 by @laanwj
  - https://github.com/bitcoin/bitcoin/issues/14595#issuecomment-436011186

Tree-SHA512: c9957a68a4a200ebd2010823a56db7e61563afedcb7c9828e86b13f3af2990e07854b622c1f3374756f94574acb3ea32de7d2a399eef6c0623f0e11265155627
2019-01-16 13:40:27 +01:00
Wladimir J. van der Laan
3ae3748ce1
Merge #14958: qa: Remove race between connecting and shutdown on separate connections
4412a59bfe qa: Remove race between connecting and shutdown on separate connections (João Barbosa)

Pull request description:

  Fixes the error https://github.com/bitcoin/bitcoin/pull/14670#issuecomment-447255352 reported by @ken2812221.

  There is a race between RPC stop and another concurrent call in the test framework. The connection must be established and the command `waitfornewblock` running before calling `stop`.

  See also https://github.com/bitcoin/bitcoin/pull/14670#issuecomment-447304513.

Tree-SHA512: 77feb8628d3b9c025ec0cf83565d4d6680cad4fb182fc93a65df8b573f3e799ba4c44e06d9001dd8a375ca0b1ee17f10e66c3902b6256d0ae2acbc64539185d7
2019-01-16 13:05:32 +01:00
João Barbosa
4412a59bfe qa: Remove race between connecting and shutdown on separate connections 2019-01-16 12:01:42 +00:00
Sjors Provoost
04215eb328
[doc] explain what .python-version does 2019-01-15 19:58:11 +01:00
Wladimir J. van der Laan
cf0c67b62c
Merge #14982: rpc: Add getrpcinfo command
a0ac15459a doc: Add getrpcinfo release notes (João Barbosa)
251a91c1bf qa: Add tests for getrpcinfo (João Barbosa)
d0730f5ce4 rpc: Add getrpcinfo command (João Barbosa)
068a8fc05f rpc: Track active commands (João Barbosa)
bf4383277d rpc: Remove unused PreCommand signal (João Barbosa)

Pull request description:

  The new `getrpcinfo` command exposes details of the RPC interface. The details can be configuration properties or runtime values/stats.

  This can be particular useful to coordinate concurrent functional tests (see #14958 from where this was extracted).

Tree-SHA512: 7292cb6087f4c429973d991aa2b53ffa1327d5a213df7d6ba5fc69b01b2e1a411f6d1609fed9234896293317dab05f65064da48b8f2b4a998eba532591d31882
2019-01-14 18:07:15 +01:00
Wladimir J. van der Laan
cebe910718
Merge #15039: wallet: Avoid leaking nLockTime fingerprint when anti-fee-sniping
fa48baf23e wallet: Avoid leaking locktime fingerprint when anti-fee-sniping (MarcoFalke)
453803adc9 [test] wallet_txn_clone: Correctly clone txin sequence (MarcoFalke)

Pull request description:

  The wallet sets the locktime to the current height of our active chain. This is fine, as long as our node is connected to other nodes. However, when we fall back and get stuck at a particular height (e.g. taking the wallet offline), the same (potentially unique) locktime is used for all transactions. This makes it easier for passive observers to cluster transactions by wallet.

  For reference, I visualized "locktime-reuse" with the data:
  * blocks 545k-555k (both inclusive)
  * locktimes<=60k
  * excluding coinbase txs

  ![distribution of height-based tx locktimes used at least twice](https://user-images.githubusercontent.com/6399679/50446163-b8256d80-0913-11e9-9832-40b76052b2b9.png)

Tree-SHA512: 2af259dd8f9f863312e2732d80ca8ba6a20c8d6d1c486b10a48479e1c85ccf13b0c38723740ebadde0f28d321cd9c133ad3e5d1e925472eb27681143bda2d0e7
2019-01-10 15:56:54 +01:00
MarcoFalke
1cfbb16b5d
Merge #15108: [tests] tidy up wallet_importmulti.py
2d5f1ea2e3 [tests] move wallet util functions to wallet_util.py (John Newbery)
6be64ef02c [tests] tidy up wallet_importmulti.py (John Newbery)

Pull request description:

  Cherry picks un-merged commits from #14952, which "fixes review comments from @ryanofsky here: https://github.com/bitcoin/bitcoin/pull/14886#pullrequestreview-183772779"

Tree-SHA512: 5f389196b0140d013a533d500f1812786a3a5cfb65980e13eaeacc459fddb55f43d05da3ab5e7cc8c997f26c0b667eed081ab6de2d125e631c70a7dd4c06e350
2019-01-09 16:58:46 -05:00
Wladimir J. van der Laan
ff7f7364d6
Merge #15087: Error if rpcpassword contains hash in conf sections
8cff83124b Error if rpcpassword contains hash in conf sections (MeshCollider)

Pull request description:

  Fixes #15075

Tree-SHA512: 08ba2a2e9a7ea228fc0e0ff9aa76da1fecbe079e3b388304a28b6399e338a4b3a38b03ab03aca880e75f14a8d2ba75ceb31a385d7989cd66db5193a79f32c4e5
2019-01-09 15:08:26 +01:00
MeshCollider
8cff83124b Error if rpcpassword contains hash in conf sections 2019-01-09 17:32:35 +13:00
Wladimir J. van der Laan
c6806ee869
Merge #15059: test: Add basic test for BIP34
fab17e8272 test: Add basic test for BIP34 (MarcoFalke)

Pull request description:

  BIP34 was disabled for testing, which explains why it had no test.

  Fix that by enabling it and adding a test.

Tree-SHA512: 9cb5702d474117ce6420226eb93ee09d6fb5fc856fabc8b67abe56a088cd727674e0e5462000e1afa83b911374036f90abdbdde56a8c236a75572ed47e10a00f
2019-01-08 15:53:39 +01:00
Wladimir J. van der Laan
1973257da0
Merge #15057: [rpc] Correct reconsiderblock help text, add test
fa38d3df69 [rpc] Correct reconsiderblock help text, add test (MarcoFalke)

Pull request description:

  Rework documentation and test to match the implementation

Tree-SHA512: d0adef6b054a341bcc1cb87783a4e4cf9be124ba6812e1ac88246a5e01b2861a8071b12dba880b2b428c37da3fa860bfec3fe3e5fbb7c28696872113faa84a9f
2019-01-07 15:56:11 +01:00
MarcoFalke
fac4e731a8
test: Run invalid_txs.InputMissing test in feature_block 2019-01-05 14:51:25 +01:00
John Newbery
2d5f1ea2e3 [tests] move wallet util functions to wallet_util.py
Adds a new wallet_util.py module and moves generic helper functions
there:

- get_key
- get_multisig
- test_address
2019-01-04 17:49:17 -08:00
John Newbery
6be64ef02c [tests] tidy up wallet_importmulti.py
Fixes review comments from PR 14886.
2019-01-04 17:46:33 -08:00
Wladimir J. van der Laan
df894fa69a
Merge #14457: test: add invalid tx templates for use in functional tests
59e387705c test: add invalid tx templates for use in functional tests (James O'Beirne)

Pull request description:

  This change adds a list of `CTransaction`-generating templates which each correspond to a specific type of invalid transaction. We then use this list to test for a wider variety of invalid tx types in `p2p_invalid_tx.py` and `feature_block.py`.

  Consolidating all invalid tx types will allow us to more easily cover all tx reject cases from a variety of tests without repeating ourselves. Validation logic doesn't differ much between mempool and block acceptance, but there *is* a difference and we should be sure we're testing both comprehensively.

  Right now, I've only added templates covering the tx reject types listed below but if this approach seems worthwhile I will expand the list to be fully comprehensive.
  ```
  bad-txns-in-belowout
  bad-txns-inputs-duplicate
  bad-txns-too-many-sigops
  bad-txns-vin-empty
  bad-txns-vout-empty
  bad-txns-vout-negative
  ```

Tree-SHA512: 05407f4a953fbd7c44c08bb49bb989cefd39a2b05ea00f5b3c92197a3f05e1b302f789e33832445734220e1c333d133aba385740b77b84139b170c583471ce20
2019-01-02 14:44:14 +01:00
Wladimir J. van der Laan
c3bb936a0f
Merge #14790: Tests: Allow running rpc_bind.py --nonloopback test without IPv6
c9066f07c9 Allow running rpc_bind.py --nonloopback test without IPv6 (Kristaps Kaupe)

Pull request description:

  Don't see a reason why this can't be tested with IPv4 only.

Tree-SHA512: 515bdf700fad420e4b1798fd4978b53e2da3ddb26e43b16d68b43071bc912c325f1ceb10046ba3d0494dab289a53c45ddc2de9064117d8c1d6bf11e88323f490
2019-01-02 14:16:12 +01:00
João Barbosa
251a91c1bf qa: Add tests for getrpcinfo 2019-01-02 12:47:32 +00:00
Ben Woosley
de23739b22
test: Fix rpc_net.py "pong" race condition
Prior to this change, the test fails with KeyError if pong has
a zero value at the time this is called, as getpeerinfo's
bytesrecv_per_msg result excludes zero-values.

https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/21310881#L62
2019-01-01 13:12:41 -08:00
MarcoFalke
fa38d3df69
[rpc] Correct reconsiderblock help text, add test 2019-01-01 18:02:12 +01:00
MarcoFalke
e756eca9e8
Merge #15054: Update copyright headers to 2018
1a49a0e310 Bump manpages (DrahtBot)
06ba77973e Update copyright headers to 2018 (DrahtBot)

Pull request description:

  * `./contrib/devtools/copyright_header.py update ./`
  * `./contrib/devtools/gen-manpages.sh`

Tree-SHA512: ca0dc5e97f4c33814d4ccd17769bbf2d23a99a71d62534fe1064fedfe47de3b5c30caf9b6deb0d70bf125e08c7ae6335ac4fcded918049d6b63b13b319d798e3
2018-12-31 12:24:27 +01:00
MarcoFalke
2741b2b6f4
Merge #15026: [test] Rename rpc_timewait to rpc_timeout
4999992c34 whitespace: Split ~300 char line into multiple ones (MarcoFalke)
fa71b38168 scripted-diff: Rename rpc_timewait to rpc_timeout (MarcoFalke)
fa3e5786d0 scripted-diff: Remove unused 'split' parameter to setup_network (MarcoFalke)

Pull request description:

  This is a bugfix, since wallet_dump currently uses the wrong name:

  18857b4c40/test/functional/wallet_dump.py (L89-L92)

  Rename all to the same name with a scripted diff (and some unrelated cleanups).

Tree-SHA512: 338ddd20dae12e6cf7aa7adbcfb239cf648017a1572b373f8431fecb184bd2a65492846d81e75a023864d9e41c94afb53044c16b79651a5937d34a5a6b772f81
2018-12-29 20:18:50 +01:00
MarcoFalke
fab17e8272
test: Add basic test for BIP34 2018-12-29 19:39:37 +01:00
MarcoFalke
96d912bea8
Merge #14738: Tests: Fix running wallet_listtransactions.py individually through test_runner.py
2474de0265 Fix running individually through test_runner.py, as suggested by @MarcoFalke (#14732) (Kristaps Kaupe)

Pull request description:

  As suggested by @MarcoFalke. Resolves #14732.

Tree-SHA512: b4e400ba06075e218dbd97d0390845f1c55be42a2b6fd70513381318cfc2693473ba1d0f9d7f379a96939d1960b53801fad7c02e06bddc50c5a835ad024c37ef
2018-12-29 18:19:27 +01:00
MarcoFalke
a1fd876122
Merge #15022: tests: Upgrade Travis OS to Xenial
b6f0db69a9 Increase timeout of featuer_assumevalid test to fix flaky tests (Graham Krizek)
aa9aca85f1 If tests are ran with (ASan + LSan), Docker needs access to ptrace (Graham Krizek)
a3b8b43663 Update Travis base OS to Xenial (Graham Krizek)

Pull request description:

  Update base Travis OS to `xenial` from `trusty`.

  Link to Travis Docs for Xenial: https://docs.travis-ci.com/user/reference/xenial/

  As noted in the documentation, Docker version is also updated from `17.06` to `18.06`

  Also includes:
      - If running Bitcoin config with LSan sanitizer, Allow ptrace in Docker run command
      - Increase timeout of feature_assumevalid test to fix flaky tests

Tree-SHA512: baf2eda0cbb9990c43c76de1aebc8dd4a3f540323ac1fe2e164ac3bcf1fe3afa3e5b026bfeb5d650dae09a6854695d5744c1130c2fa82ece86c6835ba152f68d
2018-12-29 14:37:59 +01:00
DrahtBot
06ba77973e Update copyright headers to 2018 2018-12-29 10:15:01 +01:00
Graham Krizek
b6f0db69a9
Increase timeout of featuer_assumevalid test to fix flaky tests 2018-12-28 08:28:38 -06:00
MarcoFalke
e4850ce110
Merge #14930: test: pruning: Check that verifychain can be called when pruned
fa3e874d69 test: pruning: Check that verifychain can be called when pruned (MarcoFalke)

Pull request description:

Tree-SHA512: b921e550fdad27b791665883ed9fc50210575b0f968840a45284ad0e060ddd443ea7b9b12c17916f68ad28877c58ac51a6acc76ae4ea71bf7bfb2f744b286884
2018-12-27 21:56:12 +01:00
MarcoFalke
fa48baf23e
wallet: Avoid leaking locktime fingerprint when anti-fee-sniping 2018-12-27 13:03:10 +01:00
MarcoFalke
453803adc9 [test] wallet_txn_clone: Correctly clone txin sequence 2018-12-27 00:25:52 +01:00
MeshCollider
f8a3ab3b29
Merge #14565: Overhaul importmulti logic
eacff95de Add release notes (Pieter Wuille)
bdacbda25 Overhaul importmulti logic (Pieter Wuille)

Pull request description:

  This is an alternative to #14558 (it will warn when fields are being ignored). In addition:
  * It makes sure no changes to the wallet are made when an error in the input exists.
  * It validates all arguments, and will fail if anything fails to parse.
  * Adds a whole bunch of sanity checks

Tree-SHA512: fdee0b6aca8c643663f0bc295a7c1d69c1960951493b06abf32c58977f3e565f75918dbd0402dde36e508dc746c9310a968a0ebbacccc385a57ac2a68b49c1d0
2018-12-24 23:26:17 +13:00
MarcoFalke
4999992c34
whitespace: Split ~300 char line into multiple ones 2018-12-22 17:26:49 +01:00
MarcoFalke
fa71b38168
scripted-diff: Rename rpc_timewait to rpc_timeout
-BEGIN VERIFY SCRIPT-
sed -i -e 's/self.rpc_timewait/self.rpc_timeout/g' $(git grep -l self.rpc_timewait)
-END VERIFY SCRIPT-
2018-12-22 17:23:25 +01:00
MarcoFalke
fa3e5786d0
scripted-diff: Remove unused 'split' parameter to setup_network
-BEGIN VERIFY SCRIPT-
sed -i -e 's/, split=False//g' $(git grep -l 'def setup_network')
-END VERIFY SCRIPT-
2018-12-22 17:20:52 +01:00
MarcoFalke
cc7ad2f3d9
Merge #15013: test: Avoid race in p2p_timeouts
fabcd645b9 test: Avoid racy test p2p_timeouts (MarcoFalke)

Pull request description:

  Avoid filesystem/network racyness by sleeping another second. The alternative would be to poll the `debug.log`, but that seems overkill to avoid a sleep in a test that already requires them.

Tree-SHA512: 32de75ecba38426323158fda241e37ffae73806d8c0e60b7ad73fb917e9cd25a3e5f0722cbc6f2f7a5b86501221c317783266bc2e3b4697fb120f05898ccc12d
2018-12-21 20:19:57 +01:00