Commit graph

1580 commits

Author SHA1 Message Date
MarcoFalke d88f7f8764
Merge #15471: rpc/gui: Remove 'Unknown block versions being mined' warning
ef362f2773 rpc/gui: Remove 'Unknown block versions being mined' warning (Wladimir J. van der Laan)

Pull request description:

  Due to miners inserting garbage into the version numbers causing false positives, the current version signalling has become completely useless. This removes the "unknown block versions" warning which has the tendency to scare users unnecessarily (and might get them to "update" to something bad).

  It preserves the warning in the logs. Whether this is desirable can be a point of discussion.

Tree-SHA512: 51407ccd24a571462465d9c7180f0f28307c50b82a03284abe783e181d8ab7e0638dbb710698d883f28de8a609db70763e39be2470d956e67c833da0768e43e9
2019-02-26 09:20:39 -05:00
MarcoFalke 8f470ecc53
Merge #15419: qa: Always refresh cache to be out of ibd
fa2cdc9ac2 test: Simplify create_cache (MarcoFalke)
fa25210d62 qa: Fix wallet_txn_doublespend issue (MarcoFalke)
1111aecbb5 qa: Always refresh stale cache to be out of ibd (MarcoFalke)
fab0d85802 qa: Remove mocktime unless required (MarcoFalke)

Pull request description:

  When starting a test, we are always in IBD because the timestamps on cached blocks are in the past. Usually, we solve that by generating a block at the beginning of the test.

  That is clumsy and might even lead to other problems such as #15360 and https://github.com/bitcoin/bitcoin/issues/14446#issuecomment-461926598

  So fix that by getting rid of mocktime and always refreshing the last block of the cache when starting the test framework.

  Should fix #14446

Tree-SHA512: 6af09800f9c86131349a103af617a54551f5f3f3260d38e14e3f30fdd3d91a0feb0100c56cbb12eae4aeac5571ae4b530b16345cbb831d2670237b53351a22c1
2019-02-25 11:44:21 -05:00
MarcoFalke fa2cdc9ac2
test: Simplify create_cache 2019-02-25 11:18:24 -05:00
Wladimir J. van der Laan ef362f2773 rpc/gui: Remove 'Unknown block versions being mined' warning
Due to miners inserting garbage into the version numbers, the current
version signalling has become completely useless. This removes the
"unknown block versions" warning which has the tendency to scare
users unnecessarily (and might get them to "update" to something
bad).

It preserves the warning in the logs. Whether this is desirable can
be a point of discussion.
2019-02-25 15:59:02 +01:00
Wladimir J. van der Laan 1a8a5ede9f
Merge #15401: rpc: Actually throw help when passed invalid number of params
fa4ce7038d rpc: Actually throw help when passed invalid number of params (MarcoFalke)
fa05626ca7 rpc: Add RPCHelpMan::IsValidNumArgs() (MarcoFalke)

Pull request description:

  Can be tested by

  * running the included test against an old binary (compiled without this patch)
  * calling `setban 1 "add" 3 4 5 6 7 8 9 0` in the gui

Tree-SHA512: aa6a25bbe6f40722913ea292252a62a4012c964eed9f4035335a2e2d13be98eb60f368e8a3251a104a26a62c08b2cb926b06e5ab1418ef1cf4abdd71d87c2919
2019-02-25 09:31:08 +01:00
Wladimir J. van der Laan 3e1ca1348c
Merge #15278: Improve PID file error handling
3782075a5f Move all PID file stuff to init.cpp (Hennadii Stepanov)
561e375c73 Make PID file creating errors fatal (Hennadii Stepanov)
745a2ace18 Improve PID file removing errors logging (Hennadii Stepanov)

Pull request description:

  Digging into #15240 the lack of the proper logging has been discovered.
  Fixed by this PR.

  UPDATE (inspired by @laanwj's [comment](https://github.com/bitcoin/bitcoin/pull/15278#discussion_r252641810)):
  Not being able to create the PID file is fatal now.

  Output of `bitcoind`:

  ```
  $ src/bitcoind -pid=/run/bitcoind/bitcoind.pid
  2019-02-01T23:20:10Z Bitcoin Core version v0.17.99.0-561e375c7 (release build)
  2019-02-01T23:20:10Z Assuming ancestors of block 0000000000000037a8cd3e06cd5edbfe9dd1dbcc5dacab279376ef7cfc2b4c75 have valid signatures.
  2019-02-01T23:20:10Z Setting nMinimumChainWork=00000000000000000000000000000000000000000000007dbe94253893cbd463
  2019-02-01T23:20:10Z Using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation
  2019-02-01T23:20:10Z Using RdRand as an additional entropy source
  2019-02-01T23:20:11Z Error: Unable to create the PID file '/run/bitcoind/bitcoind.pid': No such file or directory
  Error: Unable to create the PID file '/run/bitcoind/bitcoind.pid': No such file or directory
  2019-02-01T23:20:11Z Shutdown: In progress...
  2019-02-01T23:20:11Z Shutdown: Unable to remove PID file: File does not exist
  2019-02-01T23:20:11Z Shutdown: done
  ```

  Output of `bitcoin-qt`:
  ![screenshot from 2019-02-02 01-19-05](https://user-images.githubusercontent.com/32963518/52154886-9349b600-2688-11e9-8128-470f16790305.png)

  **Notes for reviewers**
  1. `CreatePidFile()` has been moved from `util/system.cpp` to `init.cpp` for the following reasons:
  - to get the ability to use `InitError()`
  - now `init.cpp` contains code of both creating PID file and removing it

  2. Regarding 0.18 release process: this PR modifies 1 string and introduces 2 new ones.

Tree-SHA512: ac07d0f800e61ec759e427d0afc0ca43d67f232e977662253963afdd0a220d34b871050f58149fc9fabd427bfc8e0d3f6a6032f2a38f30ad366fc0d074b0f2b3
2019-02-21 09:23:10 +01:00
MarcoFalke fa4ce7038d
rpc: Actually throw help when passed invalid number of params 2019-02-20 13:34:16 -05:00
MarcoFalke f9775a8655
Merge #15439: tests: remove byte.hex() to keep compatibility
1a062b85f0 tests: remove byte.hex() to keep compatibility (Akio Nakamura)

Pull request description:

  Use ```test_framework.util.bytes_to_hex_str()``` instead of ```bytes.hex()``` that new in Python 3.5 to support minimum version of Python(test).

  ```test/functional/test_framework/wallet_util.py``` is also reported to have '\.hex()' in #15397,
  but it does not matter because it calls CScript.hex() defined in wallet_util.py.

Tree-SHA512: 1019212e965f0848d235fab4da11959dffa42e8adfcd41216c10795cfc63c804b5deb5a3317f25d29940b9dcf088ab76fe3fa80d2679dc19f5f482dc5bde3283
2019-02-20 09:18:41 -05:00
MarcoFalke fa25210d62
qa: Fix wallet_txn_doublespend issue 2019-02-19 14:22:15 -05:00
MarcoFalke 1111aecbb5
qa: Always refresh stale cache to be out of ibd 2019-02-19 14:22:13 -05:00
MarcoFalke fab0d85802
qa: Remove mocktime unless required 2019-02-19 10:43:58 -05:00
Wladimir J. van der Laan 38429c4b62
Merge #15404: [test] Remove -txindex to start nodes
8e4b4f683a Address test todos by removing -txindex to nodes. Originally added when updating getrawtransaction to stop searching unspent utxos. (Amiti Uttarwar)

Pull request description:

  Original todos added when removing getrawtransaction default behavior of searching unspent utxos.

Tree-SHA512: d080953c3b0d2e5dca2265a15966dc25985a614c9cc86271ecd6276178ce428c85e262c24df92501695c32fed7beec0339b989f03cce91b57fb2efba201b7809
2019-02-19 16:31:20 +01:00
MarcoFalke 3e4fd40753
Merge #15415: [test] functional: allow custom cwd, use tmpdir as default
e3e1a5631e [test] functional: set cwd of nodes to tmpdir (Sjors Provoost)

Pull request description:

  Any process launched by bitcoind will have `self.datadir` as its `cwd`.

Tree-SHA512: 0b311643bb96c7dc2f693774620173243b3add40bf373284695af2f0071823b23485289fd2ffe152b7f63e0bfe989b16720077cfc2ce33905f9b8e7f2630f3c0
2019-02-19 10:12:28 -05:00
Sjors Provoost e3e1a5631e
[test] functional: set cwd of nodes to tmpdir 2019-02-19 08:53:51 +01:00
Akio Nakamura 1a062b85f0 tests: remove byte.hex() to keep compatibility
Use test_framework.util.bytes_to_hex_str() instead of bytes.hex() that
new in Python 3.5, to support minimum version of Python(test).
2019-02-19 16:38:44 +09:00
Akio Nakamura 1a7ba84e11 Fix lack of warning of unrecognized section names
1. Fix lack of warning by collecting all section names by moving
   m_config_sections.clear() to ArgsManager::ReadConfigFiles().
2. Add info(file name, line number) to warning message.
3. Add a test code to confirm this situation.
3. Do clear() in ReadConfigString().
2019-02-19 10:49:19 +09:00
MarcoFalke 6ba3f1fdfd
Merge #15397: Remove manual byte editing in wallet_tx_clone func test
6aaa0abc12 Remove manual byte editing in wallet_tx_clone func test (Gregory Sanders)

Pull request description:

  Adapted from @stevenroose

Tree-SHA512: 87f251579e347f870bd30fc57b0c130f00914a3dc78799826384eb049b91d49f2525d55899bf525997e23cc976ca7d10e6b56b23f7358acec307368d48a6f6f1
2019-02-18 09:16:35 -05:00
Amiti Uttarwar 8e4b4f683a Address test todos by removing -txindex to nodes.
Originally added when updating getrawtransaction to stop searching unspent utxos.
2019-02-17 21:42:08 -08:00
Wladimir J. van der Laan f60d029a2a
Merge #15368: Descriptor checksums
fd637be8d2 Add checksums to descriptors.md (Pieter Wuille)
be62903c41 Make descriptor checksums mandatory in deriveaddresses and importmulti (Pieter Wuille)
b52cb63688 Add getdescriptorinfo to compute checksum (Pieter Wuille)
3b40bff988 Descriptor checksum (Pieter Wuille)

Pull request description:

  This adds support for a descriptor-specific 8-character checksum.

  Descriptors may optionally be suffixed with a `#` plus these 8 checksum characters. Any descriptor that contains a `#` at the end must be followed by a valid checksum. If the `#` is missing entirely, it is valid without checksum.

  All RPCs are updated to report descriptors that include the checksum. On input, they are optional except in `deriveaddress` and `importmulti`, which require descriptors which include a checksum.

  A new RPC is also added to analyse descriptors (`getdescriptorinfo`), which can be used to compute the checksum for a descriptor without.

Tree-SHA512: a8294b09155eb6c67fbc178b5e2d3fbc0e9bec8b6de57a13f8835550d51c2cb32a428b3c9a188ded42b454d594e9305edbd4797906b755de77a8f33c79165f6b
2019-02-16 21:39:32 +01:00
Wladimir J. van der Laan d5b929c813
Merge #13932: Additional utility RPCs for PSBT
540729ef4b Implement analyzepsbt RPC and tests (Andrew Chow)
77542cf2a5 Move PSBT UTXO fetching to a separate method (Andrew Chow)
cb40b3abd4 Figure out what is missing during signing (Andrew Chow)
08f749c914 Implement joinpsbts RPC and tests (Andrew Chow)
7344a7b998 Implement utxoupdatepsbt RPC and tests (Andrew Chow)

Pull request description:

  This PR adds 3 new utility RPCs for interacting with PSBTs.

  `utxoupdatepsbt` updates a PSBT with UTXO information from the node. It only works with witness UTXOs because full transactions (as would be needed for non-witness UTXOs) are not available unless txindex is enabled.

  `joinpsbts` joins the inputs from multiple distinct PSBTs into one PSBT. e.g. if PSBT 1 has inputs 1 and 2, and PSBT 2 has inputs 3 and 4, `joinpsbts` would create a new PSBT with inputs 1, 2, 3, and 4.

  `analyzepsbt` analyzes a PSBT and determines the current state of it and all of its inputs, and the next step that needs to be done.

Tree-SHA512: 3c1fa302201abca76a8901d0c2be7b4ccbce334d989533c215f8b3e50e22f2f018ce6209544b26789f58f5980a253c0655111e1e20d47d5656e0414c64891a5c
2019-02-16 20:45:03 +01:00
Andrew Chow 540729ef4b Implement analyzepsbt RPC and tests 2019-02-16 11:51:02 -05:00
Pieter Wuille be62903c41 Make descriptor checksums mandatory in deriveaddresses and importmulti 2019-02-15 22:36:05 -08:00
Pieter Wuille b52cb63688 Add getdescriptorinfo to compute checksum 2019-02-15 22:36:05 -08:00
Pieter Wuille 3b40bff988 Descriptor checksum 2019-02-15 22:36:05 -08:00
Andrew Chow 08f749c914 Implement joinpsbts RPC and tests
Adds a joinpsbts RPC which combines multiple distinct PSBTs into
one PSBT.
2019-02-16 00:04:27 -05:00
Andrew Chow 7344a7b998 Implement utxoupdatepsbt RPC and tests 2019-02-16 00:04:27 -05:00
MarcoFalke eca1273c35
Merge #15383: [rpc] mining: Omit uninitialized currentblockweight, currentblocktx
fa178a6385 [rpc] mining: Omit uninitialized currentblockweight, currentblocktx (MarcoFalke)

Pull request description:

  Previously we'd report "0", which could be mistaken for a valid number. E.g. the number of transactions is 0 or the block weight is 0, whatever that means.

Tree-SHA512: ee94ab203a329e272211b726f4c23edec4b09c650ec363b77fd59ad9264165d73064f78ebb9e11b5c2c543b73c157752410a307655560531c7d5444d203aa0ea
2019-02-15 08:57:50 -05:00
Andrew Chow f4b00b70e8 Import public keys in order
Do public key imports in the order that they are specified in the import
or in the descriptor range.
2019-02-14 18:34:03 -05:00
Andrew Chow 9e1551b9ce Test pubkey import to keypool 2019-02-14 18:33:29 -05:00
Andrew Chow 9b81fd19ac Fetch keys from keypool when private keys are disabled
When private keys are disabled, still fetch keys from the keypool
if the keypool has keys. Those keys come from importing them and
adding them to the keypool.
2019-02-14 18:14:00 -05:00
MeshCollider 8d0ec74801
Merge #14021: Import key origin data through descriptors in importmulti
cb3511b9d Add release notes for importing key origin info change (Andrew Chow)
4c75a69f3 Test importing descriptors with key origin information (Andrew Chow)
02d6586d7 Import KeyOriginData when importing descriptors (Andrew Chow)
3d235dff5 Implement a function to add KeyOriginInfo to a wallet (Andrew Chow)
eab63bc26 Store key origin info in key metadata (Andrew Chow)
345bff601 Remove hdmasterkeyid (Andrew Chow)
bac8c676a Add a method to CWallet to write just CKeyMetadata (Andrew Chow)
e7652d3f6 Add WriteHDKeypath function and move *HDKeypath to util/bip32.{h,cpp} (Andrew Chow)
c45415f73 Refactor keymetadata writing to a separate method (Andrew Chow)

Pull request description:

  This PR allows for key origin data as defined by the descriptors document to be imported to the wallet when importing a descriptor using `importmulti`. This allows the `walletprocesspsbt` to include the BIP 32 derivation paths for keys that it is watching that are from a different HD wallet.

  In order to make this easier to use, a new field `hdmasterkeyfingerprint` has been added to `getaddressinfo`. Additionally I have removed `hdmasterkeyid` as was planned. I think that this API change is fine since it was going to be removed in 0.18 anyways. `CKeyMetadata` has also been extended to store key origin info to facilitate this.

Tree-SHA512: 9c7794f3c793da57e23c5abbdc3d58779ee9dea3d53168bb86c0643a4ad5a11a446264961e2f772f35eea645048cb60954ed58050002caee4e43cd9f51215097
2019-02-15 12:11:28 +13:00
Andrew Chow 4c75a69f36 Test importing descriptors with key origin information 2019-02-14 17:58:25 -05:00
MarcoFalke 31f7c6dd21
Merge #15295: fuzz: Add test/fuzz/test_runner.py and run it in travis
fa535af92c fuzz: test_runner: Better error message when built with afl (MarcoFalke)
fa7ca8ef58 qa: Add test/fuzz/test_runner.py (MarcoFalke)

Pull request description:

  Can be run with `./test/fuzz/test_runner.py` after building as described in `doc/fuzzing.md`

Tree-SHA512: f6a3cd8165ec2de4b363be4fd0a936b4a60829cce923f93fe5d6a046b1bbd64c959cdf790440bf70c0e13b0bb1b956a746a24c6fd92bddeab15b837ed50ffad2
2019-02-14 16:32:26 -05:00
Wladimir J. van der Laan 3facd9fdc4
Merge #14481: Add P2SH-P2WSH support to listunspent RPC
6ca836ab3a Add release note for listunspent P2WSH change (MeshCollider)
928beae007 Add test for P2SH-P2WSH in signrawtransactionwithkey and listunspent (MeshCollider)
314784a60f Make listunspent and signrawtransaction RPCs support witnessScript (MeshCollider)

Pull request description:

  This is a reworked version of #11708 after #12427 and the `signrawtransaction` split.

  For a P2WSH address, listunspent should return the witness script, and for a P2SH-P2WSH address, it should also return the inner witness script (because SignTransaction will automatically wrap it in P2SH if required).

  Includes a test which also tests the behaviour of #12427, and release note.

Tree-SHA512: a8e72cf16930312bf48ec47e44a68f8d7e26664043c1b4cc0983eb25aec4087e511188ff9a0f181cd7b8a0c068c60d7f1e7e3f226b79e8c48890039dcf57f7b7
2019-02-14 22:17:52 +01:00
MarcoFalke fa535af92c
fuzz: test_runner: Better error message when built with afl 2019-02-14 15:47:08 -05:00
Andrew Chow 345bff6013 Remove hdmasterkeyid 2019-02-14 14:00:29 -05:00
MeshCollider 2452c6cc0a
Merge #14978: Factor out PSBT utilities from RPCs for use in GUI code; related refactoring.
102faad81 Factor out combine / finalize / extract PSBT helpers (Glenn Willen)
78b9893d0 Remove op== on PSBTs; check compatibility in Merge (Glenn Willen)
bd0dbe876 Switch away from exceptions in refactored tx code (Glenn Willen)
c6c3d42a7 Move PSBT definitions and code to separate files (Glenn Willen)
81cd95884 Factor BroadcastTransaction out of sendrawtransaction (Glenn Willen)
c734aaa15 Split DecodePSBT into Base64 and Raw versions (Glenn Willen)
162ffefd2 Add pf_invalid arg to std::string DecodeBase{32,64} (Glenn Willen)

Pull request description:

  * Move most PSBT definitions into psbt.h.
  * Move most PSBT RPC utilities into psbt.{h,cpp}.
  * Move wallet-touching PSBT RPC utilities (FillPSBT) into
      wallet/psbtwallet.{h,cpp}.
  * Switch exceptions from JSONRPCError() to new PSBTException class.
  * Split DecodePSBT into DecodeBase64PSBT (old behavior) and DecodeRawPSBT.
  * Add one new version of DecodeBase64 utility in strencodings.h (and
      corresponding DecodeBase32 for completeness).
  * Factor BroadcastTransaction utility function out of sendrawtransaction RPC
      handler in rpc/rawtransaction.cpp

  Note: For those keeping score at home wondering why refactor, this is in anticipation of (and developed in parallel with) a change to actually introduce GUI use of all this stuff, which is already under development and working-ish.

Tree-SHA512: 2197c448e657421f430943025357597e7b06c4c377d5d4b2622b9edea52a7193c48843dd731abb3a88ac4023a9c88d211991e0a9b740c22f2e1cbe72adefe390
2019-02-14 21:49:01 +13:00
MarcoFalke fa7ca8ef58
qa: Add test/fuzz/test_runner.py 2019-02-13 17:12:28 -05:00
Gregory Sanders 6aaa0abc12 Remove manual byte editing in wallet_tx_clone func test 2019-02-13 09:49:36 -05:00
MeshCollider 928beae007 Add test for P2SH-P2WSH in signrawtransactionwithkey and listunspent 2019-02-13 14:35:16 +13:00
MarcoFalke 029d28a7aa
Merge #15238: [QA] remove some magic mining constants in functional tests
b651ef7e1c submitheader: more directly test missing prev block header (Gregory Sanders)
1e7f741745 remove some magic mining constants in functional tests (Gregory Sanders)

Pull request description:

  The fewer magic numbers the better.

  Also more directly tested a `submitheader` case of bad previous blockhash.

Tree-SHA512: 52b01a6aa199fa909eea4e9e84409a901933e545724e33149cc4132c82168199fd678809b6d94d95c9ff6ad02238a9552363620d13b8beaa5d4b67ade9ef425c
2019-02-12 16:52:06 -05:00
MarcoFalke d73918447f
Merge #15216: Scripts and tools: Replace script name with a special parameter
8c9b8a3668 Replace script name with special parameter (Hennadii Stepanov)

Pull request description:

  This PR improves UX; all others shell scripts ~(excluding travis linters)~ in the bitcoin repo have this feature.

  Before:
  ![screenshot from 2019-01-20 17-45-42](https://user-images.githubusercontent.com/32963518/51442159-b5cfec80-1ce2-11e9-8017-3b0b464ccaf8.png)

  After:
  ![screenshot from 2019-01-20 18-30-27](https://user-images.githubusercontent.com/32963518/51442166-bf595480-1ce2-11e9-9520-481518c3b288.png)

  cc: @jamesob @laanwj

Tree-SHA512: 7924e5658a2efe81fd5591390ca5af1ff0558bd9d5693363b9f8addedb1d6b90aa16f11c9b361c6fdfbd931a959255817473a240c175dee95aefc7d2d4a10a36
2019-02-12 16:39:39 -05:00
Hennadii Stepanov 8c9b8a3668
Replace script name with special parameter 2019-02-12 23:25:54 +02:00
MarcoFalke ea022d9fd0
Merge #15378: tests: Added missing tests for RPC wallet errors
dc3b2ccb5f tests: Added missing tests for RPC wallet errors (Ben Carman)

Pull request description:

Tree-SHA512: b18dcd4f7547c974c93ae67dcd92a168bdb55951b164cf174cb1e59e0daa463187068aec43108309a75d65721a5c0bcdf10a16a9869620f160121e2287559926
2019-02-12 15:27:15 -05:00
MarcoFalke fa178a6385
[rpc] mining: Omit uninitialized currentblockweight, currentblocktx 2019-02-12 11:34:57 -05:00
MarcoFalke d8794a78a8
Merge #13787: Test for Windows encoding issue
15b31032a6 appveyor: Remove outdated libraries (Chun Kuan Lee)
ae892adceb tests: accept unicode characters on Windows (Chun Kuan Lee)

Pull request description:

  This PR removes unused Windows compatible path. Encoding issues has been solved.

Tree-SHA512: d24dce035fe3b576eaa002896f815a6691e88618ea371171d7e39883e1d63b3ed4d4631ab3b3a26bd67ae4e8ae13b1ff188942ab7ac6a93d704e0de301368cd0
2019-02-12 11:15:11 -05:00
MarcoFalke 03732f8644
Merge #14543: [QA] minor p2p_sendheaders fix of height in coinbase
1cdb9bb51f minor p2p_sendheaders fix of height in coinbase (Gregory Sanders)

Pull request description:

  > \# Now announce a header that forks the last two blocks

  Doesn't effect any behavior since BIP34 isn't active in regtest for many blocks.

Tree-SHA512: 3f214b956a94250bb640f63b6ff707930e1d4cb8df1bbf0fef4012d89a94bafbde0d7b42bbe7113ec33810169281c22c6e389445921d99decb74aa56e87a0f27
2019-02-12 11:10:02 -05:00
Ben Carman dc3b2ccb5f
tests: Added missing tests for RPC wallet errors 2019-02-11 18:19:22 -06:00
Chun Kuan Lee ae892adceb tests: accept unicode characters on Windows 2019-02-12 05:17:15 +08:00
Glenn Willen c734aaa15d Split DecodePSBT into Base64 and Raw versions
Split up DecodePSBT, which both decodes base64 and then deserializes a
PartiallySignedTransaction, into two functions: DecodeBase64PSBT, which retains
the old behavior, and DecodeRawPSBT, which only performs the deserialization.

Add a test for base64 decoding failure.
2019-02-11 12:23:14 -08:00
MarcoFalke e84e0d4b5e
Merge #15337: rpc: Fix for segfault if combinepsbt called with empty inputs
30d0f7be6e rpc: Fix for segfault if combinepsbt called with empty inputs (benthecarman)

Pull request description:

  Fixes #15300

Tree-SHA512: 25e7b4e6e48d8b0d197f0ab96df308fff33e2110f8929cb48914877fa7f4c4a84f173b1378fdb2dec5d03fe7d6d1aced4b577e55f9fe180d8147d9106ebf543f
2019-02-11 08:08:17 -05:00
MeshCollider 6f4e0d1542
Merge #15226: Allow creating blank (empty) wallets (alternative)
7687f7873 [wallet] Support creating a blank wallet (Andrew Chow)

Pull request description:

  Alternative (kind of) to #14938

  This PR adds a `blank` parameter to the `createwallet` RPC to create a wallet that has no private keys initially. `sethdseed` can then be used to make a clean wallet with a custom seed. `encryptwallet` can also be used to make a wallet that is born encrypted.

  Instead of changing the version number as done in #14938, a wallet flag is used to indicate that the wallet should be blank. This flag is set at creation, and then unset when the wallet is no longer blank. A wallet becomes non-blank when a HD seed is set or anything is imported. The main change to create a blank wallet is primarily taken from #14938.

  Also with this, the term "blank wallet" is used instead of "empty wallet" to avoid confusion with wallets that have balance which would also be referred to as "empty".

  This is built on top of #15225 in order to fix GUI issues.

Tree-SHA512: 824d685e11ac2259a26b5ece99c67a7bda94a570cd921472c464243ee356b7734595ad35cc439b34357135df041ed9cba951e6edac194935c3a55a1dc4fcbdea
2019-02-11 08:08:33 +13:00
Andrew Chow 7687f7873b [wallet] Support creating a blank wallet
A blank wallet is a wallet that has no keys, script or watch only things.
A new wallet flag indicating that it is blank will be set when the wallet
is blank. Once it is no longer blank (a seed has been generated, keys or
scripts imported, etc), the flag will be unset.
2019-02-10 12:24:53 -05:00
MarcoFalke 1477933f28
Merge #15370: test_runner: Remove unused --force option
fae8b8bb1a qa: Add tool-prefix to functional test readme (MarcoFalke)
faf3d22725 test_runner: Remove unused --force option (MarcoFalke)

Pull request description:

  When someone calls the script they already have all intention to call it, no need to specify a redundant `--force`.

  The functional tests are still disabled on the travis windows cross builds, where they'd run into issues when run under Wine.

Tree-SHA512: ada0dd9b3c0cd28c5832a12c5e04c029dc3bfe5ddf366fd0abc24fb7914d2e0f0a873fe756ade7ba780a561abe9bc731838c289accc421deda481269e08514cd
2019-02-08 10:40:34 -05:00
MarcoFalke fae8b8bb1a
qa: Add tool-prefix to functional test readme 2019-02-08 08:33:07 -05:00
Wladimir J. van der Laan 6fc656a410
Merge #14242: Avoid triggering undefined behaviour (std::memset(nullptr, 0, 0)) if an invalid string is passed to DecodeSecret(...)
d855e4cac8 Avoid triggering undefined behaviour (std::memset(nullptr, 0, 0)) if an invalid string is passed to DecodeSecret(...) (practicalswift)

Pull request description:

  Avoid triggering undefined behaviour (`std::memset(nullptr, 0, 0)`) if an invalid string is passed to `DecodeSecret(...)`.

  Background reading: [memcpy (and friends) with NULL pointers](https://www.imperialviolet.org/2016/06/26/nonnull.html)

  Steps to reproduce:

  ```
  ./configure --with-sanitizers=undefined && make check && ./test/functional/test_runner.py
  ```

Tree-SHA512: b8325ced4f724d9c03065e0747af56b1f297a90d9fb09a24d46c3231a90dce3df6299f2c41f863b5cec18eaeded7b46ee4b93d9a52adc2541eb4c44d2c0965d9
2019-02-08 12:30:21 +01:00
MarcoFalke faf3d22725
test_runner: Remove unused --force option 2019-02-07 19:16:57 -05:00
Wladimir J. van der Laan 9127bd7aba
Merge #14491: Allow descriptor imports with importmulti
b985e9c850 Add release notes for importmulti descriptor support (MeshCollider)
fbb5e935ea Add test for importing via descriptor (MeshCollider)
9f48053d8f [wallet] Allow descriptor imports with importmulti (MeshCollider)
d2b381cc91 [wallet] Refactor ProcessImport() to call ProcessImportLegacy() (John Newbery)
4cac0ddd25 [wallet] Add ProcessImportLegacy() (John Newbery)
a1b25e12a5 [wallet] Refactor ProcessImport() (John Newbery)

Pull request description:

  ~~Based on #14454 #14565, last two commits only are for review.~~

  Best reviewed with `?w=1`

  Allows a descriptor to be imported into the wallet using `importmulti` RPC. Start and end of range can be specified for ranged descriptors. The descriptor is implicitly converted to old structures on import.

  Also adds a simple test of a P2SH-P2WPKH address being imported as a descriptor. More tests to come, as well as release notes.

Tree-SHA512: 160eb6fd574c4ae5b70e0109f7e5ccc95d9309138603408a1114ceb3c558065409c0d7afb66926bc8e1743c365a3b300c5f944ff18b2451acc0514fbeca1f2b3
2019-02-07 22:43:33 +01:00
practicalswift d855e4cac8 Avoid triggering undefined behaviour (std::memset(nullptr, 0, 0)) if an invalid string is passed to DecodeSecret(...) 2019-02-07 22:30:25 +01:00
MeshCollider 1933e38c1a
Merge #14667: Add deriveaddresses RPC util method
595283851 [rpc] util: add deriveaddresses method (Sjors Provoost)

Pull request description:

  Usage:

  ```sh
  bitcoin-cli deriveaddresses "wpkh([d34db33f/84h/0h/0h]xpub6DJ2dNUysrn5Vt36jH2KLBT2i1auw1tTSSomg8PhqNiUtx8QX2SvC9nrHu81fT41fvDUnhMjEzQgXnQjKEu3oaqMSzhSrHMxyyoEAmUHQbY/0/0)"
  [
    "bc1qg6ucjz7kgdedam7v5yarecy54uqw82yym06z3q"
  ] // part of the BIP32 test vector
  ```

  Avoids the need for external (BIP32) libraries to derive an address. Can be used in conjunction with `scantxoutset` as a poor mans wallet. Might be useful to test more complicated future descriptors.

  ~To keep it as simple as possible it only supports descriptors that result in a single address, so no `combo()` and ranges.~

  As discussed recently on IRC it might make sense to put this in a separate utility along with other descriptor and psbt utility functions which don't need a chain or wallet context. However I prefer to leave that to another PR.

Tree-SHA512: b8e53db11a8fd87638cc98766270cc3be9adc4b3e5085798a6a4e2e6ad252bf6d2189346bbb2da72d04d13f7f1e80b5cb88e8039653bea1f150602a876ef7f34
2019-02-08 08:21:52 +13:00
João Barbosa 6440e61375 qa: Drop RPC connection if --usecli 2019-02-06 00:04:28 +00:00
MarcoFalke 5029e94f85
Merge #14519: tests: add utility to easily profile node performance with perf
13782b8ba8 docs: add perf section to developer docs (James O'Beirne)
58180b5fd4 tests: add utility to easily profile node performance with perf (James O'Beirne)

Pull request description:

  Adds a context manager to easily (and selectively) profile node performance during functional test execution using `perf`.

  While writing some tests, I encountered some odd bitcoind slowness. I wrote up a utility (`TestNode.profile_with_perf`) that generates performance diagnostics for a node by running `perf` during the execution of a particular region of test code.

  `perf` usage is detailed in the excellent (and sadly unmerged) https://github.com/bitcoin/bitcoin/pull/12649; all due props to @eklitzke.

  ### Example

  ```python
  with node.profile_with_perf("large-msgs"):
      for i in range(200):
          node.p2p.send_message(some_large_msg)
      node.p2p.sync_with_ping()
  ```

  This generates a perf data file in the test node's datadir (`/tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data`).

  Running `perf report` generates nice output about where the node spent most of its time while running that part of the test:

  ```bash
  $ perf report -i /tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data --stdio \
    | c++filt \
    | less

  # To display the perf.data header info, please use --header/--header-only options.
  #
  #
  # Total Lost Samples: 0
  #
  # Samples: 135  of event 'cycles:pp'
  # Event count (approx.): 1458205679493582
  #
  # Children      Self  Command          Shared Object        Symbol
  # ........  ........  ...............  ...................  ........................................................................................................................................................................................................................................................................
  #
      70.14%     0.00%  bitcoin-net      bitcoind             [.] CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
                  |
                  ---CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

      70.14%     0.00%  bitcoin-net      bitcoind             [.] CNetMessage::readData(char const*, unsigned int)
                  |
                  ---CNetMessage::readData(char const*, unsigned int)
                     CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

      35.52%     0.00%  bitcoin-net      bitcoind             [.] std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
                  |
                  ---std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
                     CNetMessage::readData(char const*, unsigned int)
                     CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

  ...
  ```

Tree-SHA512: 9ac4ceaa88818d5eca00994e8e3c8ad42ae019550d6583972a0a4f7b0c4f61032e3d0c476b4ae58756bc5eb8f8015a19a7fc26c095bd588f31d49a37ed0c6b3e
2019-02-05 17:40:16 -05:00
MeshCollider fbb5e935ea Add test for importing via descriptor 2019-02-05 19:42:05 +13:00
benthecarman 30d0f7be6e rpc: Fix for segfault if combinepsbt called with empty inputs 2019-02-04 21:26:52 -06:00
João Barbosa d3bf3b930d qa: Test .walletlock file is closed 2019-02-04 18:50:21 +00:00
MarcoFalke fac3a054cb
test: Fix race in p2p_invalid_messages 2019-02-02 17:45:20 -05:00
Hennadii Stepanov 561e375c73
Make PID file creating errors fatal 2019-02-02 01:07:23 +02:00
MarcoFalke 5c99bb0047
Merge #15247: qa: Use wallet to retrieve raw transactions
fa5278a419 qa: Use wallet to retrieve raw transactions (MarcoFalke)
fa2198328e qa: Style-only fixes in touched files (MarcoFalke)

Pull request description:

  Instead of asking the coin database and block storage about a transaction, pull it directly from the wallet in wallet related tests.

  This refactoring only makes sense in light of #15159.

  <sub>This product may contain minor stylistic cleanups

Tree-SHA512: ec34c7150d873da9f19fead3f7e3f758baba5ef10061942384c470a47a6f320690109be9c5160f0c8bc228272a729653d44c78471455337318f657d6c164ba23
2019-02-01 09:35:14 -05:00
Wladimir J. van der Laan 3e38d40873
Merge #15235: Do not import private keys to wallets with private keys disabled
e6c58d3b01 Do not import private keys to wallets with private keys disabled (Andrew Chow)
b5c5021b64 Refactor importwallet to extract data from the file and then import (Andrew Chow)
1f77f6754c tests: unify RPC argument to cli argument conversion and handle dicts and lists (Andrew Chow)

Pull request description:

  Fixes a bug where private keys could be imported to wallets with private keys disabled. Now every RPC which can import private keys checks for whether the wallet has private keys are disabled and errors if it is. Also added an belt-and-suspenders check to `AddKeyPubkeyWithDB` to have it assert that the wallet has private keys enabled.

Tree-SHA512: 5cd04febce9aa2bd9bfd02f312c6ff8705e37278cae59efd3895f6d6e2f1b477aefd297e2dd0860791bdd3d4f3cad8eb1a404f8f3d4e2035b91314ad2c1028ae
2019-02-01 14:01:32 +01:00
Andrew Chow e6c58d3b01 Do not import private keys to wallets with private keys disabled 2019-01-31 14:29:28 -05:00
Andrew Chow 2e023419c5 tests: unify RPC argument to cli argument conversion and handle dicts and lists
When running tests with --usecli, unify the conversion from argument objects to
strings using a new function arg_to_cli(). This fixes boolean arguments when
using named arguments.

Also use json.dumps() to get the string values for arguments that are dicts and
lists so that bitcoind's JSON parser does not become confused.
2019-01-31 12:37:33 -05:00
MarcoFalke 252fd15add
Merge #13926: [Tools] bitcoin-wallet - a tool for creating and managing wallets offline
3c3e31c3a4 [tests] Add wallet-tool test (João Barbosa)
49d2374acf [tools] Add wallet inspection and modification tool (Jonas Schnelli)

Pull request description:

  Adds an offline tool `bitcoin-wallet-tool` for wallet creation and maintenance.

  Currently this tool can create a new wallet file, display information on an existing wallet, and run the salvage and zapwallettxes maintenance tasks on an existing wallet. It can later be extended to support other common wallet maintenance tasks.

  Doing wallet maintenance tasks in an offline tool makes much more sense (and is potentially safer) than having to spin up a full node.

Tree-SHA512: 75a28b8a58858d9d76c7532db40eacdefc5714ea5aab536fb1dc9756e2f7d750d69d68d59c50a68e633ce38fb5b8c3e3d4880db30fe01561e07ce58d42bceb2b
2019-01-31 11:07:51 -05:00
Wladimir J. van der Laan 4d661baf1a
Merge #15244: gdb attaching to process during tests has non-sudo solution
f96dbd1bbe gdb attaching to process during tests has non-sudo solution (Gregory Sanders)

Pull request description:

  There are some security considerations, so a link is attached.

Tree-SHA512: 67dd9c4b26b1e6d8e9a9fe766d309c0af69b752f6f544f3dce4bdcc95ae85feb9a49ac600c3f70d100629505d2340ab43932ded53b1485f80b97981e6df6a527
2019-01-31 15:29:35 +01:00
Wladimir J. van der Laan 36aeb43c01
Merge #15246: qa: Add tests for invalid message headers
fa3745bda8 qa: Add tests for invalid message headers (MarcoFalke)

Pull request description:

Tree-SHA512: b37e297cfd65a33a7af201f750a303cf437b438e40d38b1d2f562ccde67082616daa110ca1e5e3af6514ea4ca4b115362acf2ffa6263cea3c8e8189ce02dda67
2019-01-31 14:25:33 +01:00
Gregory Sanders f96dbd1bbe gdb attaching to process during tests has non-sudo solution 2019-01-31 12:25:04 +00:00
João Barbosa 3c3e31c3a4 [tests] Add wallet-tool test
Original tests by João Barbosa <joao.paulo.barbosa@gmail.com>

Additional contribution by John Newbery <john@johnnewbery.com>
2019-01-30 16:26:56 -05:00
Wladimir J. van der Laan 77339e5c24
Merge #15163: Correct units for "-dbcache" and "-prune"
6f6514a080 Correct units for "-dbcache" and "-prune" (Hennadii Stepanov)

Pull request description:

  Actually, all `dbcache`-related values in the code are measured in MiB (not in megabytes, MB) or in bytes (e.g., `nTotalCache`).

  See: https://github.com/bitcoin/bitcoin/blob/master/src/txdb.h

  ba8c8b2227/src/init.cpp (L1405-L1424)

  Also, "-prune" is fixed:
  1. The GUI values in GB are translated to the node values in MiB correctly.
  2. The maximum of the "prune" `QSpinBox` is not limited by default value of 99 (GB).

  Fix: #15106

Tree-SHA512: 151ec43b31b1074db8b345fedb1dcc10bde225899a5296bfc183f57e1553d13ac27db8db100226646769ad03c9fcab29d88763065a471757c6c41ac51108459d
2019-01-30 20:22:57 +01:00
MarcoFalke a47319dada
Merge #15159: [RPC] Remove lookup to UTXO set from GetTransaction
04da9f4834 [RPC] Update getrawtransaction interface (Amiti Uttarwar)

Pull request description:

  - stop checking unspent UTXOs for a transaction when txindex is not enabled, as per conversation here: https://github.com/bitcoin/bitcoin/issues/3220#issuecomment-377458383
  - code contributed by sipa

Tree-SHA512: aa07353bccc14b81b7803992a25d076d6bc06d15ec7c1b85828dc10aea7e0498d9b49f71783e352ab8a14b0bb2010cfb7835de3dfd1bc6f2323f460449348e66
2019-01-30 11:18:44 -05:00
Hennadii Stepanov 6f6514a080
Correct units for "-dbcache" and "-prune"
All dbcache-related values in the code are measured in MiB (not in
megabytes, MB) or in bytes.
The GUI "-prune" values in GB are translated to the node values in MiB
correctly. The maximum of the "-prune" QSpinBox is not limited by the
default value of 99 (GB).
Also, this improves log readability.
2019-01-30 07:17:22 +02:00
Sjors Provoost 595283851d
[rpc] util: add deriveaddresses method 2019-01-29 18:14:23 +01:00
MarcoFalke fa5278a419
qa: Use wallet to retrieve raw transactions 2019-01-26 22:43:56 -05:00
Amiti Uttarwar 04da9f4834 [RPC] Update getrawtransaction interface 2019-01-26 18:36:53 -08:00
MarcoFalke fa2198328e
qa: Style-only fixes in touched files 2019-01-25 11:29:10 -05:00
Ben Woosley f618c58b75
Docs: Update python docs to reflect that wildcard imports are disallowed 2019-01-24 16:57:41 -08:00
MarcoFalke fa3745bda8
qa: Add tests for invalid message headers 2019-01-24 17:08:22 -05:00
Andrew Chow 1f77f6754c tests: unify RPC argument to cli argument conversion and handle dicts and lists
When running tests with --usecli, unify the conversion from argument objects to
strings using a new function arg_to_cli(). This fixes boolean arguments when
using named arguments.

Also use json.dumps() to get the string values for arguments that are dicts and
lists so that bitcoind's JSON parser does not become confused.
2019-01-24 14:07:09 -05:00
Gregory Sanders b651ef7e1c submitheader: more directly test missing prev block header 2019-01-24 09:48:34 -05:00
Gregory Sanders 1e7f741745 remove some magic mining constants in functional tests 2019-01-24 09:48:34 -05:00
Wladimir J. van der Laan 72bd4ab867
Merge #15193: Default -whitelistforcerelay to off
a36d97d866 Default -whitelistforcerelay to off (Suhas Daftuar)

Pull request description:

  No one seems to use this "feature", and at any rate the behavior of relaying transactions when they violate local policy is error-prone, if we ever consider changing the ban behavior of our software from one version to the next.

  Defaulting this to off means that users who use -whitelist won't be unexpectedly surprised by this interaction.  If anyone is still relying on this feature, it can still be explicitly turned on.

Tree-SHA512: 52650ad464a728d1648f496751e3f713077ea3a1de7278ed03531b2e8723e63cf2f6f41b56c98c0f73ffa22c36e01d9170b409ab452c737aca35b7ecd7a6b448
2019-01-24 15:25:17 +01:00
Jonas Schnelli 73a6bac9ff
Merge #15196: [test]: Update all subprocess.check_output functions to be Python 3.4 compatible
fdf82ba18 Update all subprocess.check_output functions in CI scripts to be Python 3.4 compatible (Graham Krizek)

Pull request description:

  CI is failing the `lint` stage on every Cron run (regular PR/Push runs still pass). The failure was introduced in 74ce326 and has been broken since. The Python version running in CI was downgraded to 3.4 from 3.6. There were a couple files that were using the `encoding` argument in the `subprocess.check_output` function. This was introduced in Python 3.6 and therefore broke the scripts that were using it. The `universal_newlines` argument was used as well, but in order to use it we must be able to set encoding because of issues on some BSD systems.

  To get CI to pass, I removed all `universal_newline` and `encoding` args to the `check_ouput` function. Then I decoded all `check_output` return values. This should keep the same behavior but be Python 3.4 compatible.

Tree-SHA512: f5e5885e98cf4777be9cc254446a873eedb03bdccbd8e06772a964db95e9fcf46736aa9cdcab1d8f123ea9f4947ed6020679898d8b2f47ffb1d94c21a4b08209
2019-01-23 20:43:53 -10:00
Jonas Schnelli 82cf6813a4
Merge #14353: REST: add blockhash call, fetch blockhash by height
42ff30ec6 [Docs] add short documentation for /rest/blockhashbyheight (Jonas Schnelli)
579d418f7 [QA] add rest tests for /rest/blockhashbyheight/<HEIGHT>.<FORMAT> (Jonas Schnelli)
eb9ef04c4 REST: add "blockhashbyheight" call, fetch blockhash by height (Jonas Schnelli)

Pull request description:

  Completes the REST interface for trivial block exploring by adding a call that allows to fetch the blockhash in the main chain by a given height.

Tree-SHA512: 94be9e56718f857279b11cc16dfa8d04f3b5a762e87ae54281b4d87247c71c844895f4944d5a47f09056bf851f4c4761ac4fbdbaaee957265d14de5c1c73e8d2
2019-01-22 19:59:02 -10:00
Suhas Daftuar a36d97d866 Default -whitelistforcerelay to off 2019-01-22 12:18:45 -05:00
James O'Beirne 58180b5fd4 tests: add utility to easily profile node performance with perf
Introduces `TestNode.profile_with_perf()` context manager which
samples node execution to produce profiling data.

Also introduces a test framework flag, `--perf`, which will run
perf on all nodes for the duration of a given test.
2019-01-22 08:55:55 -05:00
Jonas Schnelli 579d418f74
[QA] add rest tests for /rest/blockhashbyheight/<HEIGHT>.<FORMAT> 2019-01-21 11:54:54 -10:00
Wladimir J. van der Laan 9bad1e0b22
Merge #15219: lint: Enable python linters via an array
948d8f4f10 lint: Enable python linters via an array (Ben Woosley)

Pull request description:

  This assures consistent recording of the enabled linters.

  This applies the same fix as #15170 to lint-python.sh

Tree-SHA512: 6d03f919e86e7c2465475c88b25dd84391282bcc11728078024daf0432a7dccddf9e4a2cdae35d6ef374971cb4e12f0fa21b58f757e25f2fe7c12ceb4f4b2c57
2019-01-21 20:52:07 +01:00
Wladimir J. van der Laan 72506ed349
Merge #15177: rest: Improve tests and documention of /headers and /block
7cf994d5cf qa: Improve tests of /rest/headers and /rest/block (João Barbosa)
0825b86b28 doc: /rest/block responds with 404 if block does not exist (João Barbosa)
be625f7c55 doc: Explain empty result of /rest/headers (João Barbosa)

Pull request description:

  Follow up of #15107.

Tree-SHA512: a7fdeed05216e3eda9604664db529237c2d0ddf422cfac139d6345a22b6e00bfe870d4e3f177423db7d4efb295ac2dc0ca2eb20c9c27c0719b89fd5428860d03
2019-01-21 17:37:55 +01:00
Ben Woosley 948d8f4f10
lint: Enable python linters via an array
This assures consistent recording of the enabled linters.
2019-01-20 17:13:22 -08:00
Graham Krizek fdf82ba181
Update all subprocess.check_output functions in CI scripts to be Python 3.4 compatible
Removing the 'universal_newlines' and 'encoding' args from the subprocess.check_outputs fuction. 'universal_newlines' is supported in 3.4, but 'encoding' is not. Without specifying 'encoding' it will make a guess at encoding, which can break things on BSD systems. We must handle encoding/decoding ourselves until we can use Python 3.6
2019-01-18 09:36:39 -06:00
practicalswift a517541794 Remove no longer needed shellcheck suppressions 2019-01-16 15:47:56 +01:00
practicalswift 0b7196ecad Fix warnings introduced in shellcheck v0.6.0 2019-01-16 15:47:56 +01:00