Commit graph

1966 commits

Author SHA1 Message Date
MarcoFalke
d5ea8f4bf3
Merge #16509: test: Adapt test framework for chains other than "regtest"
faf36838bd test: Avoid hardcoding the chain name in combine_logs (MarcoFalke)
fa8a1d7ba3 test: Adapt test framework for chains other than "regtest" (MarcoFalke)
68f546635d test: Fix “local variable 'e' is assigned to but never used” (Ben Woosley)

Pull request description:

  This is required for various work in progress:

  * testchains #8994
  * signet #16411
  * some of my locally written tests

  While it will be unused in the master branch as of now, it will make all of those pull requests shorter. Thus review for non-regtest tests can focus on the actual changes and not some test framework changes.

ACKs for top commit:
  jonatack:
    ACK faf36838bd, ran tests and reviewed the code.

Tree-SHA512: 35add66c12cab68f2fac8f7c7d47c604d3f24eae9336ff78f83e2c92b3dc08a25e7f4217199bac5393dd3fb72f945bba9c001d6fbb8efd298c88858075fcb3d6
2019-08-05 08:08:18 -04:00
MarcoFalke
c77f7cdbd1
Merge #16197: net: Use mockable time for tx download
fab3658356 [qa] Test that getdata requests work as expected (Suhas Daftuar)
fa883ab35a net: Use mockable time for tx download (MarcoFalke)

Pull request description:

  Two commits:

  * First commit changes to mockable time for tx download (refactoring, should only have an effect on regtest)
  * Second commit adds a test that uses mocktime to test tx download

ACKs for top commit:
  laanwj:
    code review ACK 16197/commits/fab365835639a3da03f8ad9a58a0db6c6c4c2314
  jamesob:
    ACK fab3658356

Tree-SHA512: 3a64a3e283ec4bab1f6e506404b11f0a564a5b61d2a7508ae738a61f035e57220484c66e0ae47d847fe9f7e3ff5cc834909d7b34a9bbcea6abe01f8742806908
2019-08-05 08:01:28 -04:00
MarcoFalke
fa566b2601
test: Add missing sync_blocks to feature_pruning 2019-08-02 15:36:06 -04:00
MarcoFalke
3a3d8b8357
Merge #16097: Refactor: Add Flags enum to ArgsManager class
e6f649cb2c test: Make tests arg type specific (Hennadii Stepanov)
b70cc5d733 Revamp option negating policy (Hennadii Stepanov)
db08edb303 Replace IsArgKnown() with FlagsOfKnownArg() (Hennadii Stepanov)
dde80c272a Use ArgsManager::NETWORK_ONLY flag (Hennadii Stepanov)
9a12733508 Remove unused m_debug_only member from Arg struct (Hennadii Stepanov)
fb4b9f9e3b scripted-diff: Use ArgsManager::DEBUG_ONLY flag (Hennadii Stepanov)
1b4b9422ca scripted-diff: Use Flags enum in AddArg() (Hennadii Stepanov)
265c1b58d8 Add Flags enum to ArgsManager (Hennadii Stepanov)
e0d187dfeb Refactor InterpretNegatedOption() function (Hennadii Stepanov)
e0e18a1017 refactoring: Check IsArgKnown() early (Hennadii Stepanov)

Pull request description:

  This PR adds the `Flags` enum to the `ArgsManager` class. Also the `m_flags` member is added to the `Arg` struct. Flags denote an allowed type of an arg value and special hints.

  This PR is only a refactoring and does not change behavior.

ACKs for top commit:
  jamesob:
    ACK e6f649cb2c
  MarcoFalke:
    ACK e6f649cb2c thanks for adding types to the command line options

Tree-SHA512: b867f8a9cbce2d2473c293d534af662d8cd5be15060ff0682e97af678974bdaac35e8bc6328ccba32f105034bcd38f169b92a6fb67798667891ce14d5d2a2dea
2019-08-02 12:18:16 -04:00
MarcoFalke
fa76285fdd
test: Explain why -whitelist is used in feature_fee_estimation
Also, Remove seemingly unused and undocumented -maxorphantx=1000
2019-08-02 11:16:24 -04:00
MarcoFalke
faff85a69a
test: Format feature_fee_estimation with pep8 2019-08-02 11:08:07 -04:00
MarcoFalke
faf36838bd
test: Avoid hardcoding the chain name in combine_logs 2019-08-02 09:04:21 -04:00
MarcoFalke
d759b5d26a
Merge #15911: Use wallet RBF default for walletcreatefundedpsbt
d6b3640ac7 [test] walletcreatefundedpsbt: check RBF is disabled when -walletrbf=0 (Sjors Provoost)
9ed062b568 [doc] rpc: remove "fallback to" from RBF default help (Sjors Provoost)
4fcb698bc2 [rpc] walletcreatefundedpsbt: use wallet default RBF (Sjors Provoost)

Pull request description:

  The `walletcreatefundedpsbt` RPC call currently ignores `-walletrbf` and defaults to not use RBF. This PR fixes that.

  This PR also replaces UniValue in `ConstructTransaction` with a `bool` in preparation of moving this helper method out of the RPC codebase entirely. This may be a bit overkill, but does slightly simplify it.

  Fixes #15878

ACKs for top commit:
  achow101:
    Code Review ACK d6b3640ac7
  l2a5b1:
    re-ACK d6b3640
  MarcoFalke:
    ACK d6b3640ac7

Tree-SHA512: 55b9bccd1ef36b54f6b34793017dc0721103099ad3761b3b04862291ee13d6915915d4dbb1a8567924fa56e5e95dfe10eec070e06701610e70c87f8ea92b2a00
2019-08-02 08:53:39 -04:00
MarcoFalke
9f54e9ab90
Merge #16493: test: Fix test failures
fa36aa4922 Test: Set -acceptnonstdtxn in feature_fee_estimation (MarcoFalke)
fa1bb53b0d test: Add -acceptnonstdtxn to self.extra_args[3] (MarcoFalke)
fa8a823169 test: Bump rpc_timeout in feature_dbcrash (MarcoFalke)

Pull request description:

  in feature_dbcrash:

  * Fixes #16488
  * Fixes #16498

  in feature_fee_estimation:

  * Fixes #16518

ACKs for top commit:
  fanquake:
    ACK fa36aa4922

Tree-SHA512: 9e79a6f954998b196e2a7452f72d2ecf7a6b7f61be610033038e2e40f2feba53e0ee242c7e3cdd94051811e8c96f8ab8031141710da29137fc3acea07cb2dc73
2019-08-02 08:17:39 -04:00
MarcoFalke
fa36aa4922
Test: Set -acceptnonstdtxn in feature_fee_estimation 2019-08-01 10:36:52 -04:00
Wladimir J. van der Laan
79816278e2
Merge #16470: test: Fail early on disconnect in mininode.wait_for_*
fac2e6a604 test: Fail early on disconnect in mininode.wait_for_* (MarcoFalke)

Pull request description:

  The node might crash or disconnect when our mininode waits for data. Due to the crash, the data is guaranteed to never arrive and we can fail early with an assert

ACKs for top commit:
  laanwj:
    ACK fac2e6a604

Tree-SHA512: 32ca844eb66bd70ea49103d51c76b953242b1886e0834d96fca8840fc984ff40346d0a799adf8f76b03514a783cb9cec69d45e00bdd328c5192c31b5d8d17af2
2019-08-01 15:13:17 +02:00
MeshCollider
6841b01340
Merge #16394: Allow createwallet to take empty passwords to make unencrypted wallets
c5d3787367 Allow createwallet to take empty passwords to make unencrypted wallets (Andrew Chow)

Pull request description:

  Allow createwallet to take the empty string as a password and interpret that as leaving the wallet unencrypted. Also warn when that happens.

  This fixes a bug where it was not possible to use the `avoid_reuse` option for new unencrypted wallets without using named arguments.Thus this allows more `createwallet` options to be added that can be set on unencrypted wallets when using positional arguments.

ACKs for top commit:
  jnewbery:
    code review ACK c5d3787367
  meshcollider:
    re-utACK c5d3787367
  ryanofsky:
    utACK c5d3787367. Changes since last review are rebasing, concatenating warning strings to avoid discarding warnings, adding release notes, and choosing an unambiguous wallet name for the test.

Tree-SHA512: 146737a728dd614ba94d4b166b27e8c9e195badd1709ccab2315afe59176d9b493dfba9b61c3ed81090f059c7e464d709deb06d99451b9a3fff667f527d6f7c9
2019-08-01 19:11:01 +12:00
MarcoFalke
5639d71a07
Merge #16293: test: Make test cases separate functions
faf8318c55 test: Split fundrawtx test into subtests (MarcoFalke)
fa6fba3bc8 test: Make local symbols in run_test members (MarcoFalke)

Pull request description:

  This prevents scope-leak of symbols that are supposed to be local to one test case.

Top commit has no ACKs.

Tree-SHA512: 9b2a4ca2cdd631ef915d2f7e6cd62375df9a0919448350aa6e5ae4aa8a8fe3ba53870f7a9a25a57736894b4e3a45e861018253ed2d57d9a64c2bb65fa270fad8
2019-07-31 18:03:53 -04:00
MarcoFalke
fa8a1d7ba3
test: Adapt test framework for chains other than "regtest"
Co-Authored-By: Jorge Timón <jtimon@jtimon.cc>
2019-07-31 17:00:25 -04:00
Ben Woosley
68f546635d test: Fix “local variable 'e' is assigned to but never used”
flake8 F841 lints, as of flake8 3.6.0
2019-07-31 16:12:12 -04:00
MarcoFalke
fa1bb53b0d
test: Add -acceptnonstdtxn to self.extra_args[3] 2019-07-30 14:48:21 -04:00
MarcoFalke
fa8a823169
test: Bump rpc_timeout in feature_dbcrash 2019-07-30 11:19:30 -04:00
fanquake
2410088003
Merge #16491: qa: fix deprecated log.warn in feature_dbcrash test
62d3f5057f qa: fix deprecated log.warn in feature_dbcrash test (Jon Atack)

Pull request description:

  This clears up the following deprecation message when running test/functional/feature_dbcrash.py:
  ```
  test/functional/feature_dbcrash.py:270:
  DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
    self.log.warn("Node %d never crashed during utxo flush!", i)
  ```

  Git grepping indicates that this was the last remaining use of `log.warn` in the functional tests.

ACKs for top commit:
  fanquake:
    ACK 62d3f5057f - checked that there were no more occurrences.

Tree-SHA512: 2fe87400f82488e44391f4897876003a98736013e819a7dbc3b3e87a5ffbfba8d5ccab81cf2b7577f40135c95e4db96e93bb8cb24de396efb4ad814fbda09559
2019-07-30 11:16:46 +08:00
Jon Atack
62d3f5057f
qa: fix deprecated log.warn in feature_dbcrash test
This clears up the following deprecation message when running the test:
```
test/functional/feature_dbcrash.py:270: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  self.log.warn("Node %d never crashed during utxo flush!", i)
```

Git grepping indicates that this was the last remaining use of `log.warn` in the functional tests.
2019-07-29 23:23:18 +02:00
Wladimir J. van der Laan
68da54987d
Merge #16471: [mempool] log correct messages when CPFP fails
42a5e912ee [mempool] log correct messages when CPFP fails (John Newbery)

Pull request description:

  Fixes a logging issue introduced in #15681

ACKs for top commit:
  laanwj:
    ACK 42a5e912ee (+utACK from bluematt that isn't registered because it has no commit id)

Tree-SHA512: ff5f423cc4d22838eea00c5b1d39ceda89cd61474c72f256a97c698eb0ec3f2156a97139f537669376132902c1e3943bf84c356a4b98a9a306b4ec57302c2761
2019-07-29 18:55:17 +02:00
Andrew Chow
c5d3787367 Allow createwallet to take empty passwords to make unencrypted wallets
Allow createwallet to take the empty string as a password and interpret that
as leaving the wallet unencrypted. Also warn when that happens.
2019-07-29 11:50:24 -04:00
MarcoFalke
74ea1f3b0f
Merge #16399: wallet: Improve wallet creation
e967cae8fa Use switch on status in RpcWallet (Fabian Jahr)
ba1f128d6c Return error for ignored passphrase through disable private keys option (Fabian Jahr)
d6649d16b5 Use strong enum for WalletCreationStatus (Fabian Jahr)
3199610ad3 Place out args at the end for CreateWallet (Fabian Jahr)

Pull request description:

  This is a follow-up PR to #16244

  The following suggestions are included:
  - Usage of `enum class` (https://github.com/bitcoin/bitcoin/pull/16244#discussion_r296434142)
  - Placing out args at the end convention (https://github.com/bitcoin/bitcoin/pull/16244#discussion_r296434172)
  - Return error when passphrase would be ignored because of disabled private keys (including functional test) (https://github.com/bitcoin/bitcoin/pull/16244#pullrequestreview-252015195)
  - Make `status` return variable of `CreateWallet` (https://github.com/bitcoin/bitcoin/pull/16244#discussion_r302107394)
  - Using a `switch` statement instead of `if/else` in `RpcWallet` (https://github.com/bitcoin/bitcoin/pull/16244#discussion_r302112502)

  Not included was:
  - "new create wallet function [could take] separate option arguments instead of wallet flags" (https://github.com/bitcoin/bitcoin/pull/16244#pullrequestreview-252015195)
  - "blank wallet and disable private keys options could be combined into a single option" (https://github.com/bitcoin/bitcoin/pull/16244#pullrequestreview-252015195)

  For these last two changes, I was not sure what an ideal solution could look like and/or this might be of slightly larger scope than the other changes, but I would be happy to work on these as well in this PR or another follow-up if I get positive feedback on that. Is there a place in the codebase that handles flags like these in a better way that I can refer to? Nonetheless, I would prefer keeping it in a separate PR unless it is a really simple change.

ACKs for top commit:
  jnewbery:
    Code review utACK e967cae8fa
  MarcoFalke:
    ACK e967cae8fa

Tree-SHA512: 3d12880ff95add9e4a5702afa26ef38080b57b216a608c113a4d0a08ba2d61142c027ba0071c6402add45db90383eee0bada12dc42820dc0d602721d7175edd5
2019-07-29 09:36:55 -04:00
MarcoFalke
3489b71512
Merge #16464: [qa] Ensure we don't generate a too-big block in p2sh sigops test
bf3be5297a [qa] Ensure we don't generate a too-big block in p2sh sigops test (Suhas Daftuar)

Pull request description:

  There's a bug in the loop that is calculating the block size in the p2sh sigops test -- we start with the size of the block when it has no transactions, and then increment by the size of each transaction we add, without regard to the changing size of the encoding for the number of transactions in the block.

  This might be fine if the block construction were deterministic, but the first transaction in the block has an ECDSA signature which can be variable length, so we see intermittent failures of this test when the initial transaction has a 70-byte signature and the block ends up being one byte too big.

  Fix this by double-checking the block size after construction.

ACKs for top commit:
  jonasschnelli:
    utACK bf3be5297a
  jnewbery:
    tested ACK bf3be5297a

Tree-SHA512: f86385b96f7a6feafa4183727f5f2c9aae8ad70060b574aad13b150f174a17ce9a0040bc51ae7a04bd08f2a5298b983a84b0aed5e86a8440189ebc63b99e64dc
2019-07-28 10:12:49 -04:00
Hennadii Stepanov
dde80c272a
Use ArgsManager::NETWORK_ONLY flag 2019-07-27 22:51:58 +03:00
Sjors Provoost
d6b3640ac7
[test] walletcreatefundedpsbt: check RBF is disabled when -walletrbf=0 2019-07-27 19:35:07 +02:00
Sjors Provoost
4fcb698bc2
[rpc] walletcreatefundedpsbt: use wallet default RBF 2019-07-27 19:24:56 +02:00
MeshCollider
c606e6fc53
Merge #15996: rpc: Deprecate totalfee argument in bumpfee
2f7eb772f6 Add RPC bumpfee totalFee deprecation test (Jon Atack)
a92d9ce8cf deprecate totalFee argument in bumpfee RPC call (Gregory Sanders)

Pull request description:

  totalFee argument is of questionable use, and should be removed in favor of feerate-based features.

  I first moved IsDeprecatedRPCEnabled because `bitcoin-wallet` doesn't link `libbitcoin_server`.

ACKs for top commit:
  ryanofsky:
    utACK 2f7eb772f6. Only change since last review is leaving IsDeprecatedRPCEnabled in its happy home, and switching to rpcEnableDeprecated instead. (Thanks!)
  jonatack:
    ACK 2f7eb772f6. Built locally, manually tested rpc bumpfee, help output ([gist](https://gist.github.com/jonatack/863673eacc02f9da39ff6d6712f9d837)), all tests pass. Travis failures appears to be unrelated, the [bitcoin builds are green](https://bitcoinbuilds.org/index.php?build=121).
  meshcollider:
    Code Review ACK 2f7eb772f6

Tree-SHA512: c97465205ee59575df37894bcbb6c4ecf8858dd8fe9d89503f9342b226768c1dcb553153bc9eb3055f7bf5eb41573e48b8efa57e083cd255793cbe5280f0026a
2019-07-27 22:22:03 +12:00
John Newbery
42a5e912ee [mempool] log correct messages when CPFP fails 2019-07-26 16:21:26 -04:00
MarcoFalke
fac2e6a604
test: Fail early on disconnect in mininode.wait_for_* 2019-07-26 16:11:26 -04:00
Jon Atack
2f7eb772f6 Add RPC bumpfee totalFee deprecation test
Next steps: remove `totalFee` from the wallet_bumpfee functional tests.
2019-07-26 14:09:03 -04:00
Gregory Sanders
a92d9ce8cf deprecate totalFee argument in bumpfee RPC call 2019-07-26 14:09:03 -04:00
Suhas Daftuar
bf3be5297a [qa] Ensure we don't generate a too-big block in p2sh sigops test 2019-07-25 14:43:30 -04:00
Suhas Daftuar
d9ab0ffa38 [qa] Fix race condition in example_test.py 2019-07-25 10:32:07 -04:00
fanquake
d5a54ce8f0
Merge #15305: [validation] Crash if disconnecting a block fails
a47df13471 [qa] Test disconnect block failure -> shutdown (Suhas Daftuar)
4433ed0f73 [validation] Crash if disconnecting a block fails (Suhas Daftuar)

Pull request description:

  If we're unable to disconnect a block during normal operation, then that is a
  failure of our local system (such as disk failure) or the chain that we are on
  (eg CVE-2018-17144), but cannot be due to failure of the (more work) chain that
  we're trying to validate.

  We should abort rather than stay on a less work chain.

  Fixes #14341.

ACKs for top commit:
  practicalswift:
    utACK a47df13471
  TheBlueMatt:
    utACK a47df13471. Didn't bother to review the test in detail, it looked fine. Debated whether invalidateblock should ever crash the node, but *not* crashing in the case of hitting a pruned block (which is the only change here) is clearly better, even if there are other cases I'd argue we should crash in.
  ryanofsky:
    utACK a47df13471. Only change since last review is new comment.
  promag:
    ACK a47df1347, it takes awhile to quit (RPC connection timeouts) but that's unrelated - hope to fix that soon.
  fanquake:
    ACK a47df13471

Tree-SHA512: 4dec8cef6e7dbbe513c138fc5821a7ceab855e603ece3c16185b51a3830ab7ebbc844a28827bf64e75326f45325991dcb672f13bd7baede53304f27289c4af8d
2019-07-25 09:05:22 +08:00
Hennadii Stepanov
ffea41f530
Enable all tests in feature_config_args.py 2019-07-24 19:15:10 +03:00
Hennadii Stepanov
753f7cccce
scripted-diff: Make translation bilingual
-BEGIN VERIFY SCRIPT-
sed -i 's/inline std::string _(const char\* psz)/inline bilingual_str _(const char\* psz)/' src/util/translation.h
sed -i 's/return G_TRANSLATION_FUN ? (G_TRANSLATION_FUN)(psz) : psz;/return bilingual_str{psz, G_TRANSLATION_FUN ? (G_TRANSLATION_FUN)(psz) : psz};/' src/util/translation.h
sed -i 's/\b_("\([^"]\|\\"\)*")/&.translated/g' $(git grep --files-with-matches '\b_("' src)
echo Hard cases - multiline strings.
sed -i 's/"Visit %s for further information about the software.")/&.translated/g' src/init.cpp
sed -i "s/\"Only rebuild the block database if you are sure that your computer's date and time are correct\")/&.translated/g" src/init.cpp
sed -i 's/" restore from a backup.")/&.translated/g' src/wallet/db.cpp
sed -i 's/" or address book entries might be missing or incorrect.")/&.translated/g' src/wallet/wallet.cpp
echo Special case.
sed -i 's/_(COPYRIGHT_HOLDERS)/&.translated/' src/util/system.cpp test/lint/lint-format-strings.py
-END VERIFY SCRIPT-
2019-07-24 16:33:20 +03:00
MarcoFalke
0626b8cbdf
Merge #16445: test: Skip flaky p2p_invalid_messages test on macOS
c3dfc91032 test: Skip flaky p2p_invalid_messages test on macOS (Fabian Jahr)

Pull request description:

  This mitigates https://github.com/bitcoin/bitcoin/issues/15400

  I had a look into the issue today and this seems to be the best we can do given that the root causes some unexpected custom error code from the macOS kernel that python/asyncio doesn't know how to handle properly yet.

Top commit has no ACKs.

Tree-SHA512: 20a0551d45c405b6eb0ae58190b701c7026c52eff6c434bc678f723a4dabf0074e5b52a8bb3d51ee7132dc29419d1e67a24696761c444c62cd4d429ec768e67d
2019-07-24 07:18:45 -04:00
William Casarin
72eaab073b tests: functional watch-only wallet tests
These test the new watch-only defaults for rpcs with include_watchonly
and includeWatching options.

Signed-off-by: William Casarin <jb55@jb55.com>
2019-07-24 03:33:53 -07:00
MarcoFalke
67923d6b3c
Merge #16366: init: Use InitError for all errors in bitcoind/qt
fa6f402bde Call node->initError instead of InitError from GUI code (Russell Yanofsky)
fad2502240 init: Use InitError for all errors in bitcoind/qt (MarcoFalke)

Pull request description:

  Using the same InitError for startup error in the daemon and the gui makes it possible to run the tests with the gui again:

  ```sh
  BITCOIND=bitcoin-qt ./test/functional/test_runner.py feature_includeconf feature_config_args

ACKs for top commit:
  hebasto:
    ACK fa6f402bde
  ryanofsky:
    utACK fa6f402bde. Only changes since last review are removing more includes and adding Node::initError method to avoid accessing node `InitError` function and global variables from GUI code.

Tree-SHA512: bd19e08dcea4019dfe40356bc5c63cb583cefed54b6c9dcfb82f1b5b00308d8e2b363549afcaea5e93bf83864dbe0917400c3b70f43a8a5bdff45c9cd34cc294
2019-07-23 18:40:46 -04:00
Fabian Jahr
c3dfc91032 test: Skip flaky p2p_invalid_messages test on macOS 2019-07-23 15:59:27 -04:00
Fabian Jahr
ba1f128d6c Return error for ignored passphrase through disable private keys option 2019-07-19 14:34:33 -04:00
Wladimir J. van der Laan
51a6e2c419
Merge #15681: [mempool] Allow one extra single-ancestor transaction per package
50cede3f5a [mempool] Allow one extra single-ancestor transaction per package (Matt Corallo)

Pull request description:

  This implements the proposed policy change from [1], which allows
  certain classes of contract protocols involving revocation
  punishments to use CPFP. Note that some such use-cases may still
  want some form of one-deep package relay, though even this alone
  may greatly simplify some lightning fee negotiation.

  [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-November/016518.html

ACKs for top commit:
  ajtowns:
    ACK 50cede3f5a -- looked over code again, compared with previous commit, compiles, etc.
  sdaftuar:
    ACK 50cede3f5a
  ryanofsky:
    utACK 50cede3f5a. Changes since last review: adding EXTRA_DESCENDANT_TX_SIZE_LIMIT constant, changing max ancestor size from 1,000,000 to nLimitAncestorSize constant (101,000), fixing test comment and getting rid of unused test node.

Tree-SHA512: b052c2a0f384855572b4579310131897b612201214b5abbb225167224e4f550049e300b471dbf320928652571e92ca2d650050b7cf39ac92b3bc1d2bcd386c1c
2019-07-19 20:00:12 +02:00
fanquake
59ce537a49
Merge #16152: Disable bloom filtering by default.
bead32e31e Add release notes for DEFAULT_BLOOM change (Matt Corallo)
f27309f55c Move DEFAULT_PEERBLOOMFILTERS from validation.h to net_processing.h (Matt Corallo)
5efcb77283 Disable bloom filtering by default. (Matt Corallo)

Pull request description:

  BIP 37 bloom filters have been well-known to be a significant DoS
  target for some time. However, in order to provide continuity for
  SPV clients relying on it, the NODE_BLOOM service flag was added,
  and left as a default, to ensure sufficient nodes exist with such a
  flag.

  NODE_BLOOM is, at this point, well-established and, as long as
  there exist 0.18 nodes with default config (which I'd anticipate
  will be true for many years), will be available from some peers. By
  that time, the continued slowdown of BIP 37-based filtering will
  likely have rendered it useless (though this is already largely the
  case). Further, BIP 37 was deliberately never updated to support
  witness-based filtering as newer wallets are expected to migrate to
  some yet-to-be-network-exposed filters.

ACKs for top commit:
  jnewbery:
    ACK bead32e31e
  kallewoof:
    ACK bead32e31e

Tree-SHA512: ecd901898e8efe1a7c82b471af0acc2373c2282ac633eb58d9aae7c35deda1999d0f79fb0485e6cecbda7246aeda00206cd82c7fa36866e2ac64705ba93f9390
2019-07-19 17:33:56 +08:00
Jonas Schnelli
024ecd7e01
QA: Fix race condition in wallet_encryption test 2019-07-18 22:31:24 +02:00
fanquake
0515406acb
Merge #16374: test: Enable passing wildcard test names to test runner from root
e142ee03e7 doc: describe how to pass wildcard names to test runner (Jon Atack)
6a7a70b8cf test: enable passing wildcards with path to test runner (Jon Atack)

Pull request description:

  Currently, passing wildcard testname args to the test runner from outside the test/functional/ directory does not work, even though developers expect it to. See these recent IRC discussions for more background: http://www.erisian.com.au/bitcoin-core-dev/log-2019-07-10.html#l-262 (lines 262 to 323) and http://www.erisian.com.au/bitcoin-core-dev/log-2019-07-11.html#l-134.

  1. [BUGFIX] Enable passing wildcards with paths. Examples:
      - `test/functional/test_runner.py test/functional/wallet*`
      - `functional/test_runner.py functional/wallet*`
      - `test/functional/test_runner.py ./test/functional/tool* test/functional/mempool*`
      - A current limitation this PR does not change: 9 test files with arguments in their filename are not picked up by wildcard search.

  2. [Docs] Describe how to pass wildcard names (multiple and with paths) to the test runner in test/README.md.

ACKs for top commit:
  jnewbery:
    tested ACK e142ee03e7
  jachiang:
    Tested ACK e142ee03e7. Thanks a lot for this fix!
  MarcoFalke:
    ACK e142ee03e7, fine with me

Tree-SHA512: cb3d994880cdc9b8918546b573a25faa5b4c7339826ac7cfe20f076aac6e731a34271609c0cf5a7ee5e4a2d5ae205298319d24bf36ef5b5d569a1a0c57883e54
2019-07-18 10:05:08 +08:00
MarcoFalke
24dbcf3808
Merge #15891: test: Require standard txs in regtest by default
fa89badf88 test: Require standard txs in regtest (MarcoFalke)
fa9b419160 test: Add test that mainnet requires standard txs (MarcoFalke)
fa613ca0a8 chainparams: Remove unused fMineBlocksOnDemand (MarcoFalke)

Pull request description:

  I don't see a reason why regtest should allow non-standard txs, as it makes testing mainnet behaviour such as #15846 unnecessarily hard and unintuitive.

  Of course, testnet policy remains unchanged to allow propagation of non-standard txs.

ACKs for top commit:
  ajtowns:
    ACK fa89badf88

Tree-SHA512: c4c675affb054868850bd2683aa07f4c741a448cbacb2ea8334191e105f426b0790fe6a468be61e9c5880d24154f7bf1c7075051697172dce92180c1bc3a1c90
2019-07-16 16:10:17 -04:00
MarcoFalke
8f9725c83f
Merge #16390: qa: Add --filter option to test_runner.py
1a62425260 qa: Add --filter option to test_runner.py (João Barbosa)

Pull request description:

  Allows to run functional tests like:
  ```sh
  test/functional/test_runner.py --filter wallet
  ```

ACKs for top commit:
  jonatack:
    ACK 1a62425260

Tree-SHA512: 53199e01da3b2e0112843c1c68c69d8fd7fc9bb6a6cb45a81c324973c4824ebf5fef574f9efab81a64d52e397e25d979ae40f0eaba35afb771e80012768f0b08
2019-07-16 09:01:34 -04:00
João Barbosa
1a62425260 qa: Add --filter option to test_runner.py 2019-07-16 01:00:27 +01:00
MarcoFalke
0822b44d8a
Merge #15282: test: Replace hard-coded hex tx with class in test framework
8f250ab788 TEST: Replace hard-coded hex tx with classes (Steven Roose)

Pull request description:

  Came across these breaking Elements.

ACKs for top commit:
  MarcoFalke:
    ACK 8f250ab788
  instagibbs:
    utACK 8f250ab788

Tree-SHA512: e8615dad4cda0beea4b0c7d4951a467fb9882a0a64d49c9b5ecf167369ea62a3fe5348e2401153162b0ccadecdb128492c94be36ebb881c3c42659626d86eda8
2019-07-15 08:02:07 -04:00
Jon Atack
e142ee03e7
doc: describe how to pass wildcard names to test runner 2019-07-15 10:14:23 +02:00
Jon Atack
6a7a70b8cf
test: enable passing wildcards with path to test runner
Currently, passing wildcard testname args to the test runner from outside the `test/functional/` directory does not work. See this recent IRC discussion for more background: http://www.erisian.com.au/bitcoin-core-dev/log-2019-07-10.html#l-262 (lines 262 to 323).

This small change enables passing multiple wildcards with paths, as long as the paths are coherent. Examples:
  - test/functional/test_runner.py test/functional/wallet*
  - functional/test_runner.py functional/wallet*
  - test/functional/test_runner.py ./test/functional/tool* test/functional/mempool*

A current limitation that this PR does not change: 9 test files with arguments in their name are not picked up by wildcard search.

- Squashed commit: non-mutating version

- Squashed commit: minor code optimisation
2019-07-15 10:13:39 +02:00
Wladimir J. van der Laan
536590f358
Merge #16334: test: rpc_users: Also test rpcauth.py with password.
e263a343d4 test: rpc_users: Make variable names more clear. (Carl Dong)
830dc2dd0f test: rpc_users: Also test rpcauth.py with specified password. (Carl Dong)
c73d871799 test: rpc_users: Add function for testing auth params. (Carl Dong)
604e2a997f test: rpc_users: Add function for auth'd requests. (Carl Dong)

Pull request description:

  Fixes #14758

  First two commits are tidy-ups which I feel are worthwhile as they are very straightforward, cut down the file by 50%, and made the final diff more minimal. Happy to squash after review.

ACKs for top commit:
  laanwj:
    ACK e263a343d4

Tree-SHA512: aa75c48570a87060238932d4c68e17234e158077f6195fb4917367e1ecc565e3cd8dd0ae51f9159ddd3d03742739680391bc1246454302db22d4a608c0633e80
2019-07-12 21:28:47 +02:00
MarcoFalke
fad2502240
init: Use InitError for all errors in bitcoind/qt
Also, remove unused <boost/thread.hpp> include (and others)
2019-07-11 19:39:25 -04:00
Wladimir J. van der Laan
6c1e45c4c4
Merge #16322: wallet: Fix -maxtxfee check by moving it to CWallet::CreateTransaction
0d101a340c test: Add test for maxtxfee option (MarcoFalke)
177550101b wallet: Remove unreachable code in CreateTransaction (MarcoFalke)
5c1b9714cb wallet: Fix -maxtxfee check by moving it to CWallet::CreateTransaction (João Barbosa)

Pull request description:

  Follow up to #16257, this PR makes `bumpfee` aware of `-maxtxfee`.

  It also prevents dangling locked unspents when calling `fundrawtransaction` - because the previous check was after `LockCoin`.

ACKs for top commit:
  MarcoFalke:
    re-ACK 0d101a340c, only change is small test fixup

Tree-SHA512: 3464b24ae7cd4e72ed41438c6661828ba1304af020f05da62720b23668ae734e16cf47c6d97e150cc84ef631ee099b16fc786c858f3d089905845437338fd512
2019-07-10 14:00:52 +02:00
MarcoFalke
0d101a340c test: Add test for maxtxfee option 2019-07-10 11:58:46 +01:00
Matt Corallo
50cede3f5a [mempool] Allow one extra single-ancestor transaction per package
This implements the proposed policy change from [1], which allows
certain classes of contract protocols involving revocation
punishments to use CPFP. Note that some such use-cases may still
want some form of one-deep package relay, though even this alone
may greatly simplify some lightning fee negotiation.

[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-November/016518.html
2019-07-09 15:46:25 -04:00
MarcoFalke
fa8489a155
test: Add test for BIP30 duplicate tx 2019-07-09 14:33:07 -04:00
MarcoFalke
77770d95e2
test: Properly serialize BIP34 coinbase height 2019-07-09 14:12:33 -04:00
Carl Dong
e263a343d4
test: rpc_users: Make variable names more clear. 2019-07-08 16:13:35 -04:00
Carl Dong
830dc2dd0f
test: rpc_users: Also test rpcauth.py with specified password. 2019-07-08 16:13:34 -04:00
Carl Dong
c73d871799
test: rpc_users: Add function for testing auth params. 2019-07-08 16:13:33 -04:00
Carl Dong
604e2a997f
test: rpc_users: Add function for auth'd requests. 2019-07-08 16:13:32 -04:00
Wladimir J. van der Laan
345f42a9e3
Merge #14505: test: Add linter to make sure single parameter constructors are marked explicit
c4606b8432 Add Travis check for single parameter constructors not marked "explicit" (practicalswift)

Pull request description:

  Make single parameter constructors `explicit` (C++11).

  Rationale from the developer notes:

  > - By default, declare single-argument constructors `explicit`.
  >   - *Rationale*: This is a precaution to avoid unintended conversions that might
  >   arise when single-argument constructors are used as implicit conversion
  >   functions.

ACKs for top commit:
  laanwj:
    ACK c4606b8432

Tree-SHA512: 3e6fd51935fd93b2604b2188664692973d0897469f814cd745b5147d71b99ea5d73c1081cfde9f6393f51f56969e412fcda35d2d54e938a3235b8d40945f31fd
2019-07-08 20:29:00 +02:00
MarcoFalke
4882040182
Merge #16291: gui: Stop translating PACKAGE_NAME
fa64b947bb util: No translation of `Bitcoin Core` in the copyright (MarcoFalke)
fab85208f6 qt: Run «make translate» in ./src/ (MarcoFalke)
fabe87d2c9 scripted-diff: Avoid passing PACKAGE_NAME for translation (MarcoFalke)
fa5e9f157e build: Stop translating PACKAGE_NAME (MarcoFalke)

Pull request description:

  Generally the package name is not translated, but the package description is.

  E.g. `GIMP` or `Firefox` are always called that way regardless of the system language. However, "`Firefox` webbrowser" or "`GIMP` image manipulation program" are translated.

ACKs for top commit:
  hebasto:
    ACK fa64b947bb, I have not tested the code, but I have reviewed it and it looks OK, I agree it can be merged.

Tree-SHA512: 626f811531182d0ba0ef1044930d32726773349bcb49b10261288a86ee6b80a183db30a87d817d5b0d501fad058ac22d6272311716b4f5a154f17c6f391a5a1a
2019-07-08 13:39:59 -04:00
Wladimir J. van der Laan
b5fa2319d8
Merge #15687: test: tool wallet test coverage for unexpected writes to wallet
7195fa792f test: Tool wallet test coverage for unexpected writes to wallet (Jon Atack)
3bf2b3a37b test: Split tool_wallet.py test into subtests (Jon Atack)
1eb13f09a9 test: Add log messages to test/functional/tool_wallet.py (Jon Atack)

Pull request description:

  This pull request adds test coverage in `test/functional/tool_wallet.py` to reproduce unexpected writes to the wallet as described in https://github.com/bitcoin/bitcoin/issues/15608 and serve as a benchmark for fixing the issue:

  - Wallet tool `info` unexpectedly writes to the wallet file if the wallet file permissions are read/write.

  - Wallet tool `info` raises with "Error loading . Is wallet being used by another process?" if the wallet file permissions are read-only.

  Goals:

  1. Reproduce the reported issue, define the current unexpected behavior, and add test coverage to guide a future fix. Add debug-level logging for sanity checking and commented-out assertions to be uncommented when fixing the issue. Add the same coverage to the wallet tool create test and the getwalletinfo test as regression tests while fixing the issue.

  2. Add info log messages as there are currently none in the test file.

  3. Split the tests out to separate functions as per review feedback.

  Thanks to Marco Falke for pointing me in the right direction.

ACKs for top commit:
  laanwj:
    code review ACK 7195fa792f

Tree-SHA512: 16a41cce989c8f819cf5b02c6cf8ea84653ede2738fb402f6c36cf4dc075b424dff3e2c73a1cfa1ec9c75f614675baecc71e588845a2596db06ba0957db2df7b
2019-07-08 14:57:06 +02:00
Jon Atack
7195fa792f
test: Tool wallet test coverage for unexpected writes to wallet
This commit adds test coverage in `test/functional/tool_wallet.py` to reproduce unexpected writes to the wallet as described in https://github.com/bitcoin/bitcoin/issues/15608:

    - wallet tool `info` unexpectedly writes to the wallet file if the wallet file permissions are read/write.

    - wallet tool `info` raises with "Error loading . Is wallet being used by another process?" if the wallet file permissions are read-only.

1. Reproduce the reported issue, define the current unexpected behavior, and add test coverage to guide a future fix in the form of commented-out assertions to be uncommented when testing/fixing.

2. Provisionally extend the same coverage to the wallet tool create test and the getwalletinfo test as regression tests while fixing the issue.

3. Add some logging for sanity checking.

------

Changes after rebase:

5. Make wallet_path an instance method instead of a function in tool_wallet.py as per Marco Falke review suggestion.

6. Assert wallet permissions instead of logging them in tool_wallet.py. This ran into an issue with Appveyor keeping permissions at 666 so allowed for 666 as a workaround.

7. Change the added logging from info to debug level.

8. More helpful assertions order in tool_wallet.py#assert_tool_output. This change makes #assert_tool_output raise "Error loading wallet.dat. Is wallet being used by another process?" rather than a less-helpful message when debugging the read-only wallet permissions issue.
2019-07-08 13:02:28 +02:00
Jon Atack
3bf2b3a37b
test: Split tool_wallet.py test into subtests
as per Marco Falke review suggestion.
2019-07-08 13:02:23 +02:00
Jon Atack
1eb13f09a9
test: Add log messages to test/functional/tool_wallet.py
and update code comments as per Python PEP 8 style guide.
2019-07-08 12:33:41 +02:00
fanquake
584168c7f9
Merge #16326: [RPC] add new utxoupdatepsbt arguments to the CRPCCommand and CPRCCvertParam tables
91cc18f602 [docs] Add release notes for PR 15427 (John Newbery)
3b11420b3c [RPC] add new utxoupdatepsbt arguments to the CRPCCommand and CPRCConvertParam tables (John Newbery)

Pull request description:

  The new `descriptors` argument was not added to the CRPCCommand and CPRCCvertParam tables, meaning that it couldn't be used with bitcoin-cli or named arguments.

  Before this PR:

  ```
  > bitcoin-cli utxoupdatepsbt 'cHNidP8BAFMCAAAAAYCdwVRx2X3o4KHx5tAMsN1ddp51MbfWsietjfMbl5HtAAAAAAD/////AQDh9QUAAAAAF6kUW+rtEOi4nk9rpw2F5XZl1dd8ehGHAAAAAAAAAA==' "[{\"desc\":\"sh(wpkh([bd50871a/0h/0h/0h]03895c66337b38699bfafff1084ad35bc347fac4f4e5e5fe5eb7dd81155280db53))\"}]"
  error code: -3
  error message:
  Expected type array, got string
  > bitcoin-cli --named utxoupdatepsbt psbt='cHNidP8BAFMCAAAAAYCdwVRx2X3o4KHx5tAMsN1ddp51MbfWsietjfMbl5HtAAAAAAD/////AQDh9QUAAAAAF6kUW+rtEOi4nk9rpw2F5XZl1dd8ehGHAAAAAAAAAA==' descriptors="[{\"desc\":\"sh(wpkh([bd50871a/0h/0h/0h]03895c66337b38699bfafff1084ad35bc347fac4f4e5e5fe5eb7dd81155280db53))\"}]"
  error code: -8
  error message:
  Unknown named parameter descriptors
  ```

  After this PR:

  ```
  bitcoin-cli utxoupdatepsbt 'cHNidP8BAFMCAAAAAYCdwVRx2X3o4KHx5tAMsN1ddp51MbfWsietjfMbl5HtAAAAAAD/////AQDh9QUAAAAAF6kUW+rtEOi4nk9rpw2F5XZl1dd8ehGHAAAAAAAAAA==' "[{\"desc\":\"sh(wpkh([bd50871a/0h/0h/0h]03895c66337b38699bfafff1084ad35bc347fac4f4e5e5fe5eb7dd81155280db53))\"}]"
  cHNidP8BAFMCAAAAAYCdwVRx2X3o4KHx5tAMsN1ddp51MbfWsietjfMbl5HtAAAAAAD/////AQDh9QUAAAAAF6kUW+rtEOi4nk9rpw2F5XZl1dd8ehGHAAAAAAAAAA==
  bitcoin-cli --named utxoupdatepsbt psbt='cHNidP8BAFMCAAAAAYCdwVRx2X3o4KHx5tAMsN1ddp51MbfWsietjfMbl5HtAAAAAAD/////AQDh9QUAAAAAF6kUW+rtEOi4nk9rpw2F5XZl1dd8ehGHAAAAAAAAAA==' descriptors="[{\"desc\":\"sh(wpkh([bd50871a/0h/0h/0h]03895c66337b38699bfafff1084ad35bc347fac4f4e5e5fe5eb7dd81155280db53))\"}]"
  cHNidP8BAFMCAAAAAYCdwVRx2X3o4KHx5tAMsN1ddp51MbfWsietjfMbl5HtAAAAAAD/////AQDh9QUAAAAAF6kUW+rtEOi4nk9rpw2F5XZl1dd8ehGHAAAAAAAAAA==
  ```

ACKs for top commit:
  promag:
    ACK 91cc18f.
  fanquake:
    re-ACK 91cc18f602

Tree-SHA512: 279b2339a5cac17e363002e4ab743e251d6757c904c89f1970575bdce18d4f63d5e13507e171bf2bdc1bf6dd457db345a4b11b15d4ff71b96c2fedc4ffe52b23
2019-07-07 09:51:58 +08:00
fanquake
1088b90cba
Merge #16327: scripts and tools: Update ShellCheck linter
1ac454a384 Enable ShellCheck rules (Hennadii Stepanov)

Pull request description:

  Enable some simple ShellCheck rules.

  Note for reviewers: `bash` and `shellcheck` on macOS are different from ones on Ubuntu.
  For local tests the latest `shellcheck` version 0.6.0 should be used (see #15166).

ACKs for top commit:
  practicalswift:
    utACK 1ac454a384
  dongcarl:
    utACK 1ac454a
  fanquake:
    ACK 1ac454a384

Tree-SHA512: 8d0a3a5c09fe1a0c22120178f5e6b80f81f746f8c3356b7701ff301c117acb2edea8fe08f08fb54ed73f94b1617515fb239fa28e7ab4121f74872e6494b6f20e
2019-07-05 09:19:23 +08:00
Hennadii Stepanov
1ac454a384
Enable ShellCheck rules
Enabled ShellCheck rules:
  SC1087
  SC2001
  SC2004
  SC2005
  SC2006
  SC2016
  SC2028
  SC2048
  SC2066 (note that IFS already contains only a line feed)
  SC2116
  SC2166
  SC2181
  SC2206
  SC2207
  SC2230
  SC2236
2019-07-04 19:35:25 +03:00
John Newbery
3b11420b3c [RPC] add new utxoupdatepsbt arguments to the CRPCCommand and CPRCConvertParam tables
The new `descriptors` argument needs to be added to the Command and
ConvertParams tables to by usable as a named argument and by
bitcoin-cli.

Also update the test to use named arguments to test this.
2019-07-04 08:02:23 -04:00
MarcoFalke
1381ddbcfc
Merge #16329: test: Add tests for getblockchaininfo.softforks
faf6caf4da test: Add tests for getblockchaininfo.softforks (MarcoFalke)

Pull request description:

ACKs for top commit:
  laanwj:
    Code review ACK faf6caf4da

Tree-SHA512: 8a0bb3b648f18fdba7a36a960d70c6217fd7312cf2ef52b3b911be0d7f1d27c5c50856946d7e6cb81d96c081913b7308cc5f9d89af34518439ff4ada024441da
2019-07-03 12:11:46 -04:00
Wladimir J. van der Laan
11de669d8b
Merge #16325: rpc: Clarify that block count means height excl genesis
fab0c820fa rpc: Clarify that block count means height excl genesis (MarcoFalke)

Pull request description:

  There is a common misconception that the block count returned by the blockchain rpcs includes the genesis block. See for example the discussion in https://github.com/bitcoin/bitcoin/pull/16292#issuecomment-506303256.

  However, it really returns the height, which is `0` for the genesis block.

  So clarify that and also remove the misleading "longest blockchain" comment.

  Finally, fix the wallet test that incorrectly used this rpc.

ACKs for top commit:
  instagibbs:
    utACK fab0c820fa
  promag:
    ACK fab0c82, sorry for the misconception.

Tree-SHA512: 0d087cbb628d3866352bca6420402f392e6a997e579941701a408a7fca355d84645045661f39b022e4479cc07f85a6cddaa9095b6fd9911b245692482420a5e4
2019-07-03 14:49:40 +02:00
Wladimir J. van der Laan
9339008a9d
Merge #15483: rpc: Adding a 'logpath' entry to getrpcinfo
8a6810d0d2 Add a 'logpath' field to getrpcinfo (darosior)

Pull request description:

  as discussed in #15438

ACKs for top commit:
  laanwj:
    Tested ACK 8a6810d0d2

Tree-SHA512: 752c7d90f670677c8144efb338c5c97c2264f85f1e65e031fd5a44f04230b6eafbabd0f634db263eb42c25642ecc1c4b1b602d4735e3fab07ec00b566134ddab
2019-07-03 14:35:58 +02:00
MarcoFalke
e06067387e
Merge #16250: signrawtransactionwithkey: report error when missing redeemScript/witnessScript
01174596e6 signrawtransactionwithkey: report error when missing redeemScript/witnessScript param (Anthony Towns)

Pull request description:

  Adding support for "witnessScript" as an alternative to "redeemScript" when using "signrawtransactionwithkey" meant that the `RPCTypeCheckObj()` call in `SignTransaction` can't error out just because either parameter is missing -- it's only a problem if both are missing, which isn't a state `RPCTypeCheckObj()` tests for. This results in the regression described in #16249. This patch adds some code to test for this case and give a similar error, namely:

      error code: -8
      error message:
      Missing redeemScript/witnessScript

  Fixes: #16249

ACKs for top commit:
  meshcollider:
    utACK 01174596e6
  promag:
    ACK 01174596e. Could also write test without `dict`/`del`:

Tree-SHA512: cf51346b7dea551b7f18f2a93c2a336a293b2535c62c03a5263cd2be8c58cf0cc302891da659c167e88ad1a68a756472c3c07e99f71627c61d32886fc5a3a353
2019-07-02 17:21:02 -04:00
MarcoFalke
faf8318c55
test: Split fundrawtx test into subtests 2019-07-02 17:19:18 -04:00
MarcoFalke
fa6fba3bc8
test: Make local symbols in run_test members 2019-07-02 17:17:43 -04:00
MarcoFalke
faf6caf4da
test: Add tests for getblockchaininfo.softforks 2019-07-02 15:25:13 -04:00
MarcoFalke
fab0c820fa
rpc: Clarify that block count means height excl genesis 2019-07-02 12:28:21 -04:00
João Barbosa
5c1b9714cb wallet: Fix -maxtxfee check by moving it to CWallet::CreateTransaction 2019-07-02 16:13:39 +01:00
Wladimir J. van der Laan
2f717fb5cd
Merge #15427: Add support for descriptors to utxoupdatepsbt
26fe9b9909 Add support for descriptors to utxoupdatepsbt (Pieter Wuille)
3135c1a2d2 Abstract out UpdatePSBTOutput from FillPSBT (Pieter Wuille)
fb90ec3c33 Abstract out EvalDescriptorStringOrObject from scantxoutset (Pieter Wuille)
eaf4f88734 Abstract out IsSegWitOutput from utxoupdatepsbt (Pieter Wuille)

Pull request description:

  This adds a descriptors argument to the `utxoupdatepsbt` RPC. This means:
  * Input and output scripts and keys will be filled in when known.
  * P2SH-witness inputs will be filled in from the UTXO set when a descriptor is provided that shows they're spending segwit outputs.

  This also moves some (newly) shared code to separate functions: `UpdatePSBTOutput` (an analogue to `SignPSBTInput`), `IsSegWitOutput`, and `EvalDescriptorStringOrObject` (implementing the string or object notation parsing used in `scantxoutset`).

ACKs for top commit:
  jnewbery:
    utACK 26fe9b9909
  laanwj:
    utACK 26fe9b9909 (will hold merging until response to promag's comments)
  promag:
    ACK 26fe9b9, checked refactors and tests look comprehensive. Still missing a release note but can be added later.

Tree-SHA512: 1d833b7351b59d6c5ded6da399ff371a8a2a6ad04c0a8f90e6e46105dc737fa6f2740b1e5340280d59e01f42896c40b720c042f44417e38dfbee6477b894b245
2019-07-02 16:53:22 +02:00
Wladimir J. van der Laan
6c21a801f3
Merge #15538: wallet_bumpfee.py: Make sure coin selection produces change
276972cb95 wallet_bumpfee.py: Make sure coin selection produces change (Gregory Sanders)

Pull request description:

  I was hitting the case where change-less transactions were being made.

ACKs for top commit:
  ryanofsky:
    utACK 276972cb95

Tree-SHA512: e2b7a50363daddd3ee749cacfc9d3d685a6c0c7e3e48118bb60131d205bf83ea06cdd66b69dfa3bd4dbb3bbf2b5b673d7225171486ae72fc762e5dabe2c01ef5
2019-07-02 14:49:26 +02:00
Steven Roose
8f250ab788
TEST: Replace hard-coded hex tx with classes 2019-07-01 18:06:01 +01:00
Wladimir J. van der Laan
1212808762
Merge #16257: [wallet] abort when attempting to fund a transaction above -maxtxfee
806b0052c3 [wallet] abort when attempting to fund a transaction above maxtxfee (Sjors Provoost)

Pull request description:

  `FundTransaction` calls `GetMinimumFee` which, when the fee rate is absurdly high, quietly reduces the fee to `-maxtxfee`.

  Becaue an absurdly high fee rate is usually the result of a fat finger, aborting seems safer behavior.

  Before:
  ```
  bitcoin-cli walletcreatefundedpsbt '[]' '[{"tb1q...": 0.01}]' 0 '{"feeRate": 10}' true
  {
    "psbt": "cHNidP8...gAA=",
    "fee": 0.10000000,
    "changepos": 1
  }

  ```

  After:
  ```
  bitcoin-cli walletcreatefundedpsbt '[]' '[{"tb1q...": 0.01}]' 0 '{"feeRate": 10}' true
  error code: -25
  error message:
  Fee exceeds maximum configured by -maxtxfee
  ```

  QT still checks the max fee rate as expected:
  <img width="566" alt="Schermafbeelding 2019-06-20 om 19 52 00" src="https://user-images.githubusercontent.com/10217/59888424-a2aa7100-9395-11e9-8ae6-8a3c1f7de585.png">

ACKs for top commit:
  laanwj:
    Code review ACK 806b0052c3

Tree-SHA512: bee95811711cdab100b614d2347921407af3b400aea613ca156953ed3f60b924ad29a1d335bd0e240c0b7c0fbb360226bab03294d226a5560cdf2a3f21e6d406
2019-07-01 16:03:37 +02:00
Sjors Provoost
806b0052c3
[wallet] abort when attempting to fund a transaction above maxtxfee
FundTransaction calls GetMinimumFee which, when the fee rate is absurdly high, quietly reduced the fee to -maxtxfee. Becaue an absurdly high fee rate is usually the result of a fat finger, aborting seems safer behavior.
2019-06-28 22:44:38 -04:00
MarcoFalke
fa815255c7
test: Add missing sync_all to wallet_balance test 2019-06-28 11:19:37 -04:00
MarcoFalke
fa64b947bb
util: No translation of Bitcoin Core in the copyright 2019-06-27 15:06:46 -04:00
practicalswift
c4606b8432 Add Travis check for single parameter constructors not marked "explicit" 2019-06-26 16:57:14 +02:00
Anthony Towns
01174596e6 signrawtransactionwithkey: report error when missing redeemScript/witnessScript param 2019-06-25 12:37:08 +10:00
MeshCollider
2cbcc55ba6
Merge #16239: wallet/rpc: follow-up clean-up/fixes to avoid_reuse
71d0344cf2 docs: release note wording (Karl-Johan Alm)
3d2ff37913 wallet/rpc: use static help text (Karl-Johan Alm)
53c3c1ea9e wallet/rpc/getbalances: add entry for 'mine.used' balance in results (Karl-Johan Alm)

Pull request description:

  This addresses a few remaining issues pointed out in #13756:

  * First commit addresses https://github.com/bitcoin/bitcoin/pull/13756#discussion_r284907468
  * Second commit addresses https://github.com/bitcoin/bitcoin/pull/13756#discussion_r294868973

  Ping jnewbery and achow101 as they pointed out these issues.

ACKs for commit 71d034:
  jnewbery:
    ACK 71d0344cf2
  meshcollider:
    re-utACK 71d0344cf2

Tree-SHA512: 5e28822af0574ad07dbbed21aa2fe7866bf5770b4c0a1c150ad0da8af3152bcfb7170330a7497fa500326c594740ecf63733cf58325821e2811d7b911d5783a0
2019-06-22 22:00:10 +12:00
MarcoFalke
fa89badf88
test: Require standard txs in regtest 2019-06-21 16:45:16 -04:00
Karl-Johan Alm
53c3c1ea9e
wallet/rpc/getbalances: add entry for 'mine.used' balance in results 2019-06-22 02:45:40 +09:00
MeshCollider
fd333e15a5
Merge #16226: Move ismine to the wallet module
e61de6306f Change ismine to take a CWallet instead of CKeyStore (Andrew Chow)
7c611e2000 Move ismine to wallet module (Andrew Chow)

Pull request description:

  `IsMine` isn't used outside of the wallet except for the tests. It also doesn't make sense to be outside of the wallet. This PR moves `IsMine` into the wallet module and for it to take a `CWallet` instead of `CKeyStore`. The test that used `IsMine` is also moved to the wallet tests.

  This is first [prerequisites](https://github.com/bitcoin-core/bitcoin-devwiki/wiki/Wallet-Class-Structure-Changes#ismine) for the wallet structure changes.

ACKs for commit e61de6:
  MarcoFalke:
    re-ACK e61de6306f (only change is rebase with git auto-merge)
  meshcollider:
    Very light code review ACK e61de6306f

Tree-SHA512: 1cb4ad12652aef7922ab7460c6d413e8b9d1855dca78c0a286ae49d5c0765bc7996c55f262c742001d434eb9bd4215dc2cc7aae1b371ee1a82d46b32c17e6341
2019-06-21 19:59:48 +12:00
MeshCollider
303ec103ba
Merge #16026: Ensure that uncompressed public keys in a multisig always returns a legacy address
a49503402b Make and get the multisig redeemscript and destination in one function instead of two (Andrew Chow)

Pull request description:

  `CreateMultisigRedeemscript()` is changed to `AddAndGetMultisigDestination()` so that the process of constructing the redeemScript and then getting the `CTxDestination` are done in the same function. This allows that function to see what the keys in the multisig are so that the correct address type is returned from `AddAndGetDestinationForScript()`.

  This only effects the `createmultisig` and `addmultisigaddress` RPCs and does not change signing logic as #16022 does.

  Alternative to #16022 and #16012

  Fixes #16011

ACKs for commit a49503:

Tree-SHA512: 5b0154a714deea3b2cc3a54beb420c95eeeacf4ca30c40ca80940d9d640f8b03611b0fc14c2f0710bfd8a79e8d27ad7d9ae380b4b83d52b40ab201624f2a63f0
2019-06-21 19:44:08 +12:00
MarcoFalke
ea45967e85
Merge #16234: test: Add test for unknown args
fa7dd88b71 test: Add test for unknown args (MarcoFalke)

Pull request description:

  Currently uncovered.

  Further reading:

  * https://marcofalke.github.io/btc_cov/total.coverage/src/util/system.cpp.gcov.html
  *  Fail on unknown config file options #15021

ACKs for commit fa7dd8:
  promag:
    ACK fa7dd88b71, tests looks good to me.
  hebasto:
    ACK fa7dd88b71, I have tested the code.

Tree-SHA512: 86ab370ce8e85925f945a52e81457b5678d71bbabcef01205a97782b780003f363552e0bad1ff678bccc784f82c6b511c3b88de3f8f25f62b0b713c387950564
2019-06-20 16:59:17 -04:00
Andrew Chow
a49503402b Make and get the multisig redeemscript and destination in one function instead of two
Instead of creating a redeemScript with CreateMultisigRedeemscript and
then getting the destination with AddAndGetDestinationForScript, do
both in the same function.

CreateMultisigRedeemscript is changed to AddAndGetMultisigDestination.
It creates the redeemScript and returns it via an output parameter. Then
it calls AddAndGetDestinationForScript to add the destination to the
keystore and get the proper destination.

This allows us to inspect the public keys in the redeemScript before creating
the destination so that the correct destination is used when uncompressed
pubkeys are in the multisig.
2019-06-20 11:02:00 -04:00
Andrew Chow
e61de6306f Change ismine to take a CWallet instead of CKeyStore 2019-06-19 18:06:30 -04:00
MarcoFalke
413e438ea9
Merge #16243: doc: Remove travis badge from readme
e91f0a7af2 doc: Remove travis badge from readme (MarcoFalke)

Pull request description:

  The readme(s) are shipped in the released source-code archive, in which case the travis badge is useless since it doesn't link to the travis result of the correct commit/tag/branch. GitHub embeds the correct links for each tag or commit that ci ran on, so we don't need this link in the readme.

ACKs for commit e91f0a:
  hebasto:
    ACK e91f0a7af2

Tree-SHA512: 860435a58b38a9bd0bc62a1e74b3a63c138c9a2f09008a090d5ecc7fd86fa908d2e5eda41d16606507a238d9488fa5323405364a9556b670684a2e4838aead2d
2019-06-19 15:52:19 -04:00
Suhas Daftuar
fab3658356
[qa] Test that getdata requests work as expected
We should eventually request a transaction from all peers that announce
it (assuming we never receive it).

We should prefer requesting from outbound peers over inbound peers.

Enforce the max tx requests in flight, and the eventual expiry of those
requests.

Test author:    Suhas Daftuar <sdaftuar@gmail.com>
Adjusted by:    MarcoFalke
2019-06-19 12:50:14 -04:00
MarcoFalke
e91f0a7af2 doc: Remove travis badge from readme 2019-06-19 11:39:27 -04:00
MeshCollider
44d8172323
Merge #13756: wallet: "avoid_reuse" wallet flag for improved privacy
5ebc6b0eb2 bitcoind: update -avoidpartialspends description to account for auto-enable for avoid_reuse wallets (Karl-Johan Alm)
ada258f8c8 doc: release notes for avoid_reuse (Karl-Johan Alm)
27669551da wallet: enable avoid_partial_spends by default if avoid_reuse is set (Karl-Johan Alm)
8f2e208f7c test: add test for avoidreuse feature (Karl-Johan Alm)
0bdfbd34cf wallet/rpc: add 'avoid_reuse' option to RPC commands (Karl-Johan Alm)
f904723e0d wallet/rpc: add setwalletflag RPC and MUTABLE_WALLET_FLAGS (Karl-Johan Alm)
8247a0da3a wallet: enable avoid_reuse feature (Karl-Johan Alm)
eec15662fa wallet: avoid reuse flags (Karl-Johan Alm)
58928098c2 wallet: make IsWalletFlagSet() const (Karl-Johan Alm)
129a5bafd9 wallet: rename g_known_wallet_flags constant to KNOWN_WALLET_FLAGS (Karl-Johan Alm)

Pull request description:

  Add a new wallet flag called `avoid_reuse` which, when enabled, will keep track of when a specific destination has been spent from, and will actively "blacklist" any new UTXOs which send to an already-spent-from destination.

  This improves privacy, as a payer could otherwise begin tracking a payee's wallet by regularly peppering a known UTXO with dust outputs, which would then be scooped up and used in payments by the payee, allowing the payer to map out (1) the inputs owned by the payee and (2) the destinations to which the payee is making payments.

  This replaces #10386 and together with the (now merged) #12257 it addresses #10065 in full. The concerns raised in https://github.com/bitcoin/bitcoin/pull/10386#issuecomment-302361381 are also addressed due to #12257.

  ~~Note: this builds on top of #15780.~~ (merged)

ACKs for commit 5ebc6b:
  jnewbery:
    ACK 5ebc6b0eb
  laanwj:
    Concept and code-review ACK 5ebc6b0eb2
  meshcollider:
    Code review ACK 5ebc6b0eb2
  achow101:
    ACK 5ebc6b0eb2 modulo above nits

Tree-SHA512: fdef45826af544cbbb45634ac367852cc467ec87081d86d08b53ca849e588617e9a0a255b7e7bb28692d15332de58d6c3d274ac003355220e4213d7d9070742e
2019-06-19 11:33:03 +12:00
MarcoFalke
fa410f67aa
test: Suppress false positive leak in secure_allocator<RNGState> 2019-06-18 16:37:28 -04:00
MarcoFalke
fa35c4239f
test: Log output even if fuzzer failed 2019-06-18 16:30:36 -04:00
MarcoFalke
fa9b419160
test: Add test that mainnet requires standard txs 2019-06-18 14:48:17 -04:00
MarcoFalke
fa7dd88b71
test: Add test for unknown args 2019-06-18 14:32:21 -04:00
MarcoFalke
0853d8d2fd
Merge #16112: util: Log early messages
faa2a47cd7 logging: Add threadsafety comments (MarcoFalke)
0b282f9b00 Log early messages with -printtoconsole (Anthony Towns)
412987430c Replace OpenDebugLog() with StartLogging() (Anthony Towns)

Pull request description:

  Early log messages are dropped on the floor and they'd never make it to the console or debug log. This can be tested by running the test included in this pull request without re-compiling the `bitcoind`.

  Fix that by buffering early messages and flushing them as soon as all logging options have been initialized and logging has been started.

  This pull request is identical to  "Log early messages with -printtoconsole" (#13088)  by **ajtowns**, with the following changes:
  * Rebased
  * Added docstrings for `m_buffering` and `StartLogging`
  * Switch `CCriticalSection` (aka `RecursiveMutex`) to just `Mutex` in the last commit
  * Added tests

  Fixes #16098
  Fixes #13157
  Closes #13088

ACKs for commit faa2a4:
  ajtowns:
    utACK faa2a47cd7
  hebasto:
    ACK faa2a47cd7
  kristapsk:
    ACK faa2a47cd7 (ran added functional test before / after recompiling, didn't do additional testing)

Tree-SHA512: 685e2882642fe2a43ce171d42862582dadb840d03cda8236a994322c389ca2a1f3f431b179b2726c155c61793543bb340c568a5455d97f8b83bc7d307a85d387
2019-06-18 12:32:32 -04:00
MarcoFalke
e2182b02b5
Merge #16171: Remove -mempoolreplacement to prevent needless block prop slowness.
8053e5cdad Remove -mempoolreplacement to prevent needless block prop slowness. (Matt Corallo)

Pull request description:

  At this point there is no reasonable excuse to disable opt-in RBF,
  and, unlike when this option was added, there are now significant
  issues created when disabling it (in the form of compact block
  reconstruction failures). Further, it breaks a lot of modern wallet
  behavior.

  This removes an option that is:

  * (a) only useful when a large portion of (other) miners enforce it as well
  * (b) is detrimental to everyone (income for miners, RBF notifications for others) who uses it individually otherwise
  * (c) is effectively unused
  * (d) is often confused with disabling RBF (rather than just remaining stubbornly unaware of it while the rest of the network lets it through)

ACKs for commit 8053e5:
  practicalswift:
    utACK 8053e5cdad
  promag:
    Deprecation would save from unlikely rantings, still ACK 8053e5c.
  jtimon:
    utACK 8053e5cdad
  ajtowns:
    ACK 8053e5cdad -- quick code review, checked tests work
  MarcoFalke:
    ACK 8053e5cdad

Tree-SHA512: 01aee8905b2487fc38a3a86649d422d2d2345bc60f878889ebda4b8680783e1f1a97c2000c27ef086719501be2abc2911b2039a259a5e5c04f3b24ff02b0427e
2019-06-18 10:04:14 -04:00
MeshCollider
22b6c4ed75
Merge #15899: rpc: Document iswitness flag and fix bug in converttopsbt
fa499b5f02 rpc: bugfix: Properly use iswitness in converttopsbt (MarcoFalke)
fa5c5cd141 rpc: Switch touched RPCs to IsValidNumArgs (MarcoFalke)

Pull request description:

  When a serialized transaction has inputs, there is no risk in only trying to deserialize it with witness allowed. (This is how all transactions from p2p are deserialized.) In fact, it would avoid a common issue where a transaction with inputs can be deserialized in two ways:
  * Fixes #12989
  * Fixes #15872
  * Fixes #15701
  * Fixes #13738
  * ...

  When a serialized transaction has no inputs, there is no risk in only trying to deserialze it with witness disallowed. (A transaction without inputs can't have corresponding witness data)

ACKs for commit fa499b:
  meshcollider:
    utACK fa499b5f02
  ryanofsky:
    utACK fa499b5f02. Changes since last review: consolidating commits and making iswitness documentation the same across methods.
  PastaPastaPasta:
    utACK fa499b5f02

Tree-SHA512: a64423a3131f3f0222a40da557c8b590c9ff01b45bcd40796f77a1a64ae74c6680a6be9d01ece95c492dfbcc7e2810409d2c2b336c2894af00bb213972fc85c6
2019-06-19 00:52:39 +12:00
MarcoFalke
d9bafca20c
Merge #15982: tests: Make msg_block a witness block
fa1d766717 tests: Make msg_block a witness block (MarcoFalke)
fa52eb55c9 test: Remove True argument to CBlock::serialize (MarcoFalke)

Pull request description:

  Unnamed arguments are confusing as to what they mean without looking up the function signature.

  Since segwit is active by default in regtest, and all blocks are serialized with witness (#15664), remove the argument `with_witness=True` from all calls to `CBlock::serialize` and `BlockTransactions::serialize`.

ACKs for commit fa1d76:
  laanwj:
    code-review ACK fa1d766717

Tree-SHA512: 2c550646f99c9ca86a223ca988c61a730f5e6646807adeaa7174fb2424a32cea3fef8bcd3e0b12e162e7ff192877d0c02fd0654df6ee1a9b821b065707c2dcbc
2019-06-17 11:18:10 -04:00
MarcoFalke
1a274bce4b
Merge #16205: Refactor: Replace fprintf with tfm::format
fa8f195195 Replace remaining fprintf with tfm::format manually (MarcoFalke)
fac03ec43a scripted-diff: Replace fprintf with tfm::format (MarcoFalke)
fa72a64b90 tinyformat: Add doc to Bitcoin Core specific strprintf (MarcoFalke)

Pull request description:

  This should be a refactor except in the cases where we use the wrong format specifier [1], in which case this patch is a bug fix.

  [1] : e.g.  depends: Add libevent compatibility patch for windows #8730

ACKs for commit fa8f19:
  promag:
    ACK fa8f195195. Ideally this should be rebased before merge.
  practicalswift:
    utACK fa8f195195
  Empact:
    ACK fa8f195195
  laanwj:
    code review and lightly tested ACK fa8f195195
  jonatack:
    ACK fa8f195195 from light code review, building, and running linter/unit tests/extended functional tests.

Tree-SHA512: 65f648b0bc383e3266a5bdb4ad8c8a1908a719635d49e1cd321b91254be24dbc7e22290370178e29b98ddcb3fec0889de9cbae273c7140abc9793d849534a743
2019-06-17 06:06:41 -04:00
fanquake
47d981e827
Merge #16186: doc/lint: Fix spelling errors identified by codespell 1.15.0
b748bf6f50 Fix spelling errors identified by codespell 1.15.0 (Ben Woosley)

Pull request description:

  Note all changes are to comments / documentation.

  After this commit, the only remaining output is:

  ```
    $ test/lint/lint-spelling.sh
    src/test/base32_tests.cpp:14: fo  ==> of, for
    src/test/base64_tests.cpp:14: fo  ==> of, for
    ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
  ```

  Note:
  * I ignore several valid alternative spellings ~, but changed homogenous
  to homogeneous as the latter is a more specific term according to the
  Google dictionary definitions I found~
  * homogenous is present in tinyformat, hence should be addressed upstream
  * process' is correct only if there are plural processes

ACKs for commit b748bf:
  practicalswift:
    ACK b748bf6f50
  fanquake:
    ACK b748bf6f50

Tree-SHA512: 9add7044643ce015e0a44d8b27a3f300d72c485ffff550fb6491a17f14528085289ec5caddfe02f291ea9b2cded38a0dd3079652a054e2d7fe2ff4f7b53db5d7
2019-06-16 09:57:09 +08:00
MarcoFalke
fa8f195195
Replace remaining fprintf with tfm::format manually 2019-06-13 11:46:38 -04:00
Ben Woosley
b748bf6f50
Fix spelling errors identified by codespell 1.15.0
After this commit, the only remaining output is:

  $ test/lint/lint-spelling.sh
  src/test/base32_tests.cpp:14: fo  ==> of, for
  src/test/base64_tests.cpp:14: fo  ==> of, for
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt

Note:
* I ignore several valid alternative spellings
* homogenous is present in tinyformat, hence should be addressed upstream
* process' is correct only if there are plural processes
2019-06-11 17:18:16 +02:00
João Barbosa
f402012ccf
fixup: Fix prunning test 2019-06-11 10:21:52 +02:00
Matt Corallo
8053e5cdad Remove -mempoolreplacement to prevent needless block prop slowness.
At this point there is no reasonable excuse to disable opt-in RBF,
and, unlike when this option was added, there are now significant
issues created when disabling it (in the form of compact block
reconstruction failures). Further, it breaks a lot of modern wallet
behavior.
2019-06-08 09:32:33 -04:00
Wladimir J. van der Laan
5d2ccf0ce9
Merge #15024: Allow specific private keys to be derived from descriptor
53b7de629d Add test for dumping the private key imported from descriptor (MeshCollider)
2857bc4a64 Extend importmulti descriptor tests (MeshCollider)
81a884bbd0 Import private keys from descriptor with importmulti if provided (MeshCollider)
a4d1bd1a29 Add private key derivation functions to descriptors (MeshCollider)

Pull request description:

  ~This is based on #14491, review the last 3 commits only.~

  Currently, descriptors have an Expand() function which returns public keys and scripts for a specific index of a ranged descriptor. But the private key for a specific index is not given. This allows private keys for specific indices to be derived. This also allows those keys to be imported through the `importmulti` RPC rather than having to provide them separately.

ACKs for commit 53b7de:
  achow101:
    ACK 53b7de629d

Tree-SHA512: c060bc01358a1adc76d3d470fefc2bdd39c837027f452e9bc4bd2e726097e1ece4af9d5627efd942a5f8819271e15ba54f010b169b50a9435a1f0f40fd1cebf3
2019-06-07 15:46:36 +02:00
MeshCollider
53b7de629d Add test for dumping the private key imported from descriptor 2019-06-08 01:34:58 +12:00
MeshCollider
2857bc4a64 Extend importmulti descriptor tests 2019-06-07 21:14:55 +12:00
MarcoFalke
d0f81a96d9
Merge #16129: refactor: Remove unused includes
67f4e9c522 Include core_io.h from core_read.cpp (practicalswift)
eca9767673 Make reasoning about dependencies easier by not including unused dependencies (practicalswift)

Pull request description:

  Make reasoning about dependencies easier by not including unused dependencies.

  Please note that the removed headers are _not_ "transitively included" by other still included headers. Thus the removals are real.

  As an added bonus this change means less work for the preprocessor/compiler. At least 51 393 lines of code no longer needs to be processed:

  ```
  $ git diff -u HEAD~1 | grep -E '^\-#include ' | cut -f2 -d"<" | cut -f1 -d">" | \
        sed 's%^%src/%g' | xargs cat | wc -l
  51393
  ```

  Note that 51 393 is the lower bound: the real number is likely much higher when taking into account transitively included headers :-)

ACKs for commit 67f4e9:

Tree-SHA512: 0c8868aac59813f099ce53d5307eed7962dd6f2ff3546768ef9e5c4508b87f8210f1a22c7e826c3c06bebbf28bdbfcf1628ed354c2d0fdb9a31a42cefb8fdf13
2019-06-06 16:41:40 +02:00
Matt Corallo
5efcb77283 Disable bloom filtering by default.
BIP 37 bloom filters have been well-known to be a significant DoS
target for some time. However, in order to provide continuity for
SPV clients relying on it, the NODE_BLOOM service flag was added,
and left as a default, to ensure sufficient nodes exist with such a
flag.

NODE_BLOOM is, at this point, well-established and, as long as
there exist 0.18 nodes with default config (which I'd anticipate
will be true for many years), will be available from some peers. By
that time, the continued slowdown of BIP 37-based filtering will
likely have rendered it useless (though this is already largely the
case). Further, BIP 37 was deliberately never updated to support
witness-based filtering as newer wallets are expected to migrate to
some yet-to-be-network-exposed filters.
2019-06-06 10:01:13 -04:00
MeshCollider
81a884bbd0 Import private keys from descriptor with importmulti if provided 2019-06-06 22:03:55 +12:00
Suhas Daftuar
a47df13471 [qa] Test disconnect block failure -> shutdown 2019-06-05 05:05:49 -04:00
whythat
7860c98bd5 wallet: do not encrypt wallets with disabled private keys 2019-06-04 16:39:34 +03:00
lucash-dev
0c62e3aa73 New regression testing for CVE-2018-17144, CVE-2012-2459, and CVE-2010-5137.
CVE-2018-17144 and CVE-2012-2459 are only partially tested for regression.
- CVE-2018-17144 is not tested for the inflation bug.
- CVE-2012-2459 is only tested for the mutated block being rejected, not
for the original block being accepted afterwards.

This commit fixes that limitation.

Also added functional test for CVE-2010-5137.
2019-06-02 10:25:03 -07:00
lucash-dev
38bfca6bb2 Added comments referencing multiple CVEs in tests and production code.
This commit adds comments referencing multiple CVEs both in production and test code.
CVEs covered in this commit:

CVE-2010-5137
CVE-2010-5139
CVE-2010-5141
CVE-2012-1909
CVE-2012-2459
CVE-2012-3789
CVE-2018-17144
2019-06-02 10:25:03 -07:00
practicalswift
eca9767673 Make reasoning about dependencies easier by not including unused dependencies 2019-06-02 17:15:23 +02:00
practicalswift
3c5254a820 Limit Python linting to files in the repo 2019-05-30 22:36:54 +02:00
Karl-Johan Alm
8f2e208f7c
test: add test for avoidreuse feature 2019-05-29 18:40:31 +09:00
Anthony Towns
0b282f9b00 Log early messages with -printtoconsole
This ensures log messages prior to StartLogging() are replayed to
the console as well as to the debug log file.
2019-05-28 14:26:42 -04:00
MarcoFalke
63b9efa73d
Merge #16042: test: Bump MAX_NODES to 12
fa47330397 test: Speed up cache creation (MarcoFalke)
fa6ad7a5ec test: Bump MAX_NODES to 12 (MarcoFalke)

Pull request description:

  When testing a combination of settings that affect the datadir (e.g. prune, blockfilter, ...) we may need a lot of datadirs.
  Bump the maximum number of nodes proactively from 8 to 12, so that caches get populated with 12 node dirs, as opposed to 8.

  Also, add an assert that the list of deterministic keys is exactly the number of max nodes (and not more than that.

  Also, create the cache faster.

ACKs for commit fa4733:
  laanwj:
    utACK fa47330397

Tree-SHA512: 9803c765ed52d344102f5a3bce57b05d88a7429dcb05ed66ed6c881fda8d87c2834d02d21b95fe9f39c0efe3b8527e13cf94f006588cde22e8c2cd50b2d517a6
2019-05-24 07:01:03 -04:00
MarcoFalke
854ffcae80
Merge #16078: test: replace tx hash with txid in rawtransaction test
0784af16ef remove parameters -addresstype=legacy in rpc_rawtransaction test (LongShao007)
a65dafa8f1 replace tx hash with txid in test rawtransaction (LongShao007)

Pull request description:

  The transaction hash is different from txid for witness transactions, so we should use txid instead of hash.

ACKs for commit 0784af:

Tree-SHA512: 98b699eb5f25c3a603b11eb7072efe9bc69c0c0ecc7f996405de31bc45d92105970e09fd8e4f75b42a46498817f596d36d9b28eae7d24e63a4f2f2abfcee0eab
2019-05-24 06:46:40 -04:00
LongShao007
0784af16ef
remove parameters -addresstype=legacy in rpc_rawtransaction test 2019-05-24 16:44:37 +08:00
Steven Roose
bb41e632ca
wallet_balance.py: Prevent edge cases 2019-05-23 16:18:23 +01:00
LongShao007
a65dafa8f1
replace tx hash with txid in test rawtransaction 2019-05-23 20:18:53 +08:00
Wladimir J. van der Laan
bb291b50f2
Merge #16021: p2p: Avoid logging transaction decode errors to stderr
fa2b52af32 Disallow extended encoding for non-witness transactions (take 3) (MarcoFalke)

Pull request description:

  (previous title "p2p: Disallow extended encoding for non-witness transactions (take 3)")

  Remote peers can send us illegally encoded txs and thus have us write to stderr. Fix that by not writing to stderr.

  This is a follow up to the previous (incomplete) attempts at this:

  *  Disallow extended encoding for non-witness transactions #14039
  *  Add test for superfluous witness record in deserialization #15893

ACKs for commit fa2b52:
  laanwj:
    utACK fa2b52af32
  ryanofsky:
    utACK fa2b52af32. Would change title to something like "Avoid logging transaction decode errors to stderr" instead of "Disallow extended encoding for non-witness transactions." The current title is confusing because this PR isn't really allowing or disallowing anything, just logging the condition differently. "Disallow" also seems to contradict the "Allow exceptions from..." comments in the actual code.

Tree-SHA512: c66990e69b432d00dc1c5510bf976a1188664d0890a32d1e5c6459094e7e27da82a5d227627afcbc203676f5540eec74b7d9b1d71d2c62d3b2069e1781824b4d
2019-05-20 17:29:21 +02:00
MarcoFalke
fa47330397
test: Speed up cache creation 2019-05-17 13:21:48 -04:00
MarcoFalke
fa6ad7a5ec
test: Bump MAX_NODES to 12 2019-05-17 12:19:27 -04:00
darosior
8a6810d0d2
Add a 'logpath' field to getrpcinfo 2019-05-16 23:01:00 +02:00
MarcoFalke
fa499b5f02
rpc: bugfix: Properly use iswitness in converttopsbt
Also explain the param in all RPCs
2019-05-16 15:56:04 -04:00
Wladimir J. van der Laan
df7addc4c6
Merge #15990: Add tests and documentation for blocksonly
fa8ced32a6 doc: Mention blocksonly in reduce-traffic.md, unhide option (MarcoFalke)
fa320de79f test: Add test for p2p_blocksonly (MarcoFalke)
fa3872e7b4 test: Format predicate source as multiline on error (MarcoFalke)
fa1dce7329 net: Rename ::fRelayTxes to ::g_relay_txes (MarcoFalke)

Pull request description:

  This is de-facto no longer hidden

ACKs for commit fa8ced:
  jamesob:
    utACK fa8ced32a6

Tree-SHA512: 474fbdee6cbd035ed9068a066b6056c1f909ec7520be0417820fcd1672ab3069b53f55c5147968978d9258fd3a3933fe1a9ef8e4f6e14fb6ebbd79701a0a1245
2019-05-16 19:05:24 +02:00
Wladimir J. van der Laan
1c719f78d3
Merge #15006: Add option to create an encrypted wallet
662d1171d9 Add option to create an encrypted wallet (Andrew Chow)

Pull request description:

  This PR adds a new `passphrase` argument to `createwallet` which will create a wallet that is encrypted with that passphrase.

  This is built on #15226 because it needs to first create an empty wallet, then encrypt the empty wallet and generate new keys that have only been stored in an encrypted state.

ACKs for commit 662d11:
  laanwj:
    utACK 662d1171d9
  jnewbery:
    Looks great. utACK 662d1171d9

Tree-SHA512: a53fc9a0f341eaec1614eb69abcf2d48eb4394bc89041ab69bfc05a63436ed37c65ad586c07fd37dc258ac7c7d5e4f7f93b4191407f5824bbf063b4c50894c4a
2019-05-16 18:20:14 +02:00
MarcoFalke
d5931f3676
Merge #15870: wallet: Only fail rescan when blocks have actually been pruned
fa7e311e16 [doc] rpcwallet: Only fail rescan when blocks have been pruned (MarcoFalke)
aaaa57c2aa scripted-diff: Bump copyright headers in wallet (MarcoFalke)
faf3729242 wallet: Only fail rescan when blocks have actually been pruned (MarcoFalke)

Pull request description:

  This brings the behaviour of the import* calls closer to importmulti. After this change, the difference between importmulti and the other import* calls is

  * that in importmulti you can "opt-out" of scanning early blocks by setting a later timestamp.
  * that in importmulti the wallet will successfully import the data, but fail to rescan. Whereas in the other calls, the wallet will abort before importing the data.

ACKs for commit fa7e31:
  promag:
    utACK fa7e311e16.
  jnewbery:
    utACK fa7e311e16

Tree-SHA512: a57d52ffea94b64e0eb9b5d3a7a63031325833908297dd14eb0c5251ffea3b2113b131003f1db4e9599e014369165a57f107a7150bb65e4c791e5fe742f33cb8
2019-05-16 11:18:27 -04:00
Julian Fleischer
f3b90f2e05
Run all lint scripts
The description reads:

```
# This script runs all contrib/devtools/lint-*.sh files, and fails if any exit
# with a non-zero status code.
```

This runs all scripts and returns with a non-zero exit code if any failed.
2019-05-16 16:42:59 +02:00
MarcoFalke
fad0ce59e9
tests: Fail if RPC has been added without tests 2019-05-15 14:27:18 -04:00
MarcoFalke
3503a69ba2
Merge #15963: [tests] Make random seed logged and settable
a407b6fdf3 [tests] Make random seed logged and settable (John Newbery)

Pull request description:

  This allows tests which use randomness to be reproducibly run on failure.

ACKs for commit a407b6:
  jonatack:
    re-ACK a407b6fdf3
  jb55:
    great! utACK a407b6fdf3

Tree-SHA512: e1e89e6e76d11ddec71a8f0f077227e4b46303f80461b170900d3f95d4dcc4187b0d1decfd63562ea970aaaf530ef032a3e64ed1669aac29033d95161855fda3
2019-05-14 09:00:58 -04:00
MarcoFalke
fa2b52af32
Disallow extended encoding for non-witness transactions (take 3) 2019-05-14 08:33:55 -04:00
Andrew Chow
662d1171d9 Add option to create an encrypted wallet 2019-05-13 22:49:34 -04:00
MarcoFalke
fa320de79f
test: Add test for p2p_blocksonly 2019-05-13 10:44:42 -04:00
Pieter Wuille
26fe9b9909 Add support for descriptors to utxoupdatepsbt
This adds a descriptors argument to the utxoupdatepsbt RPC. This means:
* Input and output scripts and keys will be filled in when known
* P2SH-witness outputs will be filled in from the UTXO set when a descriptor
  is provided to show they're segwit outputs.
2019-05-10 14:36:58 -07:00
MarcoFalke
e2371f842f
Merge #14802: rpc: faster getblockstats using BlockUndo data
d20d756752 rpc: faster getblockstats using BlockUndo data (Felix Weis)

Pull request description:

  Using undo data for a block (rev?????.dat) we can retrieve value information about prevouts and calculate the final transaction fee (rate). This approach is about 80x faster, drops the requirement for `-txindex`, and works for all non-pruned blocks.

  ```
  # 2018-11-25T16:36:19Z Bitcoin Core version v0.17.99.0-edc715240-dirty (release build)
  seq 550100 550200  0.00s user 0.00s system 62% cpu 0.004 total
  xargs -n1 src/bitcoin-cli getblockstats  0.21s user 0.19s system 17% cpu 2.302 total

  # 2018-11-25T16:39:17Z Bitcoin Core version v0.17.0 (release build)
  seq 550100 550200  0.00s user 0.00s system 87% cpu 0.002 total
  xargs -n1 src/bitcoin-cli getblockstats  0.24s user 0.22s system 0% cpu 3:19.42 total
  ```

ACKs for commit d20d75:
  MarcoFalke:
    re-utACK d20d756752

Tree-SHA512: 5babc3eb8d2fee2cb23dc12f522656b80737a540cbf2b13390a8f388304c46c064cca76f896b46a6e2abae8cc582d28e1ab20dd4bb17ad6142f20630c2d30c54
2019-05-10 13:20:48 -04:00
Felix Weis
d20d756752 rpc: faster getblockstats using BlockUndo data
Using undo data for a block (rev?????.dat) we can retrieve value information about prevouts and calculate the final transaction fee (rate). This approach is about 80x faster, drops the requirement for -txindex, and works for all non-pruned blocks.
2019-05-10 08:33:14 -04:00
MarcoFalke
14959753a4
Merge #15744: refactor: Extract ParseDescriptorRange
510c6532ba Extract ParseDescriptorRange (Ben Woosley)

Pull request description:

  So as to be consistently informative when the checks fail, and
  to protect against unintentional divergence among the checks.

ACKs for commit 510c65:
  meshcollider:
    Oh apologies, yes. Thanks :) utACK 510c6532ba
  MarcoFalke:
    utACK 510c6532ba
  sipa:
    utACK 510c6532ba

Tree-SHA512: b1f0792bfaa163890a20654a0fc2c4c4a996659916bf5f4a495662436b39326692a1a0c825caafd859e48c05f5dd1865c4f7c28092be5074edda3c94f94f9f8b
2019-05-10 08:09:44 -04:00
MarcoFalke
fa3872e7b4
test: Format predicate source as multiline on error 2019-05-10 07:29:54 -04:00
John Newbery
a407b6fdf3 [tests] Make random seed logged and settable
This allows tests which use randomness to be reproducibly run on failure.
2019-05-09 11:28:04 -04:00
MarcoFalke
fa1d766717
tests: Make msg_block a witness block
This diff has been generated with the following script, but is better
reviewed without looking at the script.

 # -BEGIN VERIFY SCRIPT-
echo "Use msg_witness_block everywhere, except for tests that require msg_block"
 # This could be a separate commit, but it is combined with the
 # following scripts to reduce the overall diff
sed -i -e 's/msg_block/msg_witness_block/g' ./test/functional/{feature_assumevalid,feature_cltv,feature_dersig,feature_versionbits_warning,p2p_fingerprint,p2p_sendheaders,p2p_unrequested_blocks,example_test,rpc_blockchain}.py

echo "Rename msg_block to msg_no_witness_block"
 # Rename msg_block to msg_no_witness_block in all tests (not the
 # framework)
sed -i -e 's/msg_block/msg_no_witness_block/g' $(git grep -l msg_block ./test/functional/*.py)
 # Derive msg_no_witness_block from msg_block
 # Make msg_block a witness block in messages.py
patch -p1 --fuzz 0 << EOF
diff --git a/test/functional/test_framework/messages.py b/test/functional/test_framework/messages.py
index 00190e4cbd..e454ed5987 100755
--- a/test/functional/test_framework/messages.py
+++ b/test/functional/test_framework/messages.py
@@ -1133 +1133 @@ class msg_block:
-        return self.block.serialize(with_witness=False)
+        return self.block.serialize()
@@ -1155 +1155 @@ class msg_generic:
-class msg_witness_block(msg_block):
+class msg_no_witness_block(msg_block):
@@ -1158,2 +1158 @@ class msg_witness_block(msg_block):
-        r = self.block.serialize()
-        return r
+        return self.block.serialize(with_witness=False)
@@ -1445 +1444 @@ class msg_blocktxn:
-        r += self.block_transactions.serialize(with_witness=False)
+        r += self.block_transactions.serialize()
@@ -1452 +1451 @@ class msg_blocktxn:
-class msg_witness_blocktxn(msg_blocktxn):
+class msg_no_witness_blocktxn(msg_blocktxn):
@@ -1456,3 +1455 @@ class msg_witness_blocktxn(msg_blocktxn):
-        r = b""
-        r += self.block_transactions.serialize()
-        return r
+        return self.block_transactions.serialize(with_witness=False)
EOF
 # Conclude rename of msg_block to msg_no_witness_block
sed -i -e 's/msg_witness_block/msg_block/g' $(git grep -l msg_witness_block)
 # -END VERIFY SCRIPT-
2019-05-08 11:53:02 -04:00
MarcoFalke
fa52eb55c9
test: Remove True argument to CBlock::serialize
Unnamed arguments are confusing as to what they mean without looking up
the function signature.

Since segwit is active by default in regtest, and all blocks are
serialized with witness (#15664, c459c5f), remove the argument
`with_witness=True` from all calls to `CBlock::serialize` and
`BlockTransactions::serialize`.

This diff has been created with a script, but is better reviewed without
a scripted diff.

sed -i --regexp-extended -e 's/block(_?[2a-z]*)\.serialize\([a-z_]*=?True/block\1.serialize(/g' $(git grep -l serialize ./test)
2019-05-08 10:06:21 -04:00
MarcoFalke
c459c5f701
Merge #15664: change default Python block serialization to witness
124ea38e39 change default Python block serialization to witness (Gregory Sanders)

Pull request description:

ACKs for commit 124ea3:
  stevenroose:
    ACK 124ea38e39

Tree-SHA512: 52877934f8a3c761cb89a618daffe73e86b008d9d32d48721392b7626aaa10d3b9aa26e4c59337729e0a2d01fc48648eef5ec3d72de531a685a2cf4f4d7579ab
2019-05-08 09:42:14 -04:00
MarcoFalke
faf3729242
wallet: Only fail rescan when blocks have actually been pruned 2019-05-06 14:03:56 -04:00
MarcoFalke
8ec7121a45
Merge #15927: [tests] log thread names by default in functional tests
7b29ec277b [tests] Comment for why logging config is set as command-line args. (John Newbery)
ba534ccd56 [tests] log thread names by default in functional tests (John Newbery)

Pull request description:

  More detailed logs are better

ACKs for commit 7b29ec:
  jamesob:
    utACK 7b29ec277b

Tree-SHA512: 327cfedb7b7bf32f7ce1e2de5f70c7092041a8e868e14285a79176277c6cf47ebea27027f68787332f8ad21c7f64d2640dd21813eda5b2bd0e5208a65364a879
2019-05-06 12:59:19 -04:00
Wladimir J. van der Laan
a3d2d6b067
Merge #15930: rpc: Add balances RPC
facfb4111d rpc: Deprecate getunconfirmedbalance and getwalletinfo balances (MarcoFalke)
999931cf8f rpc: Add getbalances RPC (MarcoFalke)
fad13e925e rpcwallet: Make helper methods const on CWallet (MarcoFalke)
fad40ec915 wallet: Use IsValidNumArgs in getwalletinfo rpc (MarcoFalke)

Pull request description:

  This exposes the `CWallet::GetBalance()` struct over RPC.

  In the future, incorrectly named rpcs such as `getunconfirmedbalance` or rpcs redundant to this such as `getbalance` could be removed.

ACKs for commit facfb4:
  jnewbery:
    utACK facfb4111d

Tree-SHA512: 1f54fedce55df9a8ea82d2b6265354b39a956072621876ebaee2355aac0e23c7b64340c3279502415598c095858529e18b50789be956250aafda1cd3a8d948a5
2019-05-06 11:36:42 +02:00
Wladimir J. van der Laan
d7d7d31506
Merge #15141: Rewrite DoS interface between validation and net_processing
0ff1c2a838 Separate reason for premature spends (coinbase/locktime) (Suhas Daftuar)
54470e767b Assert validation reasons are contextually correct (Suhas Daftuar)
2120c31521 [refactor] Update some comments in validation.cpp as we arent doing DoS there (Matt Corallo)
12dbdd7a41 [refactor] Drop unused state.DoS(), state.GetDoS(), state.CorruptionPossible() (Matt Corallo)
aa502b88d1 scripted-diff: Remove DoS calls to CValidationState (Matt Corallo)
7721ad64f4 [refactor] Prep for scripted-diff by removing some \ns which annoy sed. (Matt Corallo)
5e78c5734b Allow use of state.Invalid() for all reasons (Matt Corallo)
6b34bc6b6f Fix handling of invalid headers (Suhas Daftuar)
ef54b486d5 [refactor] Use Reasons directly instead of DoS codes (Matt Corallo)
9ab2a0412e CorruptionPossible -> BLOCK_MUTATED (Matt Corallo)
6e55b292b0 CorruptionPossible -> TX_WITNESS_MUTATED (Matt Corallo)
7df16e70e6 LookupBlockIndex -> CACHED_INVALID (Matt Corallo)
c8b0d22698 [refactor] Drop redundant nDoS, corruptionPossible, SetCorruptionPossible (Matt Corallo)
34477ccd39 [refactor] Add useful-for-dos "reason" field to CValidationState (Matt Corallo)
6a7f8777a0 Ban all peers for all block script failures (Suhas Daftuar)
7b999103e2 Clean up banning levels (Matt Corallo)
b8b4c80146 [refactor] drop IsInvalid(nDoSOut) (Matt Corallo)
8818729013 [refactor] Refactor misbehavior ban decisions to MaybePunishNode() (Matt Corallo)
00e11e61c0 [refactor] rename stateDummy -> orphan_state (Matt Corallo)
f34fa719cf Drop obsolete sigops comment (Matt Corallo)

Pull request description:

  This is a rebase of #11639 with some fixes for the last few comments which were not yet addressed.

  The original PR text, with some strikethroughs of text that is no longer correct:

  > This cleans up an old main-carryover - it made sense that main could decide what DoS scores to assign things because the DoS scores were handled in a different part of main, but now validation is telling net_processing what DoS scores to assign to different things, which is utter nonsense. Instead, we replace CValidationState's nDoS and CorruptionPossible with a general ValidationInvalidReason, which net_processing can handle as it sees fit. I keep the behavior changes here to a minimum, but in the future we can utilize these changes for other smarter behavior, such as disconnecting/preferring to rotate outbound peers based on them providing things which are invalid due to SOFT_FORK because we shouldn't ban for such cases.
  >
  > This is somewhat complementary with, though obviously conflicts heavily with #11523, which added enums in place of DoS scores, as well as a few other cleanups (which are still relevant).
  >
  > Compared with previous bans, the following changes are made:
  >
  > Txn with empty vin/vout or null prevouts move from 10 DoS
  > points to 100.
  > Loose transactions with a dependency loop now result in a ban
  > instead of 10 DoS points.
  > ~~BIP68-violation no longer results in a ban as it is SOFT_FORK.~~
  > ~~Non-SegWit SigOp violation no longer results in a ban as it
  > considers P2SH sigops and is thus SOFT_FORK.~~
  > ~~Any script violation in a block no longer results in a ban as
  > it may be the result of a SOFT_FORK. This should likely be
  > fixed in the future by differentiating between them.~~
  > Proof of work failure moves from 50 DoS points to a ban.
  > Blocks with timestamps under MTP now result in a ban, blocks
  > too far in the future continue to not result in a ban.
  > Inclusion of non-final transactions in a block now results in a
  > ban instead of 10 DoS points.

  Note: The change to ban all peers for consensus violations is actually NOT the change I'd like to make -- I'd prefer to only ban outbound peers in those situations.  The current behavior is a bit of a mess, however, and so in the interests of advancing this PR I tried to keep the changes to a minimum.  I plan to revisit the behavior in a followup PR.

  EDIT: One reviewer suggested I add some additional context for this PR:

  > The goal of this work was to make net_processing aware of the actual reasons for validation failures, rather than just deal with opaque numbers instructing it to do something.
  >
  > In the future, I'd like to make it so that we use more context to decide how to punish a peer. One example is to differentiate inbound and outbound peer misbehaviors. Another potential example is if we'd treat RECENT_CONSENSUS_CHANGE failures differently (ie after the next consensus change is implemented), and perhaps again we'd want to treat some peers differently than others.

ACKs for commit 0ff1c2:
  jnewbery:
    utACK 0ff1c2a838
  ryanofsky:
    utACK 0ff1c2a838. Only change is dropping the first commit (f3883a321bf4ab289edcd9754b12cae3a648b175), and dropping the temporary `assert(level == GetDoS())` that was in 35ee77f2832eaffce30042e00785c310c5540cdc (now c8b0d22698)

Tree-SHA512: e915a411100876398af5463d0a885920e44d473467bb6af991ef2e8f2681db6c1209bb60f848bd154be72d460f039b5653df20a6840352c5f7ea5486d9f777a3
2019-05-04 11:58:57 +02:00
MarcoFalke
fa08c5cb99
test_runner: Move pruning back to extended 2019-05-03 16:16:39 -04:00
John Newbery
7b29ec277b [tests] Comment for why logging config is set as command-line args. 2019-05-03 12:47:27 -04:00
Matt Corallo
34477ccd39 [refactor] Add useful-for-dos "reason" field to CValidationState
This is a first step towards cleaning up our DoS interface - make
validation return *why* something is invalid, and let net_processing
figure out what that implies in terms of banning/disconnection/etc.

Behavior change: peers will now be banned for providing blocks
with premature coinbase spends.

Co-authored-by: Anthony Towns <aj@erisian.com.au>
                Suhas Daftuar <sdaftuar@gmail.com>
2019-05-02 14:55:13 -04:00
Matt Corallo
7b999103e2 Clean up banning levels
Compared with previous bans, the following changes are made:
 * Txn with empty vin/vout or null prevouts move from 10 DoS
   points to 100.
 * Loose transactions with a dependency loop now result in a ban
   instead of 10 DoS points.
 * Many pre-segwit soft-fork errors now result in a ban.
   Note: Transactions that violate soft-fork script flags since P2SH do not generally
   result in a ban. Also, banning behavior for invalid blocks is dependent on
   whether the node is validating with multiple script check threads, due to a long-
   standing bug. That inconsistency is still present after this commit.
 * Proof of work failure moves from 50 DoS points to a ban.
 * Blocks with timestamps under MTP now result in a ban, blocks
   too far in the future continue to *not* result in a ban.
 * Inclusion of non-final transactions in a block now results in a
   ban instead of 10 DoS points.

Co-authored-by: Anthony Towns <aj@erisian.com.au>
2019-05-02 11:00:19 -04:00
MarcoFalke
999931cf8f
rpc: Add getbalances RPC 2019-05-02 10:10:23 -04:00
MarcoFalke
12aa2ac988
Merge #15323: rpc: Expose g_is_mempool_loaded via getmempoolinfo
effe81f750 Move g_is_mempool_loaded into CTxMemPool::m_is_loaded (Ben Woosley)
bb8ae2c419 rpc: Expose g_is_mempool_loaded via getmempoolinfo and /rest/mempool/info.json (Ben Woosley)

Pull request description:

  And use it to fix a race condition in mempool_persist.py:
  https://travis-ci.org/Empact/bitcoin/jobs/487577243

  Since e.g. getrawmempool returns errors based on this status, this
  enables users to test it for readiness.

  Fixes #12863

ACKs for commit effe81:
  MarcoFalke:
    utACK effe81f750
  jnewbery:
    utACK effe81f750

Tree-SHA512: 74328b0c17a97efb8a000d4ee49b9a673c2b6dde7ea30c43a6a2eff961a233351c9471f9a42344412135786c02bdf2ee1b2526651bb8fed68bd94d2120c4ef86
2019-05-01 10:06:15 -04:00
MarcoFalke
86edb79e97
Merge #15841: [test] combine_logs: append node stderr and stdout if it exists
fa90a89eee [test] combine_logs: append node stderr and stdout if it exists (MarcoFalke)

Pull request description:

  See issue:

  * tests: bitcoind stdout and error should be passed to the logger #13519

ACKs for commit fa90a8:
  laanwj:
    utACK fa90a89eee

Tree-SHA512: 39c4596e2e133c9011ab01bc4dc24e884d0a8cce7a67d3765f17c288d3ffbd438e1ff6016d0f817a981b27fce17fa77a1ff56787ddb1ea55123ce9ecffb44c08
2019-05-01 08:39:51 -04:00
MarcoFalke
2eb8c5d7a2
Merge #15758: qa: Add further tests to wallet_balance
fa79a783d6 test: Add reorg test to wallet_balance (MarcoFalke)
fad03cd046 test: Check that wallet txs not in the mempool are untrusted (MarcoFalke)
fa195315e6 test: Add getunconfirmedbalance test with conflicts (MarcoFalke)
fa464e8211 test: Add wallet_balance test for watchonly (MarcoFalke)

Pull request description:

  Second commit can be reviewed with `--ignore-all-space`

ACKs for commit fa79a7:
  jnewbery:
    utACK fa79a783d6

Tree-SHA512: ec4919a3c93b6dcb35d58e7c65bdffe7f4c8cb87b9287f3679631c1823ef5bd72789f233def94e60c1ab332711601751645566f5997ce250af55b328ed60e917
2019-05-01 08:35:05 -04:00
John Newbery
ba534ccd56 [tests] log thread names by default in functional tests 2019-04-30 15:06:26 -05:00
MarcoFalke
fa79a783d6
test: Add reorg test to wallet_balance 2019-04-30 15:14:41 -04:00
MarcoFalke
fad03cd046
test: Check that wallet txs not in the mempool are untrusted 2019-04-30 15:11:52 -04:00
MarcoFalke
fa195315e6
test: Add getunconfirmedbalance test with conflicts 2019-04-30 15:11:51 -04:00
MarcoFalke
fa464e8211
test: Add wallet_balance test for watchonly 2019-04-30 15:11:50 -04:00
MarcoFalke
45d8b71778
Merge #15696: [qa] test_runner: Move feature_pruning to base tests
fafb55e2c2 [qa] test_runner: Move feature_pruning to base tests (MarcoFalke)
8728a66782 [tests] fix block time in feature_pruning.py (John Newbery)

Pull request description:

ACKs for commit fafb55:

Tree-SHA512: 88abef94379fbad6629da11dccb080d5f0644490d6f2cc2756a33fac34bcf72e84245cef596dfae5a40f7a99b3f4da0dd85d306d4c1b452d310d3f36eef75a8b
2019-04-30 10:09:54 -04:00
MarcoFalke
fafb55e2c2
[qa] test_runner: Move feature_pruning to base tests 2019-04-30 08:55:28 -04:00
John Newbery
8728a66782 [tests] fix block time in feature_pruning.py 2019-04-29 18:48:22 -04:00
MarcoFalke
ce6762030f
Merge #15897: QA/mininode: Send all headers upfront in send_blocks_and_test to avoid sending an unconnected one
9f9db39041 QA/mininode: Send all headers upfront in send_blocks_and_test to avoid sending an unconnected one (Luke Dashjr)

Pull request description:

  While this doesn't currently trigger any problems, the network protocol does expect headers to be sent connectable in normal circumstances, and if too many are sent out of order will disconnect the peer.

ACKs for commit 9f9db3:

Tree-SHA512: 25b88718e4ba3d31aed2de7ece23fab9a0737fd6536c5e618ea8eb5a3a217dab0dffaebc4892df7993bcea7efb7c4fb5085fabebe99535b8f7fdde3c19df54ff
2019-04-29 15:03:51 -04:00
MarcoFalke
fac174e2d1
lint: Check that all wallet args are hidden 2019-04-28 12:43:50 -04:00
MeshCollider
b025aa3b9e
Merge #15846: [POLICY] Make sending to future native witness outputs standard
c634b1e20 [POLICY] Make sending to future native witness outputs standard (Pieter Wuille)

Pull request description:

  As discussed in the April 18 2019 IRC meeting.

  This makes sending to future Segwit versions via native outputs (bech32) standard for relay, mempool acceptance, and mining. The reasons are:
  * This may interfere with smooth adoption of future segwit versions, if they're defined (by the sender wallet/node).
  * It violates BIP173 ("Version 0 witness addresses are always 42 or 62 characters, but implementations MUST allow the use of any version."), though admittedly this code was written before BIP173.
  * It doesn't protect much, as P2SH-embedded segwit cannot be filtered in this way.
  * As a general policy, the sender shouldn't care what the receiver likes his outputs to be.

  Note that _spending_ such outputs (including P2SH-embedded ones) remains nonstandard, as that is actually required for softfork safety.

ACKs for commit c634b1:
  MarcoFalke:
    utACK c634b1e207
  harding:
    Tested ACK c634b1e207
  meshcollider:
    utACK c634b1e207

Tree-SHA512: e37168a1be9f445a04d4280593f0a92bdae33eee00ecd803d5eb16acb5c9cfc0f1f0a1dfbd5a0cc73da2c9928ec11cbdac7911513a78f85b789ae0d00e1b5962
2019-04-27 21:50:45 +12:00
MarcoFalke
5046d4e911
Merge #15896: QA: feature_filelock, interface_bitcoin_cli: Use PACKAGE_NAME in messages rather than hardcoding Bitcoin Core
fcc443b636 QA: feature_filelock, interface_bitcoin_cli: Use PACKAGE_NAME in messages rather than hardcoding Bitcoin Core (Luke Dashjr)

Pull request description:

ACKs for commit fcc443:
  practicalswift:
    utACK fcc443b636

Tree-SHA512: f87cfea3cb2ac716a5c9a507141dcba18cb0e3cbe17a4114ed11fa283c3d38551cc245ef68f8816c51538d492991e71019d20a9ca4acd22af4f99e631c04d33e
2019-04-26 08:16:38 -04:00
MarcoFalke
f73a3c618b
Merge #15895: QA: Avoid re-reading config.ini unnecessarily
a014373d81 QA: Avoid re-reading config.ini unnecessarily (Luke Dashjr)

Pull request description:

  BitcoinTestFramework.main already loads and stores config.ini on the object itself; just access that instead of re-reading the file to check for features

ACKs for commit a01437:
  practicalswift:
    utACK a014373d81
  hebasto:
    utACK a014373d81

Tree-SHA512: 36e3dda36cf4fae243feb1655da2891d1b78c86319be9bd9809c20054fa0cb75749370b05aa9d589a4dcab6322d8cdf4e874c5175144ed23ba63b2ed338538ca
2019-04-26 07:49:43 -04:00
Gregory Sanders
cc556e4a30 Add test for superfluous witness record in deserialization 2019-04-25 21:01:09 -04:00
Gregory Sanders
25b0786581 Fix missing input template by making minimal tx 2019-04-25 21:01:09 -04:00
Luke Dashjr
9f9db39041 QA/mininode: Send all headers upfront in send_blocks_and_test to avoid sending an unconnected one 2019-04-25 20:47:07 +00:00
Luke Dashjr
fcc443b636 QA: feature_filelock, interface_bitcoin_cli: Use PACKAGE_NAME in messages rather than hardcoding Bitcoin Core 2019-04-25 20:43:04 +00:00
Luke Dashjr
a014373d81 QA: Avoid re-reading config.ini unnecessarily
BitcoinTestFramework.main already loads and stores config.ini on the object itself; just access that instead of re-reading the file to check for features
2019-04-25 20:41:17 +00:00
MarcoFalke
8cca1fbea9
Merge #14818: Bugfix: test/functional/rpc_psbt: Remove check for specific error message that depends on uncertain assumptions
c87fc71f7e Bugfix: test/functional/rpc_psbt: Correct test description comment (Luke Dashjr)
097c4aa379 Bugfix: test/functional/rpc_psbt: Remove check for specific error message that depends on uncertain assumptions (Luke Dashjr)

Pull request description:

  When converttopsbt is called with a signed transaction, it either fails with "TX decode failed" if one or more inputs were segwit, or "Inputs must not have scriptSigs and scriptWitnesses" otherwise.
  Since no effort is made by the test to ensure the inputs are segwit or not, avoid checking the exact message used.
  The error code is still checked to ensure it is of the correct kind of failure.

ACKs for commit c87fc7:
  instagibbs:
    utACK  c87fc71f7e
  achow101:
    utACK c87fc71f7e

Tree-SHA512: 61312b5d49aa50652902f30ba9693dfba9e5b7e6478f23becda20202d8b328ddb3e040f2199b617a68df133a5f1f8b5d68bc19d4621303f17c1963dca01bd9ef
2019-04-25 11:24:14 -04:00
MarcoFalke
40a720acb8
Merge #15697: qa: Make swap_magic_bytes in p2p_invalid_messages atomic
faca95effd qa: Make swap_magic_bytes in p2p_invalid_messages atomic (MarcoFalke)

Pull request description:

  Otherwise, this will lead to errors logged in the network thread:

  https://travis-ci.org/MarcoFalke/bitcoin/jobs/513076282#L2765

ACKs for commit faca95:

Tree-SHA512: 8bc8280f9c0e8d40bc68c0ff1cab1c3386cbfef0728dbab5f29c2606dd65a9ac3c695b0c286be707a9f2bd62a6f87ac2032d7749fc2cf8b9fa1eba3c4cf70933
2019-04-23 13:12:54 -04:00
Jonas Schnelli
e9e777e21b
Merge #15874: Resolve the qt/guiutil <-> qt/optionsmodel CD
fa1c8e297 Resolve the qt/guiutil <-> qt/optionsmodal CD (251)

Pull request description:

  This pull request attempts to resolve the `qt/guiutil` <-> `qt/optionsmodel` circular dependency.

  The `Intro` class in `qt/intro` has a static member function `getDefaultDataDirectory` which is used by `qt/optionsmodel` and creates the circular dependency
  `qt/guiutil -> qt/walletmodel -> qt/optionsmodel -> qt/intro -> qt/guiutil`.

  This circular dependency is resolved by moving `Intro::getDefaultDataDirectory` to `GUIUtil::getDefaultDataDirectory` without modifying the implementation.

ACKs for commit fa1c8e:
  MarcoFalke:
    utACK fa1c8e2978
  promag:
    utACK fa1c8e2.
  hebasto:
    utACK fa1c8e2978
  practicalswift:
    utACK fa1c8e2978
  jonasschnelli:
    utACK fa1c8e2978

Tree-SHA512: 58cc4aee937c943d8de9dc97ef1789decfddb0287308f44e7e3a3b497c19e51da184988e17207544fff410168ec98dd49a3e62c47e84ad1f0cf6ef7247a80fb5
2019-04-23 18:11:01 +02:00
MarcoFalke
caceff5546
Merge #15866: test: Add missing syncwithvalidationinterfacequeue to wallet_import_rescan
fa465e4da4 test: Add missing syncwithvalidationinterfacequeue to wallet_import_rescan (MarcoFalke)

Pull request description:

  Fixes #15865

ACKs for commit fa465e:
  promag:
    utACK fa465e4da4.

Tree-SHA512: efc1d82b92aefb1f0e6c54ed0a2d69c547cd9dee7ff8d57a665022fefce01bcf726d394f0665abe4fbd3451abf3cb08eed6ff45def831857f917a988e13a1055
2019-04-23 09:43:15 -04:00
251
fa1c8e2978 Resolve the qt/guiutil <-> qt/optionsmodal CD
This pull request attempts to resolve the `qt/guiutil` <-> `qt/optionsmodel`
circular dependency.

The circular dependency is resolved by moving the `Intro::getDefaultDataDirectory`
member function to `GUIUtil::getDefaultDataDirectory`.
2019-04-23 13:26:06 +02:00
MarcoFalke
fa465e4da4
test: Add missing syncwithvalidationinterfacequeue to wallet_import_rescan 2019-04-22 09:01:19 -04:00
MarcoFalke
08bd21a3bd
Merge #15826: Pure python EC
b67978529a Add comments to Python ECDSA implementation (John Newbery)
8c7b9324ca Pure python EC (Pieter Wuille)

Pull request description:

  This removes the dependency on OpenSSL for the interaction tests, by providing a pure-Python
  toy implementation of secp256k1.

ACKs for commit b67978:
  jnewbery:
    utACK b67978529a

Tree-SHA512: 181445eb08b316c46937b80dc10aa50d103ab1fdddaf834896c0ea22204889f7b13fd33cbcbd00ddba15f7e4686fe0d9f8e8bb4c0ad0e9587490c90be83966dc
2019-04-22 08:10:05 -04:00
MarcoFalke
ae2c19f578
Merge #15655: Resolve the checkpoints <-> validation circular dependency
418d3230f8 Resolve the checkpoints <-> validation CD. (251)

Pull request description:

  This pull request attempts to resolve the `checkpoints -> validation -> checkpoints` circular dependency.

  The circular dependency is resolved by moving the `CheckPoints::GetLastCheckpoint(const CCheckpointData& data)` function to `validation.cpp` where it used exclusively by the private function `ContextualCheckBlockHeader(const CBlockHeader& block, CValidationState& state, const CChainParams& params, const CBlockIndex* pindexPrev, int64_t nAdjustedTime)`.

ACKs for commit 418d32:
  promag:
    utACK 418d323, only `GetLastCheckpoint` usage is in `validation.cpp` and so makes sense to move it there.
  practicalswift:
    utACK 418d3230f8
  MarcoFalke:
    utACK 418d3230f8
  sipa:
    utACK 418d3230f8

Tree-SHA512: 03c3556bc192e65f5e3fa76fd545d4ee7d63d3fb06b132f7a1fa6131aa21ddd2e5b2d19e2222dfe524f422daaca30efde219bed188db8c74ff4b088876b5bc16
2019-04-19 09:34:01 -04:00
John Newbery
b67978529a Add comments to Python ECDSA implementation 2019-04-18 13:23:26 -07:00
Pieter Wuille
c634b1e207 [POLICY] Make sending to future native witness outputs standard 2019-04-18 12:46:07 -07:00
Pieter Wuille
8c7b9324ca Pure python EC
This removes the dependency on OpenSSL for the interaction tests, by providing a pure-Python
toy implementation of secp256k1.
2019-04-18 11:58:32 -07:00
MarcoFalke
e4beef611a
Merge #14121: Index for BIP 157 block filters
c7efb652f3 blockfilter: Update BIP 158 test vectors. (Jim Posen)
19308c9e21 rpc: Add getblockfilter RPC method. (Jim Posen)
ff35105096 init: Add CLI option to enable block filter index. (Jim Posen)
accc8b8b18 index: Access functions for global block filter indexes. (Jim Posen)
2bc90e4e7b test: Unit test for block filter index reorg handling. (Jim Posen)
6bcf0998c0 test: Unit tests for block index filter. (Jim Posen)
b5e8200db7 index: Implement lookup methods on block filter index. (Jim Posen)
75a76e3619 index: Implement block filter index with write operations. (Jim Posen)
2ad2338ef9 serialize: Serialization support for big-endian 32-bit ints. (Jim Posen)
ba6ff9a6f7 blockfilter: Functions to translate filter types to/from names. (Jim Posen)
62b7a4f094 index: Ensure block locator is not stale after chain reorg. (Jim Posen)
4368384f1d index: Allow atomic commits of index state to be extended. (Jim Posen)

Pull request description:

  This introduces a new BlockFilterIndex class, which is required for BIP 157 support.

  The index is uses the asynchronous BaseIndex infrastructure driven by the ValidationInterface callbacks. Filters are stored sequentially in flat files and the disk location of each filter is indexed in LevelDB along with the filter hash and header. The index is designed to ensure persistence of filters reorganized out of the main chain to simplify the BIP 157 net implementation.

  Stats (block height = 565500):
  - Syncing the index from scratch takes 45m
  - Total index size is 3.8 GiB

ACKs for commit c7efb6:
  MarcoFalke:
    utACK c7efb652f3
  ryanofsky:
    Slightly tested ACK c7efb652f3 (I just rebuilt the index with the updated PR and tested the RPC). Changes since last review: rebase, fixed compile errors in internal commits, new comments, updated error messages, tweaked cache size logic, renamed commit method, renamed constants and globals, fixed whitespace, extra BlockFilterIndex::Init error check.

Tree-SHA512: f8ed7a9b6f76df45933aa5eba92b27b3af83f6df2ccb3728a5c89eec80f654344dc14f055f6f63eb9b3a7649dd8af6553fe14969889e7e2fd2f8461574d18f28
2019-04-18 09:48:25 -04:00