c004ffc9b4 Make handling of invalid in IsMine more uniform (Pieter Wuille)
a53f0feff8 Add some checks for invalid recursion in IsMine (Pieter Wuille)
b5802a9f5f Simplify IsMine logic (Pieter Wuille)
4e91820531 Make IsMine stop distinguishing solvable/unsolvable (Pieter Wuille)
6d714c3419 Make coincontrol use IsSolvable to determine solvability (Pieter Wuille)
Pull request description:
Our current `IsMine` logic does several things with outputs:
* Determine "spendability" (roughly corresponding to "could we sign for this")
* Determine "watching" (is this an output directly or indirectly a watched script)
* Determine invalidity (is this output definitely not legally spendable, detecting accidental uncompressed pubkeys in witnesses)
* Determine "solvability" (would we be able to sign for this ignoring the fact that we may be missing some private keys).
The last item (solvability) is mostly unrelated and only rarely needed (there is just one instance, inside the wallet's coin control logic). This PR changes that instance to use the separate `IsSolvable` function, and stop `IsMine` from distinguishing between solvable and unsolvable.
As an extra, this also simplifies the `IsMine` logic and adds some extra checks (which wouldn't be hit unless someone adds already invalid scripts to their wallet).
Tree-SHA512: 95a6ef75fbf2eedc5ed938c48a8e5d77dcf09c933372acdd0333129fb7301994a78498f9aacce2c8db74275e19260549dd67a83738e187d40b5090cc04f33adf
80b4910f7d wallet: Use shared pointer to retain wallet instance (João Barbosa)
Pull request description:
Currently there are 3 places where it makes sense to retain a wallet shared pointer:
- `vpwallets`;
- `interfaces::Wallet` interface instance - used by the UI;
- wallet RPC functions - given by `GetWalletForJSONRPCRequest`.
The way it is now it is possible to have, for instance, listunspent RPC and in parallel unload the wallet (once #13111 is merged) without blocking. Once the RPC finishes, the shared pointer will release the wallet.
It is also possible to get all existing wallets without blocking because the caller keeps a local list of shared pointers.
This is mostly relevant for wallet unloading.
This PR replaces #11402.
Tree-SHA512: b7e37c7e1ab56626085afe2d40b1628e8d4f0dbda08df01b7e618ecd2d894ce9b83d4219443f444ba889096286eff002f163cb0a48f37063b62e9ba4ccfa6cce
fac1223a56 Cache witness hash in CTransaction (MarcoFalke)
faab55fbb1 Make CMutableTransaction constructor explicit (MarcoFalke)
Pull request description:
This speeds up:
* compactblocks (v2)
* ATMP
* validation and miner (via `BlockWitnessMerkleRoot`)
* sigcache (see also unrelated #13204)
* rpc and rest (nice, but irrelevant)
This presumably slows down rescan, which uses a `CTransaction` and its `GetHash`, but never uses the `GetWitnessHash`. The slow down is proportional to the number of witness transactions in the rescan window. I.e. early in the chain there should be no measurable slow down. Later in the chain, there should be a slow down, but acceptable given the speedups in the modules mentioned above.
Tree-SHA512: 443e86acfcceb5af2163e68840c581d44159af3fd1fce266cab3504b29fcd74c50812b69a00d41582e7e1c5ea292f420ce5e892cdfab691da9c24ed1c44536c7
The new `loadwallet` RPC method allows an existing wallet to be loaded
dynamically at runtime.
`unloadwallet` and `createwallet` are not implemented. Notably,
`loadwallet` can only be used to load existing wallets, not to create a
new wallet.
Pass an error message back from CWallet::Verify(), and call
InitError/InitWarning from WalletInit::Verify().
This means that we can call CWallet::Verify() independently from
WalletInit and not have InitErrors printed to stdout. It also means that
the error can be reported to the user if dynamic wallet load fails.
This allows a single wallet to be verified. Prior to this commit, all
wallets were verified together by the WalletInit::Verify() function at
start-up.
Individual wallet verification will be done when loading wallets
dynamically at runtime.
WalletInit::Start calls postInitProcess() for each wallet. Previously
each call to postInitProcess() would attempt to schedule wallet
background flushing.
Just start wallet background flushing once from WalletInit::Start().
159c32d1f1 Add assertion to guide static analyzers. Clang Static Analyzer needs this guidance. (practicalswift)
fd447a6efe Fix dead stores. Values were stored but never read. Limit scope. (practicalswift)
Pull request description:
Fix Clang Static Analyzer warnings reported by @kallewoof in #12961:
* Fix dead stores. Values were stored but never read.
* Add assertion to guide static analyzers. See #12961 for details.
Tree-SHA512: 83dbec821f45217637316bee978e7543f2d2caeb7f7b0b3aec107fede0fff8baa756da8f6b761ae0d38537740839ac9752f6689109c38a4b05c0c041aaa3a1fb
66b0b1b2a6 Add compile time checking for all cs_wallet runtime locking assertions (practicalswift)
Pull request description:
Add compile time checking for `cs_wallet` runtime locking assertions.
This PR is a subset of #12665. The PR was broken up to make reviewing easier.
The intention is that literally all `EXCLUSIVE_LOCKS_REQUIRED`/`LOCKS_EXCLUDED`:s added in this PR should follow either directly or indirectly from `AssertLockHeld(…)`/`AssertLockNotHeld(…)`:s already existing in the repo.
Consider the case where function `A(…)` contains `AssertLockHeld(cs_foo)` (without
first locking `cs_foo` in `A`), and that `B(…)` calls `A(…)` (without first locking `cs_main`):
* It _directly_ follows that: `A(…)` should have an `EXCLUSIVE_LOCKS_REQUIRED(cs_foo)` annotation.
* It _indirectly_ follows that: `B(…)` should have an `EXCLUSIVE_LOCKS_REQUIRED(cs_foo)` annotation.
Tree-SHA512: d561d89e98a823922107e56dbd493f0f82e22edac91e51e6422f17daf2b446a70c143b7b157ca618fadd33d0ec63eb7a57dde5a83bfdf1fc19d71459b43e21fd
b6f0b4d wallet: Improve logging when BerkeleyDB environment fails to close (Tim Ruffing)
264c643 wallet: Reset BerkeleyDB handle after connection fails (Tim Ruffing)
Pull request description:
According to the BerkeleyDB docs, the DbEnv handle may not be accessed
after close() has been called. This change ensures that we create a new
handle after close() is called. This avoids a segfault when the first
connection attempt fails and then a second connection attempt tries to
call open() on the already closed DbEnv handle.
Without the patch, bitcoindd reliably crashes in the second call to `set_lg_dir()` after `close()` if
there is an issue with the database:
```
2018-05-03T13:27:21Z Bitcoin Core version v0.16.99.0-a024a1841-dirty (debug build)
[...]
2018-05-03T13:27:21Z Using wallet directory /home/tim/.bitcoin
2018-05-03T13:27:21Z init message: Verifying wallet(s)...
2018-05-03T13:27:21Z Using BerkeleyDB version Berkeley DB 4.8.30: (April 9, 2010)
2018-05-03T13:27:21Z Using wallet wallet.dat
2018-05-03T13:27:21Z BerkeleyEnvironment::Open: LogDir=/home/tim/.bitcoin/database
2018-05-03T13:27:21Z BerkeleyEnvironment::Open: Error -30974 opening database environment: DB_RUNRECOVERY: Fatal error, run database recovery
2018-05-03T13:27:21Z Moved old /home/tim/.bitcoin/database to /home/tim/.bitcoin/database.1525354041.bak. Retrying.
2018-05-03T13:27:21Z BerkeleyEnvironment::Open: LogDir=/home/tim/.bitcoin/database ErrorFile=/home/tim/.bitcoin/db.log
[1] 14533 segmentation fault (core dumped) ./src/bitcoind
```
After the fix:
```
2018-05-03T17:19:32Z Bitcoin Core version v0.16.99.0-cc09e3bd0-dirty (release build)
[...]
2018-05-03T17:19:32Z Using wallet directory /home/tim/.bitcoin
2018-05-03T17:19:32Z init message: Verifying wallet(s)...
2018-05-03T17:19:32Z Using BerkeleyDB version Berkeley DB 4.8.30: (April 9, 2010)
2018-05-03T17:19:32Z Using wallet wallet.dat
2018-05-03T17:19:32Z BerkeleyEnvironment::Open: LogDir=/home/tim/.bitcoin/database ErrorFile=/home/tim/.bitcoin/db.log
2018-05-03T17:19:32Z scheduler thread start
2018-05-03T17:19:32Z BerkeleyEnvironment::Open: Error -30974 opening database environment: DB_RUNRECOVERY: Fatal error, run database recovery
2018-05-03T17:19:32Z Moved old /home/tim/.bitcoin/database to /home/tim/.bitcoin/database.1525367972.bak. Retrying.
2018-05-03T17:19:32Z BerkeleyEnvironment::Open: LogDir=/home/tim/.bitcoin/database ErrorFile=/home/tim/.bitcoin/db.log
2018-05-03T17:19:32Z Cache configuration:
2018-05-03T17:19:32Z * Using 2.0MiB for block index database
2018-05-03T17:19:32Z * Using 8.0MiB for chain state database
2018-05-03T17:19:32Z * Using 440.0MiB for in-memory UTXO set (plus up to 286.1MiB of unused mempool space)
2018-05-03T17:19:32Z init message: Loading block index..
[...]
```
Tree-SHA512: b809b318e5014ec47d023dc3dc40826b9706bfb211fa08bc2d29f36971b96caa10ad48d9a3f96c03933be46fa4ff7e00e952ac77bfffb6563767fb08aa4f23d6
a8da482 Bump wallet version for pre split keypool (Andrew Chow)
dfcd9f3 Use a keypool of presplit keys after upgrading to hd chain split (Andrew Chow)
5c50e93 Allow -upgradewallet to upgradewallets to HD (Andrew Chow)
2bcf2b5 Test sethdseed (Andrew Chow)
b5ba01a Add 'sethdseed' RPC to initialize or replace HD seed (Chris Moore)
dd3c07a Separate HaveKey function that checks whether a key is in a keystore (Andrew Chow)
Pull request description:
Revival/rebase of #11085
Adds a new command `sethdseed` which allows you to either set or generate a new HD seed to be used. A new keypool can be generated or the original one kept and new keys added to the keypool will come from the new HD seed.
Wallets that are not HD will be upgraded to be version FEATURE_HD_SPLIT when the `sethdseed` RPC command is used.
I have also add some tests for this.
Additionally `-upgradewallet` can now be used to upgrade a wallet from non-HD to HD. When it is used for such an upgrade, the keypool will be regenerated.
Tree-SHA512: e56c792e150590429ac4a1061e8d6f7b20cca06366e184eb9bbade4cd6ae82699a28fe84f87031eadba97ad2c1606517a105f00fb7b45779c979243020071adb
Bump the wallet version to indicate support for the pre split keypool.
Also prevents any wallets from upgrading to versions between HD_SPLIT
and PRE_SPLIT_KEYPOOL.
After upgrading to HD chain split, we want to continue to use keys
from the old keypool. To do this, before we generate any new keys after
upgrading, we mark all of the keypool entries as being pre-chain
split and move them to a separate pre chain split keypool. Keys are
fetched from that keypool until it is emptied. Only then are the new
internal and external keypools used.
Changes the maximum upgradewallet version to the latest wallet version
number, 159900. Non-HD wallets will be upgraded to use HD derivation.
Non HD chain split wallets will be upgraded to HD chain split.
If a non-HD wallet is upgraded to HD, the keypool will be entirely
regenerated.
Since upgradewallet is effectively run during a first run, all of the
first run initial setup stuff is combined with the upgrade to HD
16be13345 Fix rescanblockchain rpc to property report progress (Ben Woosley)
Pull request description:
Previously it assumed tip in all cases. This also extracts a RescanVerificationProgress helper object whose role is to manage reporting, in order to simplify ScanForWalletTransactions - more lines in total, but much simpler to follow the core logic.
Tree-SHA512: 5ebed0c56fae4ccfe613ff1d7082cb6da5a86635a8993ed3af70b500a4ea43074121aea9219b2f0321fbfeb7efcb964bdc2199297a64ca0fa85d9d07aa637d40
CWallet::ScanForWalletTransactions did not previously take into account
pindexStop when calculating progress.
Renamed progress vars to progress_*.
rescanblockchain is the only rpc that uses this parameter.
According to the BerkeleyDB docs, the DbEnv handle may not be accessed
after close() has been called. This change ensures that we create a new
handle after close() is called. This avoids a segfault when the first
connection attempt fails and then a second connection attempt tries to
call open() on the already closed DbEnv handle.
1e46d8a Get rid of ambiguous OutputType::NONE value (Russell Yanofsky)
Pull request description:
Based on suggestion by @sipa https://github.com/bitcoin/bitcoin/pull/12119#issuecomment-357982763
After #12119, the NONE output type was overloaded to refer to either an output type that couldn't be parsed, or to an automatic change output mode. This change drops the NONE enum and uses a simple bool to indicate parse failure, and a new CHANGE_AUTO enum to refer the change output type.
This change is almost a pure refactoring except it makes RPCs reject empty string ("") address types instead of treating them like they were unset. This simplifies the parsing code a little bit and could prevent RPC usage mistakes. It's noted in the release notes.
Follows up #12408 by @MarcoFalke
Followups for future PRs:
- [ ] Add explicit support for specifying "auto" in `ParseOutputType` as suggested by promag and sipa: https://github.com/bitcoin/bitcoin/pull/12729#issuecomment-374799567 and https://github.com/bitcoin/bitcoin/pull/12729#discussion_r175969481
- [ ] Add wallet `AddressChangeType` method to complement `TransactionChangeType`: https://github.com/bitcoin/bitcoin/pull/12729#discussion_r175969618.
Tree-SHA512: 8b08b272bcb177a0a9e556dcd965840a7fe601ef83ca97938b879c9b1a33b5b3f96939e1bceef11ba7c644ac21bfd6c1dbc6ca715cd1da4ace50475240e4ee48
9cb6cdc Simplify semantics of ChainStateFlushed callback (Matt Corallo)
50b6533 scripted-diff: Rename SetBestChain callback ChainStateFlushed (Matt Corallo)
Pull request description:
Previously, ChainStateFlushed would fire either if a full flush
completed (which can happen due to memory limits, forced flush, or
on its own DATABASE_WRITE_INTERVAL timer) *or* on a
ChainStateFlushed-specific DATABASE_WRITE_INTERVAL timer. This is
both less clear for clients (as there are no guarantees about a
flush having actually happened prior to the call), and reults in
extra flushes not clearly intended by the code. We drop the second
case, providing a strong guarantee without removing the periodit
timer-based flushing.
This is a follow-up to discussion in #11857.
Tree-SHA512: 22ba3a0954d265d28413dbf87040790ca5b439820ee7bbadab14028295ec190de82ce5cd664426c82e58b706dc84278868026fa8d066702eb6e6962c9ace1f8e
e2f58f4 wallet: Make vpwallets usage thread safe (João Barbosa)
Pull request description:
This PR turns the functions introduced in #13017 thread safe. This is required to correctly support dynamically loading wallets, which is implemented in #10740.
Tree-SHA512: efaa09e501636cf957aa33de83719ce09dc0c2a19daff741a94ef10d6b7ba5dee538355b80c96ead995140f99f5df0c92fb0e22ae1adb8f397eb478280c8d8c7
This much more accurately captures the meaning of the callback.
-BEGIN VERIFY SCRIPT-
sed -i 's/SetBestChain/ChainStateFlushed/g' src/validationinterface.h src/validationinterface.cpp src/wallet/wallet.h src/wallet/wallet.cpp src/validation.cpp src/index/txindex.h src/index/txindex.cpp
-END VERIFY SCRIPT-
d8e9a2a Remove "rpc" category from GetWarnings (Wladimir J. van der Laan)
7da3b0a rpc: Move RPC_FORBIDDEN_BY_SAFE_MODE code to reserved section (Wladimir J. van der Laan)
2ae705d Remove Safe mode (Andrew Chow)
Pull request description:
Rebase of #10563. Safe mode was [disabled by default and deprecated in 0.16](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.16.0.md#safe-mode-disabled-by-default), so probably should be removed for 0.17.
> Rationale:
>
> Safe mode is useless. It only disables some RPC commands when large work forks are detected. Nothing else is affected by safe mode. It seems that very few people would be affected by safe mode. The people who use Core as a wallet are primarily using it through the GUI, which safe mode does not effect. In the GUI, transactions will still be made as normal; only a warning is displayed.
>
> I also don't think that we should be disabling RPC commands or any functionality in general. If we do, it should be done consistently, which safe mode is not. If we want to keep the idea of a safe mode around, I think that the current system needs to go first before a new system can be implemented.
Tree-SHA512: 067938f47ca6e879fb6c3c4e21f9946fd7c5da3cde67ef436f1666798c78d049225b9111dc97064f42b3bc549d3915229fa19ad5a634588f381e34fc65d64044
7d0f80b Use anonymous namespace instead of static functions (Pieter Wuille)
b61fb71 Mention removal of bare multisig IsMine in release notes (Pieter Wuille)
9c2a8b8 Do not treat bare multisig as IsMine (Pieter Wuille)
08f3228 Optimization: only test for witness scripts at top level (Pieter Wuille)
3619735 Track difference between scriptPubKey and P2SH execution in IsMine (Pieter Wuille)
ac6ec62 Switch to a private version of SigVersion inside IsMine (Pieter Wuille)
19fc973 Do not expose SigVersion argument to IsMine (Pieter Wuille)
fb1dfbb Remove unused IsMine overload (Pieter Wuille)
952d821 Make CScript -> CScriptID conversion explicit (Pieter Wuille)
Pull request description:
Currently our wallet code will treat bare multisig outputs (meaning scriptPubKeys with multiple public keys + `OP_CHECKMULTISIG` operator in it) as ours without the user asking for it, as long as all private keys in it are in our wallet.
This is a pointless feature. As it only works when all private keys are in one place, it's useless compared to single key outputs (P2PK, P2PKH, P2WPKH, P2SH-P2WPKH), and worse in terms of space, cost, UTXO size, and ability to test (due to lack of address format for them).
Furthermore, they are problematic in that producing a list of all `scriptPubKeys` we accept is not tractable (it involves all combinations of all public keys that are ours). In further wallet changes I'd like to move to a model where all scriptPubKeys that are treated as ours are explicit, rather than defined by whatever keys we have. The current behavior of the wallet is very hard to model in such a design, so I'd like to get rid of it.
I think there are two options:
* Remove it entirely (do not ever accept bare multisig outputs as ours, unless watched)
* Only accept bare multisig outputs in situations where the P2SH version of that output would also be acceptable
This PR implements the first option. The second option was explored in #12874.
Tree-SHA512: 917ed45b3cac864cee53e27f9a3e900390c576277fbd6751b1250becea04d692b3b426fa09065a3399931013bd579c4f3dbeeb29d51d19ed0c64da75d430ad9a
fac0db0 wallet: Make fee settings non-static members (MarcoFalke)
Pull request description:
The wallet header defined some globals (they were called "settings"), that should be class members instead.
This commit is hopefully only refactoring, apart from a multiwallet bugfix: Calling the rpc `settxfee` for one wallet, would set (and change) the fee rate for all loaded wallets. (See added test case)
Tree-SHA512: 4ab6ec2f5c714742396ded5e451ec3b1ceb771e3696492de29889d866de4365b3fbe4a2784d085c8b8bd11b1ebb8a1fec99ab2c62eee716791cfc67c0cf29e1b
cead28b [docs] Add release notes for deprecated 'account' API (John Newbery)
72c9575 [wallet] [tests] Add tests for accounts/labels APIs (John Newbery)
109e05d [wallet] [rpc] Deprecate wallet 'account' API (John Newbery)
3576ab1 [wallet] [rpc] Deprecate account RPC methods (John Newbery)
3db1ba0 [tests] Set -deprecatedrpc=accounts in tests (John Newbery)
4e671f0 [tests] Rename rpc_listtransactions.py to wallet_listtransactions.py (John Newbery)
a28b907 [wallet] [rpc] Remove duplicate entries in rpcwallet.cpp's CRPCCommand table (John Newbery)
Pull request description:
Deprecate all accounts functionality and make it only accessible by using `-deprecatedrpc=accounts`.
Accounts specific RPCs, account arguments, and account related results all require the `-deprecatedrpc=accunts` startup option now in order to see account things.
Several wallet functional tests use the accounts system. Those tests are unchanged, except to start the nodes with `-deprecatedrpc=accounts`. We can slowly migrate those tests to use the 'label' API instead of the 'account' API before accounts are fully removed.
Tree-SHA512: 89f4ae2fe6de4a1422f1817b0997ae22d63ab5a1a558362ce923a3871f3e42963405d6573c69c27f1764679cdee5b51bf52202cc407f1361bfd8066d652f3f37
1f83839 [wallet] [tests] Test disallowed multiwallet params (John Newbery)
3476e3c [wallet] Fix zapwallettxes/multiwallet interaction. (John Newbery)
Pull request description:
`-zapwallettxes` should be disallowed when starting bitcoin in multiwallet mode.
There's code in `WalletInit::ParameterInteraction()` to disallow `-zapwallettxes` when running in multiwallet mode. This code functioned as expected when passing the parameter `-zapwallettxes=1`, but not when passing the parameter `-zapwallettxes` (ie without the value specified). Fix that and add a test.
The new test in the
_[wallet] [tests] Test disallowed multiwallet params_ commit reproduces the bug and should fail against master.
Fixes#12505
Tree-SHA512: 6cd921717e9c7d2773ca84c946c310c2adec8430e37cbecdb33a620f510db3058a72bd328411812ba415111bc52a3367b332c9d15a37a92ccfd7ae785d2f32ab
