Commit graph

725 commits

Author SHA1 Message Date
Wladimir J. van der Laan
d42a4fe5aa
Merge #11041: Add LookupBlockIndex
92fabcd44 Add LookupBlockIndex function (João Barbosa)
43a32b739 Add missing cs_lock in CreateWalletFromFile (João Barbosa)
f814a3e8f Fix cs_main lock in LoadExternalBlockFile (João Barbosa)
c651df8b3 Lock cs_main while loading block index in AppInitMain (João Barbosa)
02de6a6bc Assert cs_main is held when accessing mapBlockIndex (João Barbosa)

Pull request description:

  Replace all `mapBlockIndex` lookups with the new `LookupBlockIndex()`. In some cases it avoids a second lookup.

Tree-SHA512: ca31118f028a19721f2191d86f2dd398144d04df345694575a64aeb293be2f85785201480c3c578a0ec99690516205708558c0fd4168b09313378fd4e60a8412
2018-03-13 19:12:35 +01:00
Andrew Chow
6a34ff5335 Have SelectCoinsMinConf and SelectCoins use BnB or Knapsack and use it
Allows SelectCoinsMinConf and SelectCoins be able to switch between
using BnB or Knapsack for choosing coins.

Has SelectCoinsMinConf do the preprocessing necessary to support either
BnB or Knapsack. This includes calculating the filtering the effective
values for each input.

Uses BnB in CreateTransaction to find an exact match for the output.
If BnB fails, it will fallback to the Knapsack solver.
2018-03-13 12:39:27 -04:00
Andrew Chow
cd927ff328 Move original knapsack solver tests to coinselector_tests.cpp 2018-03-13 12:39:26 -04:00
Andrew Chow
fb716f7b25 Move current coin selection algorithm to coinselection.{cpp,h}
Moves the current coin selection algorithm out of SelectCoinsMinConf
and puts it in coinselection.{cpp,h}. The new function, KnapsackSolver,
instead of taking a vector of COutputs, will take a vector of CInputCoins
that is prepared by SelectCoinsMinConf.
2018-03-13 12:39:26 -04:00
Andrew Chow
4b2716da46 Remove coinselection.h -> wallet.h circular dependency
Changes CInputCoin to coinselection and to use CTransactionRef in
order to avoid a circular dependency. Also moves other coin selection
specific variables out of wallet.h to coinselectoin.h
2018-03-13 12:39:26 -04:00
Andrew Chow
7d77eb1a5b Use a struct for output eligibility
Instead of specifying 3 parameters, use a struct for those parameters
in order to reduce the number of arguments to SelectCoinsMinConf.
2018-03-13 12:39:26 -04:00
Andrew Chow
ce7435cf1e Move output eligibility to a separate function 2018-03-13 12:39:26 -04:00
Andrew Chow
0185939be6 Implement Branch and Bound coin selection in a new file
Create a new file for coin selection logic and implement the BnB algorithm in it.
2018-03-13 12:39:17 -04:00
Andrew Chow
12ec29d3bb Calculate and store the number of bytes required to spend an input 2018-03-09 21:15:36 -05:00
practicalswift
1f45e2164a scripted-diff: Convert 11 enums into scoped enums (C++11)
-BEGIN VERIFY SCRIPT-

sed -i 's/enum DBErrors/enum class DBErrors/g' src/wallet/walletdb.h
git grep -l DB_ | xargs sed -i 's/DB_\(LOAD_OK\|CORRUPT\|NONCRITICAL_ERROR\|TOO_NEW\|LOAD_FAIL\|NEED_REWRITE\)/DBErrors::\1/g'
sed -i 's/^    DBErrors::/    /g' src/wallet/walletdb.h

sed -i 's/enum VerifyResult/enum class VerifyResult/g' src/wallet/db.h
sed -i 's/\(VERIFY_OK\|RECOVER_OK\|RECOVER_FAIL\)/VerifyResult::\1/g' src/wallet/db.cpp

sed -i 's/enum ThresholdState/enum class ThresholdState/g' src/versionbits.h
git grep -l THRESHOLD_ | xargs sed -i 's/THRESHOLD_\(DEFINED\|STARTED\|LOCKED_IN\|ACTIVE\|FAILED\)/ThresholdState::\1/g'
sed -i 's/^    ThresholdState::/    /g' src/versionbits.h

sed -i 's/enum SigVersion/enum class SigVersion/g' src/script/interpreter.h
git grep -l SIGVERSION_ | xargs sed -i 's/SIGVERSION_\(BASE\|WITNESS_V0\)/SigVersion::\1/g'
sed -i 's/^    SigVersion::/    /g' src/script/interpreter.h

sed -i 's/enum RetFormat {/enum class RetFormat {/g' src/rest.cpp
sed -i 's/RF_\(UNDEF\|BINARY\|HEX\|JSON\)/RetFormat::\1/g' src/rest.cpp
sed -i 's/^    RetFormat::/    /g' src/rest.cpp

sed -i 's/enum HelpMessageMode {/enum class HelpMessageMode {/g' src/init.h
git grep -l HMM_ | xargs sed -i 's/HMM_BITCOIN/HelpMessageMode::BITCOIN/g'
sed -i 's/^    HelpMessageMode::/    /g' src/init.h

sed -i 's/enum FeeEstimateHorizon/enum class FeeEstimateHorizon/g' src/policy/fees.h

sed -i 's/enum RBFTransactionState/enum class RBFTransactionState/g' src/policy/rbf.h
git grep -l RBF_ | xargs sed -i 's/RBF_TRANSACTIONSTATE_\(UNKNOWN\|REPLACEABLE_BIP125\|FINAL\)/RBFTransactionState::\1/g'
sed -i 's/^    RBFTransactionState::/    /g' src/policy/rbf.h

sed -i 's/enum BlockSource {/enum class BlockSource {/g' src/qt/clientmodel.h
git grep -l BLOCK_SOURCE_ | xargs sed -i 's/BLOCK_SOURCE_\(NONE\|REINDEX\|DISK\|NETWORK\)/BlockSource::\1/g'
sed -i 's/^    BlockSource::/    /g' src/qt/clientmodel.h

sed -i 's/enum FlushStateMode {/enum class FlushStateMode {/g' src/validation.cpp
sed -i 's/FLUSH_STATE_\(NONE\|IF_NEEDED\|PERIODIC\|ALWAYS\)/FlushStateMode::\1/g' src/validation.cpp
sed -i 's/^    FlushStateMode::/    /g' src/validation.cpp

sed -i 's/enum WitnessMode {/enum class WitnessMode {/g' src/test/script_tests.cpp
sed -i 's/WITNESS_\(NONE\|PKH\|SH\)/WitnessMode::\1/g' src/test/script_tests.cpp
sed -i 's/^    WitnessMode::/    /g' src/test/script_tests.cpp

-END VERIFY SCRIPT-
2018-03-09 15:03:40 +01:00
Russell Yanofsky
b4bc32a451 [wallet] Get rid of CWalletTx default constructor
No change in behavior in the normal case. But buggy mapWallet lookups with
invalid txids will now throw exceptions instead of inserting dummy entries into
the map, and potentially causing segfaults and other failures.

This also makes it a compiler error to use the mapWallet[hash] syntax which
could create dummy entries.
2018-03-07 21:12:47 -05:00
Russell Yanofsky
a128bdc9e1 [wallet] Construct CWalletTx objects in CommitTransaction
Construct CWalletTx objects in CWallet::CommitTransaction, instead of having
callers do it. This ensures CWalletTx objects are constructed in a uniform way
and all fields are set.

This also makes it possible to avoid confusing and wasteful CWalletTx copies in
https://github.com/bitcoin/bitcoin/pull/9381

There is no change in behavior.
2018-03-07 21:12:47 -05:00
Wladimir J. van der Laan
98bc27fb59
Merge #11687: External wallet files
be8ab7d08 Create new wallet databases as directories rather than files (Russell Yanofsky)
26c06f24e Allow wallet files not in -walletdir directory (Russell Yanofsky)
d8a99f65e Allow wallet files in multiple directories (Russell Yanofsky)

Pull request description:

  This change consists of three commits:

  * The first commit is a pure refactoring that removes the restriction that two wallets can only be opened at the same time if they are contained in the same directory.
  * The second commit removes the restriction that `-wallet` filenames can only refer to files in the `-walletdir` directory.
  * The third commit makes second commit a little safer by changing bitcoin to create wallet databases as directories rather than files, so they can be safely backed up.

  All three commits should be straightforward:

  *  The first commit adds around 20 lines of new code and then updates a bunch of function signatures (generally updating them to take plain fs::path parameters, instead of combinations of strings, fs::paths, and objects like CDBEnv and CWalletDBWrapper).
  * The second commit removes two `-wallet` filename checks and adds some test cases to the multiwallet unit test.
  * The third commit just changes the mapping from specified wallet paths to bdb environment & data paths.

    ---

  **Note:** For anybody looking at this PR for the first time, I think you can skip the comments before _20 Nov_ and start reading at https://github.com/bitcoin/bitcoin/pull/11687#issuecomment-345625565. Comments before  _20 Nov_ were about an earlier version of the PR that didn't include the third commit, and then confusion from not seeing the first commit.

Tree-SHA512: 00bbb120fe0df847cf57014f75f1f7f1f58b0b62fa0b3adab4560163ebdfe06ccdfff33b4231693f03c5dc23601cb41954a07bcea9a4919c8d42f7d62bcf6024
2018-03-07 17:11:54 +01:00
Wladimir J. van der Laan
b225010a80
Merge #11372: Address encoding cleanup
92f1f8b31 Split off key_io_tests from base58_tests (Pieter Wuille)
119b0f85e Split key_io (address/key encodings) off from base58 (Pieter Wuille)
ebfe217b1 Stop using CBase58Data for ext keys (Pieter Wuille)
32e69fa0d Replace CBitcoinSecret with {Encode,Decode}Secret (Pieter Wuille)

Pull request description:

  This PR contains some of the changes left as TODO in #11167 (and built on top of that PR). They are not intended for backporting.

  This removes the `CBase58`, `CBitcoinSecret`, `CBitcoinExtKey`, and `CBitcoinExtPubKey` classes, in favor of simple `Encode`/`Decode` functions. Furthermore, all Bitcoin-specific logic (addresses, WIF, BIP32) is moved to `key_io.{h,cpp}`, leaving `base58.{h,cpp}` as a pure utility that implements the base58 encoding/decoding logic.

Tree-SHA512: a5962c0ed27ad53cbe00f22af432cf11aa530e3efc9798e25c004bc9ed1b5673db5df3956e398ee2c085e3a136ac8da69fe7a7d97a05fb2eb3be0b60d0479655
2018-03-07 00:09:48 +01:00
João Barbosa
92fabcd443 Add LookupBlockIndex function 2018-03-06 19:52:19 +00:00
João Barbosa
43a32b7395 Add missing cs_lock in CreateWalletFromFile 2018-03-06 19:21:16 +00:00
Russell Yanofsky
be8ab7d082 Create new wallet databases as directories rather than files
This change should make it easier for users to make complete backups of wallets
because they can now just back up the specified `-wallet=<path>` path directly,
instead of having to back up the specified path as well as the transaction log
directory (for incompletely flushed wallets).

Another advantage of this change is that if two wallets are located in the same
directory, they will now use their own BerkeleyDB environments instead using a
shared environment. Using a shared environment makes it difficult to manage and
back up wallets separately because transaction log files will contain a mix of
data from all wallets in the environment.
2018-03-03 10:26:55 -05:00
Russell Yanofsky
d8a99f65e5 Allow wallet files in multiple directories
Remove requirement that two wallet files can only be opened at the same time if
they are contained in the same directory.

This change mostly consists of updates to function signatures (updating
functions to take fs::path arguments, instead of combinations of strings,
fs::path, and CDBEnv / CWalletDBWrapper arguments).
2018-03-03 10:26:55 -05:00
Russell Yanofsky
7ba2d57852 Fix ListCoins test failure due to unset g_wallet_allow_fallback_fee
New global variables were introduced in #11882 and not setting them causes:

    wallet/test/wallet_tests.cpp(638): error in "ListCoins": check wallet->CreateTransaction({recipient}, wtx, reservekey, fee, changePos, error, dummy) failed
    wallet/test/wallet_tests.cpp(679): error in "ListCoins": check list.begin()->second.size() == 2 failed [1 != 2]
    wallet/test/wallet_tests.cpp(686): error in "ListCoins": check available.size() == 2 failed [1 != 2]
    wallet/test/wallet_tests.cpp(705): error in "ListCoins": check list.begin()->second.size() == 2 failed [1 != 2]

It's possible to reproduce the failure reliably by running:

    src/test/test_bitcoin --log_level=test_suite --run_test=wallet_tests/ListCoins

Failures happen nondeterministically because boost test framework doesn't run
tests in a specified order, and tests that run previously can set the global
variables and mask the bug.
2018-03-02 14:50:49 -05:00
Wladimir J. van der Laan
987a80995a
Merge #11882: Disable default fallbackfee on mainnet
3f592b8 [QA] add wallet-rbf test (Jonas Schnelli)
8222e05 Disable wallet fallbackfee by default on mainnet (Jonas Schnelli)

Pull request description:

  Removes the default fallback fee on mainnet (but keeps it on testnet/regtest).

  Transactions using the fallbackfee in case the fallback fee has not been set are getting rejected.

Tree-SHA512: e54d2594b7f954e640cc513a18b0bfbe189f15e15bdeed4fe02b7677f939bca1731fef781b073127ffd4ce08a595fb118259b8826cdaa077ff7d5ae9495810db
2018-03-01 19:01:52 +01:00
Jonas Schnelli
8222e057fe
Disable wallet fallbackfee by default on mainnet 2018-02-25 09:33:46 +08:00
Jonas Schnelli
bf3353de90
Merge #12287: Optimise lock behaviour for GuessVerificationProgress()
90ba2df11 Fix missing cs_main lock for GuessVerificationProgress() (Jonas Schnelli)

Pull request description:

  `GuessVerificationProgress()` needs `cs_main` due to accessing the `pindex->nChainTx`.
  This adds a `AssertLockHeld` in `GuessVerificationProgress()` and adds the missing locks in...
  * `LoadChainTip()`
  * `ScanForWalletTransactions()` (got missed in #11281)
  * GUI, `ClientModel::getVerificationProgress()` <--- **this may have GUI performance impacts**, but could be relaxed later with a cache or something more efficient.

Tree-SHA512: 13302946571422375f32af8e396b9d2c1180f2693ea363aeba9e98c8266ddec64fe7862bfdcbb5a93a4b12165a61eec1e51e4e7d7a8515fa50879095dc163412
2018-02-25 09:13:43 +08:00
Pieter Wuille
119b0f85e2 Split key_io (address/key encodings) off from base58 2018-02-19 18:55:21 -08:00
Wladimir J. van der Laan
fd65937ec6
Merge #12356: Fix 'mempool min fee not met' debug output
bb00c95 Consistently use FormatStateMessage in RPC error output (Ben Woosley)
8b8a1c4 Add test for 'mempool min fee not met' rpc error (Ben Woosley)
c04e0f6 Fix 'mempool min fee not met' debug output (Ben Woosley)

Pull request description:

  Output the value that is tested, rather than the unmodified fee value.

  Prompted by looking into: #11955

Tree-SHA512: fc0bad47d4af375d208f657a6ccbad6ef7f4e2989ae2ce1171226c22fa92847494a2c55cca687bd5a1548663ed3313569bcc31c00d53c0c193a1b865dd8a7657
2018-02-15 16:35:42 +01:00
Wladimir J. van der Laan
6bb9c13f9a
Merge #11866: Do not un-mark fInMempool on wallet txn if ATMP fails.
6ef86c9 Do not un-mark fInMempool on wallet txn if ATMP fails. (Matt Corallo)

Pull request description:

  Irrespective of the failure reason, un-marking fInMempool
  out-of-order is incorrect - it should be unmarked when
  TransactionRemovedFromMempool fires.

  Clean up of #11839, which I think was the wrong fix.

Tree-SHA512: 580731297eeac4c4c99ec695e15b09febf62249237bc367fcd1830fc811d3166f9336e7aba7f2f6f8601960984ae22cebed781200db0f04e7cd2008db1a83f64
2018-02-14 16:52:46 +01:00
Wladimir J. van der Laan
0dfc25f82a
Merge #12381: Remove more boost threads
004f999 boost: drop boost threads for [alert|block|wallet]notify (Cory Fields)
0827267 boost: drop boost threads from torcontrol (Cory Fields)
ba91724 boost: remove useless threadGroup parameter from Discover (Cory Fields)
f26866b boost: drop boost threads for upnp (Cory Fields)

Pull request description:

  This doesn't completely get rid of boost::thread, but this batch should be easy to review, and leaves us with only threadGroup (scheduler + scriptcheck) remaining.

  Note to reviewers: The upnp diff changes a bunch of whitespace, it's much more clear with 'git diff -w'

Tree-SHA512: 5a356798d0785f93ed143d1f0afafe890bc82f0d470bc969473da2d2aa78bcb9b096f7ba11b92564d546fb447d4bd0d347e7842994ea0170aafd53fda7e0a66e
2018-02-12 10:34:49 +01:00
Cory Fields
004f999946 boost: drop boost threads for [alert|block|wallet]notify 2018-02-08 14:35:29 -05:00
Wladimir J. van der Laan
d405beea26
Merge #12333: Make CWallet::ListCoins atomic
2f960b5 [wallet] Indent only change of CWallet::AvailableCoins (João Barbosa)
1beea7a [wallet] Make CWallet::ListCoins atomic (João Barbosa)

Pull request description:

  Fix a potencial race in `CWallet::ListCoins`.

  Replaces `cs_main` and `cs_wallet` locks by assertions in `CWallet::AvailableCoins`.

Tree-SHA512: 09109f44a08b4b53f7605d950ab506d3f748490ab9aed474aa200e93f7b0b9f96f9bf60abe1c5f658240fd13d9e3267c0dd43fd3c1695d82384198ce1da8109f
2018-02-08 19:48:21 +01:00
João Barbosa
2f960b5070 [wallet] Indent only change of CWallet::AvailableCoins 2018-02-08 18:18:51 +00:00
João Barbosa
1beea7af92 [wallet] Make CWallet::ListCoins atomic 2018-02-08 18:18:28 +00:00
Ben Woosley
bb00c95c16
Consistently use FormatStateMessage in RPC error output
This will include the error code and debug output as well as the reason string.

See #11955 for the motivation.
2018-02-08 11:02:41 -05:00
Wladimir J. van der Laan
663911ed58
Merge #12282: wallet: Disallow abandon of conflicted txes
fa795cf wallet: Disallow abandon of conflicted txes (MarcoFalke)

Pull request description:

  Abandon transactions that are already conflicted is a noop, so don't try and return false/throw instead.

Tree-SHA512: fd2af4149bd2323f7f31fe18685c763790b8589319b4e467b464ab456d5e8971501ab16d124e57a22693666b06ae433ac3e59f0fd6dfbd2be2c6cae8be5bcbd8
2018-02-08 16:32:15 +01:00
MarcoFalke
fa795cf9c5
wallet: Disallow abandon of conflicted txes 2018-02-06 09:54:31 -05:00
Jonas Schnelli
90ba2df11b
Fix missing cs_main lock for GuessVerificationProgress() 2018-01-30 17:49:22 -10:00
João Barbosa
039425cf4f [wallet] Remove duplicate mapWallet lookups 2018-01-28 17:25:16 +00:00
João Barbosa
31dbd5af48 [wallet] Add change type to CCoinControl 2018-01-24 15:52:05 +00:00
Wladimir J. van der Laan
95941396ff
Merge #12119: [wallet] use P2WPKH change output if any destination is P2WPKH or P2WSH
596c446 [wallet] use P2WPKH change output if any destination is P2WPKH or P2WSH (Sjors Provoost)

Pull request description:

  If `-changetype` is not explicitly set, then regardless of `-addresstype`, the wallet will use a ~`bech32` change address~ `P2WPKH` change output if any destination is `P2WPKH` or `P2WSH`.

  This seems more intuitive to me and more in line with the spirit of [BIP-69](https://github.com/bitcoin/bips/blob/master/bip-0069.mediawiki).

  When combined with #11991 a QT user could opt to use `bech32` exclusively without having to figure out how to launch with `-changetype=bech32`, although so would #11937.

Tree-SHA512: 9238d3ccd1f3be8dfdd43444ccf45d6bdc6584ced3172a3045f3ecfec4a7cc8999db0cdb76ae49236492a84e6dbf3a1fdf18544d3eaf6d518e1f8bd241db33e7
2018-01-24 15:22:42 +01:00
Jonas Schnelli
ccd8ef65f9
Reduce cs_main lock in ReadBlockFromDisk, only read GetBlockPos under the lock 2018-01-23 20:24:53 -10:00
Jonas Schnelli
bc356b4268
Make sure WalletRescanReserver has successfully reserved the rescan 2018-01-23 20:24:18 -10:00
Jonas Schnelli
8d0b610fe8
Avoid pemanent cs_main/cs_wallet lock during wallet rescans 2018-01-23 20:22:33 -10:00
Sjors Provoost
596c44633f
[wallet] use P2WPKH change output if any destination is P2WPKH or P2WSH
Only if -changetype is not set and -addresstype is not "legacy".
2018-01-23 17:56:15 +01:00
Jonas Schnelli
d889c036cd
Merge #11403: SegWit wallet support
b224a47a1 Add address_types test (Pieter Wuille)
7ee54fd7c Support downgrading after recovered keypool witness keys (Pieter Wuille)
940a21932 SegWit wallet support (Pieter Wuille)
f37c64e47 Implicitly know about P2WPKH redeemscripts (Pieter Wuille)
57273f2b3 [test] Serialize CTransaction with witness by default (Pieter Wuille)
cf2c0b6f5 Support P2WPKH and P2SH-P2WPKH in dumpprivkey (Pieter Wuille)
37c03d3e0 Support P2WPKH addresses in create/addmultisig (Pieter Wuille)
3eaa003c8 Extend validateaddress information for P2SH-embedded witness (Pieter Wuille)
30a27dc5b Expose method to find key for a single-key destination (Pieter Wuille)
985c79552 Improve witness destination types and use them more (Pieter Wuille)
cbe197470 [refactor] GetAccount{PubKey,Address} -> GetAccountDestination (Pieter Wuille)
0c8ea6380 Abstract out IsSolvable from Witnessifier (Pieter Wuille)

Pull request description:

  This implements a minimum viable implementation of SegWit wallet support, based on top of #11389, and includes part of the functionality from #11089.

  Two new configuration options are added:
  * `-addresstype`, with options `legacy`, `p2sh`, and `bech32`. It controls what kind of addresses are produced by `getnewaddress`, `getaccountaddress`, and `createmultisigaddress`.
  * `-changetype`, with the same options, and by default equal to `-addresstype`, that controls what kind of change is used.

  All wallet private and public keys can be used for any type of address. Support for address types dependent on different derivation paths will need a major overhaul of how our internal detection of outputs work. I expect that that will happen for a next major version.

  The above also applies to imported keys, as having a distinction there but not for normal operations is a disaster for testing, and probably for comprehension of users. This has some ugly effects, like needing to associate the provided label to `importprivkey` with each style address for the corresponding key.

  To deal with witness outputs requiring a corresponding redeemscript in wallet, three approaches are used:
  * All SegWit addresses created through `getnewaddress` or multisig RPCs explicitly get their redeemscripts added to the wallet file. This means that downgrading after creating a witness address will work, as long as the wallet file is up to date.
  * All SegWit keys in the wallet get an _implicit_ redeemscript added, without it being written to the file. This means recovery of an old backup will work, as long as you use new software.
  * All keypool keys that are seen used in transactions explicitly get their redeemscripts added to the wallet files. This means that downgrading after recovering from a backup that includes a witness address will work.

  These approaches correspond to solutions 3a, 1a, and 5a respectively from https://gist.github.com/sipa/125cfa1615946d0c3f3eec2ad7f250a2. As argued there, there is no full solution for dealing with the case where you both downgrade and restore a backup, so that's also not implemented.

  `dumpwallet`, `importwallet`, `importmulti`, `signmessage` and `verifymessage` don't work with SegWit addresses yet. They're remaining TODOs, for this PR or a follow-up. Because of that, several tests unexpectedly run with `-addresstype=legacy` for now.

Tree-SHA512: d425dbe517c0422061ab8dacdc3a6ae47da071450932ed992c79559d922dff7b2574a31a8c94feccd3761c1dffb6422c50055e6dca8e3cf94a169bc95e39e959
2018-01-10 20:55:41 -10:00
Pieter Wuille
7ee54fd7c7 Support downgrading after recovered keypool witness keys 2018-01-09 15:35:31 -08:00
Pieter Wuille
940a21932b SegWit wallet support
This introduces two command line flags (-addresstype and -changetype) which control
the type of addresses/outputs created by the GUI and RPCs. Certain RPCs allow
overriding these (`getnewaddress` and `getrawchangeaddress`). Supported types
are "legacy" (P2PKH and P2SH-multisig), "p2sh-segwit" (P2SH-P2WPKH and P2SH-P2WSH-multisig),
and "bech32" (P2WPKH and P2WSH-multisig).

A few utility functions are added to the wallet to construct different address type
and to add the necessary entries to the wallet file to be compatible with earlier
versions (see `CWallet::LearnRelatedScripts`, `GetDestinationForKey`,
`GetAllDestinationsForKey`, `CWallet::AddAndGetDestinationForScript`).
2018-01-09 15:35:31 -08:00
Pieter Wuille
cbe197470e [refactor] GetAccount{PubKey,Address} -> GetAccountDestination 2018-01-03 05:42:57 -08:00
Akira Takizawa
595a7bab23 Increment MIT Licence copyright header year on files modified in 2017 2018-01-03 02:26:56 +09:00
Matt Corallo
97d2b09c12 Add helper to wait for validation interface queue to catch up 2017-12-26 11:56:00 -05:00
Wladimir J. van der Laan
c66adb286a
Merge #10839: Don't use pass by reference to const for cheaply-copied types (bool, char, etc.)
99ba0c3 Don't use pass by reference to const for cheaply-copied types (bool, char, etc.). (practicalswift)

Pull request description:

  Don't use pass by reference to const for cheaply-copied types (`bool`, `char`, etc.).

Tree-SHA512: ccad5e2695dff0b3d6de3e713ff3448f2981168cdac72d73bee10ad346b9919d8d4d588933369e54657a244b8b222fa0bef919bc56d983e1fa64b2004e51b225
2017-12-14 18:28:01 +01:00
Wladimir J. van der Laan
2ae58d5bfb
Merge #11864: Make CWallet::FundTransaction atomic
03a5dc9 [wallet] Make CWallet::FundTransaction atomic (João Barbosa)
95d4450 [wallet] Tidy up CWallet::FundTransaction (João Barbosa)

Pull request description:

  This PR fixes a race for `setLockedCoins` when `lockUnspents` is true. For instance, it should not be possible to use the same unspent in concurrent `fundrawtransaction` calls.

  Now the `cs_main` and `cs_wallet` locks are held during `CreateTransaction` and `LockCoin`(s). Also added some style nits around the change.

Tree-SHA512: ccf383c0c5f6db775655a3e9ccd200c3bd831a83afae2b7c389564c74f7227f5bea86a4775727de2c3603b188f383f8a12d3f9d6d94f7887865c31c94ce95ef6
2017-12-14 10:38:20 +01:00
João Barbosa
03a5dc9c3c [wallet] Make CWallet::FundTransaction atomic 2017-12-14 03:18:29 +00:00
João Barbosa
95d4450a41 [wallet] Tidy up CWallet::FundTransaction 2017-12-14 03:17:58 +00:00
Wladimir J. van der Laan
ef8ba7d73a
Merge #11870: wallet: Remove unnecessary mempool lock in ReacceptWalletTransactions
5b25293 wallet: Remove unnecessary mempool lock in ReacceptWalletTransactions (João Barbosa)

Pull request description:

Tree-SHA512: 13b922c6c9b5ca95a77742050f449366b80bdd7819c34e7ca09af8a4bd68085f4d0c6e0cde119c403f661499f97f2c465071a8047a7d794268e8d2dfe909e6d5
2017-12-12 20:26:55 +01:00
Wladimir J. van der Laan
22149540f9
Merge #11854: Split up key and script metadata for better type safety
9c8eca7 Split up key and script metadata for better type safety (Russell Yanofsky)

Pull request description:

  Suggested by @TheBlueMatt
  https://github.com/bitcoin/bitcoin/pull/11403#discussion_r155599383

  Combining the maps was probably never a good arrangement but is more
  problematic now in presence of WitnessV0ScriptHash and WitnessV0KeyHash types.

Tree-SHA512: 9263e9c01090fb49221e91d88a88241a9691dda3e92d86041c8e284306a64d3af5e2438249f9dcc3e6e4a5c11c1a89f975a86d55690adf95bf2636f15f99f92a
2017-12-12 19:57:43 +01:00
Wladimir J. van der Laan
5d132e8b97
Merge #10574: Remove includes in .cpp files for things the corresponding .h file already included
a720b92 Remove includes in .cpp files for things the corresponding .h file already included (practicalswift)

Pull request description:

  Remove includes in .cpp files for things the corresponding .h file already included.

  Example case:
  * `addrdb.cpp` includes `addrdb.h` and `fs.h`
  * `addrdb.h` includes `fs.h`

  Then remove the direct inclusion of `fs.h` in `addrman.cpp` and rely on the indirect inclusion of `fs.h` via the included `addrdb.h`.

  In line with the header include guideline (see #10575).

Tree-SHA512: 8704b9de3011a4c234db336a39f7d2c139e741cf0f7aef08a5d3e05197e1e18286b863fdab25ae9638af4ff86b3d52e5cab9eed66bfa2476063aa5c79f9b0346
2017-12-12 14:56:25 +01:00
João Barbosa
5b25293436 wallet: Remove unnecessary mempool lock in ReacceptWalletTransactions 2017-12-11 23:43:31 +00:00
Matt Corallo
6ef86c92e7 Do not un-mark fInMempool on wallet txn if ATMP fails.
Irrespective of the failure reason, un-marking fInMempool
out-of-order is incorrect - it should be unmarked when
TransactionRemovedFromMempool fires.
2017-12-11 10:38:16 -05:00
Gregory Sanders
6ba8f30e7b don't attempt mempool entry for wallet transactions on startup if already in mempool 2017-12-11 09:08:54 -05:00
Russell Yanofsky
9c8eca7704 Split up key and script metadata for better type safety
Suggested by Matt Corallo <git@bluematt.me>
https://github.com/bitcoin/bitcoin/pull/11403#discussion_r155599383

Combining the maps was probably never a good arrangement but is more
problematic now in presence of WitnessV0ScriptHash and WitnessV0KeyHash types.
2017-12-08 12:50:46 -05:00
MarcoFalke
fbce66a982
Merge #10493: Use range-based for loops (C++11) when looping over map elements
680bc2cbb Use range-based for loops (C++11) when looping over map elements (practicalswift)

Pull request description:

  Before this commit:

  ```c++
  for (std::map<T1, T2>::iterator x = y.begin(); x != y.end(); ++x) {
      T1 z = (*x).first;
      …
  }
  ```

  After this commit:

  ```c++
  for (auto& x : y) {
      T1 z = x.first;
      …
  }
  ```

Tree-SHA512: 954b136b7f5e6df09f39248a6b530fd9baa9ab59d7c2c7eb369fd4afbb591b7a52c92ee25f87f1745f47b41d6828b7abfd395b43daf84a55b4e6a3d45015e3a0
2017-11-30 17:10:05 -05:00
practicalswift
a720b928c8 Remove includes in .cpp files for things the corresponding .h file already included 2017-11-16 22:26:34 +01:00
MeshCollider
1a445343f6 scripted-diff: Replace #include "" with #include <> (ryanofsky)
-BEGIN VERIFY SCRIPT-
for f in \
  src/*.cpp \
  src/*.h \
  src/bench/*.cpp \
  src/bench/*.h \
  src/compat/*.cpp \
  src/compat/*.h \
  src/consensus/*.cpp \
  src/consensus/*.h \
  src/crypto/*.cpp \
  src/crypto/*.h \
  src/crypto/ctaes/*.h \
  src/policy/*.cpp \
  src/policy/*.h \
  src/primitives/*.cpp \
  src/primitives/*.h \
  src/qt/*.cpp \
  src/qt/*.h \
  src/qt/test/*.cpp \
  src/qt/test/*.h \
  src/rpc/*.cpp \
  src/rpc/*.h \
  src/script/*.cpp \
  src/script/*.h \
  src/support/*.cpp \
  src/support/*.h \
  src/support/allocators/*.h \
  src/test/*.cpp \
  src/test/*.h \
  src/wallet/*.cpp \
  src/wallet/*.h \
  src/wallet/test/*.cpp \
  src/wallet/test/*.h \
  src/zmq/*.cpp \
  src/zmq/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-
2017-11-16 08:23:01 +13:00
Wladimir J. van der Laan
927a1d7d08
Merge #10286: Call wallet notify callbacks in scheduler thread (without cs_main)
89f0312 Remove redundant pwallet nullptr check (Matt Corallo)
c4784b5 Add a dev notes document describing the new wallet RPC blocking (Matt Corallo)
3ea8b75 Give ZMQ consistent order with UpdatedBlockTip on scheduler thread (Matt Corallo)
cb06edf Fix wallet RPC race by waiting for callbacks in sendrawtransaction (Matt Corallo)
e545ded Also call other wallet notify callbacks in scheduler thread (Matt Corallo)
17220d6 Use callbacks to cache whether wallet transactions are in mempool (Matt Corallo)
5d67a78 Add calls to CWallet::BlockUntilSyncedToCurrentChain() in RPCs (Matt Corallo)
5ee3172 Add CWallet::BlockUntilSyncedToCurrentChain() (Matt Corallo)
0b2f42d Add CallFunctionInQueue to wait on validation interface queue drain (Matt Corallo)
2b4b345 Add ability to assert a lock is not held in DEBUG_LOCKORDER (Matt Corallo)
0343676 Call TransactionRemovedFromMempool in the CScheduler thread (Matt Corallo)
a7d3936 Add a CValidationInterface::TransactionRemovedFromMempool (Matt Corallo)

Pull request description:

  Based on #10179, this effectively reverts #9583, regaining most of the original speedups of #7946.

  This concludes the work of #9725, #10178, and #10179.

  See individual commit messages for more information.

Tree-SHA512: eead4809b0a75d1fb33b0765174ff52c972e45040635e38cf3686cef310859c1e6b3c00e7186cbd17374c6ae547bfbd6c1718fe36f26c76ba8a8b052d6ed7bc9
2017-11-15 16:25:40 +01:00
practicalswift
3e09b390b4 Use MakeUnique<T>(...) instead of std::unique_ptr<T>(new T(...)) 2017-11-09 16:53:34 +01:00
Wladimir J. van der Laan
99ec12666b
Merge #11074: Assert that CWallet::SyncMetaData finds oldest transaction.
6c4042a Assert that CWallet::SyncMetaData finds oldest transaction. (Eelis)

Pull request description:

  Without this assert, the Clang static analyzer warns about subsequent dereferencing of copyFrom, because it can't be sure that it's not nullptr. See #9573.

Tree-SHA512: 83cbcb32c52c94fcfefbc90ec7de2011dacd6bdb0da35adc401b8d8dda6a86de2fa0403e2158592268c2cf15eef4f3d887d98c90f1031d4735d5f4bf9dbc1d23
2017-11-09 15:20:18 +01:00
Wladimir J. van der Laan
77ba4bf960
Merge #10368: [wallet] Remove helper conversion operator from wallet
5a5e4e9 [wallet] Remove CTransaction&() helper conversion operator from wallet implementation. (Karl-Johan Alm)

Pull request description:

  The `CTransaction&()` operator in `CMerkleTx` makes conversion into `CTransaction`s transparent, but was marked as to-be-removed in favor of explicitly getting the `tx` ivar, presumably as the operator can lead to ambiguous behavior and makes the code harder to follow.

  This PR removes the operator and adapts callers. This includes some cases of `static_cast<CTransaction>(wtx)` → `*wtx.tx`, which is definitely an improvement.

Tree-SHA512: 95856fec7194d6a79615ea1c322abfcd6bcedf6ffd0cfa89bbdd332ce13035fa52dd4b828d20df673072dde1be64b79c513529a6f422dd5f0961ce722a32d56a
2017-11-09 14:23:13 +01:00
Wladimir J. van der Laan
937613d215
Merge #11467: Fix typos. Use nullptr instead of NULL.
0aacfa4 Remove accidental stray semicolon (practicalswift)
68feb49 Use nullptr instead of NULL (practicalswift)
c6b07fd Fix a vs. an typo (practicalswift)

Pull request description:

  Minor cleanups:
  * Typo: Fix a vs. an typo
  * Typo: Remove accidental stray semicolon (only remaining instance in repo)
  * Correctness/consistency: Use `nullptr` instead of `NULL` (only remaining instance in repo)

Tree-SHA512: 47142e557da9d3fa0b532c46edeb7f356a1f6dc5973e60b0e496badff3581ff696eade542d49da777ac7f2e895129cc8487ccdb1984ff828434fa86f9a56dad0
2017-10-18 17:01:43 +02:00
Matt Corallo
17220d6325 Use callbacks to cache whether wallet transactions are in mempool
This avoid calling out to mempool state during coin selection,
balance calculation, etc. In the next commit we ensure all wallet
callbacks from CValidationInterface happen in the same queue,
serialized with each other. This helps to avoid re-introducing one
of the issues described in #9584 [1] by further disconnecting
wallet from current chain/mempool state.

Thanks to @morcos for the suggestion to do this.

Note that there are several race conditions introduced here:

 * If a user calls sendrawtransaction from RPC, adding a
   transaction which is "trusted" (ie from them) and pays them
   change, it may not be immediately used by coin selection until
   the notification callbacks finish running. No such race is
   introduced in normal transaction-sending RPCs as this case is
   explicitly handled.

 * Until Block{Connected,Disconnected} and
   TransactionAddedToMempool calls also run in the CSceduler
   background thread, there is a race where
   TransactionAddedToMempool might be called after a
   Block{Connected,Disconnected} call happens.

 * Wallet will write a new best chain from the SetBestChain
   callback prior to having processed the transaction from that
   block.

[1] "you could go to select coins, need to use 0-conf change, but
such 0-conf change may have been included in a block who's
callbacks have not yet been processed - resulting in thinking they
are not in mempool and, thus, not selectable."
2017-10-13 19:30:14 -04:00
Matt Corallo
5ee3172636 Add CWallet::BlockUntilSyncedToCurrentChain()
This blocks until the wallet has synced up to the current height.
2017-10-13 19:29:54 -04:00
Jonas Schnelli
8c2de827e9
Merge #7061: [Wallet] Add RPC call "rescanblockchain <startheight> <stopheight>"
7a91ceb5e [QA] Add RPC based rescan test (Jonas Schnelli)
c77170fbd [Wallet] add rescanblockchain <start_height> <stop_height> RPC command (Jonas Schnelli)

Pull request description:

  A RPC rescan command is much more flexible for the following reasons:
  * You can define the start and end-height
  * It can be called during runtime
  * It can work in multiwallet environment

Tree-SHA512: df67177bad6ad1d08e5a621f095564524fa3eb87204c2048ef7265e77013e4b1b29f991708f807002329a507a254f35e79a4ed28a2d18d4b3da7a75d57ce0ea5
2017-10-13 15:23:22 -07:00
Jonas Schnelli
c77170fbdb
[Wallet] add rescanblockchain <start_height> <stop_height> RPC command 2017-10-12 11:59:21 -07:00
Wladimir J. van der Laan
a865b38bf3
Merge #11133: Document assumptions that are being made to avoid division by zero
55509f1 Document assumptions that are being made to avoid division by zero (practicalswift)

Pull request description:

  Document assumptions (via `assert(…)`:s) that are being made to avoid division by zero.

  Rationale:
  * Make it clear to human reviewers and non-human static analyzers that what might look like potential division by zero cases are written the way they are intentionally (these cases are currently flagged by various static analyzers).

Tree-SHA512: bbb67b1370afd8f39bda35f9e3a20f4325f017d94cc1bfac3b0d36c9f34c2d95a9efe11efe44db29fb4aadd25d8276d8f0e03c8806ac64f0d21d821912e13b8e
2017-10-12 13:40:16 +02:00
practicalswift
680bc2cbb3 Use range-based for loops (C++11) when looping over map elements
Before this commit:

  for (std::map<T1, T2>::iterator x = y.begin(); x != y.end(); ++x) {
  }

After this commit:

  for (auto& x : y) {
  }
2017-10-09 21:31:58 +02:00
practicalswift
68feb49105 Use nullptr instead of NULL 2017-10-09 14:26:53 +02:00
Wladimir J. van der Laan
a1f7f18709
Merge #10939: [init] Check non-emptiness of -blocknotify command prior to executing
cffe85f Skip sys::system(...) call in case of empty command (practicalswift)
6fb8f5f Check that -blocknotify command is non-empty before executing (practicalswift)

Pull request description:

  Check that `-blocknotify` command is non-empty before executing.

  To make the `BlockNotifyCallback(...)` (`-blocknotify`) behaviour consistent with that of:
  * `AlertNotify(...)` (`-alertnotify`)
  * `AddToWallet(...)` (`-walletnotify`)

Tree-SHA512: 18272166793a5a8b9cc2a727bfbcea53d38c329a55bc975c02db601329d608a61c20e026ce4b616193ecd3810dca4d3e2cb3bf773898a51872008a8dba96763e
2017-10-04 14:54:09 +02:00
MarcoFalke
e542728cde
Merge #11303: Fix estimatesmartfee rounding display issue
1789e4675 Force explicit double -> int conversion for CFeeRate constructor (Matt Corallo)
53a6590f4 Make float <-> int casts explicit outside of test, qt, CFeeRate (Matt Corallo)
0b1b9148c Remove countMaskInv caching in bench framework (Matt Corallo)

Pull request description:

  This fixes an issue where estimatesmartfee which matches at the min relay fee will return 999 sat/byte instead of 1000 sat/byte due to a float rounding issue. I went ahead and made all float <-> int conversion outside of test/qt explicit (test only had one or two more, Qt had quite a few, including many in the Qt headers themselves) and added overloads to CFeeRate to force callers to do an explicit round themselves. Easy to test with -Wfloat-conversion.

Tree-SHA512: 66087b08e5dfca67506da54ae057c2f9d86184415e8fa4fa0199e38839e06a3ce96c836fcb7593b7d960065f5240c594ff3a0cfa14333ac528421f5aeac835c9
2017-09-30 18:07:55 +02:00
MarcoFalke
93d20a734d
Merge #11309: Minor cleanups for AcceptToMemoryPool
bf64c3cb3 Ignore transactions added to mempool during a reorg for fee estimation purposes. (Alex Morcos)
04f78ab5b Do not reject based on mempool min fee when bypass_limits is set. (Alex Morcos)
fd849e1b0 Change AcceptToMemoryPool function signature (Alex Morcos)

Pull request description:

  First commit just removes default arguments from `AcceptToMemoryPool` and consolidates two arguments, it does not change behavior.

  Second commit finally fixes the fact that we're not meant to reject based on mempool min fee when adding a transaction from a disconnected block during a reorg as mentioned [here](https://github.com/bitcoin/bitcoin/pull/9602#issue-202197849)

  Third commit makes fee estimation ignore transactions added from a disconnected block during a reorg. I think this was another source of fee estimates returning estimates below 1000 sat/kB as in #11303.

Tree-SHA512: 30925ca8b341915bb214f1d2590b36b7931f2e125b7660150e38ae70338f00db5aa7f1608546dddb181446924177eb7cf62ea8bd2583068acc074d6c3f86bc0c
2017-09-29 15:07:57 +02:00
Wladimir J. van der Laan
aa624b61c9
Merge #11167: Full BIP173 (Bech32) support
8213838 [Qt] tolerate BIP173/bech32 addresses during input validation (Jonas Schnelli)
06eaca6 [RPC] Wallet: test importing of native witness scripts (NicolasDorier)
fd0041a Use BIP173 addresses in segwit.py test (Pieter Wuille)
e278f12 Support BIP173 in addwitnessaddress (Pieter Wuille)
c091b99 Implement BIP173 addresses and tests (Pieter Wuille)
bd355b8 Add regtest testing to base58_tests (Pieter Wuille)
6565c55 Convert base58_tests from type/payload to scriptPubKey comparison (Pieter Wuille)
8fd2267 Import Bech32 C++ reference code & tests (Pieter Wuille)
1e46ebd Implement {Encode,Decode}Destination without CBitcoinAddress (Pieter Wuille)

Pull request description:

  Builds on top of #11117.

  This adds support for:
  * Creating BIP173 addresses for testing (through `addwitnessaddress`, though by default it still produces P2SH versions)
  * Sending to BIP173 addresses (including non-v0 ones)
  * Analysing BIP173 addresses (through `validateaddress`)

  It includes a reformatted version of the [C++ Bech32 reference code](https://github.com/sipa/bech32/tree/master/ref/c%2B%2B) and an independent implementation of the address encoding/decoding logic (integrated with CTxDestination). All BIP173 test vectors are included.

  Not included (and intended for other PRs):
  * Full wallet support for SegWit (which would include automatically adding witness scripts to the wallet during automatic keypool topup, SegWit change outputs, ...) [see #11403]
  * Splitting base58.cpp and tests/base58_tests.cpp up into base58-specific code, and "address encoding"-code [see #11372]
  * Error locating in UI for BIP173 addresses.

Tree-SHA512: 238031185fd07f3ac873c586043970cc2db91bf7735c3c168cb33a3db39a7bda81d4891b649685bb17ef90dc63af0328e7705d8cd3e8dafd6c4d3c08fb230341
2017-09-29 10:18:45 +02:00
Pieter Wuille
c091b99379 Implement BIP173 addresses and tests 2017-09-28 16:24:30 -07:00
Wladimir J. van der Laan
dc597bb895
Merge #11225: wallet: update stored witness in AddToWallet
d01a968 wallet: update stored witness in AddToWallet (Suhas Daftuar)

Pull request description:

  Replace witness-stripped wallet transactions with full transactions;
  this can happen when upgrading from a pre-segwit wallet to a segwit-
  aware wallet.

Tree-SHA512: a348b16b38ae738fa75cf7d3ff50ebd0d0071d5d6061c9a10dc3325fc34f6bc96a67aea21fde460ca20f6178768ee0af04d6d8785b35647f436a9083c4270b07
2017-09-26 16:24:28 +02:00
Wladimir J. van der Laan
551d7bf604
Merge #11132: Document assumptions that are being made to avoid NULL pointer dereferences
fdc3293 Document assumptions that are being made to avoid NULL pointer dereferences (practicalswift)

Pull request description:

  Document assumptions (via `assert(…)`:s) that are being made avoid `NULL` pointer dereferences.

  Rationale:
  * Make it clear to human reviewers and non-human static analyzers that what might look like potential `NULL` pointer dereferences are written the way they are intentionally (these cases are currently flagged by various static analyzers).

Tree-SHA512: b424328195e2680e1e4ec546298f718c49e5ad182147dc004de580693db1b50eec4065e1c4f232bdb302baa12954265a50ba21cb5ba4ff30248535b2de778672
2017-09-20 19:07:40 +02:00
Alex Morcos
fd849e1b03 Change AcceptToMemoryPool function signature
Combine fLimitFree and fOverrideMempoolLimit into a single boolean:
bypass_limits.  This is used to indicate that mempool limiting based on feerate
should be bypassed.  It is used when readding transactions from a reorg and then
the mempool is trimmed to size after all transactions are added and they can be
evaluated in the context of their descendants. No changes to behavior.
2017-09-12 12:30:26 -04:00
MarcoFalke
fadf31ef02 wallet: Display non-HD error on first run 2017-09-12 12:40:41 +02:00
Matt Corallo
53a6590f49 Make float <-> int casts explicit outside of test, qt, CFeeRate 2017-09-11 15:51:37 -04:00
Wladimir J. van der Laan
c22a53cd63
Merge #11250: Bump wallet version to 159900 and remove the usehd option
713a92073 Remove usehd option and warn when it is used (Andrew Chow)
d4c18f733 Bump wallet version number to 159900 (Andrew Chow)

Pull request description:

  Bump the wallet version number to 159900 so that new wallets made without a default key will no longer work on previous versions at all. Also remove the `usehd` option to avoid weird interaction with wallet version numbers and HD-ness of wallets.

Tree-SHA512: dd7965505bfad6a926c79afd423236f509229a398a8398076f8d57d90a5974243f9459a61225c4daee560c796f427445c9e55a3ad528a3a97a9123ca6a1269ab
2017-09-08 02:28:12 +02:00
Andrew Chow
713a92073b Remove usehd option and warn when it is used
Removed the -usehd option so wallets cannot be made to be non-hd
anymore. A warning will be displayed when the option is set.
2017-09-07 16:37:44 -07:00
Andrew Chow
d4c18f7330 Bump wallet version number to 159900 2017-09-07 16:37:07 -07:00
Wladimir J. van der Laan
e6ab88a452
Merge #10916: add missing lock to crypter GetKeys()
fe09b0197 add missing lock to crypter GetKeys() (Marko Bencun)
5cb3da04b keystore GetKeys(): return result instead of writing to reference (Marko Bencun)

Pull request description:

  Issue: #10905

  First commit makes GetKeys() return the result instead of writing to a reference to remove some useless lines.

Tree-SHA512: bb51255b5a6cf5488c3d5dee89f539d41f0717f018441d120047f877e0a705a133fb3b7a97d1cf8f73b5d2ed93dd2dbdfcd6f394e40105af2a12e01d397cb402
2017-09-07 22:04:18 +02:00
Pieter Wuille
5c8ff0d448 Introduce wrappers around CBitcoinAddress
This patch removes the need for the intermediary Base58 type
CBitcoinAddress, by providing {Encode,Decode,IsValid}Destination
function that directly operate on the conversion between strings
and CTxDestination.
2017-09-06 13:07:45 -07:00
practicalswift
fdc329376c Document assumptions that are being made to avoid NULL pointer dereferences 2017-09-06 10:45:37 +02:00
Suhas Daftuar
d01a9682b1 wallet: update stored witness in AddToWallet
Replace witness-stripped wallet transactions with full transactions;
this can happen when upgrading from a pre-segwit wallet to a segwit-
aware wallet.
2017-09-03 08:57:17 -04:00
Wladimir J. van der Laan
df91e11ae1
Merge #11126: Acquire cs_main lock before cs_wallet during wallet initialization
de9a1db Acquire cs_main lock before cs_wallet during wallet initialization (Russell Yanofsky)

Pull request description:

  `CWallet::MarkConflicted` may acquire the `cs_main` lock after `CWalletDB::LoadWallet` acquires the `cs_wallet` lock during wallet initialization. (`CWalletDB::LoadWallet` calls `ReadKeyValue` which calls `CWallet::LoadToWallet` which calls `CWallet::MarkConflicted`). This is the opposite order that `cs_main` and `cs_wallet` locks are acquired in the rest of the code, and so leads to `POTENTIAL DEADLOCK DETECTED` errors if bitcoin is built with `-DDEBUG_LOCKORDER`.

  This commit changes `CWallet::LoadWallet` (which calls `CWalletDB::LoadWallet`) to acquire both locks in the standard order.

  Error was reported by @luke-jr in https://botbot.me/freenode/bitcoin-core-dev/msg/90244330/

Tree-SHA512: 353fe21bc0a4a2828b41876897001a3c414d4b115ee7430925bd391d8bc396fca81661145d00996c1ba1a01516d9acf8b89fb5c3da27092f5f3aa7e37ef26ffa
2017-08-28 10:56:41 +02:00
Wladimir J. van der Laan
9c833f471c
Merge #11145: Fix rounding bug in calculation of minimum change
6af49dd Output a bit more information for fee calculation report. (Alex Morcos)
a54c7b9 Fix rounding errors in calculation of minimum change size (Alex Morcos)

Pull request description:

  Thanks to @juscamarena for reporting this.

  Please backport to 0.15.

  There was a potential rounding error where the fee for the change added to the fee for the original tx could be less than the fee for the tx including change.

  This is fixed in the first commit.  The second commit adds one more snippet of information in the fee calculation report.  I actually realized that there is more information that would be nice to report, but we can add that post 0.15.

  An open question is whether we should be returning failure if the test in line 2885 is hit or just resetting pick_new_inputs and continuing.  Originally I made it a failure to avoid any possible infinite loops.  But the case hit here is an example of where that logic possibly backfired.

Tree-SHA512: efe049781acc1f6a8ad429a689359ac6f7b7c44cdfc9578a866dff4a2f6596e8de474a89d25c704f31ef4f8c89af770e98b75ef06c25419d5a6dfc87247bf274
2017-08-28 10:06:00 +02:00
practicalswift
55509f1a11 Document assumptions that are being made to avoid division by zero 2017-08-28 09:20:50 +02:00
Alex Morcos
6af49dddea Output a bit more information for fee calculation report. 2017-08-25 14:59:36 -05:00
Alex Morcos
a54c7b94f8 Fix rounding errors in calculation of minimum change size 2017-08-25 14:59:35 -05:00
Wladimir J. van der Laan
07c92b98e2
Merge #10976: [MOVEONLY] Move some static functions out of wallet.h/cpp
f01103c MOVEONLY: Init functions wallet/wallet.cpp -> wallet/init.cpp (Russell Yanofsky)
e7fe320 MOVEONLY: Fee functions wallet/wallet.cpp -> wallet/fees.cpp (Russell Yanofsky)
d97fe20 Move some static functions out of wallet.h/cpp (Russell Yanofsky)

Pull request description:

  This just moves some static wallet fee and init functions out of `wallet/wallet.cpp` and into new `wallet/fees.cpp` and `wallet/init.cpp` source files. There is one commit updating declarations and callers, followed by two MOVEONLY commits actually moving the function bodies.

  This change is desirable because wallet.h/cpp are monolithic and hard to navigate, so pulling things out and grouping together pieces of related functionality should improve the organization.

  Another motivation is the wallet process separation work in https://github.com/bitcoin/bitcoin/pull/10973, where (at least initially) parameter parsing and fee estimation are still done in the main process rather than the wallet process, and having functions that run in different processes scrambled up throughout wallet.cpp is unnecessarily confusing.

Tree-SHA512: 6e6982ff82b2ab4e681c043907e2b1801ceb9513394730070f16c46ad338278a863f5b3759aa13db76a259b268b1c919c81f4e339f0796a3cfb990161e8c316d
2017-08-25 21:30:42 +02:00
Russell Yanofsky
de9a1db2ed Acquire cs_main lock before cs_wallet during wallet initialization
CWallet::MarkConflicted may acquire the cs_main lock after
CWalletDB::LoadWallet acquires the cs_wallet lock during wallet initialization.
(CWalletDB::LoadWallet calls ReadKeyValue which calls CWallet::LoadToWallet
which calls CWallet::MarkConflicted). This is the opposite order that cs_main
and cs_wallet locks are acquired in the rest of the code, and so leads to
POTENTIAL DEADLOCK DETECTED errors if bitcoin is built with -DDEBUG_LOCKORDER.

This commit changes CWallet::LoadWallet (which calls CWalletDB::LoadWallet) to
acquire both locks in the standard order. It also fixes some tests that were
acquiring wallet and main locks out of order and failed with the new locking in
CWallet::LoadWallet.

Error was reported by Luke Dashjr <luke-jr@utopios.org> in
https://botbot.me/freenode/bitcoin-core-dev/msg/90244330/
2017-08-24 14:12:21 -04:00
Wladimir J. van der Laan
fc5c237d4a
Merge #11007: wallet: Fix potential memory leak when loading a corrupted wallet file
c06755f wallet: Fix memory leak when loading a corrupted wallet file (practicalswift)

Pull request description:

  Fix potential memory leak when loading a corrupted wallet file.

Tree-SHA512: 4b836e4ee1fe4267213bb126af0c1174f964ff015fbe28d0a7e679eab877c275769906b3c08f885763958f6a9b559e1b5e6c7bff1df340bf2dfa2acd57500818
2017-08-22 09:42:43 +02:00
Wladimir J. van der Laan
262167393d
Merge #10952: [wallet] Remove vchDefaultKey and have better first run detection
e53615b Remove vchDefaultKey and have better first run detection (Andrew Chow)

Pull request description:

  Removes vchDefaultKey which was only used for first run detection. Improves wallet first run detection by checking to see if any keys were read from the database.

  This also fixes a (rather contrived) case where an encrypted non-HD wallet has corruption such that the default key is no longer valid and is loaded into a Core version that supports HD wallets. This causes a runtime exception since a new hd master key is generated as the software believes the wallet file is newly created but cannot add the generated key to the wallet since it is encrypted. I was only able to replicate this error by creating a non-hd wallet, encrypting it, then editing the wallet using `db_dump` and `db_load` before loading the wallet with hd enabled. This problem has been reported by [two](https://bitcointalk.org/index.php?topic=1993244.0) [users](https://bitcointalk.org/index.php?topic=1746976.msg17511261#msg17511261) so it is something that can happen, although that raises the question of "what corrupted the default key".

  ~P.S. I don't know what's up with the whitespace changes. I think my text editor is doing something stupid but I don't think those are important enough to attempt undoing them.~ Undid those

Tree-SHA512: 63b485f356566e8ffa033ad9b7101f7f6b56372b29ec2a43b947b0eeb1ada4c2cfe24740515d013aedd5f51aa1890dfbe499d2c5c062fc1b5d272324728a7d55
2017-08-18 18:56:49 +02:00
Wladimir J. van der Laan
0e5b7486cb
Merge #11044: [wallet] Keypool topup cleanups
67ceff4 [wallet] Add logging to MarkReserveKeysAsUsed (John Newbery)
1221f60 [wallet] Remove keypool_topup_cleanups (John Newbery)

Pull request description:

  A couple of minor cleanups suggested by @ryanofsky here: https://github.com/bitcoin/bitcoin/pull/11022#pullrequestreview-55598940

  Does not affect functionality. Not required for v0.15.

Tree-SHA512: d8d0698fd26ea49a4157e68669d5511095760c3a1ecfa3f917e3f273efbafb55c51a202d677614216eae3f796b6e8d17506b2ec2d4799a94f18981b396e65eec
2017-08-18 17:27:17 +02:00
Wladimir J. van der Laan
fc51565cbd
Merge #11039: Avoid second mapWallet lookup
8f2f1e0 wallet: Avoid second mapWallet lookup (João Barbosa)

Pull request description:

  All calls to `mapWallet.count()` have the intent to detect if a `txid` exists and most are followed by a second lookup to retrieve the `CWalletTx`.

  This PR replaces all `mapWallet.count()` calls with `mapWallet.find()` to avoid the second lookup.

Tree-SHA512: 96b7de7f5520ebf789a1aec1949a4e9c74e13683869cee012f717e5be8e51097d068e2347a36e89097c9a89f1ed1a1529db71760dac9b572e36a3e9ac1155f29
2017-08-18 16:25:59 +02:00
Eelis
6c4042a5d0 Assert that CWallet::SyncMetaData finds oldest transaction.
This fixes one of the Clang static analyzer warnings mentioned in #9573.
2017-08-17 15:30:37 +02:00
John Newbery
67ceff4039 [wallet] Add logging to MarkReserveKeysAsUsed 2017-08-16 17:23:20 -04:00
John Newbery
1221f60c94 [wallet] Remove keypool_topup_cleanups
Unused function. Mostly reverts c25d90f125

c25d90f... was merged as part of PR 11022 but is not required.
2017-08-16 17:22:49 -04:00
practicalswift
36d326e8b0 Use nullptr instead of zero (0) as the null pointer constant 2017-08-16 10:24:18 +02:00
MarcoFalke
4fb2586661
Merge #10956: Fix typos
9d5e98ff8 Fix typos. (practicalswift)

Pull request description:

  Fix some typos not covered by #10705.

Tree-SHA512: f06e9541f6ae13ef5d6731399b61795997b21a8816abeb1749c93e99a5c47354e6cbd4a3d145f4dc6ef8a13db179799a3121ecbb7288abf3e8d81cdf81500d37
2017-08-16 00:37:18 +02:00
Andrew Chow
e53615b443 Remove vchDefaultKey and have better first run detection
Removes vchDefaultKey which was only used for first run detection.
Improves wallet first run detection by checking to see if any keys
were read from the database.

This will now also check for a valid defaultkey for backwards
compatibility reasons and to check for any corruption.

Keys will stil be generated on the first one, but there won't be
any shown in the address book as was previously done.
2017-08-15 15:05:53 -07:00
João Barbosa
8f2f1e0458 wallet: Avoid second mapWallet lookup 2017-08-14 23:06:06 +01:00
practicalswift
c06755f5cf wallet: Fix memory leak when loading a corrupted wallet file 2017-08-14 17:54:35 +02:00
practicalswift
6fb8f5f17c Check that -blocknotify command is non-empty before executing
To make BlockNotifyCallback(...) (-blocknotify) consistent with:
* AlertNotify(...) (-alertnotify)
* AddToWallet(...) (-walletnotify)
2017-08-14 17:50:31 +02:00
practicalswift
99ba0c3008 Don't use pass by reference to const for cheaply-copied types (bool, char, etc.). 2017-08-14 17:46:15 +02:00
Russell Yanofsky
f01103c1e0 MOVEONLY: Init functions wallet/wallet.cpp -> wallet/init.cpp 2017-08-14 11:19:38 -04:00
Russell Yanofsky
e7fe3208a8 MOVEONLY: Fee functions wallet/wallet.cpp -> wallet/fees.cpp 2017-08-14 11:19:38 -04:00
Russell Yanofsky
d97fe2016c Move some static functions out of wallet.h/cpp
This commit just moves a few function declarations and updates callers.
Function bodies are moved in two followup MOVEONLY commits.

This change is desirable because wallet.h/cpp are monolithic and hard to
navigate, so pulling things out and grouping together pieces of related
functionality should improve the organization.

Another proximate motivation is the wallet process separation work in
https://github.com/bitcoin/bitcoin/pull/10973, where (at least initially)
parameter parsing and fee estimation are still done in the main process rather
than the wallet process, and having functions that run in different processes
scrambled up throughout wallet.cpp is unnecessarily confusing.
2017-08-14 11:19:38 -04:00
Marko Bencun
bb81e17355 scripted-diff: stop using the gArgs wrappers
They were temporary additions to ease the transition.

-BEGIN VERIFY SCRIPT-
find src/ -name "*.cpp" ! -wholename "src/util.h" ! -wholename "src/util.cpp" | xargs perl -i -pe 's/(?<!\.)(ParseParameters|ReadConfigFile|IsArgSet|(Soft|Force)?(Get|Set)(|Bool|)Arg(s)?)\(/gArgs.\1(/g'
-END VERIFY SCRIPT-
2017-08-14 17:02:10 +02:00
Wladimir J. van der Laan
ce74799a3c
Merge #10483: scripted-diff: Use the C++11 keyword nullptr to denote the pointer literal instead of the macro NULL
90d4d89 scripted-diff: Use the C++11 keyword nullptr to denote the pointer literal instead of the macro NULL (practicalswift)

Pull request description:

  Since C++11 the macro `NULL` may be:
  * an integer literal with value zero, or
  * a prvalue of type `std::nullptr_t`

  By using the C++11 keyword `nullptr` we are guaranteed a prvalue of type `std::nullptr_t`.

  For a more thorough discussion, see "A name for the null pointer: nullptr" (Sutter &
  Stroustrup), http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2431.pdf

  With this patch applied there are no `NULL` macro usages left in the repo:

  ```
  $ git grep NULL -- "*.cpp" "*.h" | egrep -v '(/univalue/|/secp256k1/|/leveldb/|_NULL|NULLDUMMY|torcontrol.*NULL|NULL cert)' | wc -l
  0
  ```

  The road towards `nullptr` (C++11) is split into two PRs:
  * `NULL` → `nullptr` is handled in PR #10483 (scripted, this PR)
  * `0` → `nullptr` is handled in PR #10645 (manual)

Tree-SHA512: 3c395d66f2ad724a8e6fed74b93634de8bfc0c0eafac94e64e5194c939499fefd6e68f047de3083ad0b4eff37df9a8a3a76349aa17d55eabbd8e0412f140a297
2017-08-14 16:30:59 +02:00
Wladimir J. van der Laan
653a46dd91
Merge #11022: Basic keypool topup
d34957e [wallet] [tests] Add keypool topup functional test (Jonas Schnelli)
095142d [wallet] keypool mark-used and topup (John Newbery)
c25d90f [wallet] Add HasUnusedKeys() helper (John Newbery)
f2123e3 [wallet] Cache keyid -> keypool id mappings (John Newbery)
83f1ec3 [wallet] Don't hold cs_LastBlockFile while calling setBestChain (John Newbery)
2376bfc [wallet] [moveonly] Move LoadKeyPool to cpp (Matt Corallo)
cab8557 [wallet] [moveonly] Move CAffectedKeysVisitor (Jonas Schnelli)

Pull request description:

  This PR contains the first part of #10882 :

  - if a key from the keypool is used, mark all keys up to that key as used, and then try to top up the keypool
  - top up the keypool on startup

  Notably, it does not stop the node or prevent the best block from advancing if the keypool drops below a threshold (which means that transactions may be missed and funds lost if restoring from an old HD wallet backup).

Tree-SHA512: ac681fefeaf7ec2aab2fa1da93d12273ea80bd05eb48d7b3b551ea6e5d975dd97ba7de52b7fba52993823280ac4079cc36cf78a27dac708107ebf8fb6326142b
2017-08-14 16:08:44 +02:00
John Newbery
095142d1f9 [wallet] keypool mark-used and topup
This commit adds basic keypool mark-used and topup:

- try to topup the keypool on initial load
- if a key in the keypool is used, mark all keys before that as used and
try to top up
2017-08-10 13:44:02 -04:00
practicalswift
9d5e98ff80 Fix typos. 2017-08-08 17:12:57 +02:00
practicalswift
90d4d89230 scripted-diff: Use the C++11 keyword nullptr to denote the pointer literal instead of the macro NULL
-BEGIN VERIFY SCRIPT-
sed -i 's/\<NULL\>/nullptr/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h src/qt/*/*.cpp src/qt/*/*.h src/wallet/*/*.cpp src/wallet/*/*.h src/support/allocators/*.h
sed -i 's/Prefer nullptr, otherwise SAFECOOKIE./Prefer NULL, otherwise SAFECOOKIE./g' src/torcontrol.cpp
sed -i 's/tor: Using nullptr authentication/tor: Using NULL authentication/g' src/torcontrol.cpp
sed -i 's/METHODS=nullptr/METHODS=NULL/g' src/test/torcontrol_tests.cpp src/torcontrol.cpp
sed -i 's/nullptr certificates/NULL certificates/g' src/qt/paymentserver.cpp
sed -i 's/"nullptr"/"NULL"/g' src/torcontrol.cpp src/test/torcontrol_tests.cpp
-END VERIFY SCRIPT-
2017-08-07 07:36:37 +02:00
Matt Corallo
01699fb283 Fix resendwallettransactions assert failure if -walletbroadcast=0 2017-08-06 21:40:56 -04:00
John Newbery
c25d90f125 [wallet] Add HasUnusedKeys() helper 2017-08-04 16:42:53 -04:00
John Newbery
f2123e3a7b [wallet] Cache keyid -> keypool id mappings 2017-08-04 11:08:39 -04:00
Wladimir J. van der Laan
2e857bb619
Merge #10942: Eliminate fee overpaying edge case when subtracting fee from recipients
49d903e Eliminate fee overpaying edge case when subtracting fee from recipients (Alex Morcos)

Pull request description:

  I'm not sure if this is the cause of the issue in #10034 , but this was a known edge case.  I just didn't realize how simple the fix is.

  Could use a couple more eyes to make sure nothing silly can go wrong here, but if we all agree it's this simple, we can add this as another 0.15 bug fix.

Tree-SHA512: db1dd1e83363a3c231267b626d3a388893ee70ba1972056fe2c339c5c9e4fbfd30f7fe837c30cc7be884d454797fd4c619b9d631a8d5eeb55cdb07402a83acb3
2017-08-03 12:26:12 +02:00
Matt Corallo
2376bfcf24 [wallet] [moveonly] Move LoadKeyPool to cpp 2017-08-02 17:31:35 -04:00
Jonas Schnelli
cab8557e35 [wallet] [moveonly] Move CAffectedKeysVisitor 2017-08-02 17:31:35 -04:00
Alex Morcos
bdd5543ad6 Clarify help message for -discardfee 2017-07-28 09:54:59 -04:00
John Newbery
d84e78ec39 [wallet] Specify wallet name in wallet loading errors 2017-07-28 11:25:38 +01:00
João Barbosa
a6da027d83 Reject invalid wallet files 2017-07-28 11:23:43 +01:00
João Barbosa
3ef77a0c12 Reject duplicate wallet filenames 2017-07-28 11:23:42 +01:00
Alex Morcos
49d903e696 Eliminate fee overpaying edge case when subtracting fee from recipients 2017-07-27 22:04:17 -04:00
Wladimir J. van der Laan
04d395e832
Merge #10854: Avoid using sizes on non-fixed-width types to derive protocol constants.
095b917 Avoid using sizes on non-fixed-width types to derive protocol constants. (Gregory Maxwell)

Pull request description:

  Thanks to awemany for pointing this out.

  This replaces #10172 which appears to be abandoned, but uses the constants as requested on that PR.

Tree-SHA512: 032c0d75b3aaf807a7d0c7fb8ff5515acc45ad58bd00fe81413f900fe02bad900534a970403b9bb568e132c9eddea6043e958daf625e8acc84375bd41ee2e2ef
2017-07-26 08:48:15 +02:00
Marko Bencun
5cb3da04b8 keystore GetKeys(): return result instead of writing to reference
Issue: #10905

By returning the result, a few useless lines can be removed.

Return-value-optimization means there should be no copy.
2017-07-23 23:38:52 +02:00
Wladimir J. van der Laan
fd2814ef11
Merge #10862: Remove unused variable int64_t nEnd. Fix typo: "conditon" → "condition".
5a6671c Fix typo: "conditon" → "condition" (practicalswift)
35aff43 Remove unused variable int64_t nEnd (practicalswift)

Pull request description:

  * Remove unused variable `int64_t nEnd`. Last use of `nEnd` removed in commit 1fc8c3d.
  * Fix typo: "conditon" → "condition". Typo introduced in commit 439c4e8.

Tree-SHA512: 61624e6f70828c485fe46dbe00df76f1a07b7a5849d41bf7d279323b687420e60e9b85192f611a37211f17f3dea8eb3f6f6dc65d90c92e5516404fd81d37785a
2017-07-20 16:56:45 +02:00
Wladimir J. van der Laan
9022aa3722
Merge #10817: Redefine Dust and add a discard_rate
f4d00e6 Add a discard_rate (Alex Morcos)
b138585 Remove factor of 3 from definition of dust. (Alex Morcos)

Pull request description:

  The definition of dust is redefined to remove the factor of 3.

  Dust is redefined to be the value of an output such that it would
  cost that value in fees to (create and) spend the output at the dust
  relay rate.  The previous definition was that it would cost 1/3 of the
  value.  The default dust relay rate is correspondingly increased to
  3000 sat/kB so the actual default dust output value of 546 satoshis
  for a non-segwit output remains unchanged.  This commit is a refactor
  only unless a dustrelayfee is passed on the commandline in which case
  that number now needs to be increased by a factor of 3 to get the same
  behavior.  -dustrelayfee is a hidden command line option.

  Note: It's not exactly a refactor due to edge case changes in rounding
  as evidenced by the required change to the unit test.

  A discard_rate is added which defaults to 10,000 sat/kB

  Any change output which would be dust at the discard_rate you are
  willing to discard completely and add to fee (as well as continuing to
  pay the fee that would have been needed for creating the change)

  This would be a nice addition for 0.15 and I think will remain useful for 0.16 with the new coin selection algorithms in discussion, but its not crucial.

  It does add translation strings, but we could (should?) avoid that by hiding the option

Tree-SHA512: 5b6f655354d0ab6b8b6cac1e8d1fe3136d10beb15c6d948fb15bfb105155a9d03684c6240624039b3eed6428b7e60e54216cc8b2f90c4600701e39f646284a9b
2017-07-19 16:46:49 +02:00
Matt Corallo
cf82a9e704 Do not allow users to get keys from keypool without reserving them
fundrawtransaction allows users to add a change output and then
not have it removed from keypool. While it would be nice to have
users follow the normal CreateTransaction/CommitTransaction process
we use internally, there isnt much benefit in exposing this option,
especially with HD wallets, while there is ample room for users to
misunderstand or misuse this option.

This could be particularly nasty in some use-cases (especially
pre-HD-split) - eg a user might fundrawtransaction, then call
getnewaddress, hand out the address for someone to pay them, then
sendrawtransaction. This may result in the user thinking they have
received payment, even though it was really just their own change!

This could obviously result in needless key-reuse.
2017-07-18 11:20:47 -04:00
practicalswift
35aff43217 Remove unused variable int64_t nEnd
Last use of nEnd removed in commit 1fc8c3de0c.
2017-07-18 14:02:15 +02:00
Alex Morcos
f4d00e63f7 Add a discard_rate
Any change output which would be dust at the discard_rate you are
willing to discard completely and add to fee (as well as continuing to
pay the fee that would have been needed for creating the change).
2017-07-17 13:40:30 -04:00
Gregory Maxwell
095b917464 Avoid using sizes on non-fixed-width types to derive protocol constants.
Thanks to awemany for pointing this out.
2017-07-17 17:00:00 +00:00
Matt Corallo
1fc8c3de0c No longer ever reuse keypool indexes
This fixes an issue where you could reserve a keypool entry, then
top up the keypool, writing out a new key at the given index, then
return they key from the pool. This isnt likely to cause issues,
but given there is no reason to ever re-use keypool indexes
(they're 64 bits...), best to avoid it alltogether.
2017-07-17 12:12:48 -04:00
Wladimir J. van der Laan
0b019357ff
Merge #10831: Batch flushing operations to the walletdb during top up and increase keypool size.
b0e8e2d Print one log message per keypool top-up, not one per key. (Gregory Maxwell)
41dc163 Increase wallet default keypool size to 1000. (Gregory Maxwell)
30d8f3a Pushdown walletdb though CWallet::AddKeyPubKey to avoid flushes. (Gregory Maxwell)
3a53f19 Pushdown walletdb object through GenerateNewKey/DeriveNewChildKey. (Gregory Maxwell)

Pull request description:

  This carries the walletdb object from top-up through GenerateNewKey/DeriveNewChildKey/CWallet::AddKeyPubKey, which allows us to avoid the flush on destruction until the top up finishes instead of flushing the wallet for every key.

  This speeds up adding keys by well over 10x on my laptop (actually something like 17x), I wouldn't be surprised if it were an even bigger speedup on spinning rust.

  Then it increases the keypool size to 1000. I would have preferred to use 10,000 but in the case where the user creates a new wallet and then turns on encryption it seems kind of dumb to have >400KB of marked-used born unencrypted keys just laying around.

  (Thanks to Matt for cluesticking me on how to bypass the crypter spaghetti)

Tree-SHA512: 868303de38fce4c3f67d7fe133f765f15435c94b39d252d7450b5fee5c607a3cc2f5e531861a69d8c8877bf130e0ff4c539f97500a6bc0ff6d67e4a42c9385c7
2017-07-17 17:16:12 +02:00
Gregory Maxwell
b0e8e2de84 Print one log message per keypool top-up, not one per key. 2017-07-17 13:46:15 +00:00
Gregory Maxwell
30d8f3a18e Pushdown walletdb though CWallet::AddKeyPubKey to avoid flushes.
This prevents the wallet from being flushed between each and
 every key during top-up.  This results in a >10x speed-up
 for the top-up.
2017-07-17 13:46:07 +00:00
Wladimir J. van der Laan
6859ad2936
Merge #10706: Improve wallet fee logic and fix GUI bugs
11590d3 Properly bound check conf_target in wallet RPC calls (Alex Morcos)
fd29d3d Remove checking of mempool min fee from estimateSmartFee. (Alex Morcos)
2fffaa9 Make QT fee displays use GetMinimumFee instead of estimateSmartFee (Alex Morcos)
1983ca6 Use CoinControl to pass custom fee setting from QT. (Alex Morcos)
03ee701 Refactor to use CoinControl in GetMinimumFee and FeeBumper (Alex Morcos)
ecd81df Make CoinControl a required argument to CreateTransaction (Alex Morcos)

Pull request description:

  This builds on #10589  (first 5 commits from that PR, last 5 commits are new)

  The first couple commits refactor to use the CCoinControl class to pass fee calculation parameters around.

  This allows for fixing the buggy interaction in QT between the global payTxFee which can be modified by the RPC call settxfee or temporarily modified by the QT custom fee settings.  Before these changes the GUI could sometimes send a transaction with a recently set payTxFee and not respect the settings displayed in the GUI.   After these changes, using the GUI does not involve the global transaction confirm target or payTxFee.

  The prospective fee displays in the smart fee slider and the coin control dialog are changed to use the fee calculation from GetMinimumFee, this simplifies the code and makes them slightly more correct in edge cases.

  Maxing the fee calculation with the mempool min fee is move from estimateSmartFee to GetMinimumFee.

  This fixes a long standing bug, and should be tagged for 0.15 as it is holding up finalizing the estimatesmartfee RPC API before release.

Tree-SHA512: 4d36a1bd5934aa62f3806d380fcafbef73e9fe5bdf190fc5259a3e3a13349e5ce796e50e7068c46dc630ccf56d061bce5804f0bfe2e082bb01ca725b63efd4c1
2017-07-17 09:25:33 +02:00
MarcoFalke
bf0a08be28
Merge #10330: [wallet] fix zapwallettxes interaction with persistent mempool
4c3b538 [logs] fix zapwallettxes startup logs (John Newbery)
e7a2181 [wallet] fix zapwallettxes interaction with persistent mempool (John Newbery)
ff7365e [tests] fix flake8 warnings in zapwallettxes.py (John Newbery)

Pull request description:

  zapwallettxes previously did not interact well with persistent mempool.
  zapwallettxes would cause wallet transactions to be zapped, but they
  would then be reloaded from the mempool on startup. This commit softsets
  persistmempool to false if zapwallettxes is enabled so transactions are
  actually zapped.

  This PR also fixes the zapwallettxes.py functional test, which did not properly test this feature. The test line:

  ```py
       assert_raises(JSONRPCException, self.nodes[0].gettransaction, [txid3])
       #there must be a expection because the unconfirmed wallettx0 must be gone by now
  ```
  is not actually testing the presence of the transaction since the RPC is being called incorrectly (with an array instead of a string). The `assert_raises()` passes since an assert is raised, but it's not the one the test writer had in mind!

  Fixes #9710 .

Tree-SHA512: e3236efc7a2fd2b3bf1d9e2e8a7726d470c57f5d95cf41b7bde264edc8817bd36a6f3feff52f8de8db0ef64b7247c88b24e7ff7cefaa706cba86fe4e2135a508
2017-07-17 04:23:52 +02:00
Gregory Maxwell
3a53f19718 Pushdown walletdb object through GenerateNewKey/DeriveNewChildKey.
This is needed but not sufficient for batching the wallet flushing
 when topping up the keypool.
2017-07-16 00:11:31 +00:00
Pieter Wuille
5cfdda2503
Merge #10235: Track keypool entries as internal vs external in memory
d40a72ccb Clarify *(--.end()) iterator semantics in CWallet::TopUpKeyPool (Matt Corallo)
28301b978 Meet code style on lines changed in the previous commit (Matt Corallo)
4a3fc3562 Track keypool entries as internal vs external in memory (Matt Corallo)

Pull request description:

  This is an alternative version of #10184. As @jonasschnelli points out there, the performance regressions are pretty minimal, but given that this is a pretty simple, mechanical change, its probably worth doing.

Tree-SHA512: e83f9ebf2998f8164d1b2eebe5e6dcdeadea8c30b7612861f830758c08bf4093cd6a67b3bcfa9cfcb139e5e0b106fc8898a975fc69f334981aefc756568ab613
2017-07-15 14:02:05 -07:00
John Newbery
4c3b538c61 [logs] fix zapwallettxes startup logs 2017-07-15 15:31:26 -04:00
John Newbery
e7a2181b49 [wallet] fix zapwallettxes interaction with persistent mempool
zapwallettxes previously did not interact well with persistent mempool.
zapwallettxes would cause wallet transactions to be zapped, but they
would then be reloaded from the mempool on startup. This commit softsets
persistmempool to false if zapwallettxes is enabled so transactions are
actually zapped.
2017-07-15 15:15:25 -04:00
Alex Morcos
fd29d3df29 Remove checking of mempool min fee from estimateSmartFee.
This check has been moved to the wallet logic GetMinimumFee. The rpc call to
estimatesmartfee will now no longer return a result maxed with the mempool min
fee, but automated fee calculations from the wallet will produce the same result
as before and coincontrol and sendcoins dialogs in the GUI will correctly
display the right prospective fee.

changes to policy/fees.cpp include a big whitespace indentation change.
2017-07-14 23:41:40 -04:00
Alex Morcos
2fffaa9738 Make QT fee displays use GetMinimumFee instead of estimateSmartFee
Remove helper function (CalculateEstimateType) for determining whether
estimates should be conservative or not, now that this is only called
once from GetMinimumFee and incorporate the logic directly there.
2017-07-14 23:41:37 -04:00
Alex Morcos
03ee701161 Refactor to use CoinControl in GetMinimumFee and FeeBumper
Improve parameter precedence in coin_control
2017-07-14 23:10:58 -04:00
Alex Morcos
ecd81dfa3c Make CoinControl a required argument to CreateTransaction 2017-07-14 23:07:18 -04:00
Matt Corallo
d40a72ccbb Clarify *(--.end()) iterator semantics in CWallet::TopUpKeyPool 2017-07-14 21:25:24 -04:00
Matt Corallo
28301b9780 Meet code style on lines changed in the previous commit 2017-07-14 21:25:24 -04:00
Matt Corallo
4a3fc35629 Track keypool entries as internal vs external in memory
This resolves a super minor performance regressions in several
keypool-handling functions
2017-07-14 21:25:21 -04:00
Alex Morcos
dd97a529ad Properly forbid -salvagewallet and -zapwallettxes for multi wallet. 2017-07-13 13:48:28 -04:00
Alex Morcos
0f402b9263 Fix rare edge case of paying too many fees when transaction has no change.
Due to the iterative process of selecting new coins in each loop a new fee is
calculated that needs to be met each time.  In the typical case if the most
recent iteration of the loop produced a much smaller transaction and we have now
gathered inputs with too many fees, we can just reduce the change.  However in
the case where there is no change output, it is possible to end up with a
transaction which drastically overpays fees.  This commit addresses that case,
by creating a change output if the overpayment is large enough to support it,
this is accomplished by rerunning the transaction creation loop without
selecting new coins.

Thanks to instagibbs for working on this as well
2017-07-11 12:20:41 -04:00
Alex Morcos
253cd7ec4f Only reserve key for scriptChange once in CreateTransaction
This does not affect behavior but allows us to have access to an output to
scriptChange even if we currently do not have change in the transaction.
2017-07-11 12:17:56 -04:00
Wladimir J. van der Laan
104f5f21dc
Merge #10589: More economical fee estimates for RBF and RPC options to control
f135923 Add RPC options for RBF, confirmation target, and conservative fee estimation. (Alex Morcos)
f0bf33d Change default fee estimation mode. (Alex Morcos)
e0738e3 remove default argument from estimateSmartFee (Alex Morcos)
d507c30 Introduce a fee estimate mode. (Alex Morcos)
cfaef69 remove default argument from GetMinimumFee (Alex Morcos)

Tree-SHA512: 49c3a49a6893790a7e8b4e93a48f123dd5307af26c2017800683b76b4df8fc904ba73402917878676242c7440e3e04288d0c1ff3c2c907418724efc03cedab50
2017-07-11 11:58:02 +02:00
Alex Morcos
f0bf33da83 Change default fee estimation mode.
Fee estimates will default to be non-conservative if the transaction in question is opt-in-RBF.
2017-07-06 22:20:23 -04:00
Alex Morcos
d507c301bc Introduce a fee estimate mode.
GetMinimumFee now passes the conservative argument into estimateSmartFee.
Call CalculateEstimateType(mode) before calling GetMinimumFee or estimateSmartFee to determine the value of this argument.
CCoinControl can now be used to control this mode.
2017-07-06 22:20:23 -04:00
Alex Morcos
cfaef69ace remove default argument from GetMinimumFee 2017-07-06 22:20:23 -04:00
darksh1ne
9bbf60047e Use method name from __func__ macro
Use __func__ macro in std::runtime_exception to:
1. fix method name in CWalletTx::GetAvailableWatchOnlyCredit()
2. refactor CWalletTx::GetAvailableCredit()
2017-07-04 22:22:53 +03:00
Wladimir J. van der Laan
bef02fb6ac
Merge #10412: Improve wallet rescan API
deaf48b Handle TIMESTAMP_WINDOW within CWallet::RescanFromTime (Russell Yanofsky)
5b2be2b Make CWallet::RescanFromTime comment less ambiguous (Russell Yanofsky)
9bb66ab Add RescanFromTime method and use from rpcdump (Russell Yanofsky)
ccf84bb Move birthday optimization out of ScanForWalletTransactions (Russell Yanofsky)

Tree-SHA512: cd38433b8f5c5e44ecfba830a6a26bd9a9d0f4a22ae42bce17773d1a6fb25e1ee4289484996dad2d7acfa03059917ff062459f25030a761da7083ba5fbc87bc9
2017-06-24 16:37:42 +02:00
Russell Yanofsky
deaf48b046 Handle TIMESTAMP_WINDOW within CWallet::RescanFromTime
This way CWallet::RescanFromTime callers don't need to subtract
TIMESTAMP_WINDOW themselves.

This is pure refactoring, there is no change in behavior.
2017-06-22 17:16:24 -04:00
Russell Yanofsky
5b2be2b787 Make CWallet::RescanFromTime comment less ambiguous 2017-06-22 17:14:40 -04:00
Wladimir J. van der Laan
6bef7ca8bc
Merge #10633: doc: Fix various typos
0a5a6b9 Fixed multiple typos (Dimitris Tsapakidis)

Tree-SHA512: 57748710bcbc03945b160db5e95bd686a2c64605f25d5e11d8ed9d0e1be3b3bf287a63588dc6eb33d0cef4ff17c765fda7c226d667a357acc539c8fcf2b9bb7e
2017-06-22 20:46:37 +02:00
Wladimir J. van der Laan
209eef60a9
Merge #9343: Don't create change at dust limit
6171826 Don't create change at the dust limit, even if it means paying more than expected (Alex Morcos)

Tree-SHA512: d4cf939b97a92d63d4703ad152093c9dc60e74af590697bc21adbcdf17fc9e6ffb37d46d5eefafbd69327d1bcb79911f1b826ce1d960d895d464c4408c4995c3
2017-06-22 18:23:11 +02:00
Dimitris Tsapakidis
0a5a6b90bc Fixed multiple typos
A few "a->an" and "an->a".
"Shows, if the supplied default SOCKS5 proxy" -> "Shows if the supplied default SOCKS5 proxy". Change made on 3 occurrences.
"without fully understanding the ramification of a command" -> "without fully understanding the ramifications of a command".
Removed duplicate words such as "the the".
2017-06-22 19:18:10 +03:00
Pieter Wuille
d636f3943d
Merge #10536: Remove unreachable or otherwise redundant code
4265bf351 Remove unreachable or otherwise redundant code (practicalswift)

Tree-SHA512: bc9666ab5d20c936d78c50c0361405aca9edd116602aa9bcd71a79a904b647ac9eca0651d1a9d530189a6ac1c4e235bfc69ec1a68f7e36cc14d6848ac2206b7b
2017-06-20 16:16:38 -07:00
Wladimir J. van der Laan
c2ab38bdd5
Merge #10284: Always log debug information for fee calculation in CreateTransaction
1bebfc8 Output Fee Estimation Calculations in CreateTransaction (Alex Morcos)

Tree-SHA512: e25a27f7acbbc3a666d5d85da2554c5aaec4c923ee2fdbcfc532c29c6fbdec3c9e0d6ae6044543ecc339e7bd81df09c8d228e0b53a2c5c2dae0f1098c9453272
2017-06-15 14:20:14 +02:00
Alex Morcos
61718268b5 Don't create change at the dust limit, even if it means paying more than expected 2017-06-14 15:34:19 -04:00
Pieter Wuille
1ad3d4e126
Merge #10502: scripted-diff: Remove BOOST_FOREACH, Q_FOREACH and PAIRTYPE
1238f13cf scripted-diff: Remove PAIRTYPE (Jorge Timón)
18dc3c396 scripted-diff: Remove Q_FOREACH (Jorge Timón)
7c00c2672 scripted-diff: Fully remove BOOST_FOREACH (Jorge Timón)
a5410ac5e Small preparations for Q_FOREACH, PAIRTYPE and #include <boost/foreach.hpp> removal (Jorge Timón)

Tree-SHA512: d3ab4a173366402e7dcef31608977b757d4aa07abbbad2ee1bcbcfa311e994a4552f24e5a55272cb22c2dcf89a4b0495e02e9d9aceae4b08c0bab668f20e324c
2017-06-13 18:05:58 -07:00
Alex Morcos
1bebfc8d3a Output Fee Estimation Calculations in CreateTransaction 2017-06-13 12:35:04 -04:00
practicalswift
4265bf351a Remove unreachable or otherwise redundant code 2017-06-12 15:01:46 +02:00
Wladimir J. van der Laan
177433ad22
Merge #8694: Basic multiwallet support
c237bd7 wallet: Update formatting (Luke Dashjr)
9cbe8c8 wallet: Forbid -salvagewallet, -zapwallettxes, and -upgradewallet with multiple wallets (Luke Dashjr)
a2a5f3f wallet: Base backup filenames on original wallet filename (Luke Dashjr)
b823a4c wallet: Include actual backup filename in recovery warning message (Luke Dashjr)
84dcb45 Bugfix: wallet: Fix warningStr, errorStr argument order (Luke Dashjr)
008c360 Wallet: Move multiwallet sanity checks to CWallet::Verify, and do other checks on all wallets (Luke Dashjr)
0f08575 Wallet: Support loading multiple wallets if -wallet used more than once (Luke Dashjr)
b124cf0 Wallet: Replace pwalletMain with a vector of wallet pointers (Luke Dashjr)
19b3648 CWalletDB: Store the update counter per wallet (Luke Dashjr)
74e8738 Bugfix: ForceSetArg should replace entr(ies) in mapMultiArgs, not append (Luke Dashjr)
23fb9ad wallet: Move nAccountingEntryNumber from static/global to CWallet (Luke Dashjr)
9d15d55 Bugfix: wallet: Increment "update counter" when modifying account stuff (Luke Dashjr)
f28eb80 Bugfix: wallet: Increment "update counter" only after actually making the applicable db changes to avoid potential races (Luke Dashjr)

Tree-SHA512: 23f5dda58477307bc07997010740f1dc729164cdddefd2f9a2c9c7a877111eb1516d3e2ad4f9b104621f0b7f17369c69fcef13d28b85cb6c01d35f09a8845f23
2017-06-12 13:27:17 +02:00
Pieter Wuille
76f268b9bd
Merge #10521: Limit variable scope
90593ed92 Limit variable scope (practicalswift)

Tree-SHA512: 4719e303688a31aefbe1d239e86b21dd3c2045524e08bd628c6ba0c6c2a97de14d04305b9beafe0b1dcde7229793e6663168953f192e88ed409be5c30fd2a9a9
2017-06-09 13:09:24 -07:00
Luke Dashjr
c237bd750e wallet: Update formatting 2017-06-06 21:20:22 +00:00
Luke Dashjr
9cbe8c80ba wallet: Forbid -salvagewallet, -zapwallettxes, and -upgradewallet with multiple wallets 2017-06-06 21:20:22 +00:00
Luke Dashjr
b823a4c9f6 wallet: Include actual backup filename in recovery warning message 2017-06-06 21:17:09 +00:00
Luke Dashjr
008c360083 Wallet: Move multiwallet sanity checks to CWallet::Verify, and do other checks on all wallets 2017-06-06 21:17:09 +00:00
Luke Dashjr
0f08575be2 Wallet: Support loading multiple wallets if -wallet used more than once 2017-06-06 21:17:09 +00:00
Luke Dashjr
b124cf04ea Wallet: Replace pwalletMain with a vector of wallet pointers 2017-06-06 21:17:09 +00:00
Luke Dashjr
b005bf21a7 Introduce MAX_BIP125_RBF_SEQUENCE constant 2017-06-05 23:14:34 +00:00
Luke Dashjr
891c5eeec2 Wallet: Refactor FundTransaction to accept parameters via CCoinControl 2017-06-05 23:12:03 +00:00
Luke Dashjr
19b3648bb5 CWalletDB: Store the update counter per wallet 2017-06-05 22:27:57 +00:00
Luke Dashjr
23fb9adaea wallet: Move nAccountingEntryNumber from static/global to CWallet 2017-06-05 21:04:42 +00:00
Jorge Timón
1238f13cf6
scripted-diff: Remove PAIRTYPE
-BEGIN VERIFY SCRIPT-
sed -i 's/PAIRTYPE(\([^,]*\), \([^\)]*\))/std::pair<\1, \2>/' ./src/*.h ./src/*.cpp ./src/*/*.h ./src/*/*.cpp ./src/*/*/*.h ./src/*/*/*.cpp ;
sed -i ':a;N;$!ba;s/#define std::pair<t1, t2>    std::pair<t1, t2>\n//' ./src/utilstrencodings.h ;
-END VERIFY SCRIPT-
2017-06-05 20:14:53 +02:00
Jorge Timón
7c00c26726
scripted-diff: Fully remove BOOST_FOREACH
-BEGIN VERIFY SCRIPT-
sed -i 's/BOOST_FOREACH *(\(.*\),/for (\1 :/' ./src/*.h ./src/*.cpp ./src/*/*.h ./src/*/*.cpp ./src/*/*/*.h ./src/*/*/*.cpp ;
-END VERIFY SCRIPT-
2017-06-05 20:10:50 +02:00
Russell Yanofsky
9bb66ab660 Add RescanFromTime method and use from rpcdump
No change in behavior.
2017-06-05 09:59:58 -05:00
Russell Yanofsky
ccf84bb9c1 Move birthday optimization out of ScanForWalletTransactions
This change has no effect on wallet behavior.

On wallet startup, the transaction scan avoids reading any blocks with
timestamps older than the wallet birthday (less than nTimeFirstKey -
TIMESTAMP_WINDOW). This block skipping code currently resides in
CWallet::ScanForWalletTransactions but it doesn't really belong there because
it makes the implementation unnecessarily fragile and hard to understand, and
it never has any effect except at startup (because all other callers do their
rescans based on timestamps other than, but always greater or equal to,
nTimeFirstKey).
2017-06-05 09:59:58 -05:00
Wladimir J. van der Laan
08d0390a5f
Merge #10403: Fix importmulti failure to return rescan errors
4d2d604 Fix importmulti failure to return rescan errors (Russell Yanofsky)

Tree-SHA512: e5e6d6c5a2bb7230e1bcac1903a4b766cd57bf781fade50c6c9cd5713cd3e768db0987cfda9699b57a53d3a0a60951b96dce5283b3d3ec1f954162c439bc932b
2017-06-05 16:41:02 +02:00
Wladimir J. van der Laan
0747d33498
Merge #10455: Simplify feebumper minimum fee code slightly
88b8f0b Simplify feebumper minimum fee code slightly (Russell Yanofsky)

Tree-SHA512: 4465daef63936860d016a7fd7fd31ce62878d87cb943a90f321b07a40accdc5e7970d99de46b2bc924eb0b422144d6f01649855949395518790ecd05d300ee52
2017-06-05 16:38:07 +02:00
practicalswift
90593ed92c Limit variable scope 2017-06-05 00:52:36 +02:00
Pieter Wuille
098b01dc58
Merge #10500: Avoid CWalletTx copies in GetAddressBalances and GetAddressGroupings
b9b814a38 Avoid CWalletTx copies in GetAddressBalances and GetAddressGroupings (Russell Yanofsky)

Tree-SHA512: 96a0612ca460ba1bfb3921f3aa348400fd3afa12c40f2ca1f3f04068b1574fe824d577e0123013d8898a4990084316dc1dfb541331849f0996ceff7f4eb25e6b
2017-06-02 16:10:33 -07:00
Russell Yanofsky
b9b814a38e Avoid CWalletTx copies in GetAddressBalances and GetAddressGroupings 2017-06-01 11:48:29 -04:00
Russell Yanofsky
88b8f0b682 Simplify feebumper minimum fee code slightly
No change in behavior. Get rid of specifiedConfirmTarget if/else block and
rename specifiedConfirmTarget and ignoreUserSetFee variables to
ignoreGlobalPayTxFee.
2017-05-25 10:13:28 -04:00
Pieter Wuille
f2f7e97e8c
Merge #10347: Use range-based for loops (C++11) when looping over vector elements
211adc0 Use range-based for loops (C++11) when looping over vector elements (practicalswift)

Tree-SHA512: 0e007f20dcef99d3c7a1036265e00f689d69f42e02fd82dd8389f45b52d31947e5f9388de2610d3d9bd9f554915ce0d35ebce561e5ae3a9013956d0ee4937145
2017-05-23 14:08:20 -07:00
Wladimir J. van der Laan
ce8176d038
Merge #10295: [qt] Move some WalletModel functions into CWallet
108f04f Add missing LOCK2 in CWallet::GetAvailableBalance (Russell Yanofsky)
429aa9e [test] Move some tests from qt -> wallet (Russell Yanofsky)
d944bd7 [qt] Move some WalletModel functions into CWallet (Russell Yanofsky)
ef8ca17 [test] Add tests for some walletmodel functions (Russell Yanofsky)

Tree-SHA512: f6384d9f2ff3f7fb173d414588c3e7dc8c311b8ed2ce2b0979fb824a0ed83a7302890ccd3d83197f07f6fdcb6b1ca151584d90ea1961d88dfe8956c87087cde8
2017-05-23 19:32:52 +02:00