41a46cbb31 Speed up deriveaddresses for large ranges (Pieter Wuille)
Pull request description:
`deriveaddresses` dumps all generated addresses into a single `FlatSigningProvider`, which is also used for looking up information for future derivations. @achow101 points out that the growing data structures may unnecessary increase lookup time for later derivations.
Fix this by separating the provider used for lookups (`key_provider`) and the one we dump things into.
This gives a 10x speedup for a range of 7000 elements, and probably a larger speedup for larger ranges.
ACKs for commit 41a46c:
achow101:
Regardless, I do think this is a good change, so utACK 41a46cbb31
fanquake:
tACK 41a46cb
meshcollider:
utACK 41a46cbb31
Tree-SHA512: a1b894ce9d5195d8f9760f44acc6d67a90bb259283fd8c1524c38a222fe53e8c1d35b6653a508b121b7ad91e155c97d26c658f6bdcebf6c360546931e4a26a22
184f8785f wallet_bumpfee.py: add test for change key preservation (Gregory Sanders)
d08becff8 add functional tests for feerate bumpfee with adding inputs (Gregory Sanders)
0ea47ba7b generalize bumpfee to add inputs when needed (Gregory Sanders)
Pull request description:
When targeting a feerate using `bumpfee`, call a new function that directly uses `CWallet::CreateTransaction` and coin control to get the desired result. This allows us to get a superset of previous behavior, with an arbitrary RBF bump of a transaction provided it passes the preconditional checks and spare confirmed utxos are available.
Note(s):
0) The coin selection will use knapsack solver for the residual selection.
1) This functionality, just like knapsack coin selection in general, will hoover up negative-value inputs when given the chance.
2) Newly added inputs must be confirmed due to current Core policy. See error: `replacement-adds-unconfirmed`
3) Supporting this with `totalFee` is difficult since the "minimum total fee" option in `CreateTransaction` logic was (rightly)taken out in #10390 .
ACKs for commit 184f87:
jnewbery:
utACK 184f8785f7
Tree-SHA512: fb6542bdfb2c6010e328ec475cf9dcbff4eb2b1a1b27f78010214534908987a5635797196fa05edddffcbcf2987335872dc644a99261886d5cbb34a8f262ad3e
fa2dfbf30a travis: Bump second timeout to 33 minutes, Add rationale (MarcoFalke)
Pull request description:
People have been complaining on IRC about timeouts, but I don't think we can bump it much further. The tests take more than 15 minutes in some cases [1], so if there is less than 1000 seconds left to finish them we need to abort and save the cache. 🤷♂️.
[1] https://travis-ci.org/bitcoin/bitcoin/jobs/518788414#L3568
ACKs for commit fa2dfb:
practicalswift:
utACK fa2dfbf30a
fanquake:
utACK fa2dfbf⏰
Tree-SHA512: ae981731e574f34cc54145eec52c4bdee241943ef22a6fbdf3e14b8f36538b0ae4f83815fa6a2a5fe4b15a18a2f0c7898dca8ec435125f32d92fa8c19caf4272
The "addresses" field was confusing because it refered to public keys
using their P2PKH address. It was included in the return object when
needed for backward compatibility. Remove that compatibility now that
the -deprecatedrpc=validateaddress option has been removed.
New applications should use the 'embedded'->'address' field for P2SH or
P2WSH wrapped addresses, and 'pubkeys' for inspecting multisig
participants.
fafe5f0d09 test: Remove unused imports (MarcoFalke)
fa16a09215 scripted-diff: use self.sync_* methods (MarcoFalke)
faf77f9b90 test: Pass self to test_simple_bumpfee_succeeds (MarcoFalke)
fa6dc7c5c3 test: Add BitcoinTestFramework::sync_* methods (MarcoFalke)
fafe008cb4 test: Pass at most one node group to sync_all (MarcoFalke)
fa4680ed09 scripted-diff: Rename sync_blocks to send_blocks to avoid name collisions and confusion (MarcoFalke)
Pull request description:
This adds methods to the test framework that can be called by just `self.sync_*()`.
This avoids having to import the underlying util method. Also, in the default case, where all nodes are synced this avoid having to pass `self.nodes` explicitly.
So the effective changes are:
```diff
@@
-from test_framework.util import sync_blocks, sync_mempools
@@
- sync_blocks(self.nodes)
+ self.sync_blocks()
@@
- sync_mempools(self.nodes)
+ self.sync_mempools()
ACKs for commit fafe5f:
promag:
utACK fafe5f0.
jonatack:
ACK fafe5f0d09, nice simplification.
Tree-SHA512: 5c81840edf9fb3c5de2d7bf95ca36a5a8d23567cd1479a0f4044547c2080e9a3c5cf375357bc8eebb5b68829be050a171ab2512cfd47b89feed51fe3bad2cd72
303372c41a docs: Improve netaddress comments (Carl Dong)
Pull request description:
Improves comments for `netaddress`, making them available to Doxygen.
I think this is worthwhile because a lot of the code require some context (e.g., A lot of the things that we do to fit hostnames and tor addresses into `CNetAddr` is non-obvious, and documenting it is beneficial).
ACKs for commit 303372:
Tree-SHA512: 2a35784a01ed8ec5fdbe111a540192d31bde16afa96e4be97b0385daf290fc7469a66d7cb8905a70b920fad6a0e7400ca4e5da082d6e4af1d1aaccc0e8297720
0b3a65455a Avoid redefine warning (Peter Bushnell)
Pull request description:
Wrap preprocessor definition of NOMINMAX in ifndef conditional to suppress warning when cross compiling Windows.
`fs.cpp:6:0: warning: "NOMINMAX" redefined`
`/usr/lib/gcc/x86_64-w64-mingw32/7.3-posix/include/c++/x86_64-w64-mingw32/bits/os_defines.h:45:0: note: this is the location of the previous definition
#define NOMINMAX 1`
#define NOMINMAX was introduced in the following merge.
https://github.com/bitcoin/bitcoin/pull/14426
ACKs for commit 0b3a65:
practicalswift:
utACK 0b3a65455a
promag:
utACK 0b3a654.
Tree-SHA512: 0175195b88e63d3d44ffac2b8cc87ae7b285a45ed4e49605bca0cc82db073006c22024ef9c2f287980d357dac1099f798f1eeaa0bd75bb7a625919dc1632366c
7a9046e48 [wallet] Refactor CWalletTx::RelayWalletTransaction() (John Newbery)
Pull request description:
Refactor `CWalletTx::RelayWalletTransaction()` function.
This was a suggestion from the wallet-node separation PR: https://github.com/bitcoin/bitcoin/pull/15288#discussion_r256036330, which we deferred until after the main PR was merged.
There are also makes two minor behavior changes:
- no longer assert if fBroadcastTransactions is false. Just return false from the function.
- no longer print the relay message if p2pEnabled is set to false (since the transaction is not actually relayed).
ACKs for commit 7a9046:
promag:
utACK 7a9046e48d.
MeshCollider:
utACK 7a9046e48d
ryanofsky:
utACK 7a9046e48d. No changes at all, just rebase after base PR #15632 was merged
Tree-SHA512: 2ae6214cfadd917a1b3a892c4277e5e57c3eb791e17f67511470e6fbc634d19356554b9f9c55af6b779fdef821914aad59b7cc9e6c13ece145df003bf507d486
4d074e84a2 [build] Move AnalyzePSBT from psbt.cpp to node/psbt.cpp (Russell Yanofsky)
fd509bd1f7 [docs] Document src subdirectories and different libraries (John Newbery)
9eaeb7fb8d [build] Move wallet load functions to wallet/load unit (John Newbery)
91a25d1e71 [build] Add several util units (John Newbery)
99517866b6 [build] Move several units into common libraries (John Newbery)
0509465542 [build] Move rpc rawtransaction util functions to rpc/rawtransaction_util.cpp (John Newbery)
1acc61f874 [build] Move rpc utility methods to rpc/util (John Newbery)
4a75c9d651 [build] Move policy settings to new src/policy/settings unit (John Newbery)
fdf8888b6f [build] Move CheckTransaction from lib_server to lib_consensus (John Newbery)
Pull request description:
This is a move-only commit. No code is changing and the moves can be easily verified with:
```sh
git log -p -n1 --color-moved=dimmed_zebra
```
This commit moves functions and variables that wallet code depends on out of libbitcoin_server.a, so the bitcoin-wallet tool can be built without libbitcoin_server.a in #15639, and attempting to access server state from wallet code will result in link errors instead of silently broken code.
List of moves:
- `CheckTransaction` moves from `consensus/tx_verify.cpp` to `consensus/tx_check.cpp`
- `urlDecode` moves from `httpserver.cpp` to `util/url.cpp`
- `TransactionErrorString` moves from `node/transaction.cpp` to `util/error.cpp`
- `StringForFeeReason` and `FeeModeFromString` move from `policy/fees.cpp` to `util/fees.cpp`
- `incrementalRelayFee` `dustRelayFee` and `nBytesPerSigOp` move from `policy/policy.cpp` to `policy/settings.cpp`
- `SignalsOptInRBF` moves from `policy/rbf.cpp` to `util/rbf.cpp`
- `fIsBareMultisigStd` moves from `validation.cpp` to `policy/settings.cpp`
- `ConstructTransaction` `TxInErrorToJSON` and `SignTransaction` move from `rpc/rawtransaction.cpp` to `rpc/rawtransaction_util.cpp`
- `RPCTypeCheck` `RPCTypeCheckArgument` `RPCTypeCheckObj` `AmountFromValue` `ParseHashV``ParseHashO` `ParseHexV` `ParseHexO` `HelpExampleCli` and `HelpExampleRpc` move from `rpc/server.cpp` to `rpc/util.cpp`
- `AmountHighWarn` and `AmountErrMsg` move from `ui_interface.cpp` to `util/error.cpp`
- `FormatStateMessage` and `strMessageMagic` move from `validation.cpp` to `util/validation.cpp`
- `VerifyWallets` `LoadWallets` `StartWallets` `FlushWallets` `StopWallets` and `UnloadWallets` move from `wallet/init.cpp` to `wallet/node.cpp`
ACKs for commit 4d074e:
jnewbery:
utACK 4d074e84a2 (checked by doing the rebase myself and verifying no difference between my branch and 4d074e84a2)
Tree-SHA512: 5e1604a9fb06475f2b96da0de0baa8330f4dda834dc20a0183ef11e1e4c27631d1d1bbb9abf0054efc03d56945fdf9920f63366b6a4f200f665b742a479ff75c
Remove last few instances of accesses to node global variables from wallet
code. Also remove accesses to node globals from code in policy/policy.cpp that
isn't actually called by wallet code, but does get linked into wallet code.
This is the last change needed to allow bitcoin-wallet tool to be linked
without depending on libbitcoin_server.a, to ensure wallet code doesn't access
node global state and avoid bugs like
https://github.com/bitcoin/bitcoin/pull/15557#discussion_r267735431
c968780785 [docs] fix comment: the return value of findFork is _not_ an ancestor when the specified block is on the active chain (r8921039)
Pull request description:
The return value of findFork is an ancestor of the specified block only when specified block is _not_ on the active chain. When it is on the active chain, the return value is the specified block itself, not an ancestor of it.
ACKs for commit c96878:
promag:
utACK c968780, however comment could be shorter.
ryanofsky:
utACK c968780785. Only change since last review is squash
Tree-SHA512: bb05d734059898784c4a59b5b0344719eb4dfb2d49a0f7f705fcb2eb630702e66be81c01299185faf0c219fa9f9aa64cbdf6d5f91e0b3dce0ff420909a454a18
This refactors the CWalletTx::RelayWalletTransaction() function to be
clearer and adds comments. It also makes two minor behavior
changes:
- no longer assert if fBroadcastTransactions is false. Just return false
from the function.
- no longer print the relay message if p2pEnabled is set to false (since
the transaction is not actually relayed).
fa26eb5e8f rpc: RPCHelpMan: Always push_name when outer type is an object (MarcoFalke)
fa652b229e rpc: Add some doxygen comments to utils (MarcoFalke)
Pull request description:
Fixes two issues reported in #15737:
* > I am very perplexed as to how the code I'm looking at is generating the help text I'm seeing
So add documentation
* > This is a value for which a key is missing
So always serialize the name of the dictionary key if the outer type is a dictionary
ACKs for commit fa26eb:
promag:
Tested ACK fa26eb5.
Tree-SHA512: b6f0cee1f1123d245d4902e8e113b5260cae7f2cb39c9bfb8893c5b0b33ffb6349ad05813d560d39a94ccf655399c05fcda15d9b0733e6bd696538fe0aca7021
fae38c3dc6 doc: Fix all typos reported by codespell (MarcoFalke)
fa9058f0ed doc: Add release notes for 15629 (MarcoFalke)
fa4a922d78 qa: Add test for missing testnet section in conf file (MarcoFalke)
dddd6f0f58 init: Throw error when network specific config is ignored (MarcoFalke)
Pull request description:
This should have no effect on mainnet users, but simplifies testing, where config settings are currently ignored with only a warning. Fix this by making it an error.
Issues:
* bitcoin client 0.17.0 ignores wallet's name (file) #14523
* Can't set custom rpcport on testnet #13777
* ...
ACKs for commit fae38c:
Tree-SHA512: 2e209526898eea6e444c803ec2666989cee4ca137492d32984998733c50a70056cb54657df8dc3027a6a0612738a8afce0bc35824b868c5f22281e00e0188530
psbt.cpp definitions except for AnalyzePSBT are used by the wallet and need to
be linked into the wallet binary. AnalyzePSBT is an exception in that it is not
used by the wallet, and depends on node classes like CCoinsViewCache, and on
node global variables like nBytesPerSigOp.
So AnalyzePSBT is more at home in libbitcoin_server than libbitcoin_common, and
in any case needs to be defined in a separate object file than other PSBT
utilities, to avoid dragging link dependencies on node functions and global
variables into the wallet.
833d98ae0 [wallet] Remove unnecessary Chain::Lock parameter from ResendWalletTransactions (John Newbery)
52b760fc6 [wallet] Schedule tx rebroadcasts in wallet (John Newbery)
f463cd107 [wallet] Keep track of the best block time in the wallet (John Newbery)
Pull request description:
Remove the `Broadcast()`/`ResendWalletTransactions()` notification from the Validation interface.
Closes#15619. See that issue for discussion.
ACKs for commit 833d98:
ryanofsky:
utACK 833d98ae07. No changes, just rebase.
Tree-SHA512: 7689f2083608ebad8c95ab6692f7842754e1ebe5508bc926a89cad7105cce41007648f37341ba5feb92b30a7aa87acd3abf264a4f1874e35a7161553f6ff3595
Moves the following wallet load functions to a new wallet/load unit in
the libbitcoin_wallet library. All other functions in wallet/init remain
in libbitcoin_server:
- `VerifyWallets`
- `LoadWallets`
- `StartWallets`
- `FlushWallets`
- `StopWallets`
- `UnloadWallets`
Adds the following util units and adds them to libbitcoin_util:
- `util/url.cpp` takes `urlDecode` from `httpserver.cpp`
- `util/error.cpp` takes `TransactionErrorString` from
`node/transaction.cpp` and `AmountHighWarn` and `AmountErrMsg` from
`ui_interface.cpp`
- `util/fees.cpp` takes `StringForFeeReason` and `FeeModeFromString` from `policy/fees.cpp`
- `util/rbf.cpp` takes `SignalsOptInRBF` from `policy/rbf.cpp`
- 'util/validation.cpp` takes `FormatStateMessage` and `strMessageMagic` from 'validation.cpp`
Moves the following units into libbitcoin_util or libbitcoin_common
since they are required by multiple libraries:
- bloom
- interfaces/handler
- merkleblock
- outputtype
rpc/rawtransaction.cpp moves to libbitcoin_server since it should not be
accessed by non-node libraries. The utility following utility methods
move to their own unit rpc/rawtransaction_util since they need to be
accessed by non-node libraries:
- `ConstructTransaction`
- `TxInErrorToJSON`
- `SignTransaction`
Moves the following utility methods to rpc/util and moves that unit to
libbitcoin_common so they can be accessed by all libraries.
- `RPCTypeCheck`
- `RPCTypeCheckArgument`
- `RPCTypeCheckObj`
- `AmountFromValue`
- `ParseHashV``ParseHashO`
- `ParseHexV`
- `ParseHexO`
- `HelpExampleCli`
- `HelpExampleRpc`
This moves the following policy settings functions and globals to a new
src/policy/settings unit in lib_server:
- `incrementalRelayFee`
- `dustRelayFee`
- `nBytesPerSigOp`
- `fIsBareMultisigStd`
These settings are only required by the node and should not be accessed
by other libraries.
CheckTransaction is a context-free function that does not require access
to the blockchain or mempool. Move it from src/consensus/tx_verify in
lib_server to a new unit src/consensus/tx_check in lib_consensus so that
it can be called by non-server libraries.
