418d3230f8 Resolve the checkpoints <-> validation CD. (251)
Pull request description:
This pull request attempts to resolve the `checkpoints -> validation -> checkpoints` circular dependency.
The circular dependency is resolved by moving the `CheckPoints::GetLastCheckpoint(const CCheckpointData& data)` function to `validation.cpp` where it used exclusively by the private function `ContextualCheckBlockHeader(const CBlockHeader& block, CValidationState& state, const CChainParams& params, const CBlockIndex* pindexPrev, int64_t nAdjustedTime)`.
ACKs for commit 418d32:
promag:
utACK 418d323, only `GetLastCheckpoint` usage is in `validation.cpp` and so makes sense to move it there.
practicalswift:
utACK 418d3230f8
MarcoFalke:
utACK 418d3230f8
sipa:
utACK 418d3230f8
Tree-SHA512: 03c3556bc192e65f5e3fa76fd545d4ee7d63d3fb06b132f7a1fa6131aa21ddd2e5b2d19e2222dfe524f422daaca30efde219bed188db8c74ff4b088876b5bc16
c7efb652f3 blockfilter: Update BIP 158 test vectors. (Jim Posen)
19308c9e21 rpc: Add getblockfilter RPC method. (Jim Posen)
ff35105096 init: Add CLI option to enable block filter index. (Jim Posen)
accc8b8b18 index: Access functions for global block filter indexes. (Jim Posen)
2bc90e4e7b test: Unit test for block filter index reorg handling. (Jim Posen)
6bcf0998c0 test: Unit tests for block index filter. (Jim Posen)
b5e8200db7 index: Implement lookup methods on block filter index. (Jim Posen)
75a76e3619 index: Implement block filter index with write operations. (Jim Posen)
2ad2338ef9 serialize: Serialization support for big-endian 32-bit ints. (Jim Posen)
ba6ff9a6f7 blockfilter: Functions to translate filter types to/from names. (Jim Posen)
62b7a4f094 index: Ensure block locator is not stale after chain reorg. (Jim Posen)
4368384f1d index: Allow atomic commits of index state to be extended. (Jim Posen)
Pull request description:
This introduces a new BlockFilterIndex class, which is required for BIP 157 support.
The index is uses the asynchronous BaseIndex infrastructure driven by the ValidationInterface callbacks. Filters are stored sequentially in flat files and the disk location of each filter is indexed in LevelDB along with the filter hash and header. The index is designed to ensure persistence of filters reorganized out of the main chain to simplify the BIP 157 net implementation.
Stats (block height = 565500):
- Syncing the index from scratch takes 45m
- Total index size is 3.8 GiB
ACKs for commit c7efb6:
MarcoFalke:
utACK c7efb652f3
ryanofsky:
Slightly tested ACK c7efb652f3 (I just rebuilt the index with the updated PR and tested the RPC). Changes since last review: rebase, fixed compile errors in internal commits, new comments, updated error messages, tweaked cache size logic, renamed commit method, renamed constants and globals, fixed whitespace, extra BlockFilterIndex::Init error check.
Tree-SHA512: f8ed7a9b6f76df45933aa5eba92b27b3af83f6df2ccb3728a5c89eec80f654344dc14f055f6f63eb9b3a7649dd8af6553fe14969889e7e2fd2f8461574d18f28
fab6a0a659 test: Add test that addmultisigaddress fails for watchonly addresses (MarcoFalke)
fad81d870a test: Fixup creatmultisig documentation and whitespace (MarcoFalke)
Pull request description:
Just to make sure this is not regressed on accidentally in the future
ACKs for commit fab6a0:
jonatack:
ACK fab6a0a659
Tree-SHA512: bf8dcbc752f8910902a995e55ce486621156aa01f112990344815c4aab980298dfecc108e78245a8986a00c3871338ad16fc818a1bce9dfc6b37b9c88851e39d
aa410c2b17 rpc: Validate maxfeerate with AmountFromValue (João Barbosa)
Pull request description:
With this change `maxfeerate` can also be set as a string, accordingly to the help test:
```
maxfeerate (numeric or string,
```
Beside, there are no tests for the removed errors.
ACKs for commit aa410c:
meshcollider:
utACK aa410c2b17
MarcoFalke:
utACK aa410c2b17 Good catch
Tree-SHA512: f3bfea91dc7daa943729e270585dbf333055aeda805fbd01eaab20a7e0e6147382647c11525334382d198df0d3d45da6102b541efda5a1361f96271c98d5d89d
184f8785f wallet_bumpfee.py: add test for change key preservation (Gregory Sanders)
d08becff8 add functional tests for feerate bumpfee with adding inputs (Gregory Sanders)
0ea47ba7b generalize bumpfee to add inputs when needed (Gregory Sanders)
Pull request description:
When targeting a feerate using `bumpfee`, call a new function that directly uses `CWallet::CreateTransaction` and coin control to get the desired result. This allows us to get a superset of previous behavior, with an arbitrary RBF bump of a transaction provided it passes the preconditional checks and spare confirmed utxos are available.
Note(s):
0) The coin selection will use knapsack solver for the residual selection.
1) This functionality, just like knapsack coin selection in general, will hoover up negative-value inputs when given the chance.
2) Newly added inputs must be confirmed due to current Core policy. See error: `replacement-adds-unconfirmed`
3) Supporting this with `totalFee` is difficult since the "minimum total fee" option in `CreateTransaction` logic was (rightly)taken out in #10390 .
ACKs for commit 184f87:
jnewbery:
utACK 184f8785f7
Tree-SHA512: fb6542bdfb2c6010e328ec475cf9dcbff4eb2b1a1b27f78010214534908987a5635797196fa05edddffcbcf2987335872dc644a99261886d5cbb34a8f262ad3e
fafe5f0d09 test: Remove unused imports (MarcoFalke)
fa16a09215 scripted-diff: use self.sync_* methods (MarcoFalke)
faf77f9b90 test: Pass self to test_simple_bumpfee_succeeds (MarcoFalke)
fa6dc7c5c3 test: Add BitcoinTestFramework::sync_* methods (MarcoFalke)
fafe008cb4 test: Pass at most one node group to sync_all (MarcoFalke)
fa4680ed09 scripted-diff: Rename sync_blocks to send_blocks to avoid name collisions and confusion (MarcoFalke)
Pull request description:
This adds methods to the test framework that can be called by just `self.sync_*()`.
This avoids having to import the underlying util method. Also, in the default case, where all nodes are synced this avoid having to pass `self.nodes` explicitly.
So the effective changes are:
```diff
@@
-from test_framework.util import sync_blocks, sync_mempools
@@
- sync_blocks(self.nodes)
+ self.sync_blocks()
@@
- sync_mempools(self.nodes)
+ self.sync_mempools()
ACKs for commit fafe5f:
promag:
utACK fafe5f0.
jonatack:
ACK fafe5f0d09, nice simplification.
Tree-SHA512: 5c81840edf9fb3c5de2d7bf95ca36a5a8d23567cd1479a0f4044547c2080e9a3c5cf375357bc8eebb5b68829be050a171ab2512cfd47b89feed51fe3bad2cd72
4d074e84a2 [build] Move AnalyzePSBT from psbt.cpp to node/psbt.cpp (Russell Yanofsky)
fd509bd1f7 [docs] Document src subdirectories and different libraries (John Newbery)
9eaeb7fb8d [build] Move wallet load functions to wallet/load unit (John Newbery)
91a25d1e71 [build] Add several util units (John Newbery)
99517866b6 [build] Move several units into common libraries (John Newbery)
0509465542 [build] Move rpc rawtransaction util functions to rpc/rawtransaction_util.cpp (John Newbery)
1acc61f874 [build] Move rpc utility methods to rpc/util (John Newbery)
4a75c9d651 [build] Move policy settings to new src/policy/settings unit (John Newbery)
fdf8888b6f [build] Move CheckTransaction from lib_server to lib_consensus (John Newbery)
Pull request description:
This is a move-only commit. No code is changing and the moves can be easily verified with:
```sh
git log -p -n1 --color-moved=dimmed_zebra
```
This commit moves functions and variables that wallet code depends on out of libbitcoin_server.a, so the bitcoin-wallet tool can be built without libbitcoin_server.a in #15639, and attempting to access server state from wallet code will result in link errors instead of silently broken code.
List of moves:
- `CheckTransaction` moves from `consensus/tx_verify.cpp` to `consensus/tx_check.cpp`
- `urlDecode` moves from `httpserver.cpp` to `util/url.cpp`
- `TransactionErrorString` moves from `node/transaction.cpp` to `util/error.cpp`
- `StringForFeeReason` and `FeeModeFromString` move from `policy/fees.cpp` to `util/fees.cpp`
- `incrementalRelayFee` `dustRelayFee` and `nBytesPerSigOp` move from `policy/policy.cpp` to `policy/settings.cpp`
- `SignalsOptInRBF` moves from `policy/rbf.cpp` to `util/rbf.cpp`
- `fIsBareMultisigStd` moves from `validation.cpp` to `policy/settings.cpp`
- `ConstructTransaction` `TxInErrorToJSON` and `SignTransaction` move from `rpc/rawtransaction.cpp` to `rpc/rawtransaction_util.cpp`
- `RPCTypeCheck` `RPCTypeCheckArgument` `RPCTypeCheckObj` `AmountFromValue` `ParseHashV``ParseHashO` `ParseHexV` `ParseHexO` `HelpExampleCli` and `HelpExampleRpc` move from `rpc/server.cpp` to `rpc/util.cpp`
- `AmountHighWarn` and `AmountErrMsg` move from `ui_interface.cpp` to `util/error.cpp`
- `FormatStateMessage` and `strMessageMagic` move from `validation.cpp` to `util/validation.cpp`
- `VerifyWallets` `LoadWallets` `StartWallets` `FlushWallets` `StopWallets` and `UnloadWallets` move from `wallet/init.cpp` to `wallet/node.cpp`
ACKs for commit 4d074e:
jnewbery:
utACK 4d074e84a2 (checked by doing the rebase myself and verifying no difference between my branch and 4d074e84a2)
Tree-SHA512: 5e1604a9fb06475f2b96da0de0baa8330f4dda834dc20a0183ef11e1e4c27631d1d1bbb9abf0054efc03d56945fdf9920f63366b6a4f200f665b742a479ff75c
fae38c3dc6 doc: Fix all typos reported by codespell (MarcoFalke)
fa9058f0ed doc: Add release notes for 15629 (MarcoFalke)
fa4a922d78 qa: Add test for missing testnet section in conf file (MarcoFalke)
dddd6f0f58 init: Throw error when network specific config is ignored (MarcoFalke)
Pull request description:
This should have no effect on mainnet users, but simplifies testing, where config settings are currently ignored with only a warning. Fix this by making it an error.
Issues:
* bitcoin client 0.17.0 ignores wallet's name (file) #14523
* Can't set custom rpcport on testnet #13777
* ...
ACKs for commit fae38c:
Tree-SHA512: 2e209526898eea6e444c803ec2666989cee4ca137492d32984998733c50a70056cb54657df8dc3027a6a0612738a8afce0bc35824b868c5f22281e00e0188530
833d98ae0 [wallet] Remove unnecessary Chain::Lock parameter from ResendWalletTransactions (John Newbery)
52b760fc6 [wallet] Schedule tx rebroadcasts in wallet (John Newbery)
f463cd107 [wallet] Keep track of the best block time in the wallet (John Newbery)
Pull request description:
Remove the `Broadcast()`/`ResendWalletTransactions()` notification from the Validation interface.
Closes#15619. See that issue for discussion.
ACKs for commit 833d98:
ryanofsky:
utACK 833d98ae07. No changes, just rebase.
Tree-SHA512: 7689f2083608ebad8c95ab6692f7842754e1ebe5508bc926a89cad7105cce41007648f37341ba5feb92b30a7aa87acd3abf264a4f1874e35a7161553f6ff3595
Adds the following util units and adds them to libbitcoin_util:
- `util/url.cpp` takes `urlDecode` from `httpserver.cpp`
- `util/error.cpp` takes `TransactionErrorString` from
`node/transaction.cpp` and `AmountHighWarn` and `AmountErrMsg` from
`ui_interface.cpp`
- `util/fees.cpp` takes `StringForFeeReason` and `FeeModeFromString` from `policy/fees.cpp`
- `util/rbf.cpp` takes `SignalsOptInRBF` from `policy/rbf.cpp`
- 'util/validation.cpp` takes `FormatStateMessage` and `strMessageMagic` from 'validation.cpp`
This moves the following policy settings functions and globals to a new
src/policy/settings unit in lib_server:
- `incrementalRelayFee`
- `dustRelayFee`
- `nBytesPerSigOp`
- `fIsBareMultisigStd`
These settings are only required by the node and should not be accessed
by other libraries.
Removes the now-unused Broadcast/ResendWalletTransactions interface from
validationinterface.
The wallet_resendwallettransactions.py needs a sleep added at the start
to make sure that the rebroadcast scheduler is warmed up before the next
block is mined.
b5d398772 Take non-importing keys into account for spendability warning in descriptor import (Pieter Wuille)
6e597001a Import all origin info in importmulti; even for non-importing pubkeys (Pieter Wuille)
9a93c91c8 Keep full pubkeys in FlatSigningProvider::origins (Pieter Wuille)
Pull request description:
This fixes#15743 and #15742.
Since #15263, pubkeys are no longer imported for non-PKH (or WPKH, or any wrapped form of those) outputs, as that would incorrectly mark outputs to single-key versions of multisig policies as watched.
As a side effect, this change also caused origin info not to be imported anymore for multisig policies.
Fix this by plumbing through the full pubkey information for origins in FlatSigningProvider, and then importing all origin info we have in `importmulti` (knowing more never hurts, and additional origin information has no negative consequences like importing the pubkeys themselves).
ACKs for commit b5d398:
MeshCollider:
utACK b5d3987724
Tree-SHA512: 37caa2be8d01b8baa12f70a58eaa7c583f5f0afbe012e02936dd8790dc5dc852f880b77258b34ddb68cae30c029585f2d1c4f5d00015380557a1e8b471e500f3
8f5d9431a Add regtests for HTTP status codes. (Daniel Kraft)
Pull request description:
This adds explicit tests for the returned HTTP status codes to `interface_rpc.py` (for error cases) and the HTTP JSON-RPC client in general for success.
#15381 brought up discussion about the HTTP status codes in general, and the general opinion was that the current choice may not be ideal but should not be changed to preserve compatibility with existing JSON-RPC clients. Thus it makes sense to actually test the current status to ensure this desired compatibility is not broken accidentally.
ACKs for commit 8f5d94:
laanwj:
utACK 8f5d9431a3
promag:
utACK 8f5d943.
jonasschnelli:
utACK 8f5d9431a3
Tree-SHA512: 82503ccd134dd9145304e95cb6c61755f100bee27593d567cdd5c0c554d47e7b06d937456cab04107f46f4984930355db65d5e711008a0b05f2b8feec9f2950e
7813eb1db1 [qa] Overhaul p2p_compactblocks.py (Suhas Daftuar)
Pull request description:
Remove tests of:
- compactblock behavior in a simulated pre-segwit version of bitcoind
This should have been removed a long time ago, as it is not generally
necessary for us to test the behavior of old nodes (except perhaps if we
want to test that upgrading from an old node to a new one behaves properly)
- compactblock behavior during segwit upgrade (ie verifying that network
behavior before and after activation was as expected)
This is unnecessary to test now that segwit activation has already happened.
ACKs for commit 7813eb:
jnewbery:
utACK 7813eb1db1
Tree-SHA512: cadf035e6f822fa8cff974ed0c2e88a1d4d7da559b341e574e785fd3d309cc2c98c63bc05479265dc00550ae7b77fc3cbe815caae7f68bcff13a04367dca9b52
ea1a2d8794 [wallet] Remove ResendWalletTransactionsBefore (John Newbery)
f5162458cd [rpc] remove resendwallettransactions RPC (John Newbery)
Pull request description:
Remove resendwallettransactions RPC method
This RPC was added for testing wallet rebroadcasts. Since we now have a real test for wallet rebroadcasts, it's no longer needed.
The call in wallet_basic.py can be removed because wallet_resendwallettransactions.py tests wallet rebroadcast.
ACKs for commit ea1a2d:
MarcoFalke:
re-utACK ea1a2d8794
promag:
utACK ea1a2d8.
Tree-SHA512: 48245d947be1a2d2b8c30d2946105818c454a03b70b63534ecadf2144da64dafe1c9527ea670a5f4d1acd05ccdfc6c9be43ca636ee2ba58a8b7a7b2fc7bc88fd
Remove tests of:
- compactblock behavior in a simulated pre-segwit version of bitcoind
This should have been removed a long time ago, as it is not generally
necessary for us to test the behavior of old nodes (except perhaps if we
want to test that upgrading from an old node to a new one behaves properly)
- compactblock behavior during segwit upgrade (ie verifying that network
behavior before and after activation was as expected)
This is unnecessary to test now that segwit activation has already happened.
Includes changes by John Newbery.
This RPC was added for testing wallet rebroadcasts. Since we now have a
real test for wallet rebroadcasts, it's no longer needed.
The call in wallet_basic.py can be removed because
wallet_resendwallettransactions.py tests wallet rebroadcast.
This commit makes the pruning.py much faster.
Key insights to do this:
- pruning.py doesn't care what kind of transactions make up the big
blocks that are pruned in the test. Instead of making blocks with
several large, expensive to construct and validate transactions,
instead make the large blocks contain a single coinbase transaction with
a huge OP_RETURN txout.
- avoid stop-starting nodes where possible.
This test could probably be made even faster by using the P2P interface
for submitting blocks instead of the submitblock RPC.
529c1ae4a0 [tests] Add test for wallet rebroadcasts (John Newbery)
Pull request description:
The existing wallet_resendwallettransactions.py test only tests the
resendwallettransactions RPC. It does not test whether transactions are
actually rebroadcast, or whether the rebroadcast logic is called on a
timer.
Update the test to not use the resendwallettransactions RPC and test
that transactions are resent on a timer.
ACKs for commit 529c1a:
MarcoFalke:
re-utACK 529c1ae4a0
Tree-SHA512: 7341e7dd07cdc8ecbc08b1949121824148d2b58133a8e298ecdc5b7555713df3cecffb49854443cef9f033ef847cbf329e879a3bf57ab4e1fc733be432e9f718
The existing wallet_resendwallettransactions.py test only tests the
resendwallettransactions RPC. It does not test whether transactions are
actually rebroadcast, or whether the rebroadcast logic is called on a
timer.
This commit updates the test to not use the resendwallettransactions RPC and
test that transactions are rebroadcast on a timer.
e9d5e97561 Poly1305: tolerate the intentional unsigned wraparound in poly1305.cpp (Jonas Schnelli)
b34bf302f2 Add Poly1305 bench (Jonas Schnelli)
03be7f48fa Add Poly1305 implementation (Jonas Schnelli)
Pull request description:
This adds a currently unused Poly1305 implementation including test vectors from RFC7539.
Required for BIP151 (and related to #15512).
Tree-SHA512: f8c1ad2f686b980a7498ca50c517e2348ac7b1fe550565156f6c2b20faf764978e4fa6b5b1c3777a16e7a12e2eca3fb57a59be9c788b00d4358ee80f2959edb1
This commit resolves the checkpoints -> validation -> checkpoints
cirular dependency by moving
`CheckPoints::GetLastCheckpoint(const CCheckpointData& data)` from
`checkpoints.cpp` to `validation.cpp`.
947f73ceba [docs] remove reference to signrawtransaction in the developer docs. (John Newbery)
7b6616b78b [rpc] Remove deprecated functionality message from validateaddress help (John Newbery)
839c3f7c49 [rpc] Remove signrawtransaction warning (John Newbery)
Pull request description:
Removes some deprecated code from the RPCs:
- signrawtransaction was deprecated in 0.17 and removed in 0.18. A warning message was left in place to advise users to use signrawtransactionwithwallet and signrawtransactionwithkey. That warning can now be removed.
- validateaddress had some functionality deprecated in 0.17 and removed in 0.18. The help text for that functionality was not removed in 0.18 and can be removed now.
Tree-SHA512: 981678a697954ff2c392752e5a183b4b12c4eb94f55766ee1aa97a70d300668237db8fc5748c2772869d0155ba4a93e38817887b98160ee972a6f6ee94e3f7d9
signrawtransaction was deprecated in 0.17 and removed in 0.18. A warning
was left in place to tell users to migrate to using
signrawtransactionswithwallet or signrawtransactionwithkey. Remove the
warning now that it's been two releases since the method was removed.
