07c4838 Always return true if AppInitMain got to the end (Matt Corallo)
Pull request description:
This should fix a rare zapwallettxes failure on travis, but also
avoids having init operations (re-adding wallet transactions to
mempool) running after RPC is free'd.
I believe this was the failure at https://travis-ci.org/bitcoin/bitcoin/jobs/311747844 (from #11605).
Tree-SHA512: f0fea8c1b9265e2eeda57043d541380a3e58e4d9388fa24628a52fd56324257fcd7df0ca02e8f77f66fadd68d951893bab0f610ed9fd0a89b2ccd6bad1efa351
This should fix a very rare travis failure in zapwallettxes, but
is also more correct, as you can currently race
ReacceptWalletTransactions with stop RPC calls to get bitcoind to
(IMO) eroneously return a non-0 exit code.
This patch adds an option to configure the name and/or directory of the
debug log.
The user can specify either a relative path, in which case the path
is relative to the data directory. They can also specify an absolute
path to put the log anywhere else in the file system.
d31e5c1 Fix shutdown in case of errors during initialization (Wladimir J. van der Laan)
Pull request description:
PR #10286 introduced a few steps which are not robust to early shutdown in initialization.
Stumbled upon this with #11781, not sure if there are other scenarios that can trigger it, but it's good to harden against this in any case.
E.g.
```
$ src/bitcoind -debuglogfile=/dfdf
Error: Could not open debug log file /dfdf
Program received signal SIGSEGV, Segmentation fault.
UnregisterValidationInterface (pwalletIn=0x0) at /.../bitcoin/src/validationinterface.cpp:82
82 g_signals.m_internals->BlockChecked.disconnect(boost::bind(&CValidationInterface::BlockChecked, pwalletIn, _1, _2));
(gdb) bt
#0 UnregisterValidationInterface (pwalletIn=0x0) at /.../bitcoin/src/validationinterface.cpp:82
#1 0x00005555555a11fc in Shutdown () at /.../bitcoin/src/init.cpp:196
#2 0x00005555555961cc in AppInit (argc=<optimized out>, argv=<optimized out>) at /.../bitcoin/src/bitcoind.cpp:183
#3 0x0000555555596249 in main (argc=0, argv=0x555555ecf200) at /.../bitcoin/src/bitcoind.cpp:19
```
Tree-SHA512: 7dd9570a9803514a17781bfadf1edde47e96df4e852cce2f423cab422e005fb94d44e777af1a6ea5167b04a4d889e848ae7a61a7e0e94232247ddea32ee70fc8
c60c49b Improve help text and behavior of RPC-logging (Akio Nakamura)
Pull request description:
1. It is allowed `libevent` logging to be updated during runtime,
but still described that restriction in the help text.
So we delete these text.
2. Add a descrption about the evaluation order of `<include>` and
`<exclude>` to clarify how debug loggig categories to be set.
3. Add a description about the available logging category `"all"`
which is not explained.
4. Add `"optional"` to the help text of `<include>` and `<exclude>`.
5. Add missing new lines before `"Argument:"`.
6. `"0"`,`"1"` are allowed in both array of `<include>` and `<exclude>`.
`"0"` is **ignored** and `"1"` is treated **same as** `"all"`.
It is confusing, so forbid them.
7. It always returns all logging categories with status.
Fix the help text to match this behavior.
Tree-SHA512: c2142da1a9bf714af8ebc38ac0d82394e2073fc0bd56f136372e3db7b2af3b6746f8d6b0241fe66c1698c208c124deb076be83f07dec0d0a180ad150593af415
PR #10286 introduced a few steps which are not robust to early shutdown
in initialization.
Stumbled upon this with #11781, not sure if there are other scenarios
that can trigger it, but it's harden against this in any case.
abbd230 Move RPC registration out of AppInitParameterInteraction (Russell Yanofsky)
Pull request description:
Move to AppInitServers. This doesn't have any effects on bitcoin behavior. It was just strange to have this unrelated code in the middle of parameter interaction.
Tree-SHA512: 373e18f2ef8d21999ad36295d69326128a3086044acfc8ed537abd5497c8d3620b9832f7f6aa87c0c0939bb5e0d92be8a3c006b5997e9e6fa20334f5610c89bc
A) The changes in behavior are as follows:
1. Introduce logging category "none" as alias of "0" for
both RPC-logging and bitcoind "-debug" parameter.
2. Same as "0" is given to argument of "-debug",
if "none" or "0" is given to <include>, all other given logging
categories are ignored. The same is true for <exclude>.
(Before this PR, "0" was accepted but just be ignored itself.)
B) The changes in the help text are as follows:
1. Add a descrption about the evaluation order of <include> and
<exclude> to clarify how debug loggig categories to be set.
2. Delete text that describe restriction about libevent because
it's already allowed libevent logging to be updated during runtime.
3. Add a description for category "all", "1", "none" and "0".
4. Add "optional" to the help text of <include> and <exclude>.
5. Add missing new lines before "Argument:".
6. This RPC always returns all logging categories with status.
Fix the help text to match this behavior.
89f0312 Remove redundant pwallet nullptr check (Matt Corallo)
c4784b5 Add a dev notes document describing the new wallet RPC blocking (Matt Corallo)
3ea8b75 Give ZMQ consistent order with UpdatedBlockTip on scheduler thread (Matt Corallo)
cb06edf Fix wallet RPC race by waiting for callbacks in sendrawtransaction (Matt Corallo)
e545ded Also call other wallet notify callbacks in scheduler thread (Matt Corallo)
17220d6 Use callbacks to cache whether wallet transactions are in mempool (Matt Corallo)
5d67a78 Add calls to CWallet::BlockUntilSyncedToCurrentChain() in RPCs (Matt Corallo)
5ee3172 Add CWallet::BlockUntilSyncedToCurrentChain() (Matt Corallo)
0b2f42d Add CallFunctionInQueue to wait on validation interface queue drain (Matt Corallo)
2b4b345 Add ability to assert a lock is not held in DEBUG_LOCKORDER (Matt Corallo)
0343676 Call TransactionRemovedFromMempool in the CScheduler thread (Matt Corallo)
a7d3936 Add a CValidationInterface::TransactionRemovedFromMempool (Matt Corallo)
Pull request description:
Based on #10179, this effectively reverts #9583, regaining most of the original speedups of #7946.
This concludes the work of #9725, #10178, and #10179.
See individual commit messages for more information.
Tree-SHA512: eead4809b0a75d1fb33b0765174ff52c972e45040635e38cf3686cef310859c1e6b3c00e7186cbd17374c6ae547bfbd6c1718fe36f26c76ba8a8b052d6ed7bc9
76ea17c79 Add mutex requirement for AddToCompactExtraTransactions(…) (practicalswift)
4616c825a Use -Wthread-safety-analysis if available (+ -Werror=thread-safety-analysis if --enable-werror) (practicalswift)
7e319d639 Fix -Wthread-safety-analysis warnings. Change the sync.h primitives to std from boost. (Matt Corallo)
Pull request description:
* Add mutex requirement for `AddToCompactExtraTransactions(…)`.
* Use `-Wthread-safety-analysis` if available.
* Rebased on top of https://github.com/TheBlueMatt/bitcoin/commits/2017-08-test-10923 - now includes: Fix -Wthread-safety-analysis warnings. Change the sync.h primitives to std from boost.
Tree-SHA512: fb7365f85daa2741c276a1c899228181a8d46af51db7fbbdffceeaff121a3eb2ab74d7c8bf5e7de879bcc5042d00d24cb4649c312d51caba45a3f6135fd8b38f
Commit 1.
This code was written by @TheBlueMatt in the following branch:
* https://github.com/TheBlueMatt/bitcoin/commits/2017-08-test-10923
This commit message was written by me (@practicalswift) who also squashed
@TheBlueMatt's commits into one and tried to summarize the changes made.
Commit 2.
Remove boost include. Remove boost mentions in comments.
Move to AppInitServers. This doesn't have any effects on bitcoin behavior. It
was just strange to have this unrelated code in the middle or parameter
interaction.
If our tip hasn't updated in a while, that may be because our peers are
not relaying blocks to us that we would consider valid. Allow connection
to an additional outbound peer in that circumstance.
Also, periodically check to see if we are exceeding our target number of
outbound peers, and disconnect the one which has least recently
announced a new block to us (choosing the newest such peer in the case
of tie).
f4c4e38 [trivial] Make namespace explicit for is_regular_file (John Newbery)
Pull request description:
is_regular_file resolves using argument dependent lookup. Make the
namespace explicit so it's obvious where the function is defined.
For those not familiar with argument dependent lookups:
- http://en.cppreference.com/w/cpp/language/adl
- https://en.wikipedia.org/wiki/Argument-dependent_name_lookup
Thanks to C++ guru @ryanofsky for pointing this out to me.
Tree-SHA512: 919f1818081a8f90c5751181f87e13b06d90f8aec0ab873100434e55c85cca6e0e288ecc7f135e19e9b5dba7952e96b6393864b7840e20b69dd40e92a157928b
This is currently unused, but will by used by wallet to cache when
transactions are in the mempool, obviating the need for calls to
mempool from CWalletTx::InMempool()
15f5d3b17 Switch DNSSeed-needed metric to any-automatic-nodes, not services (Matt Corallo)
5ee88b4bd Clarify docs for requirements/handling of addnode/connect nodes (Matt Corallo)
57edc0b0c Rename fAddnode to a more-descriptive "manual_connection" (Matt Corallo)
44407100f Replace relevant services logic with a function suite. (Matt Corallo)
Pull request description:
This was mostly written as a way to clean things up so that the NETWORK_LIMITED PR (#10387) can be simplified a ton, but its also a nice standalone cleanup that will also require a bit of review because it tweaks a lot of stuff across net. The new functions are fine in protocol.h right now since they're straight-forward, but after NETWORK_LIMITED will really want to move elsewhere after @theuni moves the nServices-based selection to addrman from connman.
Adds HasAllRelevantServices and GetRelevantServices, which check
for NETWORK|WITNESS.
This changes the following:
* Removes nRelevantServices from CConnman, disconnecting it a bit
more from protocol-level logic.
* Replaces our sometimes-connect-to-!WITNESS-nodes logic with
simply always requiring WITNESS|NETWORK for outbound non-feeler
connections (feelers still only require NETWORK).
* This has the added benefit of removing nServicesExpected from
CNode - instead letting net_processing's VERSION message
handling simply check HasAllRelevantServices.
* This implies we believe WITNESS nodes to continue to be a
significant majority of nodes on the network, but also because
we cannot sync properly from !WITNESS nodes, it is strange to
continue using our valuable outbound slots on them.
* In order to prevent this change from preventing connection to
-connect= nodes which have !WITNESS, -connect nodes are now
given the "addnode" flag. This also allows outbound connections
to !NODE_NETWORK nodes for -connect nodes (which was already true
of addnodes).
* Has the (somewhat unintended) consequence of changing one of the
eviction metrics from the same
sometimes-connect-to-!WITNESS-nodes metric to requiring
HasRelevantServices.
This should make NODE_NETWORK_LIMITED much simpler to implement.
Tree-SHA512: 90606896c86cc5da14c77843b16674a6a012065e7b583d76d1c47a18215358abefcbab44ff4fab3fadcd39aa9a42d4740c6dc8874a58033bdfc8ad3fb5c649fc
Adds HasAllRelevantServices and GetRelevantServices, which check
for NETWORK|WITNESS.
This changes the following:
* Removes nRelevantServices from CConnman, disconnecting it a bit
more from protocol-level logic.
* Replaces our sometimes-connect-to-!WITNESS-nodes logic with
simply always requiring WITNESS|NETWORK for outbound non-feeler
connections (feelers still only require NETWORK).
* This has the added benefit of removing nServicesExpected from
CNode - instead letting net_processing's VERSION message
handling simply check HasAllRelevantServices.
* This implies we believe WITNESS nodes to continue to be a
significant majority of nodes on the network, but also because
we cannot sync properly from !WITNESS nodes, it is strange to
continue using our valuable outbound slots on them.
* In order to prevent this change from preventing connection to
-connect= nodes which have !WITNESS, -connect nodes are now
given the "addnode" flag. This also allows outbound connections
to !NODE_NETWORK nodes for -connect nodes (which was already true
of addnodes).
* Has the (somewhat unintended) consequence of changing one of the
eviction metrics from the same
sometimes-connect-to-!WITNESS-nodes metric to requiring
HasRelevantServices.
This should make NODE_NETWORK_LIMITED much simpler to implement.
c626dcb50 Make fUseCrypto atomic (MeshCollider)
731065b11 Consistent parameter names in txdb.h (MeshCollider)
35aeabec6 Make fReindex atomic to avoid race (MeshCollider)
58d91af59 Fix race for mapBlockIndex in AppInitMain (MeshCollider)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/11106
Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916
d291e7635b just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex
Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42
cffe85f Skip sys::system(...) call in case of empty command (practicalswift)
6fb8f5f Check that -blocknotify command is non-empty before executing (practicalswift)
Pull request description:
Check that `-blocknotify` command is non-empty before executing.
To make the `BlockNotifyCallback(...)` (`-blocknotify`) behaviour consistent with that of:
* `AlertNotify(...)` (`-alertnotify`)
* `AddToWallet(...)` (`-walletnotify`)
Tree-SHA512: 18272166793a5a8b9cc2a727bfbcea53d38c329a55bc975c02db601329d608a61c20e026ce4b616193ecd3810dca4d3e2cb3bf773898a51872008a8dba96763e
048e0c3e2 [rpc] [tests] Add deprecated RPC test (Cristian Mircea Messel)
d4cdbd6fb [rpc] Deprecate estimatefee RPC (John Newbery)
Pull request description:
Deprecates estimatefee in v0.16, for final removal in v0.17.
This commit introduces a phased removal of RPC methods. RPC method is
disabled by default in version x, but can be enabled by using the
`-deprecatedrpc=<methodname>` argument. RPC method is removed entirely in version
(x+1).
This gives users fair warning that an RPC is to be removed, and time to change client software if necessary. Deprecation warnings in RPC return values or release notes are easily ignored.
This is a more generic version of the approach I tried to use in #10841, which too late to make it into v0.15.
Tree-SHA512: 9695a600e84b812974387333e4a6805d18972da30befb754e9e4da77cd9815d00c5cc2ee0b0350bdbbdb5fdc6ba47789f8b2c6f5b15c8cd5a1deefcc4832da30
Deprecate estimatefee in v0.16, for final removal in v0.17.
This commit introduces a phased removal of RPC methods. RPC method is
disabled by default in version x, but can be enabled by using the
`-deprecatedrpc=<method>` argument. RPC method is removed entirely in
version (x+1).
* This removes block-size-limiting code in favor of GBT clients
doing the limiting themselves (if at all).
* -blockmaxsize is deprecated and only used to calculate an implied
blockmaxweight, addressing confusion from multiple users.
* getmininginfo's currentblocksize return value was returning
garbage values, and has been removed, also removing a
GetSerializeSize call in some block generation inner loops and
potentially addressing some performance edge cases.
5d2a3995e [trivial] fixup comment for VerifyWallets() (John Newbery)
43b0e81d0 [wallet] Add StartWallets() function to wallet/init.cpp (John Newbery)
290f3c56d [wallet] Add RegisterWalletRPC() function to wallet/init.cpp (John Newbery)
062d63102 [wallet] Add CloseWallets() function to wallet/init.cpp (John Newbery)
77fe07c15 [wallet] Add StopWallets() function to wallet/init.cpp (John Newbery)
2da5eafa4 [wallet] Add FlushWallets() function to wallet/init.cpp (John Newbery)
1b9cee66e [wallet] Rename WalletVerify() to VerifyWallets() (John Newbery)
9c76ba18c [wallet] Rename InitLoadWallet() to OpenWallets() (John Newbery)
Pull request description:
Apologies for the mostly code move only PR. This is a pre-req for both #10740 and #10762
All wallet component initialization/destruction functions are now in their own `wallet/init.cpp` translation unit and are no longer static functions on the CWallet class. The bitcoin_server also no longer has any knowledge that there are multiple wallets in vpwallet.
There should be no changes in behavior from this PR.
Tree-SHA512: 7c260eb094f2fa1a88d803769ba60935810968a7309f731135e4b17623b97f18c03bbcd293c942093d1efce62c6c978f9ff484d54dc9a60bc2fcb5af2d160fcd
Rationale:
- this init function can now open multiple wallets (hence
Wallet->Wallets)
- This is named as the antonym to CloseWallets(), which carries out the
opposite action.
eac64bb7a [qa] Test nMinimumChainWork (Suhas Daftuar)
0311836f6 Allow setting nMinimumChainWork on command line (Suhas Daftuar)
Pull request description:
As discussed briefly here: https://botbot.me/freenode/bitcoin-core-dev/2017-02-28/?msg=81712308&page=4
This adds a hidden command line option for setting `nMinimumChainWork`, which allows us to test this parameter in our functional tests, as well as allowing for niche use cases like syncing nodes that are otherwise disconnected from the network.
See also #10345, which proposes a new use of `nMinimumChainWork`.
Tree-SHA512: fe4d8f4f289697615c98d8760f1cc74c076110310ea0b5b875fcab78c127da9195b4eb84148aebacc7606c246e5773d3f13bd5d9559d0a8bffac20a3a28c62df
352d582ba Add vConnect to CConnman::Options (Marko Bencun)
Pull request description:
Split the "-connect" argument parsing out of CConnman and put it into
AppInitMain().
Tree-SHA512: f2d3efc4e2c5808ff98696ea20dd96df599bc472ed5afc9c3eea305d94c36a6ab50c632aa05396c7c34d1917d91b1e7ccd725656ff2631e2a36d9eac477455dc
This contains most of the changes of 10563 "remove safe mode", but doesn't
remove the safe mode yet, but put an `ObserveSafeMode()` check in
individual calls with okSafeMode=false.
This cleans up the ugly "okSafeMode" flag from the dispatch tables,
which is not a concern for the RPC server.
Extra-author: Wladimir J. van der Laan <laanwj@gmail.com>
f01103c MOVEONLY: Init functions wallet/wallet.cpp -> wallet/init.cpp (Russell Yanofsky)
e7fe320 MOVEONLY: Fee functions wallet/wallet.cpp -> wallet/fees.cpp (Russell Yanofsky)
d97fe20 Move some static functions out of wallet.h/cpp (Russell Yanofsky)
Pull request description:
This just moves some static wallet fee and init functions out of `wallet/wallet.cpp` and into new `wallet/fees.cpp` and `wallet/init.cpp` source files. There is one commit updating declarations and callers, followed by two MOVEONLY commits actually moving the function bodies.
This change is desirable because wallet.h/cpp are monolithic and hard to navigate, so pulling things out and grouping together pieces of related functionality should improve the organization.
Another motivation is the wallet process separation work in https://github.com/bitcoin/bitcoin/pull/10973, where (at least initially) parameter parsing and fee estimation are still done in the main process rather than the wallet process, and having functions that run in different processes scrambled up throughout wallet.cpp is unnecessarily confusing.
Tree-SHA512: 6e6982ff82b2ab4e681c043907e2b1801ceb9513394730070f16c46ad338278a863f5b3759aa13db76a259b268b1c919c81f4e339f0796a3cfb990161e8c316d
cd0ea4874 Changing -txindex requires -reindex, not -reindex-chainstate (Matt Corallo)
Pull request description:
If there's an 0.15.0rc3, this should go in it.
Tree-SHA512: 857e77f0af9c055a3d1d91f37474ee9e06d6bc8c5ed21b29201b6c386801e7041523949076cdf0daa4d357a5175ce49394d85a1bedfbf13f3e577bdb6da1d6ce
4aa2508 Bugfix: Use testnet RequireStandard for -acceptnonstdtxn default (Luke Dashjr)
Pull request description:
Fixes a bug introduced in #8855
`-acceptnonstdtxn` is a valid option only for testnet/regtest (in Core), and the help message reflects that. Currently, however, it is buggy in two ways:
1. It uses mainnet to get the default value, which doesn't make sense since the option is never available for mainnet, and the only time the option is available, is when the default is the opposite.
2. It uses the value of "require standard" directly as the default for "accept non-standard transactions", but these concepts are opposites: a negation must be performed to transform one to the other.
Note the combination of these bugs results in the correct boolean output, but the logic to get there is completely wrong.
Tree-SHA512: 06ce513f59ba31f7ab4b6422a08a17bb37a5652dea4c38a4bbefedd5e2752d17bfccc32a4b0508068fa4783e316bff00a821ef18a24b1a2bb02859995d188fdc
40a0f9f Enable devirtualization opportunities by using the final specifier (C++11) (practicalswift)
9a1675e optim: mark a few classes final (Cory Fields)
Pull request description:
Using gcc's ```-Wsuggest-final-types``` and lto, I identified a few easy devirtualization wins:
> wallet/wallet.h:651:7: warning: Declaring type 'struct CWallet' final would enable devirtualization of 26 calls [-Wsuggest-final-types]
>coins.h:201:7: warning: Declaring type 'struct CCoinsViewCache' final would enable devirtualization of 13 calls [-Wsuggest-final-types]
>txdb.h:67:7: warning: Declaring type 'struct CCoinsViewDB' final would enable devirtualization of 5 calls [-Wsuggest-final-types]
>zmq/zmqnotificationinterface.h:16:7: warning: Declaring type 'struct CZMQNotificationInterface' final would enable devirtualization of 4 calls [-Wsuggest-final-types]
>httpserver.cpp:42:7: warning: Declaring type 'struct HTTPWorkItem' final would enable devirtualization of 2 calls [-Wsuggest-final-types]
Tree-SHA512: 2a825fd27121ccabaacff5cde2fc8a50d1b4cc846374606caa2a71b0cd8fcb0d3c9b5b3fd342d944998610e2168048601278f8a3709cc515191a0bb2d98ba782
f4c3d2c Enable disablesafemode by default. (Gregory Maxwell)
Pull request description:
Safemode is almost useless as is-- it only triggers in limited
cases most of which aren't even concerning. There have been
several proposals to remove it. But as a simpler, safer, and
more flexible first case, simply deactivate it by default.
Anyone who wants it can re-enable and know what they've signed up for.
Tree-SHA512: f5409a3e81514c32db8eb27c7563ef85e25e56e5fc2a59eac2c30b10ec54087d982c1d3b702bedf9f3133c1f272f23805582a0f468350ba18d8b5a02bedd6401
This commit just moves a few function declarations and updates callers.
Function bodies are moved in two followup MOVEONLY commits.
This change is desirable because wallet.h/cpp are monolithic and hard to
navigate, so pulling things out and grouping together pieces of related
functionality should improve the organization.
Another proximate motivation is the wallet process separation work in
https://github.com/bitcoin/bitcoin/pull/10973, where (at least initially)
parameter parsing and fee estimation are still done in the main process rather
than the wallet process, and having functions that run in different processes
scrambled up throughout wallet.cpp is unnecessarily confusing.
90d4d89 scripted-diff: Use the C++11 keyword nullptr to denote the pointer literal instead of the macro NULL (practicalswift)
Pull request description:
Since C++11 the macro `NULL` may be:
* an integer literal with value zero, or
* a prvalue of type `std::nullptr_t`
By using the C++11 keyword `nullptr` we are guaranteed a prvalue of type `std::nullptr_t`.
For a more thorough discussion, see "A name for the null pointer: nullptr" (Sutter &
Stroustrup), http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2431.pdf
With this patch applied there are no `NULL` macro usages left in the repo:
```
$ git grep NULL -- "*.cpp" "*.h" | egrep -v '(/univalue/|/secp256k1/|/leveldb/|_NULL|NULLDUMMY|torcontrol.*NULL|NULL cert)' | wc -l
0
```
The road towards `nullptr` (C++11) is split into two PRs:
* `NULL` → `nullptr` is handled in PR #10483 (scripted, this PR)
* `0` → `nullptr` is handled in PR #10645 (manual)
Tree-SHA512: 3c395d66f2ad724a8e6fed74b93634de8bfc0c0eafac94e64e5194c939499fefd6e68f047de3083ad0b4eff37df9a8a3a76349aa17d55eabbd8e0412f140a297
Safemode is almost useless as is-- it only triggers in limited
cases most of which aren't even concerning. There have been
several proposals to remove it. But as a simpler, safer, and
more flexible first case, simply deactivate it by default.
Anyone who wants it can re-enable and know what they've signed up for.
This fixes a few cases where we should be treating a restart-after-
coinsviewdb-reset identically to a just-reset-coinsviewdb.
Thanks to @morcos for identifying the bug.
This more clearly uses fReindex vs fReset to make sure we're not
clearing our coinsdb needlessly when restarting after a reindex.
It also makes it so that restarting after shutting down mid-reindex
isn't treates specially at all during txdb loading code, as it
shouldn't be.
c0025d0 Fix segfault when shutting down before fully loading (Matt Corallo)
1385697 Order chainstate init more logically. (Matt Corallo)
ff3a219 Call RewindBlockIndex even if we're about to run -reindex-chainstate (Matt Corallo)
b0f3249 More user-friendly error message if UTXO DB runs ahead of block DB (Matt Corallo)
eda888e Fix some LoadChainTip-related init-order bugs. (Matt Corallo)
Pull request description:
This does a number of things to clean up chainstate init order,
fixing some issues as it goes:
* Order chainstate init more logically - first all of the
blocktree-related loading, then coinsdb, then
pcoinsTip/chainActive. Only create objects as needed.
* More clearly document exactly what is and isn't called in
-reindex and -reindex-chainstate both with comments noting
calls as no-ops and by adding if guards.
* Move the writing of fTxIndex to LoadBlockIndex - this fixes a
bug introduced in d6af06d68a where
InitBlockIndex was writing to fTxIndex which had not yet been
checked (because LoadChainTip hadn't yet initialized the
chainActive, which would otherwise have resulted in
InitBlockIndex being a NOP), allowing you to modify -txindex
without reindex, potentially corrupting your chainstate!
* Rename InitBlockIndex to LoadGenesisBlock, which is now a more
natural name for it. Also check mapBlockIndex instead of
chainActive, fixing a bug where we'd write the genesis block out
on every start.
* Move LoadGenesisBlock further down in init. This is a more logical
location for it, as it is after all of the blockindex-related
loading and checking, but before any of the UTXO-related loading
and checking.
* Give LoadChainTip a return value - allowing it to indicate that
the UTXO DB ran ahead of the block DB. This just provides a nicer
error message instead of the previous mysterious
assert(!setBlockIndexCandidates.empty()) error.
* Calls ActivateBestChain in case we just loaded the genesis
block in LoadChainTip, avoiding relying on the ActivateBestChain
in ThreadImport before continuing init process.
* Move all of the VerifyDB()-related stuff into a -reindex +
-reindex-chainstate if guard. It couldn't do anything useful
as chainActive.Tip() would be null at this point anyway.
Tree-SHA512: 3c96ee7ed44f4130bee3479a40c5cd99a619fda5e309c26d60b54feab9f6ec60fabab8cf47a049c9cf15e88999b2edb7f16cbe6819e97273560b201a89d90762
* Order chainstate init more logically - first all of the
blocktree-related loading, then coinsdb, then
pcoinsTip/chainActive. Only create objects as needed.
* More clearly document exactly what is and isn't called in
-reindex and -reindex-chainstate both with comments noting
calls as no-ops and by adding if guards.
* Move LoadGenesisBlock further down in init. This is a more logical
location for it, as it is after all of the blockindex-related
loading and checking, but before any of the UTXO-related loading
and checking.
* Move all of the VerifyDB()-related stuff into a -reindex +
-reindex-chainstate if guard. It couldn't do anything useful
as chainActive.Tip() would be null at this point anyway.
RewindBlockIndex works over both chainActive - disconnecting blocks
from the tip that need witness verification - and mapBlockIndex -
requiring redownload of blocks missing witness data.
It should never have been the case that the second half is skipped
if we're about to run -reindex-chainstate.
This gives LoadChainTip a return value - allowing it to indicate that
the UTXO DB ran ahead of the block DB. This just provides a nicer
error message instead of the previous mysterious
assert(!setBlockIndexCandidates.empty()) error.
This also calls ActivateBestChain in case we just loaded the genesis
block in LoadChainTip, avoiding relying on the ActivateBestChain
in ThreadImport before continuing init process.
* Move the writing of fTxIndex to LoadBlockIndex - this fixes a
bug introduced in d6af06d68a where
InitBlockIndex was writing to fTxIndex which had not yet been
checked (because LoadChainTip hadn't yet initialized the
chainActive, which would otherwise have resulted in
InitBlockIndex being a NOP), allowing you to modify -txindex
without reindex, potentially corrupting your chainstate!
* Rename InitBlockIndex to LoadGenesisBlock, which is now a more
natural name for it. Also check mapBlockIndex instead of
chainActive, fixing a bug where we'd write the genesis block out
on every start.
6b8d872 Protect SSE4 code behind a compile-time flag (Pieter Wuille)
fa9be90 Add selftest for SHA256 transform (Pieter Wuille)
c1ccb15 Add SSE4 based SHA256 (Pieter Wuille)
2991c91 Add SHA256 dispatcher (Pieter Wuille)
4d50f38 Support multi-block SHA256 transforms (Pieter Wuille)
Pull request description:
This adds an SSE4 assembly version of the SHA256 transform by Intel, and uses it at run time if SSE4 instructions are available, and use a fallback C++ implementation otherwise. Nearly every x86_64 CPU supports SSE4. The feature is only enabled when compiled with `--enable-experimental-asm`.
In order to avoid build dependencies and other complications, the original Intel YASM code was translated to GCC extended asm syntax.
This gives around a 50% speedup on the SHA256 benchmark for me.
It is based on an earlier patch by @laanwj, though only includes a single assembly version (for now), and removes the YASM dependency.
Tree-SHA512: d31c50695ceb45264291537b93c0d7497670be38edf021ca5402eaa7d4e1e0e1ae492326e28d4e93979d066168129e62d1825e0384b1b906d36f85d93dfcb43c
f4d00e6 Add a discard_rate (Alex Morcos)
b138585 Remove factor of 3 from definition of dust. (Alex Morcos)
Pull request description:
The definition of dust is redefined to remove the factor of 3.
Dust is redefined to be the value of an output such that it would
cost that value in fees to (create and) spend the output at the dust
relay rate. The previous definition was that it would cost 1/3 of the
value. The default dust relay rate is correspondingly increased to
3000 sat/kB so the actual default dust output value of 546 satoshis
for a non-segwit output remains unchanged. This commit is a refactor
only unless a dustrelayfee is passed on the commandline in which case
that number now needs to be increased by a factor of 3 to get the same
behavior. -dustrelayfee is a hidden command line option.
Note: It's not exactly a refactor due to edge case changes in rounding
as evidenced by the required change to the unit test.
A discard_rate is added which defaults to 10,000 sat/kB
Any change output which would be dust at the discard_rate you are
willing to discard completely and add to fee (as well as continuing to
pay the fee that would have been needed for creating the change)
This would be a nice addition for 0.15 and I think will remain useful for 0.16 with the new coin selection algorithms in discussion, but its not crucial.
It does add translation strings, but we could (should?) avoid that by hiding the option
Tree-SHA512: 5b6f655354d0ab6b8b6cac1e8d1fe3136d10beb15c6d948fb15bfb105155a9d03684c6240624039b3eed6428b7e60e54216cc8b2f90c4600701e39f646284a9b
Alternative to #10818, alternative solution to #10815.
After this change: All the AppInit steps before and inclusive
AppInitLockDataDirectory must not have Shutdown() called in case of
failure. Only when AppInitMain fails, Shutdown should be called.
Changes the GUI and bitcoind code to consistently do this.
This redefines dust to be the value of an output such that it would
cost that value in fees to (create and) spend the output at the dust
relay rate. The previous definition was that it would cost 1/3 of the
value. The default dust relay rate is correspondingly increased to
3000 sat/kB so the actual default dust output value of 546 satoshis
for a non-segwit output remains unchanged. This commit is a refactor
only unless a dustrelayfee is passed on the commandline in which case
that number now needs to be increased by a factor of 3 to get the same
behavior. -dustrelayfee is a hidden command line option.
Note: It's not exactly a refactor due to edge case changes in rounding
as evidenced by the required change to the unit test.
959dd8781 Avoid printing incorrect block indexing time due to uninitialized variable (practicalswift)
Tree-SHA512: a76e43c3ffa734ed5c7eadf363f345f268aa0e6ce775aba8f856fe3bbc82f240dc7c734c5ca3ac500a12eb41fae00623413e79f484d5acf809b6e400851d771d
Note that the CScheduler thread cant be running at this point,
it has already been stopped with the rest of the init threadgroup.
Thus, just calling any remaining loose callbacks during Shutdown()
is sane.
Fixes:
init.cpp: In function ‘bool AppInitMain(boost::thread_group&, CScheduler&)’:
init.cpp:1499:56: warning: ‘nStart’ may be used uninitialized in this function [-Wmaybe-uninitialized]
LogPrintf(" block index %15dms\n", GetTimeMillis() - nStart);
^
176c021 [qa] Test non-atomic chainstate writes (Suhas Daftuar)
d6af06d Dont create pcoinsTip until after ReplayBlocks. (Matt Corallo)
eaca1b7 Random db flush crash simulator (Pieter Wuille)
0580ee0 Adapt memory usage estimation for flushing (Pieter Wuille)
013a56a Non-atomic flushing using the blockchain as replay journal (Pieter Wuille)
b3a279c [MOVEONLY] Move LastCommonAncestor to chain (Pieter Wuille)
Tree-SHA512: 47ccc62303f9075c44d2a914be75bd6969ff881a857a2ff1227f05ec7def6f4c71c46680c5a28cb150c814999526797dc05cf2701fde1369c06169f46eccddee
This requires that we not access pcoinsTip in InitBlockIndex's
FlushStateToDisk (so we just skip it until later in AppInitMain)
and the LoadChainTip in LoadBlockIndex (which there is already one
later in AppinitMain, after ReplayBlocks, so skipping it there is
fine).
Includes some simplifications by Suhas Daftuar and Pieter Wuille.
5a9b508 [trivial] Add end of namespace comments (practicalswift)
Tree-SHA512: 92b0fcae4d1d3f4da9e97569ae84ef2d6e09625a5815cd0e5f0eb6dd2ecba9852fa85c184c5ae9de5117050330ce995e9867b451fa8cd5512169025990541a2b
Part of a series of changes to clean up the instantiation of connman
by decoupling the command line arguments.
We also now abort with an error when explicit binds are set with
-listen=0.
cb24c85 Use rdrand as entropy source on supported platforms (Pieter Wuille)
Tree-SHA512: c42eaa01a14e6bc097c70b6bf8540d61854c2f76cb32be69c2a3c411a126f7b4bf4a4486e4493c4cc367cc689319abde0d4adb799d29a54fd3e81767ce0766fc
Use case: TryCreateDirectory(GetDataDir() / "blocks" / "index") would
fail if the blocks directory was not explicitly created before.
The line that did so was in a weird location and could be removed as a
result.
3ff1fa8 Use override keyword on CCoinsView overrides (Russell Yanofsky)
24e44c3 Don't return stale data from CCoinsViewCache::Cursor() (Russell Yanofsky)
Tree-SHA512: 08699dae0925ffb9c018f02612ac6b7eaf73ec331e2f4f934f1fe25a2ce120735fa38596926e924897c203f7470e99f0a99cf70d2ce31ff428b105e16583a861
c237bd7 wallet: Update formatting (Luke Dashjr)
9cbe8c8 wallet: Forbid -salvagewallet, -zapwallettxes, and -upgradewallet with multiple wallets (Luke Dashjr)
a2a5f3f wallet: Base backup filenames on original wallet filename (Luke Dashjr)
b823a4c wallet: Include actual backup filename in recovery warning message (Luke Dashjr)
84dcb45 Bugfix: wallet: Fix warningStr, errorStr argument order (Luke Dashjr)
008c360 Wallet: Move multiwallet sanity checks to CWallet::Verify, and do other checks on all wallets (Luke Dashjr)
0f08575 Wallet: Support loading multiple wallets if -wallet used more than once (Luke Dashjr)
b124cf0 Wallet: Replace pwalletMain with a vector of wallet pointers (Luke Dashjr)
19b3648 CWalletDB: Store the update counter per wallet (Luke Dashjr)
74e8738 Bugfix: ForceSetArg should replace entr(ies) in mapMultiArgs, not append (Luke Dashjr)
23fb9ad wallet: Move nAccountingEntryNumber from static/global to CWallet (Luke Dashjr)
9d15d55 Bugfix: wallet: Increment "update counter" when modifying account stuff (Luke Dashjr)
f28eb80 Bugfix: wallet: Increment "update counter" only after actually making the applicable db changes to avoid potential races (Luke Dashjr)
Tree-SHA512: 23f5dda58477307bc07997010740f1dc729164cdddefd2f9a2c9c7a877111eb1516d3e2ad4f9b104621f0b7f17369c69fcef13d28b85cb6c01d35f09a8845f23
CCoinsViewCache doesn't actually support cursor iteration returning the
current contents of the cache, so raise an error when the cursor method is
called instead of returning a cursor that iterates over stale data.
Also update the gettxoutsetinfo RPC which was relying on the old behavior to be
explicit about which view it is returning data about.
This adds a new CuckooCache in validation, caching whether all of a
transaction's scripts were valid with a given set of script flags.
Unlike previous attempts at caching an entire transaction's
validity, which have nearly universally introduced consensus
failures, this only caches the validity of a transaction's
scriptSigs. As these are pure functions of the transaction and
data it commits to, this should be much safer.
This is somewhat duplicative with the sigcache, as entries in the
new cache will also have several entries in the sigcache. However,
the sigcache is kept both as ATMP relies on it and because it
prevents malleability-based DoS attacks on the new higher-level
cache. Instead, the -sigcachesize option is re-used - cutting the
sigcache size in half and using the newly freed memory for the
script execution cache.
Transactions which match the script execution cache never even have
entries in the script check thread's workqueue created.
Note that the cache is indexed only on the script execution flags
and the transaction's witness hash. While this is sufficient to
make the CScriptCheck() calls pure functions, this introduces
dependancies on the mempool calculating things such as the
PrecomputedTransactionData object, filling the CCoinsViewCache, etc
in the exact same way as ConnectBlock. I belive this is a reasonable
assumption, but should be noted carefully.
In a rather naive benchmark (reindex-chainstate up to block 284k
with cuckoocache always returning true for contains(),
-assumevalid=0 and a very large dbcache), this connected blocks
~1.7x faster.
589827975 scripted-diff: various renames for per-utxo consistency (Pieter Wuille)
a5e02bc7f Increase travis unit test timeout (Pieter Wuille)
73de2c1ff Rename CCoinsCacheEntry::coins to coin (Pieter Wuille)
119e552f7 Merge CCoinsViewCache's GetOutputFor and AccessCoin (Pieter Wuille)
580b02309 [MOVEONLY] Move old CCoins class to txdb.cpp (Pieter Wuille)
8b25d2c0c Upgrade from per-tx database to per-txout (Pieter Wuille)
b2af357f3 Reduce reserved memory space for flushing (Pieter Wuille)
41aa5b79a Pack Coin more tightly (Pieter Wuille)
97072d668 Remove unused CCoins methods (Pieter Wuille)
ce23efaa5 Extend coins_tests (Pieter Wuille)
508307968 Switch CCoinsView and chainstate db from per-txid to per-txout (Pieter Wuille)
4ec0d9e79 Refactor GetUTXOStats in preparation for per-COutPoint iteration (Pieter Wuille)
13870b56f Replace CCoins-based CTxMemPool::pruneSpent with isSpent (Pieter Wuille)
05293f3cb Remove ModifyCoins/ModifyNewCoins (Pieter Wuille)
961e48397 Switch tests from ModifyCoins to AddCoin/SpendCoin (Pieter Wuille)
8b3868c1b Switch CScriptCheck to use Coin instead of CCoins (Pieter Wuille)
c87b957a3 Only pass things committed to by tx's witness hash to CScriptCheck (Matt Corallo)
f68cdfe92 Switch from per-tx to per-txout CCoinsViewCache methods in some places (Pieter Wuille)
000391132 Introduce new per-txout CCoinsViewCache functions (Pieter Wuille)
bd83111a0 Optimization: Coin&& to ApplyTxInUndo (Pieter Wuille)
cb2c7fdac Replace CTxInUndo with Coin (Pieter Wuille)
422634e2f Introduce Coin, a single unspent output (Pieter Wuille)
7d991b55d Store/allow tx metadata in all undo records (Pieter Wuille)
c3aa0c119 Report on-disk size in gettxoutsetinfo (Pieter Wuille)
d34242430 Remove/ignore tx version in utxo and undo (Pieter Wuille)
7e0032290 Add specialization of SipHash for 256 + 32 bit data (Pieter Wuille)
e484652fc Introduce CHashVerifier to hash read data (Pieter Wuille)
f54580e7e error() in disconnect for disk corruption, not inconsistency (Pieter Wuille)
e66dbde6d Add SizeEstimate to CDBBatch (Pieter Wuille)
Tree-SHA512: ce1fb1e40c77d38915cd02189fab7a8b125c7f44d425c85579d872c3bede3a437760997907c99d7b3017ced1c2de54b2ac7223d99d83a6658fe5ef61edef1de3
This patch makes several related changes:
* Changes the CCoinsView virtual methods (GetCoins, HaveCoins, ...)
to be COutPoint/Coin-based rather than txid/CCoins-based.
* Changes the chainstate db to a new incompatible format that is also
COutPoint/Coin based.
* Implements reconstruction code for hash_serialized_2.
* Adapts the coins_tests unit tests (thanks to Russell Yanofsky).
A side effect of the new CCoinsView model is that we can no longer
use the (unreliable) test for transaction outputs in the UTXO set
to determine whether we already have a particular transaction.
38bc1ec Make more json-like output from estimaterawfee (Alex Morcos)
2d2e170 Comments and improved documentation (Alex Morcos)
ef589f8 minor cleanup: remove unnecessary variable (Alex Morcos)
3ee76d6 Introduce a scale factor (Alex Morcos)
5f1f0c6 Historical block span (Alex Morcos)
aa19b8e Clean up fee estimate debug printing (Alex Morcos)
10f7cbd Track first recorded height (Alex Morcos)
3810e97 Rewrite estimateSmartFee (Alex Morcos)
c7447ec Track failures in fee estimation. (Alex Morcos)
4186d3f Expose estimaterawfee (Alex Morcos)
2681153 minor refactor: explicitly track start of new bucket range and don't update curNearBucket on final loop. (Alex Morcos)
1ba43cc Make EstimateMedianVal smarter about small failures. (Alex Morcos)
d3e30bc Refactor to update moving average on fly (Alex Morcos)
e5007ba Change parameters for fee estimation and estimates on all 3 time horizons. (Alex Morcos)
c0a273f Change file format for fee estimates. (Alex Morcos)
Tree-SHA512: 186e7508d86a1f351bb656edcd84ee9091f5f2706331eda9ee29da9c8eb5bf67b8c1f2abf6662835560e7f613b1377099054f20767f41ddcdbc89c4f9e78946d
c1082a7 Chainparams: Use the factory for pow tests (Jorge Timón)
2351a06 Chainparams: Get rid of CChainParams& Params(std::string) (Jorge Timón)
f87f362 Chainparams: Use a regular factory for creating chainparams (Jorge Timón)
Tree-SHA512: 359c8a2a1bc9d02db7856d02810240ada28048ac088f878b575597a7255cdb0ffdd1a647085ee67a34c6a7e7ed9e6cfdb61240cf6e75139619b640dbb096072c
a750d77 Add tests for mempool persistence (John Newbery)
91c91e1 Control mempool persistence using a command line parameter. (John Newbery)
Tree-SHA512: 157d01cefd1903b8bfc5cbab42a3cc5e9c1094179bf4b64b3d34c0d4d9b976d593755bfea5c41c631cb758e1de17c6c2058c130d487d20560b7c0bafcddfa520
f110272 Remove `namespace fs=fs` (Wladimir J. van der Laan)
75594bd torcontrol: Use fs::path instead of std::string for private key path (Wladimir J. van der Laan)
2a5f574 Use fsbridge for fopen and freopen (Wladimir J. van der Laan)
bac5c9c Replace uses of boost::filesystem with fs (Wladimir J. van der Laan)
7d5172d Replace includes of boost/filesystem.h with fs.h (Wladimir J. van der Laan)
19e36bb Add fs.cpp/h (Wladimir J. van der Laan)
Tree-SHA512: 2c34f059dfa6850b9323f3389e9090a6b5f839a457a2960d182c2ecfafd9883c956f5928bb796613402d3aad68ebc78259796a7a313f4a6cfa98aaf507a66842
cd7f394 initialize flag variable to 0 (and continue if GetLogCategory() fails) (John Newbery)
Tree-SHA512: d0f2653bd0e71ed763220cb08d3a5335c5bdfe2f54ff7f9302d97f3265d7aa7f57606fe416a61aaac1535dbb046d0fb40a61f5a9d5cf234b042268e00ee7679d
This changes the logging categories to boolean flags instead of strings.
This simplifies the acceptance testing by avoiding accessing a scoped
static thread local pointer to a thread local set of strings. It
eliminates the only use of boost::thread_specific_ptr outside of
lockorder debugging.
This change allows log entries to be directed to multiple categories
and makes it easy to change the logging flags at runtime (e.g. via
an RPC, though that isn't done by this commit.)
It also eliminates the fDebug global.
Configuration of unknown logging categories now produces a warning.
f885b67 refactor: Make rest.cpp dependency on `*toJSON` in `blockchain.cpp` explicit (Wladimir J. van der Laan)
8d8f28d refactor: Move RPCNotifyBlockChange out of `rpc/server.h` (Wladimir J. van der Laan)
e6dcfee refactor: Move GetDifficulty out of `rpc/server.h` (Wladimir J. van der Laan)
Tree-SHA512: fc2656611d18442f2fddba5ac1554d958151f6785c2039afdfc36735d7e71592d9686ff6cc7b2ad95180071d7514470e62c52d697c5a1e88f851bddaf5942edb
0fb2887 Move several relay options into the Relay help group (Jameson Lopp)
Tree-SHA512: 31fdfd8c741adb6fe6806a28955f0fbbc9360b1d8c3d8a28684794822f1b3231fffab93357357d986b81a4532c9eeabb79e5ede9378ff3ad8930ceb6588d9eb6
