Commit graph

1391 commits

Author SHA1 Message Date
James O'Beirne
5693530685 refactor: have CCoins* data managed under CChainState
This change encapsulates UTXO set data within CChainState instances, removing
global data `pcoinsTip` and `pcoinsviewdb`. This is necessary if we want to
maintain multiple chainstates with their own rendering of the UTXO set.

We introduce a class CoinsViews which consolidates the construction of a
CCoins* hierarchy. Construction of its various pieces (db, coinscatcher,
in-memory cache) is split up so that we avoid flushing bad state to disk if
startup is interrupted.

We also introduce `CChainState::CanFlushToDisk()` which tells us when it is
safe to flush the chainstate based on this partial construction.

This commit could be broken into smaller pieces, but it would require more
ephemeral diffs to, e.g., temporarily change CCoinsViewDB's constructor
invocations.

Other changes:

- A parameter has been added to the CCoinsViewDB constructor that allows the
  name of the corresponding leveldb directory to be specified.

Thanks to Russell Yanofsky and Marco Falke for helpful feedback.
2019-08-15 11:04:10 -04:00
Hennadii Stepanov
dde80c272a
Use ArgsManager::NETWORK_ONLY flag 2019-07-27 22:51:58 +03:00
Hennadii Stepanov
fb4b9f9e3b
scripted-diff: Use ArgsManager::DEBUG_ONLY flag
-BEGIN VERIFY SCRIPT-
sed -i 's/unsigned int flags, const bool debug_only,/unsigned int flags,/' src/util/system.h src/util/system.cpp
sed -i 's/ArgsManager::NONE, debug_only/flags, false/' src/util/system.cpp
sed -i 's/arg.second.m_debug_only/(arg.second.m_flags \& ArgsManager::DEBUG_ONLY)/' src/util/system.cpp
sed -i 's/ArgsManager::ALLOW_ANY, true, OptionsCategory::/ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::/' $(git grep --files-with-matches 'AddArg(' src)
sed -i 's/ArgsManager::ALLOW_ANY, false, OptionsCategory::/ArgsManager::ALLOW_ANY, OptionsCategory::/' $(git grep --files-with-matches 'AddArg(' src)
-END VERIFY SCRIPT-
2019-07-27 15:05:14 +03:00
Hennadii Stepanov
1b4b9422ca
scripted-diff: Use Flags enum in AddArg()
-BEGIN VERIFY SCRIPT-
sed -i 's/const bool debug_only,/unsigned int flags, &/' src/util/system.h src/util/system.cpp
sed -i -E 's/(true|false), OptionsCategory::/ArgsManager::ALLOW_ANY, &/' $(git grep --files-with-matches 'AddArg(' src)
-END VERIFY SCRIPT-
2019-07-27 15:05:14 +03:00
Hennadii Stepanov
753f7cccce
scripted-diff: Make translation bilingual
-BEGIN VERIFY SCRIPT-
sed -i 's/inline std::string _(const char\* psz)/inline bilingual_str _(const char\* psz)/' src/util/translation.h
sed -i 's/return G_TRANSLATION_FUN ? (G_TRANSLATION_FUN)(psz) : psz;/return bilingual_str{psz, G_TRANSLATION_FUN ? (G_TRANSLATION_FUN)(psz) : psz};/' src/util/translation.h
sed -i 's/\b_("\([^"]\|\\"\)*")/&.translated/g' $(git grep --files-with-matches '\b_("' src)
echo Hard cases - multiline strings.
sed -i 's/"Visit %s for further information about the software.")/&.translated/g' src/init.cpp
sed -i "s/\"Only rebuild the block database if you are sure that your computer's date and time are correct\")/&.translated/g" src/init.cpp
sed -i 's/" restore from a backup.")/&.translated/g' src/wallet/db.cpp
sed -i 's/" or address book entries might be missing or incorrect.")/&.translated/g' src/wallet/wallet.cpp
echo Special case.
sed -i 's/_(COPYRIGHT_HOLDERS)/&.translated/' src/util/system.cpp test/lint/lint-format-strings.py
-END VERIFY SCRIPT-
2019-07-24 16:33:20 +03:00
Hennadii Stepanov
0b86e517ad
Refactor out translation.h
This is a prerequisite for introducing bilingual error messages.
Note: #includes are arranged by clang-format-diff.py script.
2019-07-24 16:32:53 +03:00
fanquake
848f245d04
Merge #16355: refactor: move CCoinsViewErrorCatcher out of init.cpp
4f050b91c7 move-onlyish: move CCoinsViewErrorCatcher out of init.cpp (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal

  ---

  This change moves `CCoinsViewErrorCatcher` out of `init` and into `coins` so that it can later be included in [a `CoinsView` instance](91284964ef (diff-349fbb003d5ae550a2e8fa658e475880R504)) under `CChainState`.

  Instead of hardcoding read failure behavior that has knowledge of qt, it accepts error callbacks via `AddReadErrCallback()`.

ACKs for top commit:
  dongcarl:
    re-ACK 4f050b91c7
  ryanofsky:
    utACK 4f050b91c7. Only change since last review is fixing const.

Tree-SHA512: eaba21606d15d2b8d0e3db7cec57779ce181af953db1ef4af80a0bc1dfb57923d0befde9d61b7be55c32224744f7fb6bd47d4e4c72f3ccfe6eaf0f4ae3765c17
2019-07-23 12:42:24 +08:00
James O'Beirne
4f050b91c7 move-onlyish: move CCoinsViewErrorCatcher out of init.cpp
and into coins.cpp. This move is necessary so that we can later include a
CCoinsViewErrorCatcher instance under CChainState.

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2019-07-21 21:00:31 -04:00
MarcoFalke
24dbcf3808
Merge #15891: test: Require standard txs in regtest by default
fa89badf88 test: Require standard txs in regtest (MarcoFalke)
fa9b419160 test: Add test that mainnet requires standard txs (MarcoFalke)
fa613ca0a8 chainparams: Remove unused fMineBlocksOnDemand (MarcoFalke)

Pull request description:

  I don't see a reason why regtest should allow non-standard txs, as it makes testing mainnet behaviour such as #15846 unnecessarily hard and unintuitive.

  Of course, testnet policy remains unchanged to allow propagation of non-standard txs.

ACKs for top commit:
  ajtowns:
    ACK fa89badf88

Tree-SHA512: c4c675affb054868850bd2683aa07f4c741a448cbacb2ea8334191e105f426b0790fe6a468be61e9c5880d24154f7bf1c7075051697172dce92180c1bc3a1c90
2019-07-16 16:10:17 -04:00
Wladimir J. van der Laan
8f604361eb
Merge #16194: refactor: share blockmetadata with BlockManager
682a1d0f20 refactoring: remove mapBlockIndex global (James O'Beirne)
55d525ab90 refactoring: make pindexBestInvalid internal to validation.cpp (James O'Beirne)
4ed55dfcd7 refactoring: add block_index_candidates arg to LoadBlockIndex (James O'Beirne)
613c46fe9e refactoring: move block metadata structures into BlockManager (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal

  ---

  Under an assumeutxo model, we have multiple CChainState instances in use at once in order to support background validation. Currently, each CChainState instance has its own mapBlockIndex, a collection of linked block headers, in addition to a few other data structures that are related to maintenance of the block tree but not necessarily to any given chainstate.

  In order to avoid duplicating this data across chainstates, this change moves chainstate-agnostic block metadata (and related behavior) into a class, `BlockManager`. Chainstates are parameterized with a reference to a blockmanager instance and in practice they share the same instance.

  Most of this change is conceptually move-only, though the diff is somewhat muddled. The first commit can be reviewed slightly more easily with `--color-moved=dimmed_zebra`. Admittedly, that commit is pretty unwieldy; I tried to split it up after the fact with `git add --patch`, but that was difficult because of git's inability to split hunks past a certain point. Some of the moves also ended up being obscured when done over separate commits.

ACKs for top commit:
  MarcoFalke:
    ACK 682a1d0f20
  ryanofsky:
    utACK 682a1d0f20, only changes since last review were rebase and fixing conflict on a moved line
  ariard:
    utACK 682a1d0. Most of the changes are move-only, with main problem being to avoid creating circular dependencies between `BlockManager` and `CChainState`. Tested, comments are mostly nits, feel free to ignore them

Tree-SHA512: 738d8d06539ba53acf4bd2d48ae000473e645bbc4e63d798d55d247a4d5a4f781b73538ed590f6407be9ab402ea9d395570ea20bff0a4b9ce747bcc1600c5108
2019-07-16 18:48:07 +02:00
MarcoFalke
4882040182
Merge #16291: gui: Stop translating PACKAGE_NAME
fa64b947bb util: No translation of `Bitcoin Core` in the copyright (MarcoFalke)
fab85208f6 qt: Run «make translate» in ./src/ (MarcoFalke)
fabe87d2c9 scripted-diff: Avoid passing PACKAGE_NAME for translation (MarcoFalke)
fa5e9f157e build: Stop translating PACKAGE_NAME (MarcoFalke)

Pull request description:

  Generally the package name is not translated, but the package description is.

  E.g. `GIMP` or `Firefox` are always called that way regardless of the system language. However, "`Firefox` webbrowser" or "`GIMP` image manipulation program" are translated.

ACKs for top commit:
  hebasto:
    ACK fa64b947bb, I have not tested the code, but I have reviewed it and it looks OK, I agree it can be merged.

Tree-SHA512: 626f811531182d0ba0ef1044930d32726773349bcb49b10261288a86ee6b80a183db30a87d817d5b0d501fad058ac22d6272311716b4f5a154f17c6f391a5a1a
2019-07-08 13:39:59 -04:00
James O'Beirne
682a1d0f20 refactoring: remove mapBlockIndex global
in lieu of ::BlockIndex().
2019-07-08 11:33:13 -04:00
Sjors Provoost
976b034b13
[build]: use #if HAVE_SYSTEM instead of defined(HAVE_SYSTEM) 2019-07-05 18:32:05 +02:00
Wladimir J. van der Laan
8c69fae944
Merge #15457: Check std::system for -[alert|block|wallet]notify
f874e14cd3 [build]: check std::system for -[alert|block|wallet]notify (Sjors Provoost)
cc3ad56ff2 [build] MSVC: set HAVE_SYSTEM for desktop apps (Sjors Provoost)
c1c91bb78d [build] detect std::system or ::wsystem (Sjors Provoost)

Pull request description:

  Platforms such as iOs and Universal Windows Platform do not support launching a process through system().

ACKs for top commit:
  laanwj:
    code review ACK f874e14cd3

Tree-SHA512: 16bb4a8fa1896046ccb22a46c8985e1aa45f5b11ecf5539eb2299e9a58f1a5b085c0c12cb6939c7493d93abce7e84fadcbfc73374c887db63da6d00c08aa476d
2019-07-05 17:33:33 +02:00
MarcoFalke
fabe87d2c9
scripted-diff: Avoid passing PACKAGE_NAME for translation
-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended -e 's/\<\w+(::\w+)?\(PACKAGE_NAME\)/PACKAGE_NAME/g' $(git grep -l --extended-regexp '\<\w+(::\w+)?\(PACKAGE_NAME\)' src)
-END VERIFY SCRIPT-
2019-06-26 11:01:57 -04:00
MarcoFalke
fa89badf88
test: Require standard txs in regtest 2019-06-21 16:45:16 -04:00
MarcoFalke
0853d8d2fd
Merge #16112: util: Log early messages
faa2a47cd7 logging: Add threadsafety comments (MarcoFalke)
0b282f9b00 Log early messages with -printtoconsole (Anthony Towns)
412987430c Replace OpenDebugLog() with StartLogging() (Anthony Towns)

Pull request description:

  Early log messages are dropped on the floor and they'd never make it to the console or debug log. This can be tested by running the test included in this pull request without re-compiling the `bitcoind`.

  Fix that by buffering early messages and flushing them as soon as all logging options have been initialized and logging has been started.

  This pull request is identical to  "Log early messages with -printtoconsole" (#13088)  by **ajtowns**, with the following changes:
  * Rebased
  * Added docstrings for `m_buffering` and `StartLogging`
  * Switch `CCriticalSection` (aka `RecursiveMutex`) to just `Mutex` in the last commit
  * Added tests

  Fixes #16098
  Fixes #13157
  Closes #13088

ACKs for commit faa2a4:
  ajtowns:
    utACK faa2a47cd7
  hebasto:
    ACK faa2a47cd7
  kristapsk:
    ACK faa2a47cd7 (ran added functional test before / after recompiling, didn't do additional testing)

Tree-SHA512: 685e2882642fe2a43ce171d42862582dadb840d03cda8236a994322c389ca2a1f3f431b179b2726c155c61793543bb340c568a5455d97f8b83bc7d307a85d387
2019-06-18 12:32:32 -04:00
MarcoFalke
e2182b02b5
Merge #16171: Remove -mempoolreplacement to prevent needless block prop slowness.
8053e5cdad Remove -mempoolreplacement to prevent needless block prop slowness. (Matt Corallo)

Pull request description:

  At this point there is no reasonable excuse to disable opt-in RBF,
  and, unlike when this option was added, there are now significant
  issues created when disabling it (in the form of compact block
  reconstruction failures). Further, it breaks a lot of modern wallet
  behavior.

  This removes an option that is:

  * (a) only useful when a large portion of (other) miners enforce it as well
  * (b) is detrimental to everyone (income for miners, RBF notifications for others) who uses it individually otherwise
  * (c) is effectively unused
  * (d) is often confused with disabling RBF (rather than just remaining stubbornly unaware of it while the rest of the network lets it through)

ACKs for commit 8053e5:
  practicalswift:
    utACK 8053e5cdad
  promag:
    Deprecation would save from unlikely rantings, still ACK 8053e5c.
  jtimon:
    utACK 8053e5cdad
  ajtowns:
    ACK 8053e5cdad -- quick code review, checked tests work
  MarcoFalke:
    ACK 8053e5cdad

Tree-SHA512: 01aee8905b2487fc38a3a86649d422d2d2345bc60f878889ebda4b8680783e1f1a97c2000c27ef086719501be2abc2911b2039a259a5e5c04f3b24ff02b0427e
2019-06-18 10:04:14 -04:00
MarcoFalke
fa8f195195
Replace remaining fprintf with tfm::format manually 2019-06-13 11:46:38 -04:00
Matt Corallo
8053e5cdad Remove -mempoolreplacement to prevent needless block prop slowness.
At this point there is no reasonable excuse to disable opt-in RBF,
and, unlike when this option was added, there are now significant
issues created when disabling it (in the form of compact block
reconstruction failures). Further, it breaks a lot of modern wallet
behavior.
2019-06-08 09:32:33 -04:00
MarcoFalke
d0f81a96d9
Merge #16129: refactor: Remove unused includes
67f4e9c522 Include core_io.h from core_read.cpp (practicalswift)
eca9767673 Make reasoning about dependencies easier by not including unused dependencies (practicalswift)

Pull request description:

  Make reasoning about dependencies easier by not including unused dependencies.

  Please note that the removed headers are _not_ "transitively included" by other still included headers. Thus the removals are real.

  As an added bonus this change means less work for the preprocessor/compiler. At least 51 393 lines of code no longer needs to be processed:

  ```
  $ git diff -u HEAD~1 | grep -E '^\-#include ' | cut -f2 -d"<" | cut -f1 -d">" | \
        sed 's%^%src/%g' | xargs cat | wc -l
  51393
  ```

  Note that 51 393 is the lower bound: the real number is likely much higher when taking into account transitively included headers :-)

ACKs for commit 67f4e9:

Tree-SHA512: 0c8868aac59813f099ce53d5307eed7962dd6f2ff3546768ef9e5c4508b87f8210f1a22c7e826c3c06bebbf28bdbfcf1628ed354c2d0fdb9a31a42cefb8fdf13
2019-06-06 16:41:40 +02:00
Sjors Provoost
f874e14cd3
[build]: check std::system for -[alert|block|wallet]notify
Platforms such as iOs do not support launching a process
through system().
2019-06-06 11:54:26 +02:00
Wladimir J. van der Laan
5d37c1bde0
Merge #15976: refactor: move methods under CChainState (pt. 1)
403e677c9 refactoring: IsInitialBlockDownload -> CChainState (James O'Beirne)
3ccbc376d refactoring: FlushStateToDisk -> CChainState (James O'Beirne)
4d6688603 refactoring: introduce ChainstateActive() (James O'Beirne)
d7c97edee move-only: make the CChainState interface public (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal

  ---

  This changeset starts moving functionality intimately related to CChainState into methods. Parameterizing these functions by a particular CChainState is necessary for the use of multiple chainstates simultaneously (e.g. for asynchronous background validation).

  In this change, we
  - make the CChainState interface public - since other units will start to invoke its methods directly,
  - introduce `::ChainstateActive()`, the CChainState equivalent for `::ChainActive()`,
  - and move `IsInitialBlockDownload()` and `FlushStateToDisk()` into methods on CChainState.

  Independent of assumeutxo, these changes better encapsulate chainstate behavior and allow easier use from a testing context.

  There are more methods that we'll move in the future, but they require other substantial changes (i.e. moving ownership of the `CCoinsView*` hierarchy into CChainState) so we'll save them for future PRs.

  ---

  The first move-only commit is most easily reviewed with `git diff ... --color-moved=dimmed_zebra`.

ACKs for commit 403e67:
  Empact:
    utACK 403e677c9e no need to address my nits herein
  Sjors:
    utACK 403e677
  ryanofsky:
    utACK 403e677c9e. Only change since previous review is removing global state comment as suggested.
  MarcoFalke:
    utACK 403e677c9e, though the diff still seems a bit bloated with some unnecessary changes in the second commit.
  promag:
    utACK 403e677 and rebased with current [master](c7cfd20a7).

Tree-SHA512: 6fcf260bb2dc201361170c0b4547405366f5f331fcc3a2bac29b24442814b7b244ca1b58aac5af716885f9a130c343b544590dff780da0bf835c7c5b3ccb2257
2019-06-05 11:56:23 +02:00
practicalswift
eca9767673 Make reasoning about dependencies easier by not including unused dependencies 2019-06-02 17:15:23 +02:00
Anthony Towns
0b282f9b00 Log early messages with -printtoconsole
This ensures log messages prior to StartLogging() are replayed to
the console as well as to the debug log file.
2019-05-28 14:26:42 -04:00
Anthony Towns
412987430c Replace OpenDebugLog() with StartLogging()
StartLogging() is used to mark the start of logging generically, whether
using -printtoconsole or -debuglogfile.
2019-05-28 14:26:06 -04:00
practicalswift
fb434159d1 Remove global symbols: Avoid using the global namespace if possible
Rename CCriticalSection to RecursiveMutex (both are AnnotatedMixin<std::recursive_mutex>)

```
$ git grep -E '(typedef|using).*(CCriticalSection|RecursiveMutex)'
src/sync.h:using RecursiveMutex = AnnotatedMixin<std::recursive_mutex>;
src/sync.h:typedef AnnotatedMixin<std::recursive_mutex> CCriticalSection;
```
2019-05-25 23:23:11 +02:00
Jonas Schnelli
7110d455eb
Merge #12980: Allow quicker shutdowns during LoadBlockIndex()
af5fa82b6 Allow quicker shutdowns during LoadBlockIndex() (Jonas Schnelli)

Pull request description:

ACKs for commit af5fa8:
  promag:
    utACK af5fa82b67.
  practicalswift:
    utACK af5fa82b67

Tree-SHA512: 1c64dcc5d8a9d3411553257cd5a598dcd29be981660e5bca9283c1d957dc56798abcf41d9969cd573088137597a23e48e62a8c476c463d3f176b86a10048f47b
2019-05-19 10:43:32 +02:00
Wladimir J. van der Laan
df7addc4c6
Merge #15990: Add tests and documentation for blocksonly
fa8ced32a6 doc: Mention blocksonly in reduce-traffic.md, unhide option (MarcoFalke)
fa320de79f test: Add test for p2p_blocksonly (MarcoFalke)
fa3872e7b4 test: Format predicate source as multiline on error (MarcoFalke)
fa1dce7329 net: Rename ::fRelayTxes to ::g_relay_txes (MarcoFalke)

Pull request description:

  This is de-facto no longer hidden

ACKs for commit fa8ced:
  jamesob:
    utACK fa8ced32a6

Tree-SHA512: 474fbdee6cbd035ed9068a066b6056c1f909ec7520be0417820fcd1672ab3069b53f55c5147968978d9258fd3a3933fe1a9ef8e4f6e14fb6ebbd79701a0a1245
2019-05-16 19:05:24 +02:00
James O'Beirne
3ccbc376dd refactoring: FlushStateToDisk -> CChainState
Also renames global methods for clarity:

- ::FlushStateToDisk() -> CChainState::ForceFlushStateToDisk()
  - This performs an unconditional flush.

- ::PruneAndFlush() -> CChainState::PruneAndFlush()
2019-05-16 09:06:54 -04:00
MarcoFalke
667a861741
Merge #14364: doc: Clarify -blocksdir usage
ccc27bdcd2 doc: Clarify -blocksdir usage (Daniel McNally)

Pull request description:

  This PR attempts to clarify and correct the `-blocksdir` argument description and default value. `-blocksdir` does not refer to the full path to the actual `blocks` directory, but rather the root/parent directory which contains the `blocks` directory. Accordingly, the default value is `<datadir>` and not `<datadir>/blocks` - this behavior of defaulting to the datadir can also be seen in init.cpp:

  ```cpp
      if (gArgs.IsArgSet("-blocksdir")) {
          path = fs::system_complete(gArgs.GetArg("-blocksdir", ""));
          if (!fs::is_directory(path)) {
              path = "";
              return path;
          }
      } else {
          path = GetDataDir(false);
      }
  ```

  It also attempts to clarify that only the `.dat` files containing block data are impacted by `-blocksdir`, not the index files.

  I believe this would close #12828.

ACKs for commit ccc27b:
  hebasto:
    utACK ccc27bdcd2

Tree-SHA512: 7b65f66b0579fd56e8c8cd4f9f22d6af56181817762a68deccd7fca51820ad82d9a0c48f5f1f012e746c67bcdae7af4555fad867cb620a9ca538d465c9d86c2b
2019-05-13 12:46:03 -04:00
MarcoFalke
fa8ced32a6
doc: Mention blocksonly in reduce-traffic.md, unhide option 2019-05-13 10:44:46 -04:00
MarcoFalke
fa1dce7329
net: Rename ::fRelayTxes to ::g_relay_txes
This helps to distinguish it from CNode::fRelayTxes and avoid bugs like
425278d17b
2019-05-09 09:10:53 -04:00
MarcoFalke
08788ce170
Merge #15890: Doc: remove text about txes always relayed from -whitelist
e0bb279999 Doc: remove text about txes always relayed from -whitelist (David A. Harding)

Pull request description:

  Updates text since -whitelistforcerelay was set to false by default in PR #15193.

ACKs for commit e0bb27:
  fanquake:
    utACK e0bb279
  MarcoFalke:
    utACK e0bb279999

Tree-SHA512: cf0c9321d72692d573039a04f8f1d048cbdf67ed86cc781523dabd3c45d2731b788f53749e6bb29d7da1ab44eb04030f352469b20489bb2a26c2c38fb61f6489
2019-05-08 15:59:42 -04:00
James O'Beirne
631940aab2 scripted-diff: replace chainActive -> ::ChainActive()
Though at the moment ChainActive() simply references `g_chainstate.m_chain`,
doing this change now clears the way for multiple chainstate usage and allows
us to script the diff.

-BEGIN VERIFY SCRIPT-
git grep -l "chainActive" | grep -E '(h|cpp)$' | xargs sed -i '/chainActive =/b; /extern CChain& chainActive/b; s/\(::\)\{0,1\}chainActive/::ChainActive()/g'
-END VERIFY SCRIPT-
2019-05-03 15:02:54 -04:00
MarcoFalke
12aa2ac988
Merge #15323: rpc: Expose g_is_mempool_loaded via getmempoolinfo
effe81f750 Move g_is_mempool_loaded into CTxMemPool::m_is_loaded (Ben Woosley)
bb8ae2c419 rpc: Expose g_is_mempool_loaded via getmempoolinfo and /rest/mempool/info.json (Ben Woosley)

Pull request description:

  And use it to fix a race condition in mempool_persist.py:
  https://travis-ci.org/Empact/bitcoin/jobs/487577243

  Since e.g. getrawmempool returns errors based on this status, this
  enables users to test it for readiness.

  Fixes #12863

ACKs for commit effe81:
  MarcoFalke:
    utACK effe81f750
  jnewbery:
    utACK effe81f750

Tree-SHA512: 74328b0c17a97efb8a000d4ee49b9a673c2b6dde7ea30c43a6a2eff961a233351c9471f9a42344412135786c02bdf2ee1b2526651bb8fed68bd94d2120c4ef86
2019-05-01 10:06:15 -04:00
MarcoFalke
2c35fe6238
Merge #15849: Thread names in logs and deadlock debug tools
8722e54e56 threads: add thread names to deadlock debugging message (James O'Beirne)
383b186c28 threads: prefix log messages with thread names (James O'Beirne)
ddd95ccb80 tests: add threadutil tests (James O'Beirne)
ae5f2b6a6c threads: introduce util/threadnames, refactor thread naming (James O'Beirne)
188ca75e5f disable HAVE_THREAD_LOCAL on unreliable platforms (James O'Beirne)

Pull request description:

  I'm resurrecting this one (from #13168) because I need it to make progress on #15735.

  It's now off by default and can be turned on with `-logthreadnames=1`.

  Ran some benchmarks (IBD from local peer from 500_000 -> 504_000) and it's within spitting distance either on or off:

  ### threadnames off (default)

  #### 2018-05-threadnames.3 vs. master (absolute)
  |                      name                      | iterations |   2018-05-threadnames.3    |           master           |
  |------------------------------------------------|-----------:|----------------------------|----------------------------|
  | ibd.local.500000.504000.dbcache=2048           |          3 | 376.1584 (± 9.2944)        | 392.3414 (± 13.4238)       |
  | ibd.local.500000.504000.dbcache=2048.mem-usage |          3 | 2236117.3333 (± 1845.9623) | 2238690.6667 (± 2669.3487) |

  #### 2018-05-threadnames.3 vs. master (relative)
  |                      name                      | iterations | 2018-05-threadnames.3 | master |
  |------------------------------------------------|-----------:|----------------------:|-------:|
  | ibd.local.500000.504000.dbcache=2048           |          3 |                     1 |  1.043 |
  | ibd.local.500000.504000.dbcache=2048.mem-usage |          3 |                     1 |  1.001 |

  ### threadnames on

  #### 2018-05-threadnames-take-2 vs. master (absolute)
  |                      name                      | iterations | 2018-05-threadnames-take-2 |           master           |
  |------------------------------------------------|-----------:|----------------------------|----------------------------|
  | ibd.local.500000.504000.dbcache=2048           |          3 | 367.6861 (± 0.3941)        | 364.1667 (± 0.9776)        |
  | ibd.local.500000.504000.dbcache=2048.mem-usage |          3 | 2238461.3333 (± 3697.8730) | 2237014.6667 (± 3307.6966) |

  #### 2018-05-threadnames-take-2 vs. master (relative)
  |                      name                      | iterations | 2018-05-threadnames-take-2 | master |
  |------------------------------------------------|-----------:|---------------------------:|-------:|
  | ibd.local.500000.504000.dbcache=2048           |          3 |                      1.010 |   1.00 |
  | ibd.local.500000.504000.dbcache=2048.mem-usage |          3 |                      1.001 |   1.00 |
  ```

ACKs for commit 8722e5:
  Empact:
    utACK 8722e54e56
  jnewbery:
    utACK 8722e54e56
  MarcoFalke:
    re-utACK 8722e54e56 (Only change since my previous review is DEFAULT_LOGTHREADNAMES=false and stylistic updates

Tree-SHA512: 50af992708295b8d680cf10025262dd964e599a356bdfc1dfc84fb18c00afabcb34d3d12d551b0677ff81f8fccad0e17c1d5b24dfecb953a913bc77fdd1a4577
2019-04-30 15:26:01 -04:00
James O'Beirne
383b186c28 threads: prefix log messages with thread names
Introduce a new flag (`-logthreadnames`) which allows toggling
of this behavior.
2019-04-29 13:49:15 -04:00
James O'Beirne
ae5f2b6a6c threads: introduce util/threadnames, refactor thread naming
This work is prerequisite to attaching thread names to log lines and deadlock
debug utilities. This code allows setting of an "internal" threadname per
thread on platforms where thread_local is available.

This commit also moves RenameThread() out of a more general module and adds a
numeric suffix to disambiguate between threads with the same name. It
explicitly names a few main threads using the new util::ThreadRename().
2019-04-29 13:42:25 -04:00
MarcoFalke
10852210bc
Merge #15877: doc: Fix -dustrelayfee= argument docs grammar
64491cb376 doc: Fix -dustrelayfee= argument docs grammar (keepkeyjon)

Pull request description:

ACKs for commit 64491c:
  fanquake:
    utACK 64491cb

Tree-SHA512: 562180e5bb065c71cda89555afd1cd5a54a98b058ab9006af3a6437fbbde46c7f3930b3fe98900bbb18f329057e00da81bc8290bdf6160d7eccc97d255b30e4b
2019-04-29 09:10:36 -04:00
practicalswift
a34081b7c3 Remove unused OpenSSL includes to make it more clear where OpenSSL is used 2019-04-29 12:53:24 +02:00
MarcoFalke
3356799ee3
Merge #15778: [wallet] Move maxtxfee from node to wallet
5c759c73b2 [wallet] Move maxTxFee to wallet (John Newbery)

Pull request description:

  Closes #15355

  Moves the `-maxtxfee` from the node to the wallet. See discussion in issue for details.

  This is a cleanup. There is no change in behaviour.

  Completes #15620

ACKs for commit 5c759c:
  MarcoFalke:
    utACK 5c759c73b2
  ryanofsky:
    utACK 5c759c73b2. Changes since last review: updated commit message and an error message and method name.
  meshcollider:
    utACK 5c759c73b2

Tree-SHA512: 2f9b2729da3940a5cda994d3f3bc11ee1a52fcc1c5e9842ea0ea63e4eb0300e8416853046776311298bc449ba07554aa46f0f245ce28598a5b0bd7347c12e752
2019-04-27 09:28:54 -04:00
David A. Harding
e0bb279999
Doc: remove text about txes always relayed from -whitelist
Updates text since -whitelistforcerelay was set to false by default in
PR #15193.
2019-04-25 13:11:33 -04:00
keepkeyjon
64491cb376
doc: Fix -dustrelayfee= argument docs grammar 2019-04-23 14:07:06 -06:00
MarcoFalke
4bd7187da8
Merge #15699: Remove no-op CClientUIInterface::[signal_name]_disconnect. Disconnect BlockNotifyGenesisWait and RPCNotifyBlockChange properly.
6dd469a3be Disconnect BlockNotifyGenesisWait and RPCNotifyBlockChange properly. Remove no-op CClientUIInterface::[signal_name]_disconnect. (practicalswift)

Pull request description:

  Resolves #15698.

  Changes:
  * Remove no-op `CClientUIInterface::[signal_name]_disconnect`.
  * Disconnect `BlockNotifyGenesisWait` and `RPCNotifyBlockChange` properly.

ACKs for commit 6dd469:
  MarcoFalke:
    utACK 6dd469a3be

Tree-SHA512: 0b50d658fa72261332bc57ddea379fd08f4bc1de392c10c628e20142d6fd244b606c39fd0665d6bc39324c1aa8c8814ac942b4659106279e33b90206aaf37411
2019-04-23 13:03:41 -04:00
MarcoFalke
ae2c19f578
Merge #15655: Resolve the checkpoints <-> validation circular dependency
418d3230f8 Resolve the checkpoints <-> validation CD. (251)

Pull request description:

  This pull request attempts to resolve the `checkpoints -> validation -> checkpoints` circular dependency.

  The circular dependency is resolved by moving the `CheckPoints::GetLastCheckpoint(const CCheckpointData& data)` function to `validation.cpp` where it used exclusively by the private function `ContextualCheckBlockHeader(const CBlockHeader& block, CValidationState& state, const CChainParams& params, const CBlockIndex* pindexPrev, int64_t nAdjustedTime)`.

ACKs for commit 418d32:
  promag:
    utACK 418d323, only `GetLastCheckpoint` usage is in `validation.cpp` and so makes sense to move it there.
  practicalswift:
    utACK 418d3230f8
  MarcoFalke:
    utACK 418d3230f8
  sipa:
    utACK 418d3230f8

Tree-SHA512: 03c3556bc192e65f5e3fa76fd545d4ee7d63d3fb06b132f7a1fa6131aa21ddd2e5b2d19e2222dfe524f422daaca30efde219bed188db8c74ff4b088876b5bc16
2019-04-19 09:34:01 -04:00
John Newbery
5c759c73b2 [wallet] Move maxTxFee to wallet
This commit moves the maxtxfee setting to the wallet. There is only
one minor behavior change:

- an error message in feebumper now refers to -maxtxfee instead of
maxTxFee.
2019-04-18 11:34:42 -04:00
MarcoFalke
e4beef611a
Merge #14121: Index for BIP 157 block filters
c7efb652f3 blockfilter: Update BIP 158 test vectors. (Jim Posen)
19308c9e21 rpc: Add getblockfilter RPC method. (Jim Posen)
ff35105096 init: Add CLI option to enable block filter index. (Jim Posen)
accc8b8b18 index: Access functions for global block filter indexes. (Jim Posen)
2bc90e4e7b test: Unit test for block filter index reorg handling. (Jim Posen)
6bcf0998c0 test: Unit tests for block index filter. (Jim Posen)
b5e8200db7 index: Implement lookup methods on block filter index. (Jim Posen)
75a76e3619 index: Implement block filter index with write operations. (Jim Posen)
2ad2338ef9 serialize: Serialization support for big-endian 32-bit ints. (Jim Posen)
ba6ff9a6f7 blockfilter: Functions to translate filter types to/from names. (Jim Posen)
62b7a4f094 index: Ensure block locator is not stale after chain reorg. (Jim Posen)
4368384f1d index: Allow atomic commits of index state to be extended. (Jim Posen)

Pull request description:

  This introduces a new BlockFilterIndex class, which is required for BIP 157 support.

  The index is uses the asynchronous BaseIndex infrastructure driven by the ValidationInterface callbacks. Filters are stored sequentially in flat files and the disk location of each filter is indexed in LevelDB along with the filter hash and header. The index is designed to ensure persistence of filters reorganized out of the main chain to simplify the BIP 157 net implementation.

  Stats (block height = 565500):
  - Syncing the index from scratch takes 45m
  - Total index size is 3.8 GiB

ACKs for commit c7efb6:
  MarcoFalke:
    utACK c7efb652f3
  ryanofsky:
    Slightly tested ACK c7efb652f3 (I just rebuilt the index with the updated PR and tested the RPC). Changes since last review: rebase, fixed compile errors in internal commits, new comments, updated error messages, tweaked cache size logic, renamed commit method, renamed constants and globals, fixed whitespace, extra BlockFilterIndex::Init error check.

Tree-SHA512: f8ed7a9b6f76df45933aa5eba92b27b3af83f6df2ccb3728a5c89eec80f654344dc14f055f6f63eb9b3a7649dd8af6553fe14969889e7e2fd2f8461574d18f28
2019-04-18 09:48:25 -04:00
Wladimir J. van der Laan
6a135fbe5b
Merge #15638: Move-only: Pull wallet code out of libbitcoin_server
4d074e84a2 [build] Move AnalyzePSBT from psbt.cpp to node/psbt.cpp (Russell Yanofsky)
fd509bd1f7 [docs] Document src subdirectories and different libraries (John Newbery)
9eaeb7fb8d [build] Move wallet load functions to wallet/load unit (John Newbery)
91a25d1e71 [build] Add several util units (John Newbery)
99517866b6 [build] Move several units into common libraries (John Newbery)
0509465542 [build] Move rpc rawtransaction util functions to rpc/rawtransaction_util.cpp (John Newbery)
1acc61f874 [build] Move rpc utility methods to rpc/util (John Newbery)
4a75c9d651 [build] Move policy settings to new src/policy/settings unit (John Newbery)
fdf8888b6f [build] Move CheckTransaction from lib_server to lib_consensus (John Newbery)

Pull request description:

  This is a move-only commit. No code is changing and the moves can be easily verified with:

  ```sh
  git log -p -n1 --color-moved=dimmed_zebra
  ```

  This commit moves functions and variables that wallet code depends on out of libbitcoin_server.a, so the bitcoin-wallet tool can be built without libbitcoin_server.a in #15639, and attempting to access server state from wallet code will result in link errors instead of silently broken code.

  List of moves:

  - `CheckTransaction` moves from `consensus/tx_verify.cpp` to `consensus/tx_check.cpp`
  - `urlDecode` moves from `httpserver.cpp` to `util/url.cpp`
  - `TransactionErrorString` moves from `node/transaction.cpp` to `util/error.cpp`
  - `StringForFeeReason` and `FeeModeFromString` move from `policy/fees.cpp` to `util/fees.cpp`
  - `incrementalRelayFee` `dustRelayFee` and `nBytesPerSigOp` move from `policy/policy.cpp` to `policy/settings.cpp`
  - `SignalsOptInRBF` moves from `policy/rbf.cpp` to `util/rbf.cpp`
  - `fIsBareMultisigStd` moves from `validation.cpp` to `policy/settings.cpp`
  - `ConstructTransaction` `TxInErrorToJSON` and `SignTransaction` move from `rpc/rawtransaction.cpp` to `rpc/rawtransaction_util.cpp`
  - `RPCTypeCheck` `RPCTypeCheckArgument` `RPCTypeCheckObj` `AmountFromValue` `ParseHashV``ParseHashO` `ParseHexV` `ParseHexO` `HelpExampleCli` and `HelpExampleRpc` move from `rpc/server.cpp` to `rpc/util.cpp`
  - `AmountHighWarn` and `AmountErrMsg` move from `ui_interface.cpp` to `util/error.cpp`
  - `FormatStateMessage` and `strMessageMagic` move from `validation.cpp` to `util/validation.cpp`
  - `VerifyWallets` `LoadWallets` `StartWallets` `FlushWallets` `StopWallets` and `UnloadWallets` move from `wallet/init.cpp` to `wallet/node.cpp`

ACKs for commit 4d074e:
  jnewbery:
    utACK 4d074e84a2 (checked by doing the rebase myself and verifying no difference between my branch and 4d074e84a2)

Tree-SHA512: 5e1604a9fb06475f2b96da0de0baa8330f4dda834dc20a0183ef11e1e4c27631d1d1bbb9abf0054efc03d56945fdf9920f63366b6a4f200f665b742a479ff75c
2019-04-10 15:51:37 +02:00
MarcoFalke
5392aee64f
Merge #15629: init: Throw error when network specific config is ignored
fae38c3dc6 doc: Fix all typos reported by codespell (MarcoFalke)
fa9058f0ed doc: Add release notes for 15629 (MarcoFalke)
fa4a922d78 qa: Add test for missing testnet section in conf file (MarcoFalke)
dddd6f0f58 init: Throw error when network specific config is ignored (MarcoFalke)

Pull request description:

  This should have no effect on mainnet users, but simplifies testing, where config settings are currently ignored with only a warning. Fix this by making it an error.

  Issues:
  *  bitcoin client 0.17.0 ignores wallet's name (file) #14523
  *  Can't set custom rpcport on testnet #13777
  * ...

ACKs for commit fae38c:

Tree-SHA512: 2e209526898eea6e444c803ec2666989cee4ca137492d32984998733c50a70056cb54657df8dc3027a6a0612738a8afce0bc35824b868c5f22281e00e0188530
2019-04-09 21:14:49 -04:00