Commit graph

431 commits

Author SHA1 Message Date
MarcoFalke
0277173b1d
Merge #10498: Use static_cast instead of C-style casts for non-fundamental types
9ad6746ccd Use static_cast instead of C-style casts for non-fundamental types (practicalswift)

Pull request description:

  A C-style cast is equivalent to try casting in the following order:

  1. `const_cast(...)`
  2. `static_cast(...)`
  3. `const_cast(static_cast(...))`
  4. `reinterpret_cast(...)`
  5. `const_cast(reinterpret_cast(...))`

  By using `static_cast<T>(...)` explicitly we avoid the possibility of an unintentional and dangerous `reinterpret_cast`. Furthermore `static_cast<T>(...)` allows for easier grepping of casts.

  For a more thorough discussion, see ["ES.49: If you must use a cast, use a named cast"](https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#es49-if-you-must-use-a-cast-use-a-named-cast) in the C++ Core Guidelines (Stroustrup & Sutter).

Tree-SHA512: bd6349b7ea157da93a47b8cf238932af5dff84731374ccfd69b9f732fabdad1f9b1cdfca67497040f14eaa85346391404f4c0495e22c467f26ca883cd2de4d3c
2018-02-07 16:15:28 -05:00
Matt Corallo
dd2de47c62 Fix fast-shutdown crash if genesis block was not loaded
If the ShutdownRequested() check at the top of ActivateBestChain()
returns false during initial genesis block load we will fail an
assertion in UTXO DB flush as the best block hash IsNull(). To work
around this, we move the check until after one round of
ActivateBestChainStep(), ensuring the genesis block gets connected.
2018-02-06 15:14:02 -05:00
Matt Corallo
02fc886363 Add braces to meet code style on line-after-the-one-changed. 2018-02-06 14:55:36 -05:00
Matt Corallo
85aa8398f5 Hold mempool.cs for the duration of ATMP.
This resolves an issue where getrawmempool() can race mempool
notification signals. Intuitively we use mempool.cs as a "read
lock" on the mempool with cs_main being the write lock, so holding
the read lock intermittently while doing write operations is
somewhat strange.
This also avoids the introduction of cs_main in getrawmempool()
which reviewers objected to in the previous fix in #12273
2018-02-06 13:51:44 -05:00
Jonas Schnelli
90ba2df11b
Fix missing cs_main lock for GuessVerificationProgress() 2018-01-30 17:49:22 -10:00
practicalswift
1340eda3b7 Fix typos 2018-01-28 13:21:25 +01:00
Jonas Schnelli
ccd8ef65f9
Reduce cs_main lock in ReadBlockFromDisk, only read GetBlockPos under the lock 2018-01-23 20:24:53 -10:00
Akira Takizawa
595a7bab23 Increment MIT Licence copyright header year on files modified in 2017 2018-01-03 02:26:56 +09:00
Matt Corallo
97d2b09c12 Add helper to wait for validation interface queue to catch up 2017-12-26 11:56:00 -05:00
Matt Corallo
36137497f1 Block ActivateBestChain to empty validationinterface queue 2017-12-26 11:54:49 -05:00
Matt Corallo
a99b76f269 Require no cs_main lock for ProcessNewBlock/ActivateBestChain
This requires the removal of some very liberal (incorrect) cs_mains
sprinkled in some tests. It adds some chainActive.Tip() races, but
the tests are all single-threaded anyway.
2017-12-26 11:54:43 -05:00
Matt Corallo
9c5a4a6ed8 Stop special-casing phashBlock handling in validation for TBV 2017-12-12 13:30:22 -05:00
Wladimir J. van der Laan
5d132e8b97
Merge #10574: Remove includes in .cpp files for things the corresponding .h file already included
a720b92 Remove includes in .cpp files for things the corresponding .h file already included (practicalswift)

Pull request description:

  Remove includes in .cpp files for things the corresponding .h file already included.

  Example case:
  * `addrdb.cpp` includes `addrdb.h` and `fs.h`
  * `addrdb.h` includes `fs.h`

  Then remove the direct inclusion of `fs.h` in `addrman.cpp` and rely on the indirect inclusion of `fs.h` via the included `addrdb.h`.

  In line with the header include guideline (see #10575).

Tree-SHA512: 8704b9de3011a4c234db336a39f7d2c139e741cf0f7aef08a5d3e05197e1e18286b863fdab25ae9638af4ff86b3d52e5cab9eed66bfa2476063aa5c79f9b0346
2017-12-12 14:56:25 +01:00
Wladimir J. van der Laan
214046f69b
Merge #10279: Add a CChainState class to validation.cpp to take another step towards clarifying internal interfaces
22fddde Avoid calling GetSerializeSize on each tx in a block if !fTxIndex (Matt Corallo)
2862aca Move some additional variables into CChainState private (Matt Corallo)
fd4d80a Create initial CChainState to hold chain state information (Matt Corallo)
e104f0f Move block writing out of AcceptBlock (Matt Corallo)
50701ba Move txindex/undo data disk location stuff out of ConnectBlock (Matt Corallo)
93a34cf Make DisconnectBlock unaware of where undo data resides on disk (Matt Corallo)

Pull request description:

  CChainState should eventually, essentially, be our exposed "libconsensus", but we're probably a few releases away, so the real goal is to clarify our internal interfaces. The main split was a big step, but validation.cpp is still a somewhat ranomly-mixed bag of functions that are pure functions which validate inputs (which should probably either merge with their callers or move into another file in consensus/), read/write data from disk, manipulate our current chain state (which moves into CChainState), and do mempool transaction validation.

  Obviously this is only a small step, but some effort is made to clean up what functions the functions in CChainState call, and obviously as things are added its easy to keep clear "CChainState::* cannot call anything except via callbacks through CValidationInterface, pure functions, or disk read/write things". Right now there are some glaring violations in mempool callbacks, and general flushing logic needs cleaning up (FlushStateToDisk maybe shouldnt be called, and there should be an API towards setDirtyBlockIndex, but I'll leave that for after @sipa's current changesets land).

Tree-SHA512: 69b8ec191b36b19c9492b4dee74c8057621fb6ec98ad219e8da0b2ed5c3ad711b10b5af9ff1117e8807ccf88918eeeab573be8448baecc9a59f099c53095985b
2017-12-12 14:36:57 +01:00
Wladimir J. van der Laan
497d0e014c
Merge #10275: [rpc] Allow fetching tx directly from specified block in getrawtransaction
434526a [test] Add tests for getrawtransaction with block hash. (Karl-Johan Alm)
b167951 [rpc] Allow getrawtransaction to take optional blockhash to fetch transaction from a block directly. (Karl-Johan Alm)
a5f5a2c [rpc] Fix fVerbose parsing (remove excess if cases). (Karl-Johan Alm)

Pull request description:

  [Reviewer hint: use [?w=1](https://github.com/bitcoin/bitcoin/pull/10275/files?w=1) to avoid seeing a bunch of indentation changes.]

  Presuming a user knows the block hash of the block containing a given transaction, this PR allows them to fetch the raw transaction, even without `-txindex`. It also enables support for getting transactions that are in orphaned blocks.

  Note that supplying a block hash will override mempool and txindex support in `GetTransaction`. The rationale behind this is that a transaction may be in multiple places (orphaned blocks) and if the user supplies an explicit block hash it should be adhered to.

  ```Bash
  $ # a41.. is a tx inside an orphan block ..3c6f.. -- first try getting it normally
  $ ./bitcoin-cli getrawtransaction a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79 1
  error code: -5
  error message:
  No such mempool transaction. Use -txindex to enable blockchain transaction queries. Use gettransaction for wallet transactions.
  $ # now try with block hash
  $ ./bitcoin-cli getrawtransaction a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79 1 0000000000000000003c6fe479122bfa4a9187493937af1734e1e5cd9f198ec7
  {
    "hex": "01000000014e7e81144e42f6d65550e59b715d470c9301fd7ac189[...]90488ac00000000",
    "inMainChain": false,
    "txid": "a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79",
    "hash": "a41e66ee1341aa9fb9475b98cfdc1fe1261faa56c0a49254f33065ec90f7cd79",
    "size": 225,
  [...]
  }
  $ # another tx 6c66... in block 462000
  $ ./bitcoin-cli getrawtransaction 6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735 1 00000000000000000217f2c12922e321f6d4aa933ce88005a9a493c503054a40
  {
    "hex": "0200000004d157[...]88acaf0c0700",
    "inMainChain": true,
    "txid": "6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735",
    "hash": "6c66b98191e9d6cc671f6817142152ebf6c5cab2ef008397b5a71ac13255a735",
    "size": 666,
  [...]
  }
  $
  ```

Tree-SHA512: 279be3818141edd3cc194a9ee65929331920afb30297ab2d6da07293a2d7311afee5c8b00c6457477d9f1f86e86786a9b56878ea3ee19fa2629b829d042d0cda
2017-12-06 12:10:21 +01:00
Karl-Johan Alm
b167951677
[rpc] Allow getrawtransaction to take optional blockhash to fetch transaction from a block directly. 2017-12-05 11:03:24 +09:00
Matt Corallo
22fdddeabb Avoid calling GetSerializeSize on each tx in a block if !fTxIndex 2017-12-04 09:39:21 -05:00
Matt Corallo
2862aca40f Move some additional variables into CChainState private 2017-12-04 09:39:20 -05:00
Matt Corallo
fd4d80a2f8 Create initial CChainState to hold chain state information 2017-12-04 09:34:46 -05:00
Matt Corallo
e104f0fb7e Move block writing out of AcceptBlock 2017-12-04 09:33:10 -05:00
Matt Corallo
50701ba5fc Move txindex/undo data disk location stuff out of ConnectBlock 2017-12-04 09:33:10 -05:00
Matt Corallo
93a34cfeec Make DisconnectBlock unaware of where undo data resides on disk 2017-12-04 09:33:10 -05:00
MarcoFalke
fbce66a982
Merge #10493: Use range-based for loops (C++11) when looping over map elements
680bc2cbb Use range-based for loops (C++11) when looping over map elements (practicalswift)

Pull request description:

  Before this commit:

  ```c++
  for (std::map<T1, T2>::iterator x = y.begin(); x != y.end(); ++x) {
      T1 z = (*x).first;
      …
  }
  ```

  After this commit:

  ```c++
  for (auto& x : y) {
      T1 z = x.first;
      …
  }
  ```

Tree-SHA512: 954b136b7f5e6df09f39248a6b530fd9baa9ab59d7c2c7eb369fd4afbb591b7a52c92ee25f87f1745f47b41d6828b7abfd395b43daf84a55b4e6a3d45015e3a0
2017-11-30 17:10:05 -05:00
Wladimir J. van der Laan
46d1ebfcf8
Merge #11737: Document partial validation in ConnectBlock()
9d811dc Document partial validation in ConnectBlock() (Suhas Daftuar)

Pull request description:

  `ConnectBlock()` relies on validation that happens in `ContextualCheckBlock()` and
  `ContextualCheckBlockHeader()`. This has implications for implementing consensus
  changes and handling software upgrade to ensure that nodes upgrading their
  software end up enforcing all the consensus rules.

Tree-SHA512: 36a252af2221b0e5d5d6f8d5f8b16f8b566ca0db2d56242130a5523302c8757599ac234594a6a946c1689b260d18a32c2c7f8c3831304e78b9832e2ce5ac435a
2017-11-29 12:16:45 +01:00
Andras Elso
e1a8ec56c5 Fix: Open files read only if requested 2017-11-21 20:35:06 +01:00
Suhas Daftuar
9d811dc18b Document partial validation in ConnectBlock()
ConnectBlock() relies on validation that happens in ContextualCheckBlock() and
ContextualCheckBlockHeader(). This has implications for implementing consensus
changes and handling software upgrade to ensure that nodes upgrading their
software end up enforcing all the consensus rules.
2017-11-20 14:31:15 -05:00
practicalswift
a720b928c8 Remove includes in .cpp files for things the corresponding .h file already included 2017-11-16 22:26:34 +01:00
MeshCollider
1a445343f6 scripted-diff: Replace #include "" with #include <> (ryanofsky)
-BEGIN VERIFY SCRIPT-
for f in \
  src/*.cpp \
  src/*.h \
  src/bench/*.cpp \
  src/bench/*.h \
  src/compat/*.cpp \
  src/compat/*.h \
  src/consensus/*.cpp \
  src/consensus/*.h \
  src/crypto/*.cpp \
  src/crypto/*.h \
  src/crypto/ctaes/*.h \
  src/policy/*.cpp \
  src/policy/*.h \
  src/primitives/*.cpp \
  src/primitives/*.h \
  src/qt/*.cpp \
  src/qt/*.h \
  src/qt/test/*.cpp \
  src/qt/test/*.h \
  src/rpc/*.cpp \
  src/rpc/*.h \
  src/script/*.cpp \
  src/script/*.h \
  src/support/*.cpp \
  src/support/*.h \
  src/support/allocators/*.h \
  src/test/*.cpp \
  src/test/*.h \
  src/wallet/*.cpp \
  src/wallet/*.h \
  src/wallet/test/*.cpp \
  src/wallet/test/*.h \
  src/zmq/*.cpp \
  src/zmq/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-
2017-11-16 08:23:01 +13:00
Wladimir J. van der Laan
927a1d7d08
Merge #10286: Call wallet notify callbacks in scheduler thread (without cs_main)
89f0312 Remove redundant pwallet nullptr check (Matt Corallo)
c4784b5 Add a dev notes document describing the new wallet RPC blocking (Matt Corallo)
3ea8b75 Give ZMQ consistent order with UpdatedBlockTip on scheduler thread (Matt Corallo)
cb06edf Fix wallet RPC race by waiting for callbacks in sendrawtransaction (Matt Corallo)
e545ded Also call other wallet notify callbacks in scheduler thread (Matt Corallo)
17220d6 Use callbacks to cache whether wallet transactions are in mempool (Matt Corallo)
5d67a78 Add calls to CWallet::BlockUntilSyncedToCurrentChain() in RPCs (Matt Corallo)
5ee3172 Add CWallet::BlockUntilSyncedToCurrentChain() (Matt Corallo)
0b2f42d Add CallFunctionInQueue to wait on validation interface queue drain (Matt Corallo)
2b4b345 Add ability to assert a lock is not held in DEBUG_LOCKORDER (Matt Corallo)
0343676 Call TransactionRemovedFromMempool in the CScheduler thread (Matt Corallo)
a7d3936 Add a CValidationInterface::TransactionRemovedFromMempool (Matt Corallo)

Pull request description:

  Based on #10179, this effectively reverts #9583, regaining most of the original speedups of #7946.

  This concludes the work of #9725, #10178, and #10179.

  See individual commit messages for more information.

Tree-SHA512: eead4809b0a75d1fb33b0765174ff52c972e45040635e38cf3686cef310859c1e6b3c00e7186cbd17374c6ae547bfbd6c1718fe36f26c76ba8a8b052d6ed7bc9
2017-11-15 16:25:40 +01:00
Luke Dashjr
ac51a26bdc During IBD, when doing pruning, prune 10% extra to avoid pruning again soon after
Pruning forces a chainstate flush, which can defeat the dbcache and harm performance significantly.
2017-11-11 09:05:48 +00:00
practicalswift
d223bc940a Use unique_ptr for pcoinscatcher/pcoinsdbview/pcoinsTip/pblocktree
* pcoinscatcher (CCoinsViewErrorCatcher)
* pcoinsdbview (CCoinsViewDB)
* pcoinsTip (CCoinsViewCache)
* pblocktree (CBlockTreeDB)
* Remove variables shadowing pcoinsdbview
2017-11-09 16:53:34 +01:00
practicalswift
7536b08c10 trivial: Fix typo – alreardy → already 2017-11-08 11:36:18 +01:00
MarcoFalke
dd561667cb
Merge #11389: Support having SegWit always active in regtest (sipa, ajtowns, jnewbery)
d61845818 Have SegWit active by default (Pieter Wuille)
4bd89210a Unit tests for always-active versionbits. (Anthony Towns)
d07ee77ab Always-active versionbits support (Pieter Wuille)
18e071841 [consensus] Pin P2SH activation to block 173805 on mainnet (John Newbery)
526023aa7 Improve handling of BIP9Deployment limits (Anthony Towns)

Pull request description:

  Most tests shouldn't have to deal with the now-historical SegWit activation transition (and other deployments, but SegWit is certainly the hardest one to accomodate).

  This PR makes a versionbits starttime of -1 equal to "always active", and enables it by default for SegWit on regtest. Individual tests can override this by using the existing `-vbparams` option.

  A few unit tests and functional tests are adapted to indeed override vbparams, as they specifically test the transition.

  This is in preparation for wallet SegWit support, but I thought having earlier eyes on it would be useful.

Tree-SHA512: 3f07a7b41cf46476e6c7a5c43244e68c9f41d223482cedaa4c02a3a7b7cd0e90cbd06b84a1f3704620559636a2268f5767d4c52d09c1b354945737046f618fe5
2017-11-07 17:05:46 -05:00
John Newbery
18e071841e [consensus] Pin P2SH activation to block 173805 on mainnet 2017-11-06 19:09:12 -08:00
Wladimir J. van der Laan
cffa5ee132
Merge #11531: Check that new headers are not a descendant of an invalid block (more effeciently)
f3d4adf Make p2p-acceptablock not an extended test (Matt Corallo)
00dcda6 [qa] test that invalid blocks on an invalid chain get a disconnect (Matt Corallo)
015a525 Reject headers building on invalid chains by tracking invalidity (Matt Corallo)
932f118 Accept unrequested blocks with work equal to our tip (Matt Corallo)
3d9c70c Stop always storing blocks from whitelisted peers (Matt Corallo)
3b4ac43 Rewrite p2p-acceptblock in preparation for slight behavior changes (Matt Corallo)

Pull request description:

  @sdaftuar pointed out that the version in #11487 was somewhat DoS-able as someone could feed you a valid chain that forked off the the last checkpoint block and force you to do lots of work just walking backwards across blocks for each new block they gave you. We came up with a few proposals but settled on the one implemented here as likely the simplest without obvious DoS issues. It uses our existing on-load mapBlockIndex walk to make sure everything that descends from an invalid block is marked as such, and then simply caches blocks which we attempted to connect but which were found to be invalid. To avoid DoS issues during IBD, this will need to depend on #11458.

  Includes tests from #11487.

Tree-SHA512: 46aff8332908e122dae72ceb5fe8cd241902c2281a87f58a5fb486bf69d46458d84a096fdcb5f3e8e07fbcf7466232b10c429f4d67855425f11b38ac0bf612e1
2017-11-01 14:42:08 +01:00
Matt Corallo
015a5258ad Reject headers building on invalid chains by tracking invalidity
This tracks the set of all known invalid-themselves blocks (ie
blocks which we attempted to connect but which were found to be
invalid). This is used to cheaply check if new headers build on an
invalid chain.

While we're at it we also resolve an edge-case in invalidateblock
on pruned nodes which results in them needing a reindex if they
fail to reorg.
2017-10-31 13:51:30 -04:00
Matt Corallo
932f118e6a Accept unrequested blocks with work equal to our tip
This is a simple change that makes our accept requirements the
same as our request requirements, (ever so slightly) further
decoupling our consensus logic from our FindNextBlocksToDownload
logic in net_processing.
2017-10-31 13:36:06 -04:00
Suhas Daftuar
37886d5e2f Disconnect outbound peers relaying invalid headers 2017-10-27 16:29:12 -04:00
Suhas Daftuar
ce8cd7a7da Don't process unrequested, low-work blocks
A peer could try to waste our resources by sending us unrequested blocks with
low work, eg to fill up our disk.  Since
e2652002b6 we no longer request blocks until we
know we're on a chain with more than nMinimumChainWork (our anti-DoS
threshold), but we would still process unrequested blocks that had more work
than our tip.  This commit fixes that behavior.
2017-10-19 20:33:45 -04:00
João Barbosa
7a5f9303a9 Avoid slow transaction search with txindex enabled 2017-10-19 16:01:45 +01:00
Pieter Wuille
26fee4f6bd
Merge #11062: [mempool] Mark mempool import fails that were found in mempool as 'already there'
258d33b41 [mempool] Mark unaccepted txs present in mempool as 'already there'. (Karl-Johan Alm)

Pull request description:

  I was investigating the reasons for failed imports in mempool and noticed that `LoadMempool()` and `pwallet->postInitProcess()` (for all wallets) are executed concurrently. The wallet will end up importing transactions that `LoadMempool()` later tries to import; the latter will fail due to the tx already being in the mempool.

  This PR changes the log message, adding an additional "already there" entry. For transactions not accepted into mempool, a check if they are in the mempool is done first, and if found, they are counted as 'already there', otherwise counted as 'failed'.

  Also slight rewording for consistency (successes, failed, expired, ... -> succeeded, failed, expired).

Tree-SHA512: 1a6134a25260917f2768365e0dfd8b278fe3f8287cab38bb028b7de3d517718a2d37696186dc7a23ceab338cc755fbbe7d45358ee94e573610fddd2a0620d6e5
2017-10-18 02:37:46 -07:00
Matt Corallo
e545dedf72 Also call other wallet notify callbacks in scheduler thread
This runs Block{Connected,Disconnected}, SetBestChain, Inventory,
and TransactionAddedToMempool on the background scheduler thread.

Of those, only BlockConnected is used outside of Wallet/ZMQ, and
is used only for orphan transaction removal in net_processing,
something which does not need to be synchronous with anything
else.

This partially reverts #9583, re-enabling some of the gains from
 #7946. This does not, however, re-enable the gains achieved by
repeatedly releasing cs_main between each transaction processed.
2017-10-13 19:30:15 -04:00
Wladimir J. van der Laan
470c730e3f
Merge #10898: Fix invalid checks (NULL checks after dereference, redundant checks, etc.)
76fed83 Avoid NULL pointer dereference when _walletModel is NULL (which is valid) (practicalswift)
4971a9a Use two boolean literals instead of re-using variable (practicalswift)
b5fb339 Remove duplicate uriParts.size() > 0 check (practicalswift)
7466991 Remove redundant check (!ecc is always true) (practicalswift)
55224af Remove redundant NULL checks after new (practicalswift)

Pull request description:

  Contains:
  * Remove redundant `NULL` checks after throwing `new`
  * Remove redundant check (`!ecc` is always true)
  * Remove duplicate `uriParts.size() > 0` check
  * Use two boolean literals instead of re-using variable

Tree-SHA512: 30e9af8a9d5c8184836f8267b492aeb4e26eca171a3be08f634b3f39b3055b9fa9f06623f6c69b294ca13bf99743f7645cfac2b25e014ff74687bd085a997895
2017-10-12 23:55:50 +02:00
Wladimir J. van der Laan
0e3a411351
Merge #8498: Near-Bugfix: Optimization: Minimize the number of times it is checked that no money...
4e955c5 Near-Bugfix: Reestablish consensus check removed in 8d7849b (Jorge Timón)
3e8c916 Introduce CheckInputsAndUpdateCoins static wrapper in txmempool.cpp (Jorge Timón)
832e074 Optimization: Minimize the number of times it is checked that no money is created (Jorge Timón)
3f0ee3e Proper indentation for CheckTxInputs and other minor fixes (Jorge Timón)

Pull request description:

  ...is created by individual transactions to 2 places (but call only once in each):

  - ConnectBlock ( before calculated fees per txs twice )
  - AcceptToMemoryPoolWorker ( before called CheckTxInputs 4 times and calculated
     fees per tx one extra time )

  Also call tx.GetValueOut() only once per call of CheckTxInputs (instead of 2)

  For more motivation:

  ~~https://github.com/bitcoin/bitcoin/blob/master/src/main.cpp#L1493~~
  https://github.com/jtimon/bitcoin/compare/0.13-consensus-inputs...jtimon:0.13-consensus-inputs-comments

  EDIT: partially replaces #6445

  Near-Bugfix as pointed out in https://github.com/bitcoin/bitcoin/pull/8498#discussion_r124346132

Tree-SHA512: c71188e7c7c2425c9170ed7b803896755a92fd22f43b136eedaa6e554106696f0b10271d0ef0d0127c1eaafbc31d12eb19143df4f1b6882feecedf6ef05ea346
2017-10-11 10:45:22 +02:00
practicalswift
680bc2cbb3 Use range-based for loops (C++11) when looping over map elements
Before this commit:

  for (std::map<T1, T2>::iterator x = y.begin(); x != y.end(); ++x) {
  }

After this commit:

  for (auto& x : y) {
  }
2017-10-09 21:31:58 +02:00
Wladimir J. van der Laan
3a93270c55
Merge #11367: [rpc] getblockchaininfo: add size_on_disk, prune_target_size
b7dfc6c [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning (Daniel Edgecumbe)

Pull request description:

Tree-SHA512: c255c27d6c922434d203ffdefda0dd3dddbd765b6a9cce5f80f5af5cb0b1c11c8aff6f4d00e96a326701d0bc81aace2f216fd1985675aa979f76c16f564a6cf6
2017-10-09 17:04:18 +02:00
MarcoFalke
e93fff1463
Merge #11107: Fix races in AppInitMain and others with lock and atomic bools
c626dcb50 Make fUseCrypto atomic (MeshCollider)
731065b11 Consistent parameter names in txdb.h (MeshCollider)
35aeabec6 Make fReindex atomic to avoid race (MeshCollider)
58d91af59 Fix race for mapBlockIndex in AppInitMain (MeshCollider)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/11106

  Also makes fReindex atomic as suggested in @TheBlueMatt comment below, and makes fUseCrypto atomic as suggested in 10916

  d291e7635b just renames the parameters in the txdb header file to make them consistent with those used in the cpp file, noticed it when looking for uses of fReindex

Tree-SHA512: b378aa7289fd505b76565cd4d48dcdc04ac5540283ea1c80442170b0f13cb6df771b1a94dd54b7fec3478a7b4668c224ec9d795f16937782724c5d020edd3a42
2017-10-05 15:03:36 +02:00
Wladimir J. van der Laan
e12522dfda
Merge #11406: Add state message print to AcceptBlock failure message.
6643b80 Add state message print to AcceptBlock failure message. (Matt Corallo)

Pull request description:

  This should make it easier to debug issues where the CheckBlock at
  the top of ProcessNewBlock fails (which does not print, in contrast
  to AcceptBlock, which always prints).

  This was motivated by #11371 which appears to be exactly such a case, and is not debuggable from the information provided. Not sure how much this would have helped in that case, but it is kinda weird that we can reject a block without ever printing why.

Tree-SHA512: 7a1c2c76080b810212da885c38e091609e409c62918cc326bb36a1096e09b2ae7e26fd4bdaefd79863d2894e2823e463005700a524940f177a59ef09f589b2f1
2017-10-04 14:35:43 +02:00
practicalswift
55224af6bd Remove redundant NULL checks after new 2017-10-02 15:47:43 +02:00
Daniel Edgecumbe
b7dfc6c4b8 [rpc] getblockchaininfo: add size_on_disk, prune_target_size, automatic_pruning
Fix pruneheight help text.
Move fPruneMode block to match output ordering with help text.
Add functional tests for new fields in getblockchaininfo.
2017-09-29 19:40:50 +01:00
MarcoFalke
93d20a734d
Merge #11309: Minor cleanups for AcceptToMemoryPool
bf64c3cb3 Ignore transactions added to mempool during a reorg for fee estimation purposes. (Alex Morcos)
04f78ab5b Do not reject based on mempool min fee when bypass_limits is set. (Alex Morcos)
fd849e1b0 Change AcceptToMemoryPool function signature (Alex Morcos)

Pull request description:

  First commit just removes default arguments from `AcceptToMemoryPool` and consolidates two arguments, it does not change behavior.

  Second commit finally fixes the fact that we're not meant to reject based on mempool min fee when adding a transaction from a disconnected block during a reorg as mentioned [here](https://github.com/bitcoin/bitcoin/pull/9602#issue-202197849)

  Third commit makes fee estimation ignore transactions added from a disconnected block during a reorg. I think this was another source of fee estimates returning estimates below 1000 sat/kB as in #11303.

Tree-SHA512: 30925ca8b341915bb214f1d2590b36b7931f2e125b7660150e38ae70338f00db5aa7f1608546dddb181446924177eb7cf62ea8bd2583068acc074d6c3f86bc0c
2017-09-29 15:07:57 +02:00
Matt Corallo
6643b80d1c Add state message print to AcceptBlock failure message.
This should make it easier to debug issues where the CheckBlock at
the top of ProcessNewBlock fails (which does not print, in contrast
to AcceptBlock, which always prints).
2017-09-26 16:23:41 -04:00
Pieter Wuille
aeed345c9b
Merge #10953: [Refactor] Combine scriptPubKey and amount as CTxOut in CScriptCheck
3a131b724 Rename out to m_tx_out in CScriptCheck (Johnson Lau)
e91211878 [Refactor] Combine scriptPubKey and amount as CTxOut in CScriptCheck (Johnson Lau)

Pull request description:

  This simplifies CScriptCheck by combining scriptPubKey and amount

Tree-SHA512: 6422363cf5394c6cfefb30c1709db6def63230b809cc7697887e4a2e8c684149208edf91dd139e031b9fe732776b2db59305f77c3cba6f333b11cceb39ef0cc2
2017-09-22 16:17:44 -07:00
practicalswift
9ad6746ccd Use static_cast instead of C-style casts for non-fundamental types
A C-style cast is equivalent to try casting in the following order:

1. const_cast(...)
2. static_cast(...)
3. const_cast(static_cast(...))
4. reinterpret_cast(...)
5. const_cast(reinterpret_cast(...))

By using static_cast<T>(...) explicitly we avoid the possibility
of an unintentional and dangerous reinterpret_cast. Furthermore
static_cast<T>(...) allows for easier grepping of casts.
2017-09-22 14:59:27 +02:00
Johnson Lau
3a131b7244 Rename out to m_tx_out in CScriptCheck 2017-09-22 14:27:03 +08:00
Jorge Timón
4e955c58e1
Near-Bugfix: Reestablish consensus check removed in 8d7849b
in 8d7849b6db

This can potentially prevent an overflow that could at least in theory
allow the creation of money.
2017-09-20 23:26:02 +02:00
Jorge Timón
832e0744cb
Optimization: Minimize the number of times it is checked that no money is created
by individual transactions to 2 places (but call only once in each):

- ConnectBlock ( before calculated fees per txs twice )
- AcceptToMemoryPoolWorker ( before called CheckTxInputs 4 times and calculated
   fees per tx one extra time )

Also call tx.GetValueOut() only once per call of CheckTxInputs (instead of 2)
2017-09-20 23:25:52 +02:00
Wladimir J. van der Laan
551d7bf604
Merge #11132: Document assumptions that are being made to avoid NULL pointer dereferences
fdc3293 Document assumptions that are being made to avoid NULL pointer dereferences (practicalswift)

Pull request description:

  Document assumptions (via `assert(…)`:s) that are being made avoid `NULL` pointer dereferences.

  Rationale:
  * Make it clear to human reviewers and non-human static analyzers that what might look like potential `NULL` pointer dereferences are written the way they are intentionally (these cases are currently flagged by various static analyzers).

Tree-SHA512: b424328195e2680e1e4ec546298f718c49e5ad182147dc004de580693db1b50eec4065e1c4f232bdb302baa12954265a50ba21cb5ba4ff30248535b2de778672
2017-09-20 19:07:40 +02:00
Dan Raviv
a0b4c24617 Trivial: Fix validation comments
- Move comment about transaction/block weight calculation so it applies not only to the GetBlockWeight function but also to GetTransactionWeight
- Fix comment in validation.cpp referencing future deployment of BIP113. It has already been deployed.
- The doc comment for BLOCK_DOWNLOAD_WINDOW wasn't updated since pruning was introduced, so it still refers to pruning as something that might happen in the future. A larger BLOCK_DOWNLOAD_WINDOW window would now, indeed, make pruning harder.
2017-09-15 14:07:41 +03:00
Alex Morcos
bf64c3cb34 Ignore transactions added to mempool during a reorg for fee estimation purposes. 2017-09-12 12:40:06 -04:00
Alex Morcos
04f78ab5b9 Do not reject based on mempool min fee when bypass_limits is set.
This should have always been the case, but we will correctly trim to size after
a reorg which is when bypass_limits is set.
2017-09-12 12:35:35 -04:00
Alex Morcos
fd849e1b03 Change AcceptToMemoryPool function signature
Combine fLimitFree and fOverrideMempoolLimit into a single boolean:
bypass_limits.  This is used to indicate that mempool limiting based on feerate
should be bypassed.  It is used when readding transactions from a reorg and then
the mempool is trimmed to size after all transactions are added and they can be
evaluated in the context of their descendants. No changes to behavior.
2017-09-12 12:30:26 -04:00
Jonas Schnelli
ea729d55b4
Merge #10770: Drop upgrade-cancel callback registration for a generic "cancelable"
ee4d1493e Drop upgrade-cancel callback registration for a generic "resumeable" (Matt Corallo)

Pull request description:

  Instead of passing a StartShutdown reference all the way up from
  txdb, give ShowProgress a "cancelable" boolean, as StartShutdown
  is pretty much always what you'll want to use to cancel. Use the
  same boolean to allow cancel during initial block verification.

Tree-SHA512: 515817aaa4b9e3e856200e00be9c2d44ecfa2d4f288fe3e02116105fe85de2650c13076ee7e45396ec1ce6ab45e53b0477cddda7cfdee5b3bd0589cb81a4c346
2017-09-07 09:21:04 -07:00
MeshCollider
35aeabec62 Make fReindex atomic to avoid race 2017-09-07 12:00:30 +12:00
Wladimir J. van der Laan
6acdb1fab7
Merge #11238: Add assertions before potential null deferences
c00199244 Fix potential null dereferences (MeshCollider)

Pull request description:

  Picked up by the static analyzer [Facebook Infer](http://fbinfer.com/) which I was playing around with for another research project. Just adding some asserts before dereferencing potentially null pointers.

Tree-SHA512: 9c01dab2d21bce75c7c7ef867236654ab538318a1fb39f96f09cdd2382a05be1a6b2db0a1169a94168864e82ffeae0686a383db6eba799742bdd89c37ac74397
2017-09-06 23:54:06 +02:00
MarcoFalke
bc561b4b7d
Merge #11099: [RPC][mempool]: Add savemempool RPC
1aa97ee08 Add savemempool RPC (Lawrence Nahum)
467cbbcbf Add return value to DumpMempool (Lawrence Nahum)

Pull request description:

  Adds a simple parameterless rpc command to dump the mempool.

  Rationale:

  Sometimes there can be a crash for whatever reason (bug, power loss, etc) causing the mempool.dat file to not be saved.

  This change allows to script/cron the rpc call to have more regular saves to the file as well as cli/ad-hoc.

  This should solve issue https://github.com/bitcoin/bitcoin/issues/11086

Tree-SHA512: e856ae9777425a4521279c9b58e69285d8e374790bebefd3284cf91931eac0e456f86224f427a087a01bf70440bf6e439fa02c8a34940eb1046ae473e98b6aaa
2017-09-06 13:49:25 -07:00
practicalswift
fdc329376c Document assumptions that are being made to avoid NULL pointer dereferences 2017-09-06 10:45:37 +02:00
Lawrence Nahum
467cbbcbfc
Add return value to DumpMempool 2017-09-06 09:25:00 +02:00
Suhas Daftuar
0311836f69 Allow setting nMinimumChainWork on command line 2017-09-05 15:05:28 -04:00
Karl-Johan Alm
258d33b41a
[mempool] Mark unaccepted txs present in mempool as 'already there'.
On startup, the wallets will start pumping wallet transactions into the mempool in a different thread while LoadMempool() is running.
This will sometimes result in transactions "failing" to be accepted into mempool, but only for the reason that they were already
put there by a wallet. The log message for mempool load would note this as a 'failure' to import, which was misleading; it should
instead mark it as the transaction already being in the mempool.
2017-09-04 14:48:06 +09:00
Johnson Lau
e912118786 [Refactor] Combine scriptPubKey and amount as CTxOut in CScriptCheck 2017-08-30 18:32:17 +08:00
Wladimir J. van der Laan
f088a1bb39
Merge #10303: [bench] Include ms/blk stats in Connect* benchmarks.
a473eff [bench] Replace 0.00(000)1 with MICRO/MILLI #defines in validation.cpp. (Karl-Johan Alm)
5f850b0 [bench] Include ms/blk stats in Connect* benchmarks. (Karl-Johan Alm)

Pull request description:

  Display the average per block runtime for the various benchmarked times in the block connect functions to give an overview of long(er) term time distribution statistics.

Tree-SHA512: 3d6f24f6b9e3dbb448a647e2cda8e7b90ad6a16d4821f49f426a8e1ebc3ce5a0cf0a8cde82213e293affba441615702dfe50822c8c818e282af03bfe383d83e0
2017-08-28 12:13:35 +02:00
MeshCollider
c001992440 Fix potential null dereferences 2017-08-23 19:47:56 +12:00
Matt Corallo
ee4d1493e2 Drop upgrade-cancel callback registration for a generic "resumeable"
Instead of passing a StartShutdown reference all the way up from
txdb, give ShowProgress a "resumeable" boolean, which is used to
inform the user if the action will be resumed, but cancel is always
allowed by just calling StartShutdown().
2017-08-20 20:04:15 -04:00
practicalswift
64fb0ac016 Declare single-argument (non-converting) constructors "explicit"
In order to avoid unintended implicit conversions.
2017-08-16 16:33:25 +02:00
Marko Bencun
bb81e17355 scripted-diff: stop using the gArgs wrappers
They were temporary additions to ease the transition.

-BEGIN VERIFY SCRIPT-
find src/ -name "*.cpp" ! -wholename "src/util.h" ! -wholename "src/util.cpp" | xargs perl -i -pe 's/(?<!\.)(ParseParameters|ReadConfigFile|IsArgSet|(Soft|Force)?(Get|Set)(|Bool|)Arg(s)?)\(/gArgs.\1(/g'
-END VERIFY SCRIPT-
2017-08-14 17:02:10 +02:00
Wladimir J. van der Laan
ce74799a3c
Merge #10483: scripted-diff: Use the C++11 keyword nullptr to denote the pointer literal instead of the macro NULL
90d4d89 scripted-diff: Use the C++11 keyword nullptr to denote the pointer literal instead of the macro NULL (practicalswift)

Pull request description:

  Since C++11 the macro `NULL` may be:
  * an integer literal with value zero, or
  * a prvalue of type `std::nullptr_t`

  By using the C++11 keyword `nullptr` we are guaranteed a prvalue of type `std::nullptr_t`.

  For a more thorough discussion, see "A name for the null pointer: nullptr" (Sutter &
  Stroustrup), http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2431.pdf

  With this patch applied there are no `NULL` macro usages left in the repo:

  ```
  $ git grep NULL -- "*.cpp" "*.h" | egrep -v '(/univalue/|/secp256k1/|/leveldb/|_NULL|NULLDUMMY|torcontrol.*NULL|NULL cert)' | wc -l
  0
  ```

  The road towards `nullptr` (C++11) is split into two PRs:
  * `NULL` → `nullptr` is handled in PR #10483 (scripted, this PR)
  * `0` → `nullptr` is handled in PR #10645 (manual)

Tree-SHA512: 3c395d66f2ad724a8e6fed74b93634de8bfc0c0eafac94e64e5194c939499fefd6e68f047de3083ad0b4eff37df9a8a3a76349aa17d55eabbd8e0412f140a297
2017-08-14 16:30:59 +02:00
Wladimir J. van der Laan
6bdf4b3c7c
Merge #11028: Avoid masking of difficulty adjustment errors by checkpoints
85c82b5 Avoid masking of difficulty adjustment errors by checkpoints (Pieter Wuille)

Pull request description:

  Currently difficulty adjustment violations are not reported for chains that branch off before the last checkpoint. Change this by moving the checkpoint check after the difficulty check.

Tree-SHA512: 33666f2c3459151b28c42041a463779e6df18f61d3dd5b1879a0af4e5b199ef74d1e33e06af68bebfdfb211569ad5fb56556bfebe9d63b5688d910ea211b839a
2017-08-14 16:12:55 +02:00
Wladimir J. van der Laan
653a46dd91
Merge #11022: Basic keypool topup
d34957e [wallet] [tests] Add keypool topup functional test (Jonas Schnelli)
095142d [wallet] keypool mark-used and topup (John Newbery)
c25d90f [wallet] Add HasUnusedKeys() helper (John Newbery)
f2123e3 [wallet] Cache keyid -> keypool id mappings (John Newbery)
83f1ec3 [wallet] Don't hold cs_LastBlockFile while calling setBestChain (John Newbery)
2376bfc [wallet] [moveonly] Move LoadKeyPool to cpp (Matt Corallo)
cab8557 [wallet] [moveonly] Move CAffectedKeysVisitor (Jonas Schnelli)

Pull request description:

  This PR contains the first part of #10882 :

  - if a key from the keypool is used, mark all keys up to that key as used, and then try to top up the keypool
  - top up the keypool on startup

  Notably, it does not stop the node or prevent the best block from advancing if the keypool drops below a threshold (which means that transactions may be missed and funds lost if restoring from an old HD wallet backup).

Tree-SHA512: ac681fefeaf7ec2aab2fa1da93d12273ea80bd05eb48d7b3b551ea6e5d975dd97ba7de52b7fba52993823280ac4079cc36cf78a27dac708107ebf8fb6326142b
2017-08-14 16:08:44 +02:00
Pieter Wuille
85c82b50d1 Avoid masking of difficulty adjustment errors by checkpoints
Currently difficulty adjustment violations are not reported for
chains that branch off before the last checkpoint. Change this
by moving the checkpoint check after the difficulty check.
2017-08-10 15:39:36 -07:00
practicalswift
90d4d89230 scripted-diff: Use the C++11 keyword nullptr to denote the pointer literal instead of the macro NULL
-BEGIN VERIFY SCRIPT-
sed -i 's/\<NULL\>/nullptr/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h src/qt/*/*.cpp src/qt/*/*.h src/wallet/*/*.cpp src/wallet/*/*.h src/support/allocators/*.h
sed -i 's/Prefer nullptr, otherwise SAFECOOKIE./Prefer NULL, otherwise SAFECOOKIE./g' src/torcontrol.cpp
sed -i 's/tor: Using nullptr authentication/tor: Using NULL authentication/g' src/torcontrol.cpp
sed -i 's/METHODS=nullptr/METHODS=NULL/g' src/test/torcontrol_tests.cpp src/torcontrol.cpp
sed -i 's/nullptr certificates/NULL certificates/g' src/qt/paymentserver.cpp
sed -i 's/"nullptr"/"NULL"/g' src/torcontrol.cpp src/test/torcontrol_tests.cpp
-END VERIFY SCRIPT-
2017-08-07 07:36:37 +02:00
John Newbery
83f1ec33ce [wallet] Don't hold cs_LastBlockFile while calling setBestChain
cs_LastBlockFile shouldn't be held while calling wallet functions.
2017-08-02 17:31:35 -04:00
Matt Corallo
e7539f8649 Fix some broken init-time prints/constants 2017-08-01 17:02:10 -04:00
Wladimir J. van der Laan
bd924241e7
Merge #10758: Fix some chainstate-init-order bugs.
c0025d0 Fix segfault when shutting down before fully loading (Matt Corallo)
1385697 Order chainstate init more logically. (Matt Corallo)
ff3a219 Call RewindBlockIndex even if we're about to run -reindex-chainstate (Matt Corallo)
b0f3249 More user-friendly error message if UTXO DB runs ahead of block DB (Matt Corallo)
eda888e Fix some LoadChainTip-related init-order bugs. (Matt Corallo)

Pull request description:

  This does a number of things to clean up chainstate init order,
  fixing some issues as it goes:

  * Order chainstate init more logically - first all of the
    blocktree-related loading, then coinsdb, then
    pcoinsTip/chainActive. Only create objects as needed.

  * More clearly document exactly what is and isn't called in
    -reindex and -reindex-chainstate both with comments noting
    calls as no-ops and by adding if guards.

  * Move the writing of fTxIndex to LoadBlockIndex - this fixes a
    bug introduced in d6af06d68a where
    InitBlockIndex was writing to fTxIndex which had not yet been
    checked (because LoadChainTip hadn't yet initialized the
    chainActive, which would otherwise have resulted in
    InitBlockIndex being a NOP), allowing you to modify -txindex
    without reindex, potentially corrupting your chainstate!

  * Rename InitBlockIndex to LoadGenesisBlock, which is now a more
    natural name for it. Also check mapBlockIndex instead of
    chainActive, fixing a bug where we'd write the genesis block out
    on every start.

  * Move LoadGenesisBlock further down in init. This is a more logical
    location for it, as it is after all of the blockindex-related
    loading and checking, but before any of the UTXO-related loading
    and checking.

  * Give LoadChainTip a return value - allowing it to indicate that
    the UTXO DB ran ahead of the block DB. This just provides a nicer
    error message instead of the previous mysterious
    assert(!setBlockIndexCandidates.empty()) error.

  * Calls ActivateBestChain in case we just loaded the genesis
    block in LoadChainTip, avoiding relying on the ActivateBestChain
    in ThreadImport before continuing init process.

  * Move all of the VerifyDB()-related stuff into a -reindex +
    -reindex-chainstate if guard. It couldn't do anything useful
    as chainActive.Tip() would be null at this point anyway.

Tree-SHA512: 3c96ee7ed44f4130bee3479a40c5cd99a619fda5e309c26d60b54feab9f6ec60fabab8cf47a049c9cf15e88999b2edb7f16cbe6819e97273560b201a89d90762
2017-08-01 12:58:38 +02:00
Matt Corallo
ff3a21919d Call RewindBlockIndex even if we're about to run -reindex-chainstate
RewindBlockIndex works over both chainActive - disconnecting blocks
from the tip that need witness verification - and mapBlockIndex -
requiring redownload of blocks missing witness data.

It should never have been the case that the second half is skipped
if we're about to run -reindex-chainstate.
2017-07-27 15:03:05 -04:00
Matt Corallo
b0f32497b8 More user-friendly error message if UTXO DB runs ahead of block DB
This gives LoadChainTip a return value - allowing it to indicate that
the UTXO DB ran ahead of the block DB. This just provides a nicer
error message instead of the previous mysterious
assert(!setBlockIndexCandidates.empty()) error.

This also calls ActivateBestChain in case we just loaded the genesis
block in LoadChainTip, avoiding relying on the ActivateBestChain
in ThreadImport before continuing init process.
2017-07-27 15:03:05 -04:00
Matt Corallo
eda888e573 Fix some LoadChainTip-related init-order bugs.
* Move the writing of fTxIndex to LoadBlockIndex - this fixes a
  bug introduced in d6af06d68a where
  InitBlockIndex was writing to fTxIndex which had not yet been
  checked (because LoadChainTip hadn't yet initialized the
  chainActive, which would otherwise have resulted in
  InitBlockIndex being a NOP), allowing you to modify -txindex
  without reindex, potentially corrupting your chainstate!

* Rename InitBlockIndex to LoadGenesisBlock, which is now a more
  natural name for it. Also check mapBlockIndex instead of
  chainActive, fixing a bug where we'd write the genesis block out
  on every start.
2017-07-27 15:03:05 -04:00
Wladimir J. van der Laan
04d395e832
Merge #10854: Avoid using sizes on non-fixed-width types to derive protocol constants.
095b917 Avoid using sizes on non-fixed-width types to derive protocol constants. (Gregory Maxwell)

Pull request description:

  Thanks to awemany for pointing this out.

  This replaces #10172 which appears to be abandoned, but uses the constants as requested on that PR.

Tree-SHA512: 032c0d75b3aaf807a7d0c7fb8ff5515acc45ad58bd00fe81413f900fe02bad900534a970403b9bb568e132c9eddea6043e958daf625e8acc84375bd41ee2e2ef
2017-07-26 08:48:15 +02:00
Wladimir J. van der Laan
7c2400cb8a
Merge #10775: nCheckDepth chain height fix
d9d1bd3 nCheckDepth chain height fix (romanornr)

Pull request description:

  ````
  if (nCheckDepth <= 0)
      nCheckDepth = 1000000000; // suffices until the year 19000
  if (nCheckDepth > chainActive.Height())
      nCheckDepth = chainActive.Height();
  ````

  These lines confuse me.
  Correct me if I am wrong, but we can't check any more blocks than we have right?
  If someone requests <= 0 it get set it into some huge number and then immediately limit it to the chain height in the following statement.
  ````
  if (nCheckDepth > chainActive.Height())
      nCheckDepth = chainActive.Height();
  ````
  when using ````--checkblocks=Z```` When Z is ````0```` or any other negative number, it will check all blocks.

  I think it should be changed to this maybe.
  ````
  if (nCheckDepth <= 0 || nCheckDepth > chainActive.Height())
      nCheckDepth = chainActive.Height();
  ````
  Which gets rid of that huge number which is confusing for any other altcoins that have a different block time.

Tree-SHA512: 8ee0ae5f33b399fa74dc16926709694ccfe1fc8a043cba2f5d00884220ac1b9b13f2df4588041f4133be634e5c7b14f4eebe24294028dafe91581a97dbe627f3
2017-07-20 17:34:54 +02:00
Gregory Maxwell
095b917464 Avoid using sizes on non-fixed-width types to derive protocol constants.
Thanks to awemany for pointing this out.
2017-07-17 17:00:00 +00:00
Pieter Wuille
f90603ac6d
Merge #10618: Remove confusing MAX_BLOCK_BASE_SIZE.
3babbcb48 Remove confusing MAX_BLOCK_BASE_SIZE. (Gregory Maxwell)

Tree-SHA512: 361293fc4e1e379cd5a0908ed0866a00e1c7a771bdf02fded158fca21b492a29c7a67fea0d13dc40b2a04204c89823bf1836fe5b63a17c9747751b9c845a3527
2017-07-14 17:49:48 -07:00
Gregory Maxwell
3babbcb487 Remove confusing MAX_BLOCK_BASE_SIZE.
Some people keep thinking that MAX_BLOCK_BASE_SIZE is a separate
 size limit from the weight limit when it fact it is superfluous,
 and used in early tests before the witness data has been
 validated or just to compute worst case sizes.  The size checks
 that use it would not behave any differently consensus wise
 if they were eliminated completely.

Its correct value is not independently settable but is a function
 of the weight limit and weight formula.

This patch just eliminates it and uses the scale factor as
 required to compute the worse case constants.

It also moves the weight factor out of primitives into consensus,
 which is a more logical place for it.
2017-07-14 19:24:17 +00:00
Pieter Wuille
66270a416e
Merge #10557: Make check to distinguish between orphan txs and old txs more efficient.
18bacec6c Make check to distinguish between orphan txs and old txs more efficient. (Alex Morcos)

Tree-SHA512: b6b4bad89aa561975dce7b68b2fdad5623af5ebcb9c38fd6a72b5f6d0544ed441df4865591ac018f7ae0df9b5c60820cb4d9e55664f5667c9268458df70fd554
2017-07-14 11:54:50 -07:00
Jonas Schnelli
7666250ffb
Merge #10810: missing white space in function arg
69a4339ef missing white space in function arg (Lawrence Nahum)

Pull request description:

Tree-SHA512: 94d832eca0b5429cf48c7c1d4489942b53182c5b7ad9f24264867e7631301173c23a48f7b2e8fd22ff1b3137bafacb3f9d4320f4df235668c9096aaefff7afe8
2017-07-13 09:21:14 +02:00
Pieter Wuille
e4fcbf797e
Merge #10780: Simplify "!foo || (foo && bar)" as "!foo || bar"
1e3a3200a Simplify "!foo || (foo && bar)" as "!foo || bar" (practicalswift)

Tree-SHA512: d5ce6d7a9f3741e7abe8542b840268aa683e276aac7587041d32385a4c1273e20c236bec7590e27540dde72defb3f879d4695822bb7bfdb092cc20d9aefa64c1
2017-07-12 18:25:35 -07:00
Lawrence Nahum
69a4339ef5
missing white space in function arg 2017-07-13 01:36:39 +02:00
practicalswift
1e3a3200af Simplify "!foo || (foo && bar)" as "!foo || bar" 2017-07-09 13:39:58 +02:00
Jeremy Rubin
4cac0d1e04 Fix subscript[0] in validation.cpp 2017-07-08 13:33:01 -07:00
romanornr
d9d1bd3267
nCheckDepth chain height fix 2017-07-08 19:49:11 +02:00
Wladimir J. van der Laan
6dbcc74a0e
Merge #10193: scripted-diff: Remove #include <boost/foreach.hpp>
b1268a1 clang-format: Delete ForEachMacros (Jorge Timón)
5995735 scripted-diff: Remove #include <boost/foreach.hpp> (Jorge Timón)
3eff827 scripted-diff: Remove BOOST_REVERSE_FOREACH (Jorge Timón)
33aed5b Fix const_reverse_iterator constructor (pass const ptr) (Jorge Timón)
300851e Introduce src/reverse_iterator.hpp and include it... (Jorge Timón)

Tree-SHA512: df3405328e9602d0a433ac134ba59a5c9a6202ef64188df2f94a59b2ce58dec7c988b25d0671c7937de516a96b2e6daeb9d04c82fa363b616ee4cf6e9cb0fac6
2017-07-04 18:05:18 +02:00
Wladimir J. van der Laan
2935b469ae
Merge #10192: Cache full script execution results in addition to signatures
e3f9c05 Add CheckInputs() unit tests (Suhas Daftuar)
a3543af Better document CheckInputs parameter meanings (Matt Corallo)
309ee1a Update -maxsigcachesize doc clarify init logprints for it (Matt Corallo)
b014668 Add CheckInputs wrapper CCoinsViewMemPool -> non-consensus-critical (Matt Corallo)
eada04e Do not print soft-fork-script warning with -promiscuousmempool (Matt Corallo)
b5fea8d Cache full script execution results in addition to signatures (Matt Corallo)
6d22b2b Pull script verify flags calculation out of ConnectBlock (Matt Corallo)

Tree-SHA512: 0c6c3c79c64fcb21e17ab60290c5c96d4fac11624c49f841a4201eec21cb480314c52a07d1e3abd4f9c764785cc57bfd178511f495aa0469addb204e96214fe4
2017-06-29 20:19:21 +02:00
Pieter Wuille
90a002ea64
Merge #10558: Address nits from per-utxo change
21d4afa12 Comment clarifications in coins.cpp (Alex Morcos)
3c8a9aeff Add belt-and-suspenders in DisconnectBlock (Alex Morcos)

Tree-SHA512: d83e12ed71674faaaaebc03ffa1e2276984c35a29db419268ac9e14a45b33ccab716e3606dff8cfe1dcee4bec6e4794d2ca90341f10d5684be80e3fee61addf8
2017-06-28 11:44:22 -07:00
Wladimir J. van der Laan
d4e551adfe
Merge #10148: Use non-atomic flushing with block replay
176c021 [qa] Test non-atomic chainstate writes (Suhas Daftuar)
d6af06d Dont create pcoinsTip until after ReplayBlocks. (Matt Corallo)
eaca1b7 Random db flush crash simulator (Pieter Wuille)
0580ee0 Adapt memory usage estimation for flushing (Pieter Wuille)
013a56a Non-atomic flushing using the blockchain as replay journal (Pieter Wuille)
b3a279c [MOVEONLY] Move LastCommonAncestor to chain (Pieter Wuille)

Tree-SHA512: 47ccc62303f9075c44d2a914be75bd6969ff881a857a2ff1227f05ec7def6f4c71c46680c5a28cb150c814999526797dc05cf2701fde1369c06169f46eccddee
2017-06-28 18:26:41 +02:00
practicalswift
aa95947ded Use the override specifier (C++11) where we expect to be overriding the virtual function of a base class 2017-06-28 02:12:06 +02:00
Suhas Daftuar
e3f9c05b96 Add CheckInputs() unit tests
Check that cached script execution results are only valid for the same
script flags; that script execution checks are returned for non-cached
transactions; and that cached results are only valid for transactions
with the same witness hash.
2017-06-27 16:05:04 -04:00
Alex Morcos
3c8a9aefff Add belt-and-suspenders in DisconnectBlock
These extra variables were previously checked before the move to per-txout database.
2017-06-27 15:04:56 -04:00
Alex Morcos
18bacec6c2 Make check to distinguish between orphan txs and old txs more efficient.
Checking for the existence in the CCoinsViewCache of the outputs of a new tx
will result in a disk hit for every output since they will not be found.  On the
other hand if those outputs exist already, then the inputs must also have been
missing, so we can move this check inside the input existence check so in the
common case of a new tx it doesn't need to run.

The purpose of the check is to avoid spamming the orphanMap with slightly old
txs which we have already seen in a block, but it is already only optimistic
(depending on the outputs not being spent), so make it even more efficient by
only checking the cache and not the entire pcoinsTip.
2017-06-27 14:14:15 -04:00
Matt Corallo
d6af06d68a Dont create pcoinsTip until after ReplayBlocks.
This requires that we not access pcoinsTip in InitBlockIndex's
FlushStateToDisk (so we just skip it until later in AppInitMain)
and the LoadChainTip in LoadBlockIndex (which there is already one
later in AppinitMain, after ReplayBlocks, so skipping it there is
fine).

Includes some simplifications by Suhas Daftuar and Pieter Wuille.
2017-06-26 10:46:51 -07:00
Pieter Wuille
0580ee08ff Adapt memory usage estimation for flushing 2017-06-26 10:45:48 -07:00
Pieter Wuille
013a56aa1a Non-atomic flushing using the blockchain as replay journal 2017-06-26 10:45:48 -07:00
Wladimir J. van der Laan
f3f1e2e7d3
Merge #9544: [trivial] Add end of namespace comments. Improve consistency.
5a9b508 [trivial] Add end of namespace comments (practicalswift)

Tree-SHA512: 92b0fcae4d1d3f4da9e97569ae84ef2d6e09625a5815cd0e5f0eb6dd2ecba9852fa85c184c5ae9de5117050330ce995e9867b451fa8cd5512169025990541a2b
2017-06-26 13:40:26 +02:00
Wladimir J. van der Laan
d609fd85ca
Merge #9176: Globals: Pass Consensus::Params through CBlockTreeDB::LoadBlockIndexGuts()
b324b28 Globals: Pass Consensus::Params through CBlockTreeDB::LoadBlockIndexGuts() (Jorge Timón)

Tree-SHA512: 9ef561a026960dc69df52603f4d60246c913bdd127465e8238f0954c6bbbc6b8b3b039bab70df38be1cbf5d06788fb631a2236b49c0167633d54366103c7c055
2017-06-25 10:26:01 +02:00
Wladimir J. van der Laan
6bef7ca8bc
Merge #10633: doc: Fix various typos
0a5a6b9 Fixed multiple typos (Dimitris Tsapakidis)

Tree-SHA512: 57748710bcbc03945b160db5e95bd686a2c64605f25d5e11d8ed9d0e1be3b3bf287a63588dc6eb33d0cef4ff17c765fda7c226d667a357acc539c8fcf2b9bb7e
2017-06-22 20:46:37 +02:00
Matt Corallo
a3543af3cc Better document CheckInputs parameter meanings 2017-06-22 12:21:09 -04:00
Matt Corallo
309ee1ae7b Update -maxsigcachesize doc clarify init logprints for it 2017-06-22 12:21:09 -04:00
Matt Corallo
b014668e27 Add CheckInputs wrapper CCoinsViewMemPool -> non-consensus-critical
This wraps CheckInputs in ATMP's cache-inputs call to check that
each scriptPubKey the CCoinsViewCache provides is the one which
was committed to by the input's transaction hash.
2017-06-22 12:21:09 -04:00
Dimitris Tsapakidis
0a5a6b90bc Fixed multiple typos
A few "a->an" and "an->a".
"Shows, if the supplied default SOCKS5 proxy" -> "Shows if the supplied default SOCKS5 proxy". Change made on 3 occurrences.
"without fully understanding the ramification of a command" -> "without fully understanding the ramifications of a command".
Removed duplicate words such as "the the".
2017-06-22 19:18:10 +03:00
Jorge Timón
3eff827f89
scripted-diff: Remove BOOST_REVERSE_FOREACH
-BEGIN VERIFY SCRIPT-
sed -i 's/BOOST_REVERSE_FOREACH(\(.*\), \(.*\))/for (\1 : reverse_iterate(\2))/' ./src/*.h ./src/*.cpp ./src/*/*.h ./src/*/*.cpp ;
-END VERIFY SCRIPT-
2017-06-22 03:48:50 +02:00
Jorge Timón
300851ec16
Introduce src/reverse_iterator.hpp and include it...
...where it will be needed

Taken from https://gist.github.com/arvidsson/7231973 with small
modifications to fit the bitcoin core project
2017-06-22 03:48:42 +02:00
Pieter Wuille
b3eb0d6485
Merge #10537: Few Minor per-utxo assert-semantics re-adds and tweak
9417d7a33 Be much more agressive in AccessCoin docs. (Matt Corallo)
f58349ca8 Restore some assert semantics in sigop cost calculations (Matt Corallo)
3533fb4d3 Return a bool in SpendCoin to restore pre-per-utxo assert semantics (Matt Corallo)
ec1271f2b Remove useless mapNextTx lookup in CTxMemPool::TrimToSize. (Matt Corallo)

Tree-SHA512: 158a4bce063eac93e1d50709500a10a7cb1fb3271f10ed445d701852fce713e2bf0da3456088e530ab005f194ef4a2adf0c7cb23226b160cecb37a79561f29ca
2017-06-20 18:27:45 -07:00
Pieter Wuille
efbcf2b1d5
Merge #10503: Use REJECT_DUPLICATE for already known and conflicted txn
d9bec888f Use REJECT_DUPLICATE for already known and conflicted txn (Pieter Wuille)

Tree-SHA512: adc6dc5caed731c3fd5c8784e8820a074e320360cdb5579c5b9299f9799dd99de60b7382d336ab1909dab8b23e744456d78aa0c3b1c8dd1af3d1b779314cf8fa
2017-06-20 18:00:21 -07:00
Pieter Wuille
1ad3d4e126
Merge #10502: scripted-diff: Remove BOOST_FOREACH, Q_FOREACH and PAIRTYPE
1238f13cf scripted-diff: Remove PAIRTYPE (Jorge Timón)
18dc3c396 scripted-diff: Remove Q_FOREACH (Jorge Timón)
7c00c2672 scripted-diff: Fully remove BOOST_FOREACH (Jorge Timón)
a5410ac5e Small preparations for Q_FOREACH, PAIRTYPE and #include <boost/foreach.hpp> removal (Jorge Timón)

Tree-SHA512: d3ab4a173366402e7dcef31608977b757d4aa07abbbad2ee1bcbcfa311e994a4552f24e5a55272cb22c2dcf89a4b0495e02e9d9aceae4b08c0bab668f20e324c
2017-06-13 18:05:58 -07:00
Wladimir J. van der Laan
b7296bcea0
Merge #10550: Don't return stale data from CCoinsViewCache::Cursor()
3ff1fa8 Use override keyword on CCoinsView overrides (Russell Yanofsky)
24e44c3 Don't return stale data from CCoinsViewCache::Cursor() (Russell Yanofsky)

Tree-SHA512: 08699dae0925ffb9c018f02612ac6b7eaf73ec331e2f4f934f1fe25a2ce120735fa38596926e924897c203f7470e99f0a99cf70d2ce31ff428b105e16583a861
2017-06-12 16:29:59 +02:00
Wladimir J. van der Laan
2c2d988062
Merge #10546: Remove 33 unused Boost includes
49de096 Remove unused Boost includes (practicalswift)

Tree-SHA512: ff2dad94f9eeb1dc50fcffd0e94e1686be04e4e5bc45f58ae955d630c15cd25bb8f5583d0aa1f2f263b5a723be79747ef3c6e6b417c7be7787c0abc8d1874019
2017-06-11 15:37:15 +02:00
Andrew Chow
c45cbaf69f Fix stopatheight
Moves stopatheight check into the ActivateBestChain loop so that the block height is actually checked and stopped at the appropriate time.
2017-06-09 13:40:45 -07:00
Matt Corallo
3533fb4d33 Return a bool in SpendCoin to restore pre-per-utxo assert semantics
Since its free to do so, assert that Spends succeeded when we expect
them to.
2017-06-09 13:10:05 -04:00
practicalswift
49de096c2a Remove unused Boost includes 2017-06-09 10:25:26 +02:00
Russell Yanofsky
24e44c354d Don't return stale data from CCoinsViewCache::Cursor()
CCoinsViewCache doesn't actually support cursor iteration returning the
current contents of the cache, so raise an error when the cursor method is
called instead of returning a cursor that iterates over stale data.

Also update the gettxoutsetinfo RPC which was relying on the old behavior to be
explicit about which view it is returning data about.
2017-06-07 13:00:11 -04:00
Matt Corallo
eada04e778 Do not print soft-fork-script warning with -promiscuousmempool 2017-06-07 11:02:36 -04:00
Matt Corallo
b5fea8d0cc Cache full script execution results in addition to signatures
This adds a new CuckooCache in validation, caching whether all of a
transaction's scripts were valid with a given set of script flags.

Unlike previous attempts at caching an entire transaction's
validity, which have nearly universally introduced consensus
failures, this only caches the validity of a transaction's
scriptSigs. As these are pure functions of the transaction and
data it commits to, this should be much safer.

This is somewhat duplicative with the sigcache, as entries in the
new cache will also have several entries in the sigcache. However,
the sigcache is kept both as ATMP relies on it and because it
prevents malleability-based DoS attacks on the new higher-level
cache. Instead, the -sigcachesize option is re-used - cutting the
sigcache size in half and using the newly freed memory for the
script execution cache.

Transactions which match the script execution cache never even have
entries in the script check thread's workqueue created.

Note that the cache is indexed only on the script execution flags
and the transaction's witness hash. While this is sufficient to
make the CScriptCheck() calls pure functions, this introduces
dependancies on the mempool calculating things such as the
PrecomputedTransactionData object, filling the CCoinsViewCache, etc
in the exact same way as ConnectBlock. I belive this is a reasonable
assumption, but should be noted carefully.

In a rather naive benchmark (reindex-chainstate up to block 284k
with cuckoocache always returning true for contains(),
-assumevalid=0 and a very large dbcache), this connected blocks
~1.7x faster.
2017-06-07 11:02:36 -04:00
Matt Corallo
6d22b2b17b Pull script verify flags calculation out of ConnectBlock 2017-06-07 11:02:36 -04:00
Mario Dian
24980a3e40 Make functions in validation.cpp static and pass chainparams
Fix bugs as per PR comment

Change bool static to static bool
2017-06-06 22:21:30 +08:00
Jorge Timón
1238f13cf6
scripted-diff: Remove PAIRTYPE
-BEGIN VERIFY SCRIPT-
sed -i 's/PAIRTYPE(\([^,]*\), \([^\)]*\))/std::pair<\1, \2>/' ./src/*.h ./src/*.cpp ./src/*/*.h ./src/*/*.cpp ./src/*/*/*.h ./src/*/*/*.cpp ;
sed -i ':a;N;$!ba;s/#define std::pair<t1, t2>    std::pair<t1, t2>\n//' ./src/utilstrencodings.h ;
-END VERIFY SCRIPT-
2017-06-05 20:14:53 +02:00
Jorge Timón
7c00c26726
scripted-diff: Fully remove BOOST_FOREACH
-BEGIN VERIFY SCRIPT-
sed -i 's/BOOST_FOREACH *(\(.*\),/for (\1 :/' ./src/*.h ./src/*.cpp ./src/*/*.h ./src/*/*.cpp ./src/*/*/*.h ./src/*/*/*.cpp ;
-END VERIFY SCRIPT-
2017-06-05 20:10:50 +02:00
Pieter Wuille
d9bec888f1 Use REJECT_DUPLICATE for already known and conflicted txn 2017-06-02 00:18:47 -07:00
Pieter Wuille
1088b02f0c
Merge #10195: Switch chainstate db and cache to per-txout model
589827975 scripted-diff: various renames for per-utxo consistency (Pieter Wuille)
a5e02bc7f Increase travis unit test timeout (Pieter Wuille)
73de2c1ff Rename CCoinsCacheEntry::coins to coin (Pieter Wuille)
119e552f7 Merge CCoinsViewCache's GetOutputFor and AccessCoin (Pieter Wuille)
580b02309 [MOVEONLY] Move old CCoins class to txdb.cpp (Pieter Wuille)
8b25d2c0c Upgrade from per-tx database to per-txout (Pieter Wuille)
b2af357f3 Reduce reserved memory space for flushing (Pieter Wuille)
41aa5b79a Pack Coin more tightly (Pieter Wuille)
97072d668 Remove unused CCoins methods (Pieter Wuille)
ce23efaa5 Extend coins_tests (Pieter Wuille)
508307968 Switch CCoinsView and chainstate db from per-txid to per-txout (Pieter Wuille)
4ec0d9e79 Refactor GetUTXOStats in preparation for per-COutPoint iteration (Pieter Wuille)
13870b56f Replace CCoins-based CTxMemPool::pruneSpent with isSpent (Pieter Wuille)
05293f3cb Remove ModifyCoins/ModifyNewCoins (Pieter Wuille)
961e48397 Switch tests from ModifyCoins to AddCoin/SpendCoin (Pieter Wuille)
8b3868c1b Switch CScriptCheck to use Coin instead of CCoins (Pieter Wuille)
c87b957a3 Only pass things committed to by tx's witness hash to CScriptCheck (Matt Corallo)
f68cdfe92 Switch from per-tx to per-txout CCoinsViewCache methods in some places (Pieter Wuille)
000391132 Introduce new per-txout CCoinsViewCache functions (Pieter Wuille)
bd83111a0 Optimization: Coin&& to ApplyTxInUndo (Pieter Wuille)
cb2c7fdac Replace CTxInUndo with Coin (Pieter Wuille)
422634e2f Introduce Coin, a single unspent output (Pieter Wuille)
7d991b55d Store/allow tx metadata in all undo records (Pieter Wuille)
c3aa0c119 Report on-disk size in gettxoutsetinfo (Pieter Wuille)
d34242430 Remove/ignore tx version in utxo and undo (Pieter Wuille)
7e0032290 Add specialization of SipHash for 256 + 32 bit data (Pieter Wuille)
e484652fc Introduce CHashVerifier to hash read data (Pieter Wuille)
f54580e7e error() in disconnect for disk corruption, not inconsistency (Pieter Wuille)
e66dbde6d Add SizeEstimate to CDBBatch (Pieter Wuille)

Tree-SHA512: ce1fb1e40c77d38915cd02189fab7a8b125c7f44d425c85579d872c3bede3a437760997907c99d7b3017ced1c2de54b2ac7223d99d83a6658fe5ef61edef1de3
2017-06-01 16:20:27 -07:00
Pieter Wuille
589827975f scripted-diff: various renames for per-utxo consistency
Thanks to John Newberry for pointing these out.

-BEGIN VERIFY SCRIPT-
sed -i 's/\<GetCoins\>/GetCoin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<HaveCoins\>/HaveCoin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<HaveCoinsInCache\>/HaveCoinInCache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<IsPruned\>/IsSpent/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<FetchCoins\>/FetchCoin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<CoinsEntry\>/CoinEntry/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<vHashTxnToUncache\>/coins_to_uncache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<vHashTxToUncache\>/coins_to_uncache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<fHadTxInCache\>/had_coin_in_cache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<coinbaseids\>/coinbase_coins/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<disconnectedids\>/disconnected_coins/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<duplicateids\>/duplicate_coins/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<oldcoins\>/old_coin/g' src/test/coins_tests.cpp
sed -i 's/\<origcoins\>/orig_coin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
-END VERIFY SCRIPT-
2017-06-01 13:15:25 -07:00
Pieter Wuille
b2af357f39 Reduce reserved memory space for flushing
As the maximum amount of data that can be pulled into the cache due to
a block validation is much lower now (at most one CCoin entry per input
and per output), reduce the conservative estimate used to determine
flushing time.
2017-06-01 13:15:06 -07:00
Pieter Wuille
5083079688 Switch CCoinsView and chainstate db from per-txid to per-txout
This patch makes several related changes:
* Changes the CCoinsView virtual methods (GetCoins, HaveCoins, ...)
  to be COutPoint/Coin-based rather than txid/CCoins-based.
* Changes the chainstate db to a new incompatible format that is also
  COutPoint/Coin based.
* Implements reconstruction code for hash_serialized_2.
* Adapts the coins_tests unit tests (thanks to Russell Yanofsky).

A side effect of the new CCoinsView model is that we can no longer
use the (unreliable) test for transaction outputs in the UTXO set
to determine whether we already have a particular transaction.
2017-06-01 12:59:38 -07:00
Pieter Wuille
8b3868c1b4 Switch CScriptCheck to use Coin instead of CCoins 2017-06-01 11:56:06 -07:00
Matt Corallo
c87b957a32 Only pass things committed to by tx's witness hash to CScriptCheck
This clarifies a bit more the ways in which the new script execution
cache could break consensus in the future if additional data from
the CCoins object were to be used as a part of script execution.

After this change, any such consensus breaks should be very visible
to reviewers, hopefully ensuring no such changes can be made.
2017-06-01 11:56:06 -07:00
Pieter Wuille
f68cdfe92b Switch from per-tx to per-txout CCoinsViewCache methods in some places 2017-06-01 11:56:06 -07:00
Pieter Wuille
bd83111a0f Optimization: Coin&& to ApplyTxInUndo
This avoids a prevector copy in ApplyTxInUndo.
2017-06-01 11:56:06 -07:00
Pieter Wuille
cb2c7fdac2 Replace CTxInUndo with Coin
The earlier CTxInUndo class now holds the same information as the Coin
class. Instead of duplicating functionality, replace CTxInUndo with a
serialization adapter for Coin.
2017-06-01 11:56:06 -07:00
Karl-Johan Alm
a473eff8f9
[bench] Replace 0.00(000)1 with MICRO/MILLI #defines in validation.cpp. 2017-06-01 11:43:04 +09:00
Karl-Johan Alm
5f850b0311
[bench] Include ms/blk stats in Connect* benchmarks. 2017-06-01 11:43:04 +09:00
practicalswift
5a9b508279 [trivial] Add end of namespace comments 2017-05-31 22:21:25 +02:00
Pieter Wuille
f94b7d5bfa
Merge #10464: Introduce static DoWarning (simplify UpdateTip)
f28514576 Introduce static DoWarning (simplify UpdateTip) (Jorge Timón)

Tree-SHA512: 58b39f00bc921d47d199ef1cc15e9e36f918ec86e9182de46dc2f24fa9474853a92b18d330a274a3ea7c9b001681fdfa3d86aa2bbebb5c92ac8c5a6f7690dfc1
2017-05-30 17:23:30 -07:00
Wladimir J. van der Laan
acd9957b72
Merge #9208: Improve DisconnectTip performance
c1235e3 Add RecursiveDynamicUsage overload for std::shared_ptr (Russell Yanofsky)
71f1903 Store disconnected block transactions outside mempool during reorg (Suhas Daftuar)
9decd64 [qa] Relax assumptions on mempool behavior during reorg (Suhas Daftuar)

Tree-SHA512: c160ad853a5cd060d0307af7606a0c77907497ed7033c9599b95e73d83f68fdfcd4214bd8a83db1c5b7a58022722b9de1ed2e6ea2e02f38a7b6c717f079dd0c6
2017-05-30 18:43:03 +02:00
Jorge Timón
f285145764
Introduce static DoWarning (simplify UpdateTip) 2017-05-27 06:33:39 +02:00
Pieter Wuille
7d991b55db Store/allow tx metadata in all undo records
Previously, transaction metadata (height, coinbase or not, and before
the previous commit also nVersion) was only stored for undo records
that correspond to the last output of a transaction being spent.

This only saves 2 bytes per undo record. Change this to storing this
information for every undo record, and stop complaining for having it
in non-last output spends. This means that undo dat written with
this patch won't be readable by older versions anymore.
2017-05-26 13:33:39 -07:00
Pieter Wuille
d342424301 Remove/ignore tx version in utxo and undo
This makes the following changes:
* In undo data and the chainstate database, the transaction nVersion
  field is removed from the data structures, always written as 0, and
  ignored when reading.
* The definition of hash_serialized in gettxoutsetinfo is changed to no
  longer incude the nVersion field. It is renamed to hash_serialized_2
  to avoid confusion. The new definition also includes transaction
  height and coinbase information, as this information was missing
  before.

This depends on having a CHashVerifier-based undo data checksum
verifier.

Apart from changing the definition of serialized_hash, downgrading
after using this patch is supported, as no release ever used the value
of nVersion field in UTXO entries.
2017-05-26 13:27:50 -07:00
Pieter Wuille
e484652fc3 Introduce CHashVerifier to hash read data
This is necessary later, when we drop the nVersion field from the undo
data. At that point deserializing and reserializing the data won't
roundtrip anymore, and thus that approach can't be used to verify
checksums anymore.

With this CHashVerifier approach, we can deserialize while hashing the
exact serialized form that was used. This is both more efficient and
more correct in that case.
2017-05-26 13:24:25 -07:00
Pieter Wuille
f54580e7e4 error() in disconnect for disk corruption, not inconsistency
The error() function unconditionally reports an error. It should only
be used for actually exception situations, and not for the type of
inconsistencies that ApplyTxInUndo/DisconnectBlock can graciously deal
with.

This also makes a subtle semantics change: in ApplyTxInUndo, when a
record with metadata is encountered (indicating it is the last spend
from a tx), don't wipe the CCoins record if it wasn't empty at that
point. This makes sure that UTXO operations never affect any other
UTXOs (including those from the same tx).
2017-05-26 13:24:25 -07:00
Pieter Wuille
f2f7e97e8c
Merge #10347: Use range-based for loops (C++11) when looping over vector elements
211adc0 Use range-based for loops (C++11) when looping over vector elements (practicalswift)

Tree-SHA512: 0e007f20dcef99d3c7a1036265e00f689d69f42e02fd82dd8389f45b52d31947e5f9388de2610d3d9bd9f554915ce0d35ebce561e5ae3a9013956d0ee4937145
2017-05-23 14:08:20 -07:00
Wladimir J. van der Laan
46771514fa
Merge #9571: RPC: getblockchaininfo returns BIP signaling statistics
557c9a6 RPC: getblockchaininfo: BIP9 stats (Matthew Zipkin)

Tree-SHA512: ecf0bf47f04f92becc77acc649fdfa270e768939acce42df39d30069398d40d9a30539862f7c307e08239f78d5c58c470ca5f6e717d2ab8e24db9be0dd7bec0c
2017-05-23 19:12:29 +02:00
practicalswift
211adc074a Use range-based for loops (C++11) when looping over vector elements 2017-05-19 09:56:16 +02:00
Jorge Timón
b324b28209
Globals: Pass Consensus::Params through CBlockTreeDB::LoadBlockIndexGuts() 2017-05-18 22:23:10 +02:00
Wladimir J. van der Laan
ea6fde3f1d
Merge #8329: Consensus: MOVEONLY: Move functions for tx verification
618d07f MOVEONLY: tx functions to consensus/tx_verify.o (Jorge Timón)

Tree-SHA512: 63fa2777c070a344dbfe61974526a770d962e049881c6f371b0034b1682c1e6e24f47454f01ee35ded20ade34488e023d4467a05369662906b99a73bb5de8497
2017-05-18 21:01:49 +02:00
Alex Morcos
65d484adf9 Output line to debug.log when IsInitialBlockDownload latches to false 2017-05-11 13:12:42 -04:00
Suhas Daftuar
71f1903353 Store disconnected block transactions outside mempool during reorg
Rather than re-add disconnected block transactions back to the mempool
immediately, store them in a separate disconnectpool for later processing,
because we expect most such transactions to reappear in the chain that is
still to be connected (and thus we can avoid the work of reprocessing those
transactions through the mempool altogether).
2017-05-04 16:50:52 -04:00
Pieter Wuille
431a548faa
Merge #10297: Simplify DisconnectBlock arguments/return value
db994b2 Simplify DisconnectBlock arguments/return value (Pieter Wuille)

Tree-SHA512: 62ce1a85bde2a5baffb9173ed28f2d8008200ecf8b09332122f1516fe68b33b9d7223cc1c2fffe804e38f767874c6353b76bd483e8ad7d48c4a5e80d6b683039
2017-05-02 20:42:15 -07:00
Wladimir J. van der Laan
f4a6180607
Merge #10175: Remove excess logic.
9a763d4 Remove excess logic. (practicalswift)

Tree-SHA512: cfef280cc9cccf5042c9c49a8cd6fb6700764671cdd0d988149121f56c2d43a9ac38c5fc99c92385a9619d2d846480e02a9d655fa2586b1c284961b4634c229b
2017-05-02 14:50:30 +02:00
Matt Corallo
70d39454ee Fix potential NPD introduced in b297426c
See https://github.com/bitcoin/bitcoin/pull/10290#discussion_r113954232
for more info.
2017-05-01 11:51:06 -04:00
Pieter Wuille
db994b2de9 Simplify DisconnectBlock arguments/return value
DisconnectBlock currently has a complicated interface:

  Situation       Return value
                  pfClean != nullptr   pfClean == nullptr

  All good:       true                 true
  Failure:        false                false
  Unclean rewind: true                 false
                  with *pfClean=false

Change this to return a tristate enum instead. As an added bonus,
remove the ValidationState& argument which was unused.
2017-04-28 16:15:01 -07:00
Pieter Wuille
b297426c96 Add -stopatheight for benchmarking 2017-04-27 14:39:27 -07:00
Wladimir J. van der Laan
a6548a47a5
Merge #10201: pass Consensus::Params& to ReceivedBlockTransactions()
25660e9 pass Consensus::Params& to ReceivedBlockTransactions() (Mario Dian)

Tree-SHA512: d3a5b19d93313e4bda622b322bc9cbfb7e31486010eac40fca6eea9703f814f9667f778122ba7366bb304482a2c03e2e3325083beecac374751692361952e467
2017-04-21 18:42:06 +02:00
Wladimir J. van der Laan
86ea3c2ff2
Merge #10181: Include cleanup
1c897fc Missing includes (Jorge Timón)
a1fd450 Trivial: Remove unneeded includes from .h: (Jorge Timón)

Tree-SHA512: ada3e62cc2435e58172a88b380be371b717a05725956c15e5493b6e19fe2903e5e6e43fd22dc24699333a0e8a0c7b42eb1ae61b41cb4ba82495be18e2d4ef3c6
2017-04-20 23:24:00 +02:00
Wladimir J. van der Laan
14c948987f
Merge #9942: Refactor CBlockPolicyEstimator
68af651 MOVEONLY: move TxConfirmStats to cpp (Alex Morcos)
2332f19 Initialize TxConfirmStats in constructor (Alex Morcos)
5ba81e5 Read and Write fee estimate file directly from CBlockPolicyEstimator (Alex Morcos)
14e10aa Call estimate(Smart)Fee directly from CBlockPolicyEstimator (Alex Morcos)
dbb9e36 Give CBlockPolicyEstimator it's own lock (Alex Morcos)
f6187d6 Make processBlockTx private. (Alex Morcos)
ae7327b Make feeEstimator its own global instance of CBlockPolicyEstimator (Alex Morcos)

Tree-SHA512: dbf3bd2b30822e609a35f3da519b62d23f8a50e564750695ddebd08553b4c01874ae3e07d792c6cc78cc377d2db33b951ffedc46ac7edaf5793f9ebb931713af
2017-04-20 21:17:17 +02:00
Mario Dian
25660e91ef pass Consensus::Params& to ReceivedBlockTransactions() 2017-04-13 22:37:46 +08:00
Matt Corallo
9fececb2cb Remove CValidationInterface::UpdatedTransaction
This removes another callback from block connection logic, making it
easier to reason about the wallet-RPCs-returns-stale-info issue.

UpdatedTransaction was previously used by the GUI to display
coinbase transactions only after they have a block built on top of
them. This worked fine for in most cases, but only worked due to a
corner case if the user received a coinbase payout in a block
immediately prior to restart. In that case, the normal process of
caching the most recent coinbase transaction's hash would not work,
and instead it would only work because of the on-load -checkblocks
calling DisconnectBlock and ConnectBlock on the current tip.

In order to make this more robust, a full mapWallet loop after the
first block which is connected after restart was added.
2017-04-13 10:36:21 -04:00
Jorge Timón
a1fd450328
Trivial: Remove unneeded includes from .h:
- validation.h doesn't need to include chain.h anymore
- Remove unneeded includes from net.h
2017-04-13 02:27:27 +02:00
Matt Corallo
d89f8adf25 Make DisconnectBlock and ConnectBlock static in validation.cpp 2017-04-10 16:20:12 -04:00
Wladimir J. van der Laan
67023e9004
Merge #9725: CValidationInterface Cleanups
b1a6d4c Take a CTransactionRef in AddToWalletIfInvolvingMe to avoid a copy (Matt Corallo)
1c95e2f Use std::shared_ptr instead of boost::shared_ptr in ScriptForMining (Matt Corallo)
91f1e6c Remove dead-code tracking of requests for blocks we generated (Matt Corallo)
acad82f Add override to functions using CValidationInterface methods (Matt Corallo)
e6d5e6c Hold cs_wallet for whole block [dis]connection processing (Matt Corallo)
461e49f SyncTransaction->TxAddedToMempool/BlockConnected/Disconnected (Matt Corallo)
f404334 Handle SyncTransaction in ActivateBestChain instead of ConnectTrace (Matt Corallo)
a147687 Keep conflictedTxs in ConnectTrace per-block (Matt Corallo)
d3167ba Handle conflicted transactions directly in ConnectTrace (Matt Corallo)
29e6e23 Make ConnectTrace::blocksConnected private, hide behind accessors (Matt Corallo)
822000c Add pblock to connectTrace at the end of ConnectTip, not start (Matt Corallo)
f5e9a01 Include missing #include in zmqnotificationinterface.h (Matt Corallo)

Tree-SHA512: 8893d47559da3b28d2ef7359768547cba8a4b43b6f891d80f5848f995a84b1517bfb0f706fdc8cd43f09a1350349eb440d9724a59363ab517dfcc4fcb31b2018
2017-04-10 21:21:01 +02:00
Alex Morcos
ae7327b832 Make feeEstimator its own global instance of CBlockPolicyEstimator 2017-04-10 13:51:51 -04:00
Wladimir J. van der Laan
e19586a8a9
Merge #10135: [p2p] Send the correct error code in reject messages
5d08c9c Send the correct error code in reject messages (John Newbery)

Tree-SHA512: 0cd3ef3ae202584b138cc0bbfba4125635822e0c5a755fb9276a604b39286959ab22dabc3104aa5d7e71358cd69d965de2a333ff04bf3e8ed43cf0296ac01264
2017-04-10 14:44:22 +02:00
practicalswift
9a763d4f86 Remove excess logic. 2017-04-10 10:24:49 +02:00
Matt Corallo
461e49fee2 SyncTransaction->TxAddedToMempool/BlockConnected/Disconnected
This simplifies fixing the wallet-returns-stale-info issue as we
can now hold cs_wallet across an entire block instead of only
per-tx (though we only actually do so in the next commit).

This change also removes the NOT_IN_BLOCK constant in favor of only
passing the CBlockIndex* parameter to SyncTransactions when a new
block is being connected, instead of also when a block is being
disconnected.

This change adds a parameter to BlockConnectedDisconnected which
lists the transactions which were removed from mempool due to
confliction as a result of this operation. While its somewhat of a
shame to make block-validation-logic generate a list of mempool
changes to be included in its generated callbacks, fixing this isnt
too hard.

Further in this change-set, CValidationInterface starts listening
to mempool directly, placing it in the middle and giving it a bit
of logic to know how to route notifications from block-validation,
mempool, etc (though not listening for conflicted-removals yet).
2017-04-07 11:53:43 +02:00
Matt Corallo
f404334910 Handle SyncTransaction in ActivateBestChain instead of ConnectTrace
This makes a later change to move it all into one per-block callback
simpler.
2017-04-07 11:53:43 +02:00
Matt Corallo
a1476877ce Keep conflictedTxs in ConnectTrace per-block 2017-04-07 11:53:43 +02:00
Matt Corallo
d3167ba9bb Handle conflicted transactions directly in ConnectTrace 2017-04-07 11:53:42 +02:00
Matt Corallo
29e6e231c8 Make ConnectTrace::blocksConnected private, hide behind accessors 2017-04-07 11:53:42 +02:00
Matt Corallo
822000cf82 Add pblock to connectTrace at the end of ConnectTip, not start
This makes ConnectTip responsible for the ConnectTrace instead
of splitting the logic between ActivateBestChainStep and ConnectTip
2017-04-07 11:53:41 +02:00
Jorge Timón
618d07faa2
MOVEONLY: tx functions to consensus/tx_verify.o
Functions related to transaction verification.
2017-04-06 23:36:46 +02:00
Wladimir J. van der Laan
8c28670e92
Merge #9902: Lightweight abstraction of boost::filesystem
f110272 Remove `namespace fs=fs` (Wladimir J. van der Laan)
75594bd torcontrol: Use fs::path instead of std::string for private key path (Wladimir J. van der Laan)
2a5f574 Use fsbridge for fopen and freopen (Wladimir J. van der Laan)
bac5c9c Replace uses of boost::filesystem with fs (Wladimir J. van der Laan)
7d5172d Replace includes of boost/filesystem.h with fs.h (Wladimir J. van der Laan)
19e36bb Add fs.cpp/h (Wladimir J. van der Laan)

Tree-SHA512: 2c34f059dfa6850b9323f3389e9090a6b5f839a457a2960d182c2ecfafd9883c956f5928bb796613402d3aad68ebc78259796a7a313f4a6cfa98aaf507a66842
2017-04-06 20:35:15 +02:00
Wladimir J. van der Laan
5fc6a77aa6
Merge #10133: Clean up calculations of pcoinsTip memory usage
1b55e07 Make threshold for flushing more conservative. (Alex Morcos)
f33afd3 Lower default memory footprint slightly (Alex Morcos)
5b95a19 Make pcoinsTip memory calculations consistent (Alex Morcos)

Tree-SHA512: d0061138596cf89008397b8729d9b25293938b1ad454cc99a6fe2f6210e94f76dfa78a8f0fce4c1ba3efec4e742a9c1a3ab26676a4a8346d3e7c3055d032669b
2017-04-05 08:10:43 +02:00
Wladimir J. van der Laan
2a5f574762 Use fsbridge for fopen and freopen
Abstracts away how a path is opened to a `FILE*`.

Reduces the number of places where path is converted to a string
for anything else but printing.
2017-04-03 12:32:32 +02:00
Wladimir J. van der Laan
bac5c9cf64 Replace uses of boost::filesystem with fs
Step two in abstracting away boost::filesystem.

To repeat this, simply run:
```
git ls-files \*.cpp \*.h | xargs sed -i 's/boost::filesystem/fs/g'
```
2017-04-03 12:32:32 +02:00
Wladimir J. van der Laan
7d5172d354 Replace includes of boost/filesystem.h with fs.h
This is step one in abstracting the use of boost::filesystem.
2017-04-03 12:32:32 +02:00
Gregory Maxwell
ada0caa165 Make GetWitnessCommitmentIndex callable on blocks without a coinbase txn.
This isn't actually needed anywhere, but it's less brittle.
2017-04-02 21:28:17 +00:00
Gregory Maxwell
6b3bb3d9ba Change LogAcceptCategory to use uint32_t rather than sets of strings.
This changes the logging categories to boolean flags instead of strings.

This simplifies the acceptance testing by avoiding accessing a scoped
 static thread local pointer to a thread local set of strings.  It
 eliminates the only use of boost::thread_specific_ptr outside of
 lockorder debugging.

This change allows log entries to be directed to multiple categories
 and makes it easy to change the logging flags at runtime (e.g. via
 an RPC, though that isn't done by this commit.)

It also eliminates the fDebug global.

Configuration of unknown logging categories now produces a warning.
2017-04-01 18:53:29 +00:00
Alex Morcos
1b55e07b7a Make threshold for flushing more conservative.
Always leave a reasonable buffer of 50MB for usage from newly connected block (once over 50%) and increase the high water mark buffer to 200MB.
2017-03-31 14:30:31 -04:00
John Newbery
5d08c9c579 Send the correct error code in reject messages 2017-03-31 14:22:25 -04:00
Alex Morcos
5b95a190e8 Make pcoinsTip memory calculations consistent
Since we are more accurately measuring pcoinsTip peak usage at twice the current in dynamic usage, it makes sense to double the default (this will lead to the same effective usage and peak usage as previously).
We should also double the buffer used to avoid flushing if above 90% but still sufficient space remaining.
2017-03-31 13:15:39 -04:00
Pieter Wuille
7228ce853d Compensate for memory peak at flush time 2017-03-30 12:05:05 -07:00
Matthew Zipkin
557c9a68fb RPC: getblockchaininfo: BIP9 stats
add RPC tests for BIP9 counting stats
2017-03-24 16:57:05 -07:00
practicalswift
dbf30ff10f [trivial] Fix typos in comments 2017-03-21 19:49:08 +01:00
Wladimir J. van der Laan
1b046603b3
Merge #8665: Assert all the things!
4d51e9b Assert ConnectBlock block and pIndex are the same block (NicolasDorier)
972714c pow: GetNextWorkRequired never called with NULL pindexLast (Daniel Cousens)
cc44c8f ContextualCheckBlockHeader should never have pindexPrev to NULL (NicolasDorier)

Tree-SHA512: 7cc568bf9417267c335f21ec3d1505b26e56e5b3d5f4d3dbb555279489800aaa65a3bcd7bc376e274dd102912aec16ddbb18de2e2060b2667b41eb979cd9321e
2017-03-14 10:38:27 +01:00
Wladimir J. van der Laan
47510ad3dd
Merge #9548: Remove min reasonable fee
ad82cb0 Remove unnecessary min fee argument in CTxMemPool constructor (Alex Morcos)
2a7b56c CBlockPolicyEstimator now uses hard coded minimum bucket feerate (Alex Morcos)
ac9d3d2 Change fee estimation bucket limit variable names (Alex Morcos)

Tree-SHA512: 6e3bc7df3497ed60c7620845d222063e33a0238020f5c3316e61e0eff758078588ea8dd51196ceb59aa561ba106f8cdae62cebe521adb3247108bb49f15252d6
2017-03-07 19:49:00 +01:00
Wladimir J. van der Laan
30ff3a2fc9
Merge #9602: Remove coin age priority and free transactions - implementation
b421e6d Update example bitcoin.conf (Alex Morcos)
7d4e950 Allow setting minrelaytxfee to 0 (Alex Morcos)
359e8a0 [cleanup] Remove coin age priority completely. (Alex Morcos)
f9b9371 [rpc] Remove priorityDelta from prioritisetransaction (Alex Morcos)
49be7e1 [rpc] Remove priority information from mempool RPC calls (Alex Morcos)
0315888 [test] Remove priority from tests (Alex Morcos)
f838005 No longer allow "free" transactions (Alex Morcos)
ad727f4 [rpc] sendrawtransaction no longer bypasses minRelayTxFee (Alex Morcos)
fe282ac [cleanup] Remove estimatePriority and estimateSmartPriority (Alex Morcos)
400b151 [debug] Change -printpriority option (Alex Morcos)
272b25a [mining] Remove -blockprioritysize. (Alex Morcos)
12839cd [rpc] Remove estimatepriority and estimatesmartpriority. (Alex Morcos)
ddf58c7 wallet: Remove sendfree (MarcoFalke)

Tree-SHA512: a9a4499405923ce794ef18f9e334dbbd59dfc73a3dc2df6f85cc9c62af6f353ec2eed9c2d5e58e904f918d0d7ab738f403dd4939d9bc2276136864fe63710782
2017-03-07 19:30:23 +01:00