Commit graph

459 commits

Author SHA1 Message Date
Wladimir J. van der Laan
7884956414
Merge #9511: Don't overwrite validation state with corruption check
116419e Don't overwrite validation state with corruption check (Alex Morcos)
2017-01-23 13:18:16 +01:00
Alex Morcos
de1ae324bf Exclude RBF txs from fee estimation 2017-01-20 15:04:16 -05:00
Alex Morcos
bd92f2481c [bugfix] save feeDelta instead of priorityDelta in DumpMempool 2017-01-19 23:45:02 -05:00
Alex Morcos
5b158707f2 Use incrementalRelayFee for BIP 125 replacement 2017-01-19 20:31:29 -05:00
Matt Corallo
989989354b Move wallet callbacks into cs_main (this effectively reverts #7946) 2017-01-19 11:31:05 -05:00
Wladimir J. van der Laan
9c9af5ab2d
Merge #9499: Use recent-rejects, orphans, and recently-replaced txn for compact-block-reconstruction
c594580 Add braces around AddToCompactExtraTransactions (Matt Corallo)
1ccfe9b Clarify comment about mempool/extra conflicts (Matt Corallo)
fac4c78 Make PartiallyDownloadedBlock::InitData's second param const (Matt Corallo)
b55b416 Add extra_count lower bound to compact reconstruction debug print (Matt Corallo)
863edb4 Consider all (<100k memusage) txn for compact-block-extra-txn cache (Matt Corallo)
7f8c8ca Consider all orphan txn for compact-block-extra-txn cache (Matt Corallo)
93380c5 Use replaced transactions in compact block reconstruction (Matt Corallo)
1531652 Keep shared_ptrs to recently-replaced txn for compact blocks (Matt Corallo)
edded80 Make ATMP optionally return the CTransactionRefs it replaced (Matt Corallo)
c735540 Move ORPHAN constants from validation.h to net_processing.h (Matt Corallo)
2017-01-19 09:03:46 +01:00
Alex Morcos
ad82cb06ce Remove unnecessary min fee argument in CTxMemPool constructor 2017-01-17 13:21:35 -05:00
Pieter Wuille
812714fd80
Merge #9484: Introduce assumevalid setting to skip validation presumed valid scripts.
7b5e3fe Add assumevalid testcase (John Newbery)
e440ac7 Introduce assumevalid setting to skip presumed valid scripts. (Gregory Maxwell)
2017-01-16 14:07:30 -08:00
Pieter Wuille
e126d0c12c
Merge #9490: Replace FindLatestBefore used by importmuti with FindEarliestAtLeast.
4b06e41 Add unit test for FindEarliestAtLeast (Suhas Daftuar)
997a98a Replace FindLatestBefore used by importmuti with FindEarliestAtLeast. (Gregory Maxwell)
2017-01-13 16:06:05 -08:00
Pieter Wuille
3908fc4728
Merge #9375: Relay compact block messages prior to full block connection
02ee4eb Make most_recent_compact_block a pointer to a const (Matt Corallo)
73666ad Add comment to describe callers to ActivateBestChain (Matt Corallo)
962f7f0 Call ActivateBestChain without cs_main/with most_recent_block (Matt Corallo)
0df777d Use a temp pindex to avoid a const_cast in ProcessNewBlockHeaders (Matt Corallo)
c1ae4fc Avoid holding cs_most_recent_block while calling ReadBlockFromDisk (Matt Corallo)
9eb67f5 Ensure we meet the BIP 152 old-relay-types response requirements (Matt Corallo)
5749a85 Cache most-recently-connected compact block (Matt Corallo)
9eaec08 Cache most-recently-announced block's shared_ptr (Matt Corallo)
c802092 Relay compact block messages prior to full block connection (Matt Corallo)
6987219 Add a CValidationInterface::NewPoWValidBlock callback (Matt Corallo)
180586f Call AcceptBlock with the block's shared_ptr instead of CBlock& (Matt Corallo)
8baaba6 [qa] Avoid race in preciousblock test. (Matt Corallo)
9a0b2f4 [qa] Make compact blocks test construction using fetch methods (Matt Corallo)
8017547 Make CBlockIndex*es in net_processing const (Matt Corallo)
2017-01-13 14:52:26 -08:00
Gregory Maxwell
e440ac7ef3 Introduce assumevalid setting to skip presumed valid scripts.
This disentangles the script validation skipping from checkpoints.

A new option is introduced "assumevalid" which specifies a block whos
 ancestors we assume all have valid scriptsigs and so we do not check
 them when they are also burried under the best header by two weeks
 worth of work.

Unlike checkpoints this has no influence on consensus unless you set
 it to a block with an invalid history.  Because of this it can be
 easily be updated without risk of influencing the network consensus.

This results in a massive IBD speedup.

This approach was independently recommended by Peter Todd and Luke-Jr
 since POW based signature skipping (see PR#9180) does not have the
 verifiable properties of a specific hash and may create bad incentives.

The downside is that, like checkpoints, the defaults bitrot and older
 releases will sync slower.  On the plus side users can provide their
 own value here, and if they set it to something crazy all that will
 happen is more time will be spend validating signatures.

Checkblocks and checklevel are also moved to the hidden debug options:
 Especially now that checkblocks has a low default there is little need
 to change these settings, and users frequently misunderstand them as
 influencing security or IBD speed.  By hiding them we offset the
 space added by this new option.
2017-01-13 15:42:24 +00:00
Matt Corallo
73666ad059 Add comment to describe callers to ActivateBestChain 2017-01-12 12:15:17 -08:00
Gregory Maxwell
997a98a674 Replace FindLatestBefore used by importmuti with FindEarliestAtLeast.
In spite of the name FindLatestBefore used std::lower_bound to try
 to find the earliest block with a nTime greater or equal to the
 the requested value.  But lower_bound uses bisection and requires
 the input to be ordered with respect to the comparison operation.
 Block times are not well ordered.

I don't know what lower_bound is permitted to do when the data
 is not sufficiently ordered, but it's probably not good.
 (I could construct an implementation which would infinite loop...)

To resolve the issue this commit introduces a maximum-so-far to the
 block indexes and searches that.

For clarity the function is renamed to reflect what it actually does.

An issue that remains is that there is no grace period in importmulti:
 If a address is created at time T and a send is immediately broadcast
 and included by a miner with a slow clock there may not yet have been
 any block with at least time T.

The normal rescan has a grace period of 7200 seconds, but importmulti
 does not.
2017-01-12 14:21:43 +00:00
Wladimir J. van der Laan
fac0f30482
Merge #9472: Disentangle progress estimation from checkpoints and update it
df36371 Update estimated transaction count data (Pieter Wuille)
e356d9a Shorten variable names and switch to tx/s (Pieter Wuille)
6dd8116 Remove SIGCHECK_VERIFICATION_FACTOR (Pieter Wuille)
3641141 Move tx estimation data out of CCheckPointData (Pieter Wuille)
a4bac66 [MOVEONLY] Move progress estimation out of checkpoints (Pieter Wuille)
2017-01-12 12:13:14 +01:00
Matt Corallo
0df777db6d Use a temp pindex to avoid a const_cast in ProcessNewBlockHeaders 2017-01-11 14:47:52 -08:00
Pieter Wuille
e356d9a758 Shorten variable names and switch to tx/s 2017-01-11 08:21:37 -08:00
Alex Morcos
116419e58d Don't overwrite validation state with corruption check
AcceptToMemoryPool has several classes of return false statements.
- return state.Invalid or state.DoS directly itself
- return false and set fMissingInputs (state is valid)
- return false and state is set by failed CheckTransaction
- return false and state is set by failed CheckInputs.

This commit patches the last case where the state variable was reused for additional calls to CheckInputs to identify witness stripping as cause of validation failure.  After this commit, it should be the case that if !fMissingInputs, state is always Invalid if AcceptToMemoryPool returns false.
2017-01-10 16:53:10 -05:00
mrbandrews
1fc4ec7bf2 Add pruneblockchain RPC to enable manual block file pruning. 2017-01-10 08:14:50 -05:00
Matt Corallo
edded808fc Make ATMP optionally return the CTransactionRefs it replaced 2017-01-09 14:15:18 -05:00
Pieter Wuille
46b249e578
Merge #9408: Allow shutdown during LoadMempool, dump only when necessary
325e400 [Qt] Do proper shutdown (Jonas Schnelli)
9479f8d Allow shutdown during LoadMempool, dump only when necessary (Jonas Schnelli)
2017-01-06 09:37:43 -08:00
Pieter Wuille
f646275b90
Merge #9138: Improve fee estimation
44b64b9 Fix edge case with stale fee estimates (Alex Morcos)
78ae62d Add clarifying comments to fee estimation (Alex Morcos)
5fe0f47 Add extra logging to processBlock in fee estimation. (Alex Morcos)
dc008c4 Add IsCurrentForFeeEstimatation (Alex Morcos)
ebafdca Pass pointers to existing CTxMemPoolEntries to fee estimation (Alex Morcos)
d825838 Always update fee estimates on new blocks. (Alex Morcos)
6f06b26 rename bool to validFeeEstimate (Alex Morcos)
84f7ab0 Remove member variable hadNoDependencies from CTxMemPoolEntry (Alex Morcos)
60ac00d Don't track transactions at all during IBD. (Alex Morcos)
4df4479 Remove extraneous LogPrint from fee estimation (Alex Morcos)
2017-01-05 14:22:19 -08:00
Pieter Wuille
c252685aa5
Merge #8610: Share unused mempool memory with coincache
ba3cecf Share unused mempool memory with coincache (Pieter Wuille)
2017-01-05 13:52:24 -08:00
Matt Corallo
6987219577 Add a CValidationInterface::NewPoWValidBlock callback 2017-01-05 10:32:07 -05:00
Pieter Wuille
6dd81169fc Remove SIGCHECK_VERIFICATION_FACTOR 2017-01-04 13:18:49 -08:00
Pieter Wuille
3641141c8f Move tx estimation data out of CCheckPointData 2017-01-04 13:18:49 -08:00
Pieter Wuille
a4bac66cca [MOVEONLY] Move progress estimation out of checkpoints 2017-01-04 13:18:49 -08:00
Matt Corallo
180586fd44 Call AcceptBlock with the block's shared_ptr instead of CBlock& 2017-01-04 15:56:08 -05:00
Matt Corallo
80175472d1 Make CBlockIndex*es in net_processing const 2017-01-04 15:56:08 -05:00
Pieter Wuille
7dac1e5e9e
Merge #9107: Safer modify new coins
b50cd7a Fix dangerous condition in ModifyNewCoins. (Alex Morcos)
2017-01-04 11:56:17 -08:00
Alex Morcos
dc008c462f Add IsCurrentForFeeEstimatation
Make a more conservative notion of whether the node is caught up to the rest of the network and only count transactions as fee estimation data points if the node is caught up.
2017-01-04 12:10:17 -05:00
Alex Morcos
d825838e64 Always update fee estimates on new blocks.
All decisions about whether the transactions are valid data points are made at the time the transaction arrives. Updating on blocks all the time will now cause stale fee estimates to decay quickly when we restart a node.
2017-01-04 12:09:34 -05:00
Alex Morcos
84f7ab08d2 Remove member variable hadNoDependencies from CTxMemPoolEntry
Fee estimation can just check its own mapMemPoolTxs to determine the same information.  Note that now fee estimation for block processing must happen before those transactions are removed, but this shoudl be a speedup.
2017-01-04 12:09:33 -05:00
Alex Morcos
60ac00de85 Don't track transactions at all during IBD.
This was an oversight, where blocks and mempool tracking were ignored during IBD, but transactions that arrived during IBD but were included in blocks after IBD were not ignored.
2017-01-04 12:05:05 -05:00
Alex Morcos
b50cd7a67e Fix dangerous condition in ModifyNewCoins.
We were marking coins FRESH before being sure they were not overwriting dirty undo data. This condition was never reached in existing code because undo data was always flushed before UpdateCoins was called with new transactions, but could have been exposed in an otherwise safe refactor.
Clarify in the comments the assumptions made in ModifyNewCoins.
Add ability to undo transactions to UpdateCoins unit test.
Thanks to Russ Yanofsky for suggestion on how to make logic clearer and fixing up the ccoins_modify_new test cases.
2017-01-04 11:20:42 -05:00
Jonas Schnelli
9479f8dfcf
Allow shutdown during LoadMempool, dump only when necessary 2016-12-23 14:19:38 +01:00
Pieter Wuille
ba3cecf5c4 Share unused mempool memory with coincache
If the mempool is not completely full, treat the difference between
the maximum size and the actual usage as available for the coin cache.

This also changes the early flush trigger from (usage > 0.9 * space)
to (usage > 0.9 * space && usage > space - 100MB). This means we're not
permanently leaving 10% of the space unused when the space is large.
2016-12-22 15:04:05 -08:00
Pieter Wuille
c44e4c467c Make AcceptToMemoryPool take CTransactionRef 2016-12-21 18:18:23 -08:00
Wladimir J. van der Laan
e8cfe1ee2d
Merge #8589: Inline CTxInWitness inside CTxIn
f6fb7ac Move CTxInWitness inside CTxIn (Pieter Wuille)
2016-12-21 14:04:06 +01:00
Wladimir J. van der Laan
7f72568e6b
Merge #9236: Fix races for strMiscWarning and fLargeWork*Found, make QT runawayException use GetWarnings
749be01 Move GetWarnings() into its own file. (Gregory Maxwell)
e3ba0ef Eliminate data races for strMiscWarning and fLargeWork*Found. (Gregory Maxwell)
c63198f Make QT runawayException call GetWarnings instead of directly access strMiscWarning. (Gregory Maxwell)
2016-12-19 12:40:01 +01:00
Wladimir J. van der Laan
b68685a16a
Merge #9273: Remove unused CDiskBlockPos* argument from ProcessNewBlock
a13fa4c Remove unused CDiskBlockPos* argument from ProcessNewBlock (Matt Corallo)
2016-12-14 10:56:28 +01:00
Gregory Maxwell
da9cdd2c9c Do not run functions with necessary side-effects in assert() 2016-12-14 01:50:00 +00:00
Pieter Wuille
a1dcf2e108
Merge #9240: Remove txConflicted
a874ab5 remove internal tracking of mempool conflicts for reporting to wallet (Alex Morcos)
bf663f8 remove external usage of mempool conflict tracking (Alex Morcos)
2016-12-09 16:31:03 -08:00
Alex Morcos
b919179cbb remove no longer needed check for premature v2 txs 2016-12-07 09:58:20 -05:00
Alex Morcos
bf663f8e93 remove external usage of mempool conflict tracking 2016-12-05 13:41:25 -05:00
Pieter Wuille
f6fb7acda4 Move CTxInWitness inside CTxIn 2016-12-04 23:47:12 -08:00
Wladimir J. van der Laan
d04aebaec7
Merge #9014: Fix block-connection performance regression
dd0df81 Document ConnectBlock connectTrace postconditions (Matt Corallo)
2d6e561 Switch pblock in ProcessNewBlock to a shared_ptr (Matt Corallo)
2736c44 Make the optional pblock in ActivateBestChain a shared_ptr (Matt Corallo)
ae4db44 Create a shared_ptr for the block we're connecting in ActivateBCS (Matt Corallo)
fd9d890 Keep blocks as shared_ptrs, instead of copying txn in ConnectTip (Matt Corallo)
6fdd43b Add struct to track block-connect-time-generated info for callbacks (Matt Corallo)
2016-12-05 08:07:45 +01:00
Matt Corallo
a13fa4c80f Remove unused CDiskBlockPos* argument from ProcessNewBlock 2016-12-04 00:23:17 -08:00
Matt Corallo
dd0df81ebd Document ConnectBlock connectTrace postconditions 2016-12-04 00:18:44 -08:00
Matt Corallo
2d6e5619af Switch pblock in ProcessNewBlock to a shared_ptr
This (finally) fixes a performance regression in
b3b3c2a562
2016-12-04 00:17:30 -08:00
Matt Corallo
2736c44c8e Make the optional pblock in ActivateBestChain a shared_ptr 2016-12-04 00:13:09 -08:00
Matt Corallo
ae4db44d03 Create a shared_ptr for the block we're connecting in ActivateBCS 2016-12-04 00:13:09 -08:00
Matt Corallo
fd9d89070a Keep blocks as shared_ptrs, instead of copying txn in ConnectTip 2016-12-04 00:13:09 -08:00
Matt Corallo
6fdd43b968 Add struct to track block-connect-time-generated info for callbacks 2016-12-04 00:13:09 -08:00
Gregory Maxwell
749be013f5 Move GetWarnings() into its own file. 2016-12-03 07:17:34 +00:00
Gregory Maxwell
e3ba0ef956 Eliminate data races for strMiscWarning and fLargeWork*Found.
This moves all access to these datastructures through accessor functions
 and protects them with a lock.
2016-12-03 07:17:34 +00:00
Gregory Maxwell
c63198f1c7 Make QT runawayException call GetWarnings instead of directly access strMiscWarning.
This is a first step in avoiding racy accesses to strMiscWarning.

The change required moving GetWarnings and related globals to util.
2016-12-03 07:17:28 +00:00
Pieter Wuille
81e3228fcb Make CTransaction actually immutable 2016-12-02 18:37:43 -08:00
Pieter Wuille
a1883536b4 Switch GetTransaction to returning a CTransactionRef 2016-12-02 18:28:22 -08:00
Matt Corallo
76faa3cdfe Rename the remaining main.{h,cpp} to validation.{h,cpp} 2016-12-02 09:42:51 -08:00
Renamed from src/main.cpp (Browse further)