Commit graph

48 commits

Author SHA1 Message Date
Dimitris Tsapakidis 0a5a6b90bc Fixed multiple typos
A few "a->an" and "an->a".
"Shows, if the supplied default SOCKS5 proxy" -> "Shows if the supplied default SOCKS5 proxy". Change made on 3 occurrences.
"without fully understanding the ramification of a command" -> "without fully understanding the ramifications of a command".
Removed duplicate words such as "the the".
2017-06-22 19:18:10 +03:00
Alex Morcos 1bebfc8d3a Output Fee Estimation Calculations in CreateTransaction 2017-06-13 12:35:04 -04:00
practicalswift efc2e3302d [trivial] Fix three recently introduced typos
```
$ git blame src/policy/fees.cpp | grep becuase
3810e976 (2017-03-07 11:33:44 -0500 789)          * checks for 2*target becuase we are taking the max over all time
$ git blame src/policy/fees.h | grep successfullly
2d2e1705 (2017-04-12 12:29:03 -0400  54)  * representing that a tx was successfullly confirmed in less than or equal to
$ git blame src/wallet/feebumper.cpp | grep "hasen't"
a3878374 (2017-05-11 09:34:39 +0200 258)     // make sure the transaction still has no descendants and hasen't been mined in the meantime
```
2017-05-23 13:22:40 +02:00
Pieter Wuille 318ea50a1c
Merge #10199: Better fee estimates
38bc1ec Make more json-like output from estimaterawfee (Alex Morcos)
2d2e170 Comments and improved documentation (Alex Morcos)
ef589f8 minor cleanup: remove unnecessary variable (Alex Morcos)
3ee76d6 Introduce a scale factor (Alex Morcos)
5f1f0c6 Historical block span (Alex Morcos)
aa19b8e Clean up fee estimate debug printing (Alex Morcos)
10f7cbd Track first recorded height (Alex Morcos)
3810e97 Rewrite estimateSmartFee (Alex Morcos)
c7447ec Track failures in fee estimation. (Alex Morcos)
4186d3f Expose estimaterawfee (Alex Morcos)
2681153 minor refactor: explicitly track start of new bucket range and don't update curNearBucket on final loop. (Alex Morcos)
1ba43cc Make EstimateMedianVal smarter about small failures. (Alex Morcos)
d3e30bc Refactor to update moving average on fly (Alex Morcos)
e5007ba Change parameters for fee estimation and estimates on all 3 time horizons. (Alex Morcos)
c0a273f Change file format for fee estimates. (Alex Morcos)

Tree-SHA512: 186e7508d86a1f351bb656edcd84ee9091f5f2706331eda9ee29da9c8eb5bf67b8c1f2abf6662835560e7f613b1377099054f20767f41ddcdbc89c4f9e78946d
2017-05-17 13:15:07 -07:00
Alex Morcos 2d2e17052c Comments and improved documentation 2017-05-10 11:48:46 -04:00
Alex Morcos ef589f8d40 minor cleanup: remove unnecessary variable 2017-05-10 11:47:45 -04:00
Alex Morcos 3ee76d6de5 Introduce a scale factor
For the per confirmation number tracking of data, introduce a scale factor so that in the longer horizones confirmations are bucketed together at a resolution of the scale.  (instead of 1008 individual data points for each fee bucket, have 42 data points each covering 24 different confirmation values.. (1-24), (25-48), etc.. )
2017-05-10 11:47:45 -04:00
Alex Morcos 5f1f0c6490 Historical block span
Store in fee estimate file the block span for which we were tracking estimates, so we know what targets we can successfully evaluate with the data in the file. When restarting use either this historical block span to set valid range of targets until our current span of tracking estimates is just as long.
2017-05-10 11:47:45 -04:00
Alex Morcos 10f7cbd247 Track first recorded height
Track the first time we seen txs in a block that we have been tracking in our mempool. Used to evaluate validity of fee estimates for different targets.
2017-05-10 11:47:44 -04:00
Alex Morcos 3810e976d6 Rewrite estimateSmartFee
Change the logic of estimateSmartFee to check a 60% threshold at half the target, a 85% threshold at the target and a 95% threshold at double the target. Always check the shortest time horizon possible and ensure that estimates are monotonically decreasing.  Add a conservative mode, which makes sure that the 95% threshold is also met at longer time horizons as well.
2017-05-10 11:47:44 -04:00
Alex Morcos c7447ec303 Track failures in fee estimation.
Start tracking transactions which fail to confirm within the target and are then evicted or otherwise leave mempool.

Fix slight error in unit test.
2017-05-10 11:45:27 -04:00
Alex Morcos 4186d3fdfd Expose estimaterawfee
Track information the ranges of fee rates that were used to calculate the fee estimates (the last range of fee rates in which the data points met the threshold and the first to fail) and provide an RPC call to return this information.
2017-05-10 11:45:26 -04:00
Alex Morcos 1ba43cc0ec Make EstimateMedianVal smarter about small failures.
Instead of stopping if it encounters a "sufficient" number of transactions which don't meet the threshold for being confirmed within the target, it keeps looking to add more transactions to see if there is a temporary blip in the data.  This allows a smaller number of required data points.
2017-05-10 11:45:26 -04:00
Alex Morcos e5007bae35 Change parameters for fee estimation and estimates on all 3 time horizons.
Make feerate buckets smaller (5% instead of 10%) and make the 3 different horizons have half lifes of 3 hours, 1 day and 1 week respectively.
2017-05-10 11:45:26 -04:00
Jorge Timón 381a46e38f
Consensus: Policy: MOVEONLY: Move CFeeRate out of the consensus module
...from amount.o to policy/feerate.o

Policy, because it moves policy code to the policy directory (common module)
2017-05-03 18:00:13 +02:00
Alex Morcos c0a273f4c8 Change file format for fee estimates.
Move buckets and bucketMap to be stored as part of overall serialization of estimator.
Add some placeholder data so file format is only changed once.
Maintain 3 different TxConfirmStats with potential for different decays and scales.
2017-04-20 15:46:15 -04:00
Alex Morcos 68af651498 MOVEONLY: move TxConfirmStats to cpp 2017-04-10 13:56:50 -04:00
Alex Morcos 2332f19bef Initialize TxConfirmStats in constructor
and change to storing as a pointer.
2017-04-10 13:56:50 -04:00
Alex Morcos 5ba81e54e0 Read and Write fee estimate file directly from CBlockPolicyEstimator 2017-04-10 13:56:50 -04:00
Alex Morcos 14e10aa842 Call estimate(Smart)Fee directly from CBlockPolicyEstimator 2017-04-10 13:56:50 -04:00
Alex Morcos dbb9e3699b Give CBlockPolicyEstimator it's own lock 2017-04-10 13:51:51 -04:00
Alex Morcos f6187d6e39 Make processBlockTx private. 2017-04-10 13:51:51 -04:00
Wladimir J. van der Laan 47510ad3dd
Merge #9548: Remove min reasonable fee
ad82cb0 Remove unnecessary min fee argument in CTxMemPool constructor (Alex Morcos)
2a7b56c CBlockPolicyEstimator now uses hard coded minimum bucket feerate (Alex Morcos)
ac9d3d2 Change fee estimation bucket limit variable names (Alex Morcos)

Tree-SHA512: 6e3bc7df3497ed60c7620845d222063e33a0238020f5c3316e61e0eff758078588ea8dd51196ceb59aa561ba106f8cdae62cebe521adb3247108bb49f15252d6
2017-03-07 19:49:00 +01:00
Alex Morcos fe282acd76 [cleanup] Remove estimatePriority and estimateSmartPriority
Unused everywhere now except one test.
2017-02-27 11:23:51 -05:00
Alex Morcos 2a7b56cc0e CBlockPolicyEstimator now uses hard coded minimum bucket feerate 2017-01-17 13:21:31 -05:00
Alex Morcos ac9d3d25f7 Change fee estimation bucket limit variable names 2017-01-17 13:19:32 -05:00
Alex Morcos 5fe0f47aa7 Add extra logging to processBlock in fee estimation. 2017-01-04 12:10:18 -05:00
Alex Morcos ebafdcabb1 Pass pointers to existing CTxMemPoolEntries to fee estimation 2017-01-04 12:09:34 -05:00
Alex Morcos d825838e64 Always update fee estimates on new blocks.
All decisions about whether the transactions are valid data points are made at the time the transaction arrives. Updating on blocks all the time will now cause stale fee estimates to decay quickly when we restart a node.
2017-01-04 12:09:34 -05:00
Alex Morcos 6f06b268c1 rename bool to validFeeEstimate 2017-01-04 12:09:33 -05:00
Alex Morcos 4df44794c9 Remove extraneous LogPrint from fee estimation
Once priority estimation was removed, not all transactions in the mempool are tracked in the fee estimation mempool tracking.  So there is no error if a transaction is not found for removal.
2017-01-04 12:03:44 -05:00
Wladimir J. van der Laan 1ce7ede2a9
Merge #9288: Fix a bug if the min fee is 0 for FeeFilterRounder
f8d43b8 Avoid rollingMinimumFeeRate never being able to decay below half (Alex Morcos)
eab8e1b fix a bug if the min fee is 0 for FeeFilterRounder (Alex Morcos)
2017-01-04 12:33:56 +01:00
isle2983 27765b6403 Increment MIT Licence copyright header year on files modified in 2016
Edited via:

$ contrib/devtools/copyright_header.py update .
2016-12-31 11:01:21 -07:00
Alex Morcos eab8e1b172 fix a bug if the min fee is 0 for FeeFilterRounder 2016-12-08 15:40:14 -05:00
fanquake d8edf03db7
Remove unused var UNLIKELY_PCT from fees.h 2016-11-09 17:51:27 +08:00
Wladimir J. van der Laan 3c03dc2cfc
Merge #7730: Remove priority estimation
0bd581a add release notes for removal of priority estimation (Alex Morcos)
b2322e0 Remove priority estimation (Alex Morcos)
2016-11-07 13:36:40 +01:00
Alex Morcos b2322e0fc6 Remove priority estimation 2016-11-07 13:22:34 +01:00
isle2983 4b04e32c20 [copyright] copyright header style uniform
Three categories of modifications:

1)

1 instance of 'The Bitcoin Core developers \n',
1 instance of 'the Bitcoin Core developers\n',
3 instances of 'Bitcoin Core Developers\n', and
12 instances of 'The Bitcoin developers\n'

are made uniform with the 443 instances of 'The Bitcoin Core developers\n'

2)

3 instances of 'BitPay, Inc\.\n' are made uniform with the other 6
instances of 'BitPay Inc\.\n'

3)

4 instances where there was no '(c)' between the 'Copyright' and the year
where it deviates from the style of the local directory.
2016-11-06 10:12:50 -07:00
Wladimir J. van der Laan 5eaaa83ac1 Kill insecure_random and associated global state
There are only a few uses of `insecure_random` outside the tests.
This PR replaces uses of insecure_random (and its accompanying global
state) in the core code with an FastRandomContext that is automatically
seeded on creation.

This is meant to be used for inner loops. The FastRandomContext
can be in the outer scope, or the class itself, then rand32() is used
inside the loop. Useful e.g. for pushing addresses in CNode or the fee
rounding, or randomization for coin selection.

As a context is created per purpose, thus it gets rid of
cross-thread unprotected shared usage of a single set of globals, this
should also get rid of the potential race conditions.

- I'd say TxMempool::check is not called enough to warrant using a special
  fast random context, this is switched to GetRand() (open for
  discussion...)

- The use of `insecure_rand` in ConnectThroughProxy has been replaced by
  an atomic integer counter. The only goal here is to have a different
  credentials pair for each connection to go on a different Tor circuit,
  it does not need to be random nor unpredictable.

- To avoid having a FastRandomContext on every CNode, the context is
  passed into PushAddress as appropriate.

There remains an insecure_random for test usage in `test_random.h`.
2016-10-17 13:08:35 +02:00
MarcoFalke fada0c422c [doc] Fix doxygen comments for members 2016-04-03 11:58:01 +02:00
Alex Morcos 9e072a6e66 Implement "feefilter" P2P message.
The "feefilter" p2p message is used to inform other nodes of your mempool min fee which is the feerate that any new transaction must meet to be accepted to your mempool.  This will allow them to filter invs to you according to this feerate.
2016-03-21 10:46:25 -04:00
Chris Wheeler 9d263bd17c Typo fixes in comments 2016-01-17 11:03:56 +00:00
Suhas Daftuar e30443244a Pass reference to estimateSmartFee and cleanup whitespace 2015-11-24 08:53:14 -05:00
Alex Morcos 6303051470 EstimateSmart functions consider mempool min fee 2015-11-16 15:33:06 -05:00
Alex Morcos f22ac4a22c Increase success threshold for fee estimation to 95%
This provides more conservative estimates and reacts more quickly to a backlog.
Unfortunately the unit test for fee estimation depends on the success threshold (and the decay) chosen; also modify the unit test for the new default success thresholds.
2015-11-16 15:33:06 -05:00
Alex Morcos 22eca7da22 Add smart fee estimation functions
These are more useful fee and priority estimation functions. If there is no fee/pri high enough for the target you are aiming for, it will give you the estimate for the lowest target that you can reliably obtain.  This is better than defaulting to the minimum.  It will also pass back the target for which it returned an answer.
2015-11-16 15:33:06 -05:00
Veres Lajos 9f68ed6b6d typofixes (found by misspell_fixer) 2015-08-10 22:06:27 +01:00
Alex Morcos b649e03954 Create new BlockPolicyEstimator for fee estimates
This class groups transactions that have been confirmed in blocks into buckets, based on either their fee or their priority.  Then for each bucket, the class calculates what percentage of the transactions were confirmed within various numbers of blocks.  It does this by keeping an exponentially decaying moving history for each bucket and confirm block count of the percentage of transactions in that bucket that were confirmed within that number of blocks.

-Eliminate txs which didn't have all inputs available at entry from fee/pri calcs

-Add dynamic breakpoints and tracking of confirmation delays in mempool transactions

-Remove old CMinerPolicyEstimator and CBlockAverage code

-New smartfees.py

-Pass a flag to the estimation code, using IsInitialBlockDownload as a proxy for when we are still catching up and we shouldn't be counting how many blocks it takes for transactions to be included.

-Add a policyestimator unit test
2015-05-13 10:36:24 -04:00