Commit graph

56 commits

Author SHA1 Message Date
Wladimir J. van der Laan cec9e1ea61
Merge #10422: Fix timestamp in fee estimate debug message
8d4f401 Fix timestamp in fee estimate debug message (Alex Morcos)

Tree-SHA512: eb015ddbb9f52f0f84706a16e503d2894dceed4e796979858ca8bdd7f0ec15bf7646ca3f411b06ff099308c91ca498916778e6bd52db00365baaae1d2f4218f2
2017-06-05 16:24:26 +02:00
Pieter Wuille b40ceed98a
Merge #10419: [trivial] Fix three recently introduced typos
efc2e3302 [trivial] Fix three recently introduced typos (practicalswift)

Tree-SHA512: 99e97f3c1350299dfce9c0c35547d480f25c0b877da311d9120f113afd3089eda31b88b2378e2370f288b0c41bb69cee0fd3abca661cd93d5a56982f90709f91
2017-05-26 11:56:21 -07:00
practicalswift efc2e3302d [trivial] Fix three recently introduced typos
```
$ git blame src/policy/fees.cpp | grep becuase
3810e976 (2017-03-07 11:33:44 -0500 789)          * checks for 2*target becuase we are taking the max over all time
$ git blame src/policy/fees.h | grep successfullly
2d2e1705 (2017-04-12 12:29:03 -0400  54)  * representing that a tx was successfullly confirmed in less than or equal to
$ git blame src/wallet/feebumper.cpp | grep "hasen't"
a3878374 (2017-05-11 09:34:39 +0200 258)     // make sure the transaction still has no descendants and hasen't been mined in the meantime
```
2017-05-23 13:22:40 +02:00
practicalswift 211adc074a Use range-based for loops (C++11) when looping over vector elements 2017-05-19 09:56:16 +02:00
Alex Morcos 8d4f401269 Fix timestamp in fee estimate debug message 2017-05-18 12:38:43 -04:00
Pieter Wuille 318ea50a1c
Merge #10199: Better fee estimates
38bc1ec Make more json-like output from estimaterawfee (Alex Morcos)
2d2e170 Comments and improved documentation (Alex Morcos)
ef589f8 minor cleanup: remove unnecessary variable (Alex Morcos)
3ee76d6 Introduce a scale factor (Alex Morcos)
5f1f0c6 Historical block span (Alex Morcos)
aa19b8e Clean up fee estimate debug printing (Alex Morcos)
10f7cbd Track first recorded height (Alex Morcos)
3810e97 Rewrite estimateSmartFee (Alex Morcos)
c7447ec Track failures in fee estimation. (Alex Morcos)
4186d3f Expose estimaterawfee (Alex Morcos)
2681153 minor refactor: explicitly track start of new bucket range and don't update curNearBucket on final loop. (Alex Morcos)
1ba43cc Make EstimateMedianVal smarter about small failures. (Alex Morcos)
d3e30bc Refactor to update moving average on fly (Alex Morcos)
e5007ba Change parameters for fee estimation and estimates on all 3 time horizons. (Alex Morcos)
c0a273f Change file format for fee estimates. (Alex Morcos)

Tree-SHA512: 186e7508d86a1f351bb656edcd84ee9091f5f2706331eda9ee29da9c8eb5bf67b8c1f2abf6662835560e7f613b1377099054f20767f41ddcdbc89c4f9e78946d
2017-05-17 13:15:07 -07:00
Alex Morcos 2d2e17052c Comments and improved documentation 2017-05-10 11:48:46 -04:00
Alex Morcos ef589f8d40 minor cleanup: remove unnecessary variable 2017-05-10 11:47:45 -04:00
Alex Morcos 3ee76d6de5 Introduce a scale factor
For the per confirmation number tracking of data, introduce a scale factor so that in the longer horizones confirmations are bucketed together at a resolution of the scale.  (instead of 1008 individual data points for each fee bucket, have 42 data points each covering 24 different confirmation values.. (1-24), (25-48), etc.. )
2017-05-10 11:47:45 -04:00
Alex Morcos 5f1f0c6490 Historical block span
Store in fee estimate file the block span for which we were tracking estimates, so we know what targets we can successfully evaluate with the data in the file. When restarting use either this historical block span to set valid range of targets until our current span of tracking estimates is just as long.
2017-05-10 11:47:45 -04:00
Alex Morcos aa19b8ea44 Clean up fee estimate debug printing 2017-05-10 11:47:45 -04:00
Alex Morcos 10f7cbd247 Track first recorded height
Track the first time we seen txs in a block that we have been tracking in our mempool. Used to evaluate validity of fee estimates for different targets.
2017-05-10 11:47:44 -04:00
Alex Morcos 3810e976d6 Rewrite estimateSmartFee
Change the logic of estimateSmartFee to check a 60% threshold at half the target, a 85% threshold at the target and a 95% threshold at double the target. Always check the shortest time horizon possible and ensure that estimates are monotonically decreasing.  Add a conservative mode, which makes sure that the 95% threshold is also met at longer time horizons as well.
2017-05-10 11:47:44 -04:00
Alex Morcos c7447ec303 Track failures in fee estimation.
Start tracking transactions which fail to confirm within the target and are then evicted or otherwise leave mempool.

Fix slight error in unit test.
2017-05-10 11:45:27 -04:00
Alex Morcos 4186d3fdfd Expose estimaterawfee
Track information the ranges of fee rates that were used to calculate the fee estimates (the last range of fee rates in which the data points met the threshold and the first to fail) and provide an RPC call to return this information.
2017-05-10 11:45:26 -04:00
Alex Morcos 2681153af3 minor refactor: explicitly track start of new bucket range and don't update curNearBucket on final loop. 2017-05-10 11:45:26 -04:00
Alex Morcos 1ba43cc0ec Make EstimateMedianVal smarter about small failures.
Instead of stopping if it encounters a "sufficient" number of transactions which don't meet the threshold for being confirmed within the target, it keeps looking to add more transactions to see if there is a temporary blip in the data.  This allows a smaller number of required data points.
2017-05-10 11:45:26 -04:00
Alex Morcos d3e30bca1b Refactor to update moving average on fly 2017-05-10 11:45:26 -04:00
Alex Morcos e5007bae35 Change parameters for fee estimation and estimates on all 3 time horizons.
Make feerate buckets smaller (5% instead of 10%) and make the 3 different horizons have half lifes of 3 hours, 1 day and 1 week respectively.
2017-05-10 11:45:26 -04:00
CryptAxe 94807be8c6
Trivial: fix fee estimate write error log message 2017-04-23 10:36:26 -07:00
Alex Morcos c0a273f4c8 Change file format for fee estimates.
Move buckets and bucketMap to be stored as part of overall serialization of estimator.
Add some placeholder data so file format is only changed once.
Maintain 3 different TxConfirmStats with potential for different decays and scales.
2017-04-20 15:46:15 -04:00
Alex Morcos 68af651498 MOVEONLY: move TxConfirmStats to cpp 2017-04-10 13:56:50 -04:00
Alex Morcos 2332f19bef Initialize TxConfirmStats in constructor
and change to storing as a pointer.
2017-04-10 13:56:50 -04:00
Alex Morcos 5ba81e54e0 Read and Write fee estimate file directly from CBlockPolicyEstimator 2017-04-10 13:56:50 -04:00
Alex Morcos 14e10aa842 Call estimate(Smart)Fee directly from CBlockPolicyEstimator 2017-04-10 13:56:50 -04:00
Alex Morcos dbb9e3699b Give CBlockPolicyEstimator it's own lock 2017-04-10 13:51:51 -04:00
Gregory Maxwell 6b3bb3d9ba Change LogAcceptCategory to use uint32_t rather than sets of strings.
This changes the logging categories to boolean flags instead of strings.

This simplifies the acceptance testing by avoiding accessing a scoped
 static thread local pointer to a thread local set of strings.  It
 eliminates the only use of boost::thread_specific_ptr outside of
 lockorder debugging.

This change allows log entries to be directed to multiple categories
 and makes it easy to change the logging flags at runtime (e.g. via
 an RPC, though that isn't done by this commit.)

It also eliminates the fDebug global.

Configuration of unknown logging categories now produces a warning.
2017-04-01 18:53:29 +00:00
Wladimir J. van der Laan 47510ad3dd
Merge #9548: Remove min reasonable fee
ad82cb0 Remove unnecessary min fee argument in CTxMemPool constructor (Alex Morcos)
2a7b56c CBlockPolicyEstimator now uses hard coded minimum bucket feerate (Alex Morcos)
ac9d3d2 Change fee estimation bucket limit variable names (Alex Morcos)

Tree-SHA512: 6e3bc7df3497ed60c7620845d222063e33a0238020f5c3316e61e0eff758078588ea8dd51196ceb59aa561ba106f8cdae62cebe521adb3247108bb49f15252d6
2017-03-07 19:49:00 +01:00
Alex Morcos fe282acd76 [cleanup] Remove estimatePriority and estimateSmartPriority
Unused everywhere now except one test.
2017-02-27 11:23:51 -05:00
Alex Morcos bc8fd12289 Remove harmless read of unusued priority estimates 2017-02-21 11:12:44 -05:00
Alex Morcos 2a7b56cc0e CBlockPolicyEstimator now uses hard coded minimum bucket feerate 2017-01-17 13:21:31 -05:00
Alex Morcos ac9d3d25f7 Change fee estimation bucket limit variable names 2017-01-17 13:19:32 -05:00
Alex Morcos 44b64b933d Fix edge case with stale fee estimates 2017-01-04 12:10:18 -05:00
Alex Morcos 78ae62d264 Add clarifying comments to fee estimation 2017-01-04 12:10:18 -05:00
Alex Morcos 5fe0f47aa7 Add extra logging to processBlock in fee estimation. 2017-01-04 12:10:18 -05:00
Alex Morcos ebafdcabb1 Pass pointers to existing CTxMemPoolEntries to fee estimation 2017-01-04 12:09:34 -05:00
Alex Morcos d825838e64 Always update fee estimates on new blocks.
All decisions about whether the transactions are valid data points are made at the time the transaction arrives. Updating on blocks all the time will now cause stale fee estimates to decay quickly when we restart a node.
2017-01-04 12:09:34 -05:00
Alex Morcos 6f06b268c1 rename bool to validFeeEstimate 2017-01-04 12:09:33 -05:00
Alex Morcos 84f7ab08d2 Remove member variable hadNoDependencies from CTxMemPoolEntry
Fee estimation can just check its own mapMemPoolTxs to determine the same information.  Note that now fee estimation for block processing must happen before those transactions are removed, but this shoudl be a speedup.
2017-01-04 12:09:33 -05:00
Alex Morcos 4df44794c9 Remove extraneous LogPrint from fee estimation
Once priority estimation was removed, not all transactions in the mempool are tracked in the fee estimation mempool tracking.  So there is no error if a transaction is not found for removal.
2017-01-04 12:03:44 -05:00
Wladimir J. van der Laan 1ce7ede2a9
Merge #9288: Fix a bug if the min fee is 0 for FeeFilterRounder
f8d43b8 Avoid rollingMinimumFeeRate never being able to decay below half (Alex Morcos)
eab8e1b fix a bug if the min fee is 0 for FeeFilterRounder (Alex Morcos)
2017-01-04 12:33:56 +01:00
isle2983 27765b6403 Increment MIT Licence copyright header year on files modified in 2016
Edited via:

$ contrib/devtools/copyright_header.py update .
2016-12-31 11:01:21 -07:00
Alex Morcos eab8e1b172 fix a bug if the min fee is 0 for FeeFilterRounder 2016-12-08 15:40:14 -05:00
Alex Morcos d824ad030e Disable fee estimates for a confirm target of 1 block 2016-11-29 12:18:44 -05:00
Wladimir J. van der Laan 3c03dc2cfc
Merge #7730: Remove priority estimation
0bd581a add release notes for removal of priority estimation (Alex Morcos)
b2322e0 Remove priority estimation (Alex Morcos)
2016-11-07 13:36:40 +01:00
Alex Morcos b2322e0fc6 Remove priority estimation 2016-11-07 13:22:34 +01:00
isle2983 4b04e32c20 [copyright] copyright header style uniform
Three categories of modifications:

1)

1 instance of 'The Bitcoin Core developers \n',
1 instance of 'the Bitcoin Core developers\n',
3 instances of 'Bitcoin Core Developers\n', and
12 instances of 'The Bitcoin developers\n'

are made uniform with the 443 instances of 'The Bitcoin Core developers\n'

2)

3 instances of 'BitPay, Inc\.\n' are made uniform with the other 6
instances of 'BitPay Inc\.\n'

3)

4 instances where there was no '(c)' between the 'Copyright' and the year
where it deviates from the style of the local directory.
2016-11-06 10:12:50 -07:00
Wladimir J. van der Laan 5eaaa83ac1 Kill insecure_random and associated global state
There are only a few uses of `insecure_random` outside the tests.
This PR replaces uses of insecure_random (and its accompanying global
state) in the core code with an FastRandomContext that is automatically
seeded on creation.

This is meant to be used for inner loops. The FastRandomContext
can be in the outer scope, or the class itself, then rand32() is used
inside the loop. Useful e.g. for pushing addresses in CNode or the fee
rounding, or randomization for coin selection.

As a context is created per purpose, thus it gets rid of
cross-thread unprotected shared usage of a single set of globals, this
should also get rid of the potential race conditions.

- I'd say TxMempool::check is not called enough to warrant using a special
  fast random context, this is switched to GetRand() (open for
  discussion...)

- The use of `insecure_rand` in ConnectThroughProxy has been replaced by
  an atomic integer counter. The only goal here is to have a different
  credentials pair for each connection to go on a different Tor circuit,
  it does not need to be random nor unpredictable.

- To avoid having a FastRandomContext on every CNode, the context is
  passed into PushAddress as appropriate.

There remains an insecure_random for test usage in `test_random.h`.
2016-10-17 13:08:35 +02:00
Alex Morcos 9e072a6e66 Implement "feefilter" P2P message.
The "feefilter" p2p message is used to inform other nodes of your mempool min fee which is the feerate that any new transaction must meet to be accepted to your mempool.  This will allow them to filter invs to you according to this feerate.
2016-03-21 10:46:25 -04:00
Chris Wheeler 9d263bd17c Typo fixes in comments 2016-01-17 11:03:56 +00:00