This removes the possibility for GetCoin/HaveCoin/HaveCoinInCache to return
true while the respective coin is spent. By doing it across all calls, some
extra checks can be eliminated.
coins_tests is modified to call HaveCoin sometimes before and sometimes
after AccessCoin. A further change is needed because the semantics for
GetCoin slightly changed, causing a pruned entry in the parent cache to not
be pulled into the child in FetchCoin.
5a9b508 [trivial] Add end of namespace comments (practicalswift)
Tree-SHA512: 92b0fcae4d1d3f4da9e97569ae84ef2d6e09625a5815cd0e5f0eb6dd2ecba9852fa85c184c5ae9de5117050330ce995e9867b451fa8cd5512169025990541a2b
c1be285 chainparams: make supported service bits option explicit (Cory Fields)
d5c7c1c net: use an internal address for fixed seeds (Cory Fields)
6cdc488 net: switch to dummy internal ip for dns seed source (Cory Fields)
6d0bd5b net: do not allow resolving to an internal address (Cory Fields)
7f31762 net: add an internal subnet for representing unresolved hostnames (Cory Fields)
Tree-SHA512: 9bf1042bef546ac3ef0e0d3a9a5555eb21628ff2674a0cf8c6367194b22bfdab477adf452c0e7c56f44e0fb37debc5e14bdb623452e076fb9c492c7702601d7a
A few "a->an" and "an->a".
"Shows, if the supplied default SOCKS5 proxy" -> "Shows if the supplied default SOCKS5 proxy". Change made on 3 occurrences.
"without fully understanding the ramification of a command" -> "without fully understanding the ramifications of a command".
Removed duplicate words such as "the the".
In order to prevent mixups, our internal range is never allowed as a resolve
result. This means that no user-provided string will ever be confused with an
internal address.
We currently do two resolves for dns seeds: one for the results, and one to
serve in addrman as the source for those addresses.
There's no requirement that the source hostname resolves to the stored
identifier, only that the mapping is unique. So rather than incurring the
second lookup, combine a private subnet with a hash of the hostname.
The resulting v6 ip is guaranteed not to be publicy routable, and has only a
negligible chance of colliding with a user's internal network (which would be
of no consequence anyway).
cb24c85 Use rdrand as entropy source on supported platforms (Pieter Wuille)
Tree-SHA512: c42eaa01a14e6bc097c70b6bf8540d61854c2f76cb32be69c2a3c411a126f7b4bf4a4486e4493c4cc367cc689319abde0d4adb799d29a54fd3e81767ce0766fc
3ff1fa8 Use override keyword on CCoinsView overrides (Russell Yanofsky)
24e44c3 Don't return stale data from CCoinsViewCache::Cursor() (Russell Yanofsky)
Tree-SHA512: 08699dae0925ffb9c018f02612ac6b7eaf73ec331e2f4f934f1fe25a2ce120735fa38596926e924897c203f7470e99f0a99cf70d2ce31ff428b105e16583a861
3fb81a8 Use list initialization (C++11) for maps/vectors instead of boost::assign::map_list_of/list_of (practicalswift)
Tree-SHA512: 63a9ac9ec5799472943dce1cd92a4b14e7f1fe12758a5fc4b1efceaf2c85a4ba71dad5ccc50813527f18b192e7714c076e2478ecd6ca0d452b24e88416f872f7
227ae9b [tests] Use FastRandomContext instead of boost::random::{mt19937,uniform_int_distribution} (practicalswift)
Tree-SHA512: 1bde6c8b9498051fa2eae4913eb1f5411adea8dea1511c0df859aea57a2a7db6f5839945ddf2eccdddfa322bceacad35a5d875742db7d15e40dbea83185307bb
e94584858 scripted-diff: Use new naming style for insecure_rand* functions (Pieter Wuille)
2fcd9cc86 scripted-diff: Use randbits/bool instead of randrange where possible (Pieter Wuille)
2ada67852 Use randbits instead of ad-hoc emulation in prevector tests (Pieter Wuille)
5f0b04eed Replace rand() & ((1 << N) - 1) with randbits(N) (Pieter Wuille)
3ecabae36 Replace more rand() % NUM by randranges (Pieter Wuille)
efee1db21 scripted-diff: use insecure_rand256/randrange more (Pieter Wuille)
1119927df Add various insecure_rand wrappers for tests (Pieter Wuille)
124d13a58 Merge test_random.h into test_bitcoin.h (Pieter Wuille)
90620d66c scripted-diff: Rename cuckoo tests' local rand context (Pieter Wuille)
37e864eb9 Add FastRandomContext::rand256() and ::randbytes() (Pieter Wuille)
Tree-SHA512: d09705a3ec718ae792f7d66a75401903ba7b9c9d3fc36669d6e3b9242f0194738106be26baefc8a8e3fa6df7c9a35978c71c0c430278a028b331df23a3ea3070
This adds a new CuckooCache in validation, caching whether all of a
transaction's scripts were valid with a given set of script flags.
Unlike previous attempts at caching an entire transaction's
validity, which have nearly universally introduced consensus
failures, this only caches the validity of a transaction's
scriptSigs. As these are pure functions of the transaction and
data it commits to, this should be much safer.
This is somewhat duplicative with the sigcache, as entries in the
new cache will also have several entries in the sigcache. However,
the sigcache is kept both as ATMP relies on it and because it
prevents malleability-based DoS attacks on the new higher-level
cache. Instead, the -sigcachesize option is re-used - cutting the
sigcache size in half and using the newly freed memory for the
script execution cache.
Transactions which match the script execution cache never even have
entries in the script check thread's workqueue created.
Note that the cache is indexed only on the script execution flags
and the transaction's witness hash. While this is sufficient to
make the CScriptCheck() calls pure functions, this introduces
dependancies on the mempool calculating things such as the
PrecomputedTransactionData object, filling the CCoinsViewCache, etc
in the exact same way as ConnectBlock. I belive this is a reasonable
assumption, but should be noted carefully.
In a rather naive benchmark (reindex-chainstate up to block 284k
with cuckoocache always returning true for contains(),
-assumevalid=0 and a very large dbcache), this connected blocks
~1.7x faster.
This adds the listening address on which incoming connections were received to the
CNode and CNodeStats structures.
The address is reported in `getpeerinfo`.
This can be useful for distinguishing connections received on different listening ports
(e.g. when using a different listening port for Tor hidden service connections)
or different networks.
This patch makes several related changes:
* Changes the CCoinsView virtual methods (GetCoins, HaveCoins, ...)
to be COutPoint/Coin-based rather than txid/CCoins-based.
* Changes the chainstate db to a new incompatible format that is also
COutPoint/Coin based.
* Implements reconstruction code for hash_serialized_2.
* Adapts the coins_tests unit tests (thanks to Russell Yanofsky).
A side effect of the new CCoinsView model is that we can no longer
use the (unreliable) test for transaction outputs in the UTXO set
to determine whether we already have a particular transaction.
This clarifies a bit more the ways in which the new script execution
cache could break consensus in the future if additional data from
the CCoins object were to be used as a part of script execution.
After this change, any such consensus breaks should be very visible
to reviewers, hopefully ensuring no such changes can be made.
The earlier CTxInUndo class now holds the same information as the Coin
class. Instead of duplicating functionality, replace CTxInUndo with a
serialization adapter for Coin.
This makes the following changes:
* In undo data and the chainstate database, the transaction nVersion
field is removed from the data structures, always written as 0, and
ignored when reading.
* The definition of hash_serialized in gettxoutsetinfo is changed to no
longer incude the nVersion field. It is renamed to hash_serialized_2
to avoid confusion. The new definition also includes transaction
height and coinbase information, as this information was missing
before.
This depends on having a CHashVerifier-based undo data checksum
verifier.
Apart from changing the definition of serialized_hash, downgrading
after using this patch is supported, as no release ever used the value
of nVersion field in UTXO entries.
The error() function unconditionally reports an error. It should only
be used for actually exception situations, and not for the type of
inconsistencies that ApplyTxInUndo/DisconnectBlock can graciously deal
with.
This also makes a subtle semantics change: in ApplyTxInUndo, when a
record with metadata is encountered (indicating it is the last spend
from a tx), don't wipe the CCoins record if it wasn't empty at that
point. This makes sure that UTXO operations never affect any other
UTXOs (including those from the same tx).
693247b [test] Speed up fuzzing by ~200x when using afl-fuzz (practicalswift)
Tree-SHA512: 95922fc2616b8cb00dd531ed1140a52bbda4e04292dd8c1c60a8f49dbf6ccb797a18b61180b3fb68d695456b478a1f5ae7fda47e8ecee41dd65555487aef40a3
38bc1ec Make more json-like output from estimaterawfee (Alex Morcos)
2d2e170 Comments and improved documentation (Alex Morcos)
ef589f8 minor cleanup: remove unnecessary variable (Alex Morcos)
3ee76d6 Introduce a scale factor (Alex Morcos)
5f1f0c6 Historical block span (Alex Morcos)
aa19b8e Clean up fee estimate debug printing (Alex Morcos)
10f7cbd Track first recorded height (Alex Morcos)
3810e97 Rewrite estimateSmartFee (Alex Morcos)
c7447ec Track failures in fee estimation. (Alex Morcos)
4186d3f Expose estimaterawfee (Alex Morcos)
2681153 minor refactor: explicitly track start of new bucket range and don't update curNearBucket on final loop. (Alex Morcos)
1ba43cc Make EstimateMedianVal smarter about small failures. (Alex Morcos)
d3e30bc Refactor to update moving average on fly (Alex Morcos)
e5007ba Change parameters for fee estimation and estimates on all 3 time horizons. (Alex Morcos)
c0a273f Change file format for fee estimates. (Alex Morcos)
Tree-SHA512: 186e7508d86a1f351bb656edcd84ee9091f5f2706331eda9ee29da9c8eb5bf67b8c1f2abf6662835560e7f613b1377099054f20767f41ddcdbc89c4f9e78946d
For the per confirmation number tracking of data, introduce a scale factor so that in the longer horizones confirmations are bucketed together at a resolution of the scale. (instead of 1008 individual data points for each fee bucket, have 42 data points each covering 24 different confirmation values.. (1-24), (25-48), etc.. )
Change the logic of estimateSmartFee to check a 60% threshold at half the target, a 85% threshold at the target and a 95% threshold at double the target. Always check the shortest time horizon possible and ensure that estimates are monotonically decreasing. Add a conservative mode, which makes sure that the 95% threshold is also met at longer time horizons as well.
Cleanup request from #10287.
Change "Test #:" comments to "Test:"
Change BOOST_CHECK(... = ...) to BOOST_CHECK_EQUAL(..., ...)
Remove three unnecessary if statements
ed36de5 [tests] Update Unit Test for addrman.h/addrman.cpp (Jimmy Song)
Tree-SHA512: e7c08c19e227c34c230900e14a176b2290022b78b0ece387452e673662491c11f26249cbf1711235276c07a964c339e27b4cda9a2730ded5c0e23a650e0d72db
c1082a7 Chainparams: Use the factory for pow tests (Jorge Timón)
2351a06 Chainparams: Get rid of CChainParams& Params(std::string) (Jorge Timón)
f87f362 Chainparams: Use a regular factory for creating chainparams (Jorge Timón)
Tree-SHA512: 359c8a2a1bc9d02db7856d02810240ada28048ac088f878b575597a7255cdb0ffdd1a647085ee67a34c6a7e7ed9e6cfdb61240cf6e75139619b640dbb096072c
Add test for adding multiple addresses to address manager
Clean up unnecessary modulo operations
Add test for GetNewBucket's alternate method signature
dcb69fd [test] Unit test amount.h/amount.cpp (Jimmy Song)
Tree-SHA512: dc50e6158322a282a8b8b60c8eab7ce327a33c48113a1455b8a1784f07e6277ad499e85ec978199468a2dc34e9e288287c8803c215f810fa2d841cdda9a414f5
68af651 MOVEONLY: move TxConfirmStats to cpp (Alex Morcos)
2332f19 Initialize TxConfirmStats in constructor (Alex Morcos)
5ba81e5 Read and Write fee estimate file directly from CBlockPolicyEstimator (Alex Morcos)
14e10aa Call estimate(Smart)Fee directly from CBlockPolicyEstimator (Alex Morcos)
dbb9e36 Give CBlockPolicyEstimator it's own lock (Alex Morcos)
f6187d6 Make processBlockTx private. (Alex Morcos)
ae7327b Make feeEstimator its own global instance of CBlockPolicyEstimator (Alex Morcos)
Tree-SHA512: dbf3bd2b30822e609a35f3da519b62d23f8a50e564750695ddebd08553b4c01874ae3e07d792c6cc78cc377d2db33b951ffedc46ac7edaf5793f9ebb931713af
d93b97f Set to nullptr after delete (practicalswift)
Tree-SHA512: 7201cef4541557ffe31f52ce7527c4b08a2ff5aa1eae5268bdfee5b4843881f8fd115257bef6d1b4dfb71166951950a912ce87aef160ca89c2ca2ae264cfab1b
d7f80b6 Rename first iterator to prevent shadowing. (Pavel Janík)
b42ff60 Fix shadowing of local variables. (Pavel Janík)
c4b60b3 Make some global variables less-global (static) (Pavel Janík)
bb2aaee Prevent -Wshadow warnings with gcc versions 4.8.5, 5.3.1 and 6.2.1. (Pavel Janík)
Tree-SHA512: 3aea4e28146c8f2a31351c6e2b0cce88b6f1e567a0ea0e6131624453e7193d0904e30d81b1439d8c69e281cf0e369b895851fb882ae48d5967b5c2e2c227404e
9576b01 Enable xvfb in travis to allow running test_bitcoin-qt (Russell Yanofsky)
9e6817e Add new test_bitcoin-qt static library dependencies (Russell Yanofsky)
2754ef1 Add simple qt wallet test sending a transaction (Russell Yanofsky)
b61b34c Add braces to if statements in Qt test_main (Russell Yanofsky)
cc9503c Make qt test compatible with TestChain100Setup framework (Russell Yanofsky)
91e3035 Make test_bitcoin.cpp compatible with Qt Test framework (Russell Yanofsky)
Tree-SHA512: da491181848b8c39138e997ae5ff2df0b16eef2d9cdd0a965229b1a28d4fa862d5f1ef314a1736e5050e88858f329124d15c689659fc6e50fefde769ba24e523
On msvc14, int literal '-2147483648' is invalid, because '2147483648' is unsigned type and cant't apply minus operator to unsigned type.
To define the int literal correctly, use '-2147483647 - 1' formula that is also used to define INT_MIN in limits.h.
ad1ae7a Check and enable -Wshadow by default. (Pavel Janík)
9de90bb Do not shadow variables (gcc set) (Pavel Janík)
Tree-SHA512: 9517feb423dc8ddd63896016b25324673bfbe0bffa97f22996f59d7a3fcbdc2ebf2e43ac02bc067546f54e293e9b2f2514be145f867321e9031f895c063d9fb8
If InitBlockIndex fails, then it will segfault later. Same for the later
ActivateBestChain. BOOST_REQUIRE the result, so that an error will be
reported and the test case aborted.
7e6dcd9 random: Add fallback if getrandom syscall not available (Wladimir J. van der Laan)
7cad849 sanity: Move OS random to sanity check function (Wladimir J. van der Laan)
aa09ccb squashme: comment that NUM_OS_RANDOM_BYTES should not be changed lightly (Wladimir J. van der Laan)
224e6eb util: Specific GetOSRandom for Linux/FreeBSD/OpenBSD (Wladimir J. van der Laan)
Tree-SHA512: 9fd408b1316c69de86674f342339b2f89192fd317c8c036b5df4320f828fa263c7966146bfc1904c51137ee4a26e4cb0f560b2cd05e18cde4d808b9b92ad15c4
19cafc6 test: Replace remaining sprintf with snprintf (Wladimir J. van der Laan)
0a17714 uint256: replace sprintf with HexStr and reverse-iterator (Wladimir J. van der Laan)
Tree-SHA512: 2ba1dd4d25e1cbfff4d67b2f483448aa7c34ab5c799cddd48ba5826e5fa6df425abe35e244aaf4c52db9fccfb4d2a25a14bb4597bf9d1fce95991f270da6bb26
Use of `sprintf` is seen as a red flag as many of its uses are insecure.
OpenBSD warns about it while compiling, and some modern platforms, e.g.
[cloudlibc from cloudabi](https://github.com/NuxiNL/cloudlibc) don't
even provide it anymore.
Although our uses of these functions are secure, it can't hurt to
replace them anyway. There are only 3 occurences left, all in the
tests.
Move the OS random test to a sanity check function that is called every
time bitcoind is initialized.
Keep `src/test/random_tests.cpp` for the case that later random tests
are added, and keep a rudimentary test that just calls the sanity check.
These are available in sandboxes without access to files or
devices. Also [they are safer and more straightforward](https://en.wikipedia.org/wiki/Entropy-supplying_system_calls)
to use than `/dev/urandom` as reading from a file has quite a few edge
cases:
- Linux: `getrandom(buf, buflen, 0)`. [getrandom(2)](http://man7.org/linux/man-pages/man2/getrandom.2.html)
was introduced in version 3.17 of the Linux kernel.
- OpenBSD: `getentropy(buf, buflen)`. The [getentropy(2)](http://man.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/getentropy.2)
function appeared in OpenBSD 5.6.
- FreeBSD and NetBSD: `sysctl(KERN_ARND)`. Not sure when this was added
but it has existed for quite a while.
Alternatives:
- Linux has sysctl `CTL_KERN` / `KERN_RANDOM` / `RANDOM_UUID`
which gives 16 bytes of randomness. This may be available
on older kernels, however [sysctl is deprecated on Linux](https://lwn.net/Articles/605392/)
and even removed in some distros so we shouldn't use it.
Add tests for `GetOSRand()`:
- Test that no error happens (otherwise `RandFailure()` which aborts)
- Test that all 32 bytes are overwritten (initialize with zeros, try multiple times)
Discussion:
- When to use these? Currently they are always used when available.
Another option would be to use them only when `/dev/urandom` is not
available. But this would mean these code paths receive less testing,
and I'm not sure there is any reason to prefer `/dev/urandom`.
Closes: #9676
Minimum boost version was bumped to 1.47.0 in #8920, which
means the configure step won't even pass with older boost.
This version has boost filesystem v3, which means the
(crappy) fallbacks for older versions can go.
4b189c1 Change bumpfee result value from 'oldfee' to 'origfee'. (Alex Morcos)
0c0c63f Introduce WALLET_INCREMENTAL_RELAY_FEE (Alex Morcos)
e8021ec Use CWallet::GetMinimumFee in bumpfee (Alex Morcos)
ae9719a Refactor GetMinimumFee to give option of providing targetFee (Alex Morcos)
fe8e8ef [rpc] Add incremental relay fee to getnetworkinfo (Alex Morcos)
6b331e6 Fix to have miner test aware of new separate block min tx fee (Alex Morcos)
de6400d Fix missing use of dustRelayFee (Alex Morcos)
5b15870 Use incrementalRelayFee for BIP 125 replacement (Alex Morcos)
Prior to this commit the err variable was not guaranteed to be set before
the check ...
BOOST_CHECK_MESSAGE(err != SCRIPT_ERR_OK, ScriptErrorString(err));
0c50909 testcases: explicitly specify transaction version 1 (John Newbery)
b7e144b Add test cases to test new bitcoin-tx functionality (jnewbery)
61a1534 Add all transaction output types to bitcoin-tx. (jnewbery)
1814b08 add p2sh and segwit options to bitcoin-tx outscript command (Stanislas Marion)
05a55a6 Added EVENT_CFLAGS to test makefile to explicitly include libevent headers. (Karl-Johan Alm)
280a559 Added some simple tests for the RAII-style events. (Karl-Johan Alm)
7f7f102 Switched bitcoin-cli.cpp to use RAII unique pointers with deleters. (Karl-Johan Alm)
e5534d2 Added std::unique_ptr<> wrappers with deleters for libevent modules. (Karl-Johan Alm)
Fee estimation can just check its own mapMemPoolTxs to determine the same information. Note that now fee estimation for block processing must happen before those transactions are removed, but this shoudl be a speedup.
We were marking coins FRESH before being sure they were not overwriting dirty undo data. This condition was never reached in existing code because undo data was always flushed before UpdateCoins was called with new transactions, but could have been exposed in an otherwise safe refactor.
Clarify in the comments the assumptions made in ModifyNewCoins.
Add ability to undo transactions to UpdateCoins unit test.
Thanks to Russ Yanofsky for suggestion on how to make logic clearer and fixing up the ccoins_modify_new test cases.
This commit add testcases to test the following functions in bitcoin-tx:
- add a pay to non-standard script output
- add a P2SH output
- add a P2WSH output
- add a P2WSH wrapped in a P2SH output
- add a pay to pub key output
- add a P2WPKH output
- add a P2WPKH wrapped in a P2SH output
- add a bare multisig output
- add a multisig in P2SH output
- add a multisig in a P2WSH output
- add a multisig in a P2WSH wrapped in as P2SH output
Defers to pre-defined version if found (e.g. protobuf). For protobuf case, the definitions are identical and thus include order should not affect results.
c5c92c4 Update python tests for default tx version=2 (BtcDrak)
dab207e Preserve tx version=1 for certain tests (BtcDrak)
c5d746a tiny test fix for mempool_tests (Alex Morcos)
1f0ca1a Bump default transaction version to 2 (BtcDrak)
67dac4e Add unit tests for the CuckooCache (Jeremy Rubin)
c9e69fb Add CuckooCache implementation and replace the sigcache map_type with it (Jeremy Rubin)
SQUASHME: Change cuckoocache to only work for powers of two, to avoid mod operator
SQUASHME: Update Documentation and simplify logarithm logic
SQUASHME: OSX Build Errors
SQUASHME: minor Feedback from sipa + bluematt
SQUASHME: DOCONLY: Clarify a few comments.
This is another violation of the one definition rule, as the type
for mapOrphanTransactionsByPrev did not match the one in
net_processing.cpp anymore. As it now depends on a custom Iterator,
it seems too much hassle to correctly expose it to the tests.
Instead, this commit just removes the one test it was referenced in.
Add more comprehensive unit tests for CCoinsViewCache. Right now it is hard to
refactor caching code or fix bugs in the caching logic because you have to try
to mentally enumerate all the different states the cache might be in to make
sure a change doesn't cause unintended consequences. The new tests explicitly
enumerate relevant cache states, documenting and verifying the behavior in each
state, so it will be safer and easier to make changes to the caching code in
the future.
dd0df81 Document ConnectBlock connectTrace postconditions (Matt Corallo)
2d6e561 Switch pblock in ProcessNewBlock to a shared_ptr (Matt Corallo)
2736c44 Make the optional pblock in ActivateBestChain a shared_ptr (Matt Corallo)
ae4db44 Create a shared_ptr for the block we're connecting in ActivateBCS (Matt Corallo)
fd9d890 Keep blocks as shared_ptrs, instead of copying txn in ConnectTip (Matt Corallo)
6fdd43b Add struct to track block-connect-time-generated info for callbacks (Matt Corallo)
76faa3c Rename the remaining main.{h,cpp} to validation.{h,cpp} (Matt Corallo)
e736772 Move network-msg-processing code out of main to its own file (Matt Corallo)
87c35f5 Remove orphan state wipe from UnloadBlockIndex. (Matt Corallo)
Output instances of "BloomFilter" changed to "Bloom filter", in accordance with Wikipedia standard notation:
https://en.wikipedia.org/wiki/Bloom_filter
also to sync with the majority of cases in the self-same file
c7be56d net: push only raw data into CConnman (Cory Fields)
2ec935d net: add CVectorWriter and CNetMsgMaker (Cory Fields)
b7695c2 net: No need to check individually for disconnection anymore (Cory Fields)
fedea8a net: don't send any messages before handshake or after requested disconnect (Cory Fields)
d74e352 net: Set feelers to disconnect at the end of the version message (Cory Fields)
CVectorWriter is useful for overwriting or appending an existing byte vector.
CNetMsgMaker is a shortcut for creating messages on-the-fly which are suitable
for pushing to CConnman.
ae22357 Replace CValidationState param in ProcessNewBlock with BlockChecked (Matt Corallo)
7c98ce5 Remove pfrom parameter from ProcessNewBlock (Matt Corallo)
e2e069d Revert "RPC: Give more details when "generate" fails" (Matt Corallo)
d59a518 Use fixed preallocation instead of costly GetSerializeSize (Pieter Wuille)
25a211a Add optimized CSizeComputer serializers (Pieter Wuille)
a2929a2 Make CSerAction's ForRead() constexpr (Pieter Wuille)
a603925 Avoid -Wshadow errors (Pieter Wuille)
5284721 Get rid of nType and nVersion (Pieter Wuille)
657e05a Make GetSerializeSize a wrapper on top of CSizeComputer (Pieter Wuille)
fad9b66 Make nType and nVersion private and sometimes const (Pieter Wuille)
c2c5d42 Make streams' read and write return void (Pieter Wuille)
50e8a9c Remove unused ReadVersion and WriteVersion (Pieter Wuille)
d4833ff Bump the protocol version to distinguish new banning behavior. (Suhas Daftuar)
88c3549 Fix compact block handling to not ban if block is invalid (Suhas Daftuar)
c93beac [qa] Test that invalid compactblocks don't result in ban (Suhas Daftuar)
Remove the nType and nVersion as parameters to all serialization methods
and functions. There is only one place where it's read and has an impact
(in CAddress), and even there it does not impact any of the recursively
invoked serializers.
Instead, the few places that need nType or nVersion are changed to read
it directly from the stream object, through GetType() and GetVersion()
methods which are added to all stream classes.
Given that in default GetSerializeSize implementations created by
ADD_SERIALIZE_METHODS we're already using CSizeComputer(), get rid
of the specialized GetSerializeSize methods everywhere, and just use
CSizeComputer. This removes a lot of code which isn't actually used
anywhere.
For CCompactSize and CVarInt this actually removes a more efficient
size computing algorithm, which is brought back in a later commit.
9027680 net: handle version push in InitializeNode (Cory Fields)
7588b85 net: construct CNodeStates in place (Cory Fields)
440f1d3 net: remove now-unused ssSend and Fuzz (Cory Fields)
5c2169c drop the optimistic write counter hack (Cory Fields)
ea33268 net: switch all callers to connman for pushing messages (Cory Fields)
3e32cd0 connman is in charge of pushing messages (Cory Fields)
b98c14c serialization: teach serializers variadics (Cory Fields)
Three categories of modifications:
1)
1 instance of 'The Bitcoin Core developers \n',
1 instance of 'the Bitcoin Core developers\n',
3 instances of 'Bitcoin Core Developers\n', and
12 instances of 'The Bitcoin developers\n'
are made uniform with the 443 instances of 'The Bitcoin Core developers\n'
2)
3 instances of 'BitPay, Inc\.\n' are made uniform with the other 6
instances of 'BitPay Inc\.\n'
3)
4 instances where there was no '(c)' between the 'Copyright' and the year
where it deviates from the style of the local directory.
The changes here are dense and subtle, but hopefully all is more explicit
than before.
- CConnman is now in charge of sending data rather than the nodes themselves.
This is necessary because many decisions need to be made with all nodes in
mind, and a model that requires the nodes calling up to their manager quickly
turns to spaghetti.
- The per-node-serializer (ssSend) has been replaced with a (quasi-)const
send-version. Since the send version for serialization can only change once
per connection, we now explicitly tag messages with INIT_PROTO_VERSION if
they are sent before the handshake. With this done, there's no need to lock
for access to nSendVersion.
Also, a new stream is used for each message, so there's no need to lock
during the serialization process.
- This takes care of accounting for optimistic sends, so the
nOptimisticBytesWritten hack can be removed.
- -dropmessagestest and -fuzzmessagestest have not been preserved, as I suspect
they haven't been used in years.
59ac5c5 net: Use deterministic randomness for CNode's nonce, and make it const (Cory Fields)
aff6584 net: constify a few CNode vars to indicate that they're threadsafe (Cory Fields)
- Use the python standard logging library
- Run all tests and report all failing test-cases (rather than stop after one test case fails)
- If output is different from expected output, log a contextual diff.
Refer to the right file in the top-level README.md.
Having only one file with test documentation saves some confusion about
where things are documented.
444c673 bench: Add benchmark for lockedpool allocation/deallocation (Wladimir J. van der Laan)
6567999 rpc: Add `getmemoryinfo` call (Wladimir J. van der Laan)
4536148 support: Add LockedPool (Wladimir J. van der Laan)
f4d1fc2 wallet: Get rid of LockObject and UnlockObject calls in key.h (Wladimir J. van der Laan)
999e4c9 wallet: Change CCrypter to use vectors with secure allocator (Wladimir J. van der Laan)
GetTotalBlocksEstimate is no longer used and it was the only thing
the checkpoint tests were testing.
Since checkpoints are on their way out it makes more sense to remove
the test file than to cook up a new pointless test.
This splits the output comparison for `bitcoin-tx` into two steps:
- First, check for data mismatch, parsing the data as json or hex
depending on the extension of the output file
- Then, check if the literal string matches
For either of these cases give a different error.
This prevents wild goose chases when e.g. a trailing space doesn't match
exactly, and makes sure that both test output and examples are valid
data of the purported format.
Add a pool for locked memory chunks, replacing LockedPageManager.
This is something I've been wanting to do for a long time. The current
approach of locking objects where they happen to be on the stack or heap
in-place causes a lot of mlock/munlock system call overhead, slowing
down any handling of keys.
Also locked memory is a limited resource on many operating systems (and
using a lot of it bogs down the system), so the previous approach of
locking every page that may contain any key information (but also other
information) is wasteful.
Makes it an error to use flags that have not been defined
on the libconsensus API.
There has been some confusion as to what pass to libconsensus, and
(combined with mention in the release notes) this should clear it up.
Using undocumented flags is a risk because their meaning,
and what combinations are allowed, changes from release to release.
E.g. it is no longer possible to pass (CLEANSTACK | P2SH) without
running into an assertion after the segwit changes.
a9aec5c Use BlockChecked signal to send reject messages from mapBlockSource (Matt Corallo)
7565e03 Remove SyncWithWallets wrapper function (Matt Corallo)
12ee1fe Always call UpdatedBlockTip, even if blocks were only disconnected (Matt Corallo)
f5efa28 Remove CConnman parameter from ProcessNewBlock/ActivateBestChain (Matt Corallo)
fef1010 Use CValidationInterface from chain logic to notify peer logic (Matt Corallo)
aefcb7b Move net-processing logic definitions together in main.h (Matt Corallo)
0278fb5 Remove duplicate nBlocksEstimate cmp (we already checked IsIBD()) (Matt Corallo)
87e7d72 Make validationinterface.UpdatedBlockTip more verbose (Matt Corallo)
There are only a few uses of `insecure_random` outside the tests.
This PR replaces uses of insecure_random (and its accompanying global
state) in the core code with an FastRandomContext that is automatically
seeded on creation.
This is meant to be used for inner loops. The FastRandomContext
can be in the outer scope, or the class itself, then rand32() is used
inside the loop. Useful e.g. for pushing addresses in CNode or the fee
rounding, or randomization for coin selection.
As a context is created per purpose, thus it gets rid of
cross-thread unprotected shared usage of a single set of globals, this
should also get rid of the potential race conditions.
- I'd say TxMempool::check is not called enough to warrant using a special
fast random context, this is switched to GetRand() (open for
discussion...)
- The use of `insecure_rand` in ConnectThroughProxy has been replaced by
an atomic integer counter. The only goal here is to have a different
credentials pair for each connection to go on a different Tor circuit,
it does not need to be random nor unpredictable.
- To avoid having a FastRandomContext on every CNode, the context is
passed into PushAddress as appropriate.
There remains an insecure_random for test usage in `test_random.h`.
27acfc1 [qa] Update p2p-compactblocks.py for compactblocks v2 (Suhas Daftuar)
422fac6 [qa] Add support for compactblocks v2 to mininode (Suhas Daftuar)
f5b9b8f [qa] Fix bug in mininode witness deserialization (Suhas Daftuar)
6aa28ab Use cmpctblock type 2 for segwit-enabled transfer (Pieter Wuille)
be7555f Fix overly-prescriptive p2p-segwit test for new fetch logic (Matt Corallo)
06128da Make GetFetchFlags always request witness objects from witness peers (Matt Corallo)
CConnman then passes the current best height into CNode at creation time.
This way CConnman/CNode have no dependency on main for height, and the signals
only move in one direction.
This also helps to prevent identity leakage a tiny bit. Before this change, an
attacker could theoretically make 2 connections on different interfaces. They
would connect fully on one, and only establish the initial connection on the
other. Once they receive a new block, they would relay it to your first
connection, and immediately commence the version handshake on the second. Since
the new block height is reflected immediately, they could attempt to learn
whether the two connections were correlated.
This is, of course, incredibly unlikely to work due to the small timings
involved and receipt from other senders. But it doesn't hurt to lock-in
nBestHeight at the time of connection, rather than letting the remote choose
the time.
35fe039 Rename to PrecomputedTransactionData (Pieter Wuille)
ab48c5e Unit test for sighash caching (Nicolas DORIER)
d2c5d04 Precompute sighashes (Pieter Wuille)
Tests if addresses are online or offline by briefly connecting to them. These short lived connections are referred to as feeler connections. Feeler connections are designed to increase the number of fresh online addresses in tried by selecting and connecting to addresses in new. One feeler connection is attempted on average once every two minutes.
This change was suggested as Countermeasure 4 in
Eclipse Attacks on Bitcoin’s Peer-to-Peer Network, Ethan Heilman,
Alison Kendler, Aviv Zohar, Sharon Goldberg. ePrint Archive Report
2015/263. March 2015.
9e9d644 net: fixup nits (Cory Fields)
8945384 net: Have LookupNumeric return a CService directly (Cory Fields)
21ba407 net: narrow include scope after moving to netaddress (Cory Fields)
21e5b96 net: move CNetAddr/CService/CSubNet out of netbase (Cory Fields)
1017b8a net: Add direct tests for new CSubNet constructors (Cory Fields)
b6c3ff3 net: Split resolving out of CSubNet (Cory Fields)
f96c7c4 net: Split resolving out of CService (Cory Fields)
31d6b1d net: Split resolving out of CNetAddr (Cory Fields)
9d4eb9a Do diskspace check before import thread is started (Pieter Wuille)
aa59f2e Add extra message to avoid a long 'Loading banlist' (Pieter Wuille)
0fd2a33 Use a signal to continue init after genesis activation (Pieter Wuille)
Updating documentation for adding new unit test files
Removing unneeded sentence from README
Removing uint160_tests.cpp as it DNE
Formatting command line instructions to use ``
fixing 80 char formatting issue in README
fixing more nits
mininode now supports witness transactions/blocks, blocktools
has a helper for adding witness commitments to blocks, and script
has a function to calculate hashes for signature under sigversion
1, used by segwit.
Py3 conversion by Marco Falke
Test to make sure upgraded nodes don't ask for non-wit blocks by
Gregory Sanders.
script_tests: always test bitcoinconsensus_verify_script_with_amount if VERIFY_WITNESS isn't set
Rename internal method + make it static
trim bitcoinconsensus_ prefix
Add SERIALIZE_TRANSACTION_WITNESS flag
48efec8 Fix some minor compact block issues that came up in review (Matt Corallo)
ccd06b9 Elaborate bucket size math (Pieter Wuille)
0d4cb48 Use vTxHashes to optimize InitData significantly (Matt Corallo)
8119026 Provide a flat list of txid/terators to txn in CTxMemPool (Matt Corallo)
678ee97 Add BIP 152 to implemented BIPs list (Matt Corallo)
56ba516 Add reconstruction debug logging (Matt Corallo)
2f34a2e Get our "best three" peers to announce blocks using cmpctblocks (Matt Corallo)
927f8ee Add ability to fetch CNode by NodeId (Matt Corallo)
d25cd3e Add receiver-side protocol implementation for CMPCTBLOCK stuff (Matt Corallo)
9c837d5 Add sender-side protocol implementation for CMPCTBLOCK stuff (Matt Corallo)
00c4078 Add protocol messages for short-ids blocks (Matt Corallo)
e3b2222 Add some blockencodings tests (Matt Corallo)
f4f8f14 Add TestMemPoolEntryHelper::FromTx version for CTransaction (Matt Corallo)
85ad31e Add partial-block block encodings API (Matt Corallo)
5249dac Add COMPACTSIZE wrapper similar to VARINT for serialization (Matt Corallo)
cbda71c Move context-required checks from CheckBlockHeader to Contextual... (Matt Corallo)
7c29ec9 If AcceptBlockHeader returns true, pindex will be set. (Matt Corallo)
96806c3 Stop trimming when mapTx is empty (Pieter Wuille)
54326a6 Increase maximum orphan size to 100,000 bytes. (Gregory Maxwell)
8c99d1b Treat orphans as implicit inv for parents, discard when parents rejected. (Gregory Maxwell)
11cc143 Adds an expiration time for orphan tx. (Gregory Maxwell)
db0ffe8 This eliminates the primary leak that causes the orphan map to always grow to its maximum size. (Gregory Maxwell)
1b0bcc5 Track orphan by prev COutPoint rather than prev hash (Pieter Wuille)
CreateNewBlock returns a pointer for which the caller takes ownership.
Use std::unique_ptr to make this explicit and simplify handling of these
objects in getblocktemplate.
As per meeting 2016-03-31
https://bitcoincore.org/en/meetings/2016/03/31/#bad-chain-alerts
The partition checker was producing huge number of false-positives
and was disabled in 0.12.1 on the understanding it would either be
fixed in 0.13 or removed entirely from master if not.
c2dd5a3 FIX: correctly measure size of priority block (Alex Morcos)
a278764 FIX: Account for txs already added to block in addPriorityTxs (Alex Morcos)
4dc94d1 Refactor CreateNewBlock to be a method of the BlockAssembler class (Alex Morcos)
eebc232 test: Add more test vectors for siphash (Wladimir J. van der Laan)
8884830 Use C++11 thread-safe static initializers (Pieter Wuille)
c31b24f Use 64-bit SipHash of netgroups in eviction (Pieter Wuille)
9bf156b Support SipHash with arbitrary byte writes (Pieter Wuille)
053930f Avoid recalculating vchKeyedNetGroup in eviction logic. (Patrick Strateman)
288d85d Get rid of CTxMempool::lookup() entirely (Pieter Wuille)
c2a4724 Optimization: use usec in expiration and reuse nNow (Pieter Wuille)
e9b4780 Optimization: don't check the mempool at all if no mempool req ever (Pieter Wuille)
dbfb426 Optimize the relay map to use shared_ptr's (Pieter Wuille)
8d39d7a Switch CTransaction storage in mempool to std::shared_ptr (Pieter Wuille)
1b9e6d3 Add support for unique_ptr and shared_ptr to memusage (Pieter Wuille)
Add error and range-checking parsers for unsigned 32 and 64 bit numbers.
The 32-bit variant is required for parsing sequence numbers from the
command line in `bitcoin-tx` (see #8164 for discussion). I've thrown in
the 64-bit variant as a bonus, as I'm sure it will be needed at some
point.
Also adds tests, and updates `developer-notes.md`.
ae357d5 [Bitcoin-Tx] Add tests for sequence number support (Jonas Schnelli)
e59336f [bitcoin-tx] allow to set nSequence number over the in= command (Jonas Schnelli)
a946bb6 [RPC] createrawtransaction: add option to set the sequence number per input (Jonas Schnelli)
269a440 Add test for dbwrapper iterators with same-prefix keys. (Matt Corallo)
6030625 test: Add more thorough test for dbwrapper iterators (Wladimir J. van der Laan)
84c13e7 chain: Add assertion in case of missing records in index db (Wladimir J. van der Laan)
Fixing formatting
Adding test case into automatically generated test case set
Clean up commits
removing extra whitespace from eol
Removing extra whitespace on macro line
a68ec21 Use SipHash-2-4 for address relay selection (Pieter Wuille)
8cc9cfe Switch CTxMempool::mapTx to use a hash index for txids (Pieter Wuille)
382c871 Use SipHash-2-4 for CCoinsCache index (Pieter Wuille)
0b1295b Add SipHash-2-4 primitives to hash (Pieter Wuille)
1475ecf Fix de-serialization bug where AddrMan is corrupted after exception * CAddrDB modified so that when de-serialization code throws an exception Addrman is reset to a clean state * CAddrDB modified to make unit tests possible * Regression test created to ensure bug is fixed * StartNode modifed to clear adrman if CAddrDB::Read returns an error code. (EthanHeilman)
Changing NOP3 op name to OP_CHECKSEQUENCEVERIFY, renaming instances of OP_NOP3 in script_tests.json to CHECKSEQUENCEVERIFY
Cleaning up NOP3 comment
Re-adding test cases that were accidentally deleted, removing dupicated test case, fixing formatting
Removing re-labeling of OP_NOP3 to OP_CSV
Fixing whitespace issues
166e4b0 Notify other serviceQueue thread we are finished to prevent deadlocks. (Pavel Janík)
db18ab2 Reenable multithread scheduler test. (Pavel Janík)
d1d7775 Improve worst-case behavior of CScript::FindAndDelete (Patrick Strateman)
e2a30bc Unit test for CScript::FindAndDelete (Gavin Andresen)
c0f660c Replace c-style cast with c++ style static_cast. (Patrick Strateman)
ec9ad5f Replace memcmp with std::equal in CScript::FindAndDelete (Patrick Strateman)
* CAddrDB modified so that when de-serialization code throws an exception Addrman is reset to a clean state
* CAddrDB modified to make unit tests possible
* Regression test created to ensure bug is fixed
* StartNode modifed to clear adrman if CAddrDB::Read returns an error code.
This patch changes the implementation from one that stores 16 2-bit integers
in one uint32_t's, to one that stores the first bit of 64 2-bit integers in
one uint64_t and the second bit in another. This allows for 450x faster
refreshing and 2.2x faster average speed.
I made a silly mistake in a database wrapper where keys
were sorted by char instead of uint8_t. As x86 char is signed
the sorting for the block index database was messed up, resulting
in a segfault due to missing records.
Add a test to catch:
- Wrong sorting
- Seeking errors
- Iteration result not complete
Pass parent wrapper directly instead of obfuscation key. This
makes it possible for other databases which re-use this code
to use other properties from the database.
Add a namespace dbwrapper_private for private functions to be used
only in dbwrapper.h/cpp and dbwrapper_tests.
Add a method Cursor() to CCoinsView that returns a cursor which can be
used to iterate over the whole UTXO set.
- rpc: Change gettxoutsetinfo to use new Cursor method
- txdb: Remove GetStats method - Now that GetStats is implemented in
terms of Cursor, remove it.
dde46d3 Merge script_valid and script_invalid tests (Pieter Wuille)
009b503 Get rid of expect in script_tests as it's implied by scripterror (Pieter Wuille)
76da761 Make script_error a mandatory 4th field for script_tests (Pieter Wuille)
269281b Fix some misconstructed tests (Pieter Wuille)
d03e466 Fix formatting of NOPs for generated script tests (Pieter Wuille)
c7c6641 Fix JSON pretty printing in script_tests (Pieter Wuille)
The current tests for varint only check that
serialization-deserialization is a roundtrip. That is a useful test, but
it is also good to check for some exact bit patterns, to prevent a code
change that changes the serialization format from going undetected.
As the varint functions are templated, also check with different types.
