fac70134a rpc: Update createrawtransaction examples (MarcoFalke)
fa06dfce0 [rpc] createrawtransaction: Accept sorted outputs (MarcoFalke)
8acd25d85 rpc: Allow typeAny in RPCTypeCheck (MarcoFalke)
Pull request description:
The second parameter of the `createrawtransaction` is a dictionary of the outputs. This comes with at least two drawbacks:
* In case of duplicate keys, either of them might silently disappear, with no user feedback at all. A user needs to make other mistakes, but this could eventually lead to abnormal tx fees.
* A dictionary does not guarantee that keys are sorted. Again, a user needs to keep this in mind, as it could eventually lead to excessive tx fees.
Even though my scenario of loss-of-funds is unlikely to happen, I see it as a inconvenience that should be fixed.
Tree-SHA512: cd562f34f7f9f79c7d3433805971325c388c2035611be283980f4049066a622df4f0afdc11d7ac96662260ec0115147cb65e1ab5268f5a1b063242f3fe425f77
* Z is the zone designator for the zero UTC offset.
* T is the delimiter used to separate date and time.
This makes it clear for the end-user that the date/time logged is
specified in UTC and not in the local time zone.
92f1f8b31 Split off key_io_tests from base58_tests (Pieter Wuille)
119b0f85e Split key_io (address/key encodings) off from base58 (Pieter Wuille)
ebfe217b1 Stop using CBase58Data for ext keys (Pieter Wuille)
32e69fa0d Replace CBitcoinSecret with {Encode,Decode}Secret (Pieter Wuille)
Pull request description:
This PR contains some of the changes left as TODO in #11167 (and built on top of that PR). They are not intended for backporting.
This removes the `CBase58`, `CBitcoinSecret`, `CBitcoinExtKey`, and `CBitcoinExtPubKey` classes, in favor of simple `Encode`/`Decode` functions. Furthermore, all Bitcoin-specific logic (addresses, WIF, BIP32) is moved to `key_io.{h,cpp}`, leaving `base58.{h,cpp}` as a pure utility that implements the base58 encoding/decoding logic.
Tree-SHA512: a5962c0ed27ad53cbe00f22af432cf11aa530e3efc9798e25c004bc9ed1b5673db5df3956e398ee2c085e3a136ac8da69fe7a7d97a05fb2eb3be0b60d0479655
5f8cc0df1 Add a test for large tx output scripts with segwit input. (Richard Kiss)
Pull request description:
This test failed in pycoin but passed in bitcoin, so I thought I'd share it.
Tree-SHA512: 95dff4e03afea4d93ff5e99aa06004446c3df022c2e8a191cac8981107135a5ac2bd3ba1c3a9c4eda9f8f63f584cc1700b7ef57ee6ec2c66a72c699b51bdb61a
e68172ed9 Add test-before-evict discipline to addrman (Ethan Heilman)
Pull request description:
This change implement countermeasures 3 (test-before-evict) suggested in our paper: ["Eclipse Attacks on Bitcoin’s Peer-to-Peer Network"](http://cs-people.bu.edu/heilman/eclipse/).
# Design:
A collision occurs when an address, addr1, is being moved to the tried table from the new table, but maps to a position in the tried table which already contains an address (addr2). The current behavior is that addr1 would evict addr2 from the tried table.
This change ensures that during a collision, addr1 is not inserted into tried but instead inserted into a buffer (setTriedCollisions). The to-be-evicted address, addr2, is then tested by [a feeler connection](https://github.com/bitcoin/bitcoin/pull/8282). If addr2 is found to be online, we remove addr1 from the buffer and addr2 is not evicted, on the other hand if addr2 is found be offline it is replaced by addr1.
An additional small advantage of this change is that, as no more than ten addresses can be in the test buffer at once, and addresses are only cleared one at a time from the test buffer (at 2 minute intervals), thus an attacker is forced to wait at least two minutes to insert a new address into tried after filling up the test buffer. This rate limits an attacker attempting to launch an eclipse attack.
# Risk mitigation:
- To prevent this functionality from being used as a DoS vector, we limit the number of addresses which are to be tested to ten. If we have more than ten addresses to test, we drop new addresses being added to tried if they would evict an address. Since the feeler thread only creates one new connection every 2 minutes the additional network overhead is limited.
- An address in tried gains immunity from tests for 4 hours after it has been tested or successfully connected to.
# Tests:
This change includes additional addrman unittests which test this behavior.
I ran an instance of this change with a much smaller tried table (2 buckets of 64 addresses) so that collisions were much more likely and observed evictions.
```
2016-10-27 07:20:26 Swapping 208.12.64.252:8333 for 68.62.95.247:8333 in tried table
2016-10-27 07:20:26 Moving 208.12.64.252:8333 to tried
```
I documented tests we ran against similar earlier versions of this change in #6355.
# Security Benefit
This is was originally posted in PR #8282 see [this comment for full details](https://github.com/bitcoin/bitcoin/pull/8282#issuecomment-237255215).
To determine the security benefit of these larger numbers of IPs in the tried table I modeled the attack presented in [Eclipse Attacks on Bitcoin’s Peer-to-Peer Network](https://eprint.iacr.org/2015/263).
**Default node:** 595 attacker IPs for ~50% attack success.
**Default node + test-before-evict:** 620 attacker IPs for ~50% attack success.
**Feeler node:** 5540 attacker IPs for ~50% attack success.
**Feeler node + test-before-evict:** 8600 attacker IPs for ~50% attack success.
The node running feeler connections has 10 times as many online IP addresses in its tried table making an attack 10 times harder (i.e. requiring the an attacker require 10 times as many IP addresses in different /16s). Adding test-before-evict increases resistance of the node by an additional 3000 attacker IP addresses.
Below I graph the attack over even greater attacker resources (i.e. more attacker controled IP addresses). Note that test-before-evict maintains some security far longer even against an attacker with 50,000 IPs. If this node had a larger tried table test-before-evict could greatly boost a nodes resistance to eclipse attacks.
Tree-SHA512: fdad4d26aadeaad9bcdc71929b3eb4e1f855b3ee3541fbfbe25dca8d7d0a1667815402db0cb4319db6bd3fcd32d67b5bbc0e12045c4252d62d6239b7d77c4395
Changes addrman to use the test-before-evict discipline in which an
address is to be evicted from the tried table is first tested and if
it is still online it is not evicted.
Adds tests to provide test coverage for this change.
This change was suggested as Countermeasure 3 in
Eclipse Attacks on Bitcoin’s Peer-to-Peer Network, Ethan Heilman,
Alison Kendler, Aviv Zohar, Sharon Goldberg. ePrint Archive Report
2015/263. March 2015.
2736c9e05 Avoid unintentional unsigned integer wraparounds in tests (practicalswift)
Pull request description:
Avoid unintentional unsigned integer wraparounds in tests.
This is a subset of #11535 as suggested by @MarcoFalke :-)
Tree-SHA512: 4f4ee8a08870101a3f7451aefa77ae06aaf44e3c3b2f7555faa2b8a8503f97f34e34dffcf65154278f15767dc9823955f52d1aa7b39930b390e57cdf2b65e0f3
Splits signrwatransaction into a wallet version (signrawtransactionwithwallet) and
non-wallet version (signrawtransactionwithkey). signrawtransaction is marked as DEPRECATED
and will call the right signrawtransaction* command as per the parameters in order to
maintain compatibility.
Updated signrawtransactions test to use new RPCs
be45a67 Add some script tests related to BOOL ops and odd values like negative 0. (Richard Kiss)
Pull request description:
Add some script tests related to BOOL ops and odd values like negative 0.
Tree-SHA512: 8e633f7ea5eea39e31016994baf60f295fa1dc8cae27aa5fcfc741ea97136bfb3ddc57bb62b9c6bf9fe256fc09cdd184906ba8e611e297cf8d2d363da2bbf1d4
9ad6746ccd Use static_cast instead of C-style casts for non-fundamental types (practicalswift)
Pull request description:
A C-style cast is equivalent to try casting in the following order:
1. `const_cast(...)`
2. `static_cast(...)`
3. `const_cast(static_cast(...))`
4. `reinterpret_cast(...)`
5. `const_cast(reinterpret_cast(...))`
By using `static_cast<T>(...)` explicitly we avoid the possibility of an unintentional and dangerous `reinterpret_cast`. Furthermore `static_cast<T>(...)` allows for easier grepping of casts.
For a more thorough discussion, see ["ES.49: If you must use a cast, use a named cast"](https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#es49-if-you-must-use-a-cast-use-a-named-cast) in the C++ Core Guidelines (Stroustrup & Sutter).
Tree-SHA512: bd6349b7ea157da93a47b8cf238932af5dff84731374ccfd69b9f732fabdad1f9b1cdfca67497040f14eaa85346391404f4c0495e22c467f26ca883cd2de4d3c
0a22a52 Use mempool's ancestor sort in transaction selection (Suhas Daftuar)
7abfa53 Add test for new ancestor feerate sort behavior (Suhas Daftuar)
9a51319 Sort mempool by min(feerate, ancestor_feerate) (Suhas Daftuar)
6773f92 Refactor CompareTxMemPoolEntryByDescendantScore (Suhas Daftuar)
Pull request description:
This more closely approximates the desirability of a given transaction for
mining, and should result in less re-sorting when transactions get removed from
the mempool after being mined.
I measured this as approximately a 5% speedup in removeForBlock.
Tree-SHA512: ffa36b567c5dfe3e8908c545a459b6a5ec0de26e7dc81b1050dd235cac9046564b4409a3f8c5ba97bd8b30526e8fec8f78480a912e317979467f32305c3dd37b
8e617e3 Remove unused mempool index (Suhas Daftuar)
Pull request description:
We haven't used the "mining_score" index since 0.12, so remove it.
Tree-SHA512: ae37b8663194986eaeecfc2bbeca7ecb4ae6f0d8384515fa218cbc939a580d4b9f7f997c5297c3f1b3c3a0651edb092f373ac9a4808aaec30d38cb99d5f3ed70
18be3ab139 Adding test case for SINGLE|ANYONECANPAY hash type in tx_valid.json (Chris Stewart)
Pull request description:
We are missing a test vector for SINGLE|ANYONECANPAY inside of tx_valid.json. This addresses the issue #12060
Tree-SHA512: e3526113477dbf575c4a844cf489dcfa2c037c6d928af6f97413edc1a8d29cdf2143da96471cdfd3de08bf5ed178117ed67926fd70fd42ca391ac0bb0d08f3fd
b224a47a1 Add address_types test (Pieter Wuille)
7ee54fd7c Support downgrading after recovered keypool witness keys (Pieter Wuille)
940a21932 SegWit wallet support (Pieter Wuille)
f37c64e47 Implicitly know about P2WPKH redeemscripts (Pieter Wuille)
57273f2b3 [test] Serialize CTransaction with witness by default (Pieter Wuille)
cf2c0b6f5 Support P2WPKH and P2SH-P2WPKH in dumpprivkey (Pieter Wuille)
37c03d3e0 Support P2WPKH addresses in create/addmultisig (Pieter Wuille)
3eaa003c8 Extend validateaddress information for P2SH-embedded witness (Pieter Wuille)
30a27dc5b Expose method to find key for a single-key destination (Pieter Wuille)
985c79552 Improve witness destination types and use them more (Pieter Wuille)
cbe197470 [refactor] GetAccount{PubKey,Address} -> GetAccountDestination (Pieter Wuille)
0c8ea6380 Abstract out IsSolvable from Witnessifier (Pieter Wuille)
Pull request description:
This implements a minimum viable implementation of SegWit wallet support, based on top of #11389, and includes part of the functionality from #11089.
Two new configuration options are added:
* `-addresstype`, with options `legacy`, `p2sh`, and `bech32`. It controls what kind of addresses are produced by `getnewaddress`, `getaccountaddress`, and `createmultisigaddress`.
* `-changetype`, with the same options, and by default equal to `-addresstype`, that controls what kind of change is used.
All wallet private and public keys can be used for any type of address. Support for address types dependent on different derivation paths will need a major overhaul of how our internal detection of outputs work. I expect that that will happen for a next major version.
The above also applies to imported keys, as having a distinction there but not for normal operations is a disaster for testing, and probably for comprehension of users. This has some ugly effects, like needing to associate the provided label to `importprivkey` with each style address for the corresponding key.
To deal with witness outputs requiring a corresponding redeemscript in wallet, three approaches are used:
* All SegWit addresses created through `getnewaddress` or multisig RPCs explicitly get their redeemscripts added to the wallet file. This means that downgrading after creating a witness address will work, as long as the wallet file is up to date.
* All SegWit keys in the wallet get an _implicit_ redeemscript added, without it being written to the file. This means recovery of an old backup will work, as long as you use new software.
* All keypool keys that are seen used in transactions explicitly get their redeemscripts added to the wallet files. This means that downgrading after recovering from a backup that includes a witness address will work.
These approaches correspond to solutions 3a, 1a, and 5a respectively from https://gist.github.com/sipa/125cfa1615946d0c3f3eec2ad7f250a2. As argued there, there is no full solution for dealing with the case where you both downgrade and restore a backup, so that's also not implemented.
`dumpwallet`, `importwallet`, `importmulti`, `signmessage` and `verifymessage` don't work with SegWit addresses yet. They're remaining TODOs, for this PR or a follow-up. Because of that, several tests unexpectedly run with `-addresstype=legacy` for now.
Tree-SHA512: d425dbe517c0422061ab8dacdc3a6ae47da071450932ed992c79559d922dff7b2574a31a8c94feccd3761c1dffb6422c50055e6dca8e3cf94a169bc95e39e959
c99a3c32c8 [tests] util_tests.cpp: actually check ignored args (Anthony Towns)
Pull request description:
An array with 7 elements was setup for checking argument parsing, but
was passed to ParseParamaeters with argc=5, meaning the interpretation
of the last two arguments was never actually checked.
Tree-SHA512: 7b81fde49742e524f1bb67e2ec084f5909ae36125f237f0210df4587c62e5a5a8f277f13543f0a85ad145c4bb80d62339a7d50d7ed41659df318c8198ea7f428
An array with 7 elements was setup for checking argument parsing, but
was passed to ParseParamaeters with argc=5, meaning the interpretation
of the last two arguments was never actually checked.
97d2b09c12 Add helper to wait for validation interface queue to catch up (Matt Corallo)
36137497f1 Block ActivateBestChain to empty validationinterface queue (Matt Corallo)
5a933cefcc Add an interface to get the queue depth out of CValidationInterface (Matt Corallo)
a99b76f269 Require no cs_main lock for ProcessNewBlock/ActivateBestChain (Matt Corallo)
a734896038 Avoid cs_main in net_processing ActivateBestChain calls (Matt Corallo)
66aa1d58a1 Refactor ProcessGetData in anticipation of avoiding cs_main for ABC (Matt Corallo)
818075adac Create new mutex for orphans, no cs_main in PLV::BlockConnected (Matt Corallo)
Pull request description:
This should fix#11822.
It ended up bigger than I hoped for, but its not too gnarly. Note that "
Require no cs_main lock for ProcessNewBlock/ActivateBestChain" is mostly pure code-movement.
Tree-SHA512: 1127688545926f6099449dca6a4e6609eefc3abbd72f1c66e03d32bd8c7b31e82097d8307822cfd1dec0321703579cfdd82069cab6e17b1024e75eac694122cb
This requires the removal of some very liberal (incorrect) cs_mains
sprinkled in some tests. It adds some chainActive.Tip() races, but
the tests are all single-threaded anyway.
3e1ee31 [Tests] Adding unit tests for GetDifficulty in blockchain.cpp. (sean)
Pull request description:
blockchain.cpp has low unit test coverage. This commit is intended
to start improving its code coverage to reasonable levels. One or more
follow up commits will complete the task that this commit is starting
(though the usefulness of this commit is not dependent upon later
commits).
Note that these tests were not written based upon a specification of how
GetDifficulty *should* work, but rather how it actually *does* work. As
a result, if there are any bugs in the current GetDifficulty
implementation, these unit tests serve to lock them in rather than
expose them.
-- Why has blockchain.cpp been modified if this is a unit testing change?
Since the existing GetDifficulty function relies on a global variable,
chainActive, it was not suitable for unit testing purposes. Both the
existing GetDifficulty function and the unit tests now call through to
a new, more modular version of GetDifficulty that can work on any chain,
not just chainActive.
-- Why does blockchain_tests.cpp directly include blockchain.cpp instead
of blockchain.h?
While the new GetDifficulty function's signature is arguably better than
the old one's, it still isn't great, and doesn't seem to warrant inclusion
as part of the blockchain.h API, especially since only test code is
directly using it. If a better way of exposing the new GetDifficulty
function to unit tests exists, please mention it and the commit will be
updated accordingly.
-- Why is the test fixture named blockchain_difficulty_tests rather than
blockchain_tests?
The Bitcoin Core policy for naming unit test files is to match the the
file under test ("blockchain" becomes "blockchain_tests"). While this
commit complies with that, blockchain.cpp is a massive file, such that
having all of the unit tests in one file will tend towards disorder.
Since there will be a lot more tests added to this file, the intention
is to divide up different types of tests into different test fixtures
within the same file.
Tree-SHA512: a7dda9c2a9414d4819b4d2911f5637891dc19cecbecfc1463846161d2a78793151927a5ab911c69a5d3013f7668e75a1d78a65667cb9d83910cda439cbe84d62
2862b56 [tests] remove redundant univalue_tests.cpp (John Newbery)
Pull request description:
univalue unit tests were added in #4730 , and exist at `/src/test/univalue_tests.cpp` (outside the univalue tree). That test was brought into the univalue repository in https://github.com/bitcoin-core/univalue/pull/4 , which was pulled into the github repository in https://github.com/bitcoin/bitcoin/pull/11420.
That means that the univalue test exists in two places:
1. `/src/test/univalue_tests.cpp`
2. `/src/univalue/test/object.cpp`
(2) is a strict superset of (1). It adds some macros to work around boost not being a univalue dependency, and adds a few extra lines of test.
Therefore remove `/src/test/univalue_tests.cpp`
Tree-SHA512: 3747b10bbf62e9f12363905488b29945ad559ddca68c5c03d8a362de612a51f408f41a04d3712c6889bfc1632fb1a5fa0d7df0fbf02c322b3981a6d698f501b0
12781db [Tests] check specific validation error in miner tests (Sjors Provoost)
Pull request description:
## Problem
`BOOST_CHECK_THROW` merely checks that some `std::runtime_error` is
thrown, but not which one.
Here's an example of how this can cause a test to pass when a developer
introduces a consensus bug. The test for the sigops limit assumes
that `CreateNewBlock` fails with `bad-blk-sigops`. However it can
also fail with bad-txns-vout-negative, if a naive developer lowers
`BLOCKSUBSIDY` to `1*COIN`.
## Solution
`BOOST_CHECK_EXCEPTION` allows an additional predicate function. This
commit uses this for all exceptions that are checked for in
`miner_tets.cpp`:
* `bad-blk-sigops`
* `bad-cb-multiple`
* `bad-txns-inputs-missingorspent`
* `block-validation-failed`
If the function throws a different error, the test will fail. Although the message produced by Boost is a bit [confusing](http://boost.2283326.n4.nabble.com/Test-BOOST-CHECK-EXCEPTION-error-message-still-vague-tt4683257.html#a4683554), it does show which error was actually thrown. Here's what the above `1*COIN` bug would result in:
<img width="1134" alt="schermafbeelding 2017-09-02 om 23 42 29" src="https://user-images.githubusercontent.com/10217/29998976-815cabce-9038-11e7-9c46-f5f6cfb0ca7d.png">
## Other considerations
A more elegant solution in my opinion would be to subclass `std::runtime_error` for each `INVALID_TRANSACTION` type, but this would involve touching consensus code.
I put the predicates in `test_bitcoin.h` because I assume they can be reused in other test files. However [serialize_tests.cpp](https://github.com/bitcoin/bitcoin/blob/v0.15.0rc3/src/test/serialize_tests.cpp#L245) also uses `BOOST_CHECK_EXCEPTION` and it defines the predicate in the test file itself.
Instead of four `IsRejectInvalidReasonX(std::runtime_error const& e)` functions, I'd prefer something reusable like `bool IsRejectInvalidReason(String reason)(std::runtime_error const& e)`, which would be used like `BOOST_CHECK_EXCEPTION(functionThatThrows(), std::runtime_error, IsRejectInvalidReason("bad-blk-sigops")`. I couldn't figure out how to do that in C++.
Tree-SHA512: e364f19b4ac19f910f6e8d6533357f57ccddcbd9d53dcfaf923d424d2b9711446d6f36da193208b35788ca21863eadaa7becd9ad890334d334bccf8c2e63dee1
