Added functional tests for PSBT that test the RPCs. Also added all
of the BIP 174 test vectors (except for the updater tests) in the
functional tests.
Added a Unit test for the BIP 174 updater test vector.
walletprocesspsbt takes a PSBT format transaction, updates the
PSBT with any inputs related to this wallet, signs, and finalizes
the transaction. There is also an option to not sign and just
update.
walletcreatefundedpsbt creates a PSBT from user provided data
in the same form as createrawtransaction. It also funds the transaction
and takes an options argument in the same form as fundrawtransaction.
The resulting PSBT is blank with no input or output data filled
in.
decodepsbt takes a PSBT and decodes it to JSON
combinepsbt takes multiple PSBTs for the same tx and combines them.
finalizepsbt takes a PSBT and finalizes the inputs. If all inputs
are final, it extracts the network serialized transaction and returns
that instead of a PSBT unless instructed otherwise.
createpsbt is like createrawtransaction but for PSBTs instead of
raw transactions.
convertpsbt takes a network serialized transaction and converts it
into a psbt. The resulting psbt will lose all signature data and
an explicit flag must be set to allow transactions with signature
data to be converted.
The SignPSBTInput function takes a PSBTInput, SignatureData, SigningProvider,
and other data necessary for signing. It fills the SignatureData with data from
the PSBTInput, retrieves the UTXO from the PSBTInput, signs and finalizes the
input if possible, and then extracts the results from the SignatureData and
puts them back into the PSBTInput.
Added methods which move data to/from SignaturData objects to
PSBTInput and PSBTOutput objects.
Added sanity checks for PSBTs as a whole which are done immediately
after deserialization.
Added Merge methods to merge a PSBT into another one.
98b181323 [build] Tune wildcards for LIBSECP256K1 target (Karl-Johan Alm)
Pull request description:
Automake would think the target was out of date every time because e.g. '.deps' was updated.
Note: I am assuming that secp256k1 depends on `*.h`, `*.c`, ~~and `libsecp256k1-config.h`~~ (it's `.h` so already included), aside from pre-existing `include/*`. If there are other files that would require a rebuild of the `LIBSECP256K1` target, they should probably be added.
It would be neat if you could exclude specific files, rather than split it up like this, but it doesn't seem possible (https://www.gnu.org/software/make/manual/html_node/Wildcard-Function.html#Wildcard-Function)
Should probably note this:
```Bash
$ V=1 make check VERBOSE=1
Making check in src
make[1]: Entering directory '/home/user/workspace/bitcoin/src'
make[2]: Entering directory '/home/user/workspace/bitcoin/src'
make -C secp256k1 libsecp256k1.la
make[3]: Entering directory '/home/user/workspace/bitcoin/src/secp256k1'
make[3]: 'libsecp256k1.la' is up to date.
make[3]: Leaving directory '/home/user/workspace/bitcoin/src/secp256k1'
make check-TESTS check-local
make[3]: Entering directory '/home/user/workspace/bitcoin/src'
make[4]: Entering directory '/home/user/workspace/bitcoin/src'
make -C secp256k1 libsecp256k1.la
make[5]: Entering directory '/home/user/workspace/bitcoin/src/secp256k1'
make[5]: 'libsecp256k1.la' is up to date.
make[5]: Leaving directory '/home/user/workspace/bitcoin/src/secp256k1'
PASS: test/test_bitcoin.exe
```
Tree-SHA512: 62b133c76e882788dae0c14208a9f5acdbd731c2e7a248f9e01f488b8ec13f9d637d7ad0d63e18d324bb4e088f1836a936649b0fb97bee679eaadedbeed5c981
2f1a30c63 Fix MAX_STANDARD_TX_WEIGHT check (Johnson Lau)
Pull request description:
As suggested by the constant name and its comment in policy.h, a transaction with a weight of exactly MAX_STANDARD_TX_WEIGHT should be allowed. Users could be confused.
Tree-SHA512: af417de1c6a2e6796ebbb39aa0caad8764302ded155cb1bbfbe457e4567c199cc53256189832b17d4aeec369e190b3edd4c6116d5f0b8cf0ede6dfb4ed83bdd3
2dcd7b4ec logging: avoid nStart may be used uninitialized in AppInitMain warning (mruddy)
Pull request description:
Was getting the following compiler warning:
```
init.cpp: In function ‘bool AppInitMain()’:
init.cpp:1616:60: warning: ‘nStart’ may be used uninitialized in this function [-Wmaybe-uninitialized]
LogPrintf(" block index %15dms\n", GetTimeMillis() - nStart);
```
It's ok without this PR, but this PR renames `nStart` to `load_block_index_start_time`, makes it `const`, and also reduces the scope of the variable.
The logging line is moved such that the the time spent will be logged even if a shutdown is requested while the index is being loaded.
Having the log message output even when a shutdown is requested may be how this was intended to work before anyways. That could explain the leading space, as such a log message now looks like:
```
2018-06-30T11:34:05Z [0%]...[16%]...[33%]...[50%]... block index 25750ms
2018-06-30T11:34:17Z Shutdown requested. Exiting.
```
Tree-SHA512: 967048afbc31f2ce8f80ae7d33fee0bdcbe94550cf2b5b662087e2a7cff14a8bf43d909b30f930660c184ec6c3c7e1302a84e3e54fc1723f7412827f4bf2c518
e3aab295e [tests] p2p_segwit: sync_blocks in subtest wrapper. (John Newbery)
55e805085 [tests] p2p_segwit: remove unnecessary arguments from subtests. (John Newbery)
25711c269 [tests] p2p_segwit: log and assert segwit status in subtest wrapper. (John Newbery)
6839863d5 [tests] p2p_segwit: Make sure each subtest leaves utxos for the next. (John Newbery)
bfe32734d [tests] p2p_segwit: wrap subtests with subtest wrapper. (John Newbery)
2af4e398d [tests] p2p_segwit: re-order function definitions. (John Newbery)
94a0134a4 [tests] p2p_segwit: standardise comments/docstrings. (John Newbery)
f7c7f8ecf [tests] p2p_segwit: Fix flake8 warnings. (John Newbery)
Pull request description:
`p2p_segwit.py` is a very long test, composed of multiple subtests. When it fails it's difficult to debug for a couple of reasons:
- Control flow jumps between different methods in the test class, so it's a little difficult to follow the code.
- state may be carried forward unintentionally from one subtest to the next.
Improve that by wrapping the subtests with a `@subtest` decorator which:
- logs progress
- asserts state after each subtest
As usual, I've also included a few commits which generally tidy up the test and improve style.
Tree-SHA512: 3650602b3ce9823dc968cc5f2e716757feadc3dbedb3605eb79bb3df91a6db8ae53431f253b440da690e3a8e9d76de84fad4368a2663aeb40e6b9427cf948870
b81560029 Remove CombineSignatures and replace tests (Andrew Chow)
ed94c8b55 Replace CombineSignatures with ProduceSignature (Andrew Chow)
0422beb9b Make SignatureData able to store signatures and scripts (Andrew Chow)
b6edb4f5e Inline Sign1 and SignN (Andrew Chow)
Pull request description:
Currently CombineSignatures is used to create the final scriptSig or an input. However ProduceSignature is capable of doing this itself. Using both CombineSignatures and ProduceSignature results in code duplication which is unnecessary.
To move the scriptSig construction to ProduceSignatures, the SignatureData class contains two maps to hold pubkeys mapped to signatures, and script ids mapped to scripts. DataFromTransaction is extended to be able to extract signatures, their public keys, and scripts from existing ScriptSigs.
The SignaureData are then passed down to SignStep which can use the aforementioned maps to get the signatures, pubkeys, and scripts that it needs, falling back to the actual SigningProvider and SignatureCreator if the data are not available in the SignatureData.
Additionally, Sign1 and SignN have been removed and their functionality inlined into SignStep since Sign1 is really just a wrapper around CreateSig.
Since ProduceSignature can produce the final scriptSig or scriptWitness by using SignatureData which has extracted data from the transaction, CombineSignatures is unnecessary as ProduceSignature is able to replicate all of CombineSignatures' functionality.
This also furthers BIP 174 support and begins moving towards a BIP 174 style backend.
The tests have also been updated to use the new combining methodology.
Tree-SHA512: 78cd58a4ebe37f79229bd5eee2958a0bb45cd7f36d0e993eee13ff685b3665dd76ef2dfd5f47d34678995bb587f5594100ee5f6c09b1c69ee96d3684d470d01e
49d1f4cdd Detect if char equals int8_t (Chun Kuan Lee)
Pull request description:
Probably fixes#13576. I'm not able to test this. @stacepellegrino, can you test this?
Tree-SHA512: b750e00e11e6b6f6341fec668ec2254cc101c8ebdd4878f320d6cb3b07cf326761146e4ceff0b6405b7e503ff64c093a8274bd524a097e2c49382dc296972c4f
b330c3001 Docs: Improve doc of options addnode, connect, seednode (wodry)
Pull request description:
Just clarify that options `addnode`, `connect` and `seednode` can be specified multiple times.
Tree-SHA512: ed149cabe7fc1d40f2fb6ad8b643656e0ec49cfae1834c157c89170eac1241efa3c5683d97266ff921f5229f28d732c9f7ee030e7902d9a79db1e0c8716fa3db
1fabd59e7 Break circular dependency: init -> * -> init by extracting shutdown.h (Ben Woosley)
e62fdfeea Drop unused init.h includes (Ben Woosley)
Pull request description:
Most includers just wanted to react to pending shutdown.
This isolates access to `fRequestShutdown` and limits access to the shutdown api functions, including the new `CancelShutdown` for setting it to `false`.
Tree-SHA512: df42f75dfbba163576710e9a67cf1228531fd99d70a2f187bfba0bcc476d6749cf88180a97e66a81bb5b6c3c7f0917de7402d26039ba7b644cb7509b02f7e267
747ee89b4 Docs: Modify policy to not translate command-line help (Chun Kuan Lee)
Pull request description:
After #13341, command-line helps would no longer be translated.
Tree-SHA512: 73e7a7545b316cc6ec07f2527cc0e63c922d8e382abd779638583d55a4ee0f3b26ecc9d63f51da8bab2eda9829ec5d1b3162a1115c78a1add2621ba2ccdb839e
07c493f2d scripted-diff: Replace NET_TOR with NET_ONION (wodry)
Pull request description:
This is a follow-up to #13532, where @promag already asked if this renaming would make sense.
If network shall be named _Onion_ instead of _Tor_ (like in the option `onlynet`), renaming the network enum NET_TOR to NET_ONION maybe would make sense and be stringent.
Change was produced with the following script:
```
#!/bin/bash
for file in $(grep --exclude-dir='.git' --files-with-matches --binary-files=without-match --recursive NET_TOR bitcoin/)
do
sed --in-place --expression='s/NET_TOR/NET_ONION/g' $file
done
```
_Tor_ is used at many other places in the code, though.
Tree-SHA512: 4ffdeca8115031465eb64e1c76694fb77b5900c4ea465d3c13d9b6b75a1eb04c45913f83cdc8bdbef28936aeec4655f1d4905b3b98407da3263632a2128a8d23
bb582a59c Add P2WSH destination helper and use it instead of manual hashing (Pieter Wuille)
eaba1c111 Add additional unit tests for invalid IsMine combinations (Pieter Wuille)
e6b9730c4 Do not expose invalidity from IsMine (Pieter Wuille)
Pull request description:
This improves the handling of INVALID in IsMine:
* Extra INVALID conditions were added to `IsMine` (following https://github.com/bitcoin/bitcoin/pull/13142/files#r185349057), but these were untested. Add unit tests for them.
* In https://github.com/bitcoin/bitcoin/pull/13142#issuecomment-386396975 it was suggested to merge `isInvalid` into the return status. This PR takes a different approach, and removes the `isInvalid` entirely. It was only ever used inside tests, as normal users of IsMine don't care about the reason for non-mine-ness, only whether it is or not. As the unit tests are extensive enough, it seems sufficient to have a black box text (with tests for both compressed and uncompressed keys).
Some addition code simplification is done as well.
Tree-SHA512: 3267f8846f3fa4e994f57504b155b0e1bbdf13808c4c04dab7c6886c2c0b88716169cee9c5b350513297e0ca2a00812e3401acf30ac9cde5d892f9fb59ad7fef
Instead of using CombineSignatures to create the final scriptSig or
scriptWitness of an input, use ProduceSignature itself.
To allow for ProduceSignature to place signatures, pubkeys, and scripts
that it does not know about, we pass down the SignatureData to SignStep
which pulls out the information that it needs from the SignatureData.
In addition to having the scriptSig and scriptWitness, have SignatureData
also be able to store just the signatures (pubkeys mapped to sigs) and
scripts (script ids mapped to scripts).
Also have DataFromTransaction be able to extract signatures and scripts
from the scriptSig and scriptWitness of an input to put them in SignatureData.
Adds a new SignatureChecker which takes a SignatureData and puts pubkeys
and signatures into it when it successfully verifies a signature.
Adds a new field in SignatureData which stores whether the SignatureData
was complete. This allows us to also update the scriptSig and
scriptWitness to the final one when updating a SignatureData with another
one.
-BEGIN VERIFY SCRIPT-
sed --in-place'' --expression='s/NET_TOR/NET_ONION/g' $(git grep -I --files-with-matches 'NET_TOR')
-END VERIFY SCRIPT-
The --in-place'' hack is required for sed on macOS to edit files in-place without passing a backup extension.
75848bcf40 [tests] Fix p2p_sendheaders race (John Newbery)
Pull request description:
p2p_sendheaders has a race in part 1.3.
part 1.2 sends a block to the node over the 'test_node' connection, but
doesn't wait for an inv to be received on the 'inv_node' connection. If
we get to part 1.3 before that inv has been received, then the
subsequent call to check_last_inv_announcement could fail.
Tree-SHA512: ba9baffb3a9c0d379259190c737a7a4ad2e1133005a5b026af4f6b67a2978e24db39289551ad29134151879593ef5472be7e569a3557c0740fb51f5c56263d9a
f618ebc4e4 validation: count blocks correctly for check level < 3 (Karl-Johan Alm)
Pull request description:
As noted in https://github.com/bitcoin/bitcoin/pull/13428#issuecomment-396129295 there is a bug where if check level < 3, the resulting count for blocks is wrong, because `pindexState` is never updated.
Post-commit `./bitcoin-cli verifychain 1 3`:
```
2018-06-11T07:12:28Z Verifying last 3 blocks at level 1
2018-06-11T07:12:28Z [0%]...[33%]...[66%]...[99%]...[DONE].
2018-06-11T07:12:28Z No coin database inconsistencies in last 3 blocks (0 transactions)
```
Pre-commit `./bitcoin-cli verifychain 1 3`:
```
2018-06-11T07:13:34Z Verifying last 3 blocks at level 1
2018-06-11T07:13:34Z [0%]...[33%]...[66%]...[99%]...[DONE].
2018-06-11T07:13:34Z No coin database inconsistencies in last 0 blocks (0 transactions)
```
Tree-SHA512: 3d82ed26665162c9615fb0e6e91a46ed4d229a5e6797c6c420e6b0bf1be6e5e02401c6e9a93b7a5aec503a2650d8c20d1b45fe300a922379e4cef8ee26e18d96
4132ad3bf Show symbol for inbound/outbound in peer table (wodry)
Pull request description:
Fixes#13483
The address in the network peer table is prefixed with an up-arrow symbolizing an outbound connection, or an down-array symbolizing an inbound connection. See screenshot.
The user has an easy visual confirmation about the connection direction state. I really like it :)
Impact to columns sorting is grouping by inbound/outbound first, which in my opinion is an advantage, too.
Tree-SHA512: d355f679d34c3006743c06750be5f36a083c1a8376da8f5f35045fcd9df964153409946fdde5007734f23bd692c91355962dc42df31122cdcf88e4affce8bc0e
fa87da2f17 qa: Avoid start/stop of the network thread mid-test (MarcoFalke)
Pull request description:
This simplifies test writing by removing the need to handle the network thread in tests. E.g. start thread, join thread, restart thread mid-test, adding p2p connections at the "right" time, ...
Tree-SHA512: 533642f12fef5496f1933855edcdab1a7ed901d088d34911749cd0f9e044c8a6cb1f89985ac3a7f41a512943663e4e270a61978f6f072143ae050cd102d4eab8
2fca6568f2 depends: Update Qt download url (fanquake)
Pull request description:
Qt has moved the 5.7.1 downloads from the [official_releases](https://download.qt.io/official_releases/qt/) directory to it's [archive](https://download.qt.io/archive/qt/).
```
Fetching qtbase-opensource-src-5.7.1.tar.gz from http://download.qt.io/official_releases/qt/5.7/5.7.1/submodules
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0
curl: (22) The requested URL returned error: 404 Not Found
```
This updates the qt download url in depends so we aren't always falling back on [bitcoincore.org/depends-sources](https://bitcoincore.org/depends-sources).
Tree-SHA512: 2aca3260fa0550322108b5bcc8c226e13b48c3500b7d95f545651e5b60d290edde16d9f6dea45f438c39a1388cb304cc16768a9728d32d3a441667968ea35425
c2e4fc84ec bench: Simplify CoinSelection (João Barbosa)
Pull request description:
Closes#13549.
As pointed by @MarcoFalke:
- `SelectCoinsMinConf` should always succeed as there are enough coins in the wallet.
- Removed creating the coins in the wallet.
Tree-SHA512: 965c363bcaf0ca7a1dec35b5cf4866abcf190c53eb7012dc4aeb4d29830f13a7465644bfb5a47f6ea3eaa86e4d4a57fe41e7b2593bf5094b76a551c4c71625bb
ea49e06c82 tests: Fix incorrect documentation for test case cuckoocache_hit_rate_ok (practicalswift)
Pull request description:
Fix incorrect documentation for test case `cuckoocache_hit_rate_ok`.
Tree-SHA512: 2c1138953f077666c0bba784d9f5e5a627403dcd8b81851ca74793f1bf7bb5dc274af7887d51edb6669de1e9354ca7e5dfba80a3ce62f16eaaee6c6fa57b79f5
fa103a5d5e [qa] wallet_basic: Specify minimum required amount for listunspent (MarcoFalke)
Pull request description:
A value less than that would fail the tests later on anyway:
```
File "./test/functional/wallet_basic.py", line 250, in run_test
self.nodes[1].sendrawtransaction(signed_raw_tx['hex'])
test_framework.authproxy.JSONRPCException: bad-txns-in-belowout, value in (1.00) < value out (49.998) (code 16) (-26)
Tree-SHA512: 7e72ad02b5623bc078610da06c34721836822a920a4e85b12a1e0f339e3205cdc11d39763197770e649fb73376f922ff91a8f244b465195e50a6798658e04f80
