fa324a8b15 doc: Rewrite some validation doc as lock annotations (MarcoFalke)
Pull request description:
#13402 added some lock annotations in comments. This pull removes them and adds clang-readable locking annotations instead.
Tree-SHA512: 2d392efa8ac4978830a9df08b2009e69d6f1ac031f62be2275ae8d7c7e483331c7f8d458d865443af907a7af27a592421c6cca6b2df3f2877e0f369b9198f383
66b2cf1ccf Use immintrin.h everywhere for intrinsics (Pieter Wuille)
4c935e2eee Add SHA256 implementation using using Intel SHA intrinsics (Pieter Wuille)
268400d318 [Refactor] CPU feature detection logic for SHA256 (Pieter Wuille)
Pull request description:
Based on #13191.
This adds SHA256 implementations that use Intel's SHA Extension instructions (using intrinsics). This needs GCC 4.9 or Clang 3.4.
In addition to #13191, two extra implementations are provided:
* (a) A variable-length SHA256 implementation using SHA extensions.
* (b) A 2-way 64-byte input double-SHA256 implementation using SHA extensions.
Benchmarks for 9001-element Merkle tree root computation on an AMD Ryzen 1800X system:
* Using generic C++ code (pre-#10821): 6.1ms
* Using SSE4 (master, #10821): 4.6ms
* Using 4-way SSE4 specialized for 64-byte inputs (#13191): 2.8ms
* Using 8-way AVX2 specialized for 64-byte inputs (#13191): 2.1ms
* Using 2-way SHA-NI specialized for 64-byte inputs (this PR): 0.56ms
Benchmarks for 32-byte SHA256 on the same system:
* Using SSE4 (master, #10821): 190ns
* Using SHA-NI (this PR): 53ns
Benchmarks for 1000000-byte SHA256 on the same system:
* Using SSE4 (master, #10821): 2.5ms
* Using SHA-NI (this PR): 0.51ms
Tree-SHA512: 2b319e33b22579f815d91f9daf7994a5e1e799c4f73c13e15070dd54ba71f3f6438ccf77ae9cbd1ce76f972d9cbeb5f0edfea3d86f101bbc1055db70e42743b7
d280617bf5 [qa] Add a test for merkle proof malleation (Suhas Daftuar)
ed82f17000 have verifytxoutproof check the number of txns in proof structure (Gregory Sanders)
Pull request description:
Recent publication of a weakness in Bitcoin's merkle tree construction demonstrates many SPV applications vulnerable to an expensive to pull off yet still plausible attack: https://bitslog.wordpress.com/2018/06/09/leaf-node-weakness-in-bitcoin-merkle-tree-design/
This change would at least allow `verifytxoutproof` to properly validate that the proof matches a known block, with known number of transactions any time after the full block is processed. This should neuter the attack entirely.
The negative is that a header-only processed block/future syncing mode would cause this to fail until the node has imported the data required.
related: #13451
`importprunedfunds` needs this check as well. Can expand it to cover this if people like the idea.
Tree-SHA512: 0682ec2b622a38b29f3f635323e0a8b6fc071e8a6fd134c954579926ee7b516e642966bafa667016744ce49c16e19b24dbc8801f982a36ad0a6a4aff6d93f82b
85644354c5 Corrected text to reflect new[er] process of specifying fingerprints instead of individual keys. (Jeff Frontz)
Pull request description:
More accurately reflects the contents of the directory and keys.txt (i.e., contains fingerprints of keys instead of actual keys).
Tree-SHA512: c8e70e6d905faa348b269cb39c72a1b4b525bc00445e4a5ad4204f54925da4b055444828e62c45c6f8798fe9d23526eb7779d9b2edcf22679441586a558d0d05
57889e688d bitcoin-tx: Stricter check for valid integers (Daniel Kraft)
Pull request description:
Just calling `atoi` to convert strings to integers does not check for valid integers very thoroughly; in particular, it just ignores everything starting from the first non-numeral character. Even a string like "foo" is fine and silently returns 0.
This meant that `bitcoin-tx` would not fail if such a string was passed in various places where an integer is expected (like the `locktime` or an input/output index); this means that it would, for instance, silently accept a typo and interpret it in an unexpected way.
In this change, we use `ParseInt64` for parsing strings to integers, which actually verifies that the full string is valid as number. New tests in the `bitcoin-util-test` cover the new error paths.
This fixes#13599.
Tree-SHA512: 146a0af275e9f57784e5d0582d3defbac35551b54b6b7232f8a0b20db04aa611125e52aa4512ef2f8ed2cafc2a12fe586f9d10ed66d641cff090288f279b1988
800dea88a6 Upgrade Qt depends to 5.9.6 (Sebastian Kung)
70afa65b17 Fix depends Qt5.9.4 mac build (Ken Lee)
28482efefb Ugrade Qt depends to Qt5.9.4 (Sebastian Kung)
Pull request description:
With the introduction of Ubuntu 18.04 (Bionic Beaver) modern versions of gcc and mingw that allow cross compilation of versions of Qt greater than 5.8 are now readily available. This pull requests upgrades the Qt depends recipe from Qt 5.7.1 to Qt 5.9.6. Qt 5.9.x is the current LTS version and should be supported by Qt until 2020.
Tree-SHA512: 439295d594ff8954a5ba5e348a0452713721c805485be2edcb9f8603ee59e96db5a61e1c684bdff36bbfd643a79cd35c289817257af88f489d2890e7843460bf
161e8d40a4 RPC: Add new getzmqnotifications method. (Daniel Kraft)
caac39b0ac Make ZMQ notification interface instance global. (Daniel Kraft)
Pull request description:
This adds a new RPC method `getzmqnotifications`, which returns information about all active ZMQ notification endpoints. This is useful for software that layers on top of bitcoind, so it can verify that ZeroMQ is enabled and also figure out where it should listen.
See #13526.
Tree-SHA512: edce722925741c84ddbf7b3a879fc9db1907e5269d0d97138fe724035d93ee541c2118c24fa92f4197403f380d0e25c2fda5ca6c62d526792ea749cf527a99a0
5e362c0cf0 Fix command line help for -printtoconsole and -debuglogfile (Samuel B. Atwood)
Pull request description:
This is a rebased version of #13589 with the changes to the 0.16.x release notes removed.
> #13004 changed the default behavior for printtoconsole but this has not been reflected in the command line help.
> This fixes the description of -printtoconsole to reflect this change, and also provides the user with missing information on how to explicitly disable logging to debug.log.
> At present I have made the latter update to two separate places (-printtoconsole and -debuglogfile) because a user looking for information on how to disable logging is probably going to look in the "Debugging/Testing Options" section. Moving -debuglogfile from the "General" options category to the "Debugging/Testing" section could potentially remove the need for this redundancy but may be out of the scope of this PR.
Tree-SHA512: 7461d59a1864039d5a9dfcce765a1169df882f51a4ca50a6066416c0803821cd821be07be534e0bd57f0a22c0b45adb881a93abbe91962bc37d2d228f35ee712
4b6ab02122 Remove unused argument to ProcessGetBlockData(...) (practicalswift)
c469ecf22e net: Remove unused interrupt from SendMessages (fanquake)
Pull request description:
Discussed very briefly with cfields.
Includes 65b4400 from #13554 as it's a similar refactor.
Tree-SHA512: 45cd64208a5c8164242db74e6687e9344ea592bab5e7f9ba8e1bb449057fc908ec9d8b8523748a68426e4a4304e3388a138cd834698b39837b2149b72beefdc9
Makes AddAndGetDestinationForScript use a generic CKeyStore rather than
the wallet, and makes it always add the script to the keystore, rather
than only adding related (redeem) scripts.
63c16ed507 Use __cpuid_count for gnu C to avoid gitian build fail. (Chun Kuan Lee)
Pull request description:
Fixes#13538
Tree-SHA512: 161ae4db022288ae8631a166eaea2d08cf2c90bcd27218a094a754276de30b92ca9cfb5a79aa899c5a9d0534c5d7261037e7e915e1b92bc7067ab1539dc2b51e
#13004 changed the default behavior for printtoconsole but this has not been reflected in the command line help.
This fixes the description of -printtoconsole to reflect this change, and also provides the user with missing information on how to explicitly disable logging to debug.log.
At present I have made the latter update to two separate places (-printtoconsole and -debuglogfile) because a user looking for information on how to disable logging is probably going to look in the "Debugging/Testing Options" section. Moving -debuglogfile from the "General" options category to the "Debugging/Testing" section could potentially remove the need for this redundancy but may be out of the scope of this PR.
ea65182f03 [wallet] loadwallet shouldn't create new wallets. (John Newbery)
Pull request description:
A bug in the initial implementation of loadwallet meant that if the
arguement was a directory that didn't contain a wallet.dat file, a new
wallet would be created in that directory. Fix that so that if a
directory is passed in, it must contain a wallet.dat file.
Bug reported by promag (João Barbosa).
Tree-SHA512: 0a59fa8a33fde51a88544ad288b00e4995284fe16424f643076aaba42b8244fff362145217650ee53d518dfab7efbed4237632c34cdd3dcbbecaa9ecaab5fd7b
Just calling atoi to convert strings to integers does not check for
valid integers very thoroughly; in particular, it just ignores
everything starting from the first non-numeral character. Even a string
like "foo" is fine and silently returns 0.
This meant that bitcoin-tx would not fail if such a string was passed in
various places where an integer is expected (like the locktime or an
input/output index); this means that it would, for instance, silently
accept a typo and interpret it in an unexpected way.
In this change, we use ParseInt64 for parsing strings to integers,
which actually verifies that the full string is valid as number.
New tests in the bitcoin-util-test cover the new error paths.
1fc605a8ae fix bench/prevector.cpp (Akio Nakamura)
Pull request description:
This patch intends to fix some incorrect action of bench/prevector.cpp.
1. PrevectorClear()
2nd call of ```clear()``` should to operate t1 instead of t0.
This patch changes t0 to t1.
2. PREVECTOR_TEST()
PREVECTOR_TEST macro should to call both
```PrevectorXX<nontrivial_t>(state)``` and ```PrevectorXX<trivial_t>(state)```
by specific ```"name"``` which given by parameter instead of calling
```PrevectorResize<>()``` regardless of ```"name"```.
This patch changes ```"PrevectorResize<"``` of this macro to
```"Prevector ## name<"```.
Tree-SHA512: d0498c6d627d7e96fc8ccfb329ca0be2641535b1ce1923d9b1fc720825f9bf4d7281dc8d5ae929038e37b3e625189af9807cb62e6d20933d73832a6dff4b5596
98b181323 [build] Tune wildcards for LIBSECP256K1 target (Karl-Johan Alm)
Pull request description:
Automake would think the target was out of date every time because e.g. '.deps' was updated.
Note: I am assuming that secp256k1 depends on `*.h`, `*.c`, ~~and `libsecp256k1-config.h`~~ (it's `.h` so already included), aside from pre-existing `include/*`. If there are other files that would require a rebuild of the `LIBSECP256K1` target, they should probably be added.
It would be neat if you could exclude specific files, rather than split it up like this, but it doesn't seem possible (https://www.gnu.org/software/make/manual/html_node/Wildcard-Function.html#Wildcard-Function)
Should probably note this:
```Bash
$ V=1 make check VERBOSE=1
Making check in src
make[1]: Entering directory '/home/user/workspace/bitcoin/src'
make[2]: Entering directory '/home/user/workspace/bitcoin/src'
make -C secp256k1 libsecp256k1.la
make[3]: Entering directory '/home/user/workspace/bitcoin/src/secp256k1'
make[3]: 'libsecp256k1.la' is up to date.
make[3]: Leaving directory '/home/user/workspace/bitcoin/src/secp256k1'
make check-TESTS check-local
make[3]: Entering directory '/home/user/workspace/bitcoin/src'
make[4]: Entering directory '/home/user/workspace/bitcoin/src'
make -C secp256k1 libsecp256k1.la
make[5]: Entering directory '/home/user/workspace/bitcoin/src/secp256k1'
make[5]: 'libsecp256k1.la' is up to date.
make[5]: Leaving directory '/home/user/workspace/bitcoin/src/secp256k1'
PASS: test/test_bitcoin.exe
```
Tree-SHA512: 62b133c76e882788dae0c14208a9f5acdbd731c2e7a248f9e01f488b8ec13f9d637d7ad0d63e18d324bb4e088f1836a936649b0fb97bee679eaadedbeed5c981
2f1a30c63 Fix MAX_STANDARD_TX_WEIGHT check (Johnson Lau)
Pull request description:
As suggested by the constant name and its comment in policy.h, a transaction with a weight of exactly MAX_STANDARD_TX_WEIGHT should be allowed. Users could be confused.
Tree-SHA512: af417de1c6a2e6796ebbb39aa0caad8764302ded155cb1bbfbe457e4567c199cc53256189832b17d4aeec369e190b3edd4c6116d5f0b8cf0ede6dfb4ed83bdd3
2dcd7b4ec logging: avoid nStart may be used uninitialized in AppInitMain warning (mruddy)
Pull request description:
Was getting the following compiler warning:
```
init.cpp: In function ‘bool AppInitMain()’:
init.cpp:1616:60: warning: ‘nStart’ may be used uninitialized in this function [-Wmaybe-uninitialized]
LogPrintf(" block index %15dms\n", GetTimeMillis() - nStart);
```
It's ok without this PR, but this PR renames `nStart` to `load_block_index_start_time`, makes it `const`, and also reduces the scope of the variable.
The logging line is moved such that the the time spent will be logged even if a shutdown is requested while the index is being loaded.
Having the log message output even when a shutdown is requested may be how this was intended to work before anyways. That could explain the leading space, as such a log message now looks like:
```
2018-06-30T11:34:05Z [0%]...[16%]...[33%]...[50%]... block index 25750ms
2018-06-30T11:34:17Z Shutdown requested. Exiting.
```
Tree-SHA512: 967048afbc31f2ce8f80ae7d33fee0bdcbe94550cf2b5b662087e2a7cff14a8bf43d909b30f930660c184ec6c3c7e1302a84e3e54fc1723f7412827f4bf2c518
e3aab295e [tests] p2p_segwit: sync_blocks in subtest wrapper. (John Newbery)
55e805085 [tests] p2p_segwit: remove unnecessary arguments from subtests. (John Newbery)
25711c269 [tests] p2p_segwit: log and assert segwit status in subtest wrapper. (John Newbery)
6839863d5 [tests] p2p_segwit: Make sure each subtest leaves utxos for the next. (John Newbery)
bfe32734d [tests] p2p_segwit: wrap subtests with subtest wrapper. (John Newbery)
2af4e398d [tests] p2p_segwit: re-order function definitions. (John Newbery)
94a0134a4 [tests] p2p_segwit: standardise comments/docstrings. (John Newbery)
f7c7f8ecf [tests] p2p_segwit: Fix flake8 warnings. (John Newbery)
Pull request description:
`p2p_segwit.py` is a very long test, composed of multiple subtests. When it fails it's difficult to debug for a couple of reasons:
- Control flow jumps between different methods in the test class, so it's a little difficult to follow the code.
- state may be carried forward unintentionally from one subtest to the next.
Improve that by wrapping the subtests with a `@subtest` decorator which:
- logs progress
- asserts state after each subtest
As usual, I've also included a few commits which generally tidy up the test and improve style.
Tree-SHA512: 3650602b3ce9823dc968cc5f2e716757feadc3dbedb3605eb79bb3df91a6db8ae53431f253b440da690e3a8e9d76de84fad4368a2663aeb40e6b9427cf948870
b81560029 Remove CombineSignatures and replace tests (Andrew Chow)
ed94c8b55 Replace CombineSignatures with ProduceSignature (Andrew Chow)
0422beb9b Make SignatureData able to store signatures and scripts (Andrew Chow)
b6edb4f5e Inline Sign1 and SignN (Andrew Chow)
Pull request description:
Currently CombineSignatures is used to create the final scriptSig or an input. However ProduceSignature is capable of doing this itself. Using both CombineSignatures and ProduceSignature results in code duplication which is unnecessary.
To move the scriptSig construction to ProduceSignatures, the SignatureData class contains two maps to hold pubkeys mapped to signatures, and script ids mapped to scripts. DataFromTransaction is extended to be able to extract signatures, their public keys, and scripts from existing ScriptSigs.
The SignaureData are then passed down to SignStep which can use the aforementioned maps to get the signatures, pubkeys, and scripts that it needs, falling back to the actual SigningProvider and SignatureCreator if the data are not available in the SignatureData.
Additionally, Sign1 and SignN have been removed and their functionality inlined into SignStep since Sign1 is really just a wrapper around CreateSig.
Since ProduceSignature can produce the final scriptSig or scriptWitness by using SignatureData which has extracted data from the transaction, CombineSignatures is unnecessary as ProduceSignature is able to replicate all of CombineSignatures' functionality.
This also furthers BIP 174 support and begins moving towards a BIP 174 style backend.
The tests have also been updated to use the new combining methodology.
Tree-SHA512: 78cd58a4ebe37f79229bd5eee2958a0bb45cd7f36d0e993eee13ff685b3665dd76ef2dfd5f47d34678995bb587f5594100ee5f6c09b1c69ee96d3684d470d01e
49d1f4cdd Detect if char equals int8_t (Chun Kuan Lee)
Pull request description:
Probably fixes#13576. I'm not able to test this. @stacepellegrino, can you test this?
Tree-SHA512: b750e00e11e6b6f6341fec668ec2254cc101c8ebdd4878f320d6cb3b07cf326761146e4ceff0b6405b7e503ff64c093a8274bd524a097e2c49382dc296972c4f
1. PrevectorClear()
2nd call of clear() should to operate t1 instead of t0.
This patch changes t0 to t1.
2. PREVECTOR_TEST()
PREVECTOR_TEST macro should to call both
PrevectorXX<nontrivial_t>(state) and PrevectorXX<trivial_t>(state)
by specific "name" which given by parameter instead of calling
PrevectorResize<>() regardless of "name".
This patch changes "PrevectorResize<" of this macro to
"Prevector ## name<".
This adds a new RPC method "getzmqnotifications", which returns
information about all active ZMQ notification endpoints. This is useful
for software that layers on top of bitcoind, so it can verify that
ZeroMQ is enabled and also figure out where it should listen.
See https://github.com/bitcoin/bitcoin/issues/13526.
b330c3001 Docs: Improve doc of options addnode, connect, seednode (wodry)
Pull request description:
Just clarify that options `addnode`, `connect` and `seednode` can be specified multiple times.
Tree-SHA512: ed149cabe7fc1d40f2fb6ad8b643656e0ec49cfae1834c157c89170eac1241efa3c5683d97266ff921f5229f28d732c9f7ee030e7902d9a79db1e0c8716fa3db
1fabd59e7 Break circular dependency: init -> * -> init by extracting shutdown.h (Ben Woosley)
e62fdfeea Drop unused init.h includes (Ben Woosley)
Pull request description:
Most includers just wanted to react to pending shutdown.
This isolates access to `fRequestShutdown` and limits access to the shutdown api functions, including the new `CancelShutdown` for setting it to `false`.
Tree-SHA512: df42f75dfbba163576710e9a67cf1228531fd99d70a2f187bfba0bcc476d6749cf88180a97e66a81bb5b6c3c7f0917de7402d26039ba7b644cb7509b02f7e267
747ee89b4 Docs: Modify policy to not translate command-line help (Chun Kuan Lee)
Pull request description:
After #13341, command-line helps would no longer be translated.
Tree-SHA512: 73e7a7545b316cc6ec07f2527cc0e63c922d8e382abd779638583d55a4ee0f3b26ecc9d63f51da8bab2eda9829ec5d1b3162a1115c78a1add2621ba2ccdb839e
07c493f2d scripted-diff: Replace NET_TOR with NET_ONION (wodry)
Pull request description:
This is a follow-up to #13532, where @promag already asked if this renaming would make sense.
If network shall be named _Onion_ instead of _Tor_ (like in the option `onlynet`), renaming the network enum NET_TOR to NET_ONION maybe would make sense and be stringent.
Change was produced with the following script:
```
#!/bin/bash
for file in $(grep --exclude-dir='.git' --files-with-matches --binary-files=without-match --recursive NET_TOR bitcoin/)
do
sed --in-place --expression='s/NET_TOR/NET_ONION/g' $file
done
```
_Tor_ is used at many other places in the code, though.
Tree-SHA512: 4ffdeca8115031465eb64e1c76694fb77b5900c4ea465d3c13d9b6b75a1eb04c45913f83cdc8bdbef28936aeec4655f1d4905b3b98407da3263632a2128a8d23
bb582a59c Add P2WSH destination helper and use it instead of manual hashing (Pieter Wuille)
eaba1c111 Add additional unit tests for invalid IsMine combinations (Pieter Wuille)
e6b9730c4 Do not expose invalidity from IsMine (Pieter Wuille)
Pull request description:
This improves the handling of INVALID in IsMine:
* Extra INVALID conditions were added to `IsMine` (following https://github.com/bitcoin/bitcoin/pull/13142/files#r185349057), but these were untested. Add unit tests for them.
* In https://github.com/bitcoin/bitcoin/pull/13142#issuecomment-386396975 it was suggested to merge `isInvalid` into the return status. This PR takes a different approach, and removes the `isInvalid` entirely. It was only ever used inside tests, as normal users of IsMine don't care about the reason for non-mine-ness, only whether it is or not. As the unit tests are extensive enough, it seems sufficient to have a black box text (with tests for both compressed and uncompressed keys).
Some addition code simplification is done as well.
Tree-SHA512: 3267f8846f3fa4e994f57504b155b0e1bbdf13808c4c04dab7c6886c2c0b88716169cee9c5b350513297e0ca2a00812e3401acf30ac9cde5d892f9fb59ad7fef
Instead of using CombineSignatures to create the final scriptSig or
scriptWitness of an input, use ProduceSignature itself.
To allow for ProduceSignature to place signatures, pubkeys, and scripts
that it does not know about, we pass down the SignatureData to SignStep
which pulls out the information that it needs from the SignatureData.
In addition to having the scriptSig and scriptWitness, have SignatureData
also be able to store just the signatures (pubkeys mapped to sigs) and
scripts (script ids mapped to scripts).
Also have DataFromTransaction be able to extract signatures and scripts
from the scriptSig and scriptWitness of an input to put them in SignatureData.
Adds a new SignatureChecker which takes a SignatureData and puts pubkeys
and signatures into it when it successfully verifies a signature.
Adds a new field in SignatureData which stores whether the SignatureData
was complete. This allows us to also update the scriptSig and
scriptWitness to the final one when updating a SignatureData with another
one.
-BEGIN VERIFY SCRIPT-
sed --in-place'' --expression='s/NET_TOR/NET_ONION/g' $(git grep -I --files-with-matches 'NET_TOR')
-END VERIFY SCRIPT-
The --in-place'' hack is required for sed on macOS to edit files in-place without passing a backup extension.
