fa013664ae util: Add type safe GetTime (MarcoFalke)
Pull request description:
There are basically two ways to get the time in Bitcoin Core:
* get the system time (via `GetSystemTimeInSeconds` or `GetTime{Millis,Micros}`)
* get the mockable time (via `GetTime`)
Both return the same type (a plain int). This can lead to (test-only) bugs such as 99464bc38e.
Fix that by deprecating `GetTime` and adding a `GetTime<>` that returns the mockable time in a non-int type. The new util function is currently unused, but new code should it where possible.
ACKs for commit fa0136:
promag:
utACK fa013664.
Tree-SHA512: efab9c463f079fd8fd3030c479637c7b1e8be567a881234bd0f555c8f87e518e3b43ef2466128103db8fc40295aaf24e87ad76d91f338c631246fc703477e95c
c995c870aa depends: add ability to skip building zeromq (fanquake)
Pull request description:
Similar to other depends packages, add the ability to skip building `zeromq` by passing `NO_ZMQ=1`.
Fixes#15918.
ACKs for commit c995c8:
practicalswift:
utACK c995c870aa
jonasschnelli:
utACK c995c870aa
Tree-SHA512: 72269707916d5af0bc8ecdd89f61e49264dba29350f9508fe0a497e8ce8dae66f6a828cf0bf4d97b6f95356b505cb3e6c365e8476219dd56c4535c850df393c9
fadbc5d895 mempool: remove unused magic number from consistency check (Gregory Sanders)
Pull request description:
Unexplained magic numbers are no good. Since the exact number does not matter, opt for a constant that is less peculiar.
Note that this could only possibly affect mempool consistency checks which is not active by default except on regtest.
see discussion: https://github.com/bitcoin/bitcoin/issues/15080
ACKs for commit fadbc5:
practicalswift:
utACK fadbc5d895
Tree-SHA512: 80f95ebc284c5bcc5d825fab0e9f962457a411539946d68ef4c8bdea4b1f2f7f0ead88928fac0eaaa02a1175f01f5ef381613ce53b0f27c3098e90d76ecfe9af
d5dc66e280 doc: fix/improve analyzepsbt in doc/psbt.md (Jon Atack)
Pull request description:
- fix: replace "RPC" with "PSBT"
- output includes the current status of the analyzed psbt's inputs
- apply "if possible" to the fee as well as to the estimated weight and feerate, since the fee is only shown if all utxo slots in the psbt have been filled
- add "final" to the estimated weight and feerate
ACKs for commit d5dc66:
laanwj:
ACK d5dc66e280
fanquake:
utACK d5dc66e
Tree-SHA512: 61ff1ef45ec34182613b300d21cc2b17a28d1e955f70848f5be1a40c82009fe3000db3332d2cfca1833d7c881b61cc4ebc9fc779238f76d38e9e3f706cfb3551
0db94e55d wallet: Pass WalletBatch to CWallet::UnsetWalletFlag (João Barbosa)
6cb888b37 Apply the batch treatment to CWallet::SetAddressBook via ImportScriptPubKeys (Ben Woosley)
6154a09e0 Move some of ProcessImport into CWallet::Import* (Ben Woosley)
ccb26cf34 Batch writes for importmulti (Andrew Chow)
d6576e349 Have WalletBatch automatically flush every 1000 updates (Andrew Chow)
366fe0be0 Add AddWatchOnlyWithDB, AddKeyOriginWithDB, AddCScriptWithDB functions (Andrew Chow)
Pull request description:
Instead of writing each item to the wallet database individually, do them in batches so that the import runs faster.
This was tested by importing a ranged descriptor for 10,000 keys.
Current master
```
$ time src/bitcoin-cli -regtest -rpcwallet=importbig importmulti '[{"desc": "sh(wpkh([73111820/44h/1h/0h]tpubDDoT2SgEjaU5rerQpfcRDWPAcwyZ5g7xxHgVAfPwidgPDKVjm89d6jJ8AQotp35Np3m6VaysfUY1C2g68wFqUmraGbzhSsMF9YBuTGxpBaW/1/*))#3w7php47", "range": [0, 10000], "timestamp": "now", "internal": true, "keypool": false, "watchonly": true}]'
...
real 7m45.29s
```
This PR:
```
$ time src/bitcoin-cli -regtest -rpcwallet=importbig4 importmulti '[{"desc": "pkh([73111820/44h/1h/0h]tpubDDoT2SgEjaU5rerQpfcRDWPAcwyZ5g7xxHgVAfPwidgPDKVjm89d6jJ8AQotp35Np3m6VaysfUY1C2g68wFqUmraGbzhSsMF9YBuTGxpBaW/1/*)#v65yjgmc", "range": [0, 10000], "timestamp": "now", "internal": true, "keypool": false, "watchonly": true}]'
...
real 3.93s
```
Fixes#15739
ACKs for commit 0db94e:
jb55:
utACK 0db94e5
ariard:
Tested ACK 0db94e5
Empact:
re-utACK 0db94e55dc only change is re the privacy of `UnsetWalletFlagWithDB` and `AddCScriptWithDB`.
Tree-SHA512: 3481308a64c99b6129f7bd328113dc291fe58743464628931feaebdef0e6ec770ddd5c19e4f9fbc1249a200acb04aaf62a8d914d53b0a29ac1e557576659c0cc
97dce72261 contrib: add curl as a required program in gitian-build.py (fanquake)
Pull request description:
Fixes: #16109
Adds `curl` to the list of base programs required by the `gitian-build.py` script.
ACKs for commit 97dce7:
hebasto:
tACK 97dce72261 on Debian Buster RC1.
Tree-SHA512: 68847a527aa6b5d883bffd6a6fe6bbbe4b96ceddb30f55ed5ffbfa690a10c2e9c1bc7ba4520319531ab3baa7a7f64c3c8ce89a791f7c746abe73a84c2942b94d
ae7faf20d5 Exceptions should be caught by reference, not by value. (Kristaps Kaupe)
Pull request description:
Fixes this warning with GCC8/GCC9:
```
wallet/wallettool.cpp: In function ‘std::shared_ptr<CWallet> WalletTool::LoadWallet(const string&, const boost::filesystem::path&)’:
wallet/wallettool.cpp:62:25: warning: catching polymorphic type ‘const class std::runtime_error’ by value [-Wcatch-value=]
} catch (const std::runtime_error) {
^~~~~~~~~~~~~
```
Related to #15822.
ACKs for commit ae7faf:
practicalswift:
utACK ae7faf20d5
Tree-SHA512: 07eb774b3296c0b66ac5040269bff6cd8ba0294c8c95cc08c595efbd535260ff0010fa430ca057eeccd7b38c0a981a3d7a95b675d9e2996853c013dc0bfe8127
fb434159d1 Remove global symbols: Avoid using the global namespace if possible (practicalswift)
Pull request description:
Remove global symbols: Avoid using the global namespace if possible.
Partially resolves#15612 ("Reduce the number of global symbols used").
Change in global symbols as reported by `nm bitcoind` before vs after:
```
$ diff -u <(nm src/bitcoind-before | c++filt | grep -E '^[0-9a-f]+ [A-Z] ' | cut -f3- -d' ' | sort -u) \
<(nm src/bitcoind-after | c++filt | grep -E '^[0-9a-f]+ [A-Z] ' | cut -f3- -d' ' | sort -u) \
| grep -E '^[+-][^+-]'
-boundSockets
-cs_warnings
-eventHTTP
-fFeeEstimatesInitialized
-fLargeWorkForkFound
-fLargeWorkInvalidChainFound
-pathHandlers
-strMiscWarning[abi:cxx11]
-threadHTTP
```
ACKs for commit fb4341:
Tree-SHA512: d2f78f6188a992b0e0de8d107e2c494cfa0faa2de4fda634a1d3606d6515633bec86289cf2a2e78ffe467b17b795e2243cc459fb44e0dfe2fc69899506ff61c9
480e3415d7 configure: Add flag for enabling thread_local. (Carl Dong)
Pull request description:
- When aiming for glibc compatibility, don't use thread_local. Fixes#15958.
- FreeBSD has a buggy thread_local, don't use it. Fixes#16055.
I've done a Gitian build on my local machine and the symbol tests seem to pass.
ACKs for commit 480e34:
MarcoFalke:
utACK 480e3415d7
fanquake:
tACK 480e341
Tree-SHA512: 334f21f7cf271c261b115a6410afd4ed4db3e84ad79b98c6c684c1dfa42b081f16d58e77695929e27b0fa173a894b959a327fe82821a3f3ed708b305a906ddd3
fa47330397 test: Speed up cache creation (MarcoFalke)
fa6ad7a5ec test: Bump MAX_NODES to 12 (MarcoFalke)
Pull request description:
When testing a combination of settings that affect the datadir (e.g. prune, blockfilter, ...) we may need a lot of datadirs.
Bump the maximum number of nodes proactively from 8 to 12, so that caches get populated with 12 node dirs, as opposed to 8.
Also, add an assert that the list of deterministic keys is exactly the number of max nodes (and not more than that.
Also, create the cache faster.
ACKs for commit fa4733:
laanwj:
utACK fa47330397
Tree-SHA512: 9803c765ed52d344102f5a3bce57b05d88a7429dcb05ed66ed6c881fda8d87c2834d02d21b95fe9f39c0efe3b8527e13cf94f006588cde22e8c2cd50b2d517a6
0784af16ef remove parameters -addresstype=legacy in rpc_rawtransaction test (LongShao007)
a65dafa8f1 replace tx hash with txid in test rawtransaction (LongShao007)
Pull request description:
The transaction hash is different from txid for witness transactions, so we should use txid instead of hash.
ACKs for commit 0784af:
Tree-SHA512: 98b699eb5f25c3a603b11eb7072efe9bc69c0c0ecc7f996405de31bc45d92105970e09fd8e4f75b42a46498817f596d36d9b28eae7d24e63a4f2f2abfcee0eab
8afca323e3 doc: add bitcoin_config.h PACKAGE updates to release process (Jon Atack)
3ee28c506d build: bump bitcoin_config.h packages to v0.18 (Jon Atack)
Pull request description:
- Bump PACKAGE_VERSION and PACKAGE_STRING in `build_msvc/bitcoin_config.h` from 0.17 to 0.18 (follow-up to 48ed65b).
- Update `doc/release_process.md` (follow-up to e47dc4f), new version visible [here](https://github.com/jonatack/bitcoin/blob/bitcoin_config-and-release_process-updates/doc/release-process.md).
- Perhaps worth backporting the version updates to 0.18.0.
ACKs for commit 8afca3:
laanwj:
utACK 8afca323e3
Tree-SHA512: be4308636846d719d2406790b33861a5ca4775cec77b7b40f2a01e0180d55e36d821b680c923c366de6ddb576f8a94efe59bf66a5f0637cbc2ecff6c824fe602
- When aiming for glibc compatibility, don't use thread_local.
- Add a flag --enable-threadlocal, which, when specified, will
enable/disable thread_local regardless of the value of glibc_compat.
- FreeBSD has a buggy thread_local, don't use it.
9f85e9cb3d scripted-diff: Rename LockAnnotation to LockAssertion (practicalswift)
de9b5dbca3 Make sure the compile-time locking promises given via LockAnnotation:s hold also in practice at runtime (ifdef DEBUG_LOCKORDER) (practicalswift)
3a809446b3 Move LockAnnotation to make it reflect the truth (practicalswift)
cc2588579c Move LockAnnotation from threadsafety.h (imported code) to sync.h (our code) (practicalswift)
Pull request description:
`LockAnnotation lock(mutex);` is a guarantee to the compiler thread-analysis that `mutex` is locked (when it couldn't be determined otherwise).
Before this PR it was possible to make the mistake of adding a `LockAnnotation` where the correct mutex is _not_ held. This in turn makes the thread-analysis reasoning being based on incorrect premises.
This PR adds an assertion in the `LockAnnotation` ctor which checks that the guarantees given by us at compile-time are held also in practice (`ifdef DEBUG_LOCKORDER`).
Issues like the one described in #16028 will be discovered immediately with this PR merged.
Changes in this PR:
* Move `LockAnnotation` from `threadsafety.h` (imported code) to `sync.h` (our code)
* Move `LockAnnotation` in `wallet_tests` to make it reflect the truth
* Make sure the compile-time locking promises given via `LockAnnotation`:s hold also in practice at runtime (`ifdef DEBUG_LOCKORDER`)
* Rename `LockAnnotation` to `LockAssertion`
ACKs for commit 9f85e9:
ryanofsky:
utACK 9f85e9cb3d. No changes at all since last review except clean rebase after base PR #16033 was merged
Tree-SHA512: fb80e78fe362adfd6ea8405bcb142c09b99f834fe8be4397282b223ca2c3a2bb9719a074a47a043b44757f840b239a6fcd2f98d14771f8729204834ecf608c3a
e23809a05b [rpc] deriveaddresses: Correct descriptor checksum in RPCExamples (Chris Capobianco)
Pull request description:
Trvial: This fixes the descriptor checksum found the in the deriveaddresses RPC example.
The current checksum value does work, but only if the "h" used for the hardened derivation key origin are replaced with "'".
Given the discussion to switch from "'" to "h" [here](https://github.com/bitcoin/bitcoin/issues/15740), I thought it made more sense to update the checksum rather then changing all the "h" to "'" in this example.
ACKs for commit e23809:
instagibbs:
tACK e23809a05b
Tree-SHA512: 06a2b9f3e714ecde9b9a80b3b7a4082eb072e71d8abcc455ff5387e470d48839f22a70b78bbae1cf9122cb133fee46830819b6f39d67aec8c3c8d5889ae94e04
bb41e632ca wallet_balance.py: Prevent edge cases (Steven Roose)
Pull request description:
I ran into this edge case when running the test on Elements. I had a 0-value output as change.
ACKs for commit bb41e6:
Tree-SHA512: ef4c25289cafcdb4437f11ed537664dff5afedcefab75a46f985d3be70551de2d3bc8e9cfcb22c0f3d7d2eb95ff40df78b8d01dbacbf90c36bca00426937b0a2
1609809fb2 validation: Hold cs_main when reading chainActive in RewindBlockIndex (practicalswift)
Pull request description:
Fixes#15980.
Hold `cs_main` when reading `chainActive` (via `::ChainActive()`) in `RewindBlockIndex`.
ACKs for commit 160980:
MarcoFalke:
utACK 1609809fb2
Tree-SHA512: 54f180ab391f92f04950735c2bb337f0b7495826d2096f7a0f9a2da50bc29d08747f404a0495e33ca4edd4c842efbab4c4730d5e1a8b9da3e1249cf884268f4b
d8bc47fde4 depends: switch to secure download of all dependencies (Ulrich Kempken)
Pull request description:
Even if we potentially check the integrity of the downloaded file via hash comparison, we should make use of SSL since it is available.
ACKs for commit d8bc47:
jonasschnelli:
utACK d8bc47fde4
practicalswift:
utACK d8bc47fde4
dongcarl:
tACK d8bc47fde4
Tree-SHA512: e47702f6d243ed7f498ca84c193244382f16f08df6a297caa224b4468f501f3da6fe542fcf3a0dd9c24ab1b0b38bbc51478068e6006a92854ded23abf90de3c8
df0e97ccb1 RPC: Hint for importmulti in help output of importpubkey and importaddress (Kristaps Kaupe)
Pull request description:
Similar to #12702. Hint for `importmulti` also in help output of `importpubkey` and `importaddress`.
ACKs for commit df0e97:
promag:
utACK df0e97ccb1.
jonasschnelli:
utACK df0e97ccb1
Tree-SHA512: db7358d7f4d463a50874e605bbca35a1a40dbefbb1d35cf51fe2f2aa34bef90c3ca398f4ffbcb9d7d43887a03eb8d81b6ef59066a3c7eda18a7eea876f6592e7
2c448d6bc7 parameterize hard coded numbers referring to miner conf window (Jordan Baczuk)
Pull request description:
Replace hard coded values (eg. 2016) with `mainnetParams.nMinerConfirmationWindow` where appropriate. This parameterizes hard coded values in the unit test that refer to the `Miner Confirmation Window`, which currently is `2016`. This includes values not exactly 2016 but which were derived from it. Also changed `int` to `uint32_t` where appropriate to avoid compiler warnings. This makes one source of truth, and also helps people who might be adjusting this value in testing so the unit tests don't break.
ACKs for commit 2c448d:
Tree-SHA512: 9262e0b89c1baf7857b49fe2221b2b00f948f61317b321c4871a9182a86d6f8aadeb59d6b133e8a213cc9b31b4a417888fb1ad31caef16ccbbab1de33c4b8459
Some dependency sources were downloaded via http, even though https (SSL/TLS) options are available.
Even if we potentially check the integrity of the downloaded files via hash comparison, we should make
use of this additional security layer.
bdb.mk
fontconfig.mk
freetype.mk
libX11.mk
libXau.mk
libXext.mk
libxcb.mk
native_cctools.mk
native_cdrkit.mk
xcb_proto.mk
xextproto.mk
xproto.mk
xtrans.mk
zlib.mk
miniupnp was switched to official project mirror with SSL support
e0eae1b4a4 Make --setup command independent (Hennadii Stepanov)
Pull request description:
This PR allows a user to run:
```sh
./gitian-build.py --setup
```
without unused `signer` and `version` options.
In master the `signer` and `version` options are mandatory. This implies the following code is dead:
387eb5b343/contrib/gitian-build.py (L192-L200)
This PR fixes those lines of code.
Also this PR has a nice side effect: there is no more warnings about macOS build during processing `--setup` command. Ref: https://github.com/bitcoin/bitcoin/pull/13998#issuecomment-493691117
Note: https://github.com/bitcoin-core/docs/blob/master/gitian-building.md will be updated when this PR is merged.
ACKs for commit e0eae1:
Tree-SHA512: df851fe461e402229c57b410f30f1d8bc816e8a2600ece4249aa39c763566de5b661e7aa0af171d484727eb463a6d0e10cfcf459aa60ae1a5d4e12974a8615c6
0f22a0cf2f Fix gitian-build.py --verify option (Hennadii Stepanov)
4c56a798c0 Set/unset USE_LXC, USE_VBOX, USE_DOCKER explicitly (Hennadii Stepanov)
cbbd98863b Fix Docker related issues for gitian-build.py (Hennadii Stepanov)
Pull request description:
1. The Docker does not depend on `apt-cacher-ng` package. Ref: #14002.
2. Do not try to install the Docker if `docker.service` is detected on the system (e.g., the Docker was installed manually). Fix https://github.com/bitcoin/bitcoin/pull/13623#issuecomment-405684241 by **Sjors**.
3. Prevent the setting of more than one environment variable for the `gitian-builder` (an alternative to #13999). E.g., USE_LXC being set shadows USE_DOCKER; for details see [`gitian-builder/libexec/make-clean-vm`](93a62c7d7d/libexec/make-clean-vm (L7)):
```sh
VMSW=KVM
if [ -n "$USE_LXC" ]; then
VMSW=LXC
elif [ -n "$USE_VBOX" ]; then
VMSW=VBOX
elif [ -n "$USE_DOCKER" ]; then
VMSW=DOCKER
fi
```
4. The [`gitian-builder/bin/gverify`](https://github.com/devrandom/gitian-builder/blob/master/bin/gverify) script returns the exit code 1 if a signature verification ends with 'BAD SIGNATURE' or 'MISMATCH' by design. This PR allows to see the verification results for all signatures without a premature fail of the `gitian-build.py` script. Ref: #14014.
ACKs for commit 0f22a0:
Tree-SHA512: 55f8a5cffa20d0c745f51a687f3199cea015fa616e56a0aee4c25b5ca0985036c61e8cf1922515338d8c6a85f873674ebe7a9a56a5069d65a187e383150f1a83
fa2b52af32 Disallow extended encoding for non-witness transactions (take 3) (MarcoFalke)
Pull request description:
(previous title "p2p: Disallow extended encoding for non-witness transactions (take 3)")
Remote peers can send us illegally encoded txs and thus have us write to stderr. Fix that by not writing to stderr.
This is a follow up to the previous (incomplete) attempts at this:
* Disallow extended encoding for non-witness transactions #14039
* Add test for superfluous witness record in deserialization #15893
ACKs for commit fa2b52:
laanwj:
utACK fa2b52af32
ryanofsky:
utACK fa2b52af32. Would change title to something like "Avoid logging transaction decode errors to stderr" instead of "Disallow extended encoding for non-witness transactions." The current title is confusing because this PR isn't really allowing or disallowing anything, just logging the condition differently. "Disallow" also seems to contradict the "Allow exceptions from..." comments in the actual code.
Tree-SHA512: c66990e69b432d00dc1c5510bf976a1188664d0890a32d1e5c6459094e7e27da82a5d227627afcbc203676f5540eec74b7d9b1d71d2c62d3b2069e1781824b4d
4de3c15671 depends: add patch to common dependencies (fanquake)
Pull request description:
Building on a bare system:
```
/bitcoin/depends/work/download/zeromq-4.3.1/zeromq-4.3.1.tar.gz.temp: OK
Extracting zeromq...
/bitcoin/depends/sources/zeromq-4.3.1.tar.gz: OK
Preprocessing zeromq...
/bin/sh: 1: patch: not found
```
ACKs for commit 4de3c1:
practicalswift:
utACK 4de3c15671
Tree-SHA512: d1a7b6b591e9de395a3bc54d9df9f97adff5f0a8b5f7a35792c27f49a610543216b2a3f3470f1e3c7dff51276e560d77d123a6d20871b0ed3e5a83da3495c5f2
