Commit graph

20841 commits

Author SHA1 Message Date
Pieter Wuille
3135c1a2d2 Abstract out UpdatePSBTOutput from FillPSBT 2019-05-10 14:31:22 -07:00
Pieter Wuille
fb90ec3c33 Abstract out EvalDescriptorStringOrObject from scantxoutset 2019-05-10 14:31:21 -07:00
Pieter Wuille
eaf4f88734 Abstract out IsSegWitOutput from utxoupdatepsbt
This is not a pure refactor; additional functionality is added in
IsSegWitOutput which lets it recurse into P2SH when a
SigningProvider is provided that knows about the inner script.
2019-05-10 14:22:33 -07:00
MarcoFalke
e79bbb73e0
Merge #15607: [Docs] Release process updates
bd63c1ed12 [docs] Update release-notes.md (Jon Atack)
96d32a7bc0 [docs] Update release-process.md (Jon Atack)

Pull request description:

  Updates to `release-notes.md`:

  - Write an introduction explaining how to use `release-notes.md` as a template for the release notes draft wiki for collaborative editing at https://github.com/bitcoin-core/bitcoin-devwiki/wiki, as seen for the 0.17.0 and 0.18.0 releases.

  Updates to `release-process.md`:

  - Create a release notes draft wiki at https://github.com/bitcoin-core/bitcoin-devwiki/wiki.

  - As per http://www.erisian.com.au/bitcoin-core-dev/log-2019-03-28.html#l-342, for the period during which the release notes are being edited on the wiki, the version on the branch should be wiped and replaced with a link to the wiki which should be used for all announcements until final.

  - Before -final, remove the "Needs release note" label from relevant PRs/issues and merge the release notes from the wiki into the branch.

  - Create a pinned meta-issue dedicated to testing the release candidate and communicate it in release announcements where useful. The former is done in practice (e.g. https://github.com/bitcoin/bitcoin/issues/15555, https://github.com/bitcoin/bitcoin/issues/14902) and the latter addresses the discussion here yesterday: https://x0f.org/web/statuses/101753569204220416.

  - Reorganise the headers in the Branch Updates section.

  - Update the version numbers in the examples.

  - Adapt and merge in the updates in https://github.com/bitcoin/bitcoin/pull/15692.

ACKs for commit bd63c1:

Tree-SHA512: a436d6d0971bf00f081d4e2660c3b37a0f96913236c33934453387d63c11556a80e428b110f5629f07a6aa98e627f86c799d10c3f40e9f0c27b22275e04f6fa8
2019-05-10 15:58:32 -04:00
Jon Atack
bd63c1ed12
[docs] Update release-notes.md
- Write an introduction explaining how to use the release-notes.md template.

- Adapt and merge the updates in https://github.com/bitcoin/bitcoin/pull/15692.
2019-05-10 20:35:33 +02:00
Jon Atack
96d32a7bc0
[docs] Update release-process.md
- Create a release notes draft wiki for collaborative editing at https://github.com/bitcoin-core/bitcoin-devwiki/wiki as seen for releases 0.17.0 and 0.18.0.

- As per http://www.erisian.com.au/bitcoin-core-dev/log-2019-03-28.html#l-342, for the period during which the notes are being edited on the wiki, the version on the branch should be wiped and replaced with a link to the wiki which should be used for all announcements until final.

- Before final, remove the "Needs release note" label from relevant PRs/issues and merge the release notes from the wiki into the branch.

- Create a pinned meta-issue dedicated to testing the release candidate and communicate it in release announcements where useful. The former is done in practice (e.g. #15555, #14902) and the latter addresses the discussion here: https://x0f.org/web/statuses/101753569204220416.

- Adapt and merge the updates in https://github.com/bitcoin/bitcoin/pull/15692.

- Update the version numbers in all the examples.

- Reorganise the headers in the Branch Updates section.
2019-05-10 20:35:30 +02:00
MarcoFalke
e2371f842f
Merge #14802: rpc: faster getblockstats using BlockUndo data
d20d756752 rpc: faster getblockstats using BlockUndo data (Felix Weis)

Pull request description:

  Using undo data for a block (rev?????.dat) we can retrieve value information about prevouts and calculate the final transaction fee (rate). This approach is about 80x faster, drops the requirement for `-txindex`, and works for all non-pruned blocks.

  ```
  # 2018-11-25T16:36:19Z Bitcoin Core version v0.17.99.0-edc715240-dirty (release build)
  seq 550100 550200  0.00s user 0.00s system 62% cpu 0.004 total
  xargs -n1 src/bitcoin-cli getblockstats  0.21s user 0.19s system 17% cpu 2.302 total

  # 2018-11-25T16:39:17Z Bitcoin Core version v0.17.0 (release build)
  seq 550100 550200  0.00s user 0.00s system 87% cpu 0.002 total
  xargs -n1 src/bitcoin-cli getblockstats  0.24s user 0.22s system 0% cpu 3:19.42 total
  ```

ACKs for commit d20d75:
  MarcoFalke:
    re-utACK d20d756752

Tree-SHA512: 5babc3eb8d2fee2cb23dc12f522656b80737a540cbf2b13390a8f388304c46c064cca76f896b46a6e2abae8cc582d28e1ab20dd4bb17ad6142f20630c2d30c54
2019-05-10 13:20:48 -04:00
Felix Weis
d20d756752 rpc: faster getblockstats using BlockUndo data
Using undo data for a block (rev?????.dat) we can retrieve value information about prevouts and calculate the final transaction fee (rate). This approach is about 80x faster, drops the requirement for -txindex, and works for all non-pruned blocks.
2019-05-10 08:33:14 -04:00
MarcoFalke
14959753a4
Merge #15744: refactor: Extract ParseDescriptorRange
510c6532ba Extract ParseDescriptorRange (Ben Woosley)

Pull request description:

  So as to be consistently informative when the checks fail, and
  to protect against unintentional divergence among the checks.

ACKs for commit 510c65:
  meshcollider:
    Oh apologies, yes. Thanks :) utACK 510c6532ba
  MarcoFalke:
    utACK 510c6532ba
  sipa:
    utACK 510c6532ba

Tree-SHA512: b1f0792bfaa163890a20654a0fc2c4c4a996659916bf5f4a495662436b39326692a1a0c825caafd859e48c05f5dd1865c4f7c28092be5074edda3c94f94f9f8b
2019-05-10 08:09:44 -04:00
MarcoFalke
87dbf89271
Merge #15239: scripts and tools: Move non-linux build source tarballs to "bitcoin-binaries/version" directory
5c04814b2d Move non-linux source tarball to bitcoin-binaries (Hennadii Stepanov)

Pull request description:

  Currently, if a user makes a non-linux (`--os=w`, `--os=m` or `--os=wm`) gitian building with the `gitian-build.py` script, source tarballs are not moved to the `bitcoin-binaries/${VERSION}` directory.

  This PR fixes this bug.

  ~~In addition, the `src` subdirectory in the `gitian-builder/build/out` directory is no longer used as unnecessary.~~

ACKs for commit 5c0481:
  fanquake:
    utACK 5c04814
  ken2812221:
    utACK 5c04814b2d

Tree-SHA512: 8648b6cbf502c012b12642783870e37aea385bd5f4cba5cb577fee924c09685e9a117676be502e4d4783c7a8ab31a2bd495970eec42a42d78e86ac5d39323091
2019-05-10 07:59:02 -04:00
MarcoFalke
fa3872e7b4
test: Format predicate source as multiline on error 2019-05-10 07:29:54 -04:00
Jonas Schnelli
695141bf7a
Merge #15512: Add ChaCha20 encryption option (XOR)
2dfe27517 Add ChaCha20 bench (Jonas Schnelli)
2bc2b8b49 Add ChaCha20 encryption option (XOR) (Jonas Schnelli)

Pull request description:

  The current ChaCha20 implementation does not support message encryption (it can only output the keystream which is sufficient for the RNG).

  This PR adds the actual XORing of the `plaintext` with the `keystream` in order to return the desired `ciphertext`.

  Required for v2 message transport protocol.

ACKs for commit 2dfe27:
  jnewbery:
    Looks good. utACK 2dfe275171.
  jnewbery:
    utACK 2dfe275171
  sipa:
    utACK 2dfe275171
  ryanofsky:
    utACK 2dfe275171. Changes since last review are just renaming the Crypt method, adding comments, and simplifying the benchmark.

Tree-SHA512: 84bb234da2ca9fdc44bc29a786d9dd215520f81245270c1aef801ef66b6091b7793e2eb38ad6dbb084925245065c5dce9e5582f2d0fa220ab3e182d43412d5b5
2019-05-10 09:26:23 +02:00
Pieter Wuille
0b09a57aec Give WalletModel::UnlockContext move semantics 2019-05-09 18:07:33 -07:00
Wladimir J. van der Laan
79046d5749
Merge #15939: gitian: Remove Windows 32 bit build
fa193dc8e6 doc: Remove win32 from the release process (MarcoFalke)
faf666f814 Remove Windows 32 bit build (MarcoFalke)

Pull request description:

  The Windows 32 bit build has been removed from https://bitcoincore.org/en/download/, so unless there are complaints, we don't need to build it even

ACKs for commit fa193d:
  fanquake:
    utACK fa193dc8e6

Tree-SHA512: d6f2976a2e0c407698f720b00ac23ec4056626de4eff8621f4c5581120af0460afd1bdef72329cc0e7d92afca48d94ae5fce6777cb36bfabb60b8034ff08fd88
2019-05-09 21:22:24 +02:00
Wladimir J. van der Laan
de5af41e35
Merge #15452: Replace CScriptID and CKeyID in CTxDestination with dedicated types
78e407ad0c GetKeyBirthTimes should return key ids, not destinations (Gregory Sanders)
70946e7fee Replace CScriptID and CKeyID in CTxDestination with dedicated types (Gregory Sanders)

Pull request description:

  The current usage seems to be an overloading of meanings. `CScriptID` is used in the wallet as a lookup key, as well as a destination, and `CKeyID` likewise. Instead, have all destinations be dedicated types.

  New types:
  `CScriptID`->`ScriptHash`
  `CKeyID`->`PKHash`

ACKs for commit 78e407:
  ryanofsky:
    utACK 78e407ad0c. Only changes are removing extra CScriptID()s and fixing the test case.
  Sjors:
    utACK 78e407a
  meshcollider:
    utACK 78e407ad0c

Tree-SHA512: 437f59fc3afb83a40540da3351507aef5aed44e3a7f15b01ddad6226854edeee762ff0b0ef336fe3654c4cd99a205cef175211de8b639abe1130c8a6313337b9
2019-05-09 18:54:43 +02:00
Wladimir J. van der Laan
aebe990dfe
Merge #15794: docs: Clarify PR guidelines w/re documentation
f4a230b627 docs: Clarify PR guidelines w/re documentation (Carl Dong)

Pull request description:

  PRs should change documentation accordingly when the behaviour of code changes.

ACKs for commit f4a230:
  practicalswift:
    ACK f4a230b627
  fanquake:
    utACK f4a230b

Tree-SHA512: 6d9d65d7f0f9bc8f324ee16f03169df28fb512c58bb71093128cf16797b25533cdc992bc8757034a99d6fa6423a3129ca7cf2ab2da857535f409a683486fd4ab
2019-05-09 18:04:07 +02:00
Wladimir J. van der Laan
a65fd92f7b
Merge #15766: scripts and tools: Upgrade gitian image before signing
beda0dae95 Upgrade gitian image before signing (Hennadii Stepanov)

Pull request description:

  The package upgrade in the Ubuntu repositories in the period between the building and the signing causes (particularly, using LXC) an error:
  ```
  Creating package manifest
  Could not download some packages, please run gbuild --upgrade
  ```

  For example, the [`busybox-initramfs`](https://packages.ubuntu.com/bionic/busybox-initramfs) package was [upgraded](http://changelogs.ubuntu.com/changelogs/pool/main/b/busybox/busybox_1.27.2-2ubuntu3.2/changelog) from `1:1.27.2-2ubuntu3.1` to `1:1.27.2-2ubuntu3.2` on 2019-03-06.

  This PR forces gitian image upgrade for the `--sign` command.

  Ref:
  [devrandom/gitian-builder/target-bin/grab-packages.sh](04ab7c1218/target-bin/grab-packages.sh)
  ```
  #!/bin/sh

  # Get an installed package manifest

  set -e

  cd /var/cache/apt/archives

  # make sure all packages with installed versions are downloaded
  # (except for held packages, which may not be available for download)
  dpkg-query -W -f '${Status}\t${Package}=${Version}\n' | grep -v ^hold | cut -f2- | xargs -n 50 apt-get install -q --reinstall -y -d > /tmp/download.log
  grep "cannot be downloaded" /tmp/download.log && { echo Could not download some packages, please run gbuild --upgrade 1>&2 ; exit 1 ; }
  sha256sum *.deb | sort --key 2
  ```

ACKs for commit beda0d:
  laanwj:
    utACK beda0dae95
  fanquake:
    utACK beda0da

Tree-SHA512: e2e3b3e3719e098d266ceec39bd69b950344a4eb2f43ae6ad3e696add70f743b363cc83676e339f7caa207d6478029869a8af01fe1f6d5690d2857003f7d8ce8
2019-05-09 17:29:44 +02:00
John Newbery
a407b6fdf3 [tests] Make random seed logged and settable
This allows tests which use randomness to be reproducibly run on failure.
2019-05-09 11:28:04 -04:00
MarcoFalke
fa1dce7329
net: Rename ::fRelayTxes to ::g_relay_txes
This helps to distinguish it from CNode::fRelayTxes and avoid bugs like
425278d17b
2019-05-09 09:10:53 -04:00
MarcoFalke
08788ce170
Merge #15890: Doc: remove text about txes always relayed from -whitelist
e0bb279999 Doc: remove text about txes always relayed from -whitelist (David A. Harding)

Pull request description:

  Updates text since -whitelistforcerelay was set to false by default in PR #15193.

ACKs for commit e0bb27:
  fanquake:
    utACK e0bb279
  MarcoFalke:
    utACK e0bb279999

Tree-SHA512: cf0c9321d72692d573039a04f8f1d048cbdf67ed86cc781523dabd3c45d2731b788f53749e6bb29d7da1ab44eb04030f352469b20489bb2a26c2c38fb61f6489
2019-05-08 15:59:42 -04:00
MarcoFalke
fa1d766717
tests: Make msg_block a witness block
This diff has been generated with the following script, but is better
reviewed without looking at the script.

 # -BEGIN VERIFY SCRIPT-
echo "Use msg_witness_block everywhere, except for tests that require msg_block"
 # This could be a separate commit, but it is combined with the
 # following scripts to reduce the overall diff
sed -i -e 's/msg_block/msg_witness_block/g' ./test/functional/{feature_assumevalid,feature_cltv,feature_dersig,feature_versionbits_warning,p2p_fingerprint,p2p_sendheaders,p2p_unrequested_blocks,example_test,rpc_blockchain}.py

echo "Rename msg_block to msg_no_witness_block"
 # Rename msg_block to msg_no_witness_block in all tests (not the
 # framework)
sed -i -e 's/msg_block/msg_no_witness_block/g' $(git grep -l msg_block ./test/functional/*.py)
 # Derive msg_no_witness_block from msg_block
 # Make msg_block a witness block in messages.py
patch -p1 --fuzz 0 << EOF
diff --git a/test/functional/test_framework/messages.py b/test/functional/test_framework/messages.py
index 00190e4cbd..e454ed5987 100755
--- a/test/functional/test_framework/messages.py
+++ b/test/functional/test_framework/messages.py
@@ -1133 +1133 @@ class msg_block:
-        return self.block.serialize(with_witness=False)
+        return self.block.serialize()
@@ -1155 +1155 @@ class msg_generic:
-class msg_witness_block(msg_block):
+class msg_no_witness_block(msg_block):
@@ -1158,2 +1158 @@ class msg_witness_block(msg_block):
-        r = self.block.serialize()
-        return r
+        return self.block.serialize(with_witness=False)
@@ -1445 +1444 @@ class msg_blocktxn:
-        r += self.block_transactions.serialize(with_witness=False)
+        r += self.block_transactions.serialize()
@@ -1452 +1451 @@ class msg_blocktxn:
-class msg_witness_blocktxn(msg_blocktxn):
+class msg_no_witness_blocktxn(msg_blocktxn):
@@ -1456,3 +1455 @@ class msg_witness_blocktxn(msg_blocktxn):
-        r = b""
-        r += self.block_transactions.serialize()
-        return r
+        return self.block_transactions.serialize(with_witness=False)
EOF
 # Conclude rename of msg_block to msg_no_witness_block
sed -i -e 's/msg_witness_block/msg_block/g' $(git grep -l msg_witness_block)
 # -END VERIFY SCRIPT-
2019-05-08 11:53:02 -04:00
MarcoFalke
fa193dc8e6
doc: Remove win32 from the release process 2019-05-08 11:48:18 -04:00
MarcoFalke
fa52eb55c9
test: Remove True argument to CBlock::serialize
Unnamed arguments are confusing as to what they mean without looking up
the function signature.

Since segwit is active by default in regtest, and all blocks are
serialized with witness (#15664, c459c5f), remove the argument
`with_witness=True` from all calls to `CBlock::serialize` and
`BlockTransactions::serialize`.

This diff has been created with a script, but is better reviewed without
a scripted diff.

sed -i --regexp-extended -e 's/block(_?[2a-z]*)\.serialize\([a-z_]*=?True/block\1.serialize(/g' $(git grep -l serialize ./test)
2019-05-08 10:06:21 -04:00
MarcoFalke
c459c5f701
Merge #15664: change default Python block serialization to witness
124ea38e39 change default Python block serialization to witness (Gregory Sanders)

Pull request description:

ACKs for commit 124ea3:
  stevenroose:
    ACK 124ea38e39

Tree-SHA512: 52877934f8a3c761cb89a618daffe73e86b008d9d32d48721392b7626aaa10d3b9aa26e4c59337729e0a2d01fc48648eef5ec3d72de531a685a2cf4f4d7579ab
2019-05-08 09:42:14 -04:00
MarcoFalke
49c1aa5f83
Merge #15971: validation: Add compile-time checking for negative locking requirement in LimitValidationInterfaceQueue
62d50ef308 Add LOCKS_EXCLUDED(cs_main) to LimitValidationInterfaceQueue(...) which does AssertLockNotHeld(cs_main) (practicalswift)

Pull request description:

  This PR adds compile-time checking for negative locking requirements that follow from the run-time locking requirement `AssertLockNotHeld(cs_main)` in `LimitValidationInterfaceQueue(...)`.

  Changes:
  * Add `LOCKS_EXCLUDED(cs_main)` to `LimitValidationInterfaceQueue(...)` which does `AssertLockNotHeld(cs_main)`
  * Add `LOCKS_EXCLUDED(cs_main)` to `CChainState::ActivateBestChain(…)`, `CChainState:: InvalidateBlock(…)` and `CChainState::RewindBlockIndex(…)` which all call `LimitValidationInterfaceQueue(...)` which does `AssertLockNotHeld(cs_main)`
  * Add `LOCKS_EXCLUDED(cs_main)` to `InvalidateBlock(…)` which calls `CChainState::InvalidateBlock(...)` which in turn calls `LimitValidationInterfaceQueue(...)` which  does `AssertLockNotHeld(cs_main)`
  * Add `LOCKS_EXCLUDED(cs_main)` to `RewindBlockIndex(…)` which calls `CChainState::RewindBlockIndex(...)` which in turn calls `LimitValidationInterfaceQueue(...)` which does `AssertLockNotHeld(cs_main)`

ACKs for commit 62d50e:
  MarcoFalke:
    utACK 62d50ef308

Tree-SHA512: 73d092ccd08c851ae3c5d60370c369fc030c5793f5507e2faccb6f91c851ddc0ce059fbea3899f2856330d7a8c78f2ac6a2988e8268b03154f946be9e60e3be1
2019-05-08 09:19:41 -04:00
MarcoFalke
bbb7119ca8
Merge #15978: .gitignore: Don't ignore depends patches
a3592c91a2 .gitignore: Don't ignore depends patches (Carl Dong)

Pull request description:

  Ignoring patches might be useful for those who use `git format-patch` often, but in our depends folder we **_want_** to keep track of our patches.

ACKs for commit a3592c:
  practicalswift:
    utACK a3592c91a2
  laanwj:
    utACK a3592c91a2
  jonasschnelli:
    utACK a3592c91a2
  fanquake:
    utACK a3592c9

Tree-SHA512: 439479cc2ff05335c3d21551a23b34ef2f8a4d0eb05085d2422c2c8d7d7035f529b83dc4056f18cb96234d6fa74075adaa14f0e5fb4c99e6189a4af9078fd28e
2019-05-08 08:24:21 -04:00
MeshCollider
ef802ef5d6
Merge #15880: utils and libraries: Replace deprecated Boost Filesystem functions
a0a222eec Replace deprecated Boost Filesystem function (Hennadii Stepanov)
4f65af97b Remove dead code for walletFile check (Hennadii Stepanov)

Pull request description:

  Boost Filesystem `basename()` and `extension()` functions are [deprecated since v1.36.0](https://www.boost.org/doc/libs/1_36_0/libs/filesystem/doc/reference.html#Convenience-functions).

  See more: https://lists.boost.org/Archives/boost/2010/01/160905.php

  Also this PR prevents further use of deprecated Boost Filesystem functions.
  Ref: https://www.boost.org/doc/libs/1_64_0/libs/filesystem/doc/index.htm#Coding-guidelines

  Note: On my Linux system Boost 1.65.1 header `/usr/include/boost/filesystem/convenience.hpp` contains:
  ```c++
  # ifndef BOOST_FILESYSTEM_NO_DEPRECATED

      inline std::string extension(const path & p)
      {
        return p.extension().string();
      }

      inline std::string basename(const path & p)
      {
        return p.stem().string();
      }

      inline path change_extension( const path & p, const path & new_extension )
      {
        path new_p( p );
        new_p.replace_extension( new_extension );
        return new_p;
      }

  # endif
  ```

  UPDATE:
  Also removed unused code as [noted](https://github.com/bitcoin/bitcoin/pull/15880#discussion_r279386614) by **ryanofsky**.

ACKs for commit a0a222:
  Empact:
    utACK a0a222eec0
  practicalswift:
    utACK a0a222eec0
  fanquake:
    utACK a0a222e
  ryanofsky:
    utACK a0a222eec0. Only change is dropping assert and squashing first two commits.

Tree-SHA512: bc54355441c49957507eb8d3a5782b92d65674504d69779bc16b1b997b2e7424d5665eb6bfb6e10b430a6cacd2aca70af2f94e5f7f10bea24624202834ad35c7
2019-05-09 00:01:29 +12:00
MeshCollider
c3ef63a52f Show loaded wallets as disabled in open menu instead of nothing 2019-05-08 23:54:25 +12:00
practicalswift
62d50ef308 Add LOCKS_EXCLUDED(cs_main) to LimitValidationInterfaceQueue(...) which does AssertLockNotHeld(cs_main)
Add LOCKS_EXCLUDED(cs_main) to functions calling LimitValidationInterfaceQueue(...) which does AssertLockNotHeld(cs_main)

Add LOCKS_EXCLUDED(cs_main) to functions calling CChainState::InvalidateBlock(...) which calls LimitValidationInterfaceQueue(...) which in turn does AssertLockNotHeld(cs_main)

Add LOCKS_EXCLUDED(cs_main) to functions calling CChainState::RewindBlockIndex(...) which calls LimitValidationInterfaceQueue(...) which in turn does AssertLockNotHeld(cs_main)
2019-05-08 10:31:54 +02:00
Carl Dong
a3592c91a2 .gitignore: Don't ignore depends patches 2019-05-07 21:33:54 -04:00
Russell Yanofsky
f6bb11fd37 Add test for ArgsManager::GetChainName
There was some test coverage previously, but it was limited and didn't test
conflicting and negated arguments.
2019-05-07 11:51:30 -04:00
MarcoFalke
b2a6b02161
Merge #15948: refactor: rename chainActive
486c1eea86 refactoring: remove unused chainActive (James O'Beirne)
631940aab2 scripted-diff: replace chainActive -> ::ChainActive() (James O'Beirne)
a3a609079c refactoring: introduce unused ChainActive() (James O'Beirne)
1b6e6fcfd2 rename: CChainState.chainActive -> m_chain (James O'Beirne)

Pull request description:

  This is part of the assumeutxo project:

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal

  ---

  This change refactors the `chainActive` reference into a `::ChainActive()` call. It also distinguishes `CChainState`'s `CChain` data member as `m_chain` instead of the current `chainActive`, which makes it easily confused with the global data.

  The active chain must be obtained via function because its reference will be swapped at some point during runtime after loading a UTXO snapshot.

  This change, though lengthy, should be pretty easy to review since most of it is contained within a scripted-diff. Once merged, the parent PR should be easier to review.

ACKs for commit 486c1e:
  Sjors:
    utACK 486c1ee
  promag:
    utACK 486c1ee.
  practicalswift:
    utACK 486c1eea86

Tree-SHA512: 06ed8f9e77f2d25fc9bea0ba86436d80dbbce90a1e8be23e37ec4eeb26060483e60b4a5c4fba679cb1867f61e3921c24abeb9cabdfb4d0a9b1c4ddd77b17456a
2019-05-07 11:51:30 -04:00
Russell Yanofsky
4b331159df Add unit test NextString, ForEachNoDup functions
Remove testcase generating code from util_SettingsMerge so it can be reused in
new tests.

The hash value expected in util_SettingsMerge changes as a result of this, but
only because the testcases are generated in a different order, not because any
cases are added or removed. It is possible to verify this with:

    SETTINGS_MERGE_TEST_OUT=new.txt test/test_bitcoin --run_test=util_tests/util_SettingsMerge
    git checkout HEAD~1
    make test/test_bitcoin
    SETTINGS_MERGE_TEST_OUT=old.txt test/test_bitcoin --run_test=util_tests/util_SettingsMerge
    diff -u <(sort old.txt) <(sort new.txt)

The new output is a little more readable, with simpler testcases sorted first.
2019-05-07 11:51:30 -04:00
Russell Yanofsky
05bfee3451 util_SettingsMerge test cleanup
Followup to #15869. Treat "-wallet" as the network-specific argument in test
instead of "-server", to make test output clearer and be more consistent with
bitcoind. Update embedded hash to match changed output from this.
2019-05-07 11:51:30 -04:00
orient
41ab2a8924
fix static_assert for macro HAVE_THREAD_LOCAL
fix static_assert for macro HAVE_THREAD_LOCAL while define DEBUG_LOCKCONTENTION
2019-05-07 14:11:35 +08:00
grim-trigger
1b05dff080
Fix portability issue with pthreads
This change resolves the following issue:
https://github.com/bitcoin/bitcoin/issues/15951

Only tested on OpenBSD 6.5/amd64
2019-05-07 00:17:33 +00:00
Pieter Wuille
3cb9ce85d0 Document strenghtening 2019-05-06 15:15:04 -07:00
Pieter Wuille
1d207bc46f Add hash strengthening to the RNG
Once every minute, this will feed the RNG state through repeated SHA512
for 10ms. The timings of that operation are used as entropy source as
well.
2019-05-06 15:15:04 -07:00
Pieter Wuille
c01c065b9d Do not construct out-of-bound pointers in SHA512/SHA1/RIPEMD160 code 2019-05-06 15:11:10 -07:00
MarcoFalke
3632143ebb
Merge #14266: refactor: Lift prevector default vals to the member declaration
d2eee87928 Lift prevector default vals to the member declaration (Ben Woosley)

Pull request description:

  I overlooked this possibility in #14028

ACKs for commit d2eee8:
  promag:
    utACK d2eee87, change looks good because members are always initialized.
  251Labs:
    utACK d2eee87 nice one.
  ken2812221:
    utACK d2eee87928
  practicalswift:
    utACK d2eee87928
  scravy:
    utACK d2eee87928

Tree-SHA512: f2726bae1cf892fd680cf8571027bcdc2e42ba567eaa901fb5fb5423b4d11b29e745e0163d82cb513d8c81399cc85933a16ed66d4a30829382d4721ffc41dc97
2019-05-06 15:32:24 -04:00
MarcoFalke
aaaa57c2aa
scripted-diff: Bump copyright headers in wallet
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./src/wallet/
-END VERIFY SCRIPT-
2019-05-06 14:05:52 -04:00
MarcoFalke
faf3729242
wallet: Only fail rescan when blocks have actually been pruned 2019-05-06 14:03:56 -04:00
MarcoFalke
8ec7121a45
Merge #15927: [tests] log thread names by default in functional tests
7b29ec277b [tests] Comment for why logging config is set as command-line args. (John Newbery)
ba534ccd56 [tests] log thread names by default in functional tests (John Newbery)

Pull request description:

  More detailed logs are better

ACKs for commit 7b29ec:
  jamesob:
    utACK 7b29ec277b

Tree-SHA512: 327cfedb7b7bf32f7ce1e2de5f70c7092041a8e868e14285a79176277c6cf47ebea27027f68787332f8ad21c7f64d2640dd21813eda5b2bd0e5208a65364a879
2019-05-06 12:59:19 -04:00
Wladimir J. van der Laan
c5ffe8d515
Merge #15730: rpc: Show scanning details in getwalletinfo
b6c748f849 doc: Add release notes for 15730 (João Barbosa)
d3e8458365 rpc: Show scanning details in getwalletinfo (João Barbosa)
90e27abe37 wallet: Track current scanning progress (João Barbosa)
2ee811e693 wallet: Track scanning duration (João Barbosa)

Pull request description:

  Closes #15724.

ACKs for commit b6c748:
  MarcoFalke:
    re-utACK b6c748f849 (Only change since my last review is rebase, adding release notes, and returning false instead of null)
  laanwj:
    utACK b6c748f849
  jonatack:
    ACK b6c748f849, only changes appear to be rebase for https://github.com/bitcoin/bitcoin/pull/15730#discussion_r280030617 and release notes.

Tree-SHA512: 8ee98f971c15f66ce8138fc92c55e51abc9faf01866a31ac7ce2ad766aa2bb88559eabee3b5815d645c84cdf1c19dc35ec03f31461e39bc5f6040edec0b87116
2019-05-06 13:38:12 +02:00
Wladimir J. van der Laan
a3d2d6b067
Merge #15930: rpc: Add balances RPC
facfb4111d rpc: Deprecate getunconfirmedbalance and getwalletinfo balances (MarcoFalke)
999931cf8f rpc: Add getbalances RPC (MarcoFalke)
fad13e925e rpcwallet: Make helper methods const on CWallet (MarcoFalke)
fad40ec915 wallet: Use IsValidNumArgs in getwalletinfo rpc (MarcoFalke)

Pull request description:

  This exposes the `CWallet::GetBalance()` struct over RPC.

  In the future, incorrectly named rpcs such as `getunconfirmedbalance` or rpcs redundant to this such as `getbalance` could be removed.

ACKs for commit facfb4:
  jnewbery:
    utACK facfb4111d

Tree-SHA512: 1f54fedce55df9a8ea82d2b6265354b39a956072621876ebaee2355aac0e23c7b64340c3279502415598c095858529e18b50789be956250aafda1cd3a8d948a5
2019-05-06 11:36:42 +02:00
Wladimir J. van der Laan
773e16f919
Merge #15947: Install bitcoin-wallet manpage
00d110463a Install bitcoin-wallet manpage. (Daniel Kraft)

Pull request description:

  This change marks the already-existing `bitcoin-wallet.1` manpage file for installation together with the others.  Previously, only `bitcoind.1`, `bitcoin-cli.1`, `bitcoin-tx.1` and `bitcoin-qt.1` would be installed.

ACKs for commit 00d110:
  laanwj:
    utACK 00d110463a
  practicalswift:
    utACK 00d110463a

Tree-SHA512: ca846e414548f1dc774f460edca2e17d7d619c7e6f0d18db0c58c09e04f9d43c6964fcf2bacb5b1eae94de9c5fdda86abf258ef6b78b0f693715d070dfc10f08
2019-05-06 10:07:28 +02:00
Wladimir J. van der Laan
9bbaac73bb
Merge #15928: GUI: Move QRImageWidget to its own file-pair
fc929842c2 GUI: Move QRImageWidget to its own file-pair (Luke Dashjr)
77851ab682 GUI: Refactor actual QR code rendering into new QRImageWidget::setQR (Luke Dashjr)

Pull request description:

  For at least QR-code based pairing of mobile wallets with nodes, it will be desirable to render QR codes even without wallet support.

  Therefore, this prepares by moving the QRImageWidget out of a wallet-specific file into its own `qrencoder` file-pair.

ACKs for commit fc9298:
  laanwj:
    utACK fc929842c2
  jonasschnelli:
    utACK fc929842c2

Tree-SHA512: 95529a38c0573a4b3f1253fb5f11ca07a5b3a9840ec24acc7d87270212f3c9f7c5b186d9274d297517a3b80494f38a57574fb9730b1574db01688539b987bd91
2019-05-06 10:04:33 +02:00
Wladimir J. van der Laan
d7d7d31506
Merge #15141: Rewrite DoS interface between validation and net_processing
0ff1c2a838 Separate reason for premature spends (coinbase/locktime) (Suhas Daftuar)
54470e767b Assert validation reasons are contextually correct (Suhas Daftuar)
2120c31521 [refactor] Update some comments in validation.cpp as we arent doing DoS there (Matt Corallo)
12dbdd7a41 [refactor] Drop unused state.DoS(), state.GetDoS(), state.CorruptionPossible() (Matt Corallo)
aa502b88d1 scripted-diff: Remove DoS calls to CValidationState (Matt Corallo)
7721ad64f4 [refactor] Prep for scripted-diff by removing some \ns which annoy sed. (Matt Corallo)
5e78c5734b Allow use of state.Invalid() for all reasons (Matt Corallo)
6b34bc6b6f Fix handling of invalid headers (Suhas Daftuar)
ef54b486d5 [refactor] Use Reasons directly instead of DoS codes (Matt Corallo)
9ab2a0412e CorruptionPossible -> BLOCK_MUTATED (Matt Corallo)
6e55b292b0 CorruptionPossible -> TX_WITNESS_MUTATED (Matt Corallo)
7df16e70e6 LookupBlockIndex -> CACHED_INVALID (Matt Corallo)
c8b0d22698 [refactor] Drop redundant nDoS, corruptionPossible, SetCorruptionPossible (Matt Corallo)
34477ccd39 [refactor] Add useful-for-dos "reason" field to CValidationState (Matt Corallo)
6a7f8777a0 Ban all peers for all block script failures (Suhas Daftuar)
7b999103e2 Clean up banning levels (Matt Corallo)
b8b4c80146 [refactor] drop IsInvalid(nDoSOut) (Matt Corallo)
8818729013 [refactor] Refactor misbehavior ban decisions to MaybePunishNode() (Matt Corallo)
00e11e61c0 [refactor] rename stateDummy -> orphan_state (Matt Corallo)
f34fa719cf Drop obsolete sigops comment (Matt Corallo)

Pull request description:

  This is a rebase of #11639 with some fixes for the last few comments which were not yet addressed.

  The original PR text, with some strikethroughs of text that is no longer correct:

  > This cleans up an old main-carryover - it made sense that main could decide what DoS scores to assign things because the DoS scores were handled in a different part of main, but now validation is telling net_processing what DoS scores to assign to different things, which is utter nonsense. Instead, we replace CValidationState's nDoS and CorruptionPossible with a general ValidationInvalidReason, which net_processing can handle as it sees fit. I keep the behavior changes here to a minimum, but in the future we can utilize these changes for other smarter behavior, such as disconnecting/preferring to rotate outbound peers based on them providing things which are invalid due to SOFT_FORK because we shouldn't ban for such cases.
  >
  > This is somewhat complementary with, though obviously conflicts heavily with #11523, which added enums in place of DoS scores, as well as a few other cleanups (which are still relevant).
  >
  > Compared with previous bans, the following changes are made:
  >
  > Txn with empty vin/vout or null prevouts move from 10 DoS
  > points to 100.
  > Loose transactions with a dependency loop now result in a ban
  > instead of 10 DoS points.
  > ~~BIP68-violation no longer results in a ban as it is SOFT_FORK.~~
  > ~~Non-SegWit SigOp violation no longer results in a ban as it
  > considers P2SH sigops and is thus SOFT_FORK.~~
  > ~~Any script violation in a block no longer results in a ban as
  > it may be the result of a SOFT_FORK. This should likely be
  > fixed in the future by differentiating between them.~~
  > Proof of work failure moves from 50 DoS points to a ban.
  > Blocks with timestamps under MTP now result in a ban, blocks
  > too far in the future continue to not result in a ban.
  > Inclusion of non-final transactions in a block now results in a
  > ban instead of 10 DoS points.

  Note: The change to ban all peers for consensus violations is actually NOT the change I'd like to make -- I'd prefer to only ban outbound peers in those situations.  The current behavior is a bit of a mess, however, and so in the interests of advancing this PR I tried to keep the changes to a minimum.  I plan to revisit the behavior in a followup PR.

  EDIT: One reviewer suggested I add some additional context for this PR:

  > The goal of this work was to make net_processing aware of the actual reasons for validation failures, rather than just deal with opaque numbers instructing it to do something.
  >
  > In the future, I'd like to make it so that we use more context to decide how to punish a peer. One example is to differentiate inbound and outbound peer misbehaviors. Another potential example is if we'd treat RECENT_CONSENSUS_CHANGE failures differently (ie after the next consensus change is implemented), and perhaps again we'd want to treat some peers differently than others.

ACKs for commit 0ff1c2:
  jnewbery:
    utACK 0ff1c2a838
  ryanofsky:
    utACK 0ff1c2a838. Only change is dropping the first commit (f3883a321bf4ab289edcd9754b12cae3a648b175), and dropping the temporary `assert(level == GetDoS())` that was in 35ee77f2832eaffce30042e00785c310c5540cdc (now c8b0d22698)

Tree-SHA512: e915a411100876398af5463d0a885920e44d473467bb6af991ef2e8f2681db6c1209bb60f848bd154be72d460f039b5653df20a6840352c5f7ea5486d9f777a3
2019-05-04 11:58:57 +02:00
MarcoFalke
f19a3b2ded
Merge #15949: test_runner: Move pruning back to extended
fa08c5cb99 test_runner: Move pruning back to extended (MarcoFalke)

Pull request description:

  This reverts fafb55e2c2, since the test is still too slow to run with asan enabled on a network hdd

ACKs for commit fa08c5:
  jnewbery:
    utACK fa08c5cb99
  jonasschnelli:
    utACK fa08c5cb99

Tree-SHA512: de16786b9d507a72210805c3e9eef360e5fc3d4bc3a81f7175b6cc70d1bc426cde7ac97bc0d1a0d4e0813067e1e251c2dd49256552cc6b52446b475251b7c32b
2019-05-03 17:03:16 -04:00
Jonas Schnelli
2dfe275171
Add ChaCha20 bench 2019-05-03 22:52:11 +02:00