Commit graph

18189 commits

Author SHA1 Message Date
wodry
42c499614a
Docs: Improve readability of "Squashing commits"
It was not easy to read the comment lines for me because I was not sure whether the sentence ended with the line or not ("pull set commits"?).
Therefore, dots had been invented and I have added them to signal the end of a sentence. Also begin New sentence with a capital letter.

I guess, not all 'pick' words should be replaced by 'squash'? At least I found [https://www.digitalocean.com/community/tutorials/how-to-rebase-and-update-a-pull-request](this) rebase/squash documentation helpful, where is written that the first line should not be changed.
2018-06-17 10:47:50 +02:00
Wladimir J. van der Laan
a90ca4087a
Merge #13448: Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python
c8176b3cc7 Add linter: Make sure we explicitly open all text files using UTF-8 or ASCII encoding in Python (practicalswift)
634bd97001 Explicitly specify encoding when opening text files in Python code (practicalswift)

Pull request description:

  Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python.

  As requested by @laanwj in #13440.

Tree-SHA512: 1651c00fe220ceb273324abd6703aee504029b96c7ef0e3029145901762c733c9b9d24927da281394fd4681a5bff774336c04eed01fafea997bb32192c334c06
2018-06-16 15:23:14 +02:00
Wladimir J. van der Laan
fa2ea37940
Merge #13479: contrib: Fix CVE-2018-12356 by hardening the regex
9e2e5626da Fix CVE-2018-12356 by hardening the regex. (Loganaden Velvindron)

Pull request description:

  Detailed write-up here:
  https://neopg.io/blog/pass-signature-spoof/

Tree-SHA512: 2020474ff4c7b5e7f1bea932c63da62aca48d3dc6dcd04507afefad6c40f2977ed2d2916defe2e8e7936a2d498125c578077870147954a7af9b29b823cbb0b10
2018-06-16 15:20:17 +02:00
MarcoFalke
81069a75bd
Merge #13465: Avoid concurrency issue when make multiple target
cf01fd6f9c Avoid concurrency issue (Chun Kuan Lee)

Pull request description:

  From #13406, changed travis job target for Mac to `all deploy`, but this could cause a race condition.
  Simply add `.NOTPARALLEL` to avoid it.
  Related jobs:
  https://travis-ci.org/bitcoin/bitcoin/jobs/391863281
  https://travis-ci.org/bitcoin/bitcoin/jobs/391907335
  Close #13469

Tree-SHA512: 75c6585fe770dc70e6256dcdf97af37274f95a9240ed5a5cea2ca92b8411893b80327335587270351b128f56cb2e00f684db7c19b1602048132b734dad6ececa
2018-06-15 13:59:10 -04:00
Loganaden Velvindron
9e2e5626da Fix CVE-2018-12356 by hardening the regex. 2018-06-15 21:45:32 +04:00
Cory Fields
d92204c900 build: add warning to detect hidden copies in range-for loops 2018-06-15 13:40:07 -04:00
Cory Fields
466e16e0e8 cleanup: avoid hidden copies in range-for loops 2018-06-15 13:40:00 -04:00
MarcoFalke
be27048a18
Merge #13241: scripted-diff: Avoid temporary copies when looping over std::map
9b72c988a0 scripted-diff: Avoid temporary copies when looping over std::map (Ben Woosley)

Pull request description:

  The ::value_type of the std::map/std::multimap/std::unordered_map containers is
  std::pair<const Key, T>. Dropping the const results in an unnecessary copy,
  for example in C++11 range-based loops.

  For this I started with a more general scripted diff, then narrowed it down
  based on the inspection showing that all actual map/multimap/unordered_map
  variables used in loops start with m or have map in the name.

Tree-SHA512: b656d66b69ffa1eb954124aa8ae2bc5436ca50262abefa93bdda55cfcdaffc5ff90cd40539051a2bd06355ba69ddf245265cc8764eebff66d761b3aec06155a9
2018-06-15 12:32:39 -04:00
Wladimir J. van der Laan
280924e672
doc: Add historical release notes for 0.16.1
Tree-SHA512: cca8188f954eeded58f705749b2ae51b08aadf4feddafaaafc57df2f84c10a3340a373c1602e9aa290c54b67cdcce53f61f4ca2db87bd98de5449afc53e25f86
2018-06-15 18:31:20 +02:00
MarcoFalke
43fa3554b7
Merge #13402: Document validationinterace callback blocking deadlock potential.
25bc9615b7 Document validationinterace callback blocking deadlock potential. (Matt Corallo)

Pull request description:

  From the branches-I've-had-lying-around-and-forgot-to-PR department...

  This is a comment-only PR, but the comments point out an API quirk that isn't exactly trivial. None of our use-cases right now hit this, but if we were to call SyncWithValidationInterfaceQueue (eg to limit queue depth) in ATMP, I'm pretty sure we'd hit a deadlock there.

Tree-SHA512: 889dd8fc9eb15d1f2aa5ca467e783bc8f07bc543b166b032741795b0db7a0df11a2846d3cb7c69bafa8d1acf970021001b742f52be06725a932813230c5b4a7b
2018-06-15 09:58:26 -04:00
MarcoFalke
32bf4c619d
Merge #13450: Add linter: Enforce the source code file naming convention described in the developer notes
ad691f666b Add linter: Enforce the source code file naming convention described in the developer notes (practicalswift)

Pull request description:

  Add linter: Enforce the source code file naming convention described in the developer notes.

Tree-SHA512: 6458acf5383de7e81554bdd954c3a74c2bbf26286687ea69d934f11174d2f6bd573e8d2c16a7e77bbd12065e65be7700ecd7791d215f286e18f346bf964cd17d
2018-06-15 09:36:12 -04:00
MarcoFalke
9501938a44
Merge #13476: Fix incorrect shell quoting in FreeBSD build instructions.
c9924a2756 Fix incorrect shell quoting in FreeBSD build instructions. (murrayn)

Pull request description:

  The current instructions suggest:

      BDB_PREFIX='$PWD/db4'

  which results in BDB_PREFIX being set, literally, to '$PWD/db4'.

Tree-SHA512: 4a920d7a506ff94fb12793f0b8161fdfe0798382cfa88290a0eadd6926843b577328f7159a55039e2655def7ddf2e7a1eaa3dd30490292d4a920c98759becf56
2018-06-15 09:21:14 -04:00
murrayn
c9924a2756 Fix incorrect shell quoting in FreeBSD build instructions. 2018-06-15 01:34:40 -07:00
Karl-Johan Alm
f618ebc4e4
validation: count blocks correctly for check level < 3 2018-06-15 13:27:18 +09:00
MarcoFalke
2b770080a4
Merge #13422: Drop ParseHashUV in favor of calling ParseHashStr
abd2678ac1 Drop ParseHashUV in favor of calling ParseHashStr (Ben Woosley)

Pull request description:

  The one existing call already validates `get_str` will pass via `checkObject`:
  https://github.com/bitcoin/bitcoin/pull/13422/files#diff-8fe4d6985ee4acf8bfc1ed8db1e83cb5L586

  Split from #13420

Tree-SHA512: 35dfa8c28d0c3ceac7a6de7f4eb4a44d912f4c31f5d21c9438f899566ca2b34851f1a58c3417355e55d0c33abb97385f4a47e034bfc8e3cdbbf5f73813ca0582
2018-06-14 21:34:41 -04:00
practicalswift
ad691f666b Add linter: Enforce the source code file naming convention described in the developer notes 2018-06-14 22:40:41 +02:00
MarcoFalke
1939536eea
Merge #13460: doc: Remove note to install all boost dev packages
fa3d39ec53 doc: Remove note to install all boost dev packages (MarcoFalke)

Pull request description:

  Those notes are for systems that are EOL. Also, those instructions would pull in a ton of additional dependencies such as `python2`.

  Can be reviewed with `git diff HEAD~ --word-diff`

Tree-SHA512: 7e379b3f0b7fe88452baed8860438a7867a025e650a265ed0cd5d47076983f4176545b7d96a4b7f25ae9fcb325f0116d909f3ffb6ff2d7573fddd777fe5b4c79
2018-06-14 16:37:36 -04:00
Chun Kuan Lee
cf01fd6f9c Avoid concurrency issue 2018-06-14 19:43:12 +00:00
Wladimir J. van der Laan
cc7cbd756a
Merge #13451: rpc: expose CBlockIndex::nTx in getblock(header)
86edf4a2a5 expose CBlockIndex::nTx in getblock(header) (Gregory Sanders)

Pull request description:

  Recent publication of a weakness in Bitcoin's merkle tree construction demonstrates many SPV applications vulnerable to an expensive to pull off yet still plausible attack: https://bitslog.wordpress.com/2018/06/09/leaf-node-weakness-in-bitcoin-merkle-tree-design/

  Including the coinbase in the txoutproof seems the most effective fix, however results in a significant efficiency downgrade. Transactors will not even know a priori what the size of their proof will be within a couple orders of magnitude, unless they use the mid-state of SHA2 as detailed in the blog post.

  Some applications, like Elements blockchain platform that take SPV-style proofs have optional access to a bitcoind to verify these proofs of inclusion and check depth in the chain. Returning `CBlockIndex::nTx` would allow an extremely easy and compact way of checking the depth of the tree, with no additional overhead to the codebase, and works with pruned nodes.

  `getblockheader` is arguably not the place for it, but as mentioned before, is a natural workflow for us checking depth of a block in a possibly pruned node.

  We should also ensure that `verifytxoutproof` ends up validating this depth fact as well, but left this for another PR.

Tree-SHA512: af4cf48e704c6088f8da06a477fda1aaa6f8770cee9b876c4465d1075966d6a95831a88817673fe5a0d6bbcdc1ffcbc1892e2be0d838c60fc6958d33eacdcc14
2018-06-14 19:40:02 +02:00
Gregory Sanders
3c292cc190 ScanforWalletTransactions should mark input txns as dirty 2018-06-14 09:57:34 -04:00
Gregory Sanders
ed82f17000 have verifytxoutproof check the number of txns in proof structure 2018-06-14 09:54:41 -04:00
practicalswift
47776a958b Add linter: Make sure all shell scripts opt out of locale dependence using "export LC_ALL=C" 2018-06-14 15:27:52 +02:00
practicalswift
3352da8da1 Add "export LC_ALL=C" to all shell scripts 2018-06-14 15:27:52 +02:00
MarcoFalke
fa3d39ec53
doc: Remove note to install all boost dev packages 2018-06-13 15:11:31 -04:00
MarcoFalke
4a7e64fc85
Merge #13441: Prevent shared conf files from failing with different available options in different binaries
c2dfbb4a97 Add unavailable options to hidden options category (Andrew Chow)

Pull request description:

  From IRC:

  ```
  <ossifrage> FYI, bitcoin-qt from the head I built today won't start if you have "daemon=0" in the config file, so you can't use the same config for either bitcoind or bitcoin-qt
  <ossifrage> Seems like bitcoin-qt should ignore this option?
  <provoostenator> ossifrage: probably caused by 13112. Another problem is disablewallet=1 will prevent a launch if you compile bitcoind without wallet. It probably needs to be relaxed slightly.
  ```

  Adds all of the options that are unavailable due to compiling options to the hidden category so that shared config files do not break with the alternative binaries.

Tree-SHA512: 1ef43f5f7ad46ecc2865d22ee683ef22831e8f131ec99b732bb36d90381f7964bf64829595e993c2d435823fe4425a20323c8e65307cf2463a9e40b8049ab559
2018-06-13 13:49:18 -04:00
Wladimir J. van der Laan
b2221381e7
Merge #13457: tests: Drop variadic macro
faf52f953b tests: Drop variadic macro (MarcoFalke)

Pull request description:

  The C++11 constructor of `std::vector` that takes an initializer list, is not `explicit`. Thus, the macro is not required and can be dropped.

  Hopefully fixes #13456

Tree-SHA512: 4095ed205f88138a7cd5b14790cc426899966f622a924a9b3f7de646a0d801a48ffb8921da760f1f93d5481298477c8a64dbec291381bb9aa77b075bdd2659f2
2018-06-13 19:25:42 +02:00
MarcoFalke
4382f192e5
Merge #13406: travis: Change Mac goal to all deploy
3d69853090 travis: Change Mac goal to all deploy so that travis can build all executables for Mac. (Chun Kuan Lee)

Pull request description:

  Currently, travis only build bitcoin-qt for Mac with make deploy, so  I think that we can add a make step to build cli, daemon, test and bench executables.

Tree-SHA512: 346c207836ee7313a57e0cd03367ada7e95c182d0f58b06c4ee5b8f2865ba4ee077b030a5494b58a808c015afe0569fa3252d0a5a687c47b7094dc77599ce50e
2018-06-13 13:21:02 -04:00
Wladimir J. van der Laan
bad068ad9f build: Build system changes to support only Qt5 2018-06-13 18:15:47 +02:00
MarcoFalke
f532d52d39
Merge #13350: [tests] Add logging to provide anchor points when debugging p2p_sendheaders
2ce81867b2 [tests] Add logging to provide anchor points when debugging failures. (Lowell Manners)

Pull request description:

  refs #12453

Tree-SHA512: 0ad432bd848723a5b813df4d35fcd8c81d152f042499c8340a9a2a9b7918d6e785efbf9f07b63b6c0253a949044cebdb51802971c09fb8dde0efa5169f34ef20
2018-06-13 10:41:28 -04:00
Gregory Sanders
86edf4a2a5 expose CBlockIndex::nTx in getblock(header) 2018-06-13 10:20:50 -04:00
MarcoFalke
faf52f953b
tests: Drop variadic macro 2018-06-13 09:58:54 -04:00
Wladimir J. van der Laan
8eb76f3958
Merge #13445: build: Reset default -g -O2 flags when enable debug
9882d1f044 Reset default -g -O2 flags when enable debug (Chun Kuan Lee)

Pull request description:

  The default CXXFLAGS is -g -O2, this should not appear when enable debug.
  fixes #13432

Tree-SHA512: 79447f3e1fab9e6cd12f5ca49b3d42187e856e0c159ed01140ea93d6ef1fbb1af3d65b338308566330491052c0177d12abe26796513502ddde31692665a0dbb4
2018-06-13 15:49:10 +02:00
Wladimir J. van der Laan
cf7ca60923
Merge #13435: When build fails due to lib missing, indicate which one
51cd508e2f When build fails due to lib missing, indicate which one (Ben Woosley)

Pull request description:

  A failure of "lib missing" has limited utility.

Tree-SHA512: de77c077ea223bb862a6261fd99e9ddc533e28ed42837ac4262a1c0f24d73add4d2baae6be5cb5ef2012be32bcc3ae587dbc1eab0c42b633bea8eec64b18a249
2018-06-13 15:45:04 +02:00
Wladimir J. van der Laan
caabdea627
Merge #13428: validation: check the specified number of blocks (off-by-one)
f6f8026e40 validation: check the specified number of blocks (off-by-one) (Karl-Johan Alm)

Pull request description:

  ```
  echeveria | 2018-06-11 02:03:03.384975 Verifying last 3 blocks at level 3
  echeveria | 2018-06-11 02:03:23.676793 No coin database inconsistencies in last 4 blocks (6564 transactions)
  echeveria | off by one?
  sipa      | echeveria: possibly!
  kallewoof | Looks like it checks one more block than suggested. `if (pindex->nHeight < chainActive.Height()-nCheckDepth) break;` should probably be `<=`.
  sipa      | kallewoof: agree
  ```

  Post-commit:
  ```
  2018-06-11T05:24:02Z Verifying last 6 blocks at level 3
  2018-06-11T05:24:02Z [0%]...[16%]...[33%]...[50%]...[66%]...[83%]...[99%]...[DONE].
  2018-06-11T05:25:07Z No coin database inconsistencies in last 6 blocks (7258 transactions)
  ```

  Pre-commit:
  ```
  2018-06-11T05:27:11Z Verifying last 6 blocks at level 3
  2018-06-11T05:27:11Z [0%]...[16%]...[33%]...[50%]...[66%]...[83%]...[99%]...[DONE].
  2018-06-11T05:27:12Z No coin database inconsistencies in last 7 blocks (9832 transactions)
  ```

Tree-SHA512: 6e68dc4ba74232518c2ba8ea624d65893534f3619d43ccdf0b9c65992f25b68cb52cf54fa35e6e3d092d1eee5c9a8887057828895f1acdafc0ebb48f683fffdc
2018-06-13 15:42:32 +02:00
MarcoFalke
4b1edd3185
Merge #13447: travis: Increase travis_wait time while verifying commits
51ed05a2b9 travis: Increase travis_wait time while verifying commits (Chun Kuan Lee)

Pull request description:

  From https://travis-ci.org/ken2812221/bitcoin-verify-commits/builds
  I have run vecify-commits.py nightly on travis, as you can see that 30 minutes is not enough, it took 30-50 minutes to run the script with no extra options. So change it to 50 minutes would be better. Forgot to change this at #13066

Tree-SHA512: fced346edcb7dc626fa6e714d7db9a34d3e9f283ada64e19c30565ed214f72e26a57a905b4eb71ab24a16c97a1bd35e57e9a83dbb7dc1fff2db26505b810e61e
2018-06-13 09:26:45 -04:00
Chun Kuan Lee
3d69853090 travis: Change Mac goal to all deploy so that travis can build all executables for Mac. 2018-06-12 22:27:57 +00:00
Andrew Chow
c2dfbb4a97 Add unavailable options to hidden options category
Options that are not available (but known in the source code) will
cause an error if they are specified.
Make these options "available" by adding them to the hidden options
category to prevent conf files from failing when shared between binaries
that have different options available.
2018-06-12 14:33:35 -07:00
practicalswift
c8176b3cc7 Add linter: Make sure we explicitly open all text files using UTF-8 or ASCII encoding in Python 2018-06-12 21:49:04 +02:00
practicalswift
9fdf05d70c tests: Fix lock-order-inversion (potential deadlock) in DoS_tests. Reported by TSAN.
Makes `src/test/test_bitcoin --run_test=DoS_tests` pass also when
compiled with TreadSanitizer (`./configure --with-sanitizers=thread`).
2018-06-12 21:45:46 +02:00
practicalswift
634bd97001 Explicitly specify encoding when opening text files in Python code 2018-06-12 21:34:52 +02:00
Pieter Wuille
1e1eb6367f Improve coverage of SHA256 SelfTest code 2018-06-12 12:10:13 -07:00
Wladimir J. van der Laan
a607d23ae8
Merge #13393: Enable double-SHA256-for-64-byte code on 32-bit x86
57ba401abc Enable double-SHA256-for-64-byte code on 32-bit x86 (Pieter Wuille)

Pull request description:

  The SSE4 and AVX2 double-SHA256-for-64-byte input code from #13191 compiles fine on 32-bit x86 systems, but the autodetection logic in sha256.cpp doesn't enable it. Fix this.

  Note that these instruction sets are only available on CPUs that support 64-bit mode as well, so it is only beneficial in the (perhaps unlikely) scenario where a 64-bit CPU is running a 32-bit Bitcoin Core binary.

Tree-SHA512: 39d5963c1ba8c33932549d5fe98bd184932689a40aeba95043eca31dd6824f566197c546b60905555eccaf407408a5f0f200247bb0907450d309b0a70b245102
2018-06-12 18:52:26 +02:00
Chun Kuan Lee
51ed05a2b9 travis: Increase travis_wait time while verifying commits
From https://travis-ci.org/ken2812221/bitcoin-verify-commits/builds
I have run vecify-commits.py nightly on travis, as you can see that 30 minutes is not enough, it took 30-50 minutes to run the script with no extra options. So change it to 50 minutes would be better.
2018-06-12 15:46:51 +00:00
Wladimir J. van der Laan
fa4b9065a8
Merge #13066: Migrate verify-commits script to python, run in travis
e5b2cd8e75 Use python instead of slow shell script on verify-commits (Chun Kuan Lee)

Pull request description:

  The cron job that runs every day would fail because of git checkout a single commit, not a branch.

  #12708 introduce a method to check whether merges are clean.
  However, there are four merges are not clean.
  So, I add a list of merges that are dirty and ignore them.

  Also, I modify the current shell script to python, it makes the script speed up a lot.
  The python code `tree_sha512sum` was copied from `github-merge.py`

  I've re-designed this. Now we verify all the things by default.
  - Add `--disable-tree-check` option, not to check SHA-512 tree
  - Add `--clean-merge NUMBER` option, only verify commits after &lt;NUMBER> days ago

  Travis running time:

  |option|time|
  |-|-|
  |verify-commits.py|[25m47.02s(1547.02s)](https://travis-ci.org/ken2812221/bitcoin/jobs/373321423)|
  |verify-commits.py --disable-tree-check|[19m10.08s(1150.08s)](https://travis-ci.org/ken2812221/bitcoin/jobs/373321423)|
  |verify-commits.py --clean-merge 30|[9m18.18s(558.18s)](https://travis-ci.org/ken2812221/bitcoin/jobs/373321423)|
  |verify-commits.py --disable-tree-check --clean-merge 30|[1m16.51s(76.51s)](https://travis-ci.org/ken2812221/bitcoin/jobs/373321423)|

  Since the cron job always fail, I've created a respository to verify this daily.
   [![Build Status](https://travis-ci.org/ken2812221/bitcoin-verify-commits.svg?branch=master)](https://travis-ci.org/ken2812221/bitcoin-verify-commits)

Tree-SHA512: 476bcf707d92ed3d431ca5642e013036df1506120d3dd2aa718f74240063ce856abd78f4c948336c2a6230dfe5c60c6f2d52d19bdb52d647a1c5f838eaa02e3b
2018-06-12 17:25:25 +02:00
Wladimir J. van der Laan
ca2a23387b
Merge #13120: policy: Treat segwit as always active
fa7a6cf1b3 policy: Treat segwit as always active (MarcoFalke)

Pull request description:

  Now that segwit is active for a long time, there is no need to reject transactions with the reason that segwit hasn't activated.

  Strictly speaking, this is a bug fix, because with the release of 0.16, we create segwit transactions in our wallet by default without checking if they are allowed by local policy.

  More broadly, this simplifies the code as if "premature witness" was always set to true with the corresponding command line args.

Tree-SHA512: 484c26aa3a66faba6b41e8554a91a29bfc15fbf6caae3d5363a3966283143189c4bd5333a610b0669c1238f75620691264e73f6b9f1161cdacf7574d946436da
2018-06-12 17:20:34 +02:00
João Barbosa
537efe19e6 rpc: Extract GetWalletNameFromJSONRPCRequest from GetWalletForJSONRPCRequest 2018-06-12 16:11:28 +01:00
Chun Kuan Lee
e5b2cd8e75 Use python instead of slow shell script on verify-commits 2018-06-12 14:48:02 +00:00
Wladimir J. van der Laan
5315660bae
Merge #13440: qa: Log as utf-8
fa8071a098 qa: Log as utf-8 (MarcoFalke)

Pull request description:

  Explicitly read and write the log files with utf-8 as encoding

Tree-SHA512: ca28f37f34a09845c736ff6c4c21733c3c39584f52c81e48ff25e5e35979c317d0989862b2b93acc7e359fbcc20b99533365455830b2ddb41eb4d8c17314534e
2018-06-12 14:53:57 +02:00
MarcoFalke
b22115d9a3
Merge #13312: docs: Add a note about the source code filename naming convention
e56771365b Do not use uppercase characters in source code filenames (practicalswift)
419a1983ca docs: Add a note about the source code filename naming convention (practicalswift)

Pull request description:

  Add a note about the source code filename naming convention.

Tree-SHA512: 8d329bd9e19bcd26e74b0862fb0bc2369b46095dbd3e69d34859908632763abd7c3d00ccc44ee059772ad4bae4460c2bcc1c0e22fd9d8876d57e5fcd346cea4b
2018-06-12 08:02:20 -04:00
Chun Kuan Lee
9882d1f044 Reset default -g -O2 flags when enable debug 2018-06-12 07:51:56 +00:00