Commit graph

19956 commits

Author SHA1 Message Date
Wladimir J. van der Laan
2d46f1be0c
Merge #15118: Refactor block file logic
04cca33094 Style cleanup. (Jim Posen)
4c01e4e159 flatfile: Unit tests for FlatFileSeq methods. (Jim Posen)
65a489e93d scripted-diff: Rename CBlockDiskPos to FlatFilePos. (Jim Posen)
d6d8a78f26 Move CDiskBlockPos from chain to flatfile. (Jim Posen)
e0380933e3 validation: Refactor file flush logic into FlatFileSeq. (Jim Posen)
992404b31e validation: Refactor block file pre-allocation into FlatFileSeq. (Jim Posen)
e2d2abb99f validation: Refactor OpenDiskFile into method on FlatFileSeq. (Jim Posen)
9183d6ef65 validation: Extract basic block file logic into FlatFileSeq class. (Jim Posen)
62e7addb63 util: Move CheckDiskSpace to util. (Jim Posen)

Pull request description:

  This cleans up and refactors block file helpers so that they may be used by the block filter indexer. Per [design discussion](https://github.com/bitcoin/bitcoin/pull/14121#issuecomment-451252591) about storing BIP 157 block filters, it has been suggested that they are stored in the same way as block and undo data. This refactor is sufficient to simplify file operations for this use case, though in the future perhaps more pruning-related logic ought to be moved into the new classes.

  The basic abstraction is a `FlatFileSeq` which manages access to a sequence of numbered files into which raw data is written.

Tree-SHA512: b2108756777f2dad8964a1a2ef2764486e708a4a4a8cfac47b5de8bcb0625388964438eb096b10cfd9ea39212c299b5cb32fa943e768db2333cf49ea7def157e
2019-03-02 23:20:38 +01:00
MarcoFalke
dddd1d05d3
.python-version: Specify full version 3.5.6 2019-03-02 12:06:26 -05:00
MarcoFalke
faa7cdf764
scripted-diff: Update copyright in ./test
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./test/
-END VERIFY SCRIPT-
2019-03-02 10:58:35 -05:00
MarcoFalke
fa0e65b772
scripted-diff: test: Remove brackets after assert
-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended -e 's/assert ?\((.+)\)(( )*)?(#.*)?$/assert \1\3\3\4/g' $(git grep -l --extended-regexp 'assert ?\(' test)
-END VERIFY SCRIPT-
2019-03-02 10:51:35 -05:00
MarcoFalke
fab5a1e0f4
build: Require python 3.5 2019-03-02 10:40:23 -05:00
MarcoFalke
fa6bf21f5e
scripted-diff: test: Use py3.5 bytes::hex() method
-BEGIN VERIFY SCRIPT-
sed -i -e "s/def bytes_to_hex_str/def b_2_x/g" $(git grep -l bytes_to_hex_str)

export RE_B_0="[^()]*"                          # match no bracket
export RE_B_1="${RE_B_0}\(${RE_B_0}\)${RE_B_0}" # match exactly one ()
export RE_B_2="${RE_B_0}\(${RE_B_1}\)${RE_B_0}" # match wrapped (())

export RE_M="(b2x|bytes_to_hex_str)\(((${RE_B_0}|${RE_B_1}|${RE_B_2})*)\)"

sed -i --regexp-extended -e "s/${RE_M}/\2.hex()/g"      $(git grep -l -E '(b2x|bytes_to_hex_str)')

sed -i --regexp-extended -e "/  +bytes_to_hex_str( as b2x)?,/d"    $(git grep -l bytes_to_hex_str)
sed -i --regexp-extended -e "s/ +bytes_to_hex_str( as b2x)?,//g"   $(git grep -l bytes_to_hex_str)
sed -i --regexp-extended -e "s/, bytes_to_hex_str( as b2x)?//g"    $(git grep -l bytes_to_hex_str)

export RE_M="(binascii\.)?hexlify\(((${RE_B_0}|${RE_B_1}|${RE_B_2})*)\).decode\(${RE_B_0}\)"

sed -i --regexp-extended -e "s/${RE_M}/\2.hex()/g" $(git grep -l hexlify -- ':(exclude)share')

sed -i --regexp-extended -e  "/from binascii import hexlify$/d" $(git grep -l hexlify -- ':(exclude)share')
sed -i --regexp-extended -e "s/(from binascii import) .*hexlify/\1 unhexlify/g" $(git grep -l hexlify -- ':(exclude)share')

sed -i -e 's/ignore-names "/ignore-names "b_2_x,/g' ./test/lint/lint-python-dead-code.sh
-END VERIFY SCRIPT-
2019-03-02 10:40:12 -05:00
MarcoFalke
789b0bbf2a
Merge #15335: Fix lack of warning of unrecognized section names
1a7ba84e11 Fix lack of warning of unrecognized section names (Akio Nakamura)

Pull request description:

  In #14708, It was introduced that to warn when unrecognized section names are exist in the config file.
  But ```m_config_sections.clear()```  in ```ArgsManager::ReadConfigStream()``` is called every time when reading each configuration file, so it can warn about only last reading file if ```includeconf``` exists.

  This PR fix lack of warning by collecting all section names by moving ```m_config_sections.clear()```  to ```ArgsManager::ReadConfigFiles()``` .
  Also add a test code to confirm this situation.

Tree-SHA512: 26aa0cbe3e4ae2e58cbe73d4492ee5cf465fd4c3e5df2c8ca7e282b627df9e637267af1e3816386b1dc6db2398b31936925ce0e432219fec3a9b3398f01e3e65
2019-03-02 09:59:18 -05:00
MarcoFalke
849f37fa22
Merge #15338: ci: Build and run tests once on freebsd
fa1d400003 cirrus ci: Inital config (MarcoFalke)

Pull request description:

  Could be activated through https://github.com/marketplace/cirrus-ci

Tree-SHA512: 3a25ad2a58249463e97a3b31122581d5d382fa1d9c830f36c72ca6211b0822950e56ea754a6bddc8f79af21d1fe3469caee9efe0e90411a9c6a59cb98c09f845
2019-03-02 09:55:09 -05:00
MarcoFalke
80112b17e7
Merge #15296: tests: Add script checking for deterministic line coverage in unit tests
43206239a8 tests: Add script checking for deterministic line coverage (practicalswift)

Pull request description:

  Add script checking for deterministic line coverage in unit tests.

  Context: #14343 ("coverage reports non-deterministic")

  When the coverage is deterministic this script can be invoked from Travis to guard against regressions, but left inactive for now.

  Output in case of determinism:

  ```
  $ contrib/test_deterministic_coverage.sh 2
  [2019-01-30 20:08:46] Measuring coverage, run #1 of 2
  [2019-01-30 20:10:45] Measuring coverage, run #2 of 2

  Coverage test passed: Deterministic coverage across 2 runs.
  ```

  Output in case of non-determinism:

  ```
  $ contrib/test_deterministic_coverage.sh 2
  [2019-01-30 20:08:46] Measuring coverage, run #1 of 2
  [2019-01-30 20:10:45] Measuring coverage, run #2 of 2

  The line coverage is non-deterministic between runs.

  The test suite must be deterministic in the sense that the set of lines executed at least
  once must be identical between runs. This is a neccessary condition for meaningful coverage
  measuring.

  --- gcovr.run-1.txt   2019-01-30 23:14:07.419418694 +0100
  +++ gcovr.run-2.txt   2019-01-30 23:15:57.998811282 +0100
  @@ -471,7 +471,7 @@
   test/crypto_tests.cpp                        270     270   100%
   test/cuckoocache_tests.cpp                   142     142   100%
   test/dbwrapper_tests.cpp                     148     148   100%
  -test/denialofservice_tests.cpp               225     225   100%
  +test/denialofservice_tests.cpp               225     224    99%   363
   test/descriptor_tests.cpp                    116     116   100%
   test/fs_tests.cpp                             24       3    12%   14,16-17,19-20,23,25-26,29,31-32,35-36,39,41-42,45-46,49,51-52
   test/getarg_tests.cpp                        111     111   100%
  @@ -585,5 +585,5 @@
   zmq/zmqpublishnotifier.h                       5       0     0%   12,31,37,43,49
   zmq/zmqrpc.cpp                                21       0     0%   16,18,20,22,33-35,38-45,49,52,56,60,62-63
   ------------------------------------------------------------------------------
  -TOTAL                                      61561   27606    44%
  +TOTAL                                      61561   27605    44%
   ------------------------------------------------------------------------------

  ```

  In this case line 363 of `test/denialofservice_tests.cpp` was executed only in the second run. Non-determinism detected!

Tree-SHA512: 03f45590e70a87146f89aa7838beeff0925d7fd303697ff03e0e69f8a5861694be5f0dd10cb0020e3e3d40c9cf662f71dfcd838f6affb31bd5212314e0a4e3a9
2019-03-02 09:46:25 -05:00
MarcoFalke
9e3122de05
Merge #15492: [rpc] remove deprecated generate method
07cae5287c [wallet] remove unused GetScriptForMining (Sjors Provoost)
8bb3e4c487 [rpc] remove deprecated generate method (Sjors Provoost)

Pull request description:

  As announced in v0.18, the wallet generate rpc method is deprecated and will be fully removed in v0.19.

  Clients should transition to using the node rpc method `generatetoaddress`.

Tree-SHA512: 9e5e913b59f3e18440b2b7b356124c7b87ad19f81a1ab6ada06a6c396b84e734895465f569296f1ba8c12abf74863bab5fd77765c9e806c239713aa83a59485f
2019-03-02 09:43:39 -05:00
MarcoFalke
e47d8a1b7c
Merge #15514: docs: Update Transifex links
10c7642a57 docs: Update Transifex links (marcuswin)

Pull request description:

  Rebased, finished and squashed #15406.

Tree-SHA512: 118f3d4701eeba998de35ffe7664bf82336e0db2c6062b050f8b91d035e4b2537b8fc790a85cfe28d90b736b6c596695d40c003cb158a22ed9e9acf711cda5cc
2019-03-02 09:23:50 -05:00
Wladimir J. van der Laan
c9985c84f9
build: Bump version to 0.18.99
Now that 0.18 branch has been split off, master is 0.18.99 (pre-0.19).

Also clean out release notes.

Tree-SHA512: ed5ca8bed37027aa852ba16f3f1e7fcd4ebaf74fa77a2a265cb33a9c710511019c577fae7a3b1e33259e245274d5cd4601d4774948396d0cf299b38ba634346a
2019-03-02 14:28:48 +01:00
marcuswin
10c7642a57
docs: Update Transifex links 2019-03-02 17:42:33 +08:00
Alistair Mann
3eac2d57b1
docs: add "sections" info to example bitcoin.conf
Most bitcoin.conf options apply to all three networks,
however some apply only to mainnet unless specified in a section.
As stands, conf file has no indication that sections are now in use
or are in some circumstances mandatory (eg, changing rpcport for testnet.)

Proposed change notifies the reader early which options are affected,
specifically adds those options affected but not already in the example,
adds brief explanation as to what's going on and provides a skeleton template for the sections themselves.
2019-03-02 17:31:15 +08:00
Wladimir J. van der Laan
37f236acc6
Merge #15510: [rpc] deriveaddresses: add range to CRPCConvertParam
9586157c0f [rpc] deriveaddresses: add range to CRPCConvertParam (Sjors Provoost)

Pull request description:

  Missing from #15497

Tree-SHA512: 469de3f896bcd3435a480685e5257c51ba895df0311329d5e5a3cb2e1894e5358324473d998ea45221776aefe8836a7af6c4f12198a36d2d10bf6761991cfd60
2019-03-02 10:11:35 +01:00
Suhas Daftuar
20e6ea259b [addrman] Improve collision logging and address nits 2019-03-01 16:15:50 -05:00
MarcoFalke
f9dbb319d2
Merge #15485: add rpc_misc.py, mv test getmemoryinfo, add test mallocinfo
f13ad1cae0 modify test for memory locked in case locking pages failed at some point (Adam Jonas)
2fa85ebd1c add rpc_misc.py, mv test getmemoryinfo, add test mallocinfo (Adam Jonas)

Pull request description:

  Creating the `rpc_misc.py` functional test file to add space for adding tests to a file that doesn't have a lot of coverage.
    - Removing the `getmemoryinfo()` smoke test from wallet basic rather than moving it to keep the wallet decoupled. Feel like testing for reasonable memory allocation values should suffice.
    - Adding coverage for `mallocinfo()`. Introduced standard lib XML parser since the function exports an XML string that describes the current state of the memory-allocation implementation in the caller.

Tree-SHA512: ced30115622916c88d1e729969ee331272ec9f2881eb36dee4bb7331bf633a6810a57fed63a0cfaf86de698edb5162e6a035efd07c89ece1df56b69d61288072
2019-03-01 13:54:20 -05:00
Adam Jonas
f13ad1cae0 modify test for memory locked in case locking pages failed at some point 2019-03-01 10:20:57 -05:00
MarcoFalke
dc251de6a5
Merge #15506: appveyor: fix cache issue and reduce dependencies build time
aeb7fbfd69 appveyor: Don't build debug libraries instead of "build and delete" (Chun Kuan Lee)

Pull request description:

  - fix  the filename typo on `appveyor.yml`. Maybe it's the reason that appveyor cache does not work properly.
  - Build release dependency libraries only. We build both release and debug on master. This could save ~5 mins.

Tree-SHA512: 68cdaeab98a658ebcb6159ee3f2d53376496d63b21c91291a95ad2495181de9bb12bd0fbf31672dbe72222b6368ce088b6a06592db365fc247c86bc5ba79905b
2019-03-01 09:47:56 -05:00
Sjors Provoost
9586157c0f
[rpc] deriveaddresses: add range to CRPCConvertParam 2019-03-01 15:34:43 +01:00
MarcoFalke
a6d7026a45
Merge #15497: rpc: Consistent range arguments in scantxoutset/importmulti/deriveaddresses
ca253f6ebf Make deriveaddresses use stop/[start,stop] notation for ranges (Pieter Wuille)
1675b7ce55 Use stop/[start,stop] notation in importmulti desc range (Pieter Wuille)
4566011631 Add support for stop/[start,stop] ranges to scantxoutset (Pieter Wuille)
6b9f45e81b Support ranges arguments in RPC help (Pieter Wuille)
7aa6a8aefb Add ParseRange function to parse args of the form int/[int,int] (Pieter Wuille)

Pull request description:

  This introduces a consistent notation for RPC arguments in `scantxoutset`, `importmulti`, and `deriveaddresses`, either:
  * `"range" : int` to just specify the end of the range
  * `"range" : [int,int]` to specify both the begin and the end of the range.

  For `scantxoutset`, this is a backward compatible new feature. For the two other RPCs, it's an incompatible change, but neither of them has been in a release so far. Because of that non-released reason, this only makes sense in 0.18, in my opinion.

  I suggest this as an alternative to #15496, which only makes `deriveaddresses` compatible with `importmulti`, but not with the existing `scantxoutset` RPC. I also think `[int,int]` is more convenient than `{"start":int,"stop":int}`.

  I realize this is technically a feature added to `scantxoutset` after the feature freeze. If desired, I'll drop the `scantxoutset` changes.

Tree-SHA512: 1cbebb90cf34f106786dbcec7afbf3f43fb8b7e46cc7e6763faf1bc1babf12375a1b3c3cf86ee83c21ed2171d99b5a2f60331850bc613db25538c38b6a056676
2019-03-01 09:13:18 -05:00
MarcoFalke
e8612adc5d
Merge #15507: test: Bump timeout on tests that timeout on windows
fa852f0e8d test: Bump timeout on tests that timeout on windows (MarcoFalke)

Pull request description:

  Those tests build a ton of blocks and time out for me on Windows with:

  ```
  test_framework.authproxy.JSONRPCException: 'generatetoaddress' RPC took longer than 60.000000 seconds. Consider using larger timeout for calls that take longer to return. (-344)

Tree-SHA512: a8fffeaddd02c051fbcc04bfac69f6ed826b8f16616e3b2e210a469d07c3e5706baab8121f1cd7ed265481de3a6197cf371513e2afbe506cf13b1dabfe3a0005
2019-03-01 09:04:36 -05:00
Sjors Provoost
07cae5287c
[wallet] remove unused GetScriptForMining 2019-03-01 13:14:08 +01:00
Pieter Wuille
ca253f6ebf Make deriveaddresses use stop/[start,stop] notation for ranges 2019-02-28 21:40:09 -08:00
MarcoFalke
fa852f0e8d
test: Bump timeout on tests that timeout on windows 2019-02-28 18:35:09 -05:00
Chun Kuan Lee
aeb7fbfd69 appveyor: Don't build debug libraries instead of "build and delete" 2019-03-01 07:24:19 +08:00
Pieter Wuille
8d220417cd Optimization: don't add txn back to mempool after 10 invalidates 2019-02-28 14:12:26 -08:00
Pieter Wuille
9ce9c37004 Prevent callback overruns in InvalidateBlock and RewindBlockIndex 2019-02-28 14:12:26 -08:00
Pieter Wuille
9bb32eb571 Release cs_main during InvalidateBlock iterations 2019-02-28 14:12:22 -08:00
Pieter Wuille
1675b7ce55 Use stop/[start,stop] notation in importmulti desc range 2019-02-28 13:05:10 -08:00
Pieter Wuille
4566011631 Add support for stop/[start,stop] ranges to scantxoutset 2019-02-28 13:01:44 -08:00
Pieter Wuille
6b9f45e81b Support ranges arguments in RPC help 2019-02-28 13:00:13 -08:00
Pieter Wuille
7aa6a8aefb Add ParseRange function to parse args of the form int/[int,int] 2019-02-28 13:00:09 -08:00
MarcoFalke
faa9b88199
fuzz: Link BasicTestingSetup (shared with unit tests) 2019-02-28 15:45:56 -05:00
MarcoFalke
fa85468cd2
test: Move main_tests to validation_tests 2019-02-28 15:44:55 -05:00
MarcoFalke
fa02b22245
test: Remove useless test_bitcoin_main.cpp 2019-02-28 15:44:02 -05:00
MarcoFalke
fab2daa026
test: Add missing LIBBITCOIN_ZMQ to test_test_bitcoin_LDADD 2019-02-28 15:42:37 -05:00
MarcoFalke
29c24b05fb
Merge #15503: msvc: Use a single file to specify the include path
4a5e52cb51 msvc: Use a single file to specify the include path (Chun Kuan Lee)

Pull request description:

  Specify the include files in 1 line and 1 file instead of 64 lines and 16 files.

  Also, this could avoid MSVC and autoconf include path inconsistency.

Tree-SHA512: e2e283913d8118d70fd94b0fb42c3b629b0d9d94aa08a4f625945992fda830c94ecdde8dc7647e28c4f35f8466de3b38cfd7ed7ca78d611612b3b49784fa3745
2019-02-28 15:13:43 -05:00
Chun Kuan Lee
4a5e52cb51 msvc: Use a single file to specify the include path 2019-03-01 02:30:07 +08:00
Wladimir J. van der Laan
20268c6d76
Merge #15489: doc: Update release process for snap package
fa466cbc50 doc: Update release process for snap package (MarcoFalke)

Pull request description:

Tree-SHA512: 44c083a9de96b8f128e32597071c5101b1f90a0396170955f4d8b21e46ad7bfad1e0e43a955ae1dd2968aa266ff6b5d4c80115a4a1e13a4073549d48278c4c90
2019-02-28 15:39:01 +01:00
Suhas Daftuar
f71fdda3bc [addrman] Ensure collisions eventually get resolved
After 40 minutes, time out a test-before-evict entry and just evict without
testing. Otherwise, if we were unable to test an entry for some reason, we
might break using feelers altogether.
2019-02-27 16:53:44 -05:00
Sjors Provoost
8bb3e4c487
[rpc] remove deprecated generate method 2019-02-27 17:41:01 +01:00
Suhas Daftuar
4991e3c813 [net] feeler connections can be made to outbound peers in same netgroup
Fixes a bug where feelers could be stuck trying to resolve a collision in the
tried table that is to an address in the same netgroup as an existing outbound peer.

Thanks to Muoi Tran for the original bug report and detailed debug logs to track
this down.
2019-02-27 09:30:13 -05:00
Wladimir J. van der Laan
a0d4e79b4d
Merge #15477: doc: Remove misleading hint in getrawtransaction
9999879f56 refactor: Use RPCHelpMan::IsValidNumArgs in getrawtransaction (MarcoFalke)
fa9ff8fe21 doc: Remove misleading hint in getrawtransaction (MarcoFalke)

Pull request description:

  For 0.18.0

  I asked this line to be added in #15159, which was wrong because getmempoolentry does not return the raw transaction hex.

Tree-SHA512: 7ac85500c8192314347b7283cd369196bb959c124863642b6c1ce73d5662b1cbe4f42ded9c374dac6657458ab70b01810caf1235dd1d2b404bf376ebf09efa69
2019-02-27 12:25:37 +01:00
Wladimir J. van der Laan
a4f9c024c6
Merge #15468: wallet: Use fsbridge::ifstream to fix Windows path issue
6ad79cbd56 wallet: Use fsbridge::ifstream to fix Windows path issue (Chun Kuan Lee)

Pull request description:

  Fix #15460

Tree-SHA512: 1dab04184608543d49c86cbcfb679d63d35cb7bf3bde2e2d9ddf25ec8977de42b7131db5e81a305f3452858079dbcf68f6ad4624c89575d3d7e5b550687fc6ad
2019-02-27 12:05:58 +01:00
Wladimir J. van der Laan
6f43ed4c5a
Merge #15462: gui: Fix async open wallet call order
a720a98301 gui: Fix async open wallet call order (João Barbosa)

Pull request description:

  Fixes #15455. Must call `OpenWalletActivity::open` asynchronously only after all connections are made to the `OpenWalletActivity` instance, otherwise signals can be missed.

Tree-SHA512: 4e5fdbd09d2ca017ed07a1813c2707c09f96275f1498779804e322e0a4dbd7dcff0c2e9cd6ec18463cd427b88b192a8d02373de9edc2b03ba5e4b8484b264417
2019-02-27 12:00:19 +01:00
MarcoFalke
fa466cbc50
doc: Update release process for snap package 2019-02-26 19:52:44 -05:00
MarcoFalke
fa45123f66
test: Add .style.yapf 2019-02-26 18:24:37 -05:00
Adam Jonas
2fa85ebd1c add rpc_misc.py, mv test getmemoryinfo, add test mallocinfo 2019-02-26 15:08:36 -05:00
Suhas Daftuar
4d834018e3 [addrman] Improve tried table collision logging 2019-02-26 14:59:30 -05:00