fa1d766717 tests: Make msg_block a witness block (MarcoFalke)
fa52eb55c9 test: Remove True argument to CBlock::serialize (MarcoFalke)
Pull request description:
Unnamed arguments are confusing as to what they mean without looking up the function signature.
Since segwit is active by default in regtest, and all blocks are serialized with witness (#15664), remove the argument `with_witness=True` from all calls to `CBlock::serialize` and `BlockTransactions::serialize`.
ACKs for commit fa1d76:
laanwj:
code-review ACK fa1d766717
Tree-SHA512: 2c550646f99c9ca86a223ca988c61a730f5e6646807adeaa7174fb2424a32cea3fef8bcd3e0b12e162e7ff192877d0c02fd0654df6ee1a9b821b065707c2dcbc
fa8ced32a6 doc: Mention blocksonly in reduce-traffic.md, unhide option (MarcoFalke)
fa320de79f test: Add test for p2p_blocksonly (MarcoFalke)
fa3872e7b4 test: Format predicate source as multiline on error (MarcoFalke)
fa1dce7329 net: Rename ::fRelayTxes to ::g_relay_txes (MarcoFalke)
Pull request description:
This is de-facto no longer hidden
ACKs for commit fa8ced:
jamesob:
utACK fa8ced32a6
Tree-SHA512: 474fbdee6cbd035ed9068a066b6056c1f909ec7520be0417820fcd1672ab3069b53f55c5147968978d9258fd3a3933fe1a9ef8e4f6e14fb6ebbd79701a0a1245
a407b6fdf3 [tests] Make random seed logged and settable (John Newbery)
Pull request description:
This allows tests which use randomness to be reproducibly run on failure.
ACKs for commit a407b6:
jonatack:
re-ACK a407b6fdf3
jb55:
great! utACK a407b6fdf3
Tree-SHA512: e1e89e6e76d11ddec71a8f0f077227e4b46303f80461b170900d3f95d4dcc4187b0d1decfd63562ea970aaaf530ef032a3e64ed1669aac29033d95161855fda3
This diff has been generated with the following script, but is better
reviewed without looking at the script.
# -BEGIN VERIFY SCRIPT-
echo "Use msg_witness_block everywhere, except for tests that require msg_block"
# This could be a separate commit, but it is combined with the
# following scripts to reduce the overall diff
sed -i -e 's/msg_block/msg_witness_block/g' ./test/functional/{feature_assumevalid,feature_cltv,feature_dersig,feature_versionbits_warning,p2p_fingerprint,p2p_sendheaders,p2p_unrequested_blocks,example_test,rpc_blockchain}.py
echo "Rename msg_block to msg_no_witness_block"
# Rename msg_block to msg_no_witness_block in all tests (not the
# framework)
sed -i -e 's/msg_block/msg_no_witness_block/g' $(git grep -l msg_block ./test/functional/*.py)
# Derive msg_no_witness_block from msg_block
# Make msg_block a witness block in messages.py
patch -p1 --fuzz 0 << EOF
diff --git a/test/functional/test_framework/messages.py b/test/functional/test_framework/messages.py
index 00190e4cbd..e454ed5987 100755
--- a/test/functional/test_framework/messages.py
+++ b/test/functional/test_framework/messages.py
@@ -1133 +1133 @@ class msg_block:
- return self.block.serialize(with_witness=False)
+ return self.block.serialize()
@@ -1155 +1155 @@ class msg_generic:
-class msg_witness_block(msg_block):
+class msg_no_witness_block(msg_block):
@@ -1158,2 +1158 @@ class msg_witness_block(msg_block):
- r = self.block.serialize()
- return r
+ return self.block.serialize(with_witness=False)
@@ -1445 +1444 @@ class msg_blocktxn:
- r += self.block_transactions.serialize(with_witness=False)
+ r += self.block_transactions.serialize()
@@ -1452 +1451 @@ class msg_blocktxn:
-class msg_witness_blocktxn(msg_blocktxn):
+class msg_no_witness_blocktxn(msg_blocktxn):
@@ -1456,3 +1455 @@ class msg_witness_blocktxn(msg_blocktxn):
- r = b""
- r += self.block_transactions.serialize()
- return r
+ return self.block_transactions.serialize(with_witness=False)
EOF
# Conclude rename of msg_block to msg_no_witness_block
sed -i -e 's/msg_witness_block/msg_block/g' $(git grep -l msg_witness_block)
# -END VERIFY SCRIPT-
Unnamed arguments are confusing as to what they mean without looking up
the function signature.
Since segwit is active by default in regtest, and all blocks are
serialized with witness (#15664, c459c5f), remove the argument
`with_witness=True` from all calls to `CBlock::serialize` and
`BlockTransactions::serialize`.
This diff has been created with a script, but is better reviewed without
a scripted diff.
sed -i --regexp-extended -e 's/block(_?[2a-z]*)\.serialize\([a-z_]*=?True/block\1.serialize(/g' $(git grep -l serialize ./test)
9f9db39041 QA/mininode: Send all headers upfront in send_blocks_and_test to avoid sending an unconnected one (Luke Dashjr)
Pull request description:
While this doesn't currently trigger any problems, the network protocol does expect headers to be sent connectable in normal circumstances, and if too many are sent out of order will disconnect the peer.
ACKs for commit 9f9db3:
Tree-SHA512: 25b88718e4ba3d31aed2de7ece23fab9a0737fd6536c5e618ea8eb5a3a217dab0dffaebc4892df7993bcea7efb7c4fb5085fabebe99535b8f7fdde3c19df54ff
a014373d81 QA: Avoid re-reading config.ini unnecessarily (Luke Dashjr)
Pull request description:
BitcoinTestFramework.main already loads and stores config.ini on the object itself; just access that instead of re-reading the file to check for features
ACKs for commit a01437:
practicalswift:
utACK a014373d81
hebasto:
utACK a014373d81
Tree-SHA512: 36e3dda36cf4fae243feb1655da2891d1b78c86319be9bd9809c20054fa0cb75749370b05aa9d589a4dcab6322d8cdf4e874c5175144ed23ba63b2ed338538ca
BitcoinTestFramework.main already loads and stores config.ini on the object itself; just access that instead of re-reading the file to check for features
fafe5f0d09 test: Remove unused imports (MarcoFalke)
fa16a09215 scripted-diff: use self.sync_* methods (MarcoFalke)
faf77f9b90 test: Pass self to test_simple_bumpfee_succeeds (MarcoFalke)
fa6dc7c5c3 test: Add BitcoinTestFramework::sync_* methods (MarcoFalke)
fafe008cb4 test: Pass at most one node group to sync_all (MarcoFalke)
fa4680ed09 scripted-diff: Rename sync_blocks to send_blocks to avoid name collisions and confusion (MarcoFalke)
Pull request description:
This adds methods to the test framework that can be called by just `self.sync_*()`.
This avoids having to import the underlying util method. Also, in the default case, where all nodes are synced this avoid having to pass `self.nodes` explicitly.
So the effective changes are:
```diff
@@
-from test_framework.util import sync_blocks, sync_mempools
@@
- sync_blocks(self.nodes)
+ self.sync_blocks()
@@
- sync_mempools(self.nodes)
+ self.sync_mempools()
ACKs for commit fafe5f:
promag:
utACK fafe5f0.
jonatack:
ACK fafe5f0d09, nice simplification.
Tree-SHA512: 5c81840edf9fb3c5de2d7bf95ca36a5a8d23567cd1479a0f4044547c2080e9a3c5cf375357bc8eebb5b68829be050a171ab2512cfd47b89feed51fe3bad2cd72
fae38c3dc6 doc: Fix all typos reported by codespell (MarcoFalke)
fa9058f0ed doc: Add release notes for 15629 (MarcoFalke)
fa4a922d78 qa: Add test for missing testnet section in conf file (MarcoFalke)
dddd6f0f58 init: Throw error when network specific config is ignored (MarcoFalke)
Pull request description:
This should have no effect on mainnet users, but simplifies testing, where config settings are currently ignored with only a warning. Fix this by making it an error.
Issues:
* bitcoin client 0.17.0 ignores wallet's name (file) #14523
* Can't set custom rpcport on testnet #13777
* ...
ACKs for commit fae38c:
Tree-SHA512: 2e209526898eea6e444c803ec2666989cee4ca137492d32984998733c50a70056cb54657df8dc3027a6a0612738a8afce0bc35824b868c5f22281e00e0188530
8f5d9431a Add regtests for HTTP status codes. (Daniel Kraft)
Pull request description:
This adds explicit tests for the returned HTTP status codes to `interface_rpc.py` (for error cases) and the HTTP JSON-RPC client in general for success.
#15381 brought up discussion about the HTTP status codes in general, and the general opinion was that the current choice may not be ideal but should not be changed to preserve compatibility with existing JSON-RPC clients. Thus it makes sense to actually test the current status to ensure this desired compatibility is not broken accidentally.
ACKs for commit 8f5d94:
laanwj:
utACK 8f5d9431a3
promag:
utACK 8f5d943.
jonasschnelli:
utACK 8f5d9431a3
Tree-SHA512: 82503ccd134dd9145304e95cb6c61755f100bee27593d567cdd5c0c554d47e7b06d937456cab04107f46f4984930355db65d5e711008a0b05f2b8feec9f2950e
This adds explicit tests for the returned HTTP status codes to
interface_rpc.py (for error cases) and the HTTP JSON-RPC client in
general for success.
PR 15381 brought up discussion about the HTTP status codes in general,
and the general opinion was that the current choice may not be ideal
but should not be changed to preserve compatibility with existing
JSON-RPC clients. Thus it makes sense to actually test the current
status to ensure this desired compatibility is not broken accidentally.
fa2cdc9ac2 test: Simplify create_cache (MarcoFalke)
fa25210d62 qa: Fix wallet_txn_doublespend issue (MarcoFalke)
1111aecbb5 qa: Always refresh stale cache to be out of ibd (MarcoFalke)
fab0d85802 qa: Remove mocktime unless required (MarcoFalke)
Pull request description:
When starting a test, we are always in IBD because the timestamps on cached blocks are in the past. Usually, we solve that by generating a block at the beginning of the test.
That is clumsy and might even lead to other problems such as #15360 and https://github.com/bitcoin/bitcoin/issues/14446#issuecomment-461926598
So fix that by getting rid of mocktime and always refreshing the last block of the cache when starting the test framework.
Should fix#14446
Tree-SHA512: 6af09800f9c86131349a103af617a54551f5f3f3260d38e14e3f30fdd3d91a0feb0100c56cbb12eae4aeac5571ae4b530b16345cbb831d2670237b53351a22c1
8e4b4f683a Address test todos by removing -txindex to nodes. Originally added when updating getrawtransaction to stop searching unspent utxos. (Amiti Uttarwar)
Pull request description:
Original todos added when removing getrawtransaction default behavior of searching unspent utxos.
Tree-SHA512: d080953c3b0d2e5dca2265a15966dc25985a614c9cc86271ecd6276178ce428c85e262c24df92501695c32fed7beec0339b989f03cce91b57fb2efba201b7809
e3e1a5631e [test] functional: set cwd of nodes to tmpdir (Sjors Provoost)
Pull request description:
Any process launched by bitcoind will have `self.datadir` as its `cwd`.
Tree-SHA512: 0b311643bb96c7dc2f693774620173243b3add40bf373284695af2f0071823b23485289fd2ffe152b7f63e0bfe989b16720077cfc2ce33905f9b8e7f2630f3c0
fa178a6385 [rpc] mining: Omit uninitialized currentblockweight, currentblocktx (MarcoFalke)
Pull request description:
Previously we'd report "0", which could be mistaken for a valid number. E.g. the number of transactions is 0 or the block weight is 0, whatever that means.
Tree-SHA512: ee94ab203a329e272211b726f4c23edec4b09c650ec363b77fd59ad9264165d73064f78ebb9e11b5c2c543b73c157752410a307655560531c7d5444d203aa0ea
b651ef7e1c submitheader: more directly test missing prev block header (Gregory Sanders)
1e7f741745 remove some magic mining constants in functional tests (Gregory Sanders)
Pull request description:
The fewer magic numbers the better.
Also more directly tested a `submitheader` case of bad previous blockhash.
Tree-SHA512: 52b01a6aa199fa909eea4e9e84409a901933e545724e33149cc4132c82168199fd678809b6d94d95c9ff6ad02238a9552363620d13b8beaa5d4b67ade9ef425c
13782b8ba8 docs: add perf section to developer docs (James O'Beirne)
58180b5fd4 tests: add utility to easily profile node performance with perf (James O'Beirne)
Pull request description:
Adds a context manager to easily (and selectively) profile node performance during functional test execution using `perf`.
While writing some tests, I encountered some odd bitcoind slowness. I wrote up a utility (`TestNode.profile_with_perf`) that generates performance diagnostics for a node by running `perf` during the execution of a particular region of test code.
`perf` usage is detailed in the excellent (and sadly unmerged) https://github.com/bitcoin/bitcoin/pull/12649; all due props to @eklitzke.
### Example
```python
with node.profile_with_perf("large-msgs"):
for i in range(200):
node.p2p.send_message(some_large_msg)
node.p2p.sync_with_ping()
```
This generates a perf data file in the test node's datadir (`/tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data`).
Running `perf report` generates nice output about where the node spent most of its time while running that part of the test:
```bash
$ perf report -i /tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data --stdio \
| c++filt \
| less
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 135 of event 'cycles:pp'
# Event count (approx.): 1458205679493582
#
# Children Self Command Shared Object Symbol
# ........ ........ ............... ................... ........................................................................................................................................................................................................................................................................
#
70.14% 0.00% bitcoin-net bitcoind [.] CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
|
---CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
70.14% 0.00% bitcoin-net bitcoind [.] CNetMessage::readData(char const*, unsigned int)
|
---CNetMessage::readData(char const*, unsigned int)
CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
35.52% 0.00% bitcoin-net bitcoind [.] std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
|
---std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
CNetMessage::readData(char const*, unsigned int)
CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
...
```
Tree-SHA512: 9ac4ceaa88818d5eca00994e8e3c8ad42ae019550d6583972a0a4f7b0c4f61032e3d0c476b4ae58756bc5eb8f8015a19a7fc26c095bd588f31d49a37ed0c6b3e
