Previously these functions would infinitely loop if sync failed;
now they have a default timeout of 60 seconds, after which an
AssertionError is raised.
sync_blocks() has also been improved and now compares the tip
hash of each node, rather than just using block count.
b4d24e1 Report reindexing progress in GUI (Pieter Wuille)
d3d7547 Add -reindex-chainstate that does not rebuild block index (Pieter Wuille)
fb8fad1 Optimize ActivateBestChain for long chains (Pieter Wuille)
316623f Switch reindexing to AcceptBlock in-loop and ActivateBestChain afterwards (Pieter Wuille)
d253ec4 Make ProcessNewBlock dbp const and update comment (Pieter Wuille)
fa83a5d [qa] wallet: Temporarily disable salvagewallet test (MarcoFalke)
fadd048 [doc] Link to clang-format in the developer notes (MarcoFalke)
fa72f7d [doc] Remove outdated line from listunspent RPC help, fix typo (MarcoFalke)
ac40ed7 Increase timeout waiting for pruned blk00000.dat (error10)
In my ever-growing list of test failures, I was seeing this one intermittently.
```
Running 2nd level testscript pruning.py...
Initializing test directory /tmp/testY5ypCv
Warning! This test requires 4GB of disk space and takes over 30 mins (up to 2 hours)
Mining a big blockchain of 995 blocks
Check that we haven't started pruning yet because we're below PruneAfterHeight
Success
Though we're already using more than 550MB, current usage: 587
Mining 25 more blocks should cause the first block file to be pruned
Assertion failed: blk00000.dat not pruned when it should be
File "/home/error/bitcoinxt-0.11D/qa/rpc-tests/test_framework/test_framework.py", line 118, in main
self.run_test()
File "/home/error/bitcoinxt-0.11D/qa/rpc-tests/pruning.py", line 272, in run_test
self.test_height_min()
File "/home/error/bitcoinxt-0.11D/qa/rpc-tests/pruning.py", line 94, in test_height_min
raise AssertionError("blk00000.dat not pruned when it should be")
Stopping nodes
Failed
```
After digging into the test, I found that the code is waiting 10 seconds for blk00000.dat to be deleted, and then throwing this failure if it still exists after 10 seconds.
I increased this amount, had the script print the actual time taken, and ran the test a few more times. The time taken ranged between 8 to 12 seconds. So, I feel that this timeout is too short.
After changing the timeout to 30 seconds, the test passes consistently.
(cherry picked from commit 3469911c89a48dd2fefe4d1c2a0c176256e14ee0)
We can't change "softforks", but it seems far more logical to use tags
in an object rather than using an "id" field in an array.
For example, to get the csv status before, you need to iterate the
array to find the entry with 'id' field equal to "csv":
jq '.bip9_softforks | map(select(.id == "csv"))[] | .status'
Now:
jq '.bip9_softforks.csv.status'
There is no issue with fork names being incompatible with JSON tags,
since we're selecting them ourselves.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Check the Content-Type header that is returned from the RPC server. Only
if it is `application/json` the data is supposed to be parsed as JSON.
This gives better reporting if the HTTP server happens to return an error that is
not JSON-formatted, which is the case if it happens at a lower level
before JSON-RPC kicks in.
Before: `Unexpected exception caught during testing: No JSON object could be decoded`
After: `JSONRPC error: non-JSON HTTP response with '403 Forbidden' from server`
Avoid an infinite loop in encoding, by ensuring EncodeDecimal
returns a string. round(Decimal) used to convert it to
float, but it no longer does in python 3.x. Strings are
supported since #6380, so just use that.
Replace the `bitcoin-cli -rpcwait` after spawning bitcoind
with our own loop that detects when bitcoind exits prematurely.
And if one node fails to start, stop the others.
This prevents a hang in such a case (see #7463).
Adds two RPC tests for the generatetoaddress RPC, one in the wallet, and one when the wallet is disabled.
The wallet RPC Test mines Bitcoin to another node's address and checks that that node has received the Bitcoin.
The RPC test without the wallet mines Bitcoin to an arbitrary address and checks that it works. It then mines to an arbitrary invalid address and checks that that fails.
This RPC test will test both the activation mechanism of the first versionbits soft fork as well as testing many code branches of the consensus logic for BIP's 68, 112, and 113.
Add "bip125-replaceable" output field to listtransactions and gettransaction
which indicates if an unconfirmed transaction, or any unconfirmed parent, is
signaling opt-in RBF according to BIP 125.
d11fc16 [Wallet] Call notification signal when a transaction is abandoned (Jonas Schnelli)
df0e222 Add RPC test for abandoned and conflicted transactions. (Alex Morcos)
01e06d1 Add new rpc call: abandontransaction (Alex Morcos)
9e69717 Make wallet descendant searching more efficient (Alex Morcos)
fa33d97 [walletdb] Add missing LOCK() in Recover() for dummyWallet (MarcoFalke)
fa14d99 [qa] check if wallet or blochchain maintenance changes the balance (MarcoFalke)
fa0765d [qa] Cleanup wallet.py test (MarcoFalke)
Remove necessity to call create_callback_map (as well as the function
itself) from the Python P2P test framework. Invoke the appropriate
methods directly.
- Easy to forget to call it and wonder why it doesn't work
- Simplifies the code
- This makes it easier to handle new messages in subclasses
aa4b0c2 When not filtering blocks, getdata sends more in one test (Pieter Wuille)
d41e44c Actually only use filterInventoryKnown with MSG_TX inventory messages. (Gregory Maxwell)
b6a0da4 Only use filterInventoryKnown with MSG_TX inventory messages. (Patick Strateman)
6b84935 Rename setInventoryKnown filterInventoryKnown (Patick Strateman)
e206724 Remove mruset as it is no longer used. (Gregory Maxwell)
ec73ef3 Replace setInventoryKnown with a rolling bloom filter. (Gregory Maxwell)
Redo the feerate index to be based on mining score, rather than fee.
Update mempool_packages.py to test prioritisetransaction's effect on
package scores.
Make RPC tests have a default block priority size of 50000 (the old default) so we can still use free transactions in RPC tests. When priority is eliminated, we will have to make a different change if we want to continue allowing free txs.
This replaces using inv messages to announce new blocks, when a peer requests
(via the new "sendheaders" message) that blocks be announced with headers
instead of inv's.
Since headers-first was introduced, peers send getheaders messages in response
to an inv, which requires generating a block locator that is large compared to
the size of the header being requested, and requires an extra round-trip before
a reorg can be relayed. Save time by tracking headers that a peer is likely to
know about, and send a headers chain that would connect to a peer's known
headers, unless the chain would be too big, in which case we revert to sending
an inv instead.
Based off of @sipa's commit to announce all blocks in a reorg via inv,
which has been squashed into this commit.
Rebased-by: Pieter Wuille
Reduce the default limits on maximum number of transactions and the cumulative size of those transactions in both ancestor and descendant packages to 25 txs and 101kb total size.
bip65-cltv.py is based on the earlier BIP66 soft-fork RPC test
implemented by Pieter Wuille's 819bcf9b99
bip65-cltv-p2p.py is based on the earlier BIP66 P2P test by Suhas
Daftuar's d76412b068
1) created rpc-tests.py
2) deleted rpc-tests.sh
3) travis.yml points to rpc-tests.py
4) Modified Makefile.am
5) Updated README.md
6) Added tests_config.py and deleted tests-config.sh
7) Modified configure.ac with script to set correct path in tests_config.py
af3208b Resolve issue 3166. These changes decode valid SIGHASH types on signatures in assembly (asm) representations of scriptSig scripts. This squashed commit incorporates substantial helpful feedback from jtimon, laanwj, and sipa. (mruddy)
7aac6db [QT] dump banlist to disk in case of ban/unban over QT (Jonas Schnelli)
7f90ea7 [QA] adabt QT_NO_KEYWORDS for QT ban implementation (Jonas Schnelli)
07f70b2 [QA] fix netbase tests because of new CSubNet::ToString() output (Jonas Schnelli)
4ed0510 [Qt] call DumpBanlist() when baning unbaning nodes (Philip Kaufmann)
be89292 [Qt] reenabling hotkeys for ban context menu, use different words (Jonas Schnelli)
b1189cf [Qt] adapt QT ban option to banlist.dat changes (Jonas Schnelli)
65abe91 [Qt] add sorting for bantable (Philip Kaufmann)
51654de [Qt] bantable polish (Philip Kaufmann)
cdd72cd [Qt] simplify ban list signal handling (Philip Kaufmann)
43c1f5b [Qt] remove unused timer-code from banlistmodel.cpp (Jonas Schnelli)
e2b8028 net: Fix CIDR notation in ToString() (Wladimir J. van der Laan)
9e521c1 [Qt] polish ban table (Philip Kaufmann)
607809f net: use CIDR notation in CSubNet::ToString() (Jonas Schnelli)
53caec6 [Qt] bantable overhaul (Jonas Schnelli)
f0bcbc4 [Qt] bantable fix timestamp 64bit issue (Jonas Schnelli)
6135309 [Qt] banlist, UI optimizing and better signal handling (Jonas Schnelli)
770ca79 [Qt] add context menu with unban option to ban table (Jonas Schnelli)
5f42132 [Qt] add ui signal for banlist changes (Jonas Schnelli)
ad204df [Qt] add banlist table below peers table (Jonas Schnelli)
50f0908 [Qt] add ban functions to peers window (Jonas Schnelli)
ddf98d1 Make RPC tests cope with server-side timeout between requests (Wladimir J. van der Laan)
2190ea6 rpc: Split option -rpctimeout into -rpcservertimeout and -rpcclienttimeout (Wladimir J. van der Laan)
8b2d6ed http: Disable libevent debug logging, if not explicitly enabled (Wladimir J. van der Laan)
5ce43da init: Ignore SIGPIPE (Wladimir J. van der Laan)
Python's httplib does not graciously handle disconnections from the http server, resulting in BadStatusLine errors.
See https://bugs.python.org/issue3566 "httplib persistent connections violate MUST in RFC2616 sec 8.1.4."
This was fixed in Python 3.5.
Work around it for now.
Associate with each CTxMemPoolEntry all the size/fees of descendant
mempool transactions. Sort mempool by max(feerate of entry, feerate
of descendants). Update statistics on-the-fly as transactions enter
or leave the mempool.
Also add ancestor and descendant limiting, so that transactions can
be rejected if the number or size of unconfirmed ancestors exceeds
a target, or if adding a transaction would cause some other mempool
entry to have too many (or too large) a set of unconfirmed in-
mempool descendants.
Since the introduction of the ScriptForMining callback, the mining
functions (setgenerate and generate) crash with an assertion failure
(due to a NULL pointer script returned) if the keypool is empty. Fix
this by giving a proper error.
d042854 SQUASH "Implement watchonly support in fundrawtransaction" (Matt Corallo)
428a898 SQUASH "Add have-pubkey distinction to ISMINE flags" (Matt Corallo)
6bdb474 Implement watchonly support in fundrawtransaction (Matt Corallo)
f5813bd Add logic to track pubkeys as watch-only, not just scripts (Matt Corallo)
d3354c5 Add have-pubkey distinction to ISMINE flags (Matt Corallo)
5c17059 Update importaddress help to push its use to script-only (Matt Corallo)
a1d7df3 Add importpubkey method to import a watch-only pubkey (Matt Corallo)
907a425 Add p2sh option to importaddress to import redeemScripts (Matt Corallo)
983d2d9 Split up importaddress into helper functions (Matt Corallo)
cfc3dd3 Also remove pay-2-pubkey from watch when adding a priv key (Matt Corallo)
These changes decode valid SIGHASH types on signatures in assembly (asm) representations of scriptSig scripts.
This squashed commit incorporates substantial helpful feedback from jtimon, laanwj, and sipa.
59b49cd Eliminate signed/unsigned comparison warning (Suhas Daftuar)
04b5d23 Replace sleep with syncing using pings (Suhas Daftuar)
6b1066f Ignore whitelisting during IBD for unrequested blocks. (Suhas Daftuar)
bfc30b3 Ignore unrequested blocks too far ahead of tip (Suhas Daftuar)
ec249d4 util: use locale-independent parsing in ParseDouble (Wladimir J. van der Laan)
7650449 univalue: Avoid unnecessary roundtrip through double for numbers (Wladimir J. van der Laan)
e061e27 rpc: Make ValueFromAmount always return 8 decimals (Wladimir J. van der Laan)
Since unspendable outputs can't be spent, there is no threshold at which it would be uneconomic to spend them.
This primarily targets transaction outputs with `OP_RETURN`.
---
Initially based on:
commit 9cf0ae26350033d43d5dd3c95054c0d1b1641eda
Author: zathras-crypto <zathrasc@gmail.com>
Date: Wed Mar 25 02:04:02 2015 -0700
Changes:
- cherry-picked on top of bitcoin:master
- added RPC test for fundrawtransaction
JSON makes no distinction between numbers and reals, and our code
doesn't need to do so either.
This removes VREAL, as well as its specific post-processing in
`UniValue::write`. Non-monetary amounts do not need to be forcibly
formatted with 8 decimals, so the extra roundtrip was unnecessary
(and potentially loses precision).
This removes the `conflictedbalance.sh` test as well, but that test has
been broken for a long time and isn't part of any scripts.
What it does is, IMO, sufficiently tested by other tests.
In txn_clone.py, non-essential test transaction tx2 was asserted
to be unconfirmed, but would occasionally confirm depending on
network behavior. Fix by explicitly sharing with miner and
checking for 1 confirmation.
Reduce required conflict tests from 4 to 2, by moving one invocation
each of txn_clone.py and txn_doublespend.py to the extented test
section. This saves about 15 seconds per test run for me.
177a0e4 Adding CSubNet constructor over a single CNetAddr (Jonas Schnelli)
409bccf use CBanEntry as object container for banned nodes (Jonas Schnelli)
dfa174c CAddrDB/CBanDB: change filesize variables from int to uint64_t (Jonas Schnelli)
f581d3d banlist.dat: store banlist on disk (Jonas Schnelli)
This class groups transactions that have been confirmed in blocks into buckets, based on either their fee or their priority. Then for each bucket, the class calculates what percentage of the transactions were confirmed within various numbers of blocks. It does this by keeping an exponentially decaying moving history for each bucket and confirm block count of the percentage of transactions in that bucket that were confirmed within that number of blocks.
-Eliminate txs which didn't have all inputs available at entry from fee/pri calcs
-Add dynamic breakpoints and tracking of confirmation delays in mempool transactions
-Remove old CMinerPolicyEstimator and CBlockAverage code
-New smartfees.py
-Pass a flag to the estimation code, using IsInitialBlockDownload as a proxy for when we are still catching up and we shouldn't be counting how many blocks it takes for transactions to be included.
-Add a policyestimator unit test
Does what the old txnmall.sh test did.
Creates an equivalent malleated clone and tests that SyncMetaData
syncs the accounting effects from the original transaction to the
confirmed clone.
Tests error reporting of transaction signing via RPC call "signrawtransaction".
Expected results:
Test 1: create and sign a valid raw transaction with one input:
- 1) The transaction has a complete set of signatures
- 2) No script verification error occurred
Test 2: create and sign a raw transaction with one valid, one invalid and one missing input script:
- 3) The transaction has no complete set of signatures
- 4) Two script verification errors occurred
- 5) Script verification errors have certain properties ("txid", "vout", "scriptSig", "sequence", "error")
- 6) The verification errors refer to the invalid (vin 1) and missing input (vin 2)
bba2216 RPC test for "#5418 Report missing inputs in sendrawtransaction" (Jonas Schnelli)
de8e801 Report missing inputs in sendrawtransaction (Pieter Wuille)
Previously, each NodeConnCB had its own lock to synchronize data structures
used by the testing thread and the networking thread, and NodeConn provided a
separate additional lock for synchronizing access to each send buffer. This
commit replaces those locks with a single global lock (mininode_lock) that we
use to synchronize access to all data structures shared by the two threads.
Updates comptool and maxblocksinflight to use the new synchronization
semantics, eliminating previous race conditions within comptool, and re-enables
invalidblockrequest.py in travis.
2703412 Fix default binary in p2p tests to use environment variable (Suhas Daftuar)
29bff0e Add some travis debugging for python scripts (Suhas Daftuar)
d76412b Add script manipulation tools for use in mininode testing framework (Suhas Daftuar)
b93974c Add comparison tool test runner, built on mininode (Suhas Daftuar)
6c1d1ba Python p2p testing framework (Suhas Daftuar)
script.py is modified from the code in python-bitcoinlib, and provides tools
for manipulating and creating CScript objects.
bignum.py is a dependency for script.py
script_test.py is an example test that uses the script tools for running a test
that compares the behavior of two nodes, in a comptool- style test, for each of
the test cases in the bitcoin unit test script files, script_valid.json and
script_invalid.json. (This test is very slow to run, but is a proof of concept
for how we can write tests to compare consensus-critical behavior between
different versions of bitcoind.)
bipdersig-p2p.py is another example test in the comptool framework, which tests
deployment of BIP DERSIG for a single node. It uses the script.py tools for
manipulating signatures to be non-DER compliant.
comptool.py creates a tool for running a test suite on top of the mininode p2p
framework. It supports two types of tests: those for which we expect certain
behavior (acceptance or rejection of a block or transaction) and those for
which we are just comparing that the behavior of 2 or more nodes is the same.
blockstore.py defines BlockStore and TxStore, which provide db-backed maps
between block/tx hashes and the corresponding block or tx.
blocktools.py defines utility functions for creating and manipulating blocks
and transactions.
invalidblockrequest.py is an example test in the comptool framework, which
tests the behavior of a single node when sent two different types of invalid
blocks (a block with a duplicated transaction and a block with a bad coinbase
value).
mininode.py provides a framework for connecting to a bitcoin node over the p2p
network. NodeConn is the main object that manages connectivity to a node and
provides callbacks; the interface for those callbacks is defined by NodeConnCB.
Defined also are all data structures from bitcoin core that pass on the network
(CBlock, CTransaction, etc), along with de-/serialization functions.
maxblocksinflight.py is an example test using this framework that tests whether
a node is limiting the maximum number of in-flight block requests.
This also adds support to util.py for specifying the binary to use when
starting nodes (for tests that compare the behavior of different bitcoind
versions), and adds maxblocksinflight.py to the pull tester.
1ec900a Remove broken+useless lock/unlock log prints (Matt Corallo)
352ed22 Add merkle blocks test (Matt Corallo)
59ed61b Add RPC call to generate and verify merkle blocks (Matt Corallo)
30da90d Add CMerkleBlock constructor for tx set + block and an empty one (Matt Corallo)
This adds a -prune=N option to bitcoind, which if set to N>0 will enable block
file pruning. When pruning is enabled, block and undo files will be deleted to
try to keep total space used by those files to below the prune target (N, in
MB) specified by the user, subject to some constraints:
- The last 288 blocks on the main chain are always kept (MIN_BLOCKS_TO_KEEP),
- N must be at least 550MB (chosen as a value for the target that could
reasonably be met, with some assumptions about block sizes, orphan rates,
etc; see comment in main.h),
- No blocks are pruned until chainActive is at least 100,000 blocks long (on
mainnet; defined separately for mainnet, testnet, and regtest in chainparams
as nPruneAfterHeight).
This unsets NODE_NETWORK if pruning is enabled.
Also included is an RPC test for pruning (pruning.py).
Thanks to @rdponticelli for earlier work on this feature; this is based in
part off that work.
