Commit graph

109 commits

Author SHA1 Message Date
MarcoFalke
f1f1605c22
Merge #10703: [tests] Allow tests to pass when stderr is non-empty
d64ac3f [tests] Allow tests to pass when stderr is non-empty (Jonas Schnelli)

Pull request description:

  Resurrect #10241 with nits addressed

  Not sure how much people want this. Would be useful for functional tests which cause bitcoind to print to stderr.

Tree-SHA512: 28caccf7818fb3ed5a38caef7f77161b1678aa9b8fd12c2d1e76032f409f0d33c40f7ac91e0c8d908df4a44fd01cf97d657a08bae50c6ff17d07f5b2e20c3a6e
2017-07-25 21:21:09 +02:00
Jonas Schnelli
44eb9d406a
[QA] Avoid running multiwallet.py twice 2017-07-21 15:59:06 +02:00
John Newbery
3707fcd94e [wallet] [tests] Add listwallets to multiwallet test 2017-07-20 17:06:29 -04:00
Jonas Schnelli
6b9faf7470
[QA] add basic multiwallet test 2017-07-17 17:42:22 +02:00
Wladimir J. van der Laan
e4f226a133
Merge #10190: [tests] mining functional tests (including regression test for submitblock)
11ba8e9 [tests] rename getblocktemplate_proposals.py to mining.py (John Newbery)
b29dd41 [tests] add test for submit block (John Newbery)
9bf0d80 [tests] run successful test in getblocktemplate first (John Newbery)
82dc597 [tests] don't build blocks manually in getblocktemplate test (John Newbery)
f82c709 [tests] clarify assertions in getblocktemplate test (John Newbery)
66c570a [tests] Don't build the coinbase manually in getblocktemplate test (John Newbery)
38b38cd [tests] getblocktemplate_proposals.py: add logging (John Newbery)
0a3a5ff [tests] Fix flake8 warnings in getblocktemplate tests (John Newbery)
32cffe6 [tests] Fix import order in getblocktemplate test (John Newbery)

Tree-SHA512: a51a57314fa1c4c4b8a7896492ec6e677b6bed12387060def34a62e9dfbee7961f71bb5553fbd70028be61ae32eccf13fd255fa9651f908e9a5e64c28f43f00e
2017-07-11 15:21:26 +02:00
Suhas Daftuar
4ccc12a54a [qa] Rewrite BIP66 functional tests
Rewrite the BIP66 functional tests to reflect height-based activation,
and move it out of the extended test suite.

Remove the unnecessary bipdersig.py test
2017-07-06 10:09:54 -04:00
MeshCollider
1d8df01412 Fix MD formatting in REST-interface.md and spelling mistake in
test_runner.py
2017-07-01 20:29:00 +12:00
Suhas Daftuar
d4f0d87b6f [qa] Rewrite BIP65 functional tests
After 122786d0e0, BIP65 activates at
a particular height (without regard to version numbers of blocks
below that height).  Rewrite the BIP65 functional tests to take
this into account, and add a test case that exercises
OP_CHECKLOCKTIMEVERIFY in a block where the soft-fork is active.

Also moves the bip65 functional test out of the extended test suite.
2017-06-30 09:04:36 -04:00
Jonas Schnelli
d64ac3f4aa [tests] Allow tests to pass when stderr is non-empty
Tests which pass with non-empty stderr are reported as "passed with
warnings"
2017-06-29 14:32:30 +01:00
Wladimir J. van der Laan
d4e551adfe
Merge #10148: Use non-atomic flushing with block replay
176c021 [qa] Test non-atomic chainstate writes (Suhas Daftuar)
d6af06d Dont create pcoinsTip until after ReplayBlocks. (Matt Corallo)
eaca1b7 Random db flush crash simulator (Pieter Wuille)
0580ee0 Adapt memory usage estimation for flushing (Pieter Wuille)
013a56a Non-atomic flushing using the blockchain as replay journal (Pieter Wuille)
b3a279c [MOVEONLY] Move LastCommonAncestor to chain (Pieter Wuille)

Tree-SHA512: 47ccc62303f9075c44d2a914be75bd6969ff881a857a2ff1227f05ec7def6f4c71c46680c5a28cb150c814999526797dc05cf2701fde1369c06169f46eccddee
2017-06-28 18:26:41 +02:00
John Newbery
11ba8e9cdd [tests] rename getblocktemplate_proposals.py to mining.py 2017-06-27 16:02:18 +01:00
Wladimir J. van der Laan
7c87a9c748
Merge #10612: The young person's guide to the test_framework
e7ba6c1 [tests] add example test (John Newbery)
76859e6 [tests] Update functional tests documentation (John Newbery)

Tree-SHA512: 74eb464e965e16466f95b9eda7d1e89a31ef1ef204dd30e1b11ddf482336f12f33fa5ca3cc733b6eaf440c46401e663585af9caca202deddb440bbadce964a62
2017-06-27 12:07:21 +02:00
John Newbery
e7ba6c16b3 [tests] add example test 2017-06-27 05:04:52 -04:00
Suhas Daftuar
176c021d08 [qa] Test non-atomic chainstate writes
Adds new functional test, dbcrash.py, which uses -dbcrashratio to exercise the
logic for recovering from a crash during chainstate flush.

dbcrash.py is added to the extended tests, as it may take ~10 minutes to run

Use _Exit() instead of exit() for crash simulation

This eliminates stderr output such as:
    terminate called without an active exception
or
    Assertion failed: (!pthread_mutex_destroy(&m)), function ~recursive_mutex, file /usr/local/include/boost/thread/pthread/recursive_mutex.hpp, line 104.

Eliminating the stderr output on crash simulation allows testing with
test_runner.py, which reports a test as failed if stderr is produced.
2017-06-26 14:48:42 -07:00
Ricardo Velhote
c07475294a
[RPC] Add an uptime command that displays the amount of time that bitcoind has been running 2017-06-25 20:25:45 +01:00
MarcoFalke
643fa0b22d
Merge #10555: [tests] various improvements to zmq_test.py
0a4912e [tests] timeout integration tests on travis after 20 minutes (John Newbery)
7c51e9f [tests] destroy zmq context in zmq_tests.py (John Newbery)
b1bac1c [tests] in zmq test, timeout if message not received (John Newbery)
5ebd5f9 [tests] tidy up zmq_test.py (John Newbery)
4a0c08f [tests] update zmq test to use correct config.ini file (John Newbery)

Tree-SHA512: 5e607af2f2dc5c73fba4b2d3890097580a7525f6a4996c7c78f01822e45e0054fd0a225ea53fce6308804e560cced6de6cf5d16137469dcf100b2e9643e08d03
2017-06-18 14:14:25 +02:00
John Newbery
0a4912e46a [tests] timeout integration tests on travis after 20 minutes 2017-06-12 09:17:18 -04:00
John Newbery
5ebd5f9e15 [tests] tidy up zmq_test.py 2017-06-08 15:18:32 -04:00
Andrew Chow
ec98b78e1e Tests for wallet encryption stuff
Added a functional test which tests the encryptwallet, walletpassphrase, walletpassphrasechange, and walletlock RPCs
2017-06-08 11:07:11 -07:00
MarcoFalke
75e898c094
Merge #10331: Share config between util and functional tests
8ad5bde Merge bctest.py into bitcoin-util-test.py (John Newbery)
95836c5 Use shared config file for functional and util tests (John Newbery)
89fcd35 Use an .ini config file for environment vars in bitcoin-util-test.py (John Newbery)
e9265df Change help_text in bitcoin-util-test.py to a docstring. (John Newbery)
ce58e93 Change bitcoin-util-test.py to use Python3 (John Newbery)

Tree-SHA512: 66dab0b4a8546aee0dfaef134a165f1447aff4c0ec335754bbc7d9e55909721c62f09cdbf4b22d02ac1fcd5a9b66780f91e1cc4d8687fae7288cc9072a23a78f
2017-06-06 23:55:24 +02:00
MarcoFalke
8e5725666b
Merge #10433: [tests] improve tmpdir structure
b040243 [tests] improve tmpdir structure (John Newbery)

Tree-SHA512: b21ad555c3c23a43087b893ad18bd2398e1df91b82c0bf1804d07fdb582600a1c339e6f4aaca58074e52f146f459943a0e492abc045b2666d4a3a7e0e455d6dd
2017-05-22 08:59:48 +02:00
John Newbery
b040243cc3 [tests] improve tmpdir structure 2017-05-18 18:11:32 -04:00
MarcoFalke
fac79e4d18 qa: Warn when specified test is not found 2017-05-14 12:39:02 +02:00
John Newbery
95836c5eba Use shared config file for functional and util tests
The functional tests and util tests both require a config file that is
generated by ./configure. This commit merges those two config
files into a single configuration file that can be shared by both tests.

The config from config.ini is put into a Namespace object to maintain
the interface with bctest.py. A future commit could change this
interface to use a dictionary instead of a namespace.
2017-05-03 14:18:30 -04:00
Wladimir J. van der Laan
35da2aeed7
Merge #10327: [tests] remove import-abort-rescan.py
981e586 [tests] remove import-abort-rescan.py (John Newbery)

Tree-SHA512: 8658d42b5ecdefbb71c55849036ee4e8e113abbc9c549730524009fb6eacc50340ffb7ef99fb35809d4abd6b304d491a618e45ff6ded1990f01f18cbcebf2b12
2017-05-03 15:42:58 +02:00
John Newbery
981e586ab8 [tests] remove import-abort-rescan.py
Reverts PR 10225
2017-05-03 09:16:42 -04:00
Wladimir J. van der Laan
2a183de0ec
Merge #9966: Control mempool persistence using a command line parameter
a750d77 Add tests for mempool persistence (John Newbery)
91c91e1 Control mempool persistence using a command line parameter. (John Newbery)

Tree-SHA512: 157d01cefd1903b8bfc5cbab42a3cc5e9c1094179bf4b64b3d34c0d4d9b976d593755bfea5c41c631cb758e1de17c6c2058c130d487d20560b7c0bafcddfa520
2017-05-03 11:03:48 +02:00
John Newbery
b8251f6337 [tests] allow zmq test to be run in out-of-tree builds 2017-05-01 15:12:49 -04:00
Wladimir J. van der Laan
e0a7e1994e
Merge #10225: [test] Add aborttrescan tests
ed60970 [test] Test abortrescan command. (Karl-Johan Alm)

Tree-SHA512: 7f617adba65a6df8fdc4b01432992926a06c4a05da4e657653436f7716301fa5d6249d77894a097737e7fb9e118925883f2425c639058b8973680339bb8e61b6
2017-04-25 16:10:50 +02:00
MarcoFalke
1b25b6df0f
Merge #10097: Move zmq test skipping logic into individual test case.
6803e09 Move zmq test skipping logic into individual test case. (John Newbery)

Tree-SHA512: 9d166b53e9acf386c4dafc860b38e2901b331a7505bba9714a2f4e3bdef68b0316c8b76fddbfb620835ddee549cf32fffb3a96a19b0799ad94f2553f55f19a35
2017-04-23 17:54:06 +02:00
MarcoFalke
c530c15180
Merge #10023: [tests] remove maxblocksinflight.py (functionality covered by other test)
5f4bcf2 [tests] Remove maxblocksinflight testcase (John Newbery)

Tree-SHA512: 827c8b12f4b52684a973bbfc508c5906e640572e22a96b9420a7aea86ad8d4aa4d6fcc2bb091f747e2fdd18c897e0456baff254bd5e3ceaf721bd3f09a2fd60b
2017-04-23 17:43:08 +02:00
Karl-Johan Alm
ed60970c83
[test] Test abortrescan command. 2017-04-21 12:51:50 +09:00
John Newbery
6803e09e6e Move zmq test skipping logic into individual test case.
This commit uses the new skip test funcationality added in
232b6665bc to skip the zmq tests if the
python zmq module is not available or if bitcoind has been built without
zmq support.

This removes the zmq-specific logic from test_runner.py. In general it's
better if test_runner.py has no knowledge of special cases for
individual tests and is a general purpose test runner.
2017-04-20 11:25:33 -04:00
Wladimir J. van der Laan
a987def4f6
Merge #10143: [net] Allow disconnectnode RPC to be called with node id
d54297f [tests] disconnect_ban: add tests for disconnect-by-nodeid (John Newbery)
5cc3ee2 [tests] disconnect_ban: remove dependency on urllib (John Newbery)
12de2f2 [tests] disconnect_ban: use wait_until instead of sleep (John Newbery)
2077fda [tests] disconnect_ban: add logging (John Newbery)
395561b [tests] disconnectban test - only use two nodes (John Newbery)
e367ad5 [tests] rename nodehandling to disconnectban (John Newbery)
d6564a2 [tests] fix nodehandling.py flake8 warnings (John Newbery)
23e6e64 Allow disconnectnode() to be called with node id (John Newbery)

Tree-SHA512: a371bb05a24a91cdb16a7ac4fbb091d5d3bf6554b29bd69d74522cb7523d3f1c5b1989d5e7b03f3fc7369fb727098dd2a549de551b731dd480c121d9517d3576
2017-04-20 11:47:22 +02:00
John Newbery
e367ad5b44 [tests] rename nodehandling to disconnectban 2017-04-19 13:47:56 -04:00
MarcoFalke
9111df9673
Merge #10219: Tests: Order Python Tests Differently
637706d Tests: Put Extended tests first when they're included (Jimmy Song)

Tree-SHA512: 0a720b2443b3e65f4ad548ecdf822468460fc4f4ecf32385dd79511a01c9ea4c94f0bf765ca593705b19baee1bae254dfcc3952da64b9c51d75b7da7abcdcd28
2017-04-18 09:26:43 +02:00
Jimmy Song
637706dc9e Tests: Put Extended tests first when they're included
* Added documentation in tests/README.md about enabling wallet, utils and daemon.
* Change ordering to make the long-running EXTENDED_TESTS go first.
2017-04-17 15:20:35 -07:00
MarcoFalke
d86bb075bf
Merge #10197: [tests] Functional test warnings
08e51c1 [tests] Remove cache directory by default when running test_runner (John Newbery)
c85b080 [test] add warnings to test_runner (John Newbery)

Tree-SHA512: 537a8a258e410102708d1e02893f3f45abe7a3a3290536249381a7dc55d74ca78322804bf34178dec1461ec1c29d8f8358c5901ddd1633f8b301b95bcbb6ce6d
2017-04-17 22:17:22 +02:00
John Newbery
a97ed80f97 [tests] test_runner - check unicode 2017-04-17 14:16:43 -04:00
John Newbery
08e51c1c03 [tests] Remove cache directory by default when running test_runner 2017-04-17 10:31:55 -04:00
John Newbery
c85b080cc7 [test] add warnings to test_runner 2017-04-17 10:31:55 -04:00
John Newbery
5f4bcf28ef [tests] Remove maxblocksinflight testcase
maxblocksinflight tested that a node would not send get_data messages
for more than 16 new blocks at the same time. bitcoin core no longer
responds to block invs with get_data, since it does headers-first
sync'ing. This test was therefore testing nothing and can be removed.

the sendheaders test script tests that bitcoin will not send get_headers
for more than 16 blocks simultaneously.
2017-04-13 11:48:02 -04:00
Wladimir J. van der Laan
e96462fbec tests: Fix test_runner return value in case of skipped test
Currently test_runner reports an error if a test case is skipped.
This is not how it should be, only failed tests should cause it to fail.
2017-04-11 19:49:34 +02:00
John Newbery
d80baaa514 fixup - align summary row correctly and make colors/glyphs globals 2017-04-07 09:15:29 -04:00
John Newbery
bb92d839d5 [tests] Add unicode symbols for tests passing/failing/skipping 2017-04-06 13:31:47 -04:00
John Newbery
63062bda1a [tests] color test results and sort alphabetically 2017-04-05 18:36:37 -04:00
Wladimir J. van der Laan
ebfd653798
Merge #10077: [qa] Add setnetworkactive smoke test
fa697b7 [qa] Add setnetworkactive smoke test (MarcoFalke)

Tree-SHA512: 7205bae16f551e93383987392702e6853cfb06d4448735815fa116385cbf5deb6c4a8f521efdd43cf3cc59fede3b3d1ffe74e662890b74bcc21b5c13ce1f20b7
2017-04-02 15:39:26 +02:00
MarcoFalke
79af9fbd8c
Merge #10096: Check that all test scripts in test/functional are being run
29d6634 Check that all test scripts in test/functional are being run (John Newbery)

Tree-SHA512: 9231d3a119632be031c51c4f7e95a8adae58489a8ec36fc967d499c0708bae2941a3bf28f11dcd4efd59141eb54c3c920f2629f5cd8a0139d30397a19591666d
2017-03-28 11:36:42 +02:00
John Newbery
55992f1302 Add --quiet option to suppress rpc-tests.py output
rpt-tests.py outputs progress information as it runs tests. This commit
adds a --quiet option that suppresses that progress output and only
prints a summary of results (and logs from failed tests).
2017-03-27 11:55:48 -04:00
John Newbery
29d6634a69 Check that all test scripts in test/functional are being run
This commit checks that all of the python files in the test/functional
directory are listed in test_runner.py.
2017-03-27 11:33:00 -04:00
Wladimir J. van der Laan
b1a4f27576
Merge #10073: Actually run assumevalid.py
717ad13 Actually run assumevalid.py. (John Newbery)

Tree-SHA512: 6fd55789be43e26347f8f5456194aa5e7beeeba3db85411dc75581b98059ca3a85485494fca36848f21c0f48096004906a0409fd76309951b5f977bab9f45b91
2017-03-27 09:50:42 +02:00
MarcoFalke
fa697b7192 [qa] Add setnetworkactive smoke test 2017-03-25 15:33:01 +01:00
Wladimir J. van der Laan
90dd9e6c4c
Merge #9946: Fix build errors if spaces in path or parent directory
b1f584d fix build if spaces in src dir path (Matthew Zipkin)

Tree-SHA512: 5834690c1f63b85ed04bb8ed411a94da04738534364d58bd9ee333ccff7129b2bbb710f31598c40123199e023da02c1745514294af3efdbb2c2e4c1320aded35
2017-03-25 12:15:06 +01:00
Matthew Zipkin
b1f584dbc1 fix build if spaces in src dir path 2017-03-24 18:22:34 -07:00
John Newbery
717ad131f6 Actually run assumevalid.py.
assumevalid was merged as part of PR 9484, but was not added to the
test_runner, so is not run even as part of the extended tests.

This commit adds assumevalid to the list of tests in test_runner. It
also clarifies the code in assumevalid considerably.
2017-03-24 18:23:36 -04:00
John Newbery
0c1ade6a4b Skip rpcbind_test if OS/network requirements are not met. 2017-03-24 00:05:30 -04:00
John Newbery
a750d77b95 Add tests for mempool persistence
Adds tests for mempool persistence as well as for the new
-persistmempool command line parameter.
2017-03-22 11:55:38 -04:00
John Newbery
232b6665bc Allow test cases to be skipped
Currently, functional test cases can either pass or fail. There are
occasions when it is helpful to skip tests, for example if the
system they are running on does not meet the requirements for the test.
The rest of the test suite can run without being marked as a failure.

This commit adds framework for tests to skip if their requirements
aren't met.
2017-03-22 10:26:02 -04:00
John Newbery
a9bd622a65 Rename test/pull-tester/rpc-tests.py to test/functional/test_runner.py 2017-03-20 10:40:31 -04:00
Renamed from test/pull-tester/rpc-tests.py (Browse further)