Commit graph

68 commits

Author SHA1 Message Date
Russell Yanofsky
fcfb952bca Improve TestNodeCLI output parsing
Parse JSONRPCException errors, and avoid JSON decode exception if RPC method
returns a plain string.
2018-01-08 15:17:14 -05:00
John Newbery
dad596fc37 [tests] Make NodeConnCB a subclass of NodeConn
This makes NodeConnCB a subclass of NodeConn, and
removes the need for the client code to know
anything about the implementation details of NodeConnCB.

NodeConn can now be swapped out for any other implementation
of a low-level connection without changing client code.
2017-11-28 12:44:16 -05:00
MarcoFalke
3bdf242fc6
Merge #11677: qa: Remove unused NodeConn members
fafdad0d4 qa: Remove unused NodeConn members (MarcoFalke)

Pull request description:

  * `ver_send` and `ver_recv` were completely unused
  * `rpc` was only used once, in p2p-segwit. Imo better only pass it to the constructor in that single test

Tree-SHA512: 7f85554d6d0fd2096516ca3c608811d5370da66cde35d8031bdc921607a7a4efdb26355896012f75f713f8df09e28d46ba46be69fd96a5898fabb1a25cbcb8ad
2017-11-14 17:44:09 -05:00
MarcoFalke
fafdad0d46 qa: Remove unused NodeConn members 2017-11-13 17:43:39 -05:00
MarcoFalke
faaa7db25e qa: Only allow disconnecting all NodeConns 2017-11-13 15:27:10 -05:00
John Newbery
5e5725cc2b [tests] Add p2p connection to TestNode
p2p connections can now be added to TestNode instances.

This commit also updates the example test to use the new
p2p interface in TestNode to demonstrate usage.

A future commit will update the existing tests to use p2p through the
TestNode.
2017-11-08 09:04:02 -05:00
John Newbery
b86c1cd208 [tests] fix TestNode.__getattr__() method 2017-11-08 09:04:02 -05:00
João Barbosa
ce379b47b9 [test] Replace check_output with low level version 2017-09-07 00:35:28 +01:00
João Barbosa
5c18a84b9a [test] Add support for custom arguments to TestNodeCLI 2017-09-07 00:35:28 +01:00
MarcoFalke
faa8d9581a [qa] TestNode: Add wait_until_stopped helper method 2017-09-06 09:41:21 -07:00
MarcoFalke
28f788e47e
Merge #11121: TestNode tidyups
7148b74dc [tests] Functional tests must explicitly set num_nodes (John Newbery)
5448a1471 [tests] don't override __init__() in individual tests (John Newbery)
6cf094a02 [tests] Avoid passing around member variables in test_framework (John Newbery)
36b626867 [tests] TestNode: separate add_node from start_node (John Newbery)
be2a2ab6a [tests] fix - use rpc_timeout as rpc timeout (John Newbery)

Pull request description:

  Some additional tidyups after the introduction of TestNode:

  - commit 1 makes TestNode use the correct rpc timeout. This should have been included in #11077
  - commit 2 separates `add_node()` from `start_node()` as originally discussed here: https://github.com/bitcoin/bitcoin/pull/10556#discussion_r121161453 with @kallewoof . The test writer no longer needs to assign to `self.nodes` when starting/stopping nodes.
  - commit 3 adds a `set_test_params()` method, so individual tests don't need to override `__init__()` and call `super().__init__()`

Tree-SHA512: 0adb030623b96675b5c29e2890ce99ccd837ed05f721d0c91b35378c5ac01b6658174aac12f1f77402e1d38b61f39b3c43b4df85c96952565dde1cda05b0db84
2017-09-01 18:47:26 +02:00
John Newbery
36b6268670 [tests] TestNode: separate add_node from start_node
Separates the act of creating a TestNode object from starting the node.
The test_framework now keeps track of its list of TestNodes, and test
writers can call start_node() and stop_node() without having to update
the self.nodes list.
2017-09-01 12:25:54 -04:00
John Newbery
be2a2ab6a6 [tests] fix - use rpc_timeout as rpc timeout 2017-09-01 12:25:50 -04:00
MarcoFalke
4ae6d0fbef
Merge #10798: [tests] [utils] test bitcoin-cli
c6ec4358a [tests] Add bitcoin_cli.py test script (John Newbery)
b23549f6e [tests] add TestNodeCLI class for calling bitcoin-cli for a node (John Newbery)

Pull request description:

  We don't test bitcoin-cli at all. That means that we can miss inconsistencies between the bitcoin-cli client and the RPC interface, such as #10698 and #10747. It also means that the various bitcoin-cli options and features are untested and regressions could be silently introduced.

  Let's fix that.

  This PR adds bitcoin-cli testing in the python functional test_framework:

  1. Add a bitcoin_cli.py test script that tests bitcoin-cli. At the moment it only tests that the result of `getinfo` is the same if you run it as an RPC or through bitcoin-cli, but can easily be extended to test additional bitcoin-cli features

  **EDIT: `--usecli` option is moved to a separate PR. This PR now only covers the bitcoin_cli.py test.**

  2. ~Add a `--usecli` option to the test framework. This changes the test to use bitcoin-cli for all RPC calls instead of using direct HTTP requests. This is somewhat experimental. It works for most tests, but there are some cases where it can't work transparently because:~
  - ~the testcase is asserting on a specific error code, and bitcoin-cli returns a different error code from the direct RPC~
  - ~we're sending a very large RPC request (eg `submitblock`) and it can't be serialized into a shell bitcoin-cli call.~

  ~I think that even though `--usecli` doesn't work on all tests, it's still a useful experimental feature. Future potential enhancements:~
  - ~enhance the framework to automatically skip tests that are known to fail with bitcoin-cli if the `--usecli` option is used.~
  - ~run a subset of tests in Travis with `-usecli`~

  This builds on and requires the `TestNode` PR #10711 . As an aside, this is a good demonstration of how tidy it is to add additional features/interfaces now that test node logic/state is encapsulated in a TestNode class.

  Addresses #10791

Tree-SHA512: a1e6be12e8e007f6f67b3d3bbcd142d835787300831eb38e6027a1ad25ca9d79c4bc99a41b19e31ee95205cba1b3b2d21a688b5909316aad70bfc2b4eb6d8a52
2017-08-24 16:04:36 -04:00
John Newbery
2b4ea520b7 [tests] fix timeout issues from TestNode
Fixes a couple of bugs from the introduction of TestNode:

- test scripts were no longer able to specify a custom timeout for
starting a node. Therefore tests with nodes that take a long time to
start up (eg pruning.py) would fail.
- the test for whether a node has failed on start up was broken
by changing 'assert x is None' to 'assert not x'. Since
subprocess.poll() can return None (indicating the node is still running)
or 0 (indicating the node exited with return code 0), this was a
regression.
2017-08-23 10:56:31 -04:00
Wladimir J. van der Laan
c1470a058f test: Increase initial RPC timeout to 60 seconds
When running the tests locally with a parallelism of 4 on an otherwise
busy system, RPC can take quite a wait to come up.

Change the timeout to 60 seconds just to be safe.
2017-08-20 15:02:51 +02:00
John Newbery
b23549f6e6 [tests] add TestNodeCLI class for calling bitcoin-cli for a node 2017-08-15 17:52:51 -04:00
John Newbery
7897338918 [tests] Introduce TestNode
TestNode is a class responsible for all state related to a bitcoind node
under test. It stores local state, is responsible for tracking the
bitcoind process and delegates unrecognised messages to the RPC
connection.

This commit changes start_nodes and stop_nodes to start and stop the
bitcoind nodes in parallel, making test setup and teardown much faster.
2017-08-15 13:12:15 -04:00