lbrycrd/qa
Russell Yanofsky 8463aaa63c [qa] Increase wallet-dump RPC timeout
Increase wallet-dump RPC timeout from 30 seconds to 1 minute. This avoids a
timeout error that seemed to happen regularly (around 50% of builds) on a
particular jenkins server during the first getnewaddress RPC call made by the
test.

The failing stack trace looked like:

    Unexpected exception caught during testing: timeout('timed out',)

    File ".../bitcoin/qa/rpc-tests/test_framework/test_framework.py", line 146, in main
      self.run_test()
    File ".../bitcoin/qa/rpc-tests/wallet-dump.py", line 73, in run_test
      addr = self.nodes[0].getnewaddress()
    File ".../bitcoin/qa/rpc-tests/test_framework/coverage.py", line 49, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File ".../bitcoin/qa/rpc-tests/test_framework/authproxy.py", line 145, in __call__
      response = self._request('POST', self.__url.path, postdata.encode('utf-8'))
    File ".../bitcoin/qa/rpc-tests/test_framework/authproxy.py", line 121, in _request
      return self._get_response()
    File ".../bitcoin/qa/rpc-tests/test_framework/authproxy.py", line 160, in _get_response
      http_response = self.__conn.getresponse()
    File "/usr/lib/python3.4/http/client.py", line 1171, in getresponse
      response.begin()
    File "/usr/lib/python3.4/http/client.py", line 351, in begin
      version, status, reason = self._read_status()
    File "/usr/lib/python3.4/http/client.py", line 313, in _read_status
      line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
    File "/usr/lib/python3.4/socket.py", line 374, in readinto
      return self._sock.recv_into(b)
2016-11-07 12:04:01 -05:00
..
pull-tester Add importmulti rpc call 2016-10-19 15:17:13 +01:00
rpc-tests [qa] Increase wallet-dump RPC timeout 2016-11-07 12:04:01 -05:00
README.md [doc] - clarify statement about parallel jobs in rpc-tests.py 2016-08-29 21:01:38 -06:00

The pull-tester folder contains a script to call multiple tests from the rpc-tests folder.

Every pull request to the bitcoin repository is built and run through the regression test suite. You can also run all or only individual tests locally.

Test dependencies

Before running the tests, the following must be installed.

Unix

The python3-zmq library is required. On Ubuntu or Debian it can be installed via:

sudo apt-get install python3-zmq

OS X

pip3 install pyzmq

Running tests

You can run any single test by calling

qa/pull-tester/rpc-tests.py <testname>

Or you can run any combination of tests by calling

qa/pull-tester/rpc-tests.py <testname1> <testname2> <testname3> ...

Run the regression test suite with

qa/pull-tester/rpc-tests.py

Run all possible tests with

qa/pull-tester/rpc-tests.py -extended

By default, tests will be run in parallel. To specify how many jobs to run, append -parallel=n (default n=4).

If you want to create a basic coverage report for the rpc test suite, append --coverage.

Possible options, which apply to each individual test run:

  -h, --help            show this help message and exit
  --nocleanup           Leave bitcoinds and test.* datadir on exit or error
  --noshutdown          Don't stop bitcoinds after the test execution
  --srcdir=SRCDIR       Source directory containing bitcoind/bitcoin-cli
                        (default: ../../src)
  --tmpdir=TMPDIR       Root directory for datadirs
  --tracerpc            Print out all RPC calls as they are made
  --coveragedir=COVERAGEDIR
                        Write tested RPC commands into this directory

If you set the environment variable PYTHON_DEBUG=1 you will get some debug output (example: PYTHON_DEBUG=1 qa/pull-tester/rpc-tests.py wallet).

A 200-block -regtest blockchain and wallets for four nodes is created the first time a regression test is run and is stored in the cache/ directory. Each node has 25 mature blocks (25*50=1250 BTC) in its wallet.

After the first run, the cache/ blockchain and wallets are copied into a temporary directory and used as the initial test state.

If you get into a bad state, you should be able to recover with:

rm -rf cache
killall bitcoind

Writing tests

You are encouraged to write tests for new or existing features. Further information about the test framework and individual rpc tests is found in qa/rpc-tests.