LBRYCommunity/lbrycrd
1
0
Fork 0
Code Issues 29 Pull requests 1 Projects Releases 32 Packages Wiki Activity

Merge #12926: Run unit tests in parallel

Browse source 
7ef9cd8 Increase entropy in test temp directory name (Pieter Wuille)
f6dfb0f Reorder travis builds (Pieter Wuille)
156db42 tests: run tests in parallel (Cory Fields)
66f3255 tests: split up actual tests and helper files (Cory Fields)

Pull request description:

  This runs the unit tests (`src/test/test_bitcoin`) in 4 separate simultaneous processes, significantly speeding up some Travis runs (over 2x for win32).

  This uses an approach by @theuni that relies on `make` as the mechanism for distributing tests over processes (through `-j`). For every test .cpp file, we search for `BOOST_FIXTURE_TEST_SUITE` or `BOOST_AUTO_TEST_SUITE`, and then invoke the test binary for just that suite (using `-t`). The (verbose) output is stored in a temporary file, and only shown in the case of failure.

  Some makefile reshuffling is necessary to avoid trying to run tests from `src/test/test_bitcoin.cpp` for example, which contains framework/utility code but no real tests.

  Finally, order the Travis jobs from slow to fast (apart from the arm/doc job which goes first, for fast failure). This should help reducing the total wall clock time before opening a PR and finishing Travis, in case where not all jobs are started simultaneously.

  This is an alternative to #12831.

Tree-SHA512: 9f82eb4ade14ac859618da533c7d9df2aa9f5592a076dcc4939beeffd109eda33f7d5480d8f50c0d8b23bf3099759e9f3a2d4c78efb5b66b04569b39b354c185
This commit is contained in:
Wladimir J. van der Laan 2018-04-10 14:24:05 +02:00
commit dd1ca9e0b3
No known key found for this signature in database
GPG key ID: 1E4AED62986CD25D
3 changed files with 20 additions and 13 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
.travis.yml
View file

@ -25,14 +25,14 @@ env:
- HOST=arm-linux-gnueabihf PACKAGES="g++-arm-linux-gnueabihf python3-pip" DEP_OPTS="NO_QT=1" CHECK_DOC=1 GOAL="install" BITCOIN_CONFIG="--enable-glibc-back-compat --enable-reduce-exports" - HOST=arm-linux-gnueabihf PACKAGES="g++-arm-linux-gnueabihf python3-pip" DEP_OPTS="NO_QT=1" CHECK_DOC=1 GOAL="install" BITCOIN_CONFIG="--enable-glibc-back-compat --enable-reduce-exports"
# Win32 # Win32
- HOST=i686-w64-mingw32 DPKG_ADD_ARCH="i386" DEP_OPTS="NO_QT=1" PACKAGES="python3 nsis g++-mingw-w64-i686 wine1.6" RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-reduce-exports" - HOST=i686-w64-mingw32 DPKG_ADD_ARCH="i386" DEP_OPTS="NO_QT=1" PACKAGES="python3 nsis g++-mingw-w64-i686 wine1.6" RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-reduce-exports"
# Qt4 & system libs
- HOST=x86_64-unknown-linux-gnu PACKAGES="python3-zmq qt4-dev-tools libssl-dev libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-program-options-dev libboost-test-dev libboost-thread-dev libdb5.1++-dev libminiupnpc-dev libzmq3-dev libprotobuf-dev protobuf-compiler libqrencode-dev xvfb" NO_DEPENDS=1 NEED_XVFB=1 RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --enable-glibc-back-compat --enable-reduce-exports --with-gui=qt4 CPPFLAGS=-DDEBUG_LOCKORDER"
# 32-bit + dash
- HOST=i686-pc-linux-gnu PACKAGES="g++-multilib python3-zmq" DEP_OPTS="NO_QT=1" RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-zmq --enable-glibc-back-compat --enable-reduce-exports LDFLAGS=-static-libstdc++" USE_SHELL="/bin/dash"
# Win64 # Win64
- HOST=x86_64-w64-mingw32 DPKG_ADD_ARCH="i386" DEP_OPTS="NO_QT=1" PACKAGES="python3 nsis g++-mingw-w64-x86-64 wine1.6" RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-reduce-exports" - HOST=x86_64-w64-mingw32 DPKG_ADD_ARCH="i386" DEP_OPTS="NO_QT=1" PACKAGES="python3 nsis g++-mingw-w64-x86-64 wine1.6" RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-reduce-exports"
# 32-bit + dash
- HOST=i686-pc-linux-gnu PACKAGES="g++-multilib python3-zmq" DEP_OPTS="NO_QT=1" RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-zmq --enable-glibc-back-compat --enable-reduce-exports LDFLAGS=-static-libstdc++" USE_SHELL="/bin/dash"
# x86_64 Linux (uses qt5 dev package instead of depends Qt to speed up build and avoid timeout) # x86_64 Linux (uses qt5 dev package instead of depends Qt to speed up build and avoid timeout)
- HOST=x86_64-unknown-linux-gnu PACKAGES="python3-zmq qtbase5-dev qttools5-dev-tools protobuf-compiler libdbus-1-dev libharfbuzz-dev" DEP_OPTS="NO_QT=1 NO_UPNP=1 DEBUG=1 ALLOW_HOST_PACKAGES=1" RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-zmq --with-gui=qt5 --enable-glibc-back-compat --enable-reduce-exports CPPFLAGS=-DDEBUG_LOCKORDER" - HOST=x86_64-unknown-linux-gnu PACKAGES="python3-zmq qtbase5-dev qttools5-dev-tools protobuf-compiler libdbus-1-dev libharfbuzz-dev" DEP_OPTS="NO_QT=1 NO_UPNP=1 DEBUG=1 ALLOW_HOST_PACKAGES=1" RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-zmq --with-gui=qt5 --enable-glibc-back-compat --enable-reduce-exports CPPFLAGS=-DDEBUG_LOCKORDER"
# Qt4 & system libs
- HOST=x86_64-unknown-linux-gnu PACKAGES="python3-zmq qt4-dev-tools libssl-dev libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-program-options-dev libboost-test-dev libboost-thread-dev libdb5.1++-dev libminiupnpc-dev libzmq3-dev libprotobuf-dev protobuf-compiler libqrencode-dev xvfb" NO_DEPENDS=1 NEED_XVFB=1 RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --enable-glibc-back-compat --enable-reduce-exports --with-gui=qt4 CPPFLAGS=-DDEBUG_LOCKORDER"
# x86_64 Linux, No wallet # x86_64 Linux, No wallet
- HOST=x86_64-unknown-linux-gnu PACKAGES="python3" DEP_OPTS="NO_WALLET=1" RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-glibc-back-compat --enable-reduce-exports" - HOST=x86_64-unknown-linux-gnu PACKAGES="python3" DEP_OPTS="NO_WALLET=1" RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-glibc-back-compat --enable-reduce-exports"
# Cross-Mac # Cross-Mac

23
src/Makefile.test.include
View file

@ -2,7 +2,6 @@
# Distributed under the MIT software license, see the accompanying # Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php. # file COPYING or http://www.opensource.org/licenses/mit-license.php.
TESTS += test/test_bitcoin
bin_PROGRAMS += test/test_bitcoin bin_PROGRAMS += test/test_bitcoin
noinst_PROGRAMS += test/test_bitcoin_fuzzy noinst_PROGRAMS += test/test_bitcoin_fuzzy
TEST_SRCDIR = test TEST_SRCDIR = test
@ -21,6 +20,11 @@ RAW_TEST_FILES =
GENERATED_TEST_FILES = $(JSON_TEST_FILES:.json=.json.h) $(RAW_TEST_FILES:.raw=.raw.h) GENERATED_TEST_FILES = $(JSON_TEST_FILES:.json=.json.h) $(RAW_TEST_FILES:.raw=.raw.h)
BITCOIN_TEST_SUITE = \
test/test_bitcoin_main.cpp \
test/test_bitcoin.h \
test/test_bitcoin.cpp
# test_bitcoin binary # # test_bitcoin binary #
BITCOIN_TESTS =\ BITCOIN_TESTS =\
test/arith_uint256_tests.cpp \ test/arith_uint256_tests.cpp \
@ -76,9 +80,6 @@ BITCOIN_TESTS =\
test/sigopcount_tests.cpp \ test/sigopcount_tests.cpp \
test/skiplist_tests.cpp \ test/skiplist_tests.cpp \
test/streams_tests.cpp \ test/streams_tests.cpp \
test/test_bitcoin.cpp \
test/test_bitcoin.h \
test/test_bitcoin_main.cpp \
test/timedata_tests.cpp \ test/timedata_tests.cpp \
test/torcontrol_tests.cpp \ test/torcontrol_tests.cpp \
test/transaction_tests.cpp \ test/transaction_tests.cpp \
@ -90,15 +91,17 @@ BITCOIN_TESTS =\
if ENABLE_WALLET if ENABLE_WALLET
BITCOIN_TESTS += \ BITCOIN_TESTS += \
wallet/test/wallet_test_fixture.cpp \
wallet/test/wallet_test_fixture.h \
wallet/test/accounting_tests.cpp \ wallet/test/accounting_tests.cpp \
wallet/test/wallet_tests.cpp \ wallet/test/wallet_tests.cpp \
wallet/test/wallet_crypto_tests.cpp \ wallet/test/wallet_crypto_tests.cpp \
wallet/test/coinselector_tests.cpp wallet/test/coinselector_tests.cpp
BITCOIN_TEST_SUITE += \
wallet/test/wallet_test_fixture.cpp \
wallet/test/wallet_test_fixture.h
endif endif
test_test_bitcoin_SOURCES = $(BITCOIN_TESTS) $(JSON_TEST_FILES) $(RAW_TEST_FILES) test_test_bitcoin_SOURCES = $(BITCOIN_TEST_SUITE) $(BITCOIN_TESTS) $(JSON_TEST_FILES) $(RAW_TEST_FILES)
test_test_bitcoin_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) $(TESTDEFS) $(EVENT_CFLAGS) test_test_bitcoin_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) $(TESTDEFS) $(EVENT_CFLAGS)
test_test_bitcoin_LDADD = test_test_bitcoin_LDADD =
if ENABLE_WALLET if ENABLE_WALLET
@ -150,7 +153,7 @@ bitcoin_test_check: $(TEST_BINARY) FORCE
bitcoin_test_clean : FORCE bitcoin_test_clean : FORCE
rm -f $(CLEAN_BITCOIN_TEST) $(test_test_bitcoin_OBJECTS) $(TEST_BINARY) rm -f $(CLEAN_BITCOIN_TEST) $(test_test_bitcoin_OBJECTS) $(TEST_BINARY)
check-local: check-local: $(BITCOIN_TESTS:.cpp=.cpp.test)
@echo "Running test/util/bitcoin-util-test.py..." @echo "Running test/util/bitcoin-util-test.py..."
$(PYTHON) $(top_builddir)/test/util/bitcoin-util-test.py $(PYTHON) $(top_builddir)/test/util/bitcoin-util-test.py
$(AM_V_at)$(MAKE) $(AM_MAKEFLAGS) -C secp256k1 check $(AM_V_at)$(MAKE) $(AM_MAKEFLAGS) -C secp256k1 check
@ -158,6 +161,10 @@ if EMBEDDED_UNIVALUE
$(AM_V_at)$(MAKE) $(AM_MAKEFLAGS) -C univalue check $(AM_V_at)$(MAKE) $(AM_MAKEFLAGS) -C univalue check
endif endif
%.cpp.test: %.cpp
@echo Running tests: `cat $< | grep "BOOST_FIXTURE_TEST_SUITE(\|BOOST_AUTO_TEST_SUITE(" | cut -d '(' -f 2 | cut -d ',' -f 1 | cut -d ')' -f 1` from $<
$(AM_V_at)$(TEST_BINARY) -l test_suite -t "`cat $< | grep "BOOST_FIXTURE_TEST_SUITE(\|BOOST_AUTO_TEST_SUITE(" | cut -d '(' -f 2 | cut -d ',' -f 1 | cut -d ')' -f 1`" > $<.log 2>&1 || (cat $<.log && false)
%.json.h: %.json %.json.h: %.json
@$(MKDIR_P) $(@D) @$(MKDIR_P) $(@D)
@{ \ @{ \

2
src/test/test_bitcoin.cpp
View file

@ -66,7 +66,7 @@ TestingSetup::TestingSetup(const std::string& chainName) : BasicTestingSetup(cha
RegisterAllCoreRPCCommands(tableRPC); RegisterAllCoreRPCCommands(tableRPC);
ClearDatadirCache(); ClearDatadirCache();
pathTemp = fs::temp_directory_path() / strprintf("test_bitcoin_%lu_%i", (unsigned long)GetTime(), (int)(InsecureRandRange(100000))); pathTemp = fs::temp_directory_path() / strprintf("test_bitcoin_%lu_%i", (unsigned long)GetTime(), (int)(InsecureRandRange(1 << 30)));
fs::create_directories(pathTemp); fs::create_directories(pathTemp);
gArgs.ForceSetArg("-datadir", pathTemp.string()); gArgs.ForceSetArg("-datadir", pathTemp.string());