LBRYCommunity/lbrycrd
1
0
Fork 0
Code Issues 29 Pull requests 1 Projects Releases 32 Packages Wiki Activity
lbrycrd/contrib
History
MarcoFalke 80112b17e7
Merge #15296: tests: Add script checking for deterministic line coverage in unit tests 
43206239a8 tests: Add script checking for deterministic line coverage (practicalswift)

Pull request description:

  Add script checking for deterministic line coverage in unit tests.

  Context: #14343 ("coverage reports non-deterministic")

  When the coverage is deterministic this script can be invoked from Travis to guard against regressions, but left inactive for now.

  Output in case of determinism:

  ```
  $ contrib/test_deterministic_coverage.sh 2
  [2019-01-30 20:08:46] Measuring coverage, run #1 of 2
  [2019-01-30 20:10:45] Measuring coverage, run #2 of 2

  Coverage test passed: Deterministic coverage across 2 runs.
  ```

  Output in case of non-determinism:

  ```
  $ contrib/test_deterministic_coverage.sh 2
  [2019-01-30 20:08:46] Measuring coverage, run #1 of 2
  [2019-01-30 20:10:45] Measuring coverage, run #2 of 2

  The line coverage is non-deterministic between runs.

  The test suite must be deterministic in the sense that the set of lines executed at least
  once must be identical between runs. This is a neccessary condition for meaningful coverage
  measuring.

  --- gcovr.run-1.txt   2019-01-30 23:14:07.419418694 +0100
  +++ gcovr.run-2.txt   2019-01-30 23:15:57.998811282 +0100
  @@ -471,7 +471,7 @@
   test/crypto_tests.cpp                        270     270   100%
   test/cuckoocache_tests.cpp                   142     142   100%
   test/dbwrapper_tests.cpp                     148     148   100%
  -test/denialofservice_tests.cpp               225     225   100%
  +test/denialofservice_tests.cpp               225     224    99%   363
   test/descriptor_tests.cpp                    116     116   100%
   test/fs_tests.cpp                             24       3    12%   14,16-17,19-20,23,25-26,29,31-32,35-36,39,41-42,45-46,49,51-52
   test/getarg_tests.cpp                        111     111   100%
  @@ -585,5 +585,5 @@
   zmq/zmqpublishnotifier.h                       5       0     0%   12,31,37,43,49
   zmq/zmqrpc.cpp                                21       0     0%   16,18,20,22,33-35,38-45,49,52,56,60,62-63
   ------------------------------------------------------------------------------
  -TOTAL                                      61561   27606    44%
  +TOTAL                                      61561   27605    44%
   ------------------------------------------------------------------------------

  ```

  In this case line 363 of `test/denialofservice_tests.cpp` was executed only in the second run. Non-determinism detected!

Tree-SHA512: 03f45590e70a87146f89aa7838beeff0925d7fd303697ff03e0e69f8a5861694be5f0dd10cb0020e3e3d40c9cf662f71dfcd838f6affb31bd5212314e0a4e3a9
2019-03-02 09:46:25 -05:00
..
debian [Trivial] Update license year range to 2019 2018-12-31 04:27:59 +01:00
devtools tests: Add script checking for deterministic line coverage 2019-02-15 09:38:07 +01:00
gitian-descriptors Merge #14037: Add README.md to linux release tarballs 2018-08-31 13:05:15 +02:00
gitian-keys Add gitian PGP key for hebasto 2019-01-28 14:01:00 +02:00
init init: Use systemd automatic directory creation 2019-01-05 13:21:44 +08:00
linearize Update linearize-hashes.py 2019-02-05 19:54:44 -05:00
macdeploy build: Drop macports support 2019-01-16 12:13:59 -08:00
qos Scripts and tools & Docs: Used #!/usr/bin/env bash instead of obsolete #!/bin/bash, added linting for .sh files shebang and updated the Developer Notes. 2018-12-02 16:14:21 +02:00
seeds contrib: fix GetDesirableServiceFlags typo in seeds/README.md 2018-08-13 15:11:24 +02:00
testgen contrib: Adjust output to current test format 2018-08-27 20:14:47 +09:00
verify-commits Slight tweak to the verify-commits script directions 2019-01-31 10:15:52 -08:00
verifybinaries Obsolete #!/bin/bash shebang 2018-06-20 11:12:41 +03:00
windeploy Add "export LC_ALL=C" to all shell scripts 2018-06-14 15:27:52 +02:00
zmq ZMQ: add options to configure outbound message high water mark, aka SNDHWM 2018-10-19 07:36:13 -04:00
bitcoin-cli.bash-completion remove deprecated mentions of signrawtransaction from fundraw help 2019-02-04 10:26:46 -05:00
bitcoin-qt.pro Rename rpcconsole.ui => debugwindow.ui 2015-09-23 10:27:39 +02:00
bitcoin-tx.bash-completion bash-completion: Adapt for 0.12 and 0.13 2016-07-07 07:52:59 -04:00
bitcoind.bash-completion Fix inconsistencies and grammar in various files 2017-06-27 19:59:07 +12:00
filter-lcov.py Explicitly specify encoding when opening text files in Python code 2018-06-12 21:34:52 +02:00
gitian-build.py Add checksum in gitian build scripts for ossl 2019-01-09 23:49:01 +01:00
install_db4.sh Replace script name with special parameter 2019-02-12 23:25:54 +02:00
README.md contrib: Remove debian and rpm subfolders 2018-07-30 14:00:56 -04:00
valgrind.supp contrib: Add Valgrind suppressions file 2017-08-16 21:39:31 +02:00

README.md

Repository Tools

Developer tools

Specific tools for developers working on this repository. Contains the script github-merge.py for merging GitHub pull requests securely and signing them using GPG.

Verify-Commits

Tool to verify that every merge commit was signed by a developer using the above github-merge.py script.

Linearize

Construct a linear, no-fork, best version of the blockchain.

Qos

A Linux bash script that will set up traffic control (tc) to limit the outgoing bandwidth for connections to the Bitcoin network. This means one can have an always-on bitcoind instance running, and another local bitcoind/bitcoin-qt instance which connects to this node and receives blocks from it.

Seeds

Utility to generate the pnSeed[] array that is compiled into the client.

Build Tools and Keys

Packaging

The Debian subfolder contains the copyright file.

All other packaging related files can be found in the bitcoin-core/packaging repository.

Gitian-descriptors

Files used during the gitian build process. For more information about gitian, see the the Bitcoin Core documentation repository.

Gitian-keys

PGP keys used for signing Bitcoin Core Gitian release results.

MacDeploy

Scripts and notes for Mac builds.

Gitian-build

Script for running full Gitian builds.

Test and Verify Tools

TestGen

Utilities to generate test vectors for the data-driven Bitcoin tests.

Verify Binaries

This script attempts to download and verify the signature file SHA256SUMS.asc from bitcoin.org.