LBRYCommunity/lbrycrd
1
0
Fork 0
Code Issues 29 Pull requests 1 Projects Releases 32 Packages Wiki Activity
lbrycrd/test/lint
History
MarcoFalke 5029e94f85
Merge #14519: tests: add utility to easily profile node performance with perf 
13782b8ba8 docs: add perf section to developer docs (James O'Beirne)
58180b5fd4 tests: add utility to easily profile node performance with perf (James O'Beirne)

Pull request description:

  Adds a context manager to easily (and selectively) profile node performance during functional test execution using `perf`.

  While writing some tests, I encountered some odd bitcoind slowness. I wrote up a utility (`TestNode.profile_with_perf`) that generates performance diagnostics for a node by running `perf` during the execution of a particular region of test code.

  `perf` usage is detailed in the excellent (and sadly unmerged) https://github.com/bitcoin/bitcoin/pull/12649; all due props to @eklitzke.

  ### Example

  ```python
  with node.profile_with_perf("large-msgs"):
      for i in range(200):
          node.p2p.send_message(some_large_msg)
      node.p2p.sync_with_ping()
  ```

  This generates a perf data file in the test node's datadir (`/tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data`).

  Running `perf report` generates nice output about where the node spent most of its time while running that part of the test:

  ```bash
  $ perf report -i /tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data --stdio \
    | c++filt \
    | less

  # To display the perf.data header info, please use --header/--header-only options.
  #
  #
  # Total Lost Samples: 0
  #
  # Samples: 135  of event 'cycles:pp'
  # Event count (approx.): 1458205679493582
  #
  # Children      Self  Command          Shared Object        Symbol
  # ........  ........  ...............  ...................  ........................................................................................................................................................................................................................................................................
  #
      70.14%     0.00%  bitcoin-net      bitcoind             [.] CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
                  |
                  ---CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

      70.14%     0.00%  bitcoin-net      bitcoind             [.] CNetMessage::readData(char const*, unsigned int)
                  |
                  ---CNetMessage::readData(char const*, unsigned int)
                     CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

      35.52%     0.00%  bitcoin-net      bitcoind             [.] std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
                  |
                  ---std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
                     CNetMessage::readData(char const*, unsigned int)
                     CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

  ...
  ```

Tree-SHA512: 9ac4ceaa88818d5eca00994e8e3c8ad42ae019550d6583972a0a4f7b0c4f61032e3d0c476b4ae58756bc5eb8f8015a19a7fc26c095bd588f31d49a37ed0c6b3e
2019-02-05 17:40:16 -05:00
..
check-doc.py Update all subprocess.check_output functions in CI scripts to be Python 3.4 compatible 2019-01-18 09:36:39 -06:00
check-rpc-mappings.py Merge #13707: tests: Add usage note to check-rpc-mappings.py 2018-08-25 22:41:33 +02:00
commit-script-check.sh scripted-diff: Run scripted-diff in subshell 2018-12-04 07:57:02 -08:00
git-subtree-check.sh Add "export LC_ALL=C" to all shell scripts 2018-06-14 15:27:52 +02:00
lint-all.sh Obsolete #!/bin/bash shebang 2018-06-20 11:12:41 +03:00
lint-assertions.sh Add regression test: Don't assert(...) with side effects 2018-08-28 14:28:14 +02:00
lint-circular-dependencies.sh Merge #14350: Add WalletLocation class 2018-11-05 13:17:03 +01:00
lint-filenames.sh Merge #13496: Test: Harden lint-filenames.sh 2018-06-24 18:39:27 +02:00
lint-format-strings.py lint/format-strings: Correctly exclude escaped percent symbols 2018-12-14 15:43:52 +00:00
lint-format-strings.sh Fix warnings introduced in shellcheck v0.6.0 2019-01-16 15:47:56 +01:00
lint-include-guards.sh Obsolete #!/bin/bash shebang 2018-06-20 11:12:41 +03:00
lint-includes.sh scripted-diff: Replace boost::bind with std::bind 2018-10-20 02:29:59 +08:00
lint-locale-dependence.sh Update KNOWN_VIOLATIONS: Remove fixed violations 2018-11-06 17:32:33 +01:00
lint-logs.sh Obsolete #!/bin/bash shebang 2018-06-20 11:12:41 +03:00
lint-python-dead-code.sh tests: add utility to easily profile node performance with perf 2019-01-22 08:55:55 -05:00
lint-python-utf8-encoding.sh lint: Make sure we read the command line inputs using utf-8 decoding in python 2018-09-02 21:40:51 +08:00
lint-python.sh lint: Enable python linters via an array 2019-01-20 17:13:22 -08:00
lint-qt.sh test: Add lint to prevent SIGNAL/SLOT connect style 2018-08-21 09:43:55 +01:00
lint-rpc-help.sh rpc: Documentation fixups 2018-11-15 12:19:11 -05:00
lint-shebang.sh Scripts and tools & Docs: Used #!/usr/bin/env bash instead of obsolete #!/bin/bash, added linting for .sh files shebang and updated the Developer Notes. 2018-12-02 16:14:21 +02:00
lint-shell-locale.sh macOS fix: Work around empty (sub)expression error when using BSD grep 2018-08-31 00:52:48 +02:00
lint-shell.sh Remove no longer needed shellcheck suppressions 2019-01-16 15:47:56 +01:00
lint-spelling.ignore-words.txt Add ignored word: mut 2018-10-16 13:34:10 +02:00
lint-spelling.sh Revert "qa: Fix codespell error and have lint-spelling error instead of warn" 2018-10-16 13:33:05 +02:00
lint-tests.sh Obsolete #!/bin/bash shebang 2018-06-20 11:12:41 +03:00
lint-whitespace.sh Fix warnings introduced in shellcheck v0.6.0 2019-01-16 15:47:56 +01:00
README.md test: Move linters to test/lint, add readme 2018-05-24 12:02:15 -04:00

README.md

This folder contains lint scripts.

check-doc.py

Check for missing documentation of command line options.

commit-script-check.sh

Verification of scripted diffs.

git-subtree-check.sh

Run this script from the root of the repository to verify that a subtree matches the contents of the commit it claims to have been updated to.

To use, make sure that you have fetched the upstream repository branch in which the subtree is maintained:

Usage: git-subtree-check.sh DIR (COMMIT)

COMMIT may be omitted, in which case HEAD is used.

lint-all.sh

Calls other scripts with the lint- prefix.