This adds a new CHECK_DOC check that looks for newly introduced trailing
whitespace. Existing trailing whitespace (of which there is plenty!)
will not trigger an error.
This is written in a generic way so that new lint-*.sh scripts can be
added to contrib/devtools/, as I'd like to contribute additional lint
checks in the future.
77aa9e59e test: Check RPC argument mapping (Wladimir J. van der Laan)
Pull request description:
Parse the dispatch tables from the server implementation files, and the conversion table from the client (see #10751).
Perform the following consistency checks:
- Arguments defined in conversion table, must be present in dispatch table. If not, it was probably forgotten to add them to the dispatch table, and they will not work.
- Arguments defined in conversion table must have the same names as in the dispatch table. If not, they will not work.
- All aliases for an argument must either be present in the conversion table, or not. Anything in between means an oversight and some aliases won't work.
Any of these results in an error.
It also performs a consistency check to see if the same named argument is sometimes converted, and sometimes not. E.g. one RPC call might have a 'verbose' argument that is converted,
another RPC call might have one that is not converted. This is not necessarily wrong, but points at a possible error (as well as makes the API harder to memorize) - so it is emitted as a warning (could upgrade this to error).
This test is added to travis and run when `CHECK_DOC`. Currently fails with the following output:
```
* Checking consistency between dispatch tables and vRPCConvertParams
ERROR: createrawtransaction argument 3 (named optintorbf in vRPCConvertParams) is not defined in dispatch table
ERROR: getblock argument ['verbosity', 'verbose'] has conflicts in vRPCConvertParams conversion specifier [True, False]
WARNING: conversion mismatch for argument named verbose ([('getblock', False), ('getblockheader', True), ('getmempoolancestors', True), ('getmempooldescendants', True), ('getrawmempool', True), ('getrawtransaction', True)])
```
- ~#10698 fixes the first ERROR~
- #10747 fixes the second ERROR, as well as the WARNING
Update: #10698 was merged, leaving:
```
* Checking consistency between dispatch tables and vRPCConvertParams
ERROR: getblock argument ['verbosity', 'verbose'] has conflicts in vRPCConvertParams conversion specifier [True, False]
WARNING: conversion mismatch for argument named verbose ([('getblock', False), ('getblockheader', True), ('getmempoolancestors', True), ('getmempooldescendants', True), ('getrawmempool', True), ('getrawtransaction', True)])
```
Tree-SHA512: feabebfbeda5d4613b2b9d5265aa6bde4e1a0235297ffd48fa415ad7edc531d9ed7913fe76d191ac60d481a915a326f216bc93de3c671e45e1d14e97d07dea7a
Parse the dispatch tables from the server implementation files,
and the conversion table from the client.
Perform the following consistency checks:
- Arguments defined in conversion table, must be present in dispatch
table. If not, it was probably forgotten to add them to the
dispatch table, and they will not work.
- Arguments defined in conversion table must have the same names as
in the dispatch table. If not, they will not work.
- All aliases for an argument must either be present in the
conversion table, or not. Anything in between means an oversight
and some aliases won't work.
Any of these results in an error.
It also performs a consistency check to see if the same
named argument is sometimes converted, and sometimes not. E.g.
one RPC call might have a 'verbose' argument that is converted,
another RPC call might have one that is not converted. This is not
necessarily wrong, but points at a possible error (as well as
makes the API harder to memorize) - so it is emitted as a warning
(could upgrade this to error).
Fetch the destination branch as well as PR in one go. Saves a few
seconds (as well as one ssh authentication, when using a yubikey) when
using github-merge.py.
5432fc3 Fail on commit with VERIFY SCRIPT but no scripted-diff (Pieter Wuille)
Tree-SHA512: 687b6c825f6f882f9c8e9d301bc893383710bad70216fa40b95f7e24d83a7148f9c759c3e4bd905a6cd26960829f8f6bd03106dc6c83ac312bf34ad239917018
Some minor github-merge improvements I've made over time:
User interface:
- Print merge details again before signing off, to refresh your memory -
usually I'll have done lots of different things in the shell so this
will have scrolled out a long time ago.
- Require a valid answer on the prompts. One of the requested answers
must be typed, if not, the prompt will re-ask. This prevents
accidentally rejecting.
Efficiency:
- Condense "accept merge" and "sign off" prompts. There's no reason to
have this as two separate prompts, both are just opportunities to skip
out on the merge, no action is performed in between.
Merging:
- Strip spaces from github title. This avoids redundant spaces
surrounding it from getting into the commit message.
This changes tree_sha512sum() to requests the objects for hashing from
git instead of from the working tree.
The change should make the process more deterministic (it hashes what
will be pushed) and hopefully avoids the frequent miscomputed SHA512's
that happen now.
bbd7579 Fix regsig checking for subkey sigs in verify-commits (Matt Corallo)
d025bc7 Allow any subkey in verify-commits (Matt Corallo)
eddc77a Add comment re: why SHA1 is disabled (Peter Todd)
d9c450f Verify Tree-SHA512s in merge commits, enforce sigs are not SHA1 (Matt Corallo)
be908a6 Fail merge if there are any symlinks (Matt Corallo)
Tree-SHA512: bb66c59cc1c6b1c86d7d8be7adb0769c6598c0e28ad927409941f30af87d390521e82fc13700ee22e92db1bd571db3e19a152ec7b2c0349c6e06f5de62c0b65f
This removes the option from the wallet to not pay a fee on "small"
transactions which spend "old" inputs.
This code is no longer worth keeping around, as almost all miners
prefer not to include transactions which pay no fee at all.
If both numeric format specifiers and "others" are used, assume we're
dealing with a Qt-formatted message. In the case of Qt formatting (see
https://doc.qt.io/qt-5/qstring.html#arg) only numeric formats are
replaced at all. This means "(percentage: %1%)" is valid (which was
introduced in #9461), without needing any kind of escaping that would be
necessary for strprintf. Without this, this function would wrongly
detect '%)' as a printf format specifier.
Three categories of modifications:
1)
1 instance of 'The Bitcoin Core developers \n',
1 instance of 'the Bitcoin Core developers\n',
3 instances of 'Bitcoin Core Developers\n', and
12 instances of 'The Bitcoin developers\n'
are made uniform with the 443 instances of 'The Bitcoin Core developers\n'
2)
3 instances of 'BitPay, Inc\.\n' are made uniform with the other 6
instances of 'BitPay Inc\.\n'
3)
4 instances where there was no '(c)' between the 'Copyright' and the year
where it deviates from the style of the local directory.
62c2915 build: supply `-Wl,--high-entropy-va` (Wladimir J. van der Laan)
9a75d29 devtools: Check for high-entropy ASLR in 64-bit PE executables (Wladimir J. van der Laan)
d19583f improved gen-manpages.sh, includes bitcoin-tx and strips commit tag, now also runs binaries from build dir by default, added variables for more control (nomnombtc)
09546ca regenerated all manpages with commit tag stripped, also add bitcoin-tx (nomnombtc)
ae6e754 change help string --enable-man to --disable-man (nomnombtc)
a32c102 add conditional for --enable-man, default is yes (nomnombtc)
dc84b6f add doc/man to subdir if configure flag --enable-man is set (nomnombtc)
00dba72 add doc/man/Makefile.am to include manpages (nomnombtc)
eb5643b add autogenerated manpages by help2man (nomnombtc)
6edf2fd add gen-manpages.sh description to README.md (nomnombtc)
d2cd9c0 add script to generate manpages with help2man (nomnombtc)
Three subcommands to this script:
1) ./copyright_header.py report
Examines git-tracked files with extensions that match:
INCLUDE = ['*.h', '*.cpp', '*.cc', '*.c', '*.py']
Helps to:
-> Identify source files without copyright
-> Identify source files added with something other than "The Bitcoin Core
developers" holder so we can be sure it is appropriate
-> Identify unintentional typos in the copyright line
2) ./copyright_header.py update
Replaces fix-copyright-headers.py. It does file editing in native python
rather than subprocessing out to perl as was the case with
fix-copyright-headers.py. It also shares code with the 'report' functions.
3) ./copyright_header.py insert
Inserts a copyright header into a source file with the proper format and
dates.
fafe7b3 contrib: Make fix-copyright-headers.py more portable (MarcoFalke)
fa27c0a [doc] Fix typos in comments, doxygen: Fix comment syntax (MarcoFalke)
fabfd5d [qa] pull-tester: Don't mute zmq ImportError (MarcoFalke)
67a5502 init: Fix typo in help message for -whitelistforcerelay (Wladimir J. van der Laan)
- create a script to handle split debug. This will also eventually need to check
targets, and use dsymutil for osx.
- update config.guess/config.sub for bdb for aarch64.
- temporarily disable symbol checks for arm/aarch64
- quit renaming to linux32/linux64 and use the host directly
This also adds a hack to work around an Ubuntu bug in the gcc-multilib package:
https://bugs.launchpad.net/ubuntu/+source/gcc-defaults-armhf-cross/+bug/1347820
The problem is that gcc-multilib conflicts with the aarch toolchain.
gcc-multilib installs a symlink that points
/usr/include/asm -> /usr/include/x86_64-linux-gnu/asm.
Without this link, gcc -m32 can't find asm/errno.h (and others), since
/usr/include/x86_64-linux-gnu isn't in its default include path. But
/usr/include/i386-linux-gnu is (though it doesn't exist on disk).
So work around the problem by linking
/usr/include/i386-linux-gnu/asm -> /usr/include/x86_64-linux-gnu/asm.
The symlink fix is actually quite reasonable, but echoing the password into
sudo is nasty, and should probably be addressed in gitian itself. It makes more
sense to enable passwordless sudo for the build user by default.
As we are already using the API to retrieve the pull request
title, also retrieve the base branch.
This makes sure that pull requests for 0.12 automatically end up in
0.12, and pull requests for master automatically end up in master,
and so on.
It is still possible to override the branch from the command line
or using the `githubmerge.branch` git option.
Ubuntu 16.04 "xenial xerus" does not come with Python 2.x by default.
It is possible to install a python-2.7 package, but this has its own
problem: no `python` or `python2` symlink (see #7717).
This fixes the following scripts to work with python 3:
- `make check` (bctest,py, bitcoin-util-test.py)
- `make translate` (extract_strings_qt.py)
- `make symbols-check` (symbol-check.py)
- `make security-check` (security-check.py)
Explicitly call the python commands using $(PYTHON) instead
of relying on the interpreter line at the top of the scripts.
This makes github-merge.py the first developer tool to go
all Python 3 (for context see #7717).
The changes are straightforward as the script already was
`from __future__ import division,print_function,unicode_literals`.
However urllib2 changed name, and json will only accept unicode data not
bytes.
This retains py2 compatibility for now: not strictly necessary
as it's not used by the build system - but it was easy.
Use 'utf-8' instead of the Python 2 default of 'ascii' to encode/decode
commit messages.
This can be removed when switching to Python 3, as 'utf-8' is the
default there.
Necessary for merging #7422 due to the ฿ in ฿tcDrak.
This is meant to be a direct translation of the bash script,
with the difference that it retrieves the PR title from github,
thus creating pull messages like:
Merge #12345: Expose transaction temperature over RPC
------------------------------------------------------------------------
r249567 | djasper | 2015-10-07 19:00:20 +0200 (Wed, 07 Oct 2015) | 2 lines
clang-format: Add include sorting capabilities to sublime, emacs and
clang-format-diff.py.
------------------------------------------------------------------------
r231926 | djasper | 2015-03-11 15:58:38 +0100 (Wed, 11 Mar 2015) | 3 lines
clang-format: Recognize the .ts (TypeScript) extension as JavaScript.
Patch by Martin Probst. Thank you.
------------------------------------------------------------------------
r223685 | djasper | 2014-12-08 20:39:03 +0100 (Mon, 08 Dec 2014) | 1 line
clang-format: Make clang-format-diff.py format java files.
------------------------------------------------------------------------
r221990 | djasper | 2014-11-14 14:27:28 +0100 (Fri, 14 Nov 2014) | 4 lines
clang-format: Give clang-format-diff.py a -v option.
With it, it prints the file being formatted. Apparently people are
formatting thousands of files and some progress indication is helpful.
------------------------------------------------------------------------
r216945 | ed | 2014-09-02 22:59:13 +0200 (Tue, 02 Sep 2014) | 6 lines
Use /usr/bin/env python instead of /usr/bin/python.
On operating systems like the BSDs, it is typically the case that
/usr/bin/python does not exist. We should therefore use /usr/bin/env
instead. This is also done in various other scripts in tools/.
------------------------------------------------------------------------
r208766 | djasper | 2014-05-14 11:36:11 +0200 (Wed, 14 May 2014) | 1 line
clang-format: Add clang-format-diff usage examples for SVN.
------------------------------------------------------------------------
r199750 | djasper | 2014-01-21 16:40:01 +0100 (Tue, 21 Jan 2014) | 3 lines
clang-format: Enable formatting for .proto and .protodevel files.
Support for protocol buffer files seems complete enough.
------------------------------------------------------------------------
r197668 | djasper | 2013-12-19 11:21:37 +0100 (Thu, 19 Dec 2013) | 1 line
Fix usage description of clang-format-diff.py.
------------------------------------------------------------------------
r197608 | alp | 2013-12-18 22:34:07 +0100 (Wed, 18 Dec 2013) | 7 lines
clang-format-diff.py: fix -regex/-iregex matching
While debating the finer points of file extension matching, we somehow missed
the bigger problem that the current code will match anything starting with the
default or user-specified pattern (e.g. lit.site.cfg.in).
Fix this by doing what find(1) does, implicitly wrapping the pattern with ^$.
------------------------------------------------------------------------
r197542 | alp | 2013-12-18 01:58:58 +0100 (Wed, 18 Dec 2013) | 3 lines
clang-format-diff.py: add the OpenCL file extension
It's handled correctly as a C-family language.
------------------------------------------------------------------------
r197378 | alexfh | 2013-12-16 11:57:30 +0100 (Mon, 16 Dec 2013) | 14 lines
Added -iregex for case-insensitive regex to filter file names.
Summary:
-regex and -iregex both mimic options of the find utility.
Made the default list of extensions case-insensitive, so that it's not only C
and CPP extensions are accepted in upper case.
Reviewers: djasper
Reviewed By: djasper
CC: cfe-commits
Differential Revision: http://llvm-reviews.chandlerc.com/D2415
------------------------------------------------------------------------
r196917 | alp | 2013-12-10 14:51:53 +0100 (Tue, 10 Dec 2013) | 10 lines
clang-format-diff.py: Support -regex filter and more filename extensions
Add support for more filename extensions based on the list in the clang
plus JavaScript.
Also adds a -regex option so users can override defaults if they have unusual
file extensions or want to format everything in the diff.
Keeping with tradition the flag is modelled on Unix conventions, this time
matching the semantics of find(1).
------------------------------------------------------------------------
r196484 | alp | 2013-12-05 09:14:54 +0100 (Thu, 05 Dec 2013) | 4 lines
clang-format-diff.py: pass through errors to stderr, not stdout
Also use write() for unified diff output to avoid further processing by the
print function (e.g. trailing newline).
------------------------------------------------------------------------
r196336 | alp | 2013-12-04 01:48:22 +0100 (Wed, 04 Dec 2013) | 3 lines
clang-format-diff.py: Fix 'beintroduced' in help output
Also update docs to reflect recently changed -i inplace edit behaviour.
------------------------------------------------------------------------
r192505 | alexfh | 2013-10-11 23:32:01 +0200 (Fri, 11 Oct 2013) | 17 lines
Changed clang-format-diff.py to output diff by default. Added -i option to apply changes to files instead.
Summary:
"svn diff|clang-format-diff.py" will just output the diff.
Now it's possible to use:
svn diff|clang-format-diff.py|patch -p0
as an equivalent to:
svn diff|clang-format-diff.py -i
;)
Reviewers: djasper
Reviewed By: djasper
CC: cfe-commits
Differential Revision: http://llvm-reviews.chandlerc.com/D1840
------------------------------------------------------------------------
r192184 | djasper | 2013-10-08 17:54:36 +0200 (Tue, 08 Oct 2013) | 7 lines
clang-format: Don't exit with failure on empty files.
Also let clang-format-diff.py detect errors based on clang-format's
return code. Otherwise messages like "Can't find usable .clang-format,
falling back to LLVM style" can make it fail, which might be undesired.
Patch by Alp Toker. Thank you!
------------------------------------------------------------------------
r191820 | djasper | 2013-10-02 15:59:03 +0200 (Wed, 02 Oct 2013) | 18 lines
clang-format: Fix clang-format-diff.py according to diff specification.
Patch by Alp Toker. Many thanks!
Original descriptions:
clang-format-diff incorrectly modifies unchanged lines due to an error
in diff parsing.
The unified diff format has a default line change count of 1, and 0 may
be specified to indicate that no lines have been added. This patch
updates the parser to accurately reflect the diff specification.
This also has the benefit of stabilising the operation so it will
produce the same output when run multiple times on the same changeset,
which was previously not the case.
No tests added because this script is not currently tested (though we
should look into that!)
------------------------------------------------------------------------
r191137 | djasper | 2013-09-21 12:05:02 +0200 (Sat, 21 Sep 2013) | 3 lines
Fix clang-format-diff.py to accept -style again.
Copy and paste error in r190935..
------------------------------------------------------------------------
r190935 | djasper | 2013-09-18 14:14:09 +0200 (Wed, 18 Sep 2013) | 3 lines
Simplify clang-format-diff.py using new clang-format options.
clang-format's -lines parameter makes this significantly easier.
------------------------------------------------------------------------
r189765 | alexfh | 2013-09-02 18:39:23 +0200 (Mon, 02 Sep 2013) | 2 lines
Added WebKit style to the BasedOnStyle handling and to the relevant help messages.
------------------------------------------------------------------------
r182923 | djasper | 2013-05-30 13:50:20 +0200 (Thu, 30 May 2013) | 4 lines
Fix default value of clang-format-diff's -p option.
This way, it has the same default as 'patch' and also the example in the
code makes more sense as it is explicitly setting -p 1.
------------------------------------------------------------------------
r179676 | djasper | 2013-04-17 09:55:02 +0200 (Wed, 17 Apr 2013) | 2 lines
Small improvements to clang-format documentation and integration
scripts.
------------------------------------------------------------------------
r179377 | djasper | 2013-04-12 15:42:36 +0200 (Fri, 12 Apr 2013) | 1 line
Fix clang-format-diff.py script.
------------------------------------------------------------------------
r179098 | djasper | 2013-04-09 17:23:04 +0200 (Tue, 09 Apr 2013) | 5 lines
Improvements to clang-format integrations.
This adds an emacs editor integration (thanks to Ami Fischman). Also
pulls out the style into a variable for the vi integration and just
uses clang-formats defaults style in clang-format-diff.py.
------------------------------------------------------------------------
r177506 | djasper | 2013-03-20 10:53:23 +0100 (Wed, 20 Mar 2013) | 1 line
Add clang-format binary to cfe.
------------------------------------------------------------------------
s
- Add new translations (finally, after a long time)
- update-translation script was not considering new translations - oops
- fixed this, also remove (nearly) empty translations
- Update translation process, it was still describing the old repository
structure
Perform the following ELF security checks:
- PIE: Check for position independent executable (PIE), allowing for address space randomization
- NX: Check that no sections are writable and executable (including the stack)
- RELRO: Check for read-only relocations, binding at startup
- Canary: Check for use of stack canary
Also add a check to symbol-check.py that checks that only the subset of
allowed libraries is imported (to avoid incompatibilities).
This reverts commit 1078fb0885 (and thus
pull #5623). It has various issues:
- Pull request names get cut off at ", see e.g. a026a56
- Merge script no longer copes with pulls that have a milestone
attached, due to a duplicate 'title' in JSON that is not handled by the
ad-hoc parsing.
- Check that image contents match pre- and post- crushing.
- Also remove use of external tool to compute sha256 in favor of hashlib.
- contrib: remove all use of shell=True in strip_pngs.py
Using `shell=True` can be a security hazard. See e.g.
https://docs.python.org/2/library/subprocess.html#subprocess.check_output
- Catch problems such as mismatched formatting characters. Remove
messages that can give problems at runtime.
- Also remove unfinished/untranslated messages, they just take up space
in the ts and waste parsing time.
Fixes#4774.
Running git version 2.1.0 on OSX (homebrew), I get
fatal: '1q': not a non-negative integer
I'm guessing git command-line parsing got more strict recently?
3a54ad9 Full translation update (Wladimir J. van der Laan)
9dd5d79 devtools: add a script to fetch and postprocess translations (Wladimir J. van der Laan)
58c01a3 qt: add transifex configuration file (Wladimir J. van der Laan)
Run this script from the root of the repository to update all translations from transifex.
It will do the following automatically:
- create a transifex configuration file
- fetch all translations
- post-process them into valid and committable format
Add a script to check that the (Linux) executables produced by gitian
only contain allowed gcc, glibc and libstdc++ version symbols. This
makes sure they are still compatible with the minimum supported Linux
distribution versions.
contrib/devtools/fix-copyright-headers.py script to be able to perform this maintenance task with ease during the rest of the year, every year. Modifications to contrib/devtools/README.md to document what fix-copyright-headers.py does.
