45842c3d2 Improve documentation for running verify-commits.py script (Jameson Lopp)
Pull request description:
I ran into 3 different issues while trying to run the verify-commits script for the first time and I think documenting them would help save time for future developers.
1. I was trying to just run it with "python" and didn't realize I had multiple python versions installed and this script is only syntactically valid for python 3.x.
2. I needed to import the trusted keys
3. The script was hanging because it was triggering my yubikey for signature verification
Tree-SHA512: dfc7a62972ca3de528fae3c9d420c7d2d6658767a555ebbf5f4a27c04748c35ccf8bf63bfc9f264358346de0db49bfbaf2d1540793a609d81c2d9b622ee8182c
1ca0502541 scripts: Add trusted key for Samuel Dobson (Wladimir J. van der Laan)
Pull request description:
Add trusted GPG key so that Samuel Dobson (MeshCollider), who is now wallet maintainer, can merge PRs.
Tree-SHA512: 6b2ef8ac1c8965eaaf5fc677dbf3bb7109e1f9dd4b56072a3409bcd232d5f1241c4e3d5fe962fcabb32880a4dbc49e5d6fb5e087e535a53f8d5d954d4f10db1b
688f665a5e Scripts and tools & Docs: Used #!/usr/bin/env bash instead of obsolete #!/bin/bash, added linting for .sh files shebang and updated the Developer Notes. (vim88)
Pull request description:
As it was discussed in [#13510](https://github.com/bitcoin/bitcoin/pull/13510), it is better to use `#!/usr/bin/env bash` instead of `#!/bin/bash`.
Tree-SHA512: 25f71eb9a6a0cdc91568b5c6863205c5fe095f77a69e633503a2ac7805bd9013af8538e538c0c666ce96a28e3f43ce7a8df5f08d4ff007723bb588d85674f2da
af9a9918b2 Fix detection of copyright holders (Cornelius Schumacher)
Pull request description:
Fix copyright holder detection so that `copyright_header.py report` creates a clean and accurate report:
* Fix list of copyright holders in the code
* Also detect copyrights which have a comma after the date
* Exclude directories which are git subtrees
Tree-SHA512: 7ab78618aa62c7d40b6688ddcde4a85c6fc5df8275271fa85518e146c1db90760bfafaa6036b9f6afbe887fd7e9274c913786101668573a3e289b3411aa6842f
* There is a copyright notice in `src/crypto/sha256_sse4.cpp` in the
middle of the file which contains a comma before the copyright
holder name. Correctly detect this so it's added to the report.
* Add missing copyright holders so that `copyright_header.py report`
doesn't show any unexpected copyright holder names anymore.
* Exclude files from git subtrees because they are maintained at the
original source.
a4edb168b6 ZMQ: add options to configure outbound message high water mark, aka SNDHWM (mruddy)
Pull request description:
ZMQ: add options to configure outbound message high water mark, aka SNDHWM
This is my attempt at https://github.com/bitcoin/bitcoin/pull/13315
Tree-SHA512: a4cc3bcf179776899261a97c8c4f31f35d1d8950fd71a09a79c5c064879b38e600b26824c89c4091d941502ed5b0255390882f7d44baf9e6dc49d685a86e8edb
Instead of re-naming the tarballs used for the code signing step to the generically
named tarball that is used, keep the versioned naming. Only copy them to the
correct filename when they are needed at build time.
c516c3a770 [contrib] Support ARM and RISC-V symbol check (Chun Kuan Lee)
Pull request description:
Solve the TODO in the gitian-descripter
Tree-SHA512: 8115e2958af3dde43d9d9d05f0b1b1b93b1c2aa513e771a3e4e1342a5d78af2b0e40c0bbb7e9a0d15954897317e6f5a0d80996239af3b376d5ddd527f73428ae
Ajust the outputs of gen_base58_test_vectors.py to current test format.
1. Add bech32 vector generation.
2. Add Script-fragments to hexrepr in the format expected by the test.
3. Modify the metadata.
1) Change 'isTestnet' to 'chain' (main/test/regtest)
2) Remove 'addrType'
3) Add 'tryCaseFlip' (True/False on bech32)
46f8345370 contrib: Support github pull request gitian-build (Chun Kuan Lee)
Pull request description:
- Add `-p` or `--pull` to specify the version is github pull request number
- `./gitian-build.py -bpDn test 13765` should work. This is helpful for someone want to try cross compiling.
Tree-SHA512: b474b089127ffb2a67a42d99c1888bd5812a5ba725c8915a13a01cf93bde32b5b5011314e4193fa6b231caa5de2137e709ee5aa5bf889c4ac60a97aa83caac5c
3fc20632a3 qt: Set BLOCK_CHAIN_SIZE = 220 (DrahtBot)
2b6a2f4a28 Regenerate manpages (DrahtBot)
eb7daf4d60 Update copyright headers to 2018 (DrahtBot)
Pull request description:
Some trivial maintenance to avoid having to do it again after the 0.17 branch off.
(The scripts to do this are in `./contrib/`)
Tree-SHA512: 16b2af45e0351b1c691c5311d48025dc6828079e98c2aa2e600dc5910ee8aa01858ca6c356538150dc46fe14c8819ed8ec8e4ec9a0f682b9950dd41bc50518fa
284f424d5a Fix osslsigncode compile issue in gitian-build (Chun Kuan Lee)
Pull request description:
Install libssl1.0-dev that is compatible with osslsigncode.
Fixes#13762
Verifed that this gitian descriptor file can sign 0.16.2rc2.
Tree-SHA512: 3029b86e77567a4e033b5ad95826e60df12a0486ac3c4afcac48218f5c76ba49e7f1c1307ce93ffc465ca2f24e12c401e4542929263688e4bd6521aeca3ff73b
1e60713a68 contrib: Fix test-security-check fail in Ubuntu 18.04 (Chun Kuan Lee)
Pull request description:
- Fix test-security-check fail in Ubuntu 18.04. Those flags are enabled by default, so we must specify `-no` to make the executable does 'not' have those attributes.
- Drop HIGH_ENTROPY_VA. After update our gitian system to Bionic, the compiler should support HIGH_ENTROPY_VA
Tree-SHA512: 78c1f2aae1253ddd52faa1af569b7151a503a217c7ccbe21b8004d8910c45d8a27ff04695eacbdadd7192d2c91c0d59941ca20c651dd2d5052b9999163a11ae4
6b5506a286 Fix Qt's rcc determinism for depends/gitian (Fuzzbawls)
Pull request description:
With the update to Qt 5.9 having been merged, Qt's `rcc` tool now embeds a file's last modified time in it's output. Since the build system generates temporary files for all locale translations (`*.qm` files) at build time, the resulting `qrc_bitcoin_locale.cpp` file was always being generated in a non-deterministic way.
This is a backport of https://bugreports.qt.io/browse/QTBUG-62511, which is included in Qt versions 5.11+, that allows for an environment variable (`QT_RCC_SOURCE_DATE_OVERRIDE`) to override the behavior described above. This environment variable is in turn set in the gitian descriptors, as that is where determinism is vital for release purposes.
Prior to this, the `qt_libbitcoinqt_a-qrc_bitcoin_locale.o` object file (included into `libbitcoinqt.a`) was returning a different `sha256sum` for each and every build, regardless of file contents change, thus breaking determinism in the resulting binaries.
This should fix#13731
Tree-SHA512: 174017e41f9afc3950ef54a9419de81577ec900db9aec3c78ccd3d879c6aecaaeb944fde0615b933f43e6ca9d7898a27ec071cdd0b91cb772755a3012de96725
95464c7519 doc: Improve command to generate list of authors for release notes (Mitchell Cash)
1c22cc1af1 doc: Update broken links to now point to gitian-build.py (Mitchell Cash)
Pull request description:
- Update broken links
- Improve command to generate list of authors for a release
---
I also note that it asks to ping **wumpus** on IRC, to assist in generating a list of merged pulls and sort them into categories based on labels. I tried to turn this into a simple one-liner as well (something like ``git log --merges --format="- \`%h\` %s (%an)" v0.16.0..v0.16.1``), but it didn't seem to capture everything I needed.
Would it be worthwhile **wumpus** open-sourcing his code into `contrib/devtools` so there is no single point of failure (even if it can manually be worked around).
Tree-SHA512: 8e7f0880ff07ce8fe67b74de3f2c4b78dafe2c95eefb0617fa319ff196232967cb22ee75a183a39f93bfc6e0bf85547689160139312cee5956af2c069b8a3b6a
066d2973a6 contrib: correct version check (Karl-Johan Alm)
Pull request description:
[ not(major >= 3 and minor >= 4) ] fails for '4.0':
```Python
>>> major=4
>>> minor=0
>>> if not (major >= 3 and minor >= 5):
... print('This example only works with Python 3.5 and greater')
...
This example only works with Python 3.5 and greater
```
Tree-SHA512: 41cb8c5dabe0061ead37b0d6447b699c5d5e1a5551a75dd279bb2ec6d6afa51f8b27a48e01195093db7ec37b28ff6445d59646a4a2b6dcee37776afb60eac8dc
78f06e4af7 Migrate gitian-build.sh to python (Chun Kuan Lee)
Pull request description:
Fixes#13620
- Rename Mac OSX to MacOS, rename option from 'x' to 'm'
- Fix a bug from b641f60425/contrib/gitian-build.sh (L338-L342)
Tree-SHA512: ff943055d5feca345bd17b64311374db3937d14d2f21493116fd7ab7b4cb7042480abd6a3d1d7640073b00bc0badb300e8dd32618bf73ba182df417c0633397a
37d363dd4a Tools: fix BIND_NOW check in security-check.py (Conrado Gouvea)
Pull request description:
Previously, the BIND_NOW check would work only if it was the first value in FLAGS.
Tree-SHA512: 39cd83f673a6b521803be5585ada516f2db4eede718f0c5aea3377825ed1adcefed5bbb41dd9a5f24a26f7d27116cfc81bde7e26283232593b72768c5ae3d321
86967b2e35 Add option to use docker for gitian-build.sh (Andrew Chow)
Pull request description:
Updates gitian-build.sh script for the docker virtualization method recently added to gitian-builder.
Second commit depends on https://github.com/devrandom/gitian-builder/pull/185
Tree-SHA512: 0fa50b6c1be35a7a39f4a82816369637d3d0082922ac9876b80ae8156c795dec0ff34ad384b5348777042df8ddb6a72bbaeddd7002ef80557b10e2acf46632aa
253f592909 Add stdin, stdout, stderr to ignored export list (Chun Kuan Lee)
fc6a9f2ab1 Use IN6ADDR_ANY_INIT instead of in6addr_any (Cory Fields)
908c1d7745 GCC-7 and glibc-2.27 compat code (Chun Kuan Lee)
Pull request description:
The `__divmoddi4` code was modified from https://github.com/gcc-mirror/gcc/blob/master/libgcc/libgcc2.c . I manually find the older glibc version of log2f by objdump, use `.symver` to specify the certain version.
Tree-SHA512: e8d875652003618c73e019ccc420e7a25d46f4eaff1c7a1a6bfc1770b3b46f074b368b2cb14df541b5ab124cca41dede4e28fe863a670589b834ef6b8713f9c4
47776a958b Add linter: Make sure all shell scripts opt out of locale dependence using "export LC_ALL=C" (practicalswift)
3352da8da1 Add "export LC_ALL=C" to all shell scripts (practicalswift)
Pull request description:
~~Make sure `LC_ALL=C` is set when using `grep` range expressions.~~
Make sure `LC_ALL=C` is set in all shell scripts.
From the `grep(1)` documentation:
> Within a bracket expression, a range expression consists of two characters separated by a hyphen. It matches any single character that sorts between the two characters, inclusive, using the locale's collating sequence and character set. For example, in the default C locale, `[a-d]` is equivalent to `[abcd]`. Many locales sort characters in dictionary order, and in these locales `[a-d]` is typically not equivalent to `[abcd]`; it might be equivalent to `[aBbCcDd]`, for example. To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the `LC_ALL` environment variable to the value C.
Context: [Locale issue found when reviewing #13450](https://github.com/bitcoin/bitcoin/pull/13450/files#r194877736)
Tree-SHA512: fd74d2612998f9b49ef9be24410e505d8c842716f84d085157fc7f9799d40e8a7b4969de783afcf99b7fae4f91bbb4559651f7dd6578a6a081a50bdea29f0909
c8176b3cc7 Add linter: Make sure we explicitly open all text files using UTF-8 or ASCII encoding in Python (practicalswift)
634bd97001 Explicitly specify encoding when opening text files in Python code (practicalswift)
Pull request description:
Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python.
As requested by @laanwj in #13440.
Tree-SHA512: 1651c00fe220ceb273324abd6703aee504029b96c7ef0e3029145901762c733c9b9d24927da281394fd4681a5bff774336c04eed01fafea997bb32192c334c06
e5b2cd8e75 Use python instead of slow shell script on verify-commits (Chun Kuan Lee)
Pull request description:
The cron job that runs every day would fail because of git checkout a single commit, not a branch.
#12708 introduce a method to check whether merges are clean.
However, there are four merges are not clean.
So, I add a list of merges that are dirty and ignore them.
Also, I modify the current shell script to python, it makes the script speed up a lot.
The python code `tree_sha512sum` was copied from `github-merge.py`
I've re-designed this. Now we verify all the things by default.
- Add `--disable-tree-check` option, not to check SHA-512 tree
- Add `--clean-merge NUMBER` option, only verify commits after <NUMBER> days ago
Travis running time:
|option|time|
|-|-|
|verify-commits.py|[25m47.02s(1547.02s)](https://travis-ci.org/ken2812221/bitcoin/jobs/373321423)|
|verify-commits.py --disable-tree-check|[19m10.08s(1150.08s)](https://travis-ci.org/ken2812221/bitcoin/jobs/373321423)|
|verify-commits.py --clean-merge 30|[9m18.18s(558.18s)](https://travis-ci.org/ken2812221/bitcoin/jobs/373321423)|
|verify-commits.py --disable-tree-check --clean-merge 30|[1m16.51s(76.51s)](https://travis-ci.org/ken2812221/bitcoin/jobs/373321423)|
Since the cron job always fail, I've created a respository to verify this daily.
[](https://travis-ci.org/ken2812221/bitcoin-verify-commits)
Tree-SHA512: 476bcf707d92ed3d431ca5642e013036df1506120d3dd2aa718f74240063ce856abd78f4c948336c2a6230dfe5c60c6f2d52d19bdb52d647a1c5f838eaa02e3b
fa3c910bfe test: Move linters to test/lint, add readme (MarcoFalke)
Pull request description:
This moves the checks and linters from `devtools` to a subfolder in `test`. (Motivated by my opinion that the dev tools are mostly for generating code and updating the repo whereas the linters are read-only checks.)
Also, adds a readme to clarify that checks and linters are only meant to prevent bugs and user facing issues, not merely stylistic preference or inconsistencies. (This is motivated by the diversity in developers and work flows as well as existing code styles. It would be too disruptive to change all existing code to a single style or too burdensome to force all developers to adhere to a single style. Also note that our style guide is changing, so locking in at the wrong style "too early" would only waste resources.)
Tree-SHA512: 9b10e89f2aeaf0c8a9ae248aa891d74e0abf0569f8e5dfd266446efa8bfaf19f0ea0980abf0b0b22f0d8416ee90d7435d21a9f9285b66df43f370b7979173406
ab3f4dd tests: Add test for 64-bit PE, modify 32-bit test results (Chun Kuan Lee)
Pull request description:
9a75d29b6f change the error result from `PIE` to `DYNAMIC_BASE`. And there are no test for 64-bit, so I made one
Tree-SHA512: 9d5643dadf4d9fc34ea32d94655bfb98eec2f7bc2820b4b0f525d5acf1cd22f3acf38bf8904dda4f50fd9ca5a5e56d566a392e6f804eea6e50e03cba40048621
After discussion with Matt on IRC, this is not ready for prime time
until 2019 and shouldn't have been ACKed and merged.
- Revert "Add systemd service for bitcoind"
This reverts commit 2a87b1b07c.
- Revert "Sync contrib/debian from Matt Corallo's PPA"
This reverts commit 9085532d35.
Tree-SHA512: 439f4ccc3e196011af448b220adf26b0e653ac589bf4cfbbc276c1500c9d08f209c9d6101e4d232857779d9f25164cfb222ed30e3d63de116f9121e6ebde31c3
2a87b1b Add systemd service for bitcoind (ctp-tsteenholdt)
9085532 Sync contrib/debian from Matt Corallo's PPA (ctp-tsteenholdt)
Pull request description:
On suggestion from @TheBlueMatt I have updated `contrib/debian` files to include a systemd service in the `bitcoind` build. Tested and working on Ubuntu 16.04 and 17.10.
This fixes Issue #12758
Tree-SHA512: b6137fafee940c7410df1242c8716a87f47c5bc60eb8df3ad0184a50c2d67ef3f2728761c742670a0ad546ab6e7ad60472a721350cd6280b3bcbdc582e50ee07
Adding systemd service for bitcoind, to provide for a simpler
out-of-the-box experience.
Configuration file is /etc/bitcoin/bitcoin.conf. This file is a
copy of the sample configuration file.
The service user 'bitcoin' is added during install. Its homedir
is in '/var/lib/bitcoin'.
bitcoind.service is disabled by default to allow the user to
configure it, before starting it the first time.
On package purge, the 'bitcoin' user as well as its homedir is
left intact, to not accidentally remove a wallet or something of
equal importance. Instead the user is presented with information
on how to perform the cleanup manually, after making sure all
important data has been backed up.
