4defdfab94 [MOVEONLY] Move unused Merkle branch code to tests (Pieter Wuille)
4437d6e1f3 8-way AVX2 implementation for double SHA256 on 64-byte inputs (Pieter Wuille)
230294bf5f 4-way SSE4.1 implementation for double SHA256 on 64-byte inputs (Pieter Wuille)
1f0e7ca09c Use SHA256D64 in Merkle root computation (Pieter Wuille)
d0c9632883 Specialized double sha256 for 64 byte inputs (Pieter Wuille)
57f34630fb Refactor SHA256 code (Pieter Wuille)
0df017889b Benchmark Merkle root computation (Pieter Wuille)
Pull request description:
This introduces a framework for specialized double-SHA256 with 64 byte inputs. 4 different implementations are provided:
* Generic C++ (reusing the normal SHA256 code)
* Specialized C++ for 64-byte inputs, but no special instructions
* 4-way using SSE4.1 intrinsics
* 8-way using AVX2 intrinsics
On my own system (AVX2 capable), I get these benchmarks for computing the Merkle root of 9001 leaves (supported lengths / special instructions / parallellism):
* 7.2 ms with varsize/naive/1way (master, non-SSE4 hardware)
* 5.8 ms with size64/naive/1way (this PR, non-SSE4 capable systems)
* 4.8 ms with varsize/SSE4/1way (master, SSE4 hardware)
* 2.9 ms with size64/SSE4/4way (this PR, SSE4 hardware)
* 1.1 ms with size64/AVX2/8way (this PR, AVX2 hardware)
Tree-SHA512: efa32d48b32820d9ce788ead4eb583949265be8c2e5f538c94bc914e92d131a57f8c1ee26c6f998e81fb0e30675d4e2eddc3360bcf632676249036018cff343e
9e49db2 Make --enable-debug to pick better options (Evan Klitzke)
Pull request description:
Cherry-picked (and rebased) 94189645e67f364c4445d62e2b00c282d885cbbf from the "up for grabs" PR: "[build] Make --enable-debug pick better options" (#12695).
See previous review in #12695.
Tree-SHA512: a93cdadcf13e2ef8519acb1ce4f41ce95057a388347bb0a86a5c164dc7d0b0d14d4bb2a466082d5a100b8d50de65c605c40abaed555e8ea77c99e28800a34439
Various changes:
* Don't check $GCC and $GXX
* Prefer -Og instead of -O0
* If -g3 isn't available, use -g
This also incidentally fixes compiler warnings with GCC and glibc when using
--enable-debug, as the old default values mixed poorly with the hardening flags.
This PR solves #12867 (needs to run autogen.sh && ./configure)
clang (Apple LLVM version 9.1.0 (clang-902.0.39.1)) warns unused
argument '-pie' during compilation.
So we check for warnings in the test using $CXXFLAG_WERROR.
Windows is alse default-pie and was special-cased because it also
warned, but we can also eliminate that case if warnings are caught.
It is redundant to check for the presence of MSG_NOSIGNAL macro in
configure.ac, define HAVE_MSG_NOSIGNAL and then check whether the later
is defined in the source code. Instead we can check directly whether
MSG_NOSIGNAL is defined. Same for MSG_DONTWAIT.
In addition to that, the checks we had in configure.ac produce a
compiler warning about unused variable and thus could fail if
-Werror is present and erroneously proclaim that the macros are
not available.
8172d3a configure: UniValue 1.0.4 is required for pushKV(, bool) (Luke Dashjr)
Pull request description:
The breaking changes (#12193) are already merged, so this blocks 0.17.0.
It depends on jgarzik/univalue#42 or jgarzik/univalue#50 being merged and released in UniValue 1.0.4.
Tree-SHA512: 3a21bbc72d6632bd07ee60ad7780b9ee95908357bcf59b4795b693d8a5d8c88943d6451482f11916ff5417e3bdbb9916062f87d0d73e79f50eb95ddabe21f943
cfaac2a60 Add build support for 'gprof' profiling. (murrayn)
Pull request description:
Support for profiling build: `./configure --enable-profiling`
Tree-SHA512: ea983cfce385f1893bb4ab7f94ac141b7d620951dc430da3bbc92ae1357fb05521eac689216e66dc87040171a8a57e76dd7ad98036e12a2896cfe5ab544347f0
a777244e4 Build: Add a makefile target for Doxygen documentation (Andrea Comand)
Pull request description:
You can now build the doxygen documentation with `make docs` and clean it with `make clean-docs`.
Fixes: #11949
Tree-SHA512: f2361ec7f771227367dd04bba1a444b44e59f13901463a678a5f2f579a10a56d67db2e29552e754e312a1c472a31593b6af189cbaac5cd351a428c57baf5ace7
Also clean out release notes.
Tree-SHA512: c4d5b52c089e14438be37381e1b0dab3711cc72aa8d345d1024169fff0055f3d021c8ca9d46fb794110694ebcbf7cbca0a12619f650873c9d381530adea7100e
76ea17c79 Add mutex requirement for AddToCompactExtraTransactions(…) (practicalswift)
4616c825a Use -Wthread-safety-analysis if available (+ -Werror=thread-safety-analysis if --enable-werror) (practicalswift)
7e319d639 Fix -Wthread-safety-analysis warnings. Change the sync.h primitives to std from boost. (Matt Corallo)
Pull request description:
* Add mutex requirement for `AddToCompactExtraTransactions(…)`.
* Use `-Wthread-safety-analysis` if available.
* Rebased on top of https://github.com/TheBlueMatt/bitcoin/commits/2017-08-test-10923 - now includes: Fix -Wthread-safety-analysis warnings. Change the sync.h primitives to std from boost.
Tree-SHA512: fb7365f85daa2741c276a1c899228181a8d46af51db7fbbdffceeaff121a3eb2ab74d7c8bf5e7de879bcc5042d00d24cb4649c312d51caba45a3f6135fd8b38f
5ac072caa Fix boost headers included as user instead of system headers (Dan Raviv)
Pull request description:
In most of the project, boost headers are included as system headers.
Fix the few inconsistent places where they aren't.
Tree-SHA512: 280af33a7bdc9d68a15b729fa88e1e7627e20a054b8d52a12cc5350c1ac9e9c90fb09f0aa97a00960969f75bcf3403dc52b834c94448b814efa63bfaf3b82663
Now that 0.15 branch has been split off, master is 0.15.99 (pre-0.16).
Also clean out release notes.
Tree-SHA512: 160f712fae7bfc51e49e36a27ab01f5c243b79a19a70312df95f9bc5cb8067c70aa88911f741fd1625bee549a70655eaea24d98f6049d98c3c14ee1b3143f4cb
ee2d10a Check if sys/random.h is required for getentropy on OSX. (James Hilliard)
Pull request description:
This should check and include sys/random.h if required for osx as mentioned [here](https://github.com/bitcoin/bitcoin/pull/9821#issuecomment-290936636).
Tree-SHA512: e9491f67f2e8b2e6bcdbcbb8063295e844d5627daf5336e3e17b4a8027d888fa65a08e4580a745abdc35ffd8d86b4fc7434daaac172c4a06ab7566a2ed0bfb92
912da1dcc Use AC_ARG_VAR to set ARFLAGS. (René Nyffenegger)
Pull request description:
Override the default of ARFLAGS of `cru` to `cr`.
When building, ar produces a warning for each archive, for example
```
AR libbitcoin_server.a
/usr/bin/ar: `u' modifier ignored since `D' is the default (see `U')
```
Since `u` is the default anyway, it cannot hurt to remove it.
Tree-SHA512: 7466764f847b70f0f67db25dac87a7794477abf1997cb946682f394fe80ae86ac3ed52cbadb35f0c18a87467755bde5a5158430444cd26fb60fa363cc7bd486d
d5711f4 Filter subtrees and and benchmarks from coverage report (Andrew Chow)
405b86a Replace lcov -r commands with faster way (Andrew Chow)
c8914b9 Have `make cov` optionally include branch coverage statistics (Andrew Chow)
Tree-SHA512: 9c349a7baeb7430ea586617c52f91177df58e3546d6dc573e26815ddb79e30ab1873542d85ac1daca5e1fb2c6d6c8965824b42d027b6b0496a744af57b095852
Added an option to configure to allow for branch coverage statistics gathering.
Disabled logprint macro when coverage testing is on so that unnecessary branches are not analyzed.
8ad5bde Merge bctest.py into bitcoin-util-test.py (John Newbery)
95836c5 Use shared config file for functional and util tests (John Newbery)
89fcd35 Use an .ini config file for environment vars in bitcoin-util-test.py (John Newbery)
e9265df Change help_text in bitcoin-util-test.py to a docstring. (John Newbery)
ce58e93 Change bitcoin-util-test.py to use Python3 (John Newbery)
Tree-SHA512: 66dab0b4a8546aee0dfaef134a165f1447aff4c0ec335754bbc7d9e55909721c62f09cdbf4b22d02ac1fcd5a9b66780f91e1cc4d8687fae7288cc9072a23a78f
bctest.py is only used as an import by bitcoin-util-test.py. There's no
value in keeping it as a separate module, so let's merge them into a
single module to keep building and packaging simpler.
bitcoin-test-util is importable as a module, so if any future modules
really want to import the code from bctest.py, they can import
bitcoin-test-util and call the bctest functions by name.
This is a well-intentioned but realistically annoying warning. Unfortunately,
it's too easy for a warning in one header to cause dozens of repeated warnings.
The functional tests and util tests both require a config file that is
generated by ./configure. This commit merges those two config
files into a single configuration file that can be shared by both tests.
The config from config.ini is put into a Namespace object to maintain
the interface with bctest.py. A future commit could change this
interface to use a dictionary instead of a namespace.
glibc-specific: On 32-bit systems set the number of arenas to 1. By
default, since glibc 2.10, the C library will create up to two heap
arenas per core. This is known to cause excessive virtual address space
usage in our usage. Work around it by setting the maximum number of
arenas to 1.
This adds a mode argument to `getmemoryinfo`. By default the output
will remain the same. However if a mode argument of `mallocinfo` is
provided the result of glibc `malloc_info` (if available) will
be returned as a string, as-is.
This is useful for tracking heap usage over time or troubleshooting
memory fragmentation issues.
Instead of the WIN32-specific workaround, detect lack of `MSG_DONTWAIT`
in the build system. This allows other platforms without `MSG_DONTWAIT`
to work too.
ad1ae7a Check and enable -Wshadow by default. (Pavel Janík)
9de90bb Do not shadow variables (gcc set) (Pavel Janík)
Tree-SHA512: 9517feb423dc8ddd63896016b25324673bfbe0bffa97f22996f59d7a3fcbdc2ebf2e43ac02bc067546f54e293e9b2f2514be145f867321e9031f895c063d9fb8
7e6dcd9 random: Add fallback if getrandom syscall not available (Wladimir J. van der Laan)
7cad849 sanity: Move OS random to sanity check function (Wladimir J. van der Laan)
aa09ccb squashme: comment that NUM_OS_RANDOM_BYTES should not be changed lightly (Wladimir J. van der Laan)
224e6eb util: Specific GetOSRandom for Linux/FreeBSD/OpenBSD (Wladimir J. van der Laan)
Tree-SHA512: 9fd408b1316c69de86674f342339b2f89192fd317c8c036b5df4320f828fa263c7966146bfc1904c51137ee4a26e4cb0f560b2cd05e18cde4d808b9b92ad15c4
These are available in sandboxes without access to files or
devices. Also [they are safer and more straightforward](https://en.wikipedia.org/wiki/Entropy-supplying_system_calls)
to use than `/dev/urandom` as reading from a file has quite a few edge
cases:
- Linux: `getrandom(buf, buflen, 0)`. [getrandom(2)](http://man7.org/linux/man-pages/man2/getrandom.2.html)
was introduced in version 3.17 of the Linux kernel.
- OpenBSD: `getentropy(buf, buflen)`. The [getentropy(2)](http://man.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/getentropy.2)
function appeared in OpenBSD 5.6.
- FreeBSD and NetBSD: `sysctl(KERN_ARND)`. Not sure when this was added
but it has existed for quite a while.
Alternatives:
- Linux has sysctl `CTL_KERN` / `KERN_RANDOM` / `RANDOM_UUID`
which gives 16 bytes of randomness. This may be available
on older kernels, however [sysctl is deprecated on Linux](https://lwn.net/Articles/605392/)
and even removed in some distros so we shouldn't use it.
Add tests for `GetOSRand()`:
- Test that no error happens (otherwise `RandFailure()` which aborts)
- Test that all 32 bytes are overwritten (initialize with zeros, try multiple times)
Discussion:
- When to use these? Currently they are always used when available.
Another option would be to use them only when `/dev/urandom` is not
available. But this would mean these code paths receive less testing,
and I'm not sure there is any reason to prefer `/dev/urandom`.
Closes: #9676
2fb98f6 Fix bug in dmg builder so that it actually reads in the configuration file (Don Patterson)
b01667c Mention RSVG dependency when creating the disk image on OSX (Jonas Schnelli)
09aefb5 build: Fix 'make deploy' for OSX (Cory Fields)
76faa3c Rename the remaining main.{h,cpp} to validation.{h,cpp} (Matt Corallo)
e736772 Move network-msg-processing code out of main to its own file (Matt Corallo)
87c35f5 Remove orphan state wipe from UnloadBlockIndex. (Matt Corallo)
OBJCXX's std flags don't get defined by our cxx macro. Rather than hard-coding
to c++11, just force OBJCXX to be the same as CXX unless the user specified
otherwise.
Simplified version of #8278. Assumes that every OS that (a) is supported
by Bitcoin Core (b) supports daemonization has the `daemon()` function
in its C library.
- Removes the fallback path for operating systems that support
daemonization but not `daemon()`. This prevents never-exercised code from
ending up in the repository (see discussion here:
https://github.com/bitcoin/bitcoin/pull/8278#issuecomment-242704745).
- Removes the windows-specific path. Windows doesn't support `daemon()`,
so it don't support daemonization there, automatically.
Original code by Matthew King, adapted by Wladimir van der Laan.
62c2915 build: supply `-Wl,--high-entropy-va` (Wladimir J. van der Laan)
9a75d29 devtools: Check for high-entropy ASLR in 64-bit PE executables (Wladimir J. van der Laan)
