So far, the documentation of memory_cleanse() is a verbatim copy of
the commit message in BoringSSL, where this code was originally
written. However, our code evolved since then, and the commit message
is not particularly helpful in the code but is rather of historical
interested in BoringSSL only.
This commit improves improves the comments around memory_cleanse()
and gives a better rationale for the method that we use. This commit
touches only comments.
Commit fbf327b138 ("Minimal code
changes to allow msvc compilation.") was indeed minimal in terms
of lines touched. But as a result of that minimalism it changed the
logic in memory_cleanse() to first call std::memset() and then
additionally the MSVC-specific SecureZeroMemory() function, and it
also moved a comment to the wrong location.
This commit removes the superfluous call to std::memset() on MSVC
and ensures that the comment is in the right position again.
a5bca13 Bugfix: Include <memory> for std::unique_ptr (Luke Dashjr)
Pull request description:
Not sure why all these includes were missing, but it's breaking builds for some users:
https://bugs.gentoo.org/show_bug.cgi?id=652142
(Added to all files with a reference to `std::unique_ptr`)
Tree-SHA512: 8a2c67513ca07b9bb52c34e8a20b15e56f8af2530310d9ee9b0a69694dd05e02e7a3683f14101a2685d457672b56addec591a0bb83900a0eb8e2a43d43200509
5fbf7c4 fix nits: variable naming, typos (Martin Ankerl)
1e0ee90 Use best-fit strategy in Arena, now O(log(n)) instead O(n) (Martin Ankerl)
Pull request description:
This replaces the first-fit algorithm used in the Arena with a best-fit. According to "Dynamic Storage Allocation: A Survey and Critical Review", Wilson et. al. 1995, http://www.scs.stanford.edu/14wi-cs140/sched/readings/wilson.pdf, both startegies work well in practice.
The advantage of using best-fit is that we can switch the O(n) allocation to O(log(n)). Additionally, some previously O(log(n)) operations are now O(1) operations by using hash maps. The end effect is that the benchmark runs about 2.5 times faster on my machine:
# Benchmark, evals, iterations, total, min, max, median
old: BenchLockedPool, 5, 530, 5.25749, 0.00196938, 0.00199755, 0.00198172
new: BenchLockedPool, 5, 1300, 5.11313, 0.000781493, 0.000793314, 0.00078606
I've run all unit tests and benchmarks, and increased the number of iterations so that BenchLockedPool takes about 5 seconds again.
Tree-SHA512: 6551e384671f93f10c60df530a29a1954bd265cc305411f665a8756525e5afe2873a8032c797d00b6e8c07e16d9827465d0b662875433147381474a44119ccce
This replaces the first-fit algorithm used in the Arena with a best-fit. According to "Dynamic Storage Allocation: A Survey and Critical Review", Wilson et. al. 1995, http://www.scs.stanford.edu/14wi-cs140/sched/readings/wilson.pdf, both startegies work well in practice.
The advantage of using best-fit is that we can switch the slow O(n) algorithm to O(log(n)) operations. Additionally, some previously O(log(n)) operations are now replaced with O(1) operations by using a hash map. The end effect is that the benchmark runs about 2.5 times faster on my machine:
old: BenchLockedPool, 5, 530, 5.25749, 0.00196938, 0.00199755, 0.00198172
new: BenchLockedPool, 5, 1300, 5.11313, 0.000781493, 0.000793314, 0.00078606
I've run all unit tests and benchmarks.
fbf327b Minimal code changes to allow msvc compilation. (Aaron Clauson)
Pull request description:
These changes are required to allow the Bitcoin source to build with Microsoft's C++ compiler (#11562 is also required).
I looked around for a better place for the typedef of ssize_t which is in random.h. The best candidate looks like src/compat.h but I figured including that header in random.h is a bigger change than the typedef. Note that the same typedef is in at least two other places including the OpenSSL and Berkeley DB headers so some of the Bitcoin code already picks it up.
Tree-SHA512: aa6cc6283015e08ab074641f9abdc116c4dc58574dc90f75e7a5af4cc82946d3052370e5cbe855fb6180c00f8dc66997d3724ff0412e4b7417e51b6602154825
Use C++11's better capability of expressing an interface of a non-copyable class by publicly deleting its copy ctor and assignment operator instead of just declaring them private.
The implementation we currently use from OpenSSL prevents the compiler from optimizing away clensing operations on blocks of memory that are about to be released, but this protection is not extended to link-time optimization. This commit copies the solution cooked up by Google compiler engineers which uses inline assembly directives to instruct the compiler not to optimize out the call under any circumstances. As the code is in-lined, this has the added advantage of removing one more OpenSSL dependency.
Regarding license compatibility, Google's contributions to BoringSSL library, including this code, is made available under the ISC license, which is MIT compatible.
BoringSSL git commit: ad1907fe73334d6c696c8539646c21b11178f20f
986255026 Use the noexcept specifier (C++11) instead of deprecated throw() (practicalswift)
Pull request description:
Use the `noexcept` specifier (C++11) instead of deprecated `throw()`.
Tree-SHA512: cf9b6b18f61f2f59bbeceb2e43b5cd07a60f5e569c8def05c410cb72326d597c80cb731059969ef89fa5fddaae1242225886e6109fcb535c4ad62d56ebcdf1ea
1ae86ec Changed event RAII helper functions to inline to deal with duplicate symbol linker errors. (Karl-Johan Alm)
fd369d2 Switched httpserver.cpp to use RAII wrapped libevents. (Kalle Alm)
Tree-SHA512: 877e431f211024d42a3b0800e860e02833398611433e8393f8d5d4970f47f4bd670b900443678c067fec110c087aaab7dc1981ccbf17f6057676fdbbda89aed9
ad1ae7a Check and enable -Wshadow by default. (Pavel Janík)
9de90bb Do not shadow variables (gcc set) (Pavel Janík)
Tree-SHA512: 9517feb423dc8ddd63896016b25324673bfbe0bffa97f22996f59d7a3fcbdc2ebf2e43ac02bc067546f54e293e9b2f2514be145f867321e9031f895c063d9fb8
05a55a6 Added EVENT_CFLAGS to test makefile to explicitly include libevent headers. (Karl-Johan Alm)
280a559 Added some simple tests for the RAII-style events. (Karl-Johan Alm)
7f7f102 Switched bitcoin-cli.cpp to use RAII unique pointers with deleters. (Karl-Johan Alm)
e5534d2 Added std::unique_ptr<> wrappers with deleters for libevent modules. (Karl-Johan Alm)
Check for unreasonable alloc size in LockedPool rather than lancing through new
Arenas until we improbably find one worthy of the quixotic request or the system
can support no more Arenas.
Add a pool for locked memory chunks, replacing LockedPageManager.
This is something I've been wanting to do for a long time. The current
approach of locking objects where they happen to be on the stack or heap
in-place causes a lot of mlock/munlock system call overhead, slowing
down any handling of keys.
Also locked memory is a limited resource on many operating systems (and
using a lot of it bogs down the system), so the previous approach of
locking every page that may contain any key information (but also other
information) is wasteful.
Replace these with vectors allocated from the secure allocator.
This avoids mlock syscall churn on stack pages, as well as makes
it possible to get rid of these functions.
Please review this commit and the previous one carefully that
no `sizeof(vectortype)` remains in the memcpys and memcmps usage
(ick!), and `.data()` or `&vec[x]` is used as appropriate instead of
&vec.
This assertion will occur any time that the client quits without
shutting down properly due to an error condition. As the user will
report this error instead of the error that was the root cause, it is
better to remove it.
This makes it easier for us to replace it if desired, since it's now only in
one spot. Also, it avoids the openssl include from allocators.h, which
essentially forced openssl to be included from every compilation unit.
