Commit graph

24 commits

Author SHA1 Message Date
Wladimir J. van der Laan
6e6b3b944d
Merge #14955: Switch all RNG code to the built-in PRNG
223de8d94d Document RNG design in random.h (Pieter Wuille)
f2e60ca985 Use secure allocator for RNG state (Pieter Wuille)
cddb31bb0a Encapsulate RNGState better (Pieter Wuille)
152146e782 DRY: Implement GetRand using FastRandomContext::randrange (Pieter Wuille)
a1f252eda8 Sprinkle some sweet noexcepts over the RNG code (Pieter Wuille)
4ea8e50837 Remove hwrand_initialized. (Pieter Wuille)
9d7032e4f0 Switch all RNG code to the built-in PRNG. (Pieter Wuille)
16e40a8b56 Integrate util/system's CInit into RNGState (Pieter Wuille)
2ccc3d3aa3 Abstract out seeding/extracting entropy into RNGState::MixExtract (Pieter Wuille)
aae8b9bf0f Add thread safety annotations to RNG state (Pieter Wuille)
d3f54d1c82 Rename some hardware RNG related functions (Pieter Wuille)
05fde14e3a Automatically initialize RNG on first use. (Pieter Wuille)
2d1cc50939 Don't log RandAddSeedPerfmon details (Pieter Wuille)
6a57ca91da Use FRC::randbytes instead of reading >32 bytes from RNG (Pieter Wuille)

Pull request description:

  This does not remove OpenSSL, but makes our own PRNG the 'main' one; for GetStrongRandBytes, the OpenSSL RNG is still used (indirectly, by feeding its output into our PRNG state).

  It includes a few policy changes (regarding what entropy is seeded when).

  Before this PR:
  * GetRand*:
    * OpenSSL
  * GetStrongRand*:
    * CPU cycle counter
    * Perfmon data (on Windows, once 10 min)
    * /dev/urandom (or equivalent)
    * rdrand (if available)
  * From scheduler when idle:
    * CPU cycle counter before and after 1ms sleep
  * At startup:
    * CPU cycle counter before and after 1ms sleep

  After this PR:
  * GetRand*:
    * Stack pointer (which indirectly identifies thread and some call stack information)
    * rdrand (if available)
    * CPU cycle counter
  * GetStrongRand*:
    * Stack pointer (which indirectly identifies thread and some call stack information)
    * rdrand (if available)
    * CPU cycle counter
    * /dev/urandom (or equivalent)
    * OpenSSL
    * CPU cycle counter again
  * From scheduler when idle:
    * Stack pointer (which indirectly identifies thread and some call stack information)
    * rdrand (if available)
    * CPU cycle counter before and after 1ms sleep
    * Perfmon data (on Windows, once every 10 min)
  * At startup:
    * Stack pointer (which indirectly identifies thread and some call stack information)
    * rdrand (if available)
    * CPU cycle counter
    * /dev/urandom (or equivalent)
    * OpenSSL
    * CPU cycle counter again
    * Perfmon data (on Windows, once every 10 min)

  The interface of random.h is also simplified, and documentation is added.

  This implements most of #14623.

Tree-SHA512: 0120e19bd4ce80a509b5c180a4f29497d299ce8242e25755880851344b825bc2d64a222bc245e659562fb5463fb7c70fbfcf003616be4dc59d0ed6534f93dd20
2019-01-21 19:46:45 +01:00
Pieter Wuille
16e40a8b56 Integrate util/system's CInit into RNGState
This guarantees that OpenSSL is initialized properly whenever randomness
is used, even when that randomness is invoked from global constructors.

Note that this patch uses Mutex directly, rather than CCriticalSection.
This is because the lock-detection code is not necessarily initialized
during global constructors.
2019-01-16 16:33:08 -08:00
Wladimir J. van der Laan
fcb6694a99
Merge #14839: [rebase] threads: fix unitialized members in sched_param
89282379ba threads: fix unitialized members in sched_param (Cory Fields)

Pull request description:

  Rebased theuni's #14342.

  Building with gcc 8.2 against musl libc, which apparently has more attributes available in its sched_param. The following warnings were produced:

      warning: missing initializer for member 'sched_param::sched_ss_low_priority' [-Wmissing-field-initializers]
      warning: missing initializer for member 'sched_param::sched_ss_repl_period' [-Wmissing-field-initializers]
      warning: missing initializer for member 'sched_param::sched_ss_init_budget' [-Wmissing-field-initializers]
      warning: missing initializer for member 'sched_param::sched_ss_max_repl' [-Wmissing-field-initializers]

  Since the current thread may have interesting non-zero values for these fields, we want to be sure to only change the intended one. Query and modify the current sched_param rather than starting from a zeroed one.

Tree-SHA512: a0bedbcf0130b3ee8261bb704e4bf6c9b760ad377c8a28c258765d54e54462b76707efc188b936b0a635cdd2bdf6b3b9298ab06ba361dc4806150b670d9702a3
2019-01-16 17:09:12 +01:00
Wladimir J. van der Laan
64ee94356f
Merge #14409: utils and libraries: Make 'blocksdir' always net specific
e4a0c3547e Improve blocksdir functional test. (Hennadii Stepanov)
c3f1821ac7 Make blockdir always net specific (Hennadii Stepanov)

Pull request description:

  The blocks directory is net specific by definition.

  Also this prevents the side effect of calling `GetBlocksDir(false)` in the non-mainnet environment.
  Currently a new node creates an unused `blocks\` directory in the root of the data directory when `-testnet` or `-regtest` is specified.

  Refs:
  - #12653
  - https://github.com/bitcoin/bitcoin/pull/12653#discussion_r174784834 by @laanwj
  - https://github.com/bitcoin/bitcoin/issues/14595#issuecomment-436011186

Tree-SHA512: c9957a68a4a200ebd2010823a56db7e61563afedcb7c9828e86b13f3af2990e07854b622c1f3374756f94574acb3ea32de7d2a399eef6c0623f0e11265155627
2019-01-16 13:40:27 +01:00
Wladimir J. van der Laan
332b3dd7c1 util: Make ToLower and ToUpper take a char
Unfortunately, `std::string` elements are (bare) chars. As these
are the most likely type to be passed to these functions, make them use
char instead of unsigned char. This avoids some casts.
2019-01-10 02:51:14 +01:00
Wladimir J. van der Laan
edb5bb3500 util: remove unused [U](BEGIN|END) macros 2019-01-10 01:58:33 +01:00
Wladimir J. van der Laan
62f3977f60
Merge #14599: Use functions guaranteed to be locale independent (IsDigit, ToLower) in {Format,Parse}Money(...), uint256::SetHex(...), etc. Remove the use of locale dependent boost::is_space(...)
8931a95bec Include util/strencodings.h which is required for IsSpace(...) (practicalswift)
7c9f790761 Update KNOWN_VIOLATIONS: Remove fixed violations (practicalswift)
587924f000 Use IsSpace(...) instead of boost::is_space (practicalswift)
c5fd143edb Use ToLower(...) instead of std::tolower (practicalswift)
e70cc8983c Use IsDigit(...) instead of std::isdigit (practicalswift)

Pull request description:

  * Use `ToLower(...)` instead of `std::tolower`. `std::tolower` is locale dependent.
  * Use `IsDigit(...)` instead of `std::isdigit`. Some implementations (e.g. Microsoft in 1252 codepage) may classify single-byte characters other than `[0-9]` as digits.
  * Update `KNOWN_VIOLATIONS`: Remove fixed violations.
  * ~~Replace use of locale dependent Boost trim (`boost::trim`) with locale independent `TrimString`.~~
  * Use` IsSpace(...)` instead of `boost::is_space`

Tree-SHA512: defed016136b530b723fa185afdbd00410925a748856ba3afa4cee60f61a67617e30f304f2b9991a67b5fe075d9624f051e14342aee176f45fbc024d59e1aa82
2019-01-09 18:17:17 +01:00
MeshCollider
8cff83124b Error if rpcpassword contains hash in conf sections 2019-01-09 17:32:35 +13:00
Cory Fields
89282379ba
threads: fix unitialized members in sched_param
Building with gcc 8.2 against musl libc, which apparently has more attributes
available in its sched_param. The following warnings were produced:

    warning: missing initializer for member 'sched_param::sched_ss_low_priority' [-Wmissing-field-initializers]
    warning: missing initializer for member 'sched_param::sched_ss_repl_period' [-Wmissing-field-initializers]
    warning: missing initializer for member 'sched_param::sched_ss_init_budget' [-Wmissing-field-initializers]
    warning: missing initializer for member 'sched_param::sched_ss_max_repl' [-Wmissing-field-initializers]

Since the current thread may have interesting non-zero values for these fields,
we want to be sure to only change the intended one. Query and modify the
current sched_param rather than starting from a zeroed one.
2018-12-09 21:08:48 +08:00
MarcoFalke
2a97f192ea
Merge #14771: test: Add BOOST_REQUIRE to getters returning optional
fa21ca09a8 test: Add BOOST_REQUIRE to getters returning optional (MarcoFalke)

Pull request description:

  Usually the returned value is already checked for equality, but for sanity we might as well require that the getter successfully returned.

Tree-SHA512: 0d613a9a721c61bd7a115ebc681a0890df09b8e5775f176ac18b3a586f2ca57bee0b5b816f5a7c314ff3ac6cbb2a4d9c434f8459e054a7c8a6934a75f0120c2a
2018-11-22 11:37:17 -05:00
MarcoFalke
fa21ca09a8
test: Add BOOST_REQUIRE to getters returning optional 2018-11-20 19:51:49 -05:00
Akio Nakamura
3fb09b9889 Warn unrecognized sections in the config file
In the config file, sections are specified by square bracket pair "[]"$,
or included in the option name itself which separated by a period"(.)".

Typicaly, [testnet] is not a correct section name and specified options
in that section are ignored but user cannot recognize what is happen.

So, add some log/stderr-warning messages if unrecognized section names
are present in the config file after checking section only args.
2018-11-20 18:28:16 +09:00
MarcoFalke
09f1d7fe72
Merge #14718: Remove unreferenced boost headers
c54e5a41c4 Remove unreferenced boost headers (Murray Nesbitt)

Pull request description:

  Building with clang (e.g. on FreeBSD) is very noisy due to `-Wthread-safety-analysis` warnings regarding boost. This change removes a number of unnecessary boost includes, and silences the rest of the warnings when building with clang. This allows more potentially interesting warnings to surface from the noise.

  Tested on FreeBSD 11.2

Tree-SHA512: 5e6a0623188b9be59aeae52866799aefb4c3c9ab5e569b07ee8d43fc92e0b5f1f76b96bb54c35c7043148df84641b4a96927fb71f6eb00460c20cd19cf250900
2018-11-19 13:50:16 -05:00
Murray Nesbitt
c54e5a41c4 Remove unreferenced boost headers 2018-11-16 03:20:44 +00:00
MarcoFalke
384967f311
Merge #13815: util: Add [[nodiscard]] to all {Decode,Parse}[...](...) functions returning bool
9cc0230cfc Add NODISCARD to all {Decode,Parse}[...](...) functions returning bool. Sort includes. (practicalswift)
579497e77a tests: Explicitly ignore the return value of DecodeBase58(...) (practicalswift)
145fe95ec7 tests: Check return value of ParseParameters(...) (practicalswift)
7c5bc2a523 miner: Default to DEFAULT_BLOCK_MIN_TX_FEE if unable to parse -blockmintxfee (practicalswift)

Pull request description:

  Changes in this PR:
  * ~~Add linter to make sure the return value of `Parse[...](...)` is checked~~
  * Add `__attribute__((warn_unused_result))` to all `{Decode,Parse}[...](...)` functions returning `bool`
  * Fix violations

  Context:
  * #13712: `wallet: Fix non-determinism in ParseHDKeypath(...). Avoid using an uninitialized variable in path calculation.` would have been prevented by this

Tree-SHA512: 41a97899f2d5a26584235fa02b1ebfb4faacd81ea97e927022955a658fa7e15d07a1443b4b7635151a43259a1adf8f2f4de3c1c75d7b5f09f0d5496463a1dae6
2018-11-15 14:39:37 -05:00
Wladimir J. van der Laan
90c0b6aca2
Merge #14494: Error if # is used in rpcpassword in conf
0385109444 Add test for rpcpassword hash error (MeshCollider)
13fe258e91 Error if rpcpassword in conf contains a hash character (MeshCollider)

Pull request description:

  Fixes #13143 now #13482 was merged

Tree-SHA512: e7d00c8df1657f6b8d0eee1e06b9ce2b1b0a2de487377699382c1b057836e1571dac313ca878b5877c862f0461ba789a50b239d2a9f34accd8a6321f126e3d2a
2018-11-12 15:23:27 +01:00
practicalswift
c5fd143edb Use ToLower(...) instead of std::tolower 2018-11-06 17:32:13 +01:00
practicalswift
e70cc8983c Use IsDigit(...) instead of std::isdigit 2018-11-06 17:32:08 +01:00
MeshCollider
13fe258e91 Error if rpcpassword in conf contains a hash character 2018-11-06 12:35:07 +13:00
Jim Posen
fef5adcc33 blockfilter: Use unordered_set instead of set in blockfilter. 2018-11-05 09:30:56 -08:00
practicalswift
9cc0230cfc Add NODISCARD to all {Decode,Parse}[...](...) functions returning bool. Sort includes. 2018-11-05 17:03:11 +01:00
practicalswift
ab8c6f24d2 Add SAFE_CHARS[SAFE_CHARS_URI]: Chars allowed in URIs (RFC 3986) 2018-11-05 13:27:04 +01:00
Hennadii Stepanov
c3f1821ac7
Make blockdir always net specific
The blocks directory is net specific by definition.

Also this prevents the side effect of calling GetBlocksDir(false) in the
non-mainnet environment.
2018-11-05 13:26:43 +02:00
Jim Posen
2068f089c8 scripted-diff: Move util files to separate directory.
-BEGIN VERIFY SCRIPT-
mkdir -p src/util
git mv src/util.h src/util/system.h
git mv src/util.cpp src/util/system.cpp
git mv src/utilmemory.h src/util/memory.h
git mv src/utilmoneystr.h src/util/moneystr.h
git mv src/utilmoneystr.cpp src/util/moneystr.cpp
git mv src/utilstrencodings.h src/util/strencodings.h
git mv src/utilstrencodings.cpp src/util/strencodings.cpp
git mv src/utiltime.h src/util/time.h
git mv src/utiltime.cpp src/util/time.cpp

sed -i 's/<util\.h>/<util\/system\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
sed -i 's/<utilmemory\.h>/<util\/memory\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
sed -i 's/<utilmoneystr\.h>/<util\/moneystr\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
sed -i 's/<utilstrencodings\.h>/<util\/strencodings\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
sed -i 's/<utiltime\.h>/<util\/time\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')

sed -i 's/BITCOIN_UTIL_H/BITCOIN_UTIL_SYSTEM_H/g' src/util/system.h
sed -i 's/BITCOIN_UTILMEMORY_H/BITCOIN_UTIL_MEMORY_H/g' src/util/memory.h
sed -i 's/BITCOIN_UTILMONEYSTR_H/BITCOIN_UTIL_MONEYSTR_H/g' src/util/moneystr.h
sed -i 's/BITCOIN_UTILSTRENCODINGS_H/BITCOIN_UTIL_STRENCODINGS_H/g' src/util/strencodings.h
sed -i 's/BITCOIN_UTILTIME_H/BITCOIN_UTIL_TIME_H/g' src/util/time.h

sed -i 's/ util\.\(h\|cpp\)/ util\/system\.\1/g' src/Makefile.am
sed -i 's/utilmemory\.\(h\|cpp\)/util\/memory\.\1/g' src/Makefile.am
sed -i 's/utilmoneystr\.\(h\|cpp\)/util\/moneystr\.\1/g' src/Makefile.am
sed -i 's/utilstrencodings\.\(h\|cpp\)/util\/strencodings\.\1/g' src/Makefile.am
sed -i 's/utiltime\.\(h\|cpp\)/util\/time\.\1/g' src/Makefile.am

sed -i 's/-> util ->/-> util\/system ->/' test/lint/lint-circular-dependencies.sh
sed -i 's/src\/util\.cpp/src\/util\/system\.cpp/g' test/lint/lint-format-strings.py test/lint/lint-locale-dependence.sh
sed -i 's/src\/utilmoneystr\.cpp/src\/util\/moneystr\.cpp/g' test/lint/lint-locale-dependence.sh
sed -i 's/src\/utilstrencodings\.\(h\|cpp\)/src\/util\/strencodings\.\1/g' test/lint/lint-locale-dependence.sh
sed -i 's/src\\utilstrencodings\.cpp/src\\util\\strencodings\.cpp/' build_msvc/libbitcoinconsensus/libbitcoinconsensus.vcxproj
-END VERIFY SCRIPT-
2018-11-04 22:46:07 -08:00