The environment is prepared by the main thread to guard against invalid locale settings and to prevent deinitialization issues of Boost path, which can result in app crashes.
This adds a -checkblockindex (defaulting to true for regtest), which occasionally
does a full consistency check for mapBlockIndex, setBlockIndexCandidates, chainActive, and
mapBlocksUnlinked.
This fixes a subtle bug involving block re-orgs and non-standard transactions.
Start with a block containing a non-standard transaction, and
one or more transactions spending it in the memory pool.
Then re-org away from that block to another chain that does
not contain the non-standard transaction.
Result before this fix: the dependent transactions get stuck
in the mempool without their parent, putting the mempool
in an inconsistent state.
Tested with a new unit test.
Make sure that chainparams and logging is properly initialized. Doing
this for every test may be overkill, but this initialization is so
simple that that does not matter.
This should fix the travis issues.
UNITTEST parameter are not used by any current tests, and the model
(modifyable parameters) is inconvenient when unit-testing. As
they are stored in a global structure eevery test
would have to (re)set up its own parameters.
For consistency it is also better to test with MAIN parameters.
Split GetNextWorkRequired() into two functions to allow the difficulty calculations to
be tested without requiring a full blockchain.
Add unit tests to cover basic difficulty calculation, plus each of the min/max actual
time, and maximum difficulty target conditions.
The fix to NegateSignatureS caused a test which had been failing
in IsValidSignatureEncoding to then fail in IsLowDERSignature.
Add new test so the original check remains exercised.
NegateSignatureS is called with a signature without a hashtype, so
do not save the last byte and append it after S negation.
Updates the two tests which were affected by this bug.
Makes it possible to compactly provide a delibrately invalid signature
for use with CHECK(MULTI)SIG. For instance with BIP19 if m != n invalid
signatures need to be provided in the scriptSig; prior to this change
those invalid signatures would need to be large DER-encoded signatures.
Note that we may want to further expand on this change in the future by
saying that only OP_0 is a "valid" invalid signature; BIP19 even with
this change is inherently malleable as the invalid signatures can be any
validly encoded DER signature.
on rare occasions, rand() was returning duped values, causing duplicate
transactions.
BuildMerkleTree happily used these, but CPartialMerkleTree caught them and
returned a null merkle root.
Rather than taking changes with rand(), use the loop counter to guarantee
unique values.
At sipa's request, also remove the remaining uses of rand().
856e862 namespace: drop most boost namespaces and a few header cleanups (Cory Fields)
9b1ab86 namespace: drop boost::assign altogether here (Cory Fields)
a324199 namespace: remove boost namespace pollution (Cory Fields)
Remove initialization from vector (as this is only used in the tests).
Also implement SetHex and GetHex in terms of uint256, to avoid
duplicate code as well as avoid endianness issues (as they
work in term of bytes).
- Methods that access the guts of arith_uint256 are removed,
as these are incompatible between endians. Use uint256 instead
- Serialization is no longer needed as arith_uint256's are never
read or written
- GetHash is never used on arith_uint256
If uint256() constructor takes a string, uint256(0) will become
dangerous when uint256 does not take integers anymore (it will go
through std::string(const char*) making a NULL string, and the explicit
keyword is no help).
SignatureHash and its test function SignatureHashOld
return uint256(1) as a special error signaling value.
Return a local static constant with the same value instead.
This is a check that is mentioned in BIP 37, but never implemented in the
reference code. As Bitcoin Core so far never decodes partial merkle trees,
this is not a problem. But perhaps others use the code as a reference.
OP_CODESEPARATOR is an actual executed instruction, not a declarative
thing, so if it's wrapped in an OP_IF it can be turned off.
Using this to implement Rivest's Paywords is left as an exercise for the
reader.
Although script_valid.json and script_invalid.json are loaded correctly by the
JSON interpreter used by bitcoin core, these same files are often used by other
libraries and do not necessarily load correctly due to the fact that newlines
contained inside strings are not valid and must instead use the escape
character \n. The files tx_valid.json and tx_invalid.json handle this
correctly, so I've changed the formatting in script_valid.json and
script_invalid.json to mirror those files.
Now that signing is deterministic, we can require exact correspondence between the
automatically generated tests and the ones read from JSON. Do this, and update
the tests to those deterministic versions. Note that some flag changes weren't
correctly applied before.
Based on an earlier patch by Peter Todd, though the rules here are different
(P2SH scripts should not have a CLEANSTACK check before the P2SH evaluation).
Add a sanity check to prevent cosmic rays from flipping a bit in the
generated public key, or bugs in the elliptic curve code. This is
simply done by signing a (randomized) message, and verifying the
result.
This turns STRICTENC turn into a softforking-safe change (even though it
is not intended as a consensus rule), and as a result guarantee that using
it for mempool validation only results in consensus-valid transactions in
the mempool.
NOP1 to NOP10 are reserved for future soft-fork upgrades. In the event
of an upgrade such NOPs have *VERIFY behavior, meaning that if their
arguments are not correct the script fails. Discouraging these NOPs by
rejecting transactions containing them from the mempool ensures that
we'll never accept transactions, nor mine blocks, with scripts that are
now invalid according to the majority of hashing power even if we're not
yet upgraded. Previously this wasn't an issue as the IsStandard() rules
didn't allow upgradable NOPs anyway, but 7f3b4e95 relaxed the
IsStandard() rules for P2SH redemptions allowing any redeemScript to be
spent.
We *do* allow upgradable NOPs in scripts so long as they are not
executed. This is harmless as there is no opportunity for the script to
be invalid post-upgrade.
1bea2bb Rename ProcessBlock to ProcessNewBlock to indicate change of behaviour, and document it (Luke Dashjr)
d29a291 Rename RPC_TRANSACTION_* errors to RPC_VERIFY_* and use RPC_VERIFY_ERROR for submitblock (Luke Dashjr)
f877aaa Bugfix: submitblock: Use a temporary CValidationState to determine accurately the outcome of ProcessBlock, now that it no longer does the full block validity check (Luke Dashjr)
24e8896 Add CValidationInterface::BlockChecked notification (Luke Dashjr)
99f41b9 MOVEONLY: core.o -> core/block.o (jtimon)
561e9e9 MOVEONLY: Move script/compressor out of script and put CTxOutCompressor (from core) with it (jtimon)
999a2ab MOVEONLY: separate CTxUndo out of core (jtimon)
4a3587d MOVEONLY: Separate CTransaction and dependencies from core (jtimon)
eda3733 MOVEONLY: Move CFeeRate and Amount constants to amount.o (jtimon)
85c579e script: add a slew of includes all around and drop includes from script.h (Cory Fields)
db8eb54 script: move ToString and ValueString out of the header (Cory Fields)
e9ca428 script: add ToByteVector() for converting anything with begin/end (Cory Fields)
066e2a1 script: move CScriptID to standard.h and add a ctor for creating them from CScripts (Cory Fields)
a873823 CAutoFile: Explicit Get() and remove unused methods (Wladimir J. van der Laan)
fef24ca Add IsNull() to class CAutoFile and remove operator ! (Ruben Dario Ponticeli)
This allows for a reversal of the current behavior.
This:
CScript foo;
CScriptID bar(foo.GetID());
Becomes:
CScript foo;
CScriptID bar(foo);
This way, CScript is no longer dependent on CScriptID or Hash();
* Delete canonical_tests.cpp, and move the tests to script_tests.cpp.
* Split off SCRIPT_VERIFY_DERSIG from SCRIPT_VERIFY_STRICTENC (the BIP62 part of it).
* Change signature STRICTENC/DERSIG semantics to fail the script entirely rather than the CHECKSIG result (softfork safety, and BIP62 requirement).
* Add many autogenerated tests for several odd cases.
* Mention specific BIP62 rules in the script verification flags.
7c70438 Get rid of the dummy CCoinsViewCache constructor arg (Pieter Wuille)
ed27e53 Add coins_tests with a large randomized CCoinViewCache test. (Pieter Wuille)
058b08c Do not keep fully spent but unwritten CCoins entries cached. (Pieter Wuille)
c9d1a81 Get rid of CCoinsView's SetCoins and SetBestBlock. (Pieter Wuille)
f28aec0 Use ModifyCoins instead of mutable GetCoins. (Pieter Wuille)
e790c37 Replace SCRIPT_VERIFY_NOCACHE by flag directly to checker (Pieter Wuille)
5c1e798 Make signature cache optional (Pieter Wuille)
c7829ea Abstract out SignatureChecker (Pieter Wuille)
4705902 Avoid introducing a virtual into CChainParams (Wladimir J. van der Laan)
5e2e7fc Suggested corrections on comments, variable names. Also new test case testing the PoW skip in UNITTEST. (SergioDemianLerner)
a25fd6b Switch testing framework from MAIN to new UNITTEST network (SergioDemianLerner)
Windows needed a few fixups to get the tests running:
1. bitcoin-tx needs a file extension in Windows. Take this opportunity to
add an env file, which pulls variables out of our build config. This can
be extended as needed, for now it's very simple.
2. After #1, split the args out of the exec key in the test data.
3. Correct the line-endings from windows stdout
93e24dd travis: use debug for one build (Cory Fields)
be6d87a script: don't read past the end (Cory Fields)
a94496f tests: don't split an empty string (Cory Fields)
00522cd depends: disable reduced exports for debug builds (Cory Fields)
1f7fff2 depends: add docs for debug (Cory Fields)
dc66ff5 depends: make LDFLAGS act like the other flags (Cory Fields)
b1efba8 depends: give miniupnpc cppflags (Cory Fields)
1d154db depends: teach qt to honor debug/release (Cory Fields)
3b63df5 depends: boost: build for debug or release as requested (Cory Fields)
7e99df7 depends: make sure openssl sees cppflags (Cory Fields)
f397304 depends: add debug/release flags for linux/osx/win (Cory Fields)
2027ad3 depends: add the debug/release concept to depends (Cory Fields)
All direct modifications are now done through ModifyCoins, and BatchWrite is
used for pushing batches of queued modifications up, so we don't need the
low-level SetCoins and SetBestBlock anymore in the top-level CCoinsView class.
Prevent denial-of-service attacks by banning
peers that send us invalid orphan transactions
and only storing orphan transactions given to
us by a peer while the peer is connected.
ad49c25 Split up util.cpp/h (Wladimir J. van der Laan)
f841aa2 Move `COIN` and `CENT` to core.h (Wladimir J. van der Laan)
6e5fd00 Move `*Version()` functions to version.h/cpp (Wladimir J. van der Laan)
b4aa769 Move `S_I*` constants and `MSG_NOSIGNAL` to compat.h (Wladimir J. van der Laan)
af8297c Move functions in wallet.h to implementation file (Wladimir J. van der Laan)
651480c move functions in main and net to implementation files (Wladimir J. van der Laan)
610a8c0 Move SetThreadPriority implementation to util.cpp instead of the header (Wladimir J. van der Laan)
f780e65 Remove unused function `ByteReverse` from util.h (Wladimir J. van der Laan)
121d6ad Remove unused `alignup` function from util.h (Wladimir J. van der Laan)
d1e26d4 Move CMedianFilter to timedata.cpp (Wladimir J. van der Laan)
Split up util.cpp/h into:
- string utilities (hex, base32, base64): no internal dependencies, no dependency on boost (apart from foreach)
- money utilities (parsesmoney, formatmoney)
- time utilities (gettime*, sleep, format date):
- and the rest (logging, argument parsing, config file parsing)
The latter is basically the environment and OS handling,
and is stripped of all utility functions, so we may want to
rename it to something else than util.cpp/h for clarity (Matt suggested
osinterface).
Breaks dependency of sha256.cpp on all the things pulled in by util.
Remove the 'state' and 'exceptmask' from serialize.h's stream implementations,
as well as related methods.
As exceptmask always included 'failbit', and setstate was always called with
bits = failbit, all it did was immediately raise an exception. Get rid of
those variables, and replace the setstate with direct exception throwing
(which also removes some dead code).
As a result, good() is never reached after a failure (there are only 2
calls, one of which is in tests), and can just be replaced by !eof().
fail(), clear(n) and exceptions() are just never called. Delete them.
d2d9dc0 script tests: add tests for CHECKMULTISIG limits (Otto Allmendinger)
89101c6 script test: test case for 5-byte bools (Otto Allmendinger)
4cac5db script tests: value with trailing 0x00 is true (Otto Allmendinger)
833ff16 script tests: values that overflow to 0 are true (Otto Allmendinger)
0072d98 script tests: BOOLAND, BOOLOR decode to integer (Otto Allmendinger)
ed02282 additional test for OP_SIZE in script_valid.json (Otto Allmendinger)
a3e192a replaced MINE_ with ISMINE_ (JaSK)
53a2148 fixed bug where validateaddress doesn't display information (JaSK)
f28707a fixed bug in ListReceived() (JaSK)
519dd1c Added MINE_ALL = (spendable|watchonly) (JaSK)
23b0506 Fixed some stuff in TransactionDesc (JaSK)
80dda36 removed default argument values for ismine filter (JaSK)
d5087d1 Use script matching rather than destination matching for watch-only. (Pieter Wuille)
0fa2f88 added includedWatchonly argument to listreceivedbyaddress/...account (JaSK)
f87ba3d added includeWatchonly argument to 'gettransaction' because it affects balance calculation (JaSK)
a5c6c5d fixed tiny glitch and improved readability like laanwj suggested (JaSK)
d7d5d23 Added argument to listtransactions and listsinceblock to include watchonly addresses (JaSK)
952877e Showing 'involvesWatchonly' property for transactions returned by 'listtransactions' and 'listsinceblock'. It is only appended when the transaction involves a watchonly address. (JaSK)
83f3543 Added argument to listaccounts to include watchonly addresses (JaSK)
d4640d7 Added argument to getbalance to include watchonly addresses and fixed errors in balance calculation. (JaSK)
d2692f6 Watchonly transactions are marked in transaction history (JaSK)
ffd40da Watchonly balances are shown separately in gui. (JaSK)
2935b21 qt: Hide unspendable outputs in coin control (Wladimir J. van der Laan)
c898846 Add support for watch-only addresses (Pieter Wuille)
Changes:
* Add Add/Have WatchOnly methods to CKeyStore, and implementations
in CBasicKeyStore.
* Add similar methods to CWallet, and support entries for it in
CWalletDB.
* Make IsMine in script/wallet return a new enum 'isminetype',
rather than a boolean. This allows distinguishing between
spendable and unspendable coins.
* Add a field fSpendable to COutput (GetAvailableCoins' return type).
* Mark watchonly coins in listunspent as 'watchonly': true.
* Add 'watchonly' to validateaddress, suppressing script/pubkey/...
in this case.
Based on a patch by Eric Lombrozo.
Conflicts:
src/qt/walletmodel.cpp
src/rpcserver.cpp
src/wallet.cpp
Two changes:
First removes a unit test that fails in my development environment
(OSX, compiled -g3 with clang).
sipa says that's not terribly surprising; the CMutableTransaction change
makes signing a little more expensive but verification quicker. The unit
test timed sign+verify-uncached versus verify-cached-five-times.
He also says the test will be invalid when libsec256kp1 is integrated
(because validation is super-optimized over signing).
core.h change fixes a compiler warning (clang -Wall : CMutableTransaction defined
as struct, declared as class in script.h).
This commit removes all the unnecessary dependencies (key, core,
netbase, sync, ...) from bitcoin-cli.
To do this it shards the chain parameters into BaseParams, which
contains just the RPC port and data directory (as used by utils and
bitcoin-cli) and Params, with the rest.
Relax the AreInputsStandard() tests for P2SH transactions --
allow any Script in a P2SH transaction to be relayed/mined,
as long as it has 15 or fewer signature operations.
Rationale: https://gist.github.com/gavinandresen/88be40c141bc67acb247
I don't have an easy way to test this, but the code changes are
straightforward and I've updated the AreInputsStandard unit tests.
21bf3d2 Add tests for BoostAsioToCNetAddr (Wladimir J. van der Laan)
fdbd707 Remove unused function WildcardMatch (Wladimir J. van der Laan)
ee21912 rpc: Use netmasks instead of wildcards for IP address matching (Wladimir J. van der Laan)
e16be73 net: Add CSubNet class for subnet matching (Wladimir J. van der Laan)
d864275 Use new function parseint32 in SplitHostPort (Wladimir J. van der Laan)
0d4ea1c util: add parseint32 function with strict error reporting (Wladimir J. van der Laan)
None of the current integer parsing functions in util
check whether the result is valid and fits in the range
of the type. This is required for less sloppy error reporting.
787ee0c Check redeemScript size does not exceed 520 byte limit (Peter Todd)
4d79098 Increase IsStandard() scriptSig length (Peter Todd)
f80cffa Do not trigger a DoS ban if SCRIPT_VERIFY_NULLDUMMY fails (Peter Todd)
6380180 Add rejection of non-null CHECKMULTISIG dummy values (Peter Todd)
29c1749 Let tx (in)valid tests use any SCRIPT_VERIFY flag (Peter Todd)
68f7d1d Create (MANDATORY|STANDARD)_SCRIPT_VERIFY_FLAGS constants (Peter Todd)
This is a source of transaction mutability as the dummy value was
previously not checked and could be modified to something other than the
usual OP_0 value.
Size specifiers are no longer needed now that we use typesafe tinyformat
for string formatting, instead of the system's sprintf.
No functional changes.
This continues the work in #3735.
Because this class replaces some usages of CBigNum, tests have been added to
verify that they function the same way. The only difference in their usage is
the handling of out-of-range numbers.
While operands are constrained to [-0x7FFFFFFF,0x7FFFFFFF], the results may
overflow. The overflowing result is technically unbounded, but in practice
it can be no bigger than the result of an operation on two operands. This
implementation limits them to the size of an int64.
CBigNum was unaware of this constraint, so it allowed for unbounded results,
which were then checked before use. CScriptNum asserts if an arithmetic
operation will overflow an int64_t, since scripts are not able to reach those
numbers anyway. Additionally, CScriptNum will throw an exception when
constructed from a vector containing more than 4 bytes This mimics the previous
CastToBigNum behavior.
deleted the empty no throw test in rpc_wallet_tests line 65
fixed some comments
starting verify tests
finished verify message tests
changed some comments
Building the tests was giving some vague error message about a doubly-defined
symbol.
The solution is to define ShutdownRequested in test_bitcoin.cpp as well
so that init.cpp does not get pulled in.
- Add license headers to source files (years based on commit dates)
in `src/test` as well as `qa`
- Add `README.md` to `src/test/data` specifying MIT license
Fixes#3848
a81cd968 introduced a malleability breaker for signatures
(using an even value for S). In e0e14e43 this was changed to
the lower of two potential values, rather than the even one.
Only the signing code was changed though, the (for now unused)
verification code wasn't adapted.
Amend to d5f1e72. It turns out that BerkelyDB was including inttypes.h
indirectly, so we cannot fix this with just macros.
Trivial commit: apply the following script to all .cpp and .h files:
# Middle
sed -i 's/"PRIx64"/x/g' "$1"
sed -i 's/"PRIu64"/u/g' "$1"
sed -i 's/"PRId64"/d/g' "$1"
# Initial
sed -i 's/PRIx64"/"x/g' "$1"
sed -i 's/PRIu64"/"u/g' "$1"
sed -i 's/PRId64"/"d/g' "$1"
# Trailing
sed -i 's/"PRIx64/x"/g' "$1"
sed -i 's/"PRIu64/u"/g' "$1"
sed -i 's/"PRId64/d"/g' "$1"
After this commit, `git grep` for PRI.64 should turn up nothing except
the defines in util.h.
Keep track of which block is being requested (and to be requested) from
each peer, and limit the number of blocks in-flight per peer. In addition,
detect stalled downloads, and disconnect if they persist for too long.
This means blocks are never requested twice, and should eliminate duplicate
downloads during synchronization.
Previously CreateNewBlock() didn't take into account the fact that
IsFinalTx() without any arguments tests if the transaction is considered
final in the *current* block, when both those functions really needed to
know if the transaction would be final in the *next* block.
Additionally the UI had a similar misunderstanding.
Also adds some basic tests to check that CreateNewBlock() is in fact
mining nLockTime-using transactions correctly.
Thanks to Wladimir J. van der Laan for rebase.
Unit tests would fail if compiled with -DDEBUG_LOCKORDER (AssertLockHeld()
would fail; AssertLockHeld() relies on the DEBUG_LOCKORDER code to keep
track of locks held).
Fixed by LOCK'ing the wallet mutex in the unit tests that manipulate the
wallet.
Unit tests for uint256.h. The file uint160_tests.cpp is no longer
needed. The ad-hoc tests which were in uint256.h are also no longer
needed. The new tests achieve 100% coverage.
Instead, use have an exception object to check if the string returned by what() on the raised exception matches the string returned by what() on the expected exception instance.
This way, we do not need to list all different possible explanatory strings for different platforms in the test code, and make it simple. (The idea is by Cory Fields.)
Before the fix, there were 6 errors such as :
serialize_tests.cpp:77: error in "noncanonical": incorrect exception std::ios_base::failure is caught
It turns out that ex.what() returns following string instead of "non-canonical ReadCompactSize()"
"non-canonical ReadCompactSize(): unspecified iostream_category error"
After the fix, unit test passed.
The test ran using Apple LLVM v5.0 on OSX 10.9 and the unit test error happened because of different error messages by different compilers.
g++ --version on my development environment.
```
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin13.0.0
Thread model: posix
```
`-logtodebugger` is a strange, obscure, WIN32-only (mostly MSVC) thing.
Let's clean up the options a bit get rid of it.
test_bitcoin was using fLogToDebugger as a way to prevent logging to
debug.log. For this, add a boolean (not exposed as option) fLogToDebugLog that
defaults to true and is disabled in the tests.
Use a fixed script instead of a CReserveKey from the wallet.
This does not affect the functionality or result of the tests as they never
check the state of the wallet in the first place.
Remove unnecessary dependencies for bitcoin-cli
(leveldb, berkelydb, wallet, RPC server)
Build system changes:
- split libbitcoin.a into libbitcoin_common.a, libbitcoin_server.a and
libbitcoin_cli.a
Code changes (movement only):
- split up HelpMessage into HelpMessage in init.cpp and HelpMessageCli
in rpcclient.cpp
- move uiInterface from init.cpp to util.cpp
Split bitcoinrpc up into
- rpcserver: bitcoind RPC server
- rpcclient: bitcoin-cli RPC client
- rpcprotocol: shared common HTTP/JSON-RPC protocol code
One step towards making bitcoin-cli independent from the rest
of the code, and thus a smaller executable that doesn't have to
be linked against leveldb.
This commit only does code movement, there are no functional changes.
The last fee drop was by 5x (from 50k satoshis to 10k satoshis)
in the 0.8.2 release which was about 6 months ago.
The current fee is (assuming a $500 exchange rate) about 5 dollar
cents. The new fee after this patch is 0.5 cents.
Miners who prefer the higher fees are obviously still able to
use the command line flags to override this setting. Miners who
choose to create smaller blocks will select the highest-fee paying
transactions anyway.
This would hopefully be the last manual adjustment ever required
before floating fees become normal.
I regenerated the alert test data; now alerts are tested
against a protocol version way above the current protocol
version.
So we won't have to regenerate them every time we bump
PROTOCOL_VERSION in the future.
Use misc methods of avoiding unnecesary header includes.
Replace int typedefs with int##_t from stdint.h.
Replace PRI64[xdu] with PRI[xdu]64 from inttypes.h.
Normalize QT_VERSION ifs where possible.
Resolve some indirect dependencies as direct ones.
Remove extern declarations from .cpp files.
0056095 Show short scriptPubKeys correctly (Peter Todd)
22de68d Relay OP_RETURN TxOut as standard transaction type (Peter Todd)
Signed-off-by: Gavin Andresen <gavinandresen@gmail.com>
Changed CDataStream::GetAndClear() to use the most obvious
get get and clear instead of a tricky swap().
Added a unit test for CDataStream insert/erase/GetAndClear.
Note: GetAndClear() is not performance critical, it is used only
by the send-a-message-to-the-network code. Bug was not noticed
before now because the send-a-message code never erased from the
stream.
class template base_uint had its own private lookup table.
This is saving 256 bytes per instantiation.
The result is not spectacular as bitcoin-qt has only shrinked of
about 1Kb but it is still valid improvement.
Also, I have replaced a for loop with a memset() call.
Made CBigNum::SetHex() use the new HexDigit() function.
Signed-off-by: Olivier Langlois <olivier@olivierlanglois.net>
Just-in-case sanity test for JSON spirit and AmountFromValue.
Also update rpc_format_monetary_values test to use ValueFromAmount,
so that ValueFromAmount is also tested.
Instead of building a full copy of a CTransaction being signed, and
then modifying bits and pieces until its fits the form necessary
for computing the signature hash, use a wrapper serializer that
only serializes the necessary bits on-the-fly.
This makes it easier to see which data is actually being hash,
reduces load on the heap, and also marginally improves performances
(around 3-4us/sigcheck here). The performance improvements are much
larger for large transactions, though.
The old implementation of SignatureHash is moved to a unit tests,
to test whether the old and new algorithm result in the same value
for randomly-constructed transactions.
This change moves test data into the binaries rather than reading them from
the disk at runtime.
Advantages:
- Tests become distributable
- Cross-compile friendly. Build on one machine and execute in an arbitrary
location on another.
- Easier testing for backports. Users can verify that tests pass without having
to track down corresponding test data.
- More trustworthy test results and easier quality assurance as tests make
fewer assumptions about their environment.
- Tests could theoretically run at client/daemon startup and exit on failure.
Disadvantages:
- Required 'hexdump' build-dependency. This is a standard bsd tool that should
be usable everywhere. It is likely already installed on all build-machines.
- Tests can no longer be fudged after build by altering test-data.
Seems it was forgotten about when IsPushOnly() and the unittests were
written. A particular oddity is that OP_RESERVED doesn't count towards
the >201 opcode limit unlike every other named opcode.
To fix a minor malleability found by Sergio Lerner (reported here:
https://bitcointalk.org/index.php?topic=8392.msg1245898#msg1245898)
The problem is that if (R,S) is a valid ECDSA signature for a given
message and public key, (R,-S) is also valid. Modulo N (the order
of the secp256k1 curve), this means that both (R,S) and (R,N-S) are
valid. Given that N is odd, S and N-S have a different lowest bit.
We solve the problem by forcing signatures to have an even S value,
excluding one of the alternatives.
This commit just changes the signing code to always produce even S
values, and adds a verification mode to check it. This code is not
enabled anywhere yet. Existing tests in key_tests.cpp verify that
the produced signatures are still valid.
The length of vectors, maps, sets, etc are serialized using
Write/ReadCompactSize -- which, unfortunately, do not use a
unique encoding.
So deserializing and then re-serializing a transaction (for example)
can give you different bits than you started with. That doesn't
cause any problems that we are aware of, but it is exactly the type
of subtle mismatch that can lead to exploits.
With this pull, reading a non-canonical CompactSize throws an
exception, which means nodes will ignore 'tx' or 'block' or
other messages that are not properly encoded.
Please check my logic... but this change is safe with respect to
causing a network split. Old clients that receive
non-canonically-encoded transactions or blocks deserialize
them into CTransaction/CBlock structures in memory, and then
re-serialize them before relaying them to peers.
And please check my logic with respect to causing a blockchain
split: there are no CompactSize fields in the block header, so
the block hash is always canonical. The merkle root in the block
header is computed on a vector<CTransaction>, so
any non-canonical encoding of the transactions in 'tx' or 'block'
messages is erased as they are read into memory by old clients,
and does not affect the block hash. And, as noted above, old
clients re-serialize (with canonical encoding) 'tx' and 'block'
messages before relaying to peers.
Fixes issue#2838; this is a tweaked version of pull#2845 that
should not leak the length of the password and is more generic,
in case we run into other situations where we need
timing-attack-resistant comparisons.
Orphan transactions were stored as a CDataStream pointer;
this changes the mapOrphanTransactions data structures to
store orphans as a CTransaction.
This also fixes CVE-2013-4627 by always re-serializing
transactions before relaying them.
The new class is accessed via the Params() method and holds
most things that vary between main, test and regtest networks.
The regtest mode has two purposes, one is to run the
bitcoind/bitcoinj comparison tool which compares two separate
implementations of the Bitcoin protocol looking for divergence.
The other is that when run, you get a local node which can mine
a single block instantly, which is highly convenient for testing
apps during development as there's no need to wait 10 minutes for
a block on the testnet.
Removed AreInputsStandard from CTransaction, made it a regular function in main.
Moved CTransaction::GetOutputFor to CCoinsViewCache.
Moved GetLegacySigOpCount and GetP2SHSigOpCount out of CTransaction into regular functions in main.
Moved GetValueIn and HaveInputs from CTransaction into CCoinsViewCache.
Moved AllowFree, ClientCheckInputs, CheckInputs, UpdateCoins, and CheckTransaction out of CTransaction and into main.
Moved IsStandard and IsFinal out of CTransaction and put them in main as IsStandardTx and IsFinalTx. Moved GetValueOut out of CTransaction into main. Moved CTxIn, CTxOut, and CTransaction into core.
Added minimum fee parameter to CTxOut::IsDust() temporarily until CTransaction is moved to core.h so that CTxOut needn't know about CTransaction.
- explicitly set the default of all GetBoolArg() calls
- rework getarg_test.cpp and util_tests.cpp to cover this change
- some indentation fixes
- move macdockiconhandler.h include in bitcoin.cpp to the "our headers"
section
This fixes test_bitcoin failures on openbsd reported by dhill on IRC.
On some systems rand() is a simple LCG over 2^31 and so it produces
an even-odd sequence. ApproximateBestSubset was only using the least
significant bit and so every run of the iterative solver would be the
same for some inputs, resulting in some pretty dumb decisions.
Using something other than the least significant bit would paper over
the issue but who knows what other way a system's rand() might get us
here. Instead we use an internal RNG with a period of something like
2^60 which is well behaved. This also makes it possible to make the
selection deterministic for the tests, if we wanted to implement that.
* During block verification (when parallelism is requested), script
check actions are stored instead of being executed immediately.
* After every processed transactions, its signature actions are
pushed to a CScriptCheckQueue, which maintains a queue and some
synchronization mechanism.
* Two or more threads (if enabled) start processing elements from
this queue,
* When the block connection code is finished processing transactions,
it joins the worker pool until the queue is empty.
As cs_main is held the entire time, and all verification must be
finished before the block continues processing, this does not reach
the best possible performance. It is a less drastic change than
some more advanced mechanisms (like doing verification out-of-band
entirely, and rolling back blocks when a failure is detected).
The -par=N flag controls the number of threads (1-16). 0 means auto,
and is the default.
These flags select features to be enabled/disabled during script
evaluation/checking, instead of several booleans passed along.
Currently these flags are defined:
* SCRIPT_VERIFY_P2SH: enable BIP16-style subscript evaluation
* SCRIPT_VERIFY_STRICTENC: enforce strict adherence to pubkey/sig encoding standards.
Flushes the blktree/ and coins/ databases, and reindexes the
block chain files, as if their contents was loaded via -loadblock.
Based on earlier work by Jeff Garzik.
signrawtransaction was unable to sign pay-to-script-hash inputs
when given the list of private keys to use. With this commit
you can provide the p2sh redemption script in the list of
inputs.
To prevent excessive copying of CCoins in and out of the CCoinsView
implementations, introduce a GetCoins() function in CCoinsViewCache
with returns a direct reference. The block validation and connection
logic is updated to require caching CCoinsViews, and exploits the
GetCoins() function heavily.
This switches bitcoin's transaction/block verification logic to use a
"coin database", which contains all unredeemed transaction output scripts,
amounts and heights.
The name ultraprune comes from the fact that instead of a full transaction
index, we only (need to) keep an index with unspent outputs. For now, the
blocks themselves are kept as usual, although they are only necessary for
serving, rescanning and reorganizing.
The basic datastructures are CCoins (representing the coins of a single
transaction), and CCoinsView (representing a state of the coins database).
There are several implementations for CCoinsView. A dummy, one backed by
the coins database (coins.dat), one backed by the memory pool, and one
that adds a cache on top of it. FetchInputs, ConnectInputs, ConnectBlock,
DisconnectBlock, ... now operate on a generic CCoinsView.
The block switching logic now builds a single cached CCoinsView with
changes to be committed to the database before any changes are made.
This means no uncommitted changes are ever read from the database, and
should ease the transition to another database layer which does not
support transactions (but does support atomic writes), like LevelDB.
For the getrawtransaction() RPC call, access to a txid-to-disk index
would be preferable. As this index is not necessary or even useful
for any other part of the implementation, it is not provided. Instead,
getrawtransaction() uses the coin database to find the block height,
and then scans that block to find the requested transaction. This is
slow, but should suffice for debug purposes.
Special serializer/deserializer for amount values. It is optimized for
values which have few non-zero digits in decimal representation. Most
amounts currently in the txout set take only 1 or 2 bytes to
represent.
Variable-length integers: bytes are a MSB base-128 encoding of the number.
The high bit in each byte signifies whether another digit follows. To make
the encoding is one-to-one, one is subtracted from all but the last digit.
Thus, the byte sequence a[] with length len, where all but the last byte
has bit 128 set, encodes the number:
(a[len-1] & 0x7F) + sum(i=1..len-1, 128^i*((a[len-i-1] & 0x7F)+1))
Properties:
* Very small (0-127: 1 byte, 128-16511: 2 bytes, 16512-2113663: 3 bytes)
* Every integer has exactly one encoding
* Encoding does not depend on size of original integer type
For backward compatibility, new accounting data is stored after a \0 in the comment string.
This way, old versions and third-party software should load and store them, but all actual use (listtransactions, for example) ignores it.
Memory locks do not stack, that is, pages which have been locked several times by calls to mlock()
will be unlocked by a single call to munlock(). This can result in keying material ending up in swap when
those functions are used naively. In this commit a class "LockedPageManager" is added
that simulates stacking memory locks by keeping a counter per page.
This does two things:
1) Now does not output to debug.log if -printtodebugger flag is passed
2) Unit tests set -printtodebugger so only test results are output to stdout
Note that -printtodebugger only actually prints to the debugger on Windows.
Implement listunspent / getrawtransaction / createrawtransaction /
signrawtransaction, to support creation and
signing-on-multiple-device multisignature transactions.
This commit adds support for .onion addresses (mapped into the IPv6
by using OnionCat's range and encoding), and the ability to connect
to them via a SOCKS5 proxy.
AvailableCoins() makes a vector of available outputs which is then passed to SelectCoinsMinConf(). This allows unit tests to test the coin selection algorithm without having the whole blockchain available.
Newlines in JSON strings are against the JSON spec,
so remove them from the script*.json unit tests to
make python's jsonrpc happy (json::spirit didn't care).
This introduces internal types:
* CKeyID: reference (hash160) of a key
* CScriptID: reference (hash160) of a script
* CTxDestination: a boost::variant of the former two
CBitcoinAddress is retrofitted to be a Base58 encoding of a
CTxDestination. This allows all internal code to only use the
internal types, and only have RPC and GUI depend on the base58 code.
Furthermore, the header dependencies are a lot saner now. base58.h is
at the top (right below rpc and gui) instead of at the bottom. For the
rest: wallet -> script -> keystore -> key. Only keystore still requires
a forward declaration of CScript. Solving that would require splitting
script into two layers.
test/DoS_tests.cpp: In member function ‘void DoS_tests::DoS_mapOrphans::test_method()’:
test/DoS_tests.cpp:200:41: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
test/DoS_tests.cpp:208:41: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
test/DoS_tests.cpp: In member function ‘void DoS_tests::DoS_checkSig::test_method()’:
test/DoS_tests.cpp:260:37: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
test/DoS_tests.cpp:267:37: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
test/DoS_tests.cpp:280:41: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
test/DoS_tests.cpp:307:37: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
Create a maximum-10MB signature verification result cache.
This should almost double the number of transactions that
can be processed on a given CPU, because before this change
ECDSA signatures were verified when transactions were added
to the memory pool and then again when they appeared in
a block.
One of the test cases currently aborts when using gcc's flag -ftrapv, due to
negating an INT64_MIN int64 variable, which is an undefined operation.
This will be fixed in a subsequent commit.
This also removes an un-needed sigops-per-byte check when accepting transactions to the memory pool (un-needed assuming only standard transactions are being accepted). And it only counts P2SH sigops after the switchover date.
This tests:
* creation of keys from base58-encoded strings
* extracting public keys and addresses
* compressed public keys
* compact signatures and key recovery
This introduces CNetAddr and CService, respectively wrapping an
(IPv6) IP address and an IP+port combination. This functionality used
to be part of CAddress, which also contains network flags and
connection attempt information. These extra fields are however not
always necessary.
These classes, along with logic for creating connections and doing
name lookups, are moved to netbase.{h,cpp}, which does not depend on
headers.h.
Furthermore, CNetAddr is mostly IPv6-ready, though IPv6
functionality is not yet enabled for the application itself.
so it takes a flag for how to interpret OP_EVAL.
Also increased IsStandard size of scriptSigs to 500 bytes, so
a 3-of-3 multisig transaction IsStandard.
OP_EVAL is a new opcode that evaluates an item on the stack as a script.
It enables a new type of bitcoin address that needs an arbitrarily
complex script to redeem.
