This resolves a case in which a mismatch could be used to bloat up the
mempool by sending transactions that pay enough fee to relay, but not
to be mined, with the default policies.
Amend to d5f1e72. It turns out that BerkelyDB was including inttypes.h
indirectly, so we cannot fix this with just macros.
Trivial commit: apply the following script to all .cpp and .h files:
# Middle
sed -i 's/"PRIx64"/x/g' "$1"
sed -i 's/"PRIu64"/u/g' "$1"
sed -i 's/"PRId64"/d/g' "$1"
# Initial
sed -i 's/PRIx64"/"x/g' "$1"
sed -i 's/PRIu64"/"u/g' "$1"
sed -i 's/PRId64"/"d/g' "$1"
# Trailing
sed -i 's/"PRIx64/x"/g' "$1"
sed -i 's/"PRIu64/u"/g' "$1"
sed -i 's/"PRId64/d"/g' "$1"
After this commit, `git grep` for PRI.64 should turn up nothing except
the defines in util.h.
As the tinyformat-based formatting system (introduced in b77dfdc) is
type-safe, no special format characters are needed to specify sizes.
Tinyformat can support (ignore) the C99 prefixes such as "ll" but
chokes on MSVC's inttypes.h defines prefixes such as "I64X". So don't
include inttypes.h and define our own for compatibility.
(an alternative would be to sweep the entire codebase using sed -i to
get rid of the size specifiers but this has less diff impact)
5770254 Copyright header updates s/2013/2014 on files whose last git commit was done in 2014. contrib/devtools/fix-copyright-headers.py script to be able to perform this maintenance task with ease during the rest of the year, every year. Modifications to contrib/devtools/README.md to document what fix-copyright-headers.py does. (gubatron)
Extend CMerkleTx::GetDepthInMainChain with the concept of
a "conflicted" transaction-- a transaction generated by the wallet
that is not in the main chain or in the mempool, and, therefore,
will likely never be confirmed.
GetDepthInMainChain() now returns -1 for conflicted transactions
(0 for unconfirmed-but-in-the-mempool, and >1 for confirmed).
This makes getbalance, getbalance '*', and listunspent all agree when there are
mutated transactions in the wallet.
Before:
listunspent: one 49BTC output
getbalance: 96 BTC (change counted twice)
getbalance '*': 46 BTC (spends counted twice)
After: all agree, 49 BTC available to spend.
contrib/devtools/fix-copyright-headers.py script to be able to perform this maintenance task with ease during the rest of the year, every year. Modifications to contrib/devtools/README.md to document what fix-copyright-headers.py does.
Keep track of which block is being requested (and to be requested) from
each peer, and limit the number of blocks in-flight per peer. In addition,
detect stalled downloads, and disconnect if they persist for too long.
This means blocks are never requested twice, and should eliminate duplicate
downloads during synchronization.
In case the total number of orphan blocks in memory exceeds a limit
(currently set to 750), a random orphan block (which is not
depended on by another orphan block) is dropped. This means it will
need to be downloaded again, but it won't consume memory until then.
c117d9e Support for error messages and a few more rejection reasons (Luke Dashjr)
14e7ffc Use standard BIP 22 rejection reasons where applicable (Luke Dashjr)
This changes the block processing logic from "try to atomically switch
to a new block" to a continuous "(dis)connect a block, aiming for the
assumed best chain".
This means the smallest atomic operations on the chainstate become
individual block connections or disconnections, instead of entire
reorganizations. It may mean that we try to reorganize to one block,
fail, and rereorganize again to the old block. This is slower, but
doesn't require unbounded RAM.
It also means that a ConnectBlock which fails may be no longer called
from the ProcessBlock which knows which node sent it. To deal with that,
a mapBlockSource is kept, and invalid blocks cause asynchronous "reject"
messages and banning (if necessary).
Previously CreateNewBlock() didn't take into account the fact that
IsFinalTx() without any arguments tests if the transaction is considered
final in the *current* block, when both those functions really needed to
know if the transaction would be final in the *next* block.
Additionally the UI had a similar misunderstanding.
Also adds some basic tests to check that CreateNewBlock() is in fact
mining nLockTime-using transactions correctly.
Thanks to Wladimir J. van der Laan for rebase.
After the tinyformat switch sprintf() family functions support passing
actual std::string objects.
Remove unnecessary c_str calls (236 of them) in logging and formatting.
012ca1c LoadWallet: acquire cs_wallet mutex before clearing setKeyPool (Wladimir J. van der Laan)
9569168 Document cs_wallet lock and add AssertLockHeld (Wladimir J. van der Laan)
19a5676 Use mutex pointer instead of name for AssertLockHeld (Wladimir J. van der Laan)
There were quite a few places where assert() was used with side effects,
making operation with NDEBUG non-functional. This commit fixes all the
cases I know about, but also adds an #error on NDEBUG because the code
is untested without assertions and may still have vulnerabilities if
used without assert.
This dead code can be resurrected from git history if
transaction replacement is ever implemented. Keeping
dead code in the source is a bad idea, because it implies
it was tested and worked at some point, which is not true.
The last fee drop was by 5x (from 50k satoshis to 10k satoshis)
in the 0.8.2 release which was about 6 months ago.
The current fee is (assuming a $500 exchange rate) about 5 dollar
cents. The new fee after this patch is 0.5 cents.
Miners who prefer the higher fees are obviously still able to
use the command line flags to override this setting. Miners who
choose to create smaller blocks will select the highest-fee paying
transactions anyway.
This would hopefully be the last manual adjustment ever required
before floating fees become normal.
Use misc methods of avoiding unnecesary header includes.
Replace int typedefs with int##_t from stdint.h.
Replace PRI64[xdu] with PRI[xdu]64 from inttypes.h.
Normalize QT_VERSION ifs where possible.
Resolve some indirect dependencies as direct ones.
Remove extern declarations from .cpp files.
caca6aa Make some globals in main non-public. (Pieter Wuille)
85eb2ce Do not use the redundant BestInvalidWork record in the block database. (Pieter Wuille)
As block index entries have a flag for marking invalid blocks, the
'best invalid work' information can be derived from there. In addition,
remove the global from main.h
- re-work -debug help message text
- make -debug log every debugging information again (even all categories)
- remove unneeded fDebug checks in front of LogPrint()/qDebug(), as that
check is done in LogPrintf() when category is != NULL (true for all
LogPrint() calls
- remove fDebug ONLY in code which is NOT performance-critical
- harmonize addrman category name
- deprecate -debugnet usage, should be used via -debug=net and remove the
corresponding global
Instead of explicitly testing for the presence of any output, and
dealing with this case specially, just interpret it as an empty
CCoins.
The case previously caught using the HaveCoins check, is now handled
by the generic outs != outsBlock test.
This required some code movement (what was CWalletTx::AcceptToMemoryPool
doing in main?), and adding a few explicit includes that used to be
implicit through init.h.
INIT_PROTO_VERSION is the initial version, after a succesful version/verack it is increased to a negotiated version.
MIN_PEER_PROTO_VERSION could be a different value to disconnect from peers older than a specified version.
Changes the response to the 'mempool' command so that if
the memory pool has more than MAX_INV_SZ transactions (50,000)
it will respond with multiple 'inv' messages.
SendMessages() tries to acquire a cs_main lock now, but this isn't nessecary
for much of its functionality. Move those parts out of the locked section,
so they can always be performed, and we hold cs_main for a shorter time.
This removes a few unused CBlockLocator methods, and moves the
construction and fork-finding logic to CChain (which can do these
more efficiently, as it has a height-indexable chain available).
It also makes CBlockLocator independent from the validation code.
971bb3e Added ping time measurement. New RPC "ping" command to request ping. Implemented "pong" message handler. New "pingtime" field in getpeerinfo, to provide results to user. New "pingwait" field, to show pings still in flight, to better see newly lagging peers. (Josh Lehan)
Changes the maximum size of a free transaction that will be created
from 10,000 bytes to 1,000 bytes.
The idea behind this change is to make the free transaction area
available to a greater number of people; with the default 27K-per-block,
just three very-large very-high-priority transactions could fill the space.
Remove the (relay/mempool) rule that all outputs of free transactions
must be greater than 0.01 XBT. Dust spam is now taken care of by making
dusty outputs non-standard.
New RPC "ping" command to request ping.
Implemented "pong" message handler.
New "pingtime" field in getpeerinfo, to provide results to user.
New "pingwait" field, to show pings still in flight, to better see newly lagging peers.
- rename URL into URI in paymentserver where correct
- add some missing Qt-coding-stuff in paymentserver
- change QSpinBox to QLineEdit as base for BitcoinAmountField in .ui files
(as this is the result when converting the BAF back into base)
- remove some c_str() and replace with QString::fromStdString()
- remove several new-lines
- remove unneeded spaces
- indentation fixes
This also makes negative transaction versions non-standard.
This avoids an issue triggered in block 256818 where transactions with
negative version numbers were incorrectly serialized into the UTXO set.
On restart nodes detect the inconsistency and refuse to start so long as
a block with these transactions is inside the self-consistency check
window, logging "coin database inconsistencies found". The software
recommends reindexing, but reindexing does not correct the problem.
This should be fixed by changing the chainstate serialization, but
working around it seems harmless for now because the version is not
used by any network rule currently.
A patch free workaround is to start with -checklevel=2 which skips
the consistency checks, but the IsStandard change is important for
miners in order to protect unpatched nodes.
There have been several incidents where mainnet experimentation with
raw transactions resulted in insane fees. This is hard to prevent
in the raw transaction api because the inputs may not be known.
Since sending doesn't work if the inputs aren't known, we can catch
it there.
This rejects fees > than 10000 * nMinRelayTxFee or 1 BTC with the
defaults and can be overridden with a bool at the rpc.
We're not seeing large reorgs that would justify waiting a large
amount past the rule required maturity, and the extra three
hours is just a nuisance. Take one more block to at least give
the 100th block time to propagate.
This reduces a peer's ability to attack network resources by
using a full bloom filter, but without reducing the usability
of bloom filters. It sets a default match everything filter
for peers and it generalizes a prior optimization to
cover more cases.
The length of vectors, maps, sets, etc are serialized using
Write/ReadCompactSize -- which, unfortunately, do not use a
unique encoding.
So deserializing and then re-serializing a transaction (for example)
can give you different bits than you started with. That doesn't
cause any problems that we are aware of, but it is exactly the type
of subtle mismatch that can lead to exploits.
With this pull, reading a non-canonical CompactSize throws an
exception, which means nodes will ignore 'tx' or 'block' or
other messages that are not properly encoded.
Please check my logic... but this change is safe with respect to
causing a network split. Old clients that receive
non-canonically-encoded transactions or blocks deserialize
them into CTransaction/CBlock structures in memory, and then
re-serialize them before relaying them to peers.
And please check my logic with respect to causing a blockchain
split: there are no CompactSize fields in the block header, so
the block hash is always canonical. The merkle root in the block
header is computed on a vector<CTransaction>, so
any non-canonical encoding of the transactions in 'tx' or 'block'
messages is erased as they are read into memory by old clients,
and does not affect the block hash. And, as noted above, old
clients re-serialize (with canonical encoding) 'tx' and 'block'
messages before relaying to peers.
Orphan transactions were stored as a CDataStream pointer;
this changes the mapOrphanTransactions data structures to
store orphans as a CTransaction.
This also fixes CVE-2013-4627 by always re-serializing
transactions before relaying them.
The new class is accessed via the Params() method and holds
most things that vary between main, test and regtest networks.
The regtest mode has two purposes, one is to run the
bitcoind/bitcoinj comparison tool which compares two separate
implementations of the Bitcoin protocol looking for divergence.
The other is that when run, you get a local node which can mine
a single block instantly, which is highly convenient for testing
apps during development as there's no need to wait 10 minutes for
a block on the testnet.
This (nearly) doesn't change fee rules at all:
* To make it into the fee transaction area, the dPriority comparison
changed from < to <=
* We now just ignore transactions > MAX_BLOCK_SIZE/4 instead of
doing some calculations to require increasingly large fees as
size increases.
Removed AreInputsStandard from CTransaction, made it a regular function in main.
Moved CTransaction::GetOutputFor to CCoinsViewCache.
Moved GetLegacySigOpCount and GetP2SHSigOpCount out of CTransaction into regular functions in main.
Moved GetValueIn and HaveInputs from CTransaction into CCoinsViewCache.
Moved AllowFree, ClientCheckInputs, CheckInputs, UpdateCoins, and CheckTransaction out of CTransaction and into main.
Moved IsStandard and IsFinal out of CTransaction and put them in main as IsStandardTx and IsFinalTx. Moved GetValueOut out of CTransaction into main. Moved CTxIn, CTxOut, and CTransaction into core.
Added minimum fee parameter to CTxOut::IsDust() temporarily until CTransaction is moved to core.h so that CTxOut needn't know about CTransaction.
- explicitly set the default of all GetBoolArg() calls
- rework getarg_test.cpp and util_tests.cpp to cover this change
- some indentation fixes
- move macdockiconhandler.h include in bitcoin.cpp to the "our headers"
section
Write bestblock records in wallets:
* Every 20160 blocks synced, no matter what (before: none during IBD)
* Every 144 blocks after IBD (before: for every block, slow)
* When creating a new wallet
* At shutdown
This should result in far fewer spurious rescans.
Remove the pnext pointer in CBlockIndex, and replace it with a
vBlockIndexByHeight vector (no effect on memory usage). pnext can
now be replaced by vBlockIndexByHeight[nHeight+1], but
FindBlockByHeight becomes constant-time.
This also means the entire mapBlockIndex structure and the block
index entries in it become purely blocktree-related data, and
independent from the currently active chain, potentially allowing
them to be protected by separate mutexes in the future.
Every block index entry currently requires a separately-allocated
CBigNum. By replacing them with uint256, it's just 32 bytes extra
in CBlockIndex itself.
This should save us a few megabytes in RAM, and less allocation
overhead.
This introduces the concept of the 'sync node', which is the one we
asked for missing blocks. In case the sync node goes away, a new one
will be selected.
For now, the heuristic is very simple, but it can easily be extended
later to add better policies.
As these were not updated when 'backporting' the 225430 checkpoint
into head.
Additionally, also report verification progress in debug.log, and
tweak the sigcheck-verification-speed-factor a bit.
Two reasons for this change:
1. Need to always use boost::thread's sleep, even on Windows, so the
sleeps can be interrupted (prior code used Windows' built-in Sleep).
2. I always forgot what units the old Sleep took.
Create a boost::thread_group object at the qt/bitcoind main-loop level
that will hold pointers to all the main-loop threads.
This will replace the vnThreadsRunning[] array.
For testing, ported the BitcoinMiner threads to use its
own boost::thread_group.
There exists a per-message-processed send buffer overflow protection,
where processing is halted when the send buffer is larger than the
allowed maximum.
This protection does not apply to individual items, however, and
getdata has the potential for causing large amounts of data to be
sent. In case several hundreds of blocks are requested in one getdata,
the send buffer can easily grow 50 megabytes above the send buffer
limit.
This commit breaks up the processing of getdata requests, remembering
them inside a CNode when too many are requested at once.
* Change CNode::vRecvMsg to be a deque instead of a vector (less copying)
* Make sure to acquire cs_vRecvMsg in CNode::CloseSocketDisconnect (as it
may be called without that lock).
Replaces CNode::vRecv buffer with a vector of CNetMessage's. This simplifies
ProcessMessages() and eliminates several redundant data copies.
Overview:
* socket thread now parses incoming message datastream into
header/data components, as encapsulated by CNetMessage
* socket thread adds each CNetMessage to a vector inside CNode
* message thread (ProcessMessages) iterates through CNode's CNetMessage vector
Message parsing is made more strict:
* Socket is disconnected, if message larger than MAX_SIZE
or if CMessageHeader deserialization fails (latter is impossible?).
Previously, code would simply eat garbage data all day long.
* Socket is disconnected, if we fail to find pchMessageStart.
We do not search through garbage, to find pchMessageStart. Each
message must begin precisely after the last message ends.
ProcessMessages() always processes a complete message, and is more efficient:
* buffer is always precisely sized, using CDataStream::resize(),
rather than progressively sized in 64k chunks. More efficient
for large messages like "block".
* whole-buffer memory copy eliminated (vRecv -> vMsg)
* other buffer-shifting memory copies eliminated (vRecv.insert, vRecv.erase)
- remove an unneeded MODAL flag, as MSG_ERROR sets MODAL
- re-order an if-clause in main to have bool checks before a function call
- fix some log messages that used wrong function names
- make a log message use a correct ellipsis
- remove some unneded spaces, brackets and line-breaks
- fix style for adding files in the Qt project
