Commit graph

9516 commits

Author SHA1 Message Date
Pieter Wuille
8b22af3ee5 Replace bytes_serialized with bogosize 2017-06-05 12:43:45 -07:00
Pieter Wuille
7cc2c670e3
Merge #10506: Fix bumpfee test after #10449
8906a9a0d Fix bumpfee test after #10449 (Russell Yanofsky)

Tree-SHA512: 0838c7696499baf0fb5ee6edf0b081752d6c37578360a7f24a7e9c700598cbc14ff95826f2f5124cca805d2609470a052bc7309211874b13be7ac1ff9e911a34
2017-06-01 16:28:37 -07:00
Pieter Wuille
1088b02f0c
Merge #10195: Switch chainstate db and cache to per-txout model
589827975 scripted-diff: various renames for per-utxo consistency (Pieter Wuille)
a5e02bc7f Increase travis unit test timeout (Pieter Wuille)
73de2c1ff Rename CCoinsCacheEntry::coins to coin (Pieter Wuille)
119e552f7 Merge CCoinsViewCache's GetOutputFor and AccessCoin (Pieter Wuille)
580b02309 [MOVEONLY] Move old CCoins class to txdb.cpp (Pieter Wuille)
8b25d2c0c Upgrade from per-tx database to per-txout (Pieter Wuille)
b2af357f3 Reduce reserved memory space for flushing (Pieter Wuille)
41aa5b79a Pack Coin more tightly (Pieter Wuille)
97072d668 Remove unused CCoins methods (Pieter Wuille)
ce23efaa5 Extend coins_tests (Pieter Wuille)
508307968 Switch CCoinsView and chainstate db from per-txid to per-txout (Pieter Wuille)
4ec0d9e79 Refactor GetUTXOStats in preparation for per-COutPoint iteration (Pieter Wuille)
13870b56f Replace CCoins-based CTxMemPool::pruneSpent with isSpent (Pieter Wuille)
05293f3cb Remove ModifyCoins/ModifyNewCoins (Pieter Wuille)
961e48397 Switch tests from ModifyCoins to AddCoin/SpendCoin (Pieter Wuille)
8b3868c1b Switch CScriptCheck to use Coin instead of CCoins (Pieter Wuille)
c87b957a3 Only pass things committed to by tx's witness hash to CScriptCheck (Matt Corallo)
f68cdfe92 Switch from per-tx to per-txout CCoinsViewCache methods in some places (Pieter Wuille)
000391132 Introduce new per-txout CCoinsViewCache functions (Pieter Wuille)
bd83111a0 Optimization: Coin&& to ApplyTxInUndo (Pieter Wuille)
cb2c7fdac Replace CTxInUndo with Coin (Pieter Wuille)
422634e2f Introduce Coin, a single unspent output (Pieter Wuille)
7d991b55d Store/allow tx metadata in all undo records (Pieter Wuille)
c3aa0c119 Report on-disk size in gettxoutsetinfo (Pieter Wuille)
d34242430 Remove/ignore tx version in utxo and undo (Pieter Wuille)
7e0032290 Add specialization of SipHash for 256 + 32 bit data (Pieter Wuille)
e484652fc Introduce CHashVerifier to hash read data (Pieter Wuille)
f54580e7e error() in disconnect for disk corruption, not inconsistency (Pieter Wuille)
e66dbde6d Add SizeEstimate to CDBBatch (Pieter Wuille)

Tree-SHA512: ce1fb1e40c77d38915cd02189fab7a8b125c7f44d425c85579d872c3bede3a437760997907c99d7b3017ced1c2de54b2ac7223d99d83a6658fe5ef61edef1de3
2017-06-01 16:20:27 -07:00
Pieter Wuille
589827975f scripted-diff: various renames for per-utxo consistency
Thanks to John Newberry for pointing these out.

-BEGIN VERIFY SCRIPT-
sed -i 's/\<GetCoins\>/GetCoin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<HaveCoins\>/HaveCoin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<HaveCoinsInCache\>/HaveCoinInCache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<IsPruned\>/IsSpent/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<FetchCoins\>/FetchCoin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<CoinsEntry\>/CoinEntry/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<vHashTxnToUncache\>/coins_to_uncache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<vHashTxToUncache\>/coins_to_uncache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<fHadTxInCache\>/had_coin_in_cache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<coinbaseids\>/coinbase_coins/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<disconnectedids\>/disconnected_coins/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<duplicateids\>/duplicate_coins/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<oldcoins\>/old_coin/g' src/test/coins_tests.cpp
sed -i 's/\<origcoins\>/orig_coin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
-END VERIFY SCRIPT-
2017-06-01 13:15:25 -07:00
Pieter Wuille
73de2c1ff3 Rename CCoinsCacheEntry::coins to coin 2017-06-01 13:15:25 -07:00
Pieter Wuille
119e552f7c Merge CCoinsViewCache's GetOutputFor and AccessCoin
They're doing the same thing now.
2017-06-01 13:15:25 -07:00
Pieter Wuille
580b023092 [MOVEONLY] Move old CCoins class to txdb.cpp
It's only used for upgrading from the old database anymore.
2017-06-01 13:15:25 -07:00
Pieter Wuille
8b25d2c0ce Upgrade from per-tx database to per-txout 2017-06-01 13:15:24 -07:00
Pieter Wuille
b2af357f39 Reduce reserved memory space for flushing
As the maximum amount of data that can be pulled into the cache due to
a block validation is much lower now (at most one CCoin entry per input
and per output), reduce the conservative estimate used to determine
flushing time.
2017-06-01 13:15:06 -07:00
Pieter Wuille
41aa5b79a3 Pack Coin more tightly 2017-06-01 13:15:06 -07:00
Pieter Wuille
97072d6685 Remove unused CCoins methods 2017-06-01 13:15:03 -07:00
Pieter Wuille
ce23efaa5c Extend coins_tests 2017-06-01 12:59:41 -07:00
Pieter Wuille
5083079688 Switch CCoinsView and chainstate db from per-txid to per-txout
This patch makes several related changes:
* Changes the CCoinsView virtual methods (GetCoins, HaveCoins, ...)
  to be COutPoint/Coin-based rather than txid/CCoins-based.
* Changes the chainstate db to a new incompatible format that is also
  COutPoint/Coin based.
* Implements reconstruction code for hash_serialized_2.
* Adapts the coins_tests unit tests (thanks to Russell Yanofsky).

A side effect of the new CCoinsView model is that we can no longer
use the (unreliable) test for transaction outputs in the UTXO set
to determine whether we already have a particular transaction.
2017-06-01 12:59:38 -07:00
Pieter Wuille
4ec0d9e794 Refactor GetUTXOStats in preparation for per-COutPoint iteration 2017-06-01 12:43:16 -07:00
Pieter Wuille
13870b56fc Replace CCoins-based CTxMemPool::pruneSpent with isSpent 2017-06-01 12:43:16 -07:00
Pieter Wuille
05293f3cb7 Remove ModifyCoins/ModifyNewCoins 2017-06-01 12:43:16 -07:00
Pieter Wuille
961e483979 Switch tests from ModifyCoins to AddCoin/SpendCoin 2017-06-01 12:43:12 -07:00
Russell Yanofsky
8906a9a0d2 Fix bumpfee test after #10449
Failure reported by Jorge Timón <jtimon@jtimon.cc>
https://github.com/bitcoin/bitcoin/pull/10449#issuecomment-305556569
2017-06-01 15:35:44 -04:00
Pieter Wuille
8b3868c1b4 Switch CScriptCheck to use Coin instead of CCoins 2017-06-01 11:56:06 -07:00
Matt Corallo
c87b957a32 Only pass things committed to by tx's witness hash to CScriptCheck
This clarifies a bit more the ways in which the new script execution
cache could break consensus in the future if additional data from
the CCoins object were to be used as a part of script execution.

After this change, any such consensus breaks should be very visible
to reviewers, hopefully ensuring no such changes can be made.
2017-06-01 11:56:06 -07:00
Pieter Wuille
f68cdfe92b Switch from per-tx to per-txout CCoinsViewCache methods in some places 2017-06-01 11:56:06 -07:00
Pieter Wuille
0003911326 Introduce new per-txout CCoinsViewCache functions
The new functions are:
* CCoinsViewCache::AddCoin: Add a single COutPoint/Coin pair.
* CCoinsViewCache::SpendCoin: Remove a single COutPoint.
* AddCoins: utility function that invokes CCoinsViewCache::AddCoin for
  each output in a CTransaction.
* AccessByTxid: utility function that searches for any output with
  a given txid.
* CCoinsViewCache::AccessCoin: retrieve the Coin for a COutPoint.
* CCoinsViewCache::HaveCoins: check whether a non-empty Coin exists
  for a given COutPoint.

The AddCoin and SpendCoin methods will eventually replace ModifyCoins
and ModifyNewCoins, AddCoins will replace CCoins::FromTx, and the new
AccessCoins and HaveCoins functions will replace their per-txid
counterparts.

Note that AccessCoin for now returns a copy of the Coin object. In a
later commit it will be change to returning a const reference (which
keeps working in all call sites).
2017-06-01 11:56:06 -07:00
Pieter Wuille
bd83111a0f Optimization: Coin&& to ApplyTxInUndo
This avoids a prevector copy in ApplyTxInUndo.
2017-06-01 11:56:06 -07:00
Pieter Wuille
cb2c7fdac2 Replace CTxInUndo with Coin
The earlier CTxInUndo class now holds the same information as the Coin
class. Instead of duplicating functionality, replace CTxInUndo with a
serialization adapter for Coin.
2017-06-01 11:56:06 -07:00
Wladimir J. van der Laan
39039b12a7
Merge #10475: [RPC] getmempoolinfo mempoolminfee is a BTC/KB feerate
f128f78 getmempool mempoolminfee is a BTC/KB feerate (Gregory Sanders)

Tree-SHA512: 5b32ffc0d2ba7ca623d4ce450fb67855d17b4fa8c3ffb8570a63dc53b3d476d52b39ad6190ac4357dbf01f6b535fa187cd8c34175a16469e9b00097e3bf2e2bc
2017-06-01 19:21:26 +02:00
Jonas Schnelli
64beb13179
Merge #10449: Overhaul Qt fee bumper
6d7104c99 [Qt] make sure transaction table entry gets updated after bump (Jonas Schnelli)
32325a3f5 [Qt] hide bump context menu action if tx already has been bumped (Jonas Schnelli)

Tree-SHA512: d3e5991145879b7f6b212d9d9c6f423609dc8e6fa7f6feb7df931691f1dec2acb6ab162c2fb7e758d3ca3f3fb14363df2f50f0e83e83068da5cc7e6de35e69d2
2017-06-01 16:05:02 +02:00
Wladimir J. van der Laan
cb1716acc7
Merge #10441: net: only enforce expected services for half of outgoing connections
b6fbfc2 net: only enforce the services required to connect (Cory Fields)

Tree-SHA512: 88943bff63213a734f3c96c45760cadaeb9ba18287c8a20c279851ebaf058a334c969028fb2180f155508e3eea4b838147382e4f2b655e7a9aa098eadc81d53e
2017-06-01 13:24:36 +02:00
Wladimir J. van der Laan
b6b150b016
Merge #10469: Fixing typo in rpcdump.cpp
16d94d3 Fixing typo in rpcdump.cpp (James Evans)

Tree-SHA512: 84ef1b91c719131196ebed6b865e282b77bee7699614e15884ba59010239a3bbc1380dc8f856c83338f071e3eb3ca41c6b10f830816e6c794531cf6a965d63a9
2017-06-01 13:19:36 +02:00
Wladimir J. van der Laan
f259263a7b
Merge #10447: Make bitcoind invalid argument error message specific
cd5622d Make bitcoind invalid argument error message specific (Wladimir J. van der Laan)

Tree-SHA512: 78999b9359685c4090f9a112e1d02bdddac22173f5f04600773b042acb9481ebb54d0f73d9d6d74249f845f497927a58d59de69bf33e3dccf096c3c80c7ebf6b
2017-06-01 10:51:03 +02:00
Wladimir J. van der Laan
ef2d062c9f
Merge #10450: Fix bumpfee rpc "errors" return value
1b6602f Fix bumpfee rpc "errors" return value (Russell Yanofsky)

Tree-SHA512: a55d4dbe28c75f31fde3b7de60479265bb8f3777f432d1db321f1dd848d5274c353f757b09ed3cad9c69c08acbf3906679d27052267db943ab2b19629f79bd81
2017-06-01 10:48:19 +02:00
Wladimir J. van der Laan
18ba984140
Merge #10467: add SeedNodes to CConnman::Options
5d67526 add SeedNodes to CConnman::Options (Marko Bencun)

Tree-SHA512: d732ab08697419f896d9598fa3c30030ddbe3b82a99a398d813ebcd5e1fc610ad1043fe0f4c51a7ec5c4d70ce93a9ac5a5c7a91fcaae046ec4f1f7c41523a360
2017-05-31 17:41:10 +02:00
Wladimir J. van der Laan
5dd69ce551
Merge #10479: [trivial] Fix comment for ForceSetArg()
42a83e5 [trivial] Fix comment for ForceSetArg() (John Newbery)

Tree-SHA512: 596db8b4bb01bca18908e254e5b364d3d4fcb0ea49d7d54314c65106e497efed65a877d80740bff88e090aeb772be1e46d4571ecca9b9263578f3f680f0bd0ce
2017-05-31 17:16:16 +02:00
Jonas Schnelli
c1c9a95379
Merge #10454: Fix broken q4 test build
620444bd5 Fix broken q4 test build (Russell Yanofsky)

Tree-SHA512: 3247458541e5586e5a2cfc0abda3c54f140c676603ea5a3bdc0d11aae335d0221bbc818fc5fa458242fd4bacd326fca1013fc0320d5c91b672b8ca9684fb7bf8
2017-05-31 10:38:30 +02:00
Pieter Wuille
f94b7d5bfa
Merge #10464: Introduce static DoWarning (simplify UpdateTip)
f28514576 Introduce static DoWarning (simplify UpdateTip) (Jorge Timón)

Tree-SHA512: 58b39f00bc921d47d199ef1cc15e9e36f918ec86e9182de46dc2f24fa9474853a92b18d330a274a3ea7c9b001681fdfa3d86aa2bbebb5c92ac8c5a6f7690dfc1
2017-05-30 17:23:30 -07:00
Wladimir J. van der Laan
acd9957b72
Merge #9208: Improve DisconnectTip performance
c1235e3 Add RecursiveDynamicUsage overload for std::shared_ptr (Russell Yanofsky)
71f1903 Store disconnected block transactions outside mempool during reorg (Suhas Daftuar)
9decd64 [qa] Relax assumptions on mempool behavior during reorg (Suhas Daftuar)

Tree-SHA512: c160ad853a5cd060d0307af7606a0c77907497ed7033c9599b95e73d83f68fdfcd4214bd8a83db1c5b7a58022722b9de1ed2e6ea2e02f38a7b6c717f079dd0c6
2017-05-30 18:43:03 +02:00
John Newbery
42a83e5455 [trivial] Fix comment for ForceSetArg() 2017-05-30 08:21:16 -04:00
Marko Bencun
5d67526026 add SeedNodes to CConnman::Options
Start of a series of changes to clean up the instantiation of connman
by decoupling the command line arguments.
2017-05-30 09:21:47 +02:00
Wladimir J. van der Laan
5c63d665e5
Merge #10460: Broadcast address every day, not 9 hours
010d68e Broadcast address every day, not 9 hours (Pieter Wuille)

Tree-SHA512: 73a8b44b4b48784616e722b1e63561e35d669c2ee890dbc44e40191fdfd8fc1ef56710f4769ab6ea1f1a48fec0576f200c0b3812ca58fb143e7524006d1afed4
2017-05-30 07:17:02 +02:00
Gregory Sanders
f128f78ddb getmempool mempoolminfee is a BTC/KB feerate 2017-05-29 14:14:07 -04:00
Cory Fields
b6fbfc2282 net: only enforce the services required to connect
also once half of all outgoing nodes have our preferred flags, require only
minimal flags from the rest.
2017-05-28 10:20:07 -04:00
James Evans
16d94d3d03 Fixing typo in rpcdump.cpp 2017-05-27 16:44:10 +01:00
Jorge Timón
f285145764
Introduce static DoWarning (simplify UpdateTip) 2017-05-27 06:33:39 +02:00
Pieter Wuille
422634e2f5 Introduce Coin, a single unspent output 2017-05-26 13:33:40 -07:00
Pieter Wuille
7d991b55db Store/allow tx metadata in all undo records
Previously, transaction metadata (height, coinbase or not, and before
the previous commit also nVersion) was only stored for undo records
that correspond to the last output of a transaction being spent.

This only saves 2 bytes per undo record. Change this to storing this
information for every undo record, and stop complaining for having it
in non-last output spends. This means that undo dat written with
this patch won't be readable by older versions anymore.
2017-05-26 13:33:39 -07:00
Pieter Wuille
c3aa0c1194 Report on-disk size in gettxoutsetinfo 2017-05-26 13:33:39 -07:00
Pieter Wuille
d342424301 Remove/ignore tx version in utxo and undo
This makes the following changes:
* In undo data and the chainstate database, the transaction nVersion
  field is removed from the data structures, always written as 0, and
  ignored when reading.
* The definition of hash_serialized in gettxoutsetinfo is changed to no
  longer incude the nVersion field. It is renamed to hash_serialized_2
  to avoid confusion. The new definition also includes transaction
  height and coinbase information, as this information was missing
  before.

This depends on having a CHashVerifier-based undo data checksum
verifier.

Apart from changing the definition of serialized_hash, downgrading
after using this patch is supported, as no release ever used the value
of nVersion field in UTXO entries.
2017-05-26 13:27:50 -07:00
Pieter Wuille
7e00322906 Add specialization of SipHash for 256 + 32 bit data
We'll need a version of SipHash for tuples of 256 bits and 32 bits
data, when CCoinsViewCache switches from using txids to COutPoints as
keys.
2017-05-26 13:24:25 -07:00
Pieter Wuille
e484652fc3 Introduce CHashVerifier to hash read data
This is necessary later, when we drop the nVersion field from the undo
data. At that point deserializing and reserializing the data won't
roundtrip anymore, and thus that approach can't be used to verify
checksums anymore.

With this CHashVerifier approach, we can deserialize while hashing the
exact serialized form that was used. This is both more efficient and
more correct in that case.
2017-05-26 13:24:25 -07:00
Pieter Wuille
f54580e7e4 error() in disconnect for disk corruption, not inconsistency
The error() function unconditionally reports an error. It should only
be used for actually exception situations, and not for the type of
inconsistencies that ApplyTxInUndo/DisconnectBlock can graciously deal
with.

This also makes a subtle semantics change: in ApplyTxInUndo, when a
record with metadata is encountered (indicating it is the last spend
from a tx), don't wipe the CCoins record if it wasn't empty at that
point. This makes sure that UTXO operations never affect any other
UTXOs (including those from the same tx).
2017-05-26 13:24:25 -07:00
Pieter Wuille
e66dbde6d1 Add SizeEstimate to CDBBatch
This allows estimating the in-memory size of a LevelDB batch.
2017-05-26 13:24:24 -07:00