Commit graph

15 commits

Author SHA1 Message Date
Olaoluwa Osuntokun
98cae74275 BIP0143+txscript: add segwit sighash, signing, and HashCache integration
This commit implements most of BIP0143 by adding logic to implement the
new sighash calculation, signing, and additionally introduces the
HashCache optimization which eliminates the O(N^2) computational
complexity for the SIGHASH_ALL sighash type.

The HashCache struct is the equivalent to the existing SigCache struct,
but for caching the reusable midstate for transactions which are
spending segwitty outputs.
2017-08-13 23:17:40 -05:00
Olaoluwa Osuntokun
48abfdf87c BIP0144+wire: add a MessageEncoding variant for serialization/deserialization
This commit modifies the existing wire.Message interface to introduce a
new MessageEncoding variant which dictates the exact encoding to be
used when serializing and deserializing messages. Such an option is now
necessary due to the segwit soft-fork package, as btcd will need to be
able to optionally encode transactions/blocks without witness data to
un-upgraded peers.

Two new functions have been introduced: ReadMessageWithEncodingN and
WriteMessageWithEncodingN which wrap BtcDecode/BtcEncode with the
desired encoding format.
2017-08-13 23:17:40 -05:00
Dave Collins
c440584efc
Implement infrastructure for BIP0009.
This commit adds all of the infrastructure needed to support BIP0009
soft forks.

The following is an overview of the changes:

- Add new configuration options to the chaincfg package which allows the
  rule deployments to be defined per chain
- Implement code to calculate the threshold state as required by BIP0009
  - Use threshold state caches that are stored to the database in order
    to accelerate startup time
  - Remove caches that are invalid due to definition changes in the
    params including additions, deletions, and changes to existing
    entries
- Detect and warn when a new unknown rule is about to activate or has
  been activated in the block connection code
- Detect and warn when 50% of the last 100 blocks have unexpected
  versions.
- Remove the latest block version from wire since it no longer applies
- Add a version parameter to the wire.NewBlockHeader function since the
  default is no longer available
- Update the miner block template generation code to use the calculated
  block version based on the currently defined rule deployments and
  their threshold states as of the previous block
- Add tests for new error type
- Add tests for threshold state cache
2016-11-30 13:51:59 -06:00
Dave Collins
bd4e64d1d4 chainhash: Abstract hash logic to new package. ()
This is mostly a backport of some of the same modifications made in
Decred along with a few additional things cleaned up.  In particular,
this updates the code to make use of the new chainhash package.

Also, since this required API changes anyways and the hash algorithm is
no longer tied specifically to SHA, all other functions throughout the
code base which had "Sha" in their name have been changed to Hash so
they are not incorrectly implying the hash algorithm.

The following is an overview of the changes:

- Remove the wire.ShaHash type
- Update all references to wire.ShaHash to the new chainhash.Hash type
- Rename the following functions and update all references:
  - wire.BlockHeader.BlockSha -> BlockHash
  - wire.MsgBlock.BlockSha -> BlockHash
  - wire.MsgBlock.TxShas -> TxHashes
  - wire.MsgTx.TxSha -> TxHash
  - blockchain.ShaHashToBig -> HashToBig
  - peer.ShaFunc -> peer.HashFunc
- Rename all variables that included sha in their name to include hash
  instead
- Update for function name changes in other dependent packages such as
  btcutil
- Update copyright dates on all modified files
- Update glide.lock file to use the required version of btcutil
2016-08-08 14:04:33 -05:00
Dave Collins
d406d9e52b wire: Consolidate tests into the wire pkg. ()
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.

This same thing should probably be done for the majority of the code
base.
2016-08-08 11:42:54 -05:00
Dave Collins
6229e35835 wire: Further reduce transaction allocs.
This commit drastically reduces the number of allocations needed to
deserialize a transaction and its scripts by using the combination of a
free list for initially deserializing the individual scripts along with
copying them into a single contiguous byte slice after the final size is
known and modifying each script in the transaction to point to its
location within the contiguous blob.

The end result is only a single allocation that holds all of the scripts
for a transaction regardless of the total number of scripts it has.

The script free list allows a maximum of 12,500 items with each buffer
being 512 bytes.  This implies it will have a peak usage of 6.1MB.  The
values were chosen based on profiling data and a desire to allow at
least 100 scripts per transaction to be simultaneously deserialized by
125 peers.

Also, while optimizing, decode directly into the existing previous
outpoint structure of each transaction input in order to avoid the extra
allocation per input that is otherwise caused when the local escapes to
the heap.

The following is a before and after comparison of the allocations
with the benchmarks that did not change removed:

benchmark              old allocs     new allocs     delta
-----------------------------------------------------------
ReadTxOut              1              0              -100.00%
ReadTxIn               2              0              -100.00%
DeserializeTxSmall     7              5              -28.57%
DeserializeTxLarge     11146          6              -99.95%
2016-06-03 17:09:14 -05:00
Dave Collins
b14032487f wire: Add several decode benchmarks. ()
This adds decode benchmarks for several of the messages that profiling
has identified to cause a lot of allocations in addition to those that
already exist.  By adding these benchmarks, it makes it easier to get
allocation and speed statistics which can in turn be used to compare
future improvements.

The following bencharmarks have been added:

DecodeGetHeaders, DecodeHeaders, DecodeGetBlocks, DecodeAddr, DecodeInv,
DecodeNotFound, and DecodeMerkleBlock

For reference, here is the benchmark data as of this commit.

DecodeGetHeaders     93261 ns/op     24120 B/op     1004 allocs/op
DecodeHeaders      2071263 ns/op    368399 B/op    18002 allocs/op
DecodeGetBlocks      92486 ns/op     24120 B/op     1004 allocs/op
DecodeAddr          850608 ns/op    136202 B/op     9002 allocs/op
DecodeInv         17107172 ns/op   3601447 B/op   150004 allocs/op
DecodeNotFound    17522225 ns/op   3601444 B/op   150004 allocs/op
DecodeMerkleBlock    21062 ns/op      5192 B/op      222 allocs/op
2016-04-25 17:32:29 -05:00
Dave Collins
e7ddaa468e wire: Don't allocate new readers in benchmarks. ()
This modifies the benchmarks in the wire package to avoid creating a new
reader for each iteration.  This is useful since it means that showing
the memory allocations will only show the function under test instead of
the allocation for the benchmark setup as well.

The following is a before and after comparison of the allocations
with the benchmarks that did not change removed:

benchmark              old allocs     new allocs     delta
------------------------------------------------------------
ReadVarInt1            2              1              -50.00%
ReadVarInt3            2              1              -50.00%
ReadVarInt5            2              1              -50.00%
ReadVarInt9            2              1              -50.00%
ReadVarStr4            4              3              -25.00%
ReadVarStr10           4              3              -25.00%
ReadOutPoint           2              1              -50.00%
ReadTxOut              4              3              -25.00%
ReadTxIn               6              5              -16.67%
DeserializeTxSmall     16             15             -6.25%
DeserializeTxLarge     33430          33428          -0.01%
ReadBlockHeader        8              7              -12.50%
2016-04-25 16:57:44 -05:00
Dave Collins
27c0f9f8d1 wire: Add large tx deserialize benchmark. ()
This adds a benchmark for deserializing a large transaction that is
often referred to as the megatransaction since it is the largest Bitcoin
transaction mined to date.  It consists of 5569 inputs and 1 output and
its hash is:

bb41a757f405890fb0f5856228e23b715702d714d59bf2b1feb70d8b2b4e3e08.

This is being done so there is a benchmark that tests more of a
worst-case scenario which is a better candidate for identifying and
testing improvements.

The following benchmark results shows the how much more intensive this
transaction is over the existing mock transaction:

DeserializeTxSmall  1000000    1751 ns/op      376 B/op     16 allocs/op
DeserializeTxLarge  300     5093980 ns/op  1672829 B/op  33430 allocs/op
2016-04-25 16:51:27 -05:00
Jouke Hofman
c17ff82061 wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
Dave Collins
80fa803875 wire: Export var length string serialization funcs.
This commit exports the ReadVarString and WriteVarString functions so
they are available for callers to use.

A variable length string is encoded as a variable length integer
containing the length of the string followed by the bytes that represent
the string itself.
2015-10-16 11:24:45 -05:00
Dave Collins
6e402deb35 Relicense to the btcsuite developers.
This commit relicenses all code in this repository to the btcsuite
developers.
2015-05-01 12:00:56 -05:00
Dave Collins
6211eef7ee wire: Add new DoubleSha256SH function.
This commit adds a new function which is similar to the DoubleSha256
function except it returns a ShaHash copy instead of a byte slice.  It
also adds a new benchmark for it.

This can be a slight optimization in certain cases where the caller
ultimately wants a ShaHash since it can avoid a heap allocation and
additional copy to convert the result to a ShaHash (the function simply
performs a type cast against the returned array which is not possible
against a []byte).

existing: DoubleSha256     500000   3081 ns/op   32 B/op   1 allocs/op
     new: DoubleSha256SH   500000   2939 ns/op    0 B/op   0 allocs/op

The hashing functions for blocks and transactions have also been updated
to make use of the new function since they directly return the ShaHash.
The transaction change in particular is quite useful since transactions
are frequently hashed and this change allows all of those hashes to avoid
an additional heap allocation.
2015-04-06 11:37:43 -05:00
Dave Collins
f5cdf2d6a8 Minor hashing-related optimizations.
This commit contains three classes of optimizations:
 - Reducing the number of unnecessary hash copies
 - Improve the performance of the DoubleSha256 function
 - A couple of minor optimizations of the ShaHash functions

The first class is a result of the Bytes function on a ShaHash making a
copy of the bytes before returning them.  It really should have been named
CloneBytes, but that would break the API now.

To address this, a comment has been added to the function which explicitly
calls out the copy behavior.  In addition, all call sites of .Bytes on a
ShaHash in the code base have been updated to simply slice the array when
a copy is not needed.  This saves a significant amount of data copying.

The second optimization modifies the DoubleSha256 function to directly use
fastsha256.Sum256 instead of the hasher interface.  This reduces the
number of allocations needed.  A benchmark for the function has been added
as well.

old: BenchmarkDoubleSha256  500000   3691 ns/op   192 B/op   3 allocs/op
new: BenchmarkDoubleSha256  500000   3081 ns/op    32 B/op   1 allocs/op

The final optimizations are for the ShaHash IsEqual and SetBytes functions
which have been modified to make use of the fact the type is an array and
remove an unneeded subslice.
2015-04-06 11:33:58 -05:00
Dave Collins
2eef3720a9 Import btcwire repo into wire directory.
This commit contains the entire btcwire repository along with several
changes needed to move all of the files into the wire directory in
order to prepare it for merging.  This does NOT update btcd or any of the
other packages to use the new location as that will be done separately.

- All import paths in the old btcwire test files have been changed to the
  new location
- All references to btcwire as the package name have been chagned to
  wire
- The coveralls badge has been removed since it unfortunately doesn't
  support coverage of sub-packages

This is ongoing work toward .
2015-01-31 14:59:57 -06:00
Renamed from bench_test.go (Browse further)