This commit modifies the existing wire.Message interface to introduce a
new MessageEncoding variant which dictates the exact encoding to be
used when serializing and deserializing messages. Such an option is now
necessary due to the segwit soft-fork package, as btcd will need to be
able to optionally encode transactions/blocks without witness data to
un-upgraded peers.
Two new functions have been introduced: ReadMessageWithEncodingN and
WriteMessageWithEncodingN which wrap BtcDecode/BtcEncode with the
desired encoding format.
The current code involves a ton of small allocations which is harsh on
the garbage collector and in turn causes a lot of addition runtime
overhead both in terms of additional memory and processing time.
In order to improve the situation, this drasticially reduces the number
of allocations by creating contiguous slices of objects and
deserializing into them. Since the final data structures consist of
slices of pointers to the objects, they are constructed by pointing them
into the appropriate offset of the contiguous slice.
This could be improved upon even further by converting all of the data
structures provided the wire package to be slices of contiguous objects
directly, however that would be a major breaking API change and would
end up requiring updating a lot more code in every caller. I do think
that ultimately the API should be changed, but the changes in this
commit already makes a massive difference and it doesn't require
touching any of the callers, so it is a good place to begin.
The following is a before and after comparison of the allocations
with the benchmarks that did not change removed:
benchmark old allocs new allocs delta
-----------------------------------------------------------
DeserializeTxLarge 16715 11146 -33.32%
DecodeGetHeaders 501 2 -99.60%
DecodeHeaders 2001 2 -99.90%
DecodeGetBlocks 501 2 -99.60%
DecodeAddr 3001 2002 -33.29%
DecodeInv 50003 3 -99.99%
DecodeNotFound 50002 3 -99.99%
DecodeMerkleBlock 107 3 -97.20%
This commit contains the entire btcwire repository along with several
changes needed to move all of the files into the wire directory in
order to prepare it for merging. This does NOT update btcd or any of the
other packages to use the new location as that will be done separately.
- All import paths in the old btcwire test files have been changed to the
new location
- All references to btcwire as the package name have been chagned to
wire
- The coveralls badge has been removed since it unfortunately doesn't
support coverage of sub-packages
This is ongoing work toward #214.
