lbcd/wire
Dave Collins f68cd7422d wire: Reduce allocs with a binary free list.
This introduces a new binary free list which provides a concurrent safe
list of unused buffers for the purpose of serializing and deserializing
primitive integers to their raw binary bytes.

For convenience, the type also provides functions for each of the
primitive unsigned integers that automatically obtain a buffer from the
free list, perform the necessary binary conversion, read from or write
to the given io.Reader or io.Writer, and return the buffer to the free
list.

A global instance of the type has been introduced with a maximum number
of 1024 items. Since each buffer is 8 bytes, it will consume a maximum
of 8KB.  Theoretically, this value would only allow up to 1024 peers
simultaneously reading and writing without having to resort to burdening
the garbage collector with additional allocations.  However, due to the
fact the code is designed in such a way that the buffers are quickly
used and returned to the free list, in practice it can support much more
than 1024 peers without involving the garbage collector since it is
highly unlikely every peer would need a buffer at the exact same time.

The following is a before and after comparison of the allocations
with the benchmarks that did not change removed:

benchmark              old allocs     new allocs     delta
-------------------------------------------------------------
WriteVarInt1           1              0              -100.00%
WriteVarInt3           1              0              -100.00%
WriteVarInt5           1              0              -100.00%
WriteVarInt9           1              0              -100.00%
ReadVarInt1            1              0              -100.00%
ReadVarInt3            1              0              -100.00%
ReadVarInt5            1              0              -100.00%
ReadVarInt9            1              0              -100.00%
ReadVarStr4            3              2              -33.33%
ReadVarStr10           3              2              -33.33%
WriteVarStr4           2              1              -50.00%
WriteVarStr10          2              1              -50.00%
ReadOutPoint           1              0              -100.00%
WriteOutPoint          1              0              -100.00%
ReadTxOut              3              1              -66.67%
WriteTxOut             2              0              -100.00%
ReadTxIn               5              2              -60.00%
WriteTxIn              3              0              -100.00%
DeserializeTxSmall     15             7              -53.33%
DeserializeTxLarge     33428          16715          -50.00%
SerializeTx            8              0              -100.00%
ReadBlockHeader        7              1              -85.71%
WriteBlockHeader       10             4              -60.00%
DecodeGetHeaders       1004           501            -50.10%
DecodeHeaders          18002          4001           -77.77%
DecodeGetBlocks        1004           501            -50.10%
DecodeAddr             9002           4001           -55.55%
DecodeInv              150005         50003          -66.67%
DecodeNotFound         150004         50002          -66.67%
DecodeMerkleBlock      222            108            -51.35%
TxSha                  10             2              -80.00%
2016-06-03 17:08:31 -05:00
..
testdata wire: Add large tx deserialize benchmark. (#678) 2016-04-25 16:51:27 -05:00
bench_test.go wire: Add several decode benchmarks. (#682) 2016-04-25 17:32:29 -05:00
blockheader.go Implement BIP0065 changeover logic for v4 blocks. 2015-10-28 13:28:50 -04:00
blockheader_test.go Implmement BlockHeader BtcEncode/BtcDecode. 2015-07-22 11:34:18 -04:00
common.go wire: Reduce allocs with a binary free list. 2016-06-03 17:08:31 -05:00
common_test.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
doc.go wire: Implement sendheaders command (BIP0130) 2016-02-05 12:41:39 -05:00
error.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
fakeconn_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
fakemessage_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
fixedIO_test.go multi: Fix several misspellings in the comments. 2016-02-25 11:17:12 -06:00
internal_test.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
invvect.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
invvect_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
message.go wire: Implement sendheaders command (BIP0130) 2016-02-05 12:41:39 -05:00
message_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgaddr.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
msgaddr_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgalert.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
msgalert_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgblock.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
msgblock_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgfilteradd.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
msgfilteradd_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgfilterclear.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgfilterclear_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgfilterload.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
msgfilterload_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msggetaddr.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msggetaddr_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msggetblocks.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
msggetblocks_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msggetdata.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
msggetdata_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msggetheaders.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
msggetheaders_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgheaders.go blockchain: Rework to use new db interface. 2016-04-11 16:47:27 -05:00
msgheaders_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msginv.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
msginv_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgmempool.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgmempool_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgmerkleblock.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
msgmerkleblock_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgnotfound.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
msgnotfound_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgping.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgping_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgpong.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgpong_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgreject.go wire: Export var length string serialization funcs. 2015-10-16 11:24:45 -05:00
msgreject_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgsendheaders.go wire: Implement sendheaders command (BIP0130) 2016-02-05 12:41:39 -05:00
msgsendheaders_test.go wire: Implement sendheaders command (BIP0130) 2016-02-05 12:41:39 -05:00
msgtx.go wire: Reduce allocs with a binary free list. 2016-06-03 17:08:31 -05:00
msgtx_test.go wire: Remove duplicate tx error path tests. 2015-07-26 13:05:09 -05:00
msgverack.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgverack_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
msgversion.go wire: Implement sendheaders command (BIP0130) 2016-02-05 12:41:39 -05:00
msgversion_test.go wire: Export (read|write)(VarInt|VarBytes). 2016-02-22 18:11:58 +01:00
netaddress.go wire: Reduce allocs with a binary free list. 2016-06-03 17:08:31 -05:00
netaddress_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00
protocol.go wire: Implement sendheaders command (BIP0130) 2016-02-05 12:41:39 -05:00
protocol_test.go wire: Implement SFNodeBloom (BIP0111). 2015-09-28 16:25:44 -04:00
README.md docs: Make various README.md files consistent. 2015-10-23 14:51:36 -05:00
shahash.go Correct a few style related issues found by golint. 2015-10-20 10:34:14 -05:00
shahash_test.go Relicense to the btcsuite developers. 2015-05-01 12:00:56 -05:00

wire

[Build Status] (https://travis-ci.org/btcsuite/btcd) ![ISC License] (http://img.shields.io/badge/license-ISC-blue.svg) [GoDoc] (http://godoc.org/github.com/btcsuite/btcd/wire)

Package wire implements the bitcoin wire protocol. A comprehensive suite of tests with 100% test coverage is provided to ensure proper functionality.

There is an associated blog post about the release of this package here.

This package has intentionally been designed so it can be used as a standalone package for any projects needing to interface with bitcoin peers at the wire protocol level.

Installation and Updating

$ go get -u github.com/btcsuite/btcd/wire

Bitcoin Message Overview

The bitcoin protocol consists of exchanging messages between peers. Each message is preceded by a header which identifies information about it such as which bitcoin network it is a part of, its type, how big it is, and a checksum to verify validity. All encoding and decoding of message headers is handled by this package.

To accomplish this, there is a generic interface for bitcoin messages named Message which allows messages of any type to be read, written, or passed around through channels, functions, etc. In addition, concrete implementations of most of the currently supported bitcoin messages are provided. For these supported messages, all of the details of marshalling and unmarshalling to and from the wire using bitcoin encoding are handled so the caller doesn't have to concern themselves with the specifics.

Reading Messages Example

In order to unmarshal bitcoin messages from the wire, use the ReadMessage function. It accepts any io.Reader, but typically this will be a net.Conn to a remote node running a bitcoin peer. Example syntax is:

	// Use the most recent protocol version supported by the package and the
	// main bitcoin network.
	pver := wire.ProtocolVersion
	btcnet := wire.MainNet

	// Reads and validates the next bitcoin message from conn using the
	// protocol version pver and the bitcoin network btcnet.  The returns
	// are a wire.Message, a []byte which contains the unmarshalled
	// raw payload, and a possible error.
	msg, rawPayload, err := wire.ReadMessage(conn, pver, btcnet)
	if err != nil {
		// Log and handle the error
	}

See the package documentation for details on determining the message type.

Writing Messages Example

In order to marshal bitcoin messages to the wire, use the WriteMessage function. It accepts any io.Writer, but typically this will be a net.Conn to a remote node running a bitcoin peer. Example syntax to request addresses from a remote peer is:

	// Use the most recent protocol version supported by the package and the
	// main bitcoin network.
	pver := wire.ProtocolVersion
	btcnet := wire.MainNet

	// Create a new getaddr bitcoin message.
	msg := wire.NewMsgGetAddr()

	// Writes a bitcoin message msg to conn using the protocol version
	// pver, and the bitcoin network btcnet.  The return is a possible
	// error.
	err := wire.WriteMessage(conn, msg, pver, btcnet)
	if err != nil {
		// Log and handle the error
	}

GPG Verification Key

All official release tags are signed by Conformal so users can ensure the code has not been tampered with and is coming from the btcsuite developers. To verify the signature perform the following:

  • Download the public key from the Conformal website at https://opensource.conformal.com/GIT-GPG-KEY-conformal.txt

  • Import the public key into your GPG keyring:

    gpg --import GIT-GPG-KEY-conformal.txt
    
  • Verify the release tag with the following command where TAG_NAME is a placeholder for the specific tag:

    git tag -v TAG_NAME
    

License

Package wire is licensed under the copyfree ISC License.