lbcd/msgblock.go

251 lines
8.6 KiB
Go
Raw Normal View History

2014-01-09 06:44:08 +01:00
// Copyright (c) 2013-2014 Conformal Systems LLC.
2013-05-08 21:31:00 +02:00
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.
package btcwire
import (
"bytes"
"fmt"
2013-05-08 21:31:00 +02:00
"io"
)
// defaultTransactionAlloc is the default size used for the backing array
// for transactions. The transaction array will dynamically grow as needed, but
// this figure is intended to provide enough space for the number of
// transactions in the vast majority of blocks without needing to grow the
// backing array multiple times.
const defaultTransactionAlloc = 2048
2013-05-08 21:31:00 +02:00
// MaxBlocksPerMsg is the maximum number of blocks allowed per message.
const MaxBlocksPerMsg = 500
// MaxBlockPayload is the maximum bytes a block message can be in bytes.
const MaxBlockPayload = 1000000 // Not actually 1MB which would be 1024 * 1024
// maxTxPerBlock is the maximum number of transactions that could
// possibly fit into a block.
const maxTxPerBlock = (MaxBlockPayload / minTxPayload) + 1
2013-05-08 21:31:00 +02:00
// TxLoc holds locator data for the offset and length of where a transaction is
// located within a MsgBlock data buffer.
type TxLoc struct {
TxStart int
TxLen int
}
// MsgBlock implements the Message interface and represents a bitcoin
// block message. It is used to deliver block and transaction information in
// response to a getdata message (MsgGetData) for a given block hash.
type MsgBlock struct {
Header BlockHeader
Transactions []*MsgTx
}
Remove BlockHeader.TxnCount field. This commit removes the TxnCount field from the BlockHeader type and updates the tests accordingly. Note that this change does not affect the actual wire protocol encoding in any way. The reason the field has been removed is it really doesn't belong there even though the wire protocol wiki entry on the official bitcoin wiki implies it does. The implication is an artifact from the way the reference implementation serializes headers (MsgHeaders) messages. It includes the transaction count, which is naturally always 0 for headers, along with every header. However, in reality, a block header does not include the transaction count. This can be evidenced by looking at how a block hash is calculated. It is only up to and including the Nonce field (a total of 80 bytes). From an API standpoint, having the field as part of the BlockHeader type results in several odd cases. For example, the transaction count for MsgBlocks (the only place that actually has a real transaction count since MsgHeaders does not) is available by taking the len of the Transactions slice. As such, having the extra field in the BlockHeader is really a useless field that could potentially get out of sync and cause the encode to fail. Another example is related to deserializing a block header from the database in order to serve it in response to a getheaders (MsgGetheaders) request. If a block header is assumed to have the transaction count as a part of it, then derserializing a block header not only consumes more than the 80 bytes that actually comprise the header as stated above, but you then need to change the transaction count to 0 before sending the headers (MsgHeaders) message. So, not only are you reading and deserializing more bytes than needed, but worse, you generally have to make a copy of it so you can change the transaction count without busting cached headers. This is part 1 of #13.
2014-01-19 02:37:33 +01:00
// AddTransaction adds a transaction to the message.
2013-05-08 21:31:00 +02:00
func (msg *MsgBlock) AddTransaction(tx *MsgTx) error {
msg.Transactions = append(msg.Transactions, tx)
return nil
}
Remove BlockHeader.TxnCount field. This commit removes the TxnCount field from the BlockHeader type and updates the tests accordingly. Note that this change does not affect the actual wire protocol encoding in any way. The reason the field has been removed is it really doesn't belong there even though the wire protocol wiki entry on the official bitcoin wiki implies it does. The implication is an artifact from the way the reference implementation serializes headers (MsgHeaders) messages. It includes the transaction count, which is naturally always 0 for headers, along with every header. However, in reality, a block header does not include the transaction count. This can be evidenced by looking at how a block hash is calculated. It is only up to and including the Nonce field (a total of 80 bytes). From an API standpoint, having the field as part of the BlockHeader type results in several odd cases. For example, the transaction count for MsgBlocks (the only place that actually has a real transaction count since MsgHeaders does not) is available by taking the len of the Transactions slice. As such, having the extra field in the BlockHeader is really a useless field that could potentially get out of sync and cause the encode to fail. Another example is related to deserializing a block header from the database in order to serve it in response to a getheaders (MsgGetheaders) request. If a block header is assumed to have the transaction count as a part of it, then derserializing a block header not only consumes more than the 80 bytes that actually comprise the header as stated above, but you then need to change the transaction count to 0 before sending the headers (MsgHeaders) message. So, not only are you reading and deserializing more bytes than needed, but worse, you generally have to make a copy of it so you can change the transaction count without busting cached headers. This is part 1 of #13.
2014-01-19 02:37:33 +01:00
// ClearTransactions removes all transactions from the message.
2013-05-08 21:31:00 +02:00
func (msg *MsgBlock) ClearTransactions() {
msg.Transactions = make([]*MsgTx, 0, defaultTransactionAlloc)
2013-05-08 21:31:00 +02:00
}
// BtcDecode decodes r using the bitcoin protocol encoding into the receiver.
// This is part of the Message interface implementation.
// See Deserialize for decoding blocks stored to disk, such as in a database, as
// opposed to decoding blocks from the wire.
2013-05-08 21:31:00 +02:00
func (msg *MsgBlock) BtcDecode(r io.Reader, pver uint32) error {
err := readBlockHeader(r, pver, &msg.Header)
if err != nil {
return err
}
Remove BlockHeader.TxnCount field. This commit removes the TxnCount field from the BlockHeader type and updates the tests accordingly. Note that this change does not affect the actual wire protocol encoding in any way. The reason the field has been removed is it really doesn't belong there even though the wire protocol wiki entry on the official bitcoin wiki implies it does. The implication is an artifact from the way the reference implementation serializes headers (MsgHeaders) messages. It includes the transaction count, which is naturally always 0 for headers, along with every header. However, in reality, a block header does not include the transaction count. This can be evidenced by looking at how a block hash is calculated. It is only up to and including the Nonce field (a total of 80 bytes). From an API standpoint, having the field as part of the BlockHeader type results in several odd cases. For example, the transaction count for MsgBlocks (the only place that actually has a real transaction count since MsgHeaders does not) is available by taking the len of the Transactions slice. As such, having the extra field in the BlockHeader is really a useless field that could potentially get out of sync and cause the encode to fail. Another example is related to deserializing a block header from the database in order to serve it in response to a getheaders (MsgGetheaders) request. If a block header is assumed to have the transaction count as a part of it, then derserializing a block header not only consumes more than the 80 bytes that actually comprise the header as stated above, but you then need to change the transaction count to 0 before sending the headers (MsgHeaders) message. So, not only are you reading and deserializing more bytes than needed, but worse, you generally have to make a copy of it so you can change the transaction count without busting cached headers. This is part 1 of #13.
2014-01-19 02:37:33 +01:00
txCount, err := readVarInt(r, pver)
if err != nil {
return err
}
// Prevent more transactions than could possibly fit into a block.
// It would be possible to cause memory exhaustion and panics without
// a sane upper bound on this count.
if txCount > maxTxPerBlock {
str := fmt.Sprintf("too many transactions to fit into a block "+
"[count %d, max %d]", txCount, maxTxPerBlock)
return messageError("MsgBlock.BtcDecode", str)
}
msg.Transactions = make([]*MsgTx, 0, txCount)
for i := uint64(0); i < txCount; i++ {
2013-05-08 21:31:00 +02:00
tx := MsgTx{}
err := tx.BtcDecode(r, pver)
if err != nil {
return err
}
msg.Transactions = append(msg.Transactions, &tx)
}
return nil
}
// Deserialize decodes a block from r into the receiver using a format that is
// suitable for long-term storage such as a database while respecting the
// Version field in the block. This function differs from BtcDecode in that
// BtcDecode decodes from the bitcoin wire protocol as it was sent across the
// network. The wire encoding can technically differ depending on the protocol
// version and doesn't even really need to match the format of a stored block at
// all. As of the time this comment was written, the encoded block is the same
// in both instances, but there is a distinct difference and separating the two
// allows the API to be flexible enough to deal with changes.
func (msg *MsgBlock) Deserialize(r io.Reader) error {
// At the current time, there is no difference between the wire encoding
// at protocol version 0 and the stable long-term storage format. As
// a result, make use of BtcDecode.
return msg.BtcDecode(r, 0)
}
// DeserializeTxLoc decodes r in the same manner Deserialize does, but it takes
// a byte buffer instead of a generic reader and returns a slice containing the start and length of
// each transaction within the raw data that is being deserialized.
func (msg *MsgBlock) DeserializeTxLoc(r *bytes.Buffer) ([]TxLoc, error) {
fullLen := r.Len()
// At the current time, there is no difference between the wire encoding
// at protocol version 0 and the stable long-term storage format. As
// a result, make use of existing wire protocol functions.
err := readBlockHeader(r, 0, &msg.Header)
if err != nil {
return nil, err
}
Remove BlockHeader.TxnCount field. This commit removes the TxnCount field from the BlockHeader type and updates the tests accordingly. Note that this change does not affect the actual wire protocol encoding in any way. The reason the field has been removed is it really doesn't belong there even though the wire protocol wiki entry on the official bitcoin wiki implies it does. The implication is an artifact from the way the reference implementation serializes headers (MsgHeaders) messages. It includes the transaction count, which is naturally always 0 for headers, along with every header. However, in reality, a block header does not include the transaction count. This can be evidenced by looking at how a block hash is calculated. It is only up to and including the Nonce field (a total of 80 bytes). From an API standpoint, having the field as part of the BlockHeader type results in several odd cases. For example, the transaction count for MsgBlocks (the only place that actually has a real transaction count since MsgHeaders does not) is available by taking the len of the Transactions slice. As such, having the extra field in the BlockHeader is really a useless field that could potentially get out of sync and cause the encode to fail. Another example is related to deserializing a block header from the database in order to serve it in response to a getheaders (MsgGetheaders) request. If a block header is assumed to have the transaction count as a part of it, then derserializing a block header not only consumes more than the 80 bytes that actually comprise the header as stated above, but you then need to change the transaction count to 0 before sending the headers (MsgHeaders) message. So, not only are you reading and deserializing more bytes than needed, but worse, you generally have to make a copy of it so you can change the transaction count without busting cached headers. This is part 1 of #13.
2014-01-19 02:37:33 +01:00
txCount, err := readVarInt(r, 0)
if err != nil {
return nil, err
}
// Prevent more transactions than could possibly fit into a block.
// It would be possible to cause memory exhaustion and panics without
// a sane upper bound on this count.
if txCount > maxTxPerBlock {
str := fmt.Sprintf("too many transactions to fit into a block "+
"[count %d, max %d]", txCount, maxTxPerBlock)
return nil, messageError("MsgBlock.DeserializeTxLoc", str)
}
// Deserialize each transaction while keeping track of its location
// within the byte stream.
msg.Transactions = make([]*MsgTx, 0, txCount)
txLocs := make([]TxLoc, txCount)
for i := uint64(0); i < txCount; i++ {
txLocs[i].TxStart = fullLen - r.Len()
tx := MsgTx{}
err := tx.Deserialize(r)
if err != nil {
return nil, err
}
msg.Transactions = append(msg.Transactions, &tx)
txLocs[i].TxLen = (fullLen - r.Len()) - txLocs[i].TxStart
}
return txLocs, nil
}
2013-05-08 21:31:00 +02:00
// BtcEncode encodes the receiver to w using the bitcoin protocol encoding.
// This is part of the Message interface implementation.
// See Serialize for encoding blocks to be stored to disk, such as in a
// database, as opposed to encoding blocks for the wire.
2013-05-08 21:31:00 +02:00
func (msg *MsgBlock) BtcEncode(w io.Writer, pver uint32) error {
err := writeBlockHeader(w, pver, &msg.Header)
if err != nil {
return err
}
Remove BlockHeader.TxnCount field. This commit removes the TxnCount field from the BlockHeader type and updates the tests accordingly. Note that this change does not affect the actual wire protocol encoding in any way. The reason the field has been removed is it really doesn't belong there even though the wire protocol wiki entry on the official bitcoin wiki implies it does. The implication is an artifact from the way the reference implementation serializes headers (MsgHeaders) messages. It includes the transaction count, which is naturally always 0 for headers, along with every header. However, in reality, a block header does not include the transaction count. This can be evidenced by looking at how a block hash is calculated. It is only up to and including the Nonce field (a total of 80 bytes). From an API standpoint, having the field as part of the BlockHeader type results in several odd cases. For example, the transaction count for MsgBlocks (the only place that actually has a real transaction count since MsgHeaders does not) is available by taking the len of the Transactions slice. As such, having the extra field in the BlockHeader is really a useless field that could potentially get out of sync and cause the encode to fail. Another example is related to deserializing a block header from the database in order to serve it in response to a getheaders (MsgGetheaders) request. If a block header is assumed to have the transaction count as a part of it, then derserializing a block header not only consumes more than the 80 bytes that actually comprise the header as stated above, but you then need to change the transaction count to 0 before sending the headers (MsgHeaders) message. So, not only are you reading and deserializing more bytes than needed, but worse, you generally have to make a copy of it so you can change the transaction count without busting cached headers. This is part 1 of #13.
2014-01-19 02:37:33 +01:00
err = writeVarInt(w, pver, uint64(len(msg.Transactions)))
if err != nil {
return err
}
2013-05-08 21:31:00 +02:00
for _, tx := range msg.Transactions {
err = tx.BtcEncode(w, pver)
if err != nil {
return err
}
}
return nil
}
// Serialize encodes the block to w using a format that suitable for long-term
// storage such as a database while respecting the Version field in the block.
// This function differs from BtcEncode in that BtcEncode encodes the block to
// the bitcoin wire protocol in order to be sent across the network. The wire
// encoding can technically differ depending on the protocol version and doesn't
// even really need to match the format of a stored block at all. As of the
// time this comment was written, the encoded block is the same in both
// instances, but there is a distinct difference and separating the two allows
// the API to be flexible enough to deal with changes.
func (msg *MsgBlock) Serialize(w io.Writer) error {
// At the current time, there is no difference between the wire encoding
// at protocol version 0 and the stable long-term storage format. As
// a result, make use of BtcEncode.
return msg.BtcEncode(w, 0)
}
// SerializeSize returns the number of bytes it would take to serialize the
// the block.
func (msg *MsgBlock) SerializeSize() int {
// Block header bytes + Serialized varint size for the number of
// transactions.
n := blockHeaderLen + VarIntSerializeSize(uint64(len(msg.Transactions)))
for _, tx := range msg.Transactions {
n += tx.SerializeSize()
}
return n
}
2013-05-08 21:31:00 +02:00
// Command returns the protocol command string for the message. This is part
// of the Message interface implementation.
func (msg *MsgBlock) Command() string {
return CmdBlock
2013-05-08 21:31:00 +02:00
}
// MaxPayloadLength returns the maximum length the payload can be for the
// receiver. This is part of the Message interface implementation.
func (msg *MsgBlock) MaxPayloadLength(pver uint32) uint32 {
Remove BlockHeader.TxnCount field. This commit removes the TxnCount field from the BlockHeader type and updates the tests accordingly. Note that this change does not affect the actual wire protocol encoding in any way. The reason the field has been removed is it really doesn't belong there even though the wire protocol wiki entry on the official bitcoin wiki implies it does. The implication is an artifact from the way the reference implementation serializes headers (MsgHeaders) messages. It includes the transaction count, which is naturally always 0 for headers, along with every header. However, in reality, a block header does not include the transaction count. This can be evidenced by looking at how a block hash is calculated. It is only up to and including the Nonce field (a total of 80 bytes). From an API standpoint, having the field as part of the BlockHeader type results in several odd cases. For example, the transaction count for MsgBlocks (the only place that actually has a real transaction count since MsgHeaders does not) is available by taking the len of the Transactions slice. As such, having the extra field in the BlockHeader is really a useless field that could potentially get out of sync and cause the encode to fail. Another example is related to deserializing a block header from the database in order to serve it in response to a getheaders (MsgGetheaders) request. If a block header is assumed to have the transaction count as a part of it, then derserializing a block header not only consumes more than the 80 bytes that actually comprise the header as stated above, but you then need to change the transaction count to 0 before sending the headers (MsgHeaders) message. So, not only are you reading and deserializing more bytes than needed, but worse, you generally have to make a copy of it so you can change the transaction count without busting cached headers. This is part 1 of #13.
2014-01-19 02:37:33 +01:00
// Block header at 80 bytes + transaction count + max transactions
// which can vary up to the MaxBlockPayload (including the block header
// and transaction count).
return MaxBlockPayload
2013-05-08 21:31:00 +02:00
}
// BlockSha computes the block identifier hash for this block.
func (msg *MsgBlock) BlockSha() (ShaHash, error) {
return msg.Header.BlockSha()
2013-05-08 21:31:00 +02:00
}
// TxShas returns a slice of hashes of all of transactions in this block.
func (msg *MsgBlock) TxShas() ([]ShaHash, error) {
shaList := make([]ShaHash, 0, len(msg.Transactions))
2013-05-08 21:31:00 +02:00
for _, tx := range msg.Transactions {
// Ignore error here since TxSha can't fail in the current
// implementation except due to run-time panics.
sha, _ := tx.TxSha()
2013-05-08 21:31:00 +02:00
shaList = append(shaList, sha)
}
return shaList, nil
}
// NewMsgBlock returns a new bitcoin block message that conforms to the
// Message interface. See MsgBlock for details.
func NewMsgBlock(blockHeader *BlockHeader) *MsgBlock {
return &MsgBlock{
Header: *blockHeader,
Transactions: make([]*MsgTx, 0, defaultTransactionAlloc),
2013-05-08 21:31:00 +02:00
}
}