This commit adds tests for the new SerializeSize functions for variable
length integers and transactions (and indirectly transaction inputs and
outputs).
This commit adds a new function named SerializeSize to the public API for
MsgTx, TxOut, and TxIn which can be used to determine how many bytes the
serialized data would take without having to actually serialize it.
The following benchmark shows the difference between using the new
function to get the serialize size for a typical transaction and
serializing into a temporary buffer and taking the length of it:
Bufffer: BenchmarkTxSerializeSizeBuffer 200000 7050 ns/op
New: BenchmarkTxSerializeSizeNew 100000000 18 ns/op
This is part of the ongoing effort to optimize serialization as noted in
conformal/btcd#27.
Most variable length integers are smaller numbers, so this commit reverses
the order of the if checks in the writeVarInt to assume smaller numbers
are more common.
This is part of the ongoing effort to optimize serialization as noted in
conformal/btcd#27
Rather than showing all errors from ProcessTransaction as an error, check
if the error is a TxRuleError meaning the transaction was rejected as
opposed to something actually going wrong and log it accordingly.
Looking up transactions from the database is an expensive operation.
This commit modifies the NotifyNewTxListener code to simply iterate the
transactions in the block instead of looking them up from the db.
Currently the wallet code needs a spent flag which ultimately shouldn't be
required. For now, the spent data is simply created on the fly which is
still significantly faster than doing database transaction lookups.
Closes#24.
This change unbreaks the case where an unknown command is sent to the
RPC server. Instead of replying back with a nil JSON id, if the
initial unmarshal was successful (and thus, the message was valid
JSON-RPC), the unmarshaled id will be used in the error reply.
We have a channel for queries and commands in server, where we pass in
args and the channel to reply from, let rpcserver use these interfaces
to provide the requistie information.
So far not all of the informaation is 100% correct, the syncpeer
information needs to be fetched from blockmanager, the subversion isn't
recorded and the number of bytes sent and recieved needs to be obtained
from btcwire. The rest should be correct.
This commit updates btcd to work with the new btcchain APIs which now
accept btcutil.Tx instead of raw btcwire.MsgTx. It also modifies the
transaction memory pool to store btcutil.Tx.
This is part of the ongoing transaction hash optimization effort noted in
conformal/btcd#25.
This change updates the doc.go documentation file with the correct use
of the new (since 8eead5217d) API used
for passing additional options when creating new script engines.
Spotted by davec@
This commit optimizes InsertBlock slightly by using the cached transaction
hashes instead of recomputing them. Also, while here, use the
ShaHash.IsEqual function to do comparisons for consistency.
This is ongoing work towards conformal/btcd#25.
ok @drahn.
This commit provides a new --cpuprofile flag that can be used to specify a
file path to write CPU profile data into. The resulting profile can then be
consumed by the 'go tool pprof' command.
Since the main SIGINT handler is running as a goroutine, the main
goroutine must be kept active long enough for it to finish or it will be
nuked when the main goroutine exits. This commit makes that happen by
slightly modifying how it waits for shutdown.
Rather than having the main goroutine only wait for the server to
shutdown, there is now a shutdown channel that is used to signal the main
goroutine to shutdown for all cases such as a graceful shutdown, a
scheduled shutdown, an RPC stop, or a SIGINT.
While here, also add a few prints to indicate a SIGINT was received and
the shutdown progress.
Profiling showed the duplicate transaction input check was taking around
6% of the total CheckTransactionSanity processing time. This was largely
due to using fmt.Sprintf to generate the map key.
This commit modifies the check instead to use the actual output as a map
key.
The following benchmark results show the difference:
Before: BenchmarkOldDuplicatInputCheck 100000 21787 ns/op
After: BenchmarkNewDuplicatInputCheck 2000000 937 ns/op
Closes#2
Profiling showed the MRU inventory handling was taking 5% of the total
block handling time. Upon inspection this is because the original
implementation was extremely inefficient using iteration to find the
oldest node for eviction.
This commit reworks it to use a map and list in order to achieve close to
O(1) performance for lookups, insertion, and eviction.
The following benchmark results show the difference:
Before: BenchmarkMruInventoryList 100 1069168 ns/op
After: BenchmarkMruInventoryList 10000000 152 ns/op
Closes#21
The ValidateTransactionScripts was recently changed to accept script flags
which pass through to the script engine in order to control its validation
behavior. This commit modifies the transaction memory pool script
validation code for this change and additionally adds the new flag to
perform canonical signtaure checking.
This removes the bip16 bool from NewScript and adds it to flags (with
the constant ScriptBip16), and also adds a new flag,
ScriptCanonicalSignatures, which will call btcec.ParseDERSignature
parsing a signature during an Execute. This is needed to emulate
bitcoind behavior, as bitcoind performs canonical signature validation
(DER format) in some places (like mempool acceptance) but not others
(like block validation).
Updated tests and test coverage file to reflect changes.
This change adds an additional signature parsing function which
performs additional checks to verify the signature is serialized in a
valid DER (and thus, unique) format, instead of allowing the less
strict BER signatures that ParseSignature will happily accept.
Added additional tests and updated test coverage to reflect changes.
This commit modifies the ValidateTransactionScripts API to accept the
recently added ScriptFlags from btcscript. This provides flexibility to
the caller to choose validation behavior based on those new flags.
Several of the bitcoin data structures contain variable length entries,
many of which have well-defined maximum limits. However, there are still
a few cases, such as variable length strings and number of transactions
which don't have clearly defined maximum limits. Instead they are only
limited by the maximum size of a message.
In order to efficiently decode messages, space is pre-allocated for the
slices which hold these variable length pieces as to avoid needing to
dynamically grow the backing arrays. Due to this however, it was
previously possible to claim extremely high slice lengths which exceed
available memory (or maximum allowed slice lengths).
This commit imposes limits to all of these cases based on calculating
the maximum possible number of elements that could fit into a message
and using those as sane upper limits.
The variable length string case was found (and tests added to hit it) by
drahn@ which prompted an audit to find all cases.
This change allows btcwallet to keep a pool of transactions that have
not yet been mined into a block, notifying wallet when transactions
are mined, as well as introducing a new way to send the
btcd:blockconnected notification with wallet-specific information as
part of the same notification. When a transaction is sent using the
RPC call 'sendrawtransaction', a notification request will be
automatically registered with the connected wallet (if using
websockets) to notify the wallet when the transaction first appears in
a block.
To perform this notification, and to avoid requiring wallets from
waiting for seperate mined tx notifications (and resend after a
timeout) or from sending an additional tx mined request for every tx
in the pool after each new block, the blockconnected notification is
now created seperately for each wallet. If the notified wallet has
sent a transaction, an additional JSON field "minedtxs" will include
an array of transaction IDs that the wallet has created and which are
included in the new block.
This new unique blockconnected notification can also be used for
additional notifications that may happen each new block in the future,
and to cut down on existing notification handlers in btcwallet, such
as for transactions to a watched address.
Rather than relying on the http package's DefaultServeMux for the RPC
server, create a unique mux specifically for the RPC server. This ensures
things, such as the http profiling handlers, do not commingle.
