This change removes transactions from a newly connected block
from the orphan pool if they exist. Additionally, any orphan
transactions that are no longer orphan transactions are moved
to the mempool and inv'd to the currently connected peers.
This commit builds on the recent limiter updates. The recent changes
introduced a new function named calcTxFee which could easily be confused
with calculating the fees of the transaction rather than its intended
purpose of calculating the minimum transaciton relay fee.
Rather than taking that approach, this commit instead renames the existing
function to calcMinRequiredTxRelayFee and uses a consistent variable name
for the serialized size of a transaction. It then moves the check for
whether or not the check should be applied based on the serialized size of
the transcation and block priority to the call site.
This approach also has the benefit of avoiding two calls to the
calculation function since it's a local at the call site.
ok @jrick, @dajohi
Before, btcd was rate limiting all transactions that had a minimum
fee of zero. Now, btcd only rate limits transactions that contain
a fee less than the calculated fee based on size.
Closes#163
This commit implements reject handling as defined by BIP0061 and bumps the
maximum supported protocol version to 70002 accordingly.
As a part of supporting this a new error type named RuleError has been
introduced which encapsulates and underlying error which could be one of
the existing TxRuleError or btcchain.RuleError types.
This allows a single high level type assertion to be used to determine if
the block or transaction was rejected due to a rule error or due to an
unexpected error. Meanwhile, an appropriate reject error can be created
from the error by pulling the underlying error out and using it.
Also, a check for minimum protocol version of 209 has been added.
Closes#133.
This commit implements the long polling portion of the getblocktemplate
RPC as defined by BIP0022. Per the specification, each block template is
returned with a longpollid which can be used in a subsequent
getblocktemplate request to keep the connection open until the server
determines the block template associated with the longpollid should be
replaced with a new one.
This is work towards #124.
This commit helps prevent transaction malleability by enforcing that the
extra dummy value on multisig transaction script contains no data for a
transaction . This syncs with a recent change in Bitcoin Core to remain
compatible.
As part of this change a new constant has been introduced which is used to
specify the script flags which are used for standard transactions. This
constant is then used in both the memory pool and the mining code to
ensure they remain in sync with one another.
Closes#131.
ok @jrick, @dajohi
This commit raises the maximum allowed size for a standard signature
script to cover a 15-of-15 multi-signature pay-to-script-hash with
compressed pubkeys.
This mirrors a recent change to remain compatible with Bitcoin Core.
ok @owinga who also helped verify and correct the script math.
Closes#128.
This change modifies the params struct to embed a *btcnet.Params,
removing the old parameter fields that are handled by the btcnet
package.
Hardcoded network checks have also been removed in favor of modifying
behavior based on the current active net's parameters.
Not all library packages, notable btcutil and btcchain, have been
updated to use btcnet yet, but with this change, each package can be
updated one at a time since the active net's btcnet.Params are
available at each callsite.
ok @davecgh
This commit implements a rebroadcast handler which deals with
rebroadcasting inventory at a random time interval between 0 and 30
minutes. It then uses the new rebroadcast logic to ensure transactions
which were submitted via the sendrawtransaction RPC are rebroadcast until
they make it into a block.
Closes#99.
This commit adds a new function named NewBlockTemplate along with
supporting infrastructure which is part of the core functionality needed
to support mining.
In particular the function creates a new block template which contains a
fully populated block with a zero nonce that is ready to be solved as well
as additional information regarding the fees and number of signature
operations for each transaction included in the block. The specific
transaction selection logic mirrors the reference implementation.
Various cleanup, optimizations, and comment suggestions provided by
@owainga. Also contains some naming suggestions and comment fixes from
@flammit.
This commit removes the word "only" from the non-standard transaction
error message when the number of items on the stack does not match the
number of expected items.
This was suggested by @mbelshe.
This change modifies the RPC server's notifiation manager from a
struct with requests, protected by a mutux, to two goroutines. The
first maintains a queue of all notifications and control requests
(registering/unregistering notifications), while the second reads from
the queue and processes notifications and requests one at a time.
Previously, to prevent slowing down block and mempool processing, each
notification would be handled by spawning a new goroutine. This lead
to cases where notifications would end up being sent to clients in a
different order than they were created. Adding a queue keeps the
order of notifications originating from the same goroutine, while also
not slowing down processing while waiting for notifications to be
processed and sent.
ok @davecgh
This mirrors a recent change in the reference implementation. Since the
vast majority of the network still uses it and transactions which aren't
considered standard won't be relayed, the rules for standard transactions
need to be kept consistent.
Ideally this should be handled via floating fees, but we're rather limited
until we have miner support and a larger network share so the probability
of a transaction being relayed and confirmed is high.
Closes#100.
ok @owainga
This change improves the mechanism by which btcd notifies a websocket
client of transaction relating to watched address and unspent outputs
in the following ways:
1. The old processedtx notification has been replaced with the new
recvtx notification. This notification, rather than parsing out
details used by wallet clients, sends the serialized transaction
(as hexadecimal) and any block details (if mined) if any transaction
output sends to one of the websocket client's watched addresses.
2. The old txspent notification has been replaced with the new
redeemingtx notification. This notification, rather than parsing
out details used by wallet clients, sends the serialized transaction
(as hexadecimal) and any block details (if mined) if any transaction
input spends a watched output.
3. When processing notifications for transaction outputs, if any output
spends to a client's watched address, a corresponding spent request
is automatically registered.
4. Transaction notifications originating from mempool now include both
transaction inputs and outputs, rather than only processing
5. When processing notifications for transaction inputs, a client's
output spent request is only removed if the transaction being
processed has also been mined into a block. In combination with the
4th change, this results in two redeemingtx notifications for
transactions which first appear in mempool and are subsequently mined
into a block.
This commit makes use of the new btcscript.HasCanonicalPushes to enforce
canonical data pushes for transactions that are considered standard.
A canonical data push is one where the fewest number of bytes possible to
encode the size of the data being pushed is used. This includes using the
small integer opcodes for single byte data that can be represented
directly.
This commit increases the block priority size to 50000 and the max
standard tx size to 100k. This matches relatively recent changes in the
reference implementation. The max block size was also increased to
750000, but since btcd does not currently create blocks, there is no
constant for it. That constant will likely be added as a part of the
getwork implementation since it requires block creation.
Closes#71.
This commit refactors the entire websocket client code to resolve several
issues with the previous implementation. Note that this commit does not
change the public API for websockets. It only consists of internal
improvements.
The following is the major issues which have been addressed:
- A slow websocket client could impede notifications to all clients
- Long-running operations such as rescans would block all other requests
until it had completed
- The above two points taken together could lead to apparant hangs since
the client doing the rescan would eventually run out of channel buffer
and block the entire group of clients until the rescan completed
- Disconnecting a websocket during certain operations could lead to a hang
- Stopping the rpc server with operations under way could lead to a hang
- There were no limits to the number of websocket clients that could
connect
The following is a summary of the major changes:
- The websocket code has been split into two entities: a
connection/notification manager and a websocket client
- The new connection/notification manager acts as the entry point from
the rest of the subsystems to feed data which potentially needs to
notify clients
- Each websocket client now has its own instance of the new websocket
client type which controls its own lifecycle
- The data flow has been completely redesigned to closely resemble the
peer data flow
- Each websocket now has its own long-lived goroutines for input, output,
and queuing of notifications
- Notifications use the new notification queue goroutine along with
queueing to ensure they dont't block on stalled or slow peers
- There is a new infrastructure for asynchronously executing long-running
commands such as a rescan while still allowing the faster operations to
continue to be serviced by the same client
- Since long-running operations now run asynchronously, they have been
limited to one at a time
- Added a limit of 10 websocket clients. This is hard coded for now, but
will be made configurable in the future
Taken together these changes make the code far easier to reason about and
update as well solve the aforementioned issues.
Further optimizations to improve performance are possible in regards to
the way the connection/notification manager works, however this commit
already contains a ton of changes, so they are being left for another
time.
Changed mempool.MaybeAcceptTransaction to accept an additional parameter
to differentiate betwee new transactions and those added from
disconnected blocks.
Added new fields to requestContexts to indicate which clients want to
receive all new transaction notifications.
Added NotifyForNewTx to rpcServer to deliver approriate transaction
notification.
The fee field of the getrawmempool RPC JSON response should be in Bitcoins
instead of Satoshi. This commit corrects that issue.
Also, add a couple of comments and fix a comment typo while here.
Since there is already a variable for the current block height in addition
to the next block height, use the existing curHeight variable instead
doing nextBlockHeight-1 in mempool add.
Also, make every subsystem within btcd use its own logger instance so each
subsystem can have its own level specified independent of the others.
This is work towards #48.
- Lock the mempool when removing transactions during a notification as
intended
- When generating the inventory vectors to serve on a mempool request,
recheck the memory pool for each hash since it's possible another thread
could have removed an entry after the initial query for available
hashes
- When a block is connected, remove any transactions which are now double
spends as a result of the newly connected transactions
When a transaction is being checked for acceptance into the transation
memory pool, it makes use of a chain function to ensure the invariant rules
for what transactions are allowed into a block are not violated. That
function returns a btcchain.RuleError if rules are violated. However,
since this code path is tailored to free-standing transactions, the error
needs to be converted to a TxRuleError so the caller can properly detect
the transaction as a rejected transaction instead of treating it like an
real error.
This commit modifies the transaction memory pool handling so that it does
not relay resurrected transactions. The other peers on the network will
also be reorganizing to the same block, so they already know about them.
This commit adds a few more checks to restrict what transactions are
allowed into the transaction memory pool and therefore are candidates
to be mined and relayed.
In particular, the following changes were made to what is considered
standard:
- nulldata scripts are now supported and considered standard
- multi-signature transaction are now checked to ensure they only have a
max of 3 pubkeys and the number of signatures doesn't exceed the number
of pubkeys
- the number of inputs to a signature script must now match the expected
number of inputs for the script type (includes support for additional
pay-to-script-hash inputs)
- the number of inputs pushed onto the stack by a redeeming sig script
must match the number of inputs consumed by the referenced pk script
- there can now only be a max of one nulldata output per transaction
This commit changes the various cases that were serializing transactions
into a buffer and taking the length to use the new faster SerializeSize
API. It also completes a TODO since the serialized size of a transaction
output is now available.
This commit updates btcd to work with the new btcchain APIs which now
accept btcutil.Tx instead of raw btcwire.MsgTx. It also modifies the
transaction memory pool to store btcutil.Tx.
This is part of the ongoing transaction hash optimization effort noted in
conformal/btcd#25.
The ValidateTransactionScripts was recently changed to accept script flags
which pass through to the script engine in order to control its validation
behavior. This commit modifies the transaction memory pool script
validation code for this change and additionally adds the new flag to
perform canonical signtaure checking.
It was previously possible for the unprotected iteration of the mempool
orphans to lead to undefined results. This commit remedies that by
reworking the locking code a bit. It also embeds the mutex directly into
the mempool struct rather than having a separate field for it so the
syntax is a slightly cleaner.
This commit adds code to properly respond to getdata requests for
transactions by fetching them from the transaction pool. Previously, we
advertised newly available transactions, but the code to respond with the
actual transaction was not written yet.
Also, fix a couple of comments and make the pushTxMsg and pushBlockMsg
functions consistent.
This commit is a first pass at improving the logging. It changes a number
of things to improve the readability of the output. The biggest addition
is message summaries for each message type when using the debug logging
level.
There is sitll more to do here such as allowing the level of each
subsystem to be independently specified, syslog support, and allowing the
logging level to be changed run-time.
The block manager handles inventory messges to know which inventory should
be requested based on what is already known and what is already in flight.
So, this commit adds logic to ask the transaction memory pool if the
transaction is already known before requesting it and tracks pending
requests into an in-flight transaction map owned by the block manager.
It also moves the transaction processing into the block manager so the
in-flight map can be properly cleaned.
Also, the loops which only remove a single element and break or return
don't need the extra logic for iteration since they don't continue
iteration after removal.
It is not safe to remove an item from a container/list while iterating the
list without first saving the next pointer since removing the item nils
the internal list element's next pointer.
This commit is a rather large one which implements transaction pool and
relay according to the protocol rules of the reference implementation.
It makes use of btcchain to ensure the transactions are valid for the
block chain and includes several stricter checks which determine if they
are "standard" or not before admitting them into the pool and relaying
them.
There are still a few TODOs around the more strict rules which determine
which transactions are willing to be mined, but the core checks which
are imperative (everything except the all of the "standard" checks really)
to operate as a good citizen on the bitcoin network are in place.
