Previously the websocket notifications for addresses were limited to
pay-to-pubkey-hash only. This commit removes that restriction so
all btcutil.Address types are supported. This includes pay-to-pubkey,
pay-to-pubkey-hash, and pay-to-script-hash.
When a spent notification and address notification is removed, the
tracking entry in the client which is used to track what to remove on
shutdown needs to be removed as well.
This commit refactors the entire websocket client code to resolve several
issues with the previous implementation. Note that this commit does not
change the public API for websockets. It only consists of internal
improvements.
The following is the major issues which have been addressed:
- A slow websocket client could impede notifications to all clients
- Long-running operations such as rescans would block all other requests
until it had completed
- The above two points taken together could lead to apparant hangs since
the client doing the rescan would eventually run out of channel buffer
and block the entire group of clients until the rescan completed
- Disconnecting a websocket during certain operations could lead to a hang
- Stopping the rpc server with operations under way could lead to a hang
- There were no limits to the number of websocket clients that could
connect
The following is a summary of the major changes:
- The websocket code has been split into two entities: a
connection/notification manager and a websocket client
- The new connection/notification manager acts as the entry point from
the rest of the subsystems to feed data which potentially needs to
notify clients
- Each websocket client now has its own instance of the new websocket
client type which controls its own lifecycle
- The data flow has been completely redesigned to closely resemble the
peer data flow
- Each websocket now has its own long-lived goroutines for input, output,
and queuing of notifications
- Notifications use the new notification queue goroutine along with
queueing to ensure they dont't block on stalled or slow peers
- There is a new infrastructure for asynchronously executing long-running
commands such as a rescan while still allowing the faster operations to
continue to be serviced by the same client
- Since long-running operations now run asynchronously, they have been
limited to one at a time
- Added a limit of 10 websocket clients. This is hard coded for now, but
will be made configurable in the future
Taken together these changes make the code far easier to reason about and
update as well solve the aforementioned issues.
Further optimizations to improve performance are possible in regards to
the way the connection/notification manager works, however this commit
already contains a ton of changes, so they are being left for another
time.
This commit adds a new configuration option, --rpcmaxclients, to limit the
number of max standard RPC clients that are served concurrently. Note
that this value does not apply to websocket connections. A future commit
will add support for limiting those separately.
Closes#68.
Rather than using a type specifically in btcd for the getrawmempool, this
commit, along with a recent commit to btcjson, changes the code over to
use the type from btcjson. This is more consistent with other RPC results
and provides a few extra benefits such as the ability for btcjson to
automatically unmarshal the results into a concrete type with proper field
types as opposed to a generic interface.
Rather than using a type specifically in btcd for the getpeerinfo, this
commit, along with a recent commit to btcjson, changes the code over to
use the type from btcjson. This is more consistent with other RPC results
and provides a few extra benefits such as the ability for btcjson to
automatically unmarshal the results into a concrete type with proper field
types as opposed to a generic interface.
Recent commits to the reference implementation have changed the syncnode
field to be present in the getpeerinfo RPC even when it is false. This
commit changes btcd to match.
The wsContext was being locked twice when NewBlockNotifyCheckTxIn is
called. Fixed by changing handlers to assume lock is acquired and
renamed methods to not be exported.
Since the Z values are normalized (which ordinarily mutates them as
needed) before checking for equality, the race detector gets confused when
using a global value for the field representation of the value 1 and
passing it into the various internal arithmetic routines and reports a
false positive.
Even though the race was a false positive and had no adverse effects, this
commit silences the race detector by creating new variables at the top
level and passing them instead of the global fieldOne variable. The
global is still used for comparison operations since those have no
potential to mutate the value and hence don't trigger the race detector.
This commit changes the server byte counters over to use a mutex instead
of the atomic package. The atomic.AddUint64 function requires the struct
fields to be 64-bit aligned on 32-bit platforms. The byte counts are
fields in the server struct and are not 64-bit aligned. While it would be
possible to arrange the fields to be aligned through various means, it
would make the code too fragile for my tastes. I prefer code that doesn't
depend on platform specific alignment.
Fixes#96.
This commit adds a new option, --logdir, which works in the same fashion
as the --datadir option. Consequently, the logging directory is name
"namespaced" by the network as well. This resolves the issue where two
btcd instances running (one for mainnet and one for testnet) would
overwrite each other's log files by default.
It also provides the user with a method to change the logging location to
non-default locations if they prefer. For example, it enables multiple
btcd instances on the same network to specify unique logging directories
(even though running multiple btcd instances on the same network is not
the most sane configuration).
Closes#95.
Since the websocket handlers run in their own separate goroutines, it's
possible that they execute after the websocket connection has been
closed and cleaned up. This commit add the necessary checks to ensure
stale data isn't added to notification lists and that requests to closed
connections are ignored.
This closes#92.
The getwork command alters the output depending on whether or not the
optional data parameter was specified. It is a JSON object when no data
was provided, and a boolean indicating whether a solution was found
when data was provided.
The getwork RPC now only accepts a single optional parameter which, if
present, is a string containing the hex-encoded block data to solve.
Also update the tests for the new correct parameters.
ok @jcvernaleo
Access to connections map and associated notification maps in rpcServer
need to be protected with s.ws.Lock to prevent race with add/remove new
clients.
This closes#88.
This commit resolves an issue where the block node index was forcing
entire blocks to be kept in memory thereby forcing excessive memory usage.
For example, prior to this change, the memory usage could consume upwards
of 1.5GB while importing bootstrap.dat via the addblock utility. With
this change the entire import takes <150MB. This also has the same memory
reduction to btcd since it uses the same code path.
Changed mempool.MaybeAcceptTransaction to accept an additional parameter
to differentiate betwee new transactions and those added from
disconnected blocks.
Added new fields to requestContexts to indicate which clients want to
receive all new transaction notifications.
Added NotifyForNewTx to rpcServer to deliver approriate transaction
notification.
Sending NotifyAllNewTXsCmd will register websocket client to receive
notifications on all new transactions.
Once registered the client will receive either AllTxNtfn or
AllVerboseTxNtfn based on the required verbose field in the
NotifyAllNewTXsCmd.
Previously the getnettotals was just looping through all of the currently
connected peers to sum the byte counts and returning that. However, the
intention of the getnettotals RPC is to get all bytes since the server was
started, so this logic was not correct.
This commit modifies the code to keep an atomic counter on the server for
bytes read/written and has each peer update the server counters as well as
the per-peer counters.
