This change modifies the RPC server's notifiation manager from a
struct with requests, protected by a mutux, to two goroutines. The
first maintains a queue of all notifications and control requests
(registering/unregistering notifications), while the second reads from
the queue and processes notifications and requests one at a time.
Previously, to prevent slowing down block and mempool processing, each
notification would be handled by spawning a new goroutine. This lead
to cases where notifications would end up being sent to clients in a
different order than they were created. Adding a queue keeps the
order of notifications originating from the same goroutine, while also
not slowing down processing while waiting for notifications to be
processed and sent.
ok @davecgh
This changes the implementation of the sendrawtransaction RPC handler
to match bitcoind behavior by always returning a rejection error for
any error processing or accepting the tx by the mempool. Previously,
if the tx was rejected for a rule error rather than an actual failure,
a client would still receive the tx sha as a result with no error.
This commit refactors the entire websocket client code to resolve several
issues with the previous implementation. Note that this commit does not
change the public API for websockets. It only consists of internal
improvements.
The following is the major issues which have been addressed:
- A slow websocket client could impede notifications to all clients
- Long-running operations such as rescans would block all other requests
until it had completed
- The above two points taken together could lead to apparant hangs since
the client doing the rescan would eventually run out of channel buffer
and block the entire group of clients until the rescan completed
- Disconnecting a websocket during certain operations could lead to a hang
- Stopping the rpc server with operations under way could lead to a hang
- There were no limits to the number of websocket clients that could
connect
The following is a summary of the major changes:
- The websocket code has been split into two entities: a
connection/notification manager and a websocket client
- The new connection/notification manager acts as the entry point from
the rest of the subsystems to feed data which potentially needs to
notify clients
- Each websocket client now has its own instance of the new websocket
client type which controls its own lifecycle
- The data flow has been completely redesigned to closely resemble the
peer data flow
- Each websocket now has its own long-lived goroutines for input, output,
and queuing of notifications
- Notifications use the new notification queue goroutine along with
queueing to ensure they dont't block on stalled or slow peers
- There is a new infrastructure for asynchronously executing long-running
commands such as a rescan while still allowing the faster operations to
continue to be serviced by the same client
- Since long-running operations now run asynchronously, they have been
limited to one at a time
- Added a limit of 10 websocket clients. This is hard coded for now, but
will be made configurable in the future
Taken together these changes make the code far easier to reason about and
update as well solve the aforementioned issues.
Further optimizations to improve performance are possible in regards to
the way the connection/notification manager works, however this commit
already contains a ton of changes, so they are being left for another
time.
This commit adds a new configuration option, --rpcmaxclients, to limit the
number of max standard RPC clients that are served concurrently. Note
that this value does not apply to websocket connections. A future commit
will add support for limiting those separately.
Closes#68.
Rather than using a type specifically in btcd for the getrawmempool, this
commit, along with a recent commit to btcjson, changes the code over to
use the type from btcjson. This is more consistent with other RPC results
and provides a few extra benefits such as the ability for btcjson to
automatically unmarshal the results into a concrete type with proper field
types as opposed to a generic interface.
Previously the getnettotals was just looping through all of the currently
connected peers to sum the byte counts and returning that. However, the
intention of the getnettotals RPC is to get all bytes since the server was
started, so this logic was not correct.
This commit modifies the code to keep an atomic counter on the server for
bytes read/written and has each peer update the server counters as well as
the per-peer counters.
This commit moves the connection endpoint for websockets to /ws instead of
/wallet. First, the former is more standard, and second the latter
presumes how the websocket is to be used.
Closes#80.
The getinfo RPC method requires access to information only available in
the wallet. Therefore, it has been moved to the list of methods which
return an error information the caller to send the request to the wallet
instead.
The getnewaddress RPC method deals with wallet-related functionality and
therefore has been moved to the list of methods which return an error
information the caller to send the request to the wallet instead.
Both of these RPC methods require access to information ony available in
the wallet. Therefore they have been moved to the list of methods which
return an error information the caller to send the request to the wallet
instead.
This commit adds full support for the getaddednodeinfo RPC command
including DNS lookups which abide by proxy/onion/tor rules when the DNS
flag is specified. Note that it returns an array of strings when the DNS
flag is not set which is different than the current version of bitcoind
which is bugged and scheduled to be fixed per issue 3581 on the bitcoind
issue tracker.
This commit improves how the legacy RPC server responds to authentication
failures so things like web browsers can react better. The following
changes have been made:
First, authentication failures were only printing the 401 error response
in the body instead of setting the http status code. This means the
response had a 200 OK header with a body of 401 Unauthorized. Therefore
the client would think everything was ok, but see the response as
malformed JSON.
Second, the spec for 401 Unauthorized responses state they must include a
WWW-Authenticate header to instruct the client how to authenticate.
Without this, browsers won't prompt the user for credentials.
The previous websocket code required HTTP auth headers to be sent in order
to use the websocket. While this makes sense for most applications, some
use cases such as javascript-based websockets from browsers do no have the
ability to send headers.
This commit modifies the authentication logic to allow an alternative
authentication mechanism when HTTP auth headers can't be sent. In
particular, it introduces a new JSON-RPC command named authenticate which
accepts the username and passphrase for the RPC server. The command is
only required if the connetion has not already been authenticated via HTTP
auth headers and it must be the first command to be received. Sending any
other command will immediately disconnect the websocket.
ok from @owainga and @jrick.
This closes#77.
The websocket.Server used the by websocket.Handler type automatically adds
a handshake function which prevents connections when the Origin header is
not set. Not all clients send this information and we already require
authentication headers as the auth mechanism anyways.
This removes the last notification that was being sent unsolicited.
Since it is no longer needed, the code to duplicate notifications to
all clients has been removed.
The RPC server was performing some of the shutdown logic in the wrong
order, that is, logging the the server has shut down, waiting for all
server goroutines to finish, and then closing a channel to notify
server goroutines to stop. These three items have been reversed to
fix a hang where goroutines currently being waited on had not shut
down because they did not receive the notification.
While here, the server waitgroup was incremented for a goroutine that
was running without it, another select statement was added to stop a
duplicate close (which never occured last commit when I added the
select statements), and the "stopping rescan" logging was moved to
debug to make the ^C shutdown logging nicer.
This changes the protocol between btcd and btcwallet to follow
JSON-RPC specifications sending notifications as requests with an
empty ID.
The notification request context handling has been greatly cleaned up
now that IDs no longer need to be saved when sending notifications.
This commit changes all code which deals with extracting addresses from
scripts to use the btcscript API ExtractPkScriptAddrs which in turn makes
use of the new btcutil.Address interface.
This provides much cleaner code for dealing with arbitrary script
destinations which is extensible without having to churn the APIs if new
destination types are added.
The name handlers for a package level is a bit too generic and could
easily cause a name collision. Even though the compiler would catch it,
use something a bit more descriptive.
Since the command to handler mappings are the most often modified and
referenced code in rpcserver.go and rpcwebsocket.go, move them near the
top of their respective files.
This commit cleans up the standard RPC command hanlding a bit by removing
the websocket specific notification channel from the handlers. This was
previously required because the sendrawtransaction, when called from a
websocket enabled connection, needs to add a notification for when the
transaction is mined.
This commit modifies that to instead implement a websocket extended
version of sendrawtransaction which invokes the standard handler and adds
the notification. In addition, the main send was modified to first look
if the command has a websocket specific handler first, and then falls back
to standard commands, rather than the previous approach of first checking
for a standard command and falling through to websocket commands. This
essentially allows websockets connections to extend commands with the same
name with additional functionality such as what was done in this commit.
The rpcserver.go file is starting to get a bit unwieldy. This commit
moves the separable websocket specific bits into a separate file named
rpcwebsocket.go.
