This is a backport of decred/dcrwallet#612.
This change moves the wait for the session RPC response (used as a
pong) to a new goroutine that does not run directly in the queue
handler. By moving this out to a new goroutine, the handler can
continue enqueuing and dequeueing notifications while waiting for the
session response. Previously, if a notifiation was sent after the
session RPC was called and before the response was received, the
rpcclient main loop would block due to being unable to enqueue the
notification.
This updates all code to make use of the new chainhash package since the
old wire.ShaHash type and related functions have been removed in favor
of the abstracted package.
Also, while here, rename all variables that included sha in their name
to include hash instead.
Finally, update glide.lock to use the required version of btcd, btcutil,
and btcrpcclient.
This commit corrects various things found by the static checkers
(comments, unkeyed fields, return after some if/else).
Add generated files and legacy files to the whitelist to be ignored.
Catch .travis.yml up with btcd so goclean can be run.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
Logging from btcrpcclient is currently not possible to set, and defaults
to nothing. Letting it inherit chain's logger can greatly simplify
debugging of connectivity issues.
Also remove a now redundant log message upon connecting to btcd.
This change introduces additional network activity with the btcd
process to ensure that the network connection is not silently dropped.
Previously, if the connection was lost (e.g. wallet runs on a laptop
and connects to remote btcd, and the laptop is suspended/resumed) the
lost connection would not be detectable since all normal RPC activity
(excluding requests from btcwallet to btcd made by the user) is in the
direction of btcd to wallet in the form of websocket notifications.
btcrpcclient commit 9ca93b30ad11ec34348d2d788c58019571bf9524 switched
the client to btcjson v2 and broke the wallet build. This change is
the minimum amount of work to make wallet work again.
The rpc server still uses btcjson v1, so there will be 2 versions of
btcjson linked in the resulting binary. This is not optimal but not
compiling is worse.
Otherwise, if the websocket connection to btcd is lost and
reestablished, there is no sync to the current best block, and btcd
will not notify wallet of new relevant transactions.
This commit converts the wallet to use the new secure hierarchical
deterministic wallet address manager package as well as the walletdb
package.
The following is an overview of modified functionality:
- The wallet must now be created before starting the executable
- A new flag --create has been added to create the new wallet using wizard
style question and answer prompts
- Starting the process without an existing wallet will instruct now
display a message to run it with --create
- Providing the --create flag with an existing wallet will simply show an
error and return
In addition the snacl package has been modified to return the memory after
performing scrypt operations to the OS.
Previously a runtime.GC was being invoked which forced it to release the
memory as far as the garbage collector is concerned, but the memory was
not released back to the OS immediatley. This modification allows the
memory to be released immedately since it won't be needed again until the
next wallet unlock.
This commit introduces a new flag, --noclienttls, which can be used to disable
TLS for the RPC client. However, the flag can only be used when the RPC
client is connecting to localhost interfaces. This is intended to prevent
accidentally leaking sensitive data when switching between local and
remote servers.
The notified chain server connection state was being passed through
the wallet and then notified to the RPC server, which prevented this
notification from ever firing if a wallet didn't exist yet. Instead,
make the RPC server register for these notifications directly from the
chain server RPC client.
I'm not happy with this notification and how it's handled in the code,
but to not break existing clients this change is being made. Fixing
the notifiation mess and modifying existing clients to use a new
notification API will need to be done sometime later.
