This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
This commit makes the creation and updating of the address manager more
explicit so it's easier to upgrade in the future.
In particular, rather than treating the initial creation as an upgrade by
relying on creating the initial buckets on the fly on each load, the code
now explicitly provides distinct create and upgrade paths that are invoked
from the Create and Open functions, respectively.
It also adds some commented out sample code to illustrate how upgrades
should be done and a check to ensure bumping the version number without
writing upgrade code results in a new error, ErrUpgrade, being returned.
Finally, a test has been added for the new functionality.
This contains the APIs to create and retrieve Voting Pools and Series (with
public/private keys) from a walletdb namespace, plus the generation of deposit
addresses (using m-of-n multi-sig P2SH scripts according to the series
configuration).
This commit converts the waddrmgr package to use the new walletdb package
semantics.
Since waddrmgr no longer controls the database, it is unable to make a
copy of the database and return it as the old ExportWatchingOnly function
required. As a result, it has been renamed to ConvertToWatchingOnly and
it now modifies the namespace provided to it. The idea is that the caller
which does control the database can now make a copy of the database, get
the waddrmgr namespace in the database copy and invoke the new function
to modify it. This also works well with other packages that might also
need to make modifications for watching-only mode.
In addition, the following changes are made:
- All places that worked with database paths now work with the
walletdb.Namespace interface
- The managerTx code is replaced to use the walletdb.Tx interface
- The code which checks if the manager already exists is updated to work
with the walletdb.Namespace interface
- The LatestDbVersion constant is now LatestMgrVersion since it no longer
controls the database
This commit implements a new secure, scalable, hierarchical deterministic
wallet address manager package.
The following is an overview of features:
- BIP0032 hierarchical deterministic keys
- BIP0043/BIP0044 multi-account hierarchy
- Strong focus on security:
- Fully encrypted database including public information such as
addresses as well as private information such as private keys and
scripts needed to redeem pay-to-script-hash transactions
- Hardened against memory scraping through the use of actively clearing
private material from memory when locked
- Different crypto keys used for public, private, and script data
- Ability for different passphrases for public and private data
- Scrypt-based key derivation
- NaCl-based secretbox cryptography (XSalsa20 and Poly1305)
- Multi-tier scalable key design to allow instant password changes
regardless of the number of addresses stored
- Import WIF keys
- Import pay-to-script-hash scripts for things such as multi-signature
transactions
- Ability to export a watching-only version which does not contain any
private key material
- Programmatically detectable errors, including encapsulation of errors
from packages it relies on
- Address synchronization capabilities
This commit only provides the implementation package. It does not
include integration into to the existing wallet code base or conversion of
existing addresses. That functionality will be provided by future
commits.
