In this commit, we add a database migration from version 4 to version 5.
We also take this opportunity to clean up the old migration code. This
is no longer needed as wallets very old can simply go back in the prior
git history to migrate to version 4, then go from there to version 5.
This changes the database access APIs and each of the "manager"
packages (waddrmgr/wstakemgr) so that transactions are opened (only)
by the wallet package and the namespace buckets that each manager
expects to operate on are passed in as parameters.
This helps improve the atomicity situation as it means that many
calls to these APIs can be grouped together into a single
database transaction.
This change does not attempt to completely fix the "half-processed"
block problem. Mined transactions are still added to the wallet
database under their own database transaction as this is how they are
notified by the consensus JSON-RPC server (as loose transactions,
without the rest of the block that contains them). It will make
updating to a fixed notification model significantly easier, as the
same "manager" APIs can still be used, but grouped into a single
atomic transaction.
This prevents treating a flag that was explicitly set to the default
as unchanged, since the explicit set is recorded in the new
*cfgutil.ExplicitString flag type.
This changes the wallet.Open function signature to remove the database
namespace parameters. This is done so that the wallet package itself
is responsible for the location and opening of these namespaces from
the database, rather than requiring the caller to open these ahead of
time.
A new wallet.Create function has also been added. This function
initializes a new wallet in an empty database, using the same
namespaces as wallet.Open will eventually use. This relieves the
caller from needing to manage wallet database namespaces explicitly.
Fixes#397.
This corrects and simplifies the shutdown logic for interrupts, the
walletrpc.WalletLoaderService/CloseWallet RPC, and the legacy stop RPC
by both stopping all wallet processes and closing the wallet database.
It appears that this behavior broke as part of the wallet package
refactor, causing occasional nil pointer panics and memory faults when
closing the wallet database with active transactions.
Fixes#282.
Fixes#283.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
This change moves the chain and network parameter definitions, along
with the default client and server ports, to a package for reuse by
other utilities (most notably, tools in the cmd dir). Along with it,
functions commonly used for config parsing and validation are moved to
an internal package since they will also be useful for distributed
tools.
Rather than the main package being responsible for opening the address
and transaction managers, the namespaces of these components are
passed as parameters to the wallet.Open function.
Additionally, the address manager Options struct has been split into
two: ScryptOptions which holds the scrypt parameters needed during
passphrase key derivation, and OpenCallbacks which is only passed to
the Open function to allow the caller to provide additional details
during upgrades.
These changes are being done in preparation for a notification server
in the wallet package, with callbacks passed to the Open and Create
functions in waddrmgr and wtxmgr. Before this could happen, the
wallet package had to be responsible for actually opening the managers
from their namespaces.
This a refactor of the btcwallet main package to create a new wallet
package.
The main feature of this package is the integration of all the other
wallet components (waddrmgr, txstore, and chain) and the Wallet type is
'runnable', so it will be continuously updating itself against changes
notified by the remote btcd instance.
It also includes several methods which provide access to information
necessary to run a wallet RPC server.
The createtemp flag is used to create a temporary simnet wallet for
use with btcsim or other testing suites.
The flag only works if the datadir and network to use it on (simnet)
are specified.
After starting btcwallet with the flag, the wallet immediately opens
and is functional.
This commit converts the wallet to use the new secure hierarchical
deterministic wallet address manager package as well as the walletdb
package.
The following is an overview of modified functionality:
- The wallet must now be created before starting the executable
- A new flag --create has been added to create the new wallet using wizard
style question and answer prompts
- Starting the process without an existing wallet will instruct now
display a message to run it with --create
- Providing the --create flag with an existing wallet will simply show an
error and return
In addition the snacl package has been modified to return the memory after
performing scrypt operations to the OS.
Previously a runtime.GC was being invoked which forced it to release the
memory as far as the garbage collector is concerned, but the memory was
not released back to the OS immediatley. This modification allows the
memory to be released immedately since it won't be needed again until the
next wallet unlock.
