This PR allows the creation of managers and accounts that are watch-only. The state of the database after creation would be identical to the state after calling
Manager.ConvertToWatchingOnly, assuming accounts with the right xpubs were created in the former case.
Co-authored-by: Ken Sedgwick <ken@bonsai.com>
We update the dropwtxmgr utility tool to take into account that the
wallet only stores MaxReorgDepth blocks, which introduced an additional
constraint when updating the wallet's synced state. The constraint
ensures that the previous block exists when updating the wallet's synced
state, but this does not hold for the birthday block since it's the
first block we'll store.
In this commit, we modify the wallet's block hash index to only store up
to MaxReorgDepth blocks. This allows us to reduce consumed storage, as
we'd be mostly storing duplicate data. We choose to store up to
MaxReorgDepth to ensure we can recover from a potential long reorg.
In this commit, we modify the dropwtxmgr tool to force a rescan upon
restart from the wallet's birthday block, rather than the chain's
genesis block. We can safely do this as we expect that no on-chain
events relevant to the wallet should happen before this block. For
older wallets which do not have their birthday block set, the rescan
should start from the genesis block.
In this commit, we add a new key/value pair to the waddrmgr's sync
bucket to store the verification status of the birthday block. This
verification status determines whether the wallet has verified the
correctness of its birthday block through its sanity check on startup.
In this commit, we add a new key/value pair within the waddrmgr's
syncBucket that will represent the birthday block of the wallet. This
can then be used to force rescans from this point, rather than from the
genesis block.
In this commit, we remove the old upgrade/migration logic of the address
manager as it's been superseded by the new approach using the
migration.Manager interface.
In this commit, we can remove the LatestVersion constant as it's no
longer needed. Instead, we'll now define the latest version as the last
entry in the slice of versions previously defined.
In this commit, we add a database migration from version 4 to version 5.
We also take this opportunity to clean up the old migration code. This
is no longer needed as wallets very old can simply go back in the prior
git history to migrate to version 4, then go from there to version 5.
In this commit, we create new key spaces to allow users to store the
encrypted master priv/pub keys. This is required as in order to create
new scopes, we must do hardened derivation from the root key.
In this commit, we make a fundamental modification bucket structure
within the database. Most buckets are no under an additional layer of
nesting: the scope. The scope encapsulates which (purpose, coin type)
pair the address, accounts, and coin type keys belong to.
In this commit, we remove all direct references to BIP 44 as upcoming
changes will shift to a model that is no longer directly dependent on
BIP 44 in favor of restoring a layer of abstraction and allowing users
to manage multiple (purpose, coin type) scopes within the same
database.
This changes the database access APIs and each of the "manager"
packages (waddrmgr/wstakemgr) so that transactions are opened (only)
by the wallet package and the namespace buckets that each manager
expects to operate on are passed in as parameters.
This helps improve the atomicity situation as it means that many
calls to these APIs can be grouped together into a single
database transaction.
This change does not attempt to completely fix the "half-processed"
block problem. Mined transactions are still added to the wallet
database under their own database transaction as this is how they are
notified by the consensus JSON-RPC server (as loose transactions,
without the rest of the block that contains them). It will make
updating to a fixed notification model significantly easier, as the
same "manager" APIs can still be used, but grouped into a single
atomic transaction.
This commit introduces two new address types to the waddrmgr. The first
address type is the native p2wkh (pay-to-witness-key-hash) output type
introduced as part of BIP0141 and the segwit soft-fork. The second
address type is a p2wkh output nested *within* a regular p2sh output.
This second address allows older wallets which are not yet aware of the
new segwit output types to transparently pay to a wallet which does
support them. Additionally, using this nested p2wkh output the wallet
gains both the space+transaction fee savings, as well as the
malleability fixes.
Both address types have been implemented as special cases of the
ManagedPubKeyAddress since they share several traits, only
differentiating in the signing mechanism needed, and the concrete
implementation of btcutil.Address returned by the address.
Two new `addressType` constants have been added to waddrmgr’s db in
order to properly serialize and deserialize the new address types.
This updates all code to make use of the new chainhash package since the
old wire.ShaHash type and related functions have been removed in favor
of the abstracted package.
Also, while here, rename all variables that included sha in their name
to include hash instead.
Finally, update glide.lock to use the required version of btcd, btcutil,
and btcrpcclient.
This commit corrects various things found by the static checkers
(comments, unkeyed fields, return after some if/else).
Add generated files and legacy files to the whitelist to be ignored.
Catch .travis.yml up with btcd so goclean can be run.
Rather than the main package being responsible for opening the address
and transaction managers, the namespaces of these components are
passed as parameters to the wallet.Open function.
Additionally, the address manager Options struct has been split into
two: ScryptOptions which holds the scrypt parameters needed during
passphrase key derivation, and OpenCallbacks which is only passed to
the Open function to allow the caller to provide additional details
during upgrades.
These changes are being done in preparation for a notification server
in the wallet package, with callbacks passed to the Open and Create
functions in waddrmgr and wtxmgr. Before this could happen, the
wallet package had to be responsible for actually opening the managers
from their namespaces.
If the account number to name index mapped the default account name to
an alias, the upgrade would not succeed and the upgrade would be
aborted (and rolled back).
This became a problem for upgrading old (pre-v3) wallets since the v3
upgrade did not rename the previous "" account to "default", but
instead just created an alias.
Fix tested by @dajohi, who ran into this issue with a wallet upgrade
from an older keystore version.
Rather than disallowing the default account to be renamed as was
proposed in #245 (and implemented in #246), the default account name
is no longer considered a reserved name by the address manager.
Instead, it is simply the initial name used for the first initial
account.
A database upgrade removes any additional aliases for the default
account in the database. This prevents a lookup for some name which
is not an account name from mapping to the default account
unexpectedly (potentially preventing incorrect account usage from the
RPC server due to bad iteraction with default parameters).
All unset account names in a JSON-RPC request are expected to be set
nil by btcjson. This behavior depends on btcsuite/btcd#399.
Additionally, the manager no longer considers the wildcard * to be a
reserved account name. Due to poor API decisions, the RPC server
overloads the meaning of account fields to optionally allow referring
to all accounts at a time, or a single account. This is not a address
manager responsibility, though, as a future cleaner API should not use
multiple differet meanings for the same field across multiple
requests. Therefore, don't burden down future APIs with this quirk
and prevent incorrect wildcard usage from the RPC server.
Closes#245.
This commit makes the creation and updating of the address manager more
explicit so it's easier to upgrade in the future.
In particular, rather than treating the initial creation as an upgrade by
relying on creating the initial buckets on the fly on each load, the code
now explicitly provides distinct create and upgrade paths that are invoked
from the Create and Open functions, respectively.
It also adds some commented out sample code to illustrate how upgrades
should be done and a check to ensure bumping the version number without
writing upgrade code results in a new error, ErrUpgrade, being returned.
Finally, a test has been added for the new functionality.
This contains the APIs to create and retrieve Voting Pools and Series (with
public/private keys) from a walletdb namespace, plus the generation of deposit
addresses (using m-of-n multi-sig P2SH scripts according to the series
configuration).
This commit converts the waddrmgr package to use the new walletdb package
semantics.
Since waddrmgr no longer controls the database, it is unable to make a
copy of the database and return it as the old ExportWatchingOnly function
required. As a result, it has been renamed to ConvertToWatchingOnly and
it now modifies the namespace provided to it. The idea is that the caller
which does control the database can now make a copy of the database, get
the waddrmgr namespace in the database copy and invoke the new function
to modify it. This also works well with other packages that might also
need to make modifications for watching-only mode.
In addition, the following changes are made:
- All places that worked with database paths now work with the
walletdb.Namespace interface
- The managerTx code is replaced to use the walletdb.Tx interface
- The code which checks if the manager already exists is updated to work
with the walletdb.Namespace interface
- The LatestDbVersion constant is now LatestMgrVersion since it no longer
controls the database
This commit implements a new secure, scalable, hierarchical deterministic
wallet address manager package.
The following is an overview of features:
- BIP0032 hierarchical deterministic keys
- BIP0043/BIP0044 multi-account hierarchy
- Strong focus on security:
- Fully encrypted database including public information such as
addresses as well as private information such as private keys and
scripts needed to redeem pay-to-script-hash transactions
- Hardened against memory scraping through the use of actively clearing
private material from memory when locked
- Different crypto keys used for public, private, and script data
- Ability for different passphrases for public and private data
- Scrypt-based key derivation
- NaCl-based secretbox cryptography (XSalsa20 and Poly1305)
- Multi-tier scalable key design to allow instant password changes
regardless of the number of addresses stored
- Import WIF keys
- Import pay-to-script-hash scripts for things such as multi-signature
transactions
- Ability to export a watching-only version which does not contain any
private key material
- Programmatically detectable errors, including encapsulation of errors
from packages it relies on
- Address synchronization capabilities
This commit only provides the implementation package. It does not
include integration into to the existing wallet code base or conversion of
existing addresses. That functionality will be provided by future
commits.
