Simnet was previously left out as it didn't have defined HD versions for
some of our key scopes. To allow testing importing accounts into simnet
wallets, we fall back to use the mainnet HD versions.
This commit also addresses an issue with simnet wallets that would arise
whenever ScopedKeyManager.AccountProperties was invoked:
`failed to retrieve account public key: unsupported net SimNet`
Following the previous commit, some external hardware signers require a
master key fingerprint to be present within the PSBT input derivation
paths so that the signer can recognize which inputs are relevant and
must be signed.
For key scopes which have an address schema where the external and
internal branches differ, we always assume that imported keys use the
external address type defined in the scope's address schema. This may
not always be the case however, and should be handled correctly.
Ideally, we generate two addresses per imported key (only if the
external and internal address types differ) and scan for both in the
chain.
This change was motivated by the need to support importing BIP-0049 keys
that use the standard address derivation scheme, where nested witness
pubkeys are used for both the external and internal branches. Our
BIP-0049 key scope is slightly different, in that addresses derived from
the internal branch use the witness pubkey address type. By having the
option of overriding the address schema for a particular account, we can
support importing standard BIP-0049 keys.
The master fingerprint corresponds to the fingerprint of the root master
public key (otherwise known as m/). This is required by some hardware
wallets for proper identification and signing.
The address schema is an optional field that allows an account to
override its corresponding address schema with a custom one.
Watch-only accounts are usually backed by an external signer as they do
not contain any private key information. Some external signers require a
root key fingerprint for identification and signing purposes. In order
to guarantee compatibility with external signers, we need to persist the
root key fingerprint within the database.
Before this change, watch-only accounts used the default account
database structure. In this commit, we introduce a new account type to
store different information for watch-only accounts only. This isn't a
breaking change as watch-only accounts have yet to be supported by the
primary user of the wallet (lnd). With this new account type, we can
avoid the empty private key fields, which are irrelevant to watch-only
accounts, and we can store the root key fingerprint.
Previously, addresses that belong to a watch-only account would have a
derivation path using the internal account number used to identify
accounts within the databse, rather than the actual account number based
on the account's master public key child index. This wasn't an issue
before as only one account would exist within the wallet, the 0 account,
which is also the default. To ensure users of the DerivationPath struct
can arrive at addresses correctly, we introduce a new field
InternalAccount to denote the internal account number and repurpose the
existing Account field to its actual meaning.
Not setting this would result in a non-sensible unix timestamp
(2288912640) being exposed when the wallet hasn't synced any blocks,
like in the case when it's waiting for the backend to sync.
This PR allows the creation of managers and accounts that are watch-only. The state of the database after creation would be identical to the state after calling
Manager.ConvertToWatchingOnly, assuming accounts with the right xpubs were created in the former case.
Co-authored-by: Ken Sedgwick <ken@bonsai.com>
Due to a no longer existing bug within the wallet, it was possible for
change addresses to be created outside of their intended key scope (the
default), so wallets affected by this now need to ensure they scan the
chain for all addresses within the default key scopes (as expected), and
all _internal_ addresses (branch used for change addresses) within any
other registered key scopes to reflect their proper balance.
The commit being reverted resulted in the discovery of a bug in which
change addresses could at times be created outside of the default key
scopes, causing us to not properly determine their spends.
It was discovered that the wallet can scan the chain for unnecessary
additional addresses that are derived by higher-level applications using
custom key scopes. This isn't much of an issue for full nodes, but it
can cause light clients to scan more than what's required, triggering
more false positive matches which lead to block retrieval.
Now, we'll only scan the chain for addresses that exist within the
default key scopes, as those are the only ones the wallet should be
concerned about.
We update the dropwtxmgr utility tool to take into account that the
wallet only stores MaxReorgDepth blocks, which introduced an additional
constraint when updating the wallet's synced state. The constraint
ensures that the previous block exists when updating the wallet's synced
state, but this does not hold for the birthday block since it's the
first block we'll store.
In this commit, we add a migration that will be used by existing wallets
to ensure they can adhere to the new requirement of storing up to
MaxReorgDepth entries within the block hash index.
In this commit, we modify the wallet's block hash index to only store up
to MaxReorgDepth blocks. This allows us to reduce consumed storage, as
we'd be mostly storing duplicate data. We choose to store up to
MaxReorgDepth to ensure we can recover from a potential long reorg.
This commit makes nextAddresses add a function to the transactions
OnCommit handler used to update the cache on successful database
transaction commit. Before this we would risk the cache and database of
get out of sync if the database transaction failed or was aborted after
the cache was updated.
In this commit, we modify the dropwtxmgr tool to force a rescan upon
restart from the wallet's birthday block, rather than the chain's
genesis block. We can safely do this as we expect that no on-chain
events relevant to the wallet should happen before this block. For
older wallets which do not have their birthday block set, the rescan
should start from the genesis block.
In this commit, we add a new key/value pair to the waddrmgr's sync
bucket to store the verification status of the birthday block. This
verification status determines whether the wallet has verified the
correctness of its birthday block through its sanity check on startup.
In this commit, we add a migration to force a rescan of users' wallets
starting from their birthday block to ensure that their balance is
reflected correctly as it is on-chain. This was inspired by the recent
bug discovered where the wallet would not watch for the confirmation of
a relevant transaction.
In this commit, we add a new migration to the waddrmgr to populate the
birthday block for existing wallets. This will deem useful when
performing rescans for whatever reason, as we'll now be able to start
from this point rather than the genesis block, incurring a longer
rescan.
The migration is not as reliable since we do not store block timestamps,
so we'll need to estimate our height by looking at the genesis timestamp
and assuming a block occurs every 10 minutes. This can be unsafe, and
cause us to actually miss on-chain events, so a sanity check will be
added before the wallet attempts to sync itself in a later commit.
In this commit, we add a new key/value pair within the waddrmgr's
syncBucket that will represent the birthday block of the wallet. This
can then be used to force rescans from this point, rather than from the
genesis block.
In this commit, we convert our unit tests to have package-level access.
We do this as an effort to reduce test code duplication when we
introduce migration tests which require access to specific unexported
functions/methods.
In this commit, we remove the old upgrade/migration logic of the address
manager as it's been superseded by the new approach using the
migration.Manager interface.
In this commit, we can remove the LatestVersion constant as it's no
longer needed. Instead, we'll now define the latest version as the last
entry in the slice of versions previously defined.
In this commit, we add an implementation of the recently introduced
migration.Manager interface for the address manager. With this, we'll
now be able to only expose the things required for the migration to
happen, but have the actual migration logic live at a much higher level.
The existing versions defined are set up in the same way as the existing
upgrade/migration logic, which will end up being superseded by this and
removed in a later commit.
