Commit graph

127 commits

Author SHA1 Message Date
Wilmer Paulino
1285049923
wallet: include addresses from relevant key scopes in rescan
Due to a no longer existing bug within the wallet, it was possible for
change addresses to be created outside of their intended key scope (the
default), so wallets affected by this now need to ensure they scan the
chain for all addresses within the default key scopes (as expected), and
all _internal_ addresses (branch used for change addresses) within any
other registered key scopes to reflect their proper balance.
2020-03-30 15:35:49 -07:00
Wilmer Paulino
43e19da868
Revert "waddrmgr+wallet: only watch addresses within default key scopes"
The commit being reverted resulted in the discovery of a bug in which
change addresses could at times be created outside of the default key
scopes, causing us to not properly determine their spends.
2020-03-30 15:31:44 -07:00
Olaoluwa Osuntokun
ae9416ad76
Merge pull request #676 from wpaulino/watch-default-account-addrs-only
waddrmgr+wallet: only watch addresses within default key scopes
2020-02-18 16:46:49 -08:00
Wilmer Paulino
51b362d7c5
waddrmgr+wallet: only watch addresses within default key scopes
It was discovered that the wallet can scan the chain for unnecessary
additional addresses that are derived by higher-level applications using
custom key scopes. This isn't much of an issue for full nodes, but it
can cause light clients to scan more than what's required, triggering
more false positive matches which lead to block retrieval.

Now, we'll only scan the chain for addresses that exist within the
default key scopes, as those are the only ones the wallet should be
concerned about.
2020-01-24 17:12:23 -08:00
Oliver Gugger
655c23db1d
waddrmgr: export fast scrypt options 2020-01-23 13:47:02 +01:00
Wilmer Paulino
81797fe29d
multi: extend walletdb bolt driver with NoFreelistSync option
This allows external callers to set the option instead. All tests
remained with the option enabled.
2019-10-03 13:28:04 -04:00
Wilmer Paulino
f710cd4a92
cmd/dropwtxmgr+waddrmgr: handle constraint when resetting synced state
We update the dropwtxmgr utility tool to take into account that the
wallet only stores MaxReorgDepth blocks, which introduced an additional
constraint when updating the wallet's synced state. The constraint
ensures that the previous block exists when updating the wallet's synced
state, but this does not hold for the birthday block since it's the
first block we'll store.
2019-07-08 13:26:43 -07:00
Wilmer Paulino
f2f46b674d
waddrmgr: add migration to maintain MaxReorgDepth block hashes stored
In this commit, we add a migration that will be used by existing wallets
to ensure they can adhere to the new requirement of storing up to
MaxReorgDepth entries within the block hash index.
2019-06-11 12:02:42 -07:00
Wilmer Paulino
e548e76684
waddrmgr: maintain a maximum of MaxReorgDepth block hashes stored
In this commit, we modify the wallet's block hash index to only store up
to MaxReorgDepth blocks. This allows us to reduce consumed storage, as
we'd be mostly storing duplicate data. We choose to store up to
MaxReorgDepth to ensure we can recover from a potential long reorg.
2019-06-11 12:02:41 -07:00
Wilmer Paulino
8ae8071c54
waddrmgr: use RLock when retrieving sync info 2019-05-23 14:04:02 -07:00
Wilmer Paulino
3cf41c92b5
waddrmgr: use methods that do not hold the manager's lock 2019-03-18 15:47:48 -07:00
Johan T. Halseth
918d9c2f88
waddrmgr/scoped_manager: add nextAddresses cache update to db tx's OnCommit()
This commit makes nextAddresses add a function to the transactions
OnCommit handler used to update the cache on successful database
transaction commit. Before this we would risk the cache and database of
get out of sync if the database transaction failed or was aborted after
the cache was updated.
2019-02-14 09:19:25 +01:00
Wilmer Paulino
177e31c0b3 waddrmgr+cmd/dropwtxmgr: start rescan from birthday block
In this commit, we modify the dropwtxmgr tool to force a rescan upon
restart from the wallet's birthday block, rather than the chain's
genesis block. We can safely do this as we expect that no on-chain
events relevant to the wallet should happen before this block.  For
older wallets which do not have their birthday block set, the rescan
should start from the genesis block.
2019-01-08 20:12:16 -08:00
Wilmer Paulino
7c377b2906
waddrmgr/sync: expose verification status in Manager's birthday block methods 2018-11-15 18:02:21 -08:00
Wilmer Paulino
6568c433fe
waddrmgr/db: store birthday block verification status
In this commit, we add a new key/value pair to the waddrmgr's sync
bucket to store the verification status of the birthday block. This
verification status determines whether the wallet has verified the
correctness of its birthday block through its sanity check on startup.
2018-11-15 18:01:19 -08:00
Wilmer Paulino
0424fd22ec
waddrmgr/migrations: add migration to force rescan from birthday block
In this commit, we add a migration to force a rescan of users' wallets
starting from their birthday block to ensure that their balance is
reflected correctly as it is on-chain. This was inspired by the recent
bug discovered where the wallet would not watch for the confirmation of
a relevant transaction.
2018-11-14 18:09:10 -08:00
Wilmer Paulino
a25899eae7
waddrmgr/migrations: add migration to populate birthday block for existing wallets
In this commit, we add a new migration to the waddrmgr to populate the
birthday block for existing wallets. This will deem useful when
performing rescans for whatever reason, as we'll now be able to start
from this point rather than the genesis block, incurring a longer
rescan.

The migration is not as reliable since we do not store block timestamps,
so we'll need to estimate our height by looking at the genesis timestamp
and assuming a block occurs every 10 minutes. This can be unsafe, and
cause us to actually miss on-chain events, so a sanity check will be
added before the wallet attempts to sync itself in a later commit.
2018-11-13 19:37:08 -08:00
Wilmer Paulino
709fa17540
waddrmgr+wallet: add waddrmgr log to the wallet 2018-11-12 08:56:31 -08:00
Wilmer Paulino
61a2a8c391
waddrmgr/db: return ErrBlockNotFound if block doesn't exist 2018-11-12 08:56:31 -08:00
Wilmer Paulino
e30cebea1b
waddrmgr/sync: add methods to Manager to get/set birthday block 2018-11-11 17:44:10 -08:00
Wilmer Paulino
e9d24382d6
waddrmgr/db: add birthday block within syncBucket
In this commit, we add a new key/value pair within the waddrmgr's
syncBucket that will represent the birthday block of the wallet. This
can then be used to force rescans from this point, rather than from the
genesis block.
2018-11-11 17:44:08 -08:00
Wilmer Paulino
f2432b1a5e
waddrmgr: remove testing work-around for unexported vars
These work-arounds are no longer needed since the tests now have
package-level access.
2018-11-07 04:02:59 -08:00
Wilmer Paulino
fb91894a20
waddrmgr: remove ErrUpgrade check in TestManager in favor of test 2018-11-07 00:44:31 -08:00
Wilmer Paulino
c2d6f1a8b3
waddrmgr: convert unit tests to package-level tests
In this commit, we convert our unit tests to have package-level access.
We do this as an effort to reduce test code duplication when we
introduce migration tests which require access to specific unexported
functions/methods.
2018-11-07 00:44:27 -08:00
Wilmer Paulino
f582eab1fa
waddrmgr: remove old migration logic
In this commit, we remove the old upgrade/migration logic of the address
manager as it's been superseded by the new approach using the
migration.Manager interface.
2018-11-05 17:58:16 -08:00
Wilmer Paulino
1e582298b2
waddrmgr/db: remove LatestVersion const in favor of getLatestVersion
In this commit, we can remove the LatestVersion constant as it's no
longer needed. Instead, we'll now define the latest version as the last
entry in the slice of versions previously defined.
2018-11-05 17:58:16 -08:00
Wilmer Paulino
b05148bb8b
waddrmgr/migrations: move migration-related functions
This commit is strictly a code move to keep all migration-related things
within the same file.
2018-11-05 17:58:16 -08:00
Wilmer Paulino
3b4f73272f
waddrmgr/migrations: add migration.Manager implementation
In this commit, we add an implementation of the recently introduced
migration.Manager interface for the address manager. With this, we'll
now be able to only expose the things required for the migration to
happen, but have the actual migration logic live at a much higher level.

The existing versions defined are set up in the same way as the existing
upgrade/migration logic, which will end up being superseded by this and
removed in a later commit.
2018-11-05 17:58:16 -08:00
Wilmer Paulino
c01bbc4758
waddrmgr/db: remove unused pubPassPhrase arg from upgradeToVersion5 2018-11-05 17:58:16 -08:00
Conner Fromknecht
85c75de4a5
waddrmgr/manager_test: use SetSecretKeyGen to safely swap keygen 2018-08-31 16:29:55 -07:00
Conner Fromknecht
644fd2bda0
waddrmgr/internal_test: remove TstRunWithReplacedSecretKey 2018-08-31 16:29:55 -07:00
Conner Fromknecht
ba58d5357f
waddrmgr/manager: guard access to newSecretKey
This commit places a mutex around calls to newSecretKey,
since the inner function needs to be swapped out
during testing. Prior to this change, the race
detector would panic since the mutation was
unprotected.
2018-08-31 16:29:54 -07:00
Olaoluwa Osuntokun
1feb87e6ae
waddrmgr: update tests to ensure correcntess of new DerivationInfo method 2018-08-14 18:17:46 -07:00
Olaoluwa Osuntokun
bd81968215
waddrmgr: add new DerivationInfo method to managedAddress, update ScopedKeyManager
In this commit, we add the new DerivationInfo method to the current
default implementation of the ManagedPubKeyAddress interface. In doing
this, we replace the account field with the derivationPath, as we can
obtain the account field from the derivationPath itself.
2018-08-14 18:17:46 -07:00
Olaoluwa Osuntokun
05d73f6899 waddrmgr: add new DerivationInfo method to ManagedPubKeyAddress
In this commit, we add a new method DerivationInfo to the
ManagedPubKeyAddress interface. This method is meant to provide callers
with the information necessary to independently derive each key returned
by the various methods provided to derive addresses.
2018-08-14 18:15:02 -07:00
Johan T. Halseth
f243546c33
waddrmgr test: move t.Parallel to actual test 2018-07-20 11:13:27 +02:00
Olaoluwa Osuntokun
2b856b5200 multi: fix linter errors 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
ccb49eaf2a waddrmgr: fix botched merge 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
7770cac383 multi: switch import paths back to upstream 2018-05-23 19:38:56 -07:00
Conner Fromknecht
d652e7dd04 waddrmgr/manager_test: use new wallet Create api with birthday 2018-05-23 19:38:56 -07:00
Conner Fromknecht
0968cfdabc waddrmgr/common_test: use new wallet Create api with birthday 2018-05-23 19:38:56 -07:00
Conner Fromknecht
57b37d38f6 waddrmgr/scoped_manager: adds extend in/ex ternal addrs 2018-05-23 19:38:56 -07:00
Conner Fromknecht
5e07326784 waddrmgr/manager: feed birthday to waddrmgr 2018-05-23 19:38:56 -07:00
Conner Fromknecht
65a276bd7b waddrmgr/db: internalBranch -> InternalBranch 2018-05-23 19:38:56 -07:00
Conner Fromknecht
1d50b92bdc waddrmgr/multi: fix scoped mgr reentry deadlock
This commit resolves a deadlock observed when attempting
to generate addresses. There were a few cases, particularly
in chainAddressRowToManaged and loadAccountInfo, which accessed
the public IsLocked() method of the Manager, even though the
shared mutex had already been acquired.

The solution is to create an internal isLocked() method, which
can be safely called assuming the manager's mutex has already been
acquired. As the comments above both of the methods in question
specify, we can assume the Manager's mutex *is* already acquired.

This commit also reduces some unnecessary code duplication, since
the recent changes left both a Locked() and IsLocked() method that
perform the same functionality. IsLocked() was favored as it more
clearly indicates that the returned value is a boolean.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
d0d94666cf waddrmgr: update test to ensure timestamp is set 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
27dfed7f27 waddrmgr: fix deadlock bug by using read mutex when necessary
In this commit, we fix a deadlock bug that was introduced recently.
This can happen when ForEachActiveAccountAddress or
ForEachActiveAddress is called, as these internally need to grab the
mutex of the manager (within the scoped manager) in order to check if
the manager is locked or not.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
d42a45f16f waddrmgr: add database migration from version 4 to 5
In this commit, we add a database migration from version 4 to version 5.
We also take this opportunity to clean up the old migration code. This
is no longer needed as wallets very old can simply go back in the prior
git history to migrate to version 4, then go from there to version 5.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
3ca68d8259 waddrmgr: add additional tests to exercise new scope specific features 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
b8104ef86e waddrmgr: update existing tests to be aware of new key scopes 2018-05-23 19:38:56 -07:00