Commit graph

292 commits

Author SHA1 Message Date
carla
9e2f2ce157
wallet: add label parameter to SendOutputs 2020-05-11 15:17:55 +02:00
carla
d2f9185f6a
wallet: add label to PublishTransaction
All label parameter to PublishTransaction. Pass in an empty string
in rpc call as a placeholder for follow up PR which will add a label
parameter to the PublishTransaction request.
2020-05-11 15:16:22 +02:00
Dev Random
4c5bc1b15d
waddrmgr: create watch-only address managers and accounts
This PR allows the creation of managers and accounts that are watch-only. The state of the database after creation would be identical to the state after calling 

Manager.ConvertToWatchingOnly, assuming accounts with the right xpubs were created in the former case.

Co-authored-by: Ken Sedgwick <ken@bonsai.com>
2020-04-24 17:44:21 -07:00
Wilmer Paulino
31c027e19f
wallet: perform recovery on all registered key scopes
In similar fashion to the previous commit, due to a no longer existing
bug within the wallet, it was possible for change addresses to be
created outside of their intended key scope (the default), so wallets
affected by this now need to ensure upon recovery that they scan the
chain for _all_ existing key scopes, rather than just the default ones,
to reflect their proper balance. Through manual testing, it was shown
that the impact of recovering the additional key scopes is negligible in
most cases for both full nodes and light clients.
2020-03-30 15:35:50 -07:00
Wilmer Paulino
1285049923
wallet: include addresses from relevant key scopes in rescan
Due to a no longer existing bug within the wallet, it was possible for
change addresses to be created outside of their intended key scope (the
default), so wallets affected by this now need to ensure they scan the
chain for all addresses within the default key scopes (as expected), and
all _internal_ addresses (branch used for change addresses) within any
other registered key scopes to reflect their proper balance.
2020-03-30 15:35:49 -07:00
Wilmer Paulino
43e19da868
Revert "waddrmgr+wallet: only watch addresses within default key scopes"
The commit being reverted resulted in the discovery of a bug in which
change addresses could at times be created outside of the default key
scopes, causing us to not properly determine their spends.
2020-03-30 15:31:44 -07:00
Wilmer Paulino
60fce250f4
wallet: derive change addresses from the provided key scope
Previously, the wallet would determine the key scope to use for change
addresses by locating the one compatible with P2WPKH addresses, but this
wasn't always safe like in the case when multiple key scopes that
supported these addresses existed within the address manager, leading
the change address to be created outside of the intended key scope.
2020-03-30 15:30:53 -07:00
Oliver Gugger
fc64a1c704
wallet: wait for chain sync on Neutrino recovery
Normally the wallet doesn't wait for the chain backend to be synced
on regtest/simnet because there we cannot be certain if we are at
the chain tip, as there are no other nodes to compare to. For
Neutrino, this is a bit different because we rely on the cfheader
server to tell us what it thinks the chain tip is. For a wallet
recovery on Neutrino we therefore need to make sure we are at least
synced up to what the server thinks is the tip.
2020-03-16 10:24:27 +01:00
Wilmer Paulino
51b362d7c5
waddrmgr+wallet: only watch addresses within default key scopes
It was discovered that the wallet can scan the chain for unnecessary
additional addresses that are derived by higher-level applications using
custom key scopes. This isn't much of an issue for full nodes, but it
can cause light clients to scan more than what's required, triggering
more false positive matches which lead to block retrieval.

Now, we'll only scan the chain for addresses that exist within the
default key scopes, as those are the only ones the wallet should be
concerned about.
2020-01-24 17:12:23 -08:00
Wilmer Paulino
81797fe29d
multi: extend walletdb bolt driver with NoFreelistSync option
This allows external callers to set the option instead. All tests
remained with the option enabled.
2019-10-03 13:28:04 -04:00
Johan T. Halseth
0149963628
wallet: define ErrReplacement
This error will be returned if the backend rejects a transaction since
it doesn't satisfy the requirements for replacement.
2019-09-24 09:00:22 +02:00
Johan T. Halseth
7533180fd3
wallet: define ErrDoubleSpend
This error is returned in cases where there are conflicting transactions
either in the mempool or chain.
2019-09-24 09:00:10 +02:00
Johan T. Halseth
33770f11b2
wallet: clean up error string matching
Also note where the errors are found.
2019-09-20 09:31:02 +02:00
Olaoluwa Osuntokun
8dddf4319f
wallet/txauthor: create new wallet/txauthor module 2019-09-05 17:30:56 -07:00
Olaoluwa Osuntokun
a608ee4e4f
wallet/txrules: create new wallet/txrules module 2019-09-05 17:24:09 -07:00
Olaoluwa Osuntokun
d510e4772d
wallet/txsizes: create new txsizes package+module, remove internal version
In this commit, we move the existing `internal/txsizes` package into its
own package and make it a module along the way. This allows projects
like `neutrino` to depend on a slimmer set of `btcwallet` related
dependencies.
2019-09-05 17:18:14 -07:00
Wilmer Paulino
02b0a7c18d
wallet: fix nil txid in publishTransaction
A nil txid could've been returned from publishTransaction even if it was
successful. This was due to the underlying SendRawTransaction call
"failing", e.g., when the transaction being broadcast has already
confirmed, but publishTranasction interpreting such failure as a
success.
2019-07-10 16:25:01 -07:00
Olaoluwa Osuntokun
2698f8434e
wallet: speed up bare wallet init when using the btcd backend
In this commit, we speed up creating a fresh wallet when using the btcd
backend. Before this commit, we would need to rescan 10k or so blocks
when creating a new wallet with the btcd backend. btcd will actually
fully scan all the blocks even though we have zero addresses or UTXOs to
look for. As a result, this can take quite some time.

In this commit we modify the starting height of the initial rescan to
start at the birthday height, and only modify it if it's unset, or the
best height of the chain is before this birthday height. As a result, we
won't always have the full 10k block re org safety horizon on disk, but
will tend to this level after we begin to sync forward.
2019-06-28 15:01:09 -07:00
Wilmer Paulino
2ff8e92e37
wallet: remove wallet best height check when checkpointing intial sync
This check is not required since the wallet's best height is the genesis
block.
2019-06-19 20:00:00 -07:00
Wilmer Paulino
426f523475
wallet: start recovery from the wallet's best height
Previously, the wallet would attempt to store the same block it
checkpointed during its initial sync when performing a recovery. This
would cause the previous block existence validation check to be in
place, which would ultimately fail because the previous block was not
stored intentionally.

To address this, we always start/resume our recovery from the wallet's
best height. This also ensures that we do not rescan the same block
again when resuming a recovery after a shutdown.
2019-06-19 19:59:51 -07:00
Olaoluwa Osuntokun
de02d6fdfb
Merge pull request #626 from Roasbeef/dust-fix-sendoutputs
wallet: use relay fee to check for dust not fee rate
2019-06-18 17:55:38 -07:00
Olaoluwa Osuntokun
1dccdec1c1
wallet: use relay fee to check for dust not fee rate
In this commit we fix a lingering bug in our output sanity checks that
would only show up during time periods of persistently higher fees.
Before this commit we would incorrectly use the fee rate instead of the
min relay fee when checking an output for dust. This would cause us to
mistakenly reject a transaction for having a dust output.

We fix this by falling back to using the current min-relayfee.
2019-06-18 17:11:01 -07:00
Wilmer Paulino
a9847c28b6
wallet: improve error logging for unsuccessful notification handling
If a block arrives while the wallet is rescanning, users would be shown
a log message that is confusing and not very helpful.
2019-06-13 18:09:05 -07:00
Wilmer Paulino
fd8aa5d541
wallet: use locateBirthdayBlock within birthdaySanityCheck
We use the recently introduced locateBirthdayBlock function within
birthdaySanityCheck as it serves as a more optimized alternative that
achieves the same purpose.
2019-06-13 18:09:04 -07:00
Wilmer Paulino
26bc5ab104
wallet: remove unused function scanChain 2019-06-13 18:09:03 -07:00
Wilmer Paulino
fa65c1b5f7
wallet: store reorg safe height upon initial sync
Currently, wallet rescans start from its known tip of the chain. Since
we no longer store blocks all the way from genesis to the tip of the
chain, performing a rescan would cause us to scan blocks all the way
from genesis, which we want to avoid. To prevent this, we set the
wallet's tip to be the current reorg safe height. This ensures that
we're unable to scan any blocks before it, and that we maintain
MaxReorgDepth blocks stored.
2019-06-13 18:09:02 -07:00
Wilmer Paulino
e754478496
wallet: modify recovery logic to not start from genesis
This commit serves as another building point to allow the wallet to not
store blocks all the way from genesis to the tip of chain. We modify the
wallet's recovery logic to now start from either its birthday block, or
the current reorg safe height if it's before the birthday, to ensure the
wallet properly only stores MaxReorgDepth blocks.

We also refactor things a bit in hopes of making the logic a bit more
readable.
2019-06-13 18:09:02 -07:00
Wilmer Paulino
2a6f24c61b
wallet: locate birthday block without scanning chain from genesis
We do this as the wallet will no longer store blocks all the way from
genesis to the tip of the chain. Instead, in order to find a reasonable
birthday block, we resort to performing a binary search for a block
timestamp that's within +/-2 hours of the birthday timestamp.
2019-06-13 18:09:01 -07:00
Wilmer Paulino
17efcdba2f
wallet: wait until chain backend is current to begin wallet sync
This serves as groundwork for only storing up to MaxReorgDepth blocks
upon initial sync. To do so, we want to make sure the chain backend
considers itself current so that we can only fetch the latest
MaxReorgDepth blocks from it.
2019-06-13 18:09:00 -07:00
Wilmer Paulino
39f81c630b
chain: add IsCurrent method to chain.Interface
IsCurrent allows us to determine if the chain backend considers itself
"current" with the chain.
2019-06-13 18:08:59 -07:00
Wilmer Paulino
1ee2a239de
wallet: make wallet initial sync synchronous
This ensures the wallet can properly do an initial sync, a recovery, or
detect if it's on a stale branch before attempting to process new blocks
at tip.

Since the rescan will be triggered synchronously as well, we'll need to
catch the wallet's quit chan when handling rescan batches in order to
allow for clean shutdowns.
2019-06-13 18:08:58 -07:00
Wilmer Paulino
bf9cc20045
wallet: check RPC error code for rejected confirmed transactions
This unifies the logic of receiving an error when broadcasting a
confirmed transaction through btcd's/bitcoind's RPC interface. The btcd
dependency update is required in order for it to match bitcoind's
behavior. For older nodes that have yet to update, the confirmed
transaction will still be caught by the "transaction already exists"
case. This is not needed for bitcoind however, because its been sending
the same RPC error code for several major releases now.
2019-05-22 17:06:37 -07:00
Olaoluwa Osuntokun
acf3b04b02
Merge pull request #604 from wpaulino/scan-chain-bug-fix
wallet: ensure bestHeight is updated before isCurrent check
2019-03-12 20:26:08 -07:00
Olaoluwa Osuntokun
ecfdd888dd
Merge pull request #602 from wpaulino/wallet-sync-dev-env
wallet/wallet: accept 0 bestHeight in developer environments
2019-03-12 20:17:22 -07:00
Wilmer Paulino
06bf42c746
wallet: use last synced block as birthday in development environments
In this commit, we address an issue with chains that are not current,
like in the often case of regtest and simnet chains. Syncing the wallet
would fail due to the chain not being current and not finding a suitable
birthday block. We fix this by just using the last synced block as the
birthday block to ensure we can properly sync to the chain.
2019-03-12 18:33:45 -07:00
Wilmer Paulino
ea7c6c3ed9
wallet: accept 0 bestHeight in developer environments
In this commit, we fix a regression in the wallet when attempting to
sync new developer test chains such as regtest and simnet. The wallet
would block syncing until a block was mined, but in order to mine a
block, an address must be generated by the wallet first. This address
generation would block as the syncing logic was already holding the
database's mutex.
2019-03-12 18:33:31 -07:00
Olaoluwa Osuntokun
25804bf90f
Merge pull request #597 from wpaulino/tx-broadcast-err-handling
wallet/wallet: remove invalid transactions when broadcast fails
2019-03-08 21:10:47 -08:00
Wilmer Paulino
5f1ab915be
wallet: ensure bestHeight is updated before isCurrent check
In this commit, we fix an issue with the wallet's initial sync logic
where we'd miss processing all of the blocks in the chain. This can
happen if the backend is considered current while we're still catching
up. To address this, we make sure we update our best height to process
those missed blocks.

Co-authored-by: Roei Erez <roeierez@gmail.com>
2019-03-08 18:31:23 -08:00
Olaoluwa Osuntokun
8c64a08971
Merge pull request #601 from Roasbeef/recovery-bug-fix-node
wallet: only log block batch if non-empty
2019-03-01 13:43:46 -03:00
Olaoluwa Osuntokun
b9da1fbd8d
wallet: only log block batch if non-empty
This fixes an issue reputed by a user that would cause btcwallet to
panic if the full node was stopped while btcwallet was still restoring
the wallet.
2019-02-27 18:38:17 -03:00
Johan T. Halseth
650f859fdb
wallet: add dryrun arg to tx create, rolling back db if set 2019-02-27 13:08:18 -03:00
Wilmer Paulino
c7ab11f310
wallet/wallet: reuse publishTransaction within resendUnminedTxs
By doing this, we defer all error string-matching to happen within
publishTransaction, which allows us to simplify some of the existing
logic and maintain consistency.
2019-02-20 14:50:27 -08:00
Wilmer Paulino
7a2b5cef76
wallet/wallet: remove invalid transactions when broadcast fails
In this commit, we rework how publishTransaction works in order to
correctly handle removing invalid transactions from the wallet's
unconfirmed transaction store. This is crucial as otherwise, invalid
transactions can remain within the wallet and be used for further
transactions, causing a chain of inaccurate transactions.

publishTransaction will now only return an error if the transaction
fails to be broadcast and it has not been previously seen in the
mempool/chain. This is intended in order to provide an easier API to
callers. Any other errors when broadcasting the transaction will cause
it to be removed from the wallet's unconfirmed transaction store to
ensure it maintains an accurate view of the chain.
2019-02-20 14:50:27 -08:00
Wilmer Paulino
7e00d1843e
wallet/wallet: refactor sending transaction to backend into own method
We do this in order to be able to reuse the new publishTransaction
method within other parts of the wallet in order to consolidate all
error string-matching within one place.
2019-02-20 13:13:44 -08:00
Johan T. Halseth
eef81a4124
wallet/wallet: assume not current if best height is 0
This is done to avoid the birthday rescan to fail if the chain backend
reports a bestheight of 0.

Earlier it could happen that we attempted to sync to the birthday, but
since only the genesis block was available, which would be rejected as
birthday block because of the timestamp, it would fail to find a block
and the sync would fail.
2019-02-06 11:26:24 +01:00
Wilmer Paulino
c853ceaa60
wallet/wallet: consolidate rollback logic
In this commit, we consolidate the existing rollback logic to carry out
its duties under one database transaction.

Co-authored-by: Roei Erez <roeierez@gmail.com>
2019-01-14 22:42:45 -08:00
Wilmer Paulino
db837f1ba3
wallet/wallet: use new syncToBirthday and recovery methods
In this commit, we refactor the wallet's syncing logic with
syncWithChain to use the newer, simpler methods: syncToBirthday and
recovery. Along the way, we also fix a bug within the wallet where it
was possible to sync past the birthday, but not sync to tip completely
and restart, which would lead to us starting a rescan from the latest
synced height, rather than from the birthday stamp.

This commit slightly changes the wallet's syncing behavior to the
following:

  1. Ensure the wallet is synced to its birthday.
  2. Perform a recovery if requested.
  3. Check for chain reorgs.
  4. Dispatch a rescan from the current synced height.

Co-authored-by: Roei Erez <roeierez@gmail.com>
2019-01-14 22:42:45 -08:00
Wilmer Paulino
29e1f0c4fb
wallet/wallet: add new recovery method
In this commit, we add a new recovery method to the wallet. This method
attempts to recover any unspent outputs which pay to any of the wallet's
addresses. Most of the logic found within it is heavily borrowed from
the existing syncWithChain method. This method is currently unused, but
it will end up replacing some of the existing sync logic in a later
commit.
2019-01-14 22:42:45 -08:00
Wilmer Paulino
089cc747db
wallet/wallet: add new syncToBirthday method
In this commit, we add a new syncToBirthday method to the wallet. This
method intends to sync the wallet's point of the view of the chain until
finding its birthday. Most of the logic found within it is heavily
borrowed from the existing syncWithChain method. This method is
currently unused, but it will end up replacing some of the existing sync
logic in a later commit.

Co-authored-by: Roei Erez <roeierez@gmail.com>
2019-01-14 22:42:45 -08:00
Wilmer Paulino
80450c9033
wallet/chainntfns: make birthdaySanityCheck return ErrBirthdayBlockNotSet 2018-11-20 13:00:45 -08:00