Commit graph

1365 commits

Author SHA1 Message Date
Wilmer Paulino
90a0744ae4
chain: batch filter fetches for neutrino chain client
This greatly speeds up wallet recovery with a neutrino backend, as we'll
avoid unnecessary round trips for each block filter fetch.
2020-03-31 13:37:56 -07:00
Olaoluwa Osuntokun
60299130c6
Merge pull request #682 from wpaulino/scan-key-scopes
wallet: rescan and recover addresses in all relevant key scopes
2020-03-31 11:10:45 -07:00
Wilmer Paulino
31c027e19f
wallet: perform recovery on all registered key scopes
In similar fashion to the previous commit, due to a no longer existing
bug within the wallet, it was possible for change addresses to be
created outside of their intended key scope (the default), so wallets
affected by this now need to ensure upon recovery that they scan the
chain for _all_ existing key scopes, rather than just the default ones,
to reflect their proper balance. Through manual testing, it was shown
that the impact of recovering the additional key scopes is negligible in
most cases for both full nodes and light clients.
2020-03-30 15:35:50 -07:00
Wilmer Paulino
1285049923
wallet: include addresses from relevant key scopes in rescan
Due to a no longer existing bug within the wallet, it was possible for
change addresses to be created outside of their intended key scope (the
default), so wallets affected by this now need to ensure they scan the
chain for all addresses within the default key scopes (as expected), and
all _internal_ addresses (branch used for change addresses) within any
other registered key scopes to reflect their proper balance.
2020-03-30 15:35:49 -07:00
Wilmer Paulino
43e19da868
Revert "waddrmgr+wallet: only watch addresses within default key scopes"
The commit being reverted resulted in the discovery of a bug in which
change addresses could at times be created outside of the default key
scopes, causing us to not properly determine their spends.
2020-03-30 15:31:44 -07:00
Wilmer Paulino
60fce250f4
wallet: derive change addresses from the provided key scope
Previously, the wallet would determine the key scope to use for change
addresses by locating the one compatible with P2WPKH addresses, but this
wasn't always safe like in the case when multiple key scopes that
supported these addresses existed within the address manager, leading
the change address to be created outside of the intended key scope.
2020-03-30 15:30:53 -07:00
Olaoluwa Osuntokun
7b2cbe89f8
Merge pull request #686 from wpaulino/walletdb-upgrade
build: update to walletdb v1.3.0
2020-03-25 19:46:02 -07:00
Olaoluwa Osuntokun
ba45925bcb
build: update to walletdb v1.3.0 2020-03-25 18:53:31 -07:00
Olaoluwa Osuntokun
60327da210
Merge pull request #684 from wpaulino/bbolt-134
walletdb: update to bolt v1.3.4
2020-03-25 18:50:50 -07:00
Olaoluwa Osuntokun
43b5326afa
walletdb: update to bolt v1.3.4 2020-03-25 15:28:30 -07:00
Olaoluwa Osuntokun
015c045a3b
Merge pull request #681 from guggero/neutrino-recovery
wallet: wait for chain sync on Neutrino recovery
2020-03-23 16:53:26 -07:00
Oliver Gugger
fc64a1c704
wallet: wait for chain sync on Neutrino recovery
Normally the wallet doesn't wait for the chain backend to be synced
on regtest/simnet because there we cannot be certain if we are at
the chain tip, as there are no other nodes to compare to. For
Neutrino, this is a bit different because we rely on the cfheader
server to tell us what it thinks the chain tip is. For a wallet
recovery on Neutrino we therefore need to make sure we are at least
synced up to what the server thinks is the tip.
2020-03-16 10:24:27 +01:00
Olaoluwa Osuntokun
ae9416ad76
Merge pull request #676 from wpaulino/watch-default-account-addrs-only
waddrmgr+wallet: only watch addresses within default key scopes
2020-02-18 16:46:49 -08:00
Olaoluwa Osuntokun
a8ff50bc4b
Merge pull request #678 from bhandras/walletdb-testing
walletdb: generic param passing for walletdb test
2020-02-18 16:42:02 -08:00
Andras Banki-Horvath
f2ed9c1c77 walletdb: generic param passing for walletdb test
This commit removes bbolt specific parameters from the interface
test to enable testing of other DB drivers trough the same entry
point.
2020-02-18 17:10:33 +01:00
Wilmer Paulino
51b362d7c5
waddrmgr+wallet: only watch addresses within default key scopes
It was discovered that the wallet can scan the chain for unnecessary
additional addresses that are derived by higher-level applications using
custom key scopes. This isn't much of an issue for full nodes, but it
can cause light clients to scan more than what's required, triggering
more false positive matches which lead to block retrieval.

Now, we'll only scan the chain for addresses that exist within the
default key scopes, as those are the only ones the wallet should be
concerned about.
2020-01-24 17:12:23 -08:00
Olaoluwa Osuntokun
704cd189ac
Merge pull request #675 from guggero/testing-scrypt
waddrmgr: export fast scrypt options
2020-01-23 18:38:27 -08:00
Oliver Gugger
655c23db1d
waddrmgr: export fast scrypt options 2020-01-23 13:47:02 +01:00
Olaoluwa Osuntokun
f93a69f8f3
Merge pull request #673 from cfromknecht/wtxmgr-update-gomod
wtxmgr: update go.mod to walletdb 1.2.0
2020-01-15 16:33:20 -08:00
Conner Fromknecht
282ee8f09f
wtxmgr: update go.mod to walletdb 1.2.0 2020-01-15 12:40:34 -08:00
Olaoluwa Osuntokun
824604d88b
Merge pull request #672 from cfromknecht/walletdb-sequence
walletdb: add sequence methods from bbolt to the main bucket interface
2020-01-15 12:35:53 -08:00
Olaoluwa Osuntokun
2c6a714c14
walletdb: add new Batch package-level function
In this commit, we add a new package-level function to emulate the
existing Batch interface for bbolt. We do this via a new super-set
interface which is then checked against in the main implementation of
the Batch method.
2020-01-15 04:52:44 -08:00
Olaoluwa Osuntokun
da2fe0e3b0
walletdb: ensure transactions are rolled back on panic 2020-01-15 04:52:34 -08:00
Olaoluwa Osuntokun
e7e46cd6f9
build: update goclean.sh to properly test all packages
In this commit, we update our `go list` command to account for the
changes to the output of the command when modules are active. If modules
are active, then any packages which are themselves a sub-module won't
properly be listed.
2020-01-15 04:51:38 -08:00
Olaoluwa Osuntokun
95064ae858
wtxmgr+build: update all packges to point towards walletdb v1.2.0 2020-01-15 04:51:25 -08:00
Olaoluwa Osuntokun
0bcbb4cc4a
walletdb: add sequence methods to main interface, update bdb to implement
In this commit, we add the trio of sequence based methods that bbolt
ships with to the main bucket interface. We do this in order to easily
allow walletdb as is to be slotted into place where bbolt is currently
used, without sacrificing any functionality.
2020-01-15 04:51:15 -08:00
Olaoluwa Osuntokun
eb2582bbde
walletdb: update examples to acccount for new 2nd param for bdb Open/Create 2020-01-15 04:50:22 -08:00
Olaoluwa Osuntokun
4aacb4a413
Merge pull request #665 from jcvernaleo/jcv_buildDocs
README: Update build instructions for switch from glide to modules
2020-01-13 14:20:16 -08:00
John C. Vernaleo
c765989b51
README: Update build instructions for switch from glide to modules 2019-11-19 15:11:22 -05:00
Olaoluwa Osuntokun
b19df70ddd
Merge pull request #664 from wpaulino/btcwallet-version-bump
version: bump version to 0.11.0
2019-11-12 19:22:12 -08:00
Wilmer Paulino
d948e49ed3
version: bump version to 0.11.0 2019-11-12 19:20:25 -08:00
Olaoluwa Osuntokun
df7568c50e
Merge pull request #663 from wpaulino/version-bump
build: update to latest versions of btcd+neutrino
2019-11-12 19:13:29 -08:00
Olaoluwa Osuntokun
af97007ead
build: update to latest versions of btcd+neutrino 2019-11-12 19:00:05 -08:00
Olaoluwa Osuntokun
d14cee90e5
Merge pull request #660 from wpaulino/zmq-reuse-msg-buffers
build+chain: reuse buffers when reading ZMQ messages from bitcoind
2019-11-12 18:45:29 -08:00
Wilmer Paulino
d7d2f14d61
build+chain: reuse buffers when reading ZMQ messages from bitcoind
`bitcoind` notifies transactions once they're accepted into the mempool
and once they're confirmed in a block. Previously, reading a message
from ZMQ would allocate a buffer with the size of the message. This can
cause nodes to perform a large number of allocations within a small
amount periodically (3000 300B allocations every 10 mins on average),
which can cause a lot of GC pressure on lower resourced nodes. To remedy
this, we introduce two static buffers, one for blocks and another for
transactions, that will be reused for every message read. Each is
constrained by its maximum expected size.
2019-11-12 18:18:59 -08:00
Olaoluwa Osuntokun
c49e7ef3ec
Merge pull request #658 from wpaulino/lingering-unconfirmed-input
wtxmgr: remove unconfirmed input reference for confirmed transcation …
2019-11-08 19:18:58 -08:00
Olaoluwa Osuntokun
ef3abfafbc
Merge pull request #655 from breez/fix-re-broadcast-flow
Fix broadcast existing transaction
2019-11-08 19:14:04 -08:00
Olaoluwa Osuntokun
b08b47817c
Merge pull request #653 from wpaulino/improve-tx-record-err-msg
wtxmgr: improve error message for failed tx record retrieval
2019-11-08 18:05:26 -08:00
Wilmer Paulino
58ac163d48
chain: trace log timed out ZMQ connections 2019-11-08 12:39:00 -08:00
Wilmer Paulino
36ca842905
chain: extract ZMQ command strings into constants 2019-11-08 12:38:28 -08:00
Wilmer Paulino
66e09f252d
wtxmgr: remove unconfirmed input reference for confirmed transcation
Previously, inserting a transaction as unconfirmed into the store and
later confirming it would leave a lingering unconfirmed input record.
This was discovered as part of
https://github.com/btcsuite/btcwallet/pull/655. This issue would only
affect the wallet if it tracked spent transaction outputs, which it
doesn't. We aim to resolve it in any case for the sake of internal
consistency.
2019-10-30 16:44:55 -07:00
Wilmer Paulino
ac731b8e52
wtxmgr: add TestInsertMempoolTxAndConfirm
This test ensures that there aren't any lingering unconfirmed records
for a transaction that existed within the store as unconfirmed before
becoming confirmed. At the moment, this is currently failing due to a
gap when moving a transaction from unconfirmed to confirmed within the
store. This will be resolved in a subsequent commit.
2019-10-30 16:44:54 -07:00
Roei Erez
1c43ed9294 wtxmgr: fix broadcast existing transaction
This commit ensures the wallet won't enter an inconsitent state
by checking tx confirmation before adding credit.
Without this fix, In the case the existing transaction is already
confirmed on-chain the flow updates the bucketUnminedCredits but
without adding en entry also to the bucketUnmined resulting in
inconsistent state.
2019-10-31 00:28:21 +02:00
Wilmer Paulino
1e2f445cdd
wtxmgr: improve error message for failed tx record retrieval 2019-10-24 09:24:32 -04:00
Olaoluwa Osuntokun
7abdd4f8ad
Merge pull request #649 from halseth/neutrino-dep-update-cfilter-fix
mod: update neutrino dep
2019-10-14 18:12:23 +02:00
Johan T. Halseth
ed825b0fd9
mod: update neutrino+btcd dep 2019-10-14 17:53:31 +02:00
Olaoluwa Osuntokun
5f205d607b
Merge pull request #648 from wpaulino/bolt-no-freelist-sync-option
multi: extend walletdb bolt driver with NoFreelistSync option
2019-10-03 21:45:42 -07:00
Wilmer Paulino
81797fe29d
multi: extend walletdb bolt driver with NoFreelistSync option
This allows external callers to set the option instead. All tests
remained with the option enabled.
2019-10-03 13:28:04 -04:00
Olaoluwa Osuntokun
95d7aa0b49
Merge pull request #647 from halseth/rbf-reject-matching
Cleanup publication error matching, add ErrDoubleSpend/ErrReplacment
2019-09-24 17:50:52 -07:00
Johan T. Halseth
0149963628
wallet: define ErrReplacement
This error will be returned if the backend rejects a transaction since
it doesn't satisfy the requirements for replacement.
2019-09-24 09:00:22 +02:00