Commit graph

1320 commits

Author SHA1 Message Date
Wilmer Paulino d2ce512b4b
build: pin to walletdb v1.3.1 2020-04-03 15:08:07 -07:00
Olaoluwa Osuntokun 2fed914b79
Merge pull request #690 from wpaulino/walletdb-revert-bolt-v1.3.4
walletdb: revert bolt v1.3.4
2020-04-03 15:01:46 -07:00
Wilmer Paulino 57be08fbbf
Revert "walletdb: update to bolt v1.3.4"
The v1.3.4 version of the bolt module contained a bug fix to allow the
use of go1.14, but it uncovered a GC issue.
2020-04-03 14:18:22 -07:00
Wilmer Paulino e51b4821ce
walletdb: use temp dir for db filepaths in tests 2020-04-03 14:13:53 -07:00
Olaoluwa Osuntokun 237a3ee772
Merge pull request #688 from wpaulino/neutrino-recovery-batch-filters
chain: batch filter fetches for neutrino chain client
2020-03-31 18:28:17 -07:00
Wilmer Paulino 90a0744ae4
chain: batch filter fetches for neutrino chain client
This greatly speeds up wallet recovery with a neutrino backend, as we'll
avoid unnecessary round trips for each block filter fetch.
2020-03-31 13:37:56 -07:00
Olaoluwa Osuntokun 60299130c6
Merge pull request #682 from wpaulino/scan-key-scopes
wallet: rescan and recover addresses in all relevant key scopes
2020-03-31 11:10:45 -07:00
Wilmer Paulino 31c027e19f
wallet: perform recovery on all registered key scopes
In similar fashion to the previous commit, due to a no longer existing
bug within the wallet, it was possible for change addresses to be
created outside of their intended key scope (the default), so wallets
affected by this now need to ensure upon recovery that they scan the
chain for _all_ existing key scopes, rather than just the default ones,
to reflect their proper balance. Through manual testing, it was shown
that the impact of recovering the additional key scopes is negligible in
most cases for both full nodes and light clients.
2020-03-30 15:35:50 -07:00
Wilmer Paulino 1285049923
wallet: include addresses from relevant key scopes in rescan
Due to a no longer existing bug within the wallet, it was possible for
change addresses to be created outside of their intended key scope (the
default), so wallets affected by this now need to ensure they scan the
chain for all addresses within the default key scopes (as expected), and
all _internal_ addresses (branch used for change addresses) within any
other registered key scopes to reflect their proper balance.
2020-03-30 15:35:49 -07:00
Wilmer Paulino 43e19da868
Revert "waddrmgr+wallet: only watch addresses within default key scopes"
The commit being reverted resulted in the discovery of a bug in which
change addresses could at times be created outside of the default key
scopes, causing us to not properly determine their spends.
2020-03-30 15:31:44 -07:00
Wilmer Paulino 60fce250f4
wallet: derive change addresses from the provided key scope
Previously, the wallet would determine the key scope to use for change
addresses by locating the one compatible with P2WPKH addresses, but this
wasn't always safe like in the case when multiple key scopes that
supported these addresses existed within the address manager, leading
the change address to be created outside of the intended key scope.
2020-03-30 15:30:53 -07:00
Olaoluwa Osuntokun 7b2cbe89f8
Merge pull request #686 from wpaulino/walletdb-upgrade
build: update to walletdb v1.3.0
2020-03-25 19:46:02 -07:00
Olaoluwa Osuntokun ba45925bcb
build: update to walletdb v1.3.0 2020-03-25 18:53:31 -07:00
Olaoluwa Osuntokun 60327da210
Merge pull request #684 from wpaulino/bbolt-134
walletdb: update to bolt v1.3.4
2020-03-25 18:50:50 -07:00
Olaoluwa Osuntokun 43b5326afa
walletdb: update to bolt v1.3.4 2020-03-25 15:28:30 -07:00
Olaoluwa Osuntokun 015c045a3b
Merge pull request #681 from guggero/neutrino-recovery
wallet: wait for chain sync on Neutrino recovery
2020-03-23 16:53:26 -07:00
Oliver Gugger fc64a1c704
wallet: wait for chain sync on Neutrino recovery
Normally the wallet doesn't wait for the chain backend to be synced
on regtest/simnet because there we cannot be certain if we are at
the chain tip, as there are no other nodes to compare to. For
Neutrino, this is a bit different because we rely on the cfheader
server to tell us what it thinks the chain tip is. For a wallet
recovery on Neutrino we therefore need to make sure we are at least
synced up to what the server thinks is the tip.
2020-03-16 10:24:27 +01:00
Olaoluwa Osuntokun ae9416ad76
Merge pull request #676 from wpaulino/watch-default-account-addrs-only
waddrmgr+wallet: only watch addresses within default key scopes
2020-02-18 16:46:49 -08:00
Olaoluwa Osuntokun a8ff50bc4b
Merge pull request #678 from bhandras/walletdb-testing
walletdb: generic param passing for walletdb test
2020-02-18 16:42:02 -08:00
Andras Banki-Horvath f2ed9c1c77 walletdb: generic param passing for walletdb test
This commit removes bbolt specific parameters from the interface
test to enable testing of other DB drivers trough the same entry
point.
2020-02-18 17:10:33 +01:00
Wilmer Paulino 51b362d7c5
waddrmgr+wallet: only watch addresses within default key scopes
It was discovered that the wallet can scan the chain for unnecessary
additional addresses that are derived by higher-level applications using
custom key scopes. This isn't much of an issue for full nodes, but it
can cause light clients to scan more than what's required, triggering
more false positive matches which lead to block retrieval.

Now, we'll only scan the chain for addresses that exist within the
default key scopes, as those are the only ones the wallet should be
concerned about.
2020-01-24 17:12:23 -08:00
Olaoluwa Osuntokun 704cd189ac
Merge pull request #675 from guggero/testing-scrypt
waddrmgr: export fast scrypt options
2020-01-23 18:38:27 -08:00
Oliver Gugger 655c23db1d
waddrmgr: export fast scrypt options 2020-01-23 13:47:02 +01:00
Olaoluwa Osuntokun f93a69f8f3
Merge pull request #673 from cfromknecht/wtxmgr-update-gomod
wtxmgr: update go.mod to walletdb 1.2.0
2020-01-15 16:33:20 -08:00
Conner Fromknecht 282ee8f09f
wtxmgr: update go.mod to walletdb 1.2.0 2020-01-15 12:40:34 -08:00
Olaoluwa Osuntokun 824604d88b
Merge pull request #672 from cfromknecht/walletdb-sequence
walletdb: add sequence methods from bbolt to the main bucket interface
2020-01-15 12:35:53 -08:00
Olaoluwa Osuntokun 2c6a714c14
walletdb: add new Batch package-level function
In this commit, we add a new package-level function to emulate the
existing Batch interface for bbolt. We do this via a new super-set
interface which is then checked against in the main implementation of
the Batch method.
2020-01-15 04:52:44 -08:00
Olaoluwa Osuntokun da2fe0e3b0
walletdb: ensure transactions are rolled back on panic 2020-01-15 04:52:34 -08:00
Olaoluwa Osuntokun e7e46cd6f9
build: update goclean.sh to properly test all packages
In this commit, we update our `go list` command to account for the
changes to the output of the command when modules are active. If modules
are active, then any packages which are themselves a sub-module won't
properly be listed.
2020-01-15 04:51:38 -08:00
Olaoluwa Osuntokun 95064ae858
wtxmgr+build: update all packges to point towards walletdb v1.2.0 2020-01-15 04:51:25 -08:00
Olaoluwa Osuntokun 0bcbb4cc4a
walletdb: add sequence methods to main interface, update bdb to implement
In this commit, we add the trio of sequence based methods that bbolt
ships with to the main bucket interface. We do this in order to easily
allow walletdb as is to be slotted into place where bbolt is currently
used, without sacrificing any functionality.
2020-01-15 04:51:15 -08:00
Olaoluwa Osuntokun eb2582bbde
walletdb: update examples to acccount for new 2nd param for bdb Open/Create 2020-01-15 04:50:22 -08:00
Olaoluwa Osuntokun 4aacb4a413
Merge pull request #665 from jcvernaleo/jcv_buildDocs
README: Update build instructions for switch from glide to modules
2020-01-13 14:20:16 -08:00
John C. Vernaleo c765989b51
README: Update build instructions for switch from glide to modules 2019-11-19 15:11:22 -05:00
Olaoluwa Osuntokun b19df70ddd
Merge pull request #664 from wpaulino/btcwallet-version-bump
version: bump version to 0.11.0
2019-11-12 19:22:12 -08:00
Wilmer Paulino d948e49ed3
version: bump version to 0.11.0 2019-11-12 19:20:25 -08:00
Olaoluwa Osuntokun df7568c50e
Merge pull request #663 from wpaulino/version-bump
build: update to latest versions of btcd+neutrino
2019-11-12 19:13:29 -08:00
Olaoluwa Osuntokun af97007ead
build: update to latest versions of btcd+neutrino 2019-11-12 19:00:05 -08:00
Olaoluwa Osuntokun d14cee90e5
Merge pull request #660 from wpaulino/zmq-reuse-msg-buffers
build+chain: reuse buffers when reading ZMQ messages from bitcoind
2019-11-12 18:45:29 -08:00
Wilmer Paulino d7d2f14d61
build+chain: reuse buffers when reading ZMQ messages from bitcoind
`bitcoind` notifies transactions once they're accepted into the mempool
and once they're confirmed in a block. Previously, reading a message
from ZMQ would allocate a buffer with the size of the message. This can
cause nodes to perform a large number of allocations within a small
amount periodically (3000 300B allocations every 10 mins on average),
which can cause a lot of GC pressure on lower resourced nodes. To remedy
this, we introduce two static buffers, one for blocks and another for
transactions, that will be reused for every message read. Each is
constrained by its maximum expected size.
2019-11-12 18:18:59 -08:00
Olaoluwa Osuntokun c49e7ef3ec
Merge pull request #658 from wpaulino/lingering-unconfirmed-input
wtxmgr: remove unconfirmed input reference for confirmed transcation …
2019-11-08 19:18:58 -08:00
Olaoluwa Osuntokun ef3abfafbc
Merge pull request #655 from breez/fix-re-broadcast-flow
Fix broadcast existing transaction
2019-11-08 19:14:04 -08:00
Olaoluwa Osuntokun b08b47817c
Merge pull request #653 from wpaulino/improve-tx-record-err-msg
wtxmgr: improve error message for failed tx record retrieval
2019-11-08 18:05:26 -08:00
Wilmer Paulino 58ac163d48
chain: trace log timed out ZMQ connections 2019-11-08 12:39:00 -08:00
Wilmer Paulino 36ca842905
chain: extract ZMQ command strings into constants 2019-11-08 12:38:28 -08:00
Wilmer Paulino 66e09f252d
wtxmgr: remove unconfirmed input reference for confirmed transcation
Previously, inserting a transaction as unconfirmed into the store and
later confirming it would leave a lingering unconfirmed input record.
This was discovered as part of
https://github.com/btcsuite/btcwallet/pull/655. This issue would only
affect the wallet if it tracked spent transaction outputs, which it
doesn't. We aim to resolve it in any case for the sake of internal
consistency.
2019-10-30 16:44:55 -07:00
Wilmer Paulino ac731b8e52
wtxmgr: add TestInsertMempoolTxAndConfirm
This test ensures that there aren't any lingering unconfirmed records
for a transaction that existed within the store as unconfirmed before
becoming confirmed. At the moment, this is currently failing due to a
gap when moving a transaction from unconfirmed to confirmed within the
store. This will be resolved in a subsequent commit.
2019-10-30 16:44:54 -07:00
Roei Erez 1c43ed9294 wtxmgr: fix broadcast existing transaction
This commit ensures the wallet won't enter an inconsitent state
by checking tx confirmation before adding credit.
Without this fix, In the case the existing transaction is already
confirmed on-chain the flow updates the bucketUnminedCredits but
without adding en entry also to the bucketUnmined resulting in
inconsistent state.
2019-10-31 00:28:21 +02:00
Wilmer Paulino 1e2f445cdd
wtxmgr: improve error message for failed tx record retrieval 2019-10-24 09:24:32 -04:00
Olaoluwa Osuntokun 7abdd4f8ad
Merge pull request #649 from halseth/neutrino-dep-update-cfilter-fix
mod: update neutrino dep
2019-10-14 18:12:23 +02:00