Commit graph

1476 commits

Author SHA1 Message Date
carla
9aed49070d
wtxmgr: update rangeBlockTransactions to use minedTxDetails
Code is duplicated across these two functions, so update
rangeBlockTransactions to use minedTxDetails. When iterating through
a block, we do have the block metadata already (which is looked up by
minedTxDetails). This change can be further optimized to split
minedTxDetails into minedTxDetails and minedTxDetailsWithoutBlock to
reduce this redundancy.
2020-05-11 15:18:06 +02:00
carla
9e2f2ce157
wallet: add label parameter to SendOutputs 2020-05-11 15:17:55 +02:00
carla
d2f9185f6a
wallet: add label to PublishTransaction
All label parameter to PublishTransaction. Pass in an empty string
in rpc call as a placeholder for follow up PR which will add a label
parameter to the PublishTransaction request.
2020-05-11 15:16:22 +02:00
carla
50869085eb
wtxmgr: add put and fetch functions for optional transaction label
Add and test functions which can be used to write optional transaction
labels to disk in their own bucket. These labels are keyed by txid and
write the labels to disk using-length value encoding scheme. Although
the length field is not required at present, it is added to allow future
extensibility without a migration.

This approach is chosen over adding this information to txRecords,
Because a migration would be required to add a field after the variable
Length serialized tx.

The put label function will overwrite existing labels if called more
than once for the same txid. User side validation of whether we want
to override this label should be performed by calling code. Labels must
be > 0 characters and <= 500 characters (an arbitrarily chosen limit).
2020-05-11 15:16:11 +02:00
Dev Random
4c5bc1b15d
waddrmgr: create watch-only address managers and accounts
This PR allows the creation of managers and accounts that are watch-only. The state of the database after creation would be identical to the state after calling 

Manager.ConvertToWatchingOnly, assuming accounts with the right xpubs were created in the former case.

Co-authored-by: Ken Sedgwick <ken@bonsai.com>
2020-04-24 17:44:21 -07:00
Olaoluwa Osuntokun
ada7ca077e
Merge pull request #691 from wpaulino/pin-walletdb-v1.3.1
build: pin to walletdb v1.3.1
2020-04-03 15:22:02 -07:00
Wilmer Paulino
d2ce512b4b
build: pin to walletdb v1.3.1 2020-04-03 15:08:07 -07:00
Olaoluwa Osuntokun
2fed914b79
Merge pull request #690 from wpaulino/walletdb-revert-bolt-v1.3.4
walletdb: revert bolt v1.3.4
2020-04-03 15:01:46 -07:00
Wilmer Paulino
57be08fbbf
Revert "walletdb: update to bolt v1.3.4"
The v1.3.4 version of the bolt module contained a bug fix to allow the
use of go1.14, but it uncovered a GC issue.
2020-04-03 14:18:22 -07:00
Wilmer Paulino
e51b4821ce
walletdb: use temp dir for db filepaths in tests 2020-04-03 14:13:53 -07:00
Olaoluwa Osuntokun
237a3ee772
Merge pull request #688 from wpaulino/neutrino-recovery-batch-filters
chain: batch filter fetches for neutrino chain client
2020-03-31 18:28:17 -07:00
Wilmer Paulino
90a0744ae4
chain: batch filter fetches for neutrino chain client
This greatly speeds up wallet recovery with a neutrino backend, as we'll
avoid unnecessary round trips for each block filter fetch.
2020-03-31 13:37:56 -07:00
Olaoluwa Osuntokun
60299130c6
Merge pull request #682 from wpaulino/scan-key-scopes
wallet: rescan and recover addresses in all relevant key scopes
2020-03-31 11:10:45 -07:00
Wilmer Paulino
31c027e19f
wallet: perform recovery on all registered key scopes
In similar fashion to the previous commit, due to a no longer existing
bug within the wallet, it was possible for change addresses to be
created outside of their intended key scope (the default), so wallets
affected by this now need to ensure upon recovery that they scan the
chain for _all_ existing key scopes, rather than just the default ones,
to reflect their proper balance. Through manual testing, it was shown
that the impact of recovering the additional key scopes is negligible in
most cases for both full nodes and light clients.
2020-03-30 15:35:50 -07:00
Wilmer Paulino
1285049923
wallet: include addresses from relevant key scopes in rescan
Due to a no longer existing bug within the wallet, it was possible for
change addresses to be created outside of their intended key scope (the
default), so wallets affected by this now need to ensure they scan the
chain for all addresses within the default key scopes (as expected), and
all _internal_ addresses (branch used for change addresses) within any
other registered key scopes to reflect their proper balance.
2020-03-30 15:35:49 -07:00
Wilmer Paulino
43e19da868
Revert "waddrmgr+wallet: only watch addresses within default key scopes"
The commit being reverted resulted in the discovery of a bug in which
change addresses could at times be created outside of the default key
scopes, causing us to not properly determine their spends.
2020-03-30 15:31:44 -07:00
Wilmer Paulino
60fce250f4
wallet: derive change addresses from the provided key scope
Previously, the wallet would determine the key scope to use for change
addresses by locating the one compatible with P2WPKH addresses, but this
wasn't always safe like in the case when multiple key scopes that
supported these addresses existed within the address manager, leading
the change address to be created outside of the intended key scope.
2020-03-30 15:30:53 -07:00
Olaoluwa Osuntokun
7b2cbe89f8
Merge pull request #686 from wpaulino/walletdb-upgrade
build: update to walletdb v1.3.0
2020-03-25 19:46:02 -07:00
Olaoluwa Osuntokun
ba45925bcb
build: update to walletdb v1.3.0 2020-03-25 18:53:31 -07:00
Olaoluwa Osuntokun
60327da210
Merge pull request #684 from wpaulino/bbolt-134
walletdb: update to bolt v1.3.4
2020-03-25 18:50:50 -07:00
Olaoluwa Osuntokun
43b5326afa
walletdb: update to bolt v1.3.4 2020-03-25 15:28:30 -07:00
Olaoluwa Osuntokun
015c045a3b
Merge pull request #681 from guggero/neutrino-recovery
wallet: wait for chain sync on Neutrino recovery
2020-03-23 16:53:26 -07:00
Oliver Gugger
fc64a1c704
wallet: wait for chain sync on Neutrino recovery
Normally the wallet doesn't wait for the chain backend to be synced
on regtest/simnet because there we cannot be certain if we are at
the chain tip, as there are no other nodes to compare to. For
Neutrino, this is a bit different because we rely on the cfheader
server to tell us what it thinks the chain tip is. For a wallet
recovery on Neutrino we therefore need to make sure we are at least
synced up to what the server thinks is the tip.
2020-03-16 10:24:27 +01:00
Olaoluwa Osuntokun
ae9416ad76
Merge pull request #676 from wpaulino/watch-default-account-addrs-only
waddrmgr+wallet: only watch addresses within default key scopes
2020-02-18 16:46:49 -08:00
Olaoluwa Osuntokun
a8ff50bc4b
Merge pull request #678 from bhandras/walletdb-testing
walletdb: generic param passing for walletdb test
2020-02-18 16:42:02 -08:00
Andras Banki-Horvath
f2ed9c1c77 walletdb: generic param passing for walletdb test
This commit removes bbolt specific parameters from the interface
test to enable testing of other DB drivers trough the same entry
point.
2020-02-18 17:10:33 +01:00
Wilmer Paulino
51b362d7c5
waddrmgr+wallet: only watch addresses within default key scopes
It was discovered that the wallet can scan the chain for unnecessary
additional addresses that are derived by higher-level applications using
custom key scopes. This isn't much of an issue for full nodes, but it
can cause light clients to scan more than what's required, triggering
more false positive matches which lead to block retrieval.

Now, we'll only scan the chain for addresses that exist within the
default key scopes, as those are the only ones the wallet should be
concerned about.
2020-01-24 17:12:23 -08:00
Olaoluwa Osuntokun
704cd189ac
Merge pull request #675 from guggero/testing-scrypt
waddrmgr: export fast scrypt options
2020-01-23 18:38:27 -08:00
Oliver Gugger
655c23db1d
waddrmgr: export fast scrypt options 2020-01-23 13:47:02 +01:00
Olaoluwa Osuntokun
f93a69f8f3
Merge pull request #673 from cfromknecht/wtxmgr-update-gomod
wtxmgr: update go.mod to walletdb 1.2.0
2020-01-15 16:33:20 -08:00
Conner Fromknecht
282ee8f09f
wtxmgr: update go.mod to walletdb 1.2.0 2020-01-15 12:40:34 -08:00
Olaoluwa Osuntokun
824604d88b
Merge pull request #672 from cfromknecht/walletdb-sequence
walletdb: add sequence methods from bbolt to the main bucket interface
2020-01-15 12:35:53 -08:00
Olaoluwa Osuntokun
2c6a714c14
walletdb: add new Batch package-level function
In this commit, we add a new package-level function to emulate the
existing Batch interface for bbolt. We do this via a new super-set
interface which is then checked against in the main implementation of
the Batch method.
2020-01-15 04:52:44 -08:00
Olaoluwa Osuntokun
da2fe0e3b0
walletdb: ensure transactions are rolled back on panic 2020-01-15 04:52:34 -08:00
Olaoluwa Osuntokun
e7e46cd6f9
build: update goclean.sh to properly test all packages
In this commit, we update our `go list` command to account for the
changes to the output of the command when modules are active. If modules
are active, then any packages which are themselves a sub-module won't
properly be listed.
2020-01-15 04:51:38 -08:00
Olaoluwa Osuntokun
95064ae858
wtxmgr+build: update all packges to point towards walletdb v1.2.0 2020-01-15 04:51:25 -08:00
Olaoluwa Osuntokun
0bcbb4cc4a
walletdb: add sequence methods to main interface, update bdb to implement
In this commit, we add the trio of sequence based methods that bbolt
ships with to the main bucket interface. We do this in order to easily
allow walletdb as is to be slotted into place where bbolt is currently
used, without sacrificing any functionality.
2020-01-15 04:51:15 -08:00
Olaoluwa Osuntokun
eb2582bbde
walletdb: update examples to acccount for new 2nd param for bdb Open/Create 2020-01-15 04:50:22 -08:00
Olaoluwa Osuntokun
4aacb4a413
Merge pull request #665 from jcvernaleo/jcv_buildDocs
README: Update build instructions for switch from glide to modules
2020-01-13 14:20:16 -08:00
John C. Vernaleo
c765989b51
README: Update build instructions for switch from glide to modules 2019-11-19 15:11:22 -05:00
Olaoluwa Osuntokun
b19df70ddd
Merge pull request #664 from wpaulino/btcwallet-version-bump
version: bump version to 0.11.0
2019-11-12 19:22:12 -08:00
Wilmer Paulino
d948e49ed3
version: bump version to 0.11.0 2019-11-12 19:20:25 -08:00
Olaoluwa Osuntokun
df7568c50e
Merge pull request #663 from wpaulino/version-bump
build: update to latest versions of btcd+neutrino
2019-11-12 19:13:29 -08:00
Olaoluwa Osuntokun
af97007ead
build: update to latest versions of btcd+neutrino 2019-11-12 19:00:05 -08:00
Olaoluwa Osuntokun
d14cee90e5
Merge pull request #660 from wpaulino/zmq-reuse-msg-buffers
build+chain: reuse buffers when reading ZMQ messages from bitcoind
2019-11-12 18:45:29 -08:00
Wilmer Paulino
d7d2f14d61
build+chain: reuse buffers when reading ZMQ messages from bitcoind
`bitcoind` notifies transactions once they're accepted into the mempool
and once they're confirmed in a block. Previously, reading a message
from ZMQ would allocate a buffer with the size of the message. This can
cause nodes to perform a large number of allocations within a small
amount periodically (3000 300B allocations every 10 mins on average),
which can cause a lot of GC pressure on lower resourced nodes. To remedy
this, we introduce two static buffers, one for blocks and another for
transactions, that will be reused for every message read. Each is
constrained by its maximum expected size.
2019-11-12 18:18:59 -08:00
Olaoluwa Osuntokun
c49e7ef3ec
Merge pull request #658 from wpaulino/lingering-unconfirmed-input
wtxmgr: remove unconfirmed input reference for confirmed transcation …
2019-11-08 19:18:58 -08:00
Olaoluwa Osuntokun
ef3abfafbc
Merge pull request #655 from breez/fix-re-broadcast-flow
Fix broadcast existing transaction
2019-11-08 19:14:04 -08:00
Olaoluwa Osuntokun
b08b47817c
Merge pull request #653 from wpaulino/improve-tx-record-err-msg
wtxmgr: improve error message for failed tx record retrieval
2019-11-08 18:05:26 -08:00
Wilmer Paulino
58ac163d48
chain: trace log timed out ZMQ connections 2019-11-08 12:39:00 -08:00