Commit graph

1026 commits

Author SHA1 Message Date
Olaoluwa Osuntokun
921dae5d5e wallet: catch remaining bitcoind errors, and generic RPC errors resendUnminedTxs
In this commit, ensure that upon restart, if any of the full-node based
backends we support reject the transaction, then we'll properly remove
the now invalid transaction from the tx store. Before this commit, we
could miss a few errors from bitcoind. To remedy this, we explicitly
catch those errors, but then also attempt to precisely catch the set of
generic json RPC errors that can be returned.
2018-05-23 19:38:56 -07:00
Wilmer Paulino
8b2aebe89e wallet: add call to modify both public and private passphrases atomically 2018-05-23 19:38:56 -07:00
Kenneth Perry (thothonegan)
0547e5a313 When determining whether to rollback, ask the chain using the chainhash instead of the possibly invalid local hash 2018-05-23 19:38:56 -07:00
Alex
3f15a85da7 chain: ensure bitcoind back-end fully rescans and sends correct ntfn 2018-05-23 19:38:56 -07:00
Alex
81c4b1d096 chain: eliminate start of sync race condition in bitcoind back-end
This commit also allows bitcoind back-end to return block timestamp
in `BlockStamp()` and prevents logging and notification of events
until after the client wallet requests notifications.
2018-05-23 19:38:56 -07:00
Alex
6d16463627 wallet: demote chainntfns error about out-of-sync blocks to debug 2018-05-23 19:38:56 -07:00
Alex
6b6ed89d87 chain: improve reorg-during-rescan handling for bitcoind 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
321c2fac2b build: switch from bolt to bbolt 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
5eaecee2c9 wallet: ensure timestamp is always set when calling waddrmgr.SetSyncedTo 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
0dcd36bf59 wallet: only remove conflicting unmined transactions
In this commit, we fix a bug introduced in an earlier commit. Before
this commit, we would *always* remove an unmined transaction if it
failed to be accepted by the network upon restart. Instead, we should
only remove transaction that are actually due to us trying to spend an
output that’s already spent, or an orphan transaction.
2018-05-23 19:38:56 -07:00
Conner Fromknecht
1d50b92bdc waddrmgr/multi: fix scoped mgr reentry deadlock
This commit resolves a deadlock observed when attempting
to generate addresses. There were a few cases, particularly
in chainAddressRowToManaged and loadAccountInfo, which accessed
the public IsLocked() method of the Manager, even though the
shared mutex had already been acquired.

The solution is to create an internal isLocked() method, which
can be safely called assuming the manager's mutex has already been
acquired. As the comments above both of the methods in question
specify, we can assume the Manager's mutex *is* already acquired.

This commit also reduces some unnecessary code duplication, since
the recent changes left both a Locked() and IsLocked() method that
perform the same functionality. IsLocked() was favored as it more
clearly indicates that the returned value is a boolean.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
3b66af9154 wallet: properly set timestamp during block connect/disconnect 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
d0d94666cf waddrmgr: update test to ensure timestamp is set 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
27dfed7f27 waddrmgr: fix deadlock bug by using read mutex when necessary
In this commit, we fix a deadlock bug that was introduced recently.
This can happen when ForEachActiveAccountAddress or
ForEachActiveAddress is called, as these internally need to grab the
mutex of the manager (within the scoped manager) in order to check if
the manager is locked or not.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
d42a45f16f waddrmgr: add database migration from version 4 to 5
In this commit, we add a database migration from version 4 to version 5.
We also take this opportunity to clean up the old migration code. This
is no longer needed as wallets very old can simply go back in the prior
git history to migrate to version 4, then go from there to version 5.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
b75bf1426e wallet: update wallet to be aware of new KeyScopes 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
115d7d17f2 rpc: update rpc servers to default to waddrmgr.KeyScopeBIP0044 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
3ca68d8259 waddrmgr: add additional tests to exercise new scope specific features 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
b8104ef86e waddrmgr: update existing tests to be aware of new key scopes 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
9a8da416ef waddrmgr: update all addresses to be aware of new addr scopes 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
641efbbd31 waddrmgr: extend the set of AddressType enum values with script+rawkey 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
a75170c650 waddrmgr: update the Manager struct to remove functionality covered by ScopedManager
In this commit, we remove many of the methods in the Manager struct as
they’ve now be been replicated within a scoped format for each of the
ScopedKeyManagers.

A major change is that we’ll now actually store the master HD private
and public keys. This required as in order to create new scopes, we
need access to the master HD private key as hardened derivation is
required in accordance with BIP43.

The initial creation of the manager namespaces has also been extended
to create the namespaces and keys for the set of default key scopes.

Finally, a series of utility method has been added to allow callers to
create ScopedKeyManagers for arbitrary sets of scopes.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
e3afc8be6a waddrmgr: create new ScopedKeyManager to manage BIP43 purposes
In this commit, we create a new struct that houses the key derivation,
address management, and account management for a particular scope. A
scope consists of a (purpose, cointype) tuple. Additionally, each
ScopedKeyManager is able to generate internal/external addresses for a
specific *address type*. This make rescans easier as for each scope, we
know what type of output to look for within the chain.

The ScopedKeyManagers have two new primary methods that weren’t
previously present within the regular Manager:
   * DeriveFromKeyPath
   * NewRawAccount

These two methods allow callers a greater degree of control over the
way that accounted are created and addressed derived.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
f5de14f16b waddrmgr: update database version to 5 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
9f71d472a3 waddrmgr: extend createManagerNS to also create name spaces for each scope 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
57cf4274ee waddrmgr: extend deletePrivateKeys to delete HD key and scope keys 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
946ca2da1e waddrmgr: add ability to store encrypt master HD priv/pub keys
In this commit, we create new key spaces to allow users to store the
encrypted master priv/pub keys. This is required as in order to create
new scopes, we must do hardened derivation from the root key.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
e8755c2bc2 waddrmgr: introduce new bucket hierarchy with scopes at the top
In this commit, we make a fundamental modification bucket structure
within the database. Most buckets are no under an additional layer of
nesting: the scope. The scope encapsulates which (purpose, coin type)
pair the address, accounts, and coin type keys belong to.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
8bd9f7713b waddrmgr: remove direct references to BIP 44 in struct/variable names
In this commit, we remove all direct references to BIP 44 as upcoming
changes will shift to a model that is no longer directly dependent on
BIP 44 in favor of restoring a layer of abstraction and allowing users
to manage multiple (purpose, coin type) scopes within the same
database.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
ea82d184bf waddrmgr: remove additional addressType values
In this commit, we remove the extra addressType values as this will no
longer be needed due to the new abstraction of scoped key managers.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
6f6abef1d6 waddrmgr: introduce concept of scopes for key managers
In this commit, we introduce the concept of scopes for individual key
managers. Each scope will lock down a key manager to a particular
purpose and coin type within the BIP0043 hierarchy. Each scope will
also have a set address type schema. This schema will be consulted when
creating addresses for a particular scoped key manager.

Finally, we introduce 3 new default scopes:
   * BIP 44
   * BIP 84
   * BIP 49++ (BIP49 but uses p2wkh for change addresses)
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
f4ef7cdd29 waddrmgr: store block timestamp in BlockStamp 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
ab4ccacbb9 wallet: extend PublishTransaction to be a general reliable broadcaster
In this commit, we extend the PublishTransction method to be a more
general semi reliable transaction broadcast mechanism. We do this by
removing the special casing for neutrino. With this change, we’ll
_always_ write any transactions to be broadcast to disk. A side effect
of this, is that if the transaction doesn’t *directly* involve any
outputs we control, then it’ll linger around until a restart, when we
try to rebroadcast, and observe that it has bene rejected.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
54b31c1a35 wallet: remove conflicting double spend transactions on start up 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
50acc9cdf5 wtxmgr: add new method to allow callers to remove conflicted unmined txns
In this commit, we add a new method to the Store object that allows
callers to *manually* remove any conflicting transactions. At times,
it’s the case that while we were offline another transaction was
broadcast that double spends our own, or with the existence of RBF,
another replacement transaction was generated. In this case, when we
come back online, the tx will be rejected. Currently, we have no way of
removing such transaction sot avoid the retransmit-then-reject-dance.
This commit fixes that by adding RemoveUnminedTx.
2018-05-23 19:38:56 -07:00
Johan T. Halseth
8bcd56fc27 wallet/size test: add vsize test for non-segwit tx 2018-05-23 19:38:56 -07:00
Johan T. Halseth
d626036401 wallet/author: use vsize when estimating fees
This commit makes use of the recently added EstimateVirtualSize
method to estimated the size of a transaction when calculating
fees. This makes fee estimation more accurate when we are spending
segwit outputs, as before we wouldn't account for the witness
descount, resulting in overshooting fee estimates.
2018-05-23 19:38:56 -07:00
Johan T. Halseth
2caa8878b4 wallet/size test: add TestEstimateVirtualSize
This commit adds a test for EstimateVirtualSize. Currently three
test transactions are added:
    - 1 P2WPKH inputs + 2 P2PKH outputs
    - 1 P2SH-P2WPKH inputs + 2 P2PKH outputs
    - 1 P2WPKH inputs + 1 P2PKH output + 1 P2WPKH change output
2018-05-23 19:38:56 -07:00
Johan T. Halseth
f782f9dc68 wallet/size: add EstimateVirtualSize
This commit adds a new method EstimateVirtualSize that calculates
the worst case estimate vsize for a transaction with a given set
of inputs and outputs. This method is aware of P2PKH, P2WPKH and
P2SH-P2WPKH inputs, and caulculates the transaction vsize with
the witness data included.
2018-05-23 19:38:56 -07:00
Conner Fromknecht
2bd4130581 chain/bitcoind: connect to zmq in Start() to return error 2018-05-23 19:38:56 -07:00
Alex
2c34216319 build: update glide to latest btcd, neutrino, and gozmq 2018-05-23 19:38:56 -07:00
Alex
30d84da3e5 build+chain: replace pebbe/zmq4 with lightninglabs/gozmq (native Go) 2018-05-23 19:38:56 -07:00
Alex
a16bfd7775 chain/neutrino: fix data races due to not using pointer for rescan 2018-05-23 19:38:56 -07:00
Alex
78a69b4802 chain/neutrino: fix deadlock 2018-05-23 19:38:56 -07:00
Alex
73dbcf3943 multi: add bitcoind back-end (only for API use for now) 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
b963eb3ba4 wallet: pass in fee-per-kb into CreateSimpleTx 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
1cbee7c506 build: update to latest btcd and neutrino 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
aa42e17e2f rpc/legacyrpc: update method to use default fee rate 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
7b9d880fee wallet: remove internal relayFee in favor of passing in fee rate when sending
In this commit, we do away with the internal relayFee all together.
Instead, we’ll pass in the fee rate when we’re crafting any
transactions. This allows the caller to manually dictate their desired
fee rate.
2018-05-23 19:38:56 -07:00
Alex
60b23c144e dropwtxmgr: re-create namespace and set sync info to rescan for lnd 2018-05-23 19:38:56 -07:00