Commit graph

963 commits

Author SHA1 Message Date
Olaoluwa Osuntokun
27dfed7f27 waddrmgr: fix deadlock bug by using read mutex when necessary
In this commit, we fix a deadlock bug that was introduced recently.
This can happen when ForEachActiveAccountAddress or
ForEachActiveAddress is called, as these internally need to grab the
mutex of the manager (within the scoped manager) in order to check if
the manager is locked or not.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
d42a45f16f waddrmgr: add database migration from version 4 to 5
In this commit, we add a database migration from version 4 to version 5.
We also take this opportunity to clean up the old migration code. This
is no longer needed as wallets very old can simply go back in the prior
git history to migrate to version 4, then go from there to version 5.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
b75bf1426e wallet: update wallet to be aware of new KeyScopes 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
115d7d17f2 rpc: update rpc servers to default to waddrmgr.KeyScopeBIP0044 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
3ca68d8259 waddrmgr: add additional tests to exercise new scope specific features 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
b8104ef86e waddrmgr: update existing tests to be aware of new key scopes 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
9a8da416ef waddrmgr: update all addresses to be aware of new addr scopes 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
641efbbd31 waddrmgr: extend the set of AddressType enum values with script+rawkey 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
a75170c650 waddrmgr: update the Manager struct to remove functionality covered by ScopedManager
In this commit, we remove many of the methods in the Manager struct as
they’ve now be been replicated within a scoped format for each of the
ScopedKeyManagers.

A major change is that we’ll now actually store the master HD private
and public keys. This required as in order to create new scopes, we
need access to the master HD private key as hardened derivation is
required in accordance with BIP43.

The initial creation of the manager namespaces has also been extended
to create the namespaces and keys for the set of default key scopes.

Finally, a series of utility method has been added to allow callers to
create ScopedKeyManagers for arbitrary sets of scopes.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
e3afc8be6a waddrmgr: create new ScopedKeyManager to manage BIP43 purposes
In this commit, we create a new struct that houses the key derivation,
address management, and account management for a particular scope. A
scope consists of a (purpose, cointype) tuple. Additionally, each
ScopedKeyManager is able to generate internal/external addresses for a
specific *address type*. This make rescans easier as for each scope, we
know what type of output to look for within the chain.

The ScopedKeyManagers have two new primary methods that weren’t
previously present within the regular Manager:
   * DeriveFromKeyPath
   * NewRawAccount

These two methods allow callers a greater degree of control over the
way that accounted are created and addressed derived.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
f5de14f16b waddrmgr: update database version to 5 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
9f71d472a3 waddrmgr: extend createManagerNS to also create name spaces for each scope 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
57cf4274ee waddrmgr: extend deletePrivateKeys to delete HD key and scope keys 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
946ca2da1e waddrmgr: add ability to store encrypt master HD priv/pub keys
In this commit, we create new key spaces to allow users to store the
encrypted master priv/pub keys. This is required as in order to create
new scopes, we must do hardened derivation from the root key.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
e8755c2bc2 waddrmgr: introduce new bucket hierarchy with scopes at the top
In this commit, we make a fundamental modification bucket structure
within the database. Most buckets are no under an additional layer of
nesting: the scope. The scope encapsulates which (purpose, coin type)
pair the address, accounts, and coin type keys belong to.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
8bd9f7713b waddrmgr: remove direct references to BIP 44 in struct/variable names
In this commit, we remove all direct references to BIP 44 as upcoming
changes will shift to a model that is no longer directly dependent on
BIP 44 in favor of restoring a layer of abstraction and allowing users
to manage multiple (purpose, coin type) scopes within the same
database.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
ea82d184bf waddrmgr: remove additional addressType values
In this commit, we remove the extra addressType values as this will no
longer be needed due to the new abstraction of scoped key managers.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
6f6abef1d6 waddrmgr: introduce concept of scopes for key managers
In this commit, we introduce the concept of scopes for individual key
managers. Each scope will lock down a key manager to a particular
purpose and coin type within the BIP0043 hierarchy. Each scope will
also have a set address type schema. This schema will be consulted when
creating addresses for a particular scoped key manager.

Finally, we introduce 3 new default scopes:
   * BIP 44
   * BIP 84
   * BIP 49++ (BIP49 but uses p2wkh for change addresses)
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
f4ef7cdd29 waddrmgr: store block timestamp in BlockStamp 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
ab4ccacbb9 wallet: extend PublishTransaction to be a general reliable broadcaster
In this commit, we extend the PublishTransction method to be a more
general semi reliable transaction broadcast mechanism. We do this by
removing the special casing for neutrino. With this change, we’ll
_always_ write any transactions to be broadcast to disk. A side effect
of this, is that if the transaction doesn’t *directly* involve any
outputs we control, then it’ll linger around until a restart, when we
try to rebroadcast, and observe that it has bene rejected.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
54b31c1a35 wallet: remove conflicting double spend transactions on start up 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
50acc9cdf5 wtxmgr: add new method to allow callers to remove conflicted unmined txns
In this commit, we add a new method to the Store object that allows
callers to *manually* remove any conflicting transactions. At times,
it’s the case that while we were offline another transaction was
broadcast that double spends our own, or with the existence of RBF,
another replacement transaction was generated. In this case, when we
come back online, the tx will be rejected. Currently, we have no way of
removing such transaction sot avoid the retransmit-then-reject-dance.
This commit fixes that by adding RemoveUnminedTx.
2018-05-23 19:38:56 -07:00
Johan T. Halseth
8bcd56fc27 wallet/size test: add vsize test for non-segwit tx 2018-05-23 19:38:56 -07:00
Johan T. Halseth
d626036401 wallet/author: use vsize when estimating fees
This commit makes use of the recently added EstimateVirtualSize
method to estimated the size of a transaction when calculating
fees. This makes fee estimation more accurate when we are spending
segwit outputs, as before we wouldn't account for the witness
descount, resulting in overshooting fee estimates.
2018-05-23 19:38:56 -07:00
Johan T. Halseth
2caa8878b4 wallet/size test: add TestEstimateVirtualSize
This commit adds a test for EstimateVirtualSize. Currently three
test transactions are added:
    - 1 P2WPKH inputs + 2 P2PKH outputs
    - 1 P2SH-P2WPKH inputs + 2 P2PKH outputs
    - 1 P2WPKH inputs + 1 P2PKH output + 1 P2WPKH change output
2018-05-23 19:38:56 -07:00
Johan T. Halseth
f782f9dc68 wallet/size: add EstimateVirtualSize
This commit adds a new method EstimateVirtualSize that calculates
the worst case estimate vsize for a transaction with a given set
of inputs and outputs. This method is aware of P2PKH, P2WPKH and
P2SH-P2WPKH inputs, and caulculates the transaction vsize with
the witness data included.
2018-05-23 19:38:56 -07:00
Conner Fromknecht
2bd4130581 chain/bitcoind: connect to zmq in Start() to return error 2018-05-23 19:38:56 -07:00
Alex
2c34216319 build: update glide to latest btcd, neutrino, and gozmq 2018-05-23 19:38:56 -07:00
Alex
30d84da3e5 build+chain: replace pebbe/zmq4 with lightninglabs/gozmq (native Go) 2018-05-23 19:38:56 -07:00
Alex
a16bfd7775 chain/neutrino: fix data races due to not using pointer for rescan 2018-05-23 19:38:56 -07:00
Alex
78a69b4802 chain/neutrino: fix deadlock 2018-05-23 19:38:56 -07:00
Alex
73dbcf3943 multi: add bitcoind back-end (only for API use for now) 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
b963eb3ba4 wallet: pass in fee-per-kb into CreateSimpleTx 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
1cbee7c506 build: update to latest btcd and neutrino 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
aa42e17e2f rpc/legacyrpc: update method to use default fee rate 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
7b9d880fee wallet: remove internal relayFee in favor of passing in fee rate when sending
In this commit, we do away with the internal relayFee all together.
Instead, we’ll pass in the fee rate when we’re crafting any
transactions. This allows the caller to manually dictate their desired
fee rate.
2018-05-23 19:38:56 -07:00
Alex
60b23c144e dropwtxmgr: re-create namespace and set sync info to rescan for lnd 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
9988a4d500 build: update to latest btcd+btcutil+neutrino 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
f4d3cc9351 build: update glide files for latest btcutil, btcd, and neutrino 2018-05-23 19:38:56 -07:00
Johan T. Halseth
255f23679f wallet: close db on Open() failure.
This commit makes sure the wallet db is closed if the call to
open the wallet fails, as subsequent calls to OpenExistingWallet
would fail to open the already open database.
2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
d59a767ab8 build: update glide files to point to latest neutrino+btcutil 2018-05-23 19:38:56 -07:00
Olaoluwa Osuntokun
ee841b07cf wallet: for ineutrino back-end insert all broadcast transactions into txstore 2018-05-23 19:38:56 -07:00
Alex
436aa312d2 build: update glide for new neutrino version 2018-05-23 19:38:56 -07:00
Alex
d7ad221ee0 chain: ensure clientMtx is never locked while waiting on channel 2018-05-23 19:38:56 -07:00
Alex
5c69110f47 wallet: ensure rescan is launched with all addresses
During the time of initial block hash catch-up, it is possible to
request an address be generated. This commit updates the active
addresses by calling `w.activeData` after the catch-up is complete.
2018-05-23 19:38:56 -07:00
Alex
5113a64617 wallet: fix nil dereference in rescanprogress ntfn handler 2018-05-23 19:38:56 -07:00
Alex
8e2c741f88 wallet: move DB access from rescan ntfns into correct goroutine 2018-05-23 19:38:56 -07:00
Alex
81a9bb67c1 wallet: set wallet birthday properly in rescans and manager 2018-05-23 19:38:56 -07:00
Alex
1b6d7a7ee3 chain: add GetBlockHeader support
This commit allows neutrino to simulate the GetBlockHeader RPC, and
allows chain client code to use the call with both btcd/RPC back end
and the neutrino back end.
2018-05-23 19:38:56 -07:00
Alex
e06434ed75 chain: add StartTime support to neutrino client
This commit adds support for using StartTime in a rescan in
neutrino. The NeutrinoClient instance can have a birthday set
such that all underlying neutrino rescans are called with that
start time.
2018-05-23 19:38:56 -07:00