2015-12-01 19:44:58 +01:00
|
|
|
// Copyright (c) 2013-2015 The btcsuite developers
|
|
|
|
// Use of this source code is governed by an ISC
|
|
|
|
// license that can be found in the LICENSE file.
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
|
2015-04-02 20:13:38 +02:00
|
|
|
package wallet
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
|
|
|
|
import (
|
2017-07-12 01:13:10 +02:00
|
|
|
"bytes"
|
2018-11-06 02:06:55 +01:00
|
|
|
"fmt"
|
|
|
|
"time"
|
2017-07-12 01:13:10 +02:00
|
|
|
|
2018-11-20 03:13:23 +01:00
|
|
|
"github.com/btcsuite/btcd/chaincfg/chainhash"
|
2018-05-15 07:11:11 +02:00
|
|
|
"github.com/btcsuite/btcd/txscript"
|
2018-11-20 03:13:23 +01:00
|
|
|
"github.com/btcsuite/btcd/wire"
|
2018-05-15 07:11:11 +02:00
|
|
|
"github.com/btcsuite/btcwallet/chain"
|
|
|
|
"github.com/btcsuite/btcwallet/waddrmgr"
|
|
|
|
"github.com/btcsuite/btcwallet/walletdb"
|
|
|
|
"github.com/btcsuite/btcwallet/wtxmgr"
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
)
|
|
|
|
|
2018-11-20 03:13:23 +01:00
|
|
|
const (
|
|
|
|
// birthdayBlockDelta is the maximum time delta allowed between our
|
|
|
|
// birthday timestamp and our birthday block's timestamp when searching
|
|
|
|
// for a better birthday block candidate (if possible).
|
|
|
|
birthdayBlockDelta = 2 * time.Hour
|
|
|
|
)
|
|
|
|
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
func (w *Wallet) handleChainNotifications() {
|
2018-07-14 04:09:23 +02:00
|
|
|
defer w.wg.Done()
|
|
|
|
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
chainClient, err := w.requireChainClient()
|
|
|
|
if err != nil {
|
|
|
|
log.Errorf("handleChainNotifications called without RPC client")
|
|
|
|
return
|
|
|
|
}
|
|
|
|
|
2017-09-21 23:14:55 +02:00
|
|
|
catchUpHashes := func(w *Wallet, client chain.Interface,
|
|
|
|
height int32) error {
|
2021-03-24 14:43:24 +01:00
|
|
|
// TODO(aakselrod): There's a race condition here, which
|
2017-09-21 23:14:55 +02:00
|
|
|
// happens when a reorg occurs between the
|
|
|
|
// rescanProgress notification and the last GetBlockHash
|
|
|
|
// call. The solution when using btcd is to make btcd
|
|
|
|
// send blockconnected notifications with each block
|
|
|
|
// the way Neutrino does, and get rid of the loop. The
|
|
|
|
// other alternative is to check the final hash and,
|
|
|
|
// if it doesn't match the original hash returned by
|
|
|
|
// the notification, to roll back and restart the
|
|
|
|
// rescan.
|
|
|
|
log.Infof("Catching up block hashes to height %d, this"+
|
|
|
|
" might take a while", height)
|
|
|
|
err := walletdb.Update(w.db, func(tx walletdb.ReadWriteTx) error {
|
|
|
|
ns := tx.ReadWriteBucket(waddrmgrNamespaceKey)
|
2018-03-11 01:38:54 +01:00
|
|
|
|
2017-09-21 23:14:55 +02:00
|
|
|
startBlock := w.Manager.SyncedTo()
|
2018-03-11 01:38:54 +01:00
|
|
|
|
2017-09-21 23:14:55 +02:00
|
|
|
for i := startBlock.Height + 1; i <= height; i++ {
|
|
|
|
hash, err := client.GetBlockHash(int64(i))
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
2018-03-11 01:38:54 +01:00
|
|
|
header, err := chainClient.GetBlockHeader(hash)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
2017-09-21 23:14:55 +02:00
|
|
|
bs := waddrmgr.BlockStamp{
|
2018-03-11 01:38:54 +01:00
|
|
|
Height: i,
|
|
|
|
Hash: *hash,
|
|
|
|
Timestamp: header.Timestamp,
|
2017-09-21 23:14:55 +02:00
|
|
|
}
|
|
|
|
err = w.Manager.SetSyncedTo(ns, &bs)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return nil
|
|
|
|
})
|
|
|
|
if err != nil {
|
|
|
|
log.Errorf("Failed to update address manager "+
|
|
|
|
"sync state for height %d: %v", height, err)
|
|
|
|
}
|
2018-03-11 01:38:54 +01:00
|
|
|
|
2017-09-21 23:14:55 +02:00
|
|
|
log.Info("Done catching up block hashes")
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
2018-07-14 04:09:23 +02:00
|
|
|
for {
|
|
|
|
select {
|
|
|
|
case n, ok := <-chainClient.Notifications():
|
|
|
|
if !ok {
|
2018-07-04 03:59:17 +02:00
|
|
|
return
|
|
|
|
}
|
2018-07-14 04:09:23 +02:00
|
|
|
|
|
|
|
var notificationName string
|
|
|
|
var err error
|
|
|
|
switch n := n.(type) {
|
|
|
|
case chain.ClientConnected:
|
2018-11-06 02:06:55 +01:00
|
|
|
// Before attempting to sync with our backend,
|
|
|
|
// we'll make sure that our birthday block has
|
|
|
|
// been set correctly to potentially prevent
|
|
|
|
// missing relevant events.
|
2018-11-20 03:13:23 +01:00
|
|
|
birthdayStore := &walletBirthdayStore{
|
|
|
|
db: w.db,
|
|
|
|
manager: w.Manager,
|
|
|
|
}
|
|
|
|
birthdayBlock, err := birthdaySanityCheck(
|
|
|
|
chainClient, birthdayStore,
|
|
|
|
)
|
2018-11-20 22:00:45 +01:00
|
|
|
if err != nil && !waddrmgr.IsError(err, waddrmgr.ErrBirthdayBlockNotSet) {
|
2021-03-24 14:43:24 +01:00
|
|
|
panic(fmt.Errorf("unable to sanity "+
|
2018-11-06 02:06:55 +01:00
|
|
|
"check wallet birthday block: %v",
|
2019-05-14 22:16:58 +02:00
|
|
|
err))
|
2018-11-06 02:06:55 +01:00
|
|
|
}
|
|
|
|
|
2019-05-14 22:16:58 +02:00
|
|
|
err = w.syncWithChain(birthdayBlock)
|
|
|
|
if err != nil && !w.ShuttingDown() {
|
2021-03-24 14:43:24 +01:00
|
|
|
panic(fmt.Errorf("unable to synchronize "+
|
2019-05-14 22:16:58 +02:00
|
|
|
"wallet to chain: %v", err))
|
|
|
|
}
|
2018-07-14 04:09:23 +02:00
|
|
|
case chain.BlockConnected:
|
|
|
|
err = walletdb.Update(w.db, func(tx walletdb.ReadWriteTx) error {
|
|
|
|
return w.connectBlock(tx, wtxmgr.BlockMeta(n))
|
|
|
|
})
|
2019-05-21 01:29:41 +02:00
|
|
|
notificationName = "block connected"
|
2018-07-14 04:09:23 +02:00
|
|
|
case chain.BlockDisconnected:
|
|
|
|
err = walletdb.Update(w.db, func(tx walletdb.ReadWriteTx) error {
|
|
|
|
return w.disconnectBlock(tx, wtxmgr.BlockMeta(n))
|
|
|
|
})
|
2019-05-21 01:29:41 +02:00
|
|
|
notificationName = "block disconnected"
|
2018-07-14 04:09:23 +02:00
|
|
|
case chain.RelevantTx:
|
|
|
|
err = walletdb.Update(w.db, func(tx walletdb.ReadWriteTx) error {
|
|
|
|
return w.addRelevantTx(tx, n.TxRecord, n.Block)
|
|
|
|
})
|
2019-05-21 01:29:41 +02:00
|
|
|
notificationName = "relevant transaction"
|
2018-07-14 04:09:23 +02:00
|
|
|
case chain.FilteredBlockConnected:
|
|
|
|
// Atomically update for the whole block.
|
|
|
|
if len(n.RelevantTxs) > 0 {
|
|
|
|
err = walletdb.Update(w.db, func(
|
|
|
|
tx walletdb.ReadWriteTx) error {
|
|
|
|
var err error
|
|
|
|
for _, rec := range n.RelevantTxs {
|
|
|
|
err = w.addRelevantTx(tx, rec,
|
|
|
|
n.Block)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return nil
|
|
|
|
})
|
|
|
|
}
|
2019-05-21 01:29:41 +02:00
|
|
|
notificationName = "filtered block connected"
|
2018-07-14 04:09:23 +02:00
|
|
|
|
|
|
|
// The following require some database maintenance, but also
|
|
|
|
// need to be reported to the wallet's rescan goroutine.
|
|
|
|
case *chain.RescanProgress:
|
|
|
|
err = catchUpHashes(w, chainClient, n.Height)
|
2019-05-21 01:29:41 +02:00
|
|
|
notificationName = "rescan progress"
|
2018-07-14 04:09:23 +02:00
|
|
|
select {
|
|
|
|
case w.rescanNotifications <- n:
|
|
|
|
case <-w.quitChan():
|
|
|
|
return
|
|
|
|
}
|
|
|
|
case *chain.RescanFinished:
|
|
|
|
err = catchUpHashes(w, chainClient, n.Height)
|
2019-05-21 01:29:41 +02:00
|
|
|
notificationName = "rescan finished"
|
2018-07-14 04:09:23 +02:00
|
|
|
w.SetChainSynced(true)
|
|
|
|
select {
|
|
|
|
case w.rescanNotifications <- n:
|
|
|
|
case <-w.quitChan():
|
|
|
|
return
|
|
|
|
}
|
2018-07-04 03:59:17 +02:00
|
|
|
}
|
2018-07-14 04:09:23 +02:00
|
|
|
if err != nil {
|
2019-05-21 01:29:41 +02:00
|
|
|
// If we received a block connected notification
|
|
|
|
// while rescanning, then we can ignore logging
|
|
|
|
// the error as we'll properly catch up once we
|
|
|
|
// process the RescanFinished notification.
|
|
|
|
if notificationName == "block connected" &&
|
|
|
|
waddrmgr.IsError(err, waddrmgr.ErrBlockNotFound) &&
|
|
|
|
!w.ChainSynced() {
|
|
|
|
|
|
|
|
log.Debugf("Received block connected "+
|
|
|
|
"notification for height %v "+
|
|
|
|
"while rescanning",
|
|
|
|
n.(chain.BlockConnected).Height)
|
|
|
|
continue
|
2018-07-14 04:09:23 +02:00
|
|
|
}
|
2019-05-21 01:29:41 +02:00
|
|
|
|
|
|
|
log.Errorf("Unable to process chain backend "+
|
|
|
|
"%v notification: %v", notificationName,
|
|
|
|
err)
|
2018-03-14 01:36:23 +01:00
|
|
|
}
|
2018-07-14 04:09:23 +02:00
|
|
|
case <-w.quit:
|
|
|
|
return
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// connectBlock handles a chain server notification by marking a wallet
|
|
|
|
// that's currently in-sync with the chain server as being synced up to
|
|
|
|
// the passed block.
|
2017-01-17 01:19:02 +01:00
|
|
|
func (w *Wallet) connectBlock(dbtx walletdb.ReadWriteTx, b wtxmgr.BlockMeta) error {
|
|
|
|
addrmgrNs := dbtx.ReadWriteBucket(waddrmgrNamespaceKey)
|
|
|
|
|
2015-06-18 18:29:13 +02:00
|
|
|
bs := waddrmgr.BlockStamp{
|
2018-03-06 19:09:20 +01:00
|
|
|
Height: b.Height,
|
|
|
|
Hash: b.Hash,
|
|
|
|
Timestamp: b.Time,
|
2015-06-18 18:29:13 +02:00
|
|
|
}
|
2017-01-17 01:19:02 +01:00
|
|
|
err := w.Manager.SetSyncedTo(addrmgrNs, &bs)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
2014-10-29 07:43:29 +01:00
|
|
|
}
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
// Notify interested clients of the connected block.
|
2017-01-17 01:19:02 +01:00
|
|
|
//
|
|
|
|
// TODO: move all notifications outside of the database transaction.
|
|
|
|
w.NtfnServer.notifyAttachedBlock(dbtx, &b)
|
|
|
|
return nil
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// disconnectBlock handles a chain server reorganize by rolling back all
|
|
|
|
// block history from the reorged block for a wallet in-sync with the chain
|
|
|
|
// server.
|
2017-01-17 01:19:02 +01:00
|
|
|
func (w *Wallet) disconnectBlock(dbtx walletdb.ReadWriteTx, b wtxmgr.BlockMeta) error {
|
|
|
|
addrmgrNs := dbtx.ReadWriteBucket(waddrmgrNamespaceKey)
|
|
|
|
txmgrNs := dbtx.ReadWriteBucket(wtxmgrNamespaceKey)
|
|
|
|
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
if !w.ChainSynced() {
|
2015-04-17 22:12:04 +02:00
|
|
|
return nil
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
2017-07-12 01:13:10 +02:00
|
|
|
// Disconnect the removed block and all blocks after it if we know about
|
|
|
|
// the disconnected block. Otherwise, the block is in the future.
|
|
|
|
if b.Height <= w.Manager.SyncedTo().Height {
|
|
|
|
hash, err := w.Manager.BlockHash(addrmgrNs, b.Height)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
if bytes.Equal(hash[:], b.Hash[:]) {
|
|
|
|
bs := waddrmgr.BlockStamp{
|
|
|
|
Height: b.Height - 1,
|
|
|
|
}
|
|
|
|
hash, err = w.Manager.BlockHash(addrmgrNs, bs.Height)
|
2015-04-17 22:12:04 +02:00
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
2017-07-12 01:13:10 +02:00
|
|
|
b.Hash = *hash
|
2018-03-06 19:09:20 +01:00
|
|
|
|
|
|
|
client := w.ChainClient()
|
|
|
|
header, err := client.GetBlockHeader(hash)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
bs.Timestamp = header.Timestamp
|
2017-07-12 01:13:10 +02:00
|
|
|
err = w.Manager.SetSyncedTo(addrmgrNs, &bs)
|
2015-05-14 20:00:07 +02:00
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
2018-08-14 03:46:05 +02:00
|
|
|
|
2017-07-12 01:13:10 +02:00
|
|
|
err = w.TxStore.Rollback(txmgrNs, b.Height)
|
2018-08-14 03:46:05 +02:00
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
// Notify interested clients of the disconnected block.
|
|
|
|
w.NtfnServer.notifyDetachedBlock(&b.Hash)
|
|
|
|
|
2015-04-17 22:12:04 +02:00
|
|
|
return nil
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
2017-01-17 01:19:02 +01:00
|
|
|
func (w *Wallet) addRelevantTx(dbtx walletdb.ReadWriteTx, rec *wtxmgr.TxRecord, block *wtxmgr.BlockMeta) error {
|
|
|
|
addrmgrNs := dbtx.ReadWriteBucket(waddrmgrNamespaceKey)
|
|
|
|
txmgrNs := dbtx.ReadWriteBucket(wtxmgrNamespaceKey)
|
2015-04-06 21:03:24 +02:00
|
|
|
|
|
|
|
// At the moment all notified transactions are assumed to actually be
|
|
|
|
// relevant. This assumption will not hold true when SPV support is
|
|
|
|
// added, but until then, simply insert the transaction because there
|
|
|
|
// should either be one or more relevant inputs or outputs.
|
2021-06-04 08:07:24 +02:00
|
|
|
exists, err := w.TxStore.InsertTxCheckIfExists(txmgrNs, rec, block)
|
2015-04-06 21:03:24 +02:00
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
2021-06-04 08:07:24 +02:00
|
|
|
// If the transaction has already been recorded, we can return early.
|
|
|
|
// Note: Returning here is safe as we're within the context of an atomic
|
|
|
|
// database transaction, so we don't need to worry about the MarkUsed
|
|
|
|
// calls below.
|
|
|
|
if exists {
|
|
|
|
return nil
|
|
|
|
}
|
|
|
|
|
2015-04-06 21:03:24 +02:00
|
|
|
// Check every output to determine whether it is controlled by a wallet
|
|
|
|
// key. If so, mark the output as a credit.
|
|
|
|
for i, output := range rec.MsgTx.TxOut {
|
|
|
|
_, addrs, _, err := txscript.ExtractPkScriptAddrs(output.PkScript,
|
2015-04-02 20:13:38 +02:00
|
|
|
w.chainParams)
|
2015-04-06 21:03:24 +02:00
|
|
|
if err != nil {
|
|
|
|
// Non-standard outputs are skipped.
|
|
|
|
continue
|
|
|
|
}
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
for _, addr := range addrs {
|
2017-01-17 01:19:02 +01:00
|
|
|
ma, err := w.Manager.Address(addrmgrNs, addr)
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
if err == nil {
|
2015-04-06 21:03:24 +02:00
|
|
|
// TODO: Credits should be added with the
|
|
|
|
// account they belong to, so wtxmgr is able to
|
|
|
|
// track per-account balances.
|
2017-01-17 01:19:02 +01:00
|
|
|
err = w.TxStore.AddCredit(txmgrNs, rec, block, uint32(i),
|
2015-04-06 21:03:24 +02:00
|
|
|
ma.Internal())
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
2017-01-17 01:19:02 +01:00
|
|
|
err = w.Manager.MarkUsed(addrmgrNs, addr)
|
2015-04-06 21:03:24 +02:00
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
log.Debugf("Marked address %v used", addr)
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
continue
|
|
|
|
}
|
2015-04-06 21:03:24 +02:00
|
|
|
|
|
|
|
// Missing addresses are skipped. Other errors should
|
|
|
|
// be propagated.
|
2015-05-28 00:21:17 +02:00
|
|
|
if !waddrmgr.IsError(err, waddrmgr.ErrAddressNotFound) {
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
return err
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
// Send notification of mined or unmined transaction to any interested
|
|
|
|
// clients.
|
|
|
|
//
|
|
|
|
// TODO: Avoid the extra db hits.
|
|
|
|
if block == nil {
|
2017-01-17 01:19:02 +01:00
|
|
|
details, err := w.TxStore.UniqueTxDetails(txmgrNs, &rec.Hash, nil)
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
if err != nil {
|
2018-02-14 06:36:39 +01:00
|
|
|
log.Errorf("Cannot query transaction details for notification: %v", err)
|
2018-09-07 04:42:13 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// It's possible that the transaction was not found within the
|
|
|
|
// wallet's set of unconfirmed transactions due to it already
|
|
|
|
// being confirmed, so we'll avoid notifying it.
|
|
|
|
//
|
|
|
|
// TODO(wilmer): ideally we should find the culprit to why we're
|
|
|
|
// receiving an additional unconfirmed chain.RelevantTx
|
|
|
|
// notification from the chain backend.
|
|
|
|
if details != nil {
|
2017-01-17 01:19:02 +01:00
|
|
|
w.NtfnServer.notifyUnminedTransaction(dbtx, details)
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
}
|
|
|
|
} else {
|
2017-01-17 01:19:02 +01:00
|
|
|
details, err := w.TxStore.UniqueTxDetails(txmgrNs, &rec.Hash, &block.Block)
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
if err != nil {
|
2018-02-14 06:36:39 +01:00
|
|
|
log.Errorf("Cannot query transaction details for notification: %v", err)
|
2018-09-07 04:42:13 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// We'll only notify the transaction if it was found within the
|
|
|
|
// wallet's set of confirmed transactions.
|
|
|
|
if details != nil {
|
2017-01-17 01:19:02 +01:00
|
|
|
w.NtfnServer.notifyMinedTransaction(dbtx, details, block)
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
}
|
|
|
|
}
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
|
|
|
|
return nil
|
|
|
|
}
|
2018-11-06 02:06:55 +01:00
|
|
|
|
2018-11-20 03:13:23 +01:00
|
|
|
// chainConn is an interface that abstracts the chain connection logic required
|
|
|
|
// to perform a wallet's birthday block sanity check.
|
|
|
|
type chainConn interface {
|
|
|
|
// GetBestBlock returns the hash and height of the best block known to
|
|
|
|
// the backend.
|
|
|
|
GetBestBlock() (*chainhash.Hash, int32, error)
|
|
|
|
|
|
|
|
// GetBlockHash returns the hash of the block with the given height.
|
|
|
|
GetBlockHash(int64) (*chainhash.Hash, error)
|
|
|
|
|
|
|
|
// GetBlockHeader returns the header for the block with the given hash.
|
|
|
|
GetBlockHeader(*chainhash.Hash) (*wire.BlockHeader, error)
|
|
|
|
}
|
|
|
|
|
|
|
|
// birthdayStore is an interface that abstracts the wallet's sync-related
|
|
|
|
// information required to perform a birthday block sanity check.
|
|
|
|
type birthdayStore interface {
|
|
|
|
// Birthday returns the birthday timestamp of the wallet.
|
|
|
|
Birthday() time.Time
|
|
|
|
|
|
|
|
// BirthdayBlock returns the birthday block of the wallet. The boolean
|
|
|
|
// returned should signal whether the wallet has already verified the
|
|
|
|
// correctness of its birthday block.
|
|
|
|
BirthdayBlock() (waddrmgr.BlockStamp, bool, error)
|
|
|
|
|
|
|
|
// SetBirthdayBlock updates the birthday block of the wallet to the
|
|
|
|
// given block. The boolean can be used to signal whether this block
|
|
|
|
// should be sanity checked the next time the wallet starts.
|
|
|
|
//
|
|
|
|
// NOTE: This should also set the wallet's synced tip to reflect the new
|
|
|
|
// birthday block. This will allow the wallet to rescan from this point
|
|
|
|
// to detect any potentially missed events.
|
|
|
|
SetBirthdayBlock(waddrmgr.BlockStamp) error
|
|
|
|
}
|
|
|
|
|
|
|
|
// walletBirthdayStore is a wrapper around the wallet's database and address
|
|
|
|
// manager that satisfies the birthdayStore interface.
|
|
|
|
type walletBirthdayStore struct {
|
|
|
|
db walletdb.DB
|
|
|
|
manager *waddrmgr.Manager
|
|
|
|
}
|
|
|
|
|
|
|
|
var _ birthdayStore = (*walletBirthdayStore)(nil)
|
2018-11-06 02:06:55 +01:00
|
|
|
|
2018-11-20 03:13:23 +01:00
|
|
|
// Birthday returns the birthday timestamp of the wallet.
|
|
|
|
func (s *walletBirthdayStore) Birthday() time.Time {
|
|
|
|
return s.manager.Birthday()
|
|
|
|
}
|
|
|
|
|
|
|
|
// BirthdayBlock returns the birthday block of the wallet.
|
|
|
|
func (s *walletBirthdayStore) BirthdayBlock() (waddrmgr.BlockStamp, bool, error) {
|
2018-11-16 02:59:45 +01:00
|
|
|
var (
|
|
|
|
birthdayBlock waddrmgr.BlockStamp
|
|
|
|
birthdayBlockVerified bool
|
|
|
|
)
|
2018-11-20 03:13:23 +01:00
|
|
|
|
|
|
|
err := walletdb.View(s.db, func(tx walletdb.ReadTx) error {
|
2018-11-06 02:06:55 +01:00
|
|
|
var err error
|
|
|
|
ns := tx.ReadBucket(waddrmgrNamespaceKey)
|
2018-11-20 03:13:23 +01:00
|
|
|
birthdayBlock, birthdayBlockVerified, err = s.manager.BirthdayBlock(ns)
|
2018-11-06 02:06:55 +01:00
|
|
|
return err
|
|
|
|
})
|
|
|
|
|
2018-11-20 03:13:23 +01:00
|
|
|
return birthdayBlock, birthdayBlockVerified, err
|
|
|
|
}
|
|
|
|
|
|
|
|
// SetBirthdayBlock updates the birthday block of the wallet to the
|
|
|
|
// given block. The boolean can be used to signal whether this block
|
|
|
|
// should be sanity checked the next time the wallet starts.
|
|
|
|
//
|
|
|
|
// NOTE: This should also set the wallet's synced tip to reflect the new
|
|
|
|
// birthday block. This will allow the wallet to rescan from this point
|
|
|
|
// to detect any potentially missed events.
|
|
|
|
func (s *walletBirthdayStore) SetBirthdayBlock(block waddrmgr.BlockStamp) error {
|
|
|
|
return walletdb.Update(s.db, func(tx walletdb.ReadWriteTx) error {
|
|
|
|
ns := tx.ReadWriteBucket(waddrmgrNamespaceKey)
|
|
|
|
err := s.manager.SetBirthdayBlock(ns, block, true)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
return s.manager.SetSyncedTo(ns, &block)
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
|
|
|
// birthdaySanityCheck is a helper function that ensures a birthday block
|
|
|
|
// correctly reflects the birthday timestamp within a reasonable timestamp
|
|
|
|
// delta. It's intended to be run after the wallet establishes its connection
|
|
|
|
// with the backend, but before it begins syncing. This is done as the second
|
|
|
|
// part to the wallet's address manager migration where we populate the birthday
|
|
|
|
// block to ensure we do not miss any relevant events throughout rescans.
|
2018-11-20 22:00:45 +01:00
|
|
|
// waddrmgr.ErrBirthdayBlockNotSet is returned if the birthday block has not
|
|
|
|
// been set yet.
|
2018-11-20 03:13:23 +01:00
|
|
|
func birthdaySanityCheck(chainConn chainConn,
|
|
|
|
birthdayStore birthdayStore) (*waddrmgr.BlockStamp, error) {
|
|
|
|
|
|
|
|
// We'll start by fetching our wallet's birthday timestamp and block.
|
|
|
|
birthdayTimestamp := birthdayStore.Birthday()
|
|
|
|
birthdayBlock, birthdayBlockVerified, err := birthdayStore.BirthdayBlock()
|
2018-11-20 22:00:45 +01:00
|
|
|
if err != nil {
|
2018-11-06 02:06:55 +01:00
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
2018-11-16 02:59:45 +01:00
|
|
|
// If the birthday block has already been verified to be correct, we can
|
|
|
|
// exit our sanity check to prevent potentially fetching a better
|
|
|
|
// candidate.
|
|
|
|
if birthdayBlockVerified {
|
|
|
|
log.Debugf("Birthday block has already been verified: "+
|
|
|
|
"height=%d, hash=%v", birthdayBlock.Height,
|
|
|
|
birthdayBlock.Hash)
|
|
|
|
|
|
|
|
return &birthdayBlock, nil
|
|
|
|
}
|
|
|
|
|
2019-05-21 01:28:41 +02:00
|
|
|
// Otherwise, we'll attempt to locate a better one now that we have
|
|
|
|
// access to the chain.
|
|
|
|
newBirthdayBlock, err := locateBirthdayBlock(chainConn, birthdayTimestamp)
|
2018-11-06 02:06:55 +01:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
2019-05-21 01:28:41 +02:00
|
|
|
if err := birthdayStore.SetBirthdayBlock(*newBirthdayBlock); err != nil {
|
2018-11-06 02:06:55 +01:00
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
2019-05-21 01:28:41 +02:00
|
|
|
return newBirthdayBlock, nil
|
2018-11-06 02:06:55 +01:00
|
|
|
}
|