lbcwallet/btcwallet.go

172 lines
5.2 KiB
Go
Raw Normal View History

Remove account support, fix races on btcd connect. This commit is the result of several big changes being made to the wallet. In particular, the "handshake" (initial sync to the chain server) was quite racy and required proper synchronization. To make fixing this race easier, several other changes were made to the internal wallet data structures and much of the RPC server ended up being rewritten. First, all account support has been removed. The previous Account struct has been replaced with a Wallet structure, which includes a keystore for saving keys, and a txstore for storing relevant transactions. This decision has been made since it is the opinion of myself and other developers that bitcoind accounts are fundamentally broken (as accounts implemented by bitcoind support both arbitrary address groupings as well as moving balances between accounts -- these are fundamentally incompatible features), and since a BIP0032 keystore is soon planned to be implemented (at which point, "accounts" can return as HD extended keys). With the keystore handling the grouping of related keys, there is no reason have many different Account structs, and the AccountManager has been removed as well. All RPC handlers that take an account option will only work with "" (the default account) or "*" if the RPC allows specifying all accounts. Second, much of the RPC server has been cleaned up. The global variables for the RPC server and chain server client have been moved to part of the rpcServer struct, and the handlers for each RPC method that are looked up change depending on which components have been set. Passthrough requests are also no longer handled specially, but when the chain server is set, a handler to perform the passthrough will be returned if the method is not otherwise a wallet RPC. The notification system for websocket clients has also been rewritten so wallet components can send notifications through channels, rather than requiring direct access to the RPC server itself, or worse still, sending directly to a websocket client's send channel. In the future, this will enable proper registration of notifications, rather than unsolicited broadcasts to every connected websocket client (see issue #84). Finally, and the main reason why much of this cleanup was necessary, the races during intial sync with the chain server have been fixed. Previously, when the 'Handshake' was run, a rescan would occur which would perform modifications to Account data structures as notifications were received. Synchronization was provided with a single binary semaphore which serialized all access to wallet and account data. However, the Handshake itself was not able to run with this lock (or else notifications would block), and many data races would occur as both notifications were being handled. If GOMAXPROCS was ever increased beyond 1, btcwallet would always immediately crash due to invalid addresses caused by the data races on startup. To fix this, the single lock for all wallet access has been replaced with mutexes for both the keystore and txstore. Handling of btcd notifications and client requests may now occur simultaneously. GOMAXPROCS has also been set to the number of logical CPUs at the beginning of main, since with the data races fixed, there's no reason to prevent the extra parallelism gained by increasing it. Closes #78. Closes #101. Closes #110.
2014-07-09 05:17:38 +02:00
/*
* Copyright (c) 2013, 2014 Conformal Systems LLC <info@conformal.com>
*
* Permission to use, copy, modify, and distribute this software for any
* purpose with or without fee is hereby granted, provided that the above
* copyright notice and this permission notice appear in all copies.
*
* THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
* WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
* ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
* WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
* ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
* OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
*/
package main
import (
"io/ioutil"
"net"
"net/http"
_ "net/http/pprof"
"os"
"runtime"
"github.com/conformal/btcwallet/chain"
)
var (
cfg *config
shutdownChan = make(chan struct{})
)
func main() {
// Use all processor cores.
runtime.GOMAXPROCS(runtime.NumCPU())
// Work around defer not working after os.Exit.
if err := walletMain(); err != nil {
os.Exit(1)
}
}
// walletMain is a work-around main function that is required since deferred
// functions (such as log flushing) are not called with calls to os.Exit.
// Instead, main runs this function and checks for a non-nil error, at which
// point any defers have already run, and if the error is non-nil, the program
// can be exited with an error exit status.
func walletMain() error {
// Load configuration and parse command line. This function also
// initializes logging and configures it accordingly.
tcfg, _, err := loadConfig()
if err != nil {
return err
}
cfg = tcfg
defer backendLog.Flush()
if cfg.Profile != "" {
go func() {
listenAddr := net.JoinHostPort("", cfg.Profile)
log.Infof("Profile server listening on %s", listenAddr)
profileRedirect := http.RedirectHandler("/debug/pprof",
http.StatusSeeOther)
http.Handle("/", profileRedirect)
log.Errorf("%v", http.ListenAndServe(listenAddr, nil))
}()
}
// Create and start HTTP server to serve wallet client connections.
// This will be updated with the wallet and chain server RPC client
// created below after each is created.
server, err := newRPCServer(cfg.SvrListeners, cfg.RPCMaxClients,
cfg.RPCMaxWebsockets)
if err != nil {
log.Errorf("Unable to create HTTP server: %v", err)
return err
}
server.Start()
// Shutdown the server if an interrupt signal is received.
addInterruptHandler(server.Stop)
// Create channel so that the goroutine which opens the chain server
// connection can pass the conn to the goroutine which opens the wallet.
// Buffer the channel so sends are not blocked, since if the wallet is
// not yet created, the wallet open goroutine does not read this.
chainSvrChan := make(chan *chain.Client, 1)
go func() {
// Read CA certs and create the RPC client.
var certs []byte
if !cfg.DisableClientTLS {
certs, err = ioutil.ReadFile(cfg.CAFile)
if err != nil {
log.Warnf("Cannot open CA file: %v", err)
// If there's an error reading the CA file, continue
// with nil certs and without the client connection
certs = nil
}
Remove account support, fix races on btcd connect. This commit is the result of several big changes being made to the wallet. In particular, the "handshake" (initial sync to the chain server) was quite racy and required proper synchronization. To make fixing this race easier, several other changes were made to the internal wallet data structures and much of the RPC server ended up being rewritten. First, all account support has been removed. The previous Account struct has been replaced with a Wallet structure, which includes a keystore for saving keys, and a txstore for storing relevant transactions. This decision has been made since it is the opinion of myself and other developers that bitcoind accounts are fundamentally broken (as accounts implemented by bitcoind support both arbitrary address groupings as well as moving balances between accounts -- these are fundamentally incompatible features), and since a BIP0032 keystore is soon planned to be implemented (at which point, "accounts" can return as HD extended keys). With the keystore handling the grouping of related keys, there is no reason have many different Account structs, and the AccountManager has been removed as well. All RPC handlers that take an account option will only work with "" (the default account) or "*" if the RPC allows specifying all accounts. Second, much of the RPC server has been cleaned up. The global variables for the RPC server and chain server client have been moved to part of the rpcServer struct, and the handlers for each RPC method that are looked up change depending on which components have been set. Passthrough requests are also no longer handled specially, but when the chain server is set, a handler to perform the passthrough will be returned if the method is not otherwise a wallet RPC. The notification system for websocket clients has also been rewritten so wallet components can send notifications through channels, rather than requiring direct access to the RPC server itself, or worse still, sending directly to a websocket client's send channel. In the future, this will enable proper registration of notifications, rather than unsolicited broadcasts to every connected websocket client (see issue #84). Finally, and the main reason why much of this cleanup was necessary, the races during intial sync with the chain server have been fixed. Previously, when the 'Handshake' was run, a rescan would occur which would perform modifications to Account data structures as notifications were received. Synchronization was provided with a single binary semaphore which serialized all access to wallet and account data. However, the Handshake itself was not able to run with this lock (or else notifications would block), and many data races would occur as both notifications were being handled. If GOMAXPROCS was ever increased beyond 1, btcwallet would always immediately crash due to invalid addresses caused by the data races on startup. To fix this, the single lock for all wallet access has been replaced with mutexes for both the keystore and txstore. Handling of btcd notifications and client requests may now occur simultaneously. GOMAXPROCS has also been set to the number of logical CPUs at the beginning of main, since with the data races fixed, there's no reason to prevent the extra parallelism gained by increasing it. Closes #78. Closes #101. Closes #110.
2014-07-09 05:17:38 +02:00
}
rpcc, err := chain.NewClient(activeNet.Params, cfg.RPCConnect,
cfg.BtcdUsername, cfg.BtcdPassword, certs, cfg.DisableClientTLS)
Remove account support, fix races on btcd connect. This commit is the result of several big changes being made to the wallet. In particular, the "handshake" (initial sync to the chain server) was quite racy and required proper synchronization. To make fixing this race easier, several other changes were made to the internal wallet data structures and much of the RPC server ended up being rewritten. First, all account support has been removed. The previous Account struct has been replaced with a Wallet structure, which includes a keystore for saving keys, and a txstore for storing relevant transactions. This decision has been made since it is the opinion of myself and other developers that bitcoind accounts are fundamentally broken (as accounts implemented by bitcoind support both arbitrary address groupings as well as moving balances between accounts -- these are fundamentally incompatible features), and since a BIP0032 keystore is soon planned to be implemented (at which point, "accounts" can return as HD extended keys). With the keystore handling the grouping of related keys, there is no reason have many different Account structs, and the AccountManager has been removed as well. All RPC handlers that take an account option will only work with "" (the default account) or "*" if the RPC allows specifying all accounts. Second, much of the RPC server has been cleaned up. The global variables for the RPC server and chain server client have been moved to part of the rpcServer struct, and the handlers for each RPC method that are looked up change depending on which components have been set. Passthrough requests are also no longer handled specially, but when the chain server is set, a handler to perform the passthrough will be returned if the method is not otherwise a wallet RPC. The notification system for websocket clients has also been rewritten so wallet components can send notifications through channels, rather than requiring direct access to the RPC server itself, or worse still, sending directly to a websocket client's send channel. In the future, this will enable proper registration of notifications, rather than unsolicited broadcasts to every connected websocket client (see issue #84). Finally, and the main reason why much of this cleanup was necessary, the races during intial sync with the chain server have been fixed. Previously, when the 'Handshake' was run, a rescan would occur which would perform modifications to Account data structures as notifications were received. Synchronization was provided with a single binary semaphore which serialized all access to wallet and account data. However, the Handshake itself was not able to run with this lock (or else notifications would block), and many data races would occur as both notifications were being handled. If GOMAXPROCS was ever increased beyond 1, btcwallet would always immediately crash due to invalid addresses caused by the data races on startup. To fix this, the single lock for all wallet access has been replaced with mutexes for both the keystore and txstore. Handling of btcd notifications and client requests may now occur simultaneously. GOMAXPROCS has also been set to the number of logical CPUs at the beginning of main, since with the data races fixed, there's no reason to prevent the extra parallelism gained by increasing it. Closes #78. Closes #101. Closes #110.
2014-07-09 05:17:38 +02:00
if err != nil {
log.Errorf("Cannot create chain server RPC client: %v", err)
return
}
err = rpcc.Start()
if err != nil {
log.Warnf("Connection to Bitcoin RPC chain server " +
"unsuccessful -- available RPC methods will be limited")
}
// Even if Start errored, we still add the server disconnected.
// All client methods will then error, so it's obvious to a
// client that the there was a connection problem.
server.SetChainServer(rpcc)
chainSvrChan <- rpcc
}()
// Create a channel to report unrecoverable errors during the loading of
// the wallet files. These may include OS file handling errors or
// issues deserializing the wallet files, but does not include missing
// wallet files (as that must be handled by creating a new wallet).
walletOpenErrors := make(chan error)
go func() {
defer close(walletOpenErrors)
// Open wallet structures from disk.
w, err := openWallet()
if err != nil {
if os.IsNotExist(err) {
// If the keystore file is missing, notify the server
// that generating new wallets is ok.
server.SetWallet(nil)
return
} else {
// If the keystore file exists but another error was
// encountered, we cannot continue.
log.Errorf("Cannot load wallet files: %v", err)
walletOpenErrors <- err
return
}
}
server.SetWallet(w)
// Start wallet goroutines and handle RPC client notifications
// if the chain server connection was opened.
select {
case chainSvr := <-chainSvrChan:
w.Start(chainSvr)
Remove account support, fix races on btcd connect. This commit is the result of several big changes being made to the wallet. In particular, the "handshake" (initial sync to the chain server) was quite racy and required proper synchronization. To make fixing this race easier, several other changes were made to the internal wallet data structures and much of the RPC server ended up being rewritten. First, all account support has been removed. The previous Account struct has been replaced with a Wallet structure, which includes a keystore for saving keys, and a txstore for storing relevant transactions. This decision has been made since it is the opinion of myself and other developers that bitcoind accounts are fundamentally broken (as accounts implemented by bitcoind support both arbitrary address groupings as well as moving balances between accounts -- these are fundamentally incompatible features), and since a BIP0032 keystore is soon planned to be implemented (at which point, "accounts" can return as HD extended keys). With the keystore handling the grouping of related keys, there is no reason have many different Account structs, and the AccountManager has been removed as well. All RPC handlers that take an account option will only work with "" (the default account) or "*" if the RPC allows specifying all accounts. Second, much of the RPC server has been cleaned up. The global variables for the RPC server and chain server client have been moved to part of the rpcServer struct, and the handlers for each RPC method that are looked up change depending on which components have been set. Passthrough requests are also no longer handled specially, but when the chain server is set, a handler to perform the passthrough will be returned if the method is not otherwise a wallet RPC. The notification system for websocket clients has also been rewritten so wallet components can send notifications through channels, rather than requiring direct access to the RPC server itself, or worse still, sending directly to a websocket client's send channel. In the future, this will enable proper registration of notifications, rather than unsolicited broadcasts to every connected websocket client (see issue #84). Finally, and the main reason why much of this cleanup was necessary, the races during intial sync with the chain server have been fixed. Previously, when the 'Handshake' was run, a rescan would occur which would perform modifications to Account data structures as notifications were received. Synchronization was provided with a single binary semaphore which serialized all access to wallet and account data. However, the Handshake itself was not able to run with this lock (or else notifications would block), and many data races would occur as both notifications were being handled. If GOMAXPROCS was ever increased beyond 1, btcwallet would always immediately crash due to invalid addresses caused by the data races on startup. To fix this, the single lock for all wallet access has been replaced with mutexes for both the keystore and txstore. Handling of btcd notifications and client requests may now occur simultaneously. GOMAXPROCS has also been set to the number of logical CPUs at the beginning of main, since with the data races fixed, there's no reason to prevent the extra parallelism gained by increasing it. Closes #78. Closes #101. Closes #110.
2014-07-09 05:17:38 +02:00
case <-server.quit:
}
}()
// Check for unrecoverable errors during the wallet startup, and return
// the error, if any.
err, ok := <-walletOpenErrors
Remove account support, fix races on btcd connect. This commit is the result of several big changes being made to the wallet. In particular, the "handshake" (initial sync to the chain server) was quite racy and required proper synchronization. To make fixing this race easier, several other changes were made to the internal wallet data structures and much of the RPC server ended up being rewritten. First, all account support has been removed. The previous Account struct has been replaced with a Wallet structure, which includes a keystore for saving keys, and a txstore for storing relevant transactions. This decision has been made since it is the opinion of myself and other developers that bitcoind accounts are fundamentally broken (as accounts implemented by bitcoind support both arbitrary address groupings as well as moving balances between accounts -- these are fundamentally incompatible features), and since a BIP0032 keystore is soon planned to be implemented (at which point, "accounts" can return as HD extended keys). With the keystore handling the grouping of related keys, there is no reason have many different Account structs, and the AccountManager has been removed as well. All RPC handlers that take an account option will only work with "" (the default account) or "*" if the RPC allows specifying all accounts. Second, much of the RPC server has been cleaned up. The global variables for the RPC server and chain server client have been moved to part of the rpcServer struct, and the handlers for each RPC method that are looked up change depending on which components have been set. Passthrough requests are also no longer handled specially, but when the chain server is set, a handler to perform the passthrough will be returned if the method is not otherwise a wallet RPC. The notification system for websocket clients has also been rewritten so wallet components can send notifications through channels, rather than requiring direct access to the RPC server itself, or worse still, sending directly to a websocket client's send channel. In the future, this will enable proper registration of notifications, rather than unsolicited broadcasts to every connected websocket client (see issue #84). Finally, and the main reason why much of this cleanup was necessary, the races during intial sync with the chain server have been fixed. Previously, when the 'Handshake' was run, a rescan would occur which would perform modifications to Account data structures as notifications were received. Synchronization was provided with a single binary semaphore which serialized all access to wallet and account data. However, the Handshake itself was not able to run with this lock (or else notifications would block), and many data races would occur as both notifications were being handled. If GOMAXPROCS was ever increased beyond 1, btcwallet would always immediately crash due to invalid addresses caused by the data races on startup. To fix this, the single lock for all wallet access has been replaced with mutexes for both the keystore and txstore. Handling of btcd notifications and client requests may now occur simultaneously. GOMAXPROCS has also been set to the number of logical CPUs at the beginning of main, since with the data races fixed, there's no reason to prevent the extra parallelism gained by increasing it. Closes #78. Closes #101. Closes #110.
2014-07-09 05:17:38 +02:00
if ok {
return err
}
// Wait for the server to shutdown either due to a stop RPC request
// or an interrupt.
server.WaitForShutdown()
log.Info("Shutdown complete")
return nil
}