2016-08-08 21:49:09 +02:00
|
|
|
// Copyright (c) 2013-2016 The btcsuite developers
|
2015-12-01 19:44:58 +01:00
|
|
|
// Use of this source code is governed by an ISC
|
|
|
|
// license that can be found in the LICENSE file.
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
|
|
|
|
package chain
|
|
|
|
|
|
|
|
import (
|
|
|
|
"errors"
|
|
|
|
"sync"
|
|
|
|
"time"
|
|
|
|
|
2015-05-01 07:55:49 +02:00
|
|
|
"github.com/btcsuite/btcd/btcjson"
|
2015-02-06 06:51:37 +01:00
|
|
|
"github.com/btcsuite/btcd/chaincfg"
|
2016-08-08 21:49:09 +02:00
|
|
|
"github.com/btcsuite/btcd/chaincfg/chainhash"
|
2017-08-25 02:30:43 +02:00
|
|
|
"github.com/btcsuite/btcd/rpcclient"
|
2015-01-15 17:48:58 +01:00
|
|
|
"github.com/btcsuite/btcutil"
|
2014-10-29 07:43:29 +01:00
|
|
|
"github.com/btcsuite/btcwallet/waddrmgr"
|
2015-04-06 21:03:24 +02:00
|
|
|
"github.com/btcsuite/btcwallet/wtxmgr"
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
)
|
|
|
|
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
// RPCClient represents a persistent client connection to a bitcoin RPC server
|
2015-02-06 06:12:48 +01:00
|
|
|
// for information regarding the current best block chain.
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
type RPCClient struct {
|
2017-08-25 02:30:43 +02:00
|
|
|
*rpcclient.Client
|
|
|
|
connConfig *rpcclient.ConnConfig // Work around unexported field
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
chainParams *chaincfg.Params
|
|
|
|
reconnectAttempts int
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
|
|
|
|
enqueueNotification chan interface{}
|
|
|
|
dequeueNotification chan interface{}
|
2014-10-29 07:43:29 +01:00
|
|
|
currentBlock chan *waddrmgr.BlockStamp
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
|
|
|
|
quit chan struct{}
|
|
|
|
wg sync.WaitGroup
|
|
|
|
started bool
|
|
|
|
quitMtx sync.Mutex
|
|
|
|
}
|
|
|
|
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
// NewRPCClient creates a client connection to the server described by the
|
|
|
|
// connect string. If disableTLS is false, the remote RPC certificate must be
|
|
|
|
// provided in the certs slice. The connection is not established immediately,
|
|
|
|
// but must be done using the Start method. If the remote server does not
|
|
|
|
// operate on the same bitcoin network as described by the passed chain
|
|
|
|
// parameters, the connection will be disconnected.
|
|
|
|
func NewRPCClient(chainParams *chaincfg.Params, connect, user, pass string, certs []byte,
|
|
|
|
disableTLS bool, reconnectAttempts int) (*RPCClient, error) {
|
|
|
|
|
|
|
|
if reconnectAttempts < 0 {
|
|
|
|
return nil, errors.New("reconnectAttempts must be positive")
|
|
|
|
}
|
|
|
|
|
|
|
|
client := &RPCClient{
|
2017-08-25 02:30:43 +02:00
|
|
|
connConfig: &rpcclient.ConnConfig{
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
Host: connect,
|
|
|
|
Endpoint: "ws",
|
|
|
|
User: user,
|
|
|
|
Pass: pass,
|
|
|
|
Certificates: certs,
|
2017-03-09 00:23:21 +01:00
|
|
|
DisableAutoReconnect: false,
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
DisableConnectOnNew: true,
|
|
|
|
DisableTLS: disableTLS,
|
|
|
|
},
|
2015-02-06 06:51:37 +01:00
|
|
|
chainParams: chainParams,
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
reconnectAttempts: reconnectAttempts,
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
enqueueNotification: make(chan interface{}),
|
|
|
|
dequeueNotification: make(chan interface{}),
|
2014-10-29 07:43:29 +01:00
|
|
|
currentBlock: make(chan *waddrmgr.BlockStamp),
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
quit: make(chan struct{}),
|
|
|
|
}
|
2017-08-25 02:30:43 +02:00
|
|
|
ntfnCallbacks := &rpcclient.NotificationHandlers{
|
2014-07-28 16:33:00 +02:00
|
|
|
OnClientConnected: client.onClientConnect,
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
OnBlockConnected: client.onBlockConnected,
|
|
|
|
OnBlockDisconnected: client.onBlockDisconnected,
|
|
|
|
OnRecvTx: client.onRecvTx,
|
|
|
|
OnRedeemingTx: client.onRedeemingTx,
|
|
|
|
OnRescanFinished: client.onRescanFinished,
|
|
|
|
OnRescanProgress: client.onRescanProgress,
|
|
|
|
}
|
2017-08-25 02:30:43 +02:00
|
|
|
rpcClient, err := rpcclient.New(client.connConfig, ntfnCallbacks)
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
client.Client = rpcClient
|
|
|
|
return client, nil
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
2015-02-06 06:12:48 +01:00
|
|
|
// Start attempts to establish a client connection with the remote server.
|
|
|
|
// If successful, handler goroutines are started to process notifications
|
|
|
|
// sent by the server. After a limited number of connection attempts, this
|
|
|
|
// function gives up, and therefore will not block forever waiting for the
|
|
|
|
// connection to be established to a server that may not exist.
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
func (c *RPCClient) Start() error {
|
|
|
|
err := c.Connect(c.reconnectAttempts)
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
// Verify that the server is running on the expected network.
|
|
|
|
net, err := c.GetCurrentNet()
|
|
|
|
if err != nil {
|
|
|
|
c.Disconnect()
|
|
|
|
return err
|
|
|
|
}
|
2015-02-06 06:51:37 +01:00
|
|
|
if net != c.chainParams.Net {
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
c.Disconnect()
|
|
|
|
return errors.New("mismatched networks")
|
|
|
|
}
|
|
|
|
|
|
|
|
c.quitMtx.Lock()
|
|
|
|
c.started = true
|
|
|
|
c.quitMtx.Unlock()
|
|
|
|
|
|
|
|
c.wg.Add(1)
|
|
|
|
go c.handler()
|
|
|
|
return nil
|
|
|
|
}
|
|
|
|
|
2015-02-06 06:12:48 +01:00
|
|
|
// Stop disconnects the client and signals the shutdown of all goroutines
|
|
|
|
// started by Start.
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
func (c *RPCClient) Stop() {
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
c.quitMtx.Lock()
|
|
|
|
select {
|
|
|
|
case <-c.quit:
|
|
|
|
default:
|
|
|
|
close(c.quit)
|
|
|
|
c.Client.Shutdown()
|
|
|
|
|
|
|
|
if !c.started {
|
|
|
|
close(c.dequeueNotification)
|
|
|
|
}
|
|
|
|
}
|
2015-09-15 22:05:58 +02:00
|
|
|
c.quitMtx.Unlock()
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
2015-02-06 06:12:48 +01:00
|
|
|
// WaitForShutdown blocks until both the client has finished disconnecting
|
|
|
|
// and all handlers have exited.
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
func (c *RPCClient) WaitForShutdown() {
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
c.Client.WaitForShutdown()
|
|
|
|
c.wg.Wait()
|
|
|
|
}
|
|
|
|
|
2015-02-06 06:12:48 +01:00
|
|
|
// Notifications returns a channel of parsed notifications sent by the remote
|
|
|
|
// bitcoin RPC server. This channel must be continually read or the process
|
|
|
|
// may abort for running out memory, as unread notifications are queued for
|
|
|
|
// later reads.
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
func (c *RPCClient) Notifications() <-chan interface{} {
|
2015-02-06 06:12:48 +01:00
|
|
|
return c.dequeueNotification
|
|
|
|
}
|
|
|
|
|
|
|
|
// BlockStamp returns the latest block notified by the client, or an error
|
|
|
|
// if the client has been shut down.
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
func (c *RPCClient) BlockStamp() (*waddrmgr.BlockStamp, error) {
|
2015-02-06 06:12:48 +01:00
|
|
|
select {
|
|
|
|
case bs := <-c.currentBlock:
|
|
|
|
return bs, nil
|
|
|
|
case <-c.quit:
|
|
|
|
return nil, errors.New("disconnected")
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2015-04-06 21:03:24 +02:00
|
|
|
// parseBlock parses a btcws definition of the block a tx is mined it to the
|
|
|
|
// Block structure of the wtxmgr package, and the block index. This is done
|
2017-08-25 02:30:43 +02:00
|
|
|
// here since rpcclient doesn't parse this nicely for us.
|
2015-04-06 21:03:24 +02:00
|
|
|
func parseBlock(block *btcjson.BlockDetails) (*wtxmgr.BlockMeta, error) {
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
if block == nil {
|
2015-04-06 21:03:24 +02:00
|
|
|
return nil, nil
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
2016-08-08 21:49:09 +02:00
|
|
|
blkHash, err := chainhash.NewHashFromStr(block.Hash)
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
if err != nil {
|
2015-04-06 21:03:24 +02:00
|
|
|
return nil, err
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
2015-04-06 21:03:24 +02:00
|
|
|
blk := &wtxmgr.BlockMeta{
|
|
|
|
Block: wtxmgr.Block{
|
|
|
|
Height: block.Height,
|
2016-08-08 21:49:09 +02:00
|
|
|
Hash: *blkHash,
|
2015-04-06 21:03:24 +02:00
|
|
|
},
|
|
|
|
Time: time.Unix(block.Time, 0),
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
2015-04-06 21:03:24 +02:00
|
|
|
return blk, nil
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
func (c *RPCClient) onClientConnect() {
|
2015-09-15 22:05:58 +02:00
|
|
|
select {
|
|
|
|
case c.enqueueNotification <- ClientConnected{}:
|
|
|
|
case <-c.quit:
|
|
|
|
}
|
2014-07-28 16:33:00 +02:00
|
|
|
}
|
|
|
|
|
2016-08-08 21:49:09 +02:00
|
|
|
func (c *RPCClient) onBlockConnected(hash *chainhash.Hash, height int32, time time.Time) {
|
2015-09-15 22:05:58 +02:00
|
|
|
select {
|
|
|
|
case c.enqueueNotification <- BlockConnected{
|
2015-06-18 18:29:13 +02:00
|
|
|
Block: wtxmgr.Block{
|
|
|
|
Hash: *hash,
|
|
|
|
Height: height,
|
|
|
|
},
|
|
|
|
Time: time,
|
2015-09-15 22:05:58 +02:00
|
|
|
}:
|
|
|
|
case <-c.quit:
|
2015-06-18 18:29:13 +02:00
|
|
|
}
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
2016-08-08 21:49:09 +02:00
|
|
|
func (c *RPCClient) onBlockDisconnected(hash *chainhash.Hash, height int32, time time.Time) {
|
2015-09-15 22:05:58 +02:00
|
|
|
select {
|
|
|
|
case c.enqueueNotification <- BlockDisconnected{
|
2015-06-18 18:29:13 +02:00
|
|
|
Block: wtxmgr.Block{
|
|
|
|
Hash: *hash,
|
|
|
|
Height: height,
|
|
|
|
},
|
|
|
|
Time: time,
|
2015-09-15 22:05:58 +02:00
|
|
|
}:
|
|
|
|
case <-c.quit:
|
2015-06-18 18:29:13 +02:00
|
|
|
}
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
func (c *RPCClient) onRecvTx(tx *btcutil.Tx, block *btcjson.BlockDetails) {
|
2015-04-06 21:03:24 +02:00
|
|
|
blk, err := parseBlock(block)
|
|
|
|
if err != nil {
|
|
|
|
// Log and drop improper notification.
|
|
|
|
log.Errorf("recvtx notification bad block: %v", err)
|
|
|
|
return
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
2015-04-06 21:03:24 +02:00
|
|
|
|
|
|
|
rec, err := wtxmgr.NewTxRecordFromMsgTx(tx.MsgTx(), time.Now())
|
|
|
|
if err != nil {
|
|
|
|
log.Errorf("Cannot create transaction record for relevant "+
|
|
|
|
"tx: %v", err)
|
|
|
|
return
|
|
|
|
}
|
2015-09-15 22:05:58 +02:00
|
|
|
select {
|
|
|
|
case c.enqueueNotification <- RelevantTx{rec, blk}:
|
|
|
|
case <-c.quit:
|
|
|
|
}
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
func (c *RPCClient) onRedeemingTx(tx *btcutil.Tx, block *btcjson.BlockDetails) {
|
2015-04-06 21:03:24 +02:00
|
|
|
// Handled exactly like recvtx notifications.
|
|
|
|
c.onRecvTx(tx, block)
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
2016-08-08 21:49:09 +02:00
|
|
|
func (c *RPCClient) onRescanProgress(hash *chainhash.Hash, height int32, blkTime time.Time) {
|
2015-09-15 22:05:58 +02:00
|
|
|
select {
|
|
|
|
case c.enqueueNotification <- &RescanProgress{hash, height, blkTime}:
|
|
|
|
case <-c.quit:
|
|
|
|
}
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
2016-08-08 21:49:09 +02:00
|
|
|
func (c *RPCClient) onRescanFinished(hash *chainhash.Hash, height int32, blkTime time.Time) {
|
2015-09-15 22:05:58 +02:00
|
|
|
select {
|
|
|
|
case c.enqueueNotification <- &RescanFinished{hash, height, blkTime}:
|
|
|
|
case <-c.quit:
|
|
|
|
}
|
|
|
|
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// handler maintains a queue of notifications and the current state (best
|
|
|
|
// block) of the chain.
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
func (c *RPCClient) handler() {
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
hash, height, err := c.GetBestBlock()
|
|
|
|
if err != nil {
|
2015-09-15 22:05:58 +02:00
|
|
|
log.Errorf("Failed to receive best block from chain server: %v", err)
|
|
|
|
c.Stop()
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
c.wg.Done()
|
2015-09-15 22:05:58 +02:00
|
|
|
return
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
2014-10-29 07:43:29 +01:00
|
|
|
bs := &waddrmgr.BlockStamp{Hash: *hash, Height: height}
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
|
|
|
|
// TODO: Rather than leaving this as an unbounded queue for all types of
|
|
|
|
// notifications, try dropping ones where a later enqueued notification
|
|
|
|
// can fully invalidate one waiting to be processed. For example,
|
|
|
|
// blockconnected notifications for greater block heights can remove the
|
|
|
|
// need to process earlier blockconnected notifications still waiting
|
|
|
|
// here.
|
|
|
|
|
|
|
|
var notifications []interface{}
|
|
|
|
enqueue := c.enqueueNotification
|
|
|
|
var dequeue chan interface{}
|
|
|
|
var next interface{}
|
2015-09-15 22:05:58 +02:00
|
|
|
pingChan := time.After(time.Minute)
|
2017-04-21 22:48:18 +02:00
|
|
|
pingChanReset := make(chan (<-chan time.Time))
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
out:
|
|
|
|
for {
|
|
|
|
select {
|
|
|
|
case n, ok := <-enqueue:
|
|
|
|
if !ok {
|
|
|
|
// If no notifications are queued for handling,
|
|
|
|
// the queue is finished.
|
|
|
|
if len(notifications) == 0 {
|
|
|
|
break out
|
|
|
|
}
|
|
|
|
// nil channel so no more reads can occur.
|
|
|
|
enqueue = nil
|
|
|
|
continue
|
|
|
|
}
|
|
|
|
if len(notifications) == 0 {
|
|
|
|
next = n
|
|
|
|
dequeue = c.dequeueNotification
|
|
|
|
}
|
|
|
|
notifications = append(notifications, n)
|
2015-09-15 22:05:58 +02:00
|
|
|
pingChan = time.After(time.Minute)
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
|
|
|
|
case dequeue <- next:
|
|
|
|
if n, ok := next.(BlockConnected); ok {
|
2015-06-18 18:29:13 +02:00
|
|
|
bs = &waddrmgr.BlockStamp{
|
2016-03-25 20:11:25 +01:00
|
|
|
Height: n.Height,
|
|
|
|
Hash: n.Hash,
|
2015-06-18 18:29:13 +02:00
|
|
|
}
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
notifications[0] = nil
|
|
|
|
notifications = notifications[1:]
|
|
|
|
if len(notifications) != 0 {
|
|
|
|
next = notifications[0]
|
|
|
|
} else {
|
|
|
|
// If no more notifications can be enqueued, the
|
|
|
|
// queue is finished.
|
|
|
|
if enqueue == nil {
|
|
|
|
break out
|
|
|
|
}
|
|
|
|
dequeue = nil
|
|
|
|
}
|
|
|
|
|
2015-09-15 22:05:58 +02:00
|
|
|
case <-pingChan:
|
2017-04-21 22:48:18 +02:00
|
|
|
// No notifications were received in the last 60s. Ensure the
|
|
|
|
// connection is still active by making a new request to the server.
|
|
|
|
//
|
|
|
|
// This MUST wait for the response in a new goroutine so as to not
|
|
|
|
// block channel sends enqueueing more notifications. Doing so
|
|
|
|
// would cause a deadlock and after the timeout expires, the client
|
|
|
|
// would be shut down.
|
|
|
|
//
|
|
|
|
// TODO: A minute timeout is used to prevent the handler loop from
|
|
|
|
// blocking here forever, but this is much larger than it needs to
|
|
|
|
// be due to dcrd processing websocket requests synchronously (see
|
2017-06-06 02:54:35 +02:00
|
|
|
// https://github.com/roasbeef/btcd/issues/504). Decrease this to
|
2017-04-21 22:48:18 +02:00
|
|
|
// something saner like 3s when the above issue is fixed.
|
2015-09-15 22:05:58 +02:00
|
|
|
type sessionResult struct {
|
|
|
|
err error
|
|
|
|
}
|
|
|
|
sessionResponse := make(chan sessionResult, 1)
|
|
|
|
go func() {
|
|
|
|
_, err := c.Session()
|
|
|
|
sessionResponse <- sessionResult{err}
|
|
|
|
}()
|
2017-04-21 22:48:18 +02:00
|
|
|
go func() {
|
|
|
|
select {
|
|
|
|
case resp := <-sessionResponse:
|
|
|
|
if resp.err != nil {
|
|
|
|
log.Errorf("Failed to receive session "+
|
|
|
|
"result: %v", resp.err)
|
|
|
|
c.Stop()
|
|
|
|
}
|
|
|
|
pingChanReset <- time.After(time.Minute)
|
|
|
|
|
|
|
|
case <-time.After(time.Minute):
|
|
|
|
log.Errorf("Timeout waiting for session RPC")
|
2015-09-15 22:05:58 +02:00
|
|
|
c.Stop()
|
|
|
|
}
|
2017-04-21 22:48:18 +02:00
|
|
|
}()
|
2015-09-15 22:05:58 +02:00
|
|
|
|
2017-04-21 22:48:18 +02:00
|
|
|
case ch := <-pingChanReset:
|
|
|
|
pingChan = ch
|
2015-09-15 22:05:58 +02:00
|
|
|
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
case c.currentBlock <- bs:
|
|
|
|
|
|
|
|
case <-c.quit:
|
|
|
|
break out
|
|
|
|
}
|
|
|
|
}
|
2015-09-15 22:05:58 +02:00
|
|
|
|
|
|
|
c.Stop()
|
Remove account support, fix races on btcd connect.
This commit is the result of several big changes being made to the
wallet. In particular, the "handshake" (initial sync to the chain
server) was quite racy and required proper synchronization. To make
fixing this race easier, several other changes were made to the
internal wallet data structures and much of the RPC server ended up
being rewritten.
First, all account support has been removed. The previous Account
struct has been replaced with a Wallet structure, which includes a
keystore for saving keys, and a txstore for storing relevant
transactions. This decision has been made since it is the opinion of
myself and other developers that bitcoind accounts are fundamentally
broken (as accounts implemented by bitcoind support both arbitrary
address groupings as well as moving balances between accounts -- these
are fundamentally incompatible features), and since a BIP0032 keystore
is soon planned to be implemented (at which point, "accounts" can
return as HD extended keys). With the keystore handling the grouping
of related keys, there is no reason have many different Account
structs, and the AccountManager has been removed as well. All RPC
handlers that take an account option will only work with "" (the
default account) or "*" if the RPC allows specifying all accounts.
Second, much of the RPC server has been cleaned up. The global
variables for the RPC server and chain server client have been moved
to part of the rpcServer struct, and the handlers for each RPC method
that are looked up change depending on which components have been set.
Passthrough requests are also no longer handled specially, but when
the chain server is set, a handler to perform the passthrough will be
returned if the method is not otherwise a wallet RPC. The
notification system for websocket clients has also been rewritten so
wallet components can send notifications through channels, rather than
requiring direct access to the RPC server itself, or worse still,
sending directly to a websocket client's send channel. In the future,
this will enable proper registration of notifications, rather than
unsolicited broadcasts to every connected websocket client (see
issue #84).
Finally, and the main reason why much of this cleanup was necessary,
the races during intial sync with the chain server have been fixed.
Previously, when the 'Handshake' was run, a rescan would occur which
would perform modifications to Account data structures as
notifications were received. Synchronization was provided with a
single binary semaphore which serialized all access to wallet and
account data. However, the Handshake itself was not able to run with
this lock (or else notifications would block), and many data races
would occur as both notifications were being handled. If GOMAXPROCS
was ever increased beyond 1, btcwallet would always immediately crash
due to invalid addresses caused by the data races on startup. To fix
this, the single lock for all wallet access has been replaced with
mutexes for both the keystore and txstore. Handling of btcd
notifications and client requests may now occur simultaneously.
GOMAXPROCS has also been set to the number of logical CPUs at the
beginning of main, since with the data races fixed, there's no reason
to prevent the extra parallelism gained by increasing it.
Closes #78.
Closes #101.
Closes #110.
2014-07-09 05:17:38 +02:00
|
|
|
close(c.dequeueNotification)
|
|
|
|
c.wg.Done()
|
|
|
|
}
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
|
2017-08-25 02:30:43 +02:00
|
|
|
// POSTClient creates the equivalent HTTP POST rpcclient.Client.
|
|
|
|
func (c *RPCClient) POSTClient() (*rpcclient.Client, error) {
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
configCopy := *c.connConfig
|
|
|
|
configCopy.HTTPPostMode = true
|
2017-08-25 02:30:43 +02:00
|
|
|
return rpcclient.New(&configCopy, nil)
|
Modernize the RPC server.
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).
The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default. Enabling the new server requires
setting a listen address (--experimenalrpclisten). This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default. Both RPC servers can be run at the same time,
but require binding to different listen addresses.
In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers. This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.
Type safety has also been improved in the legacy RPC server. Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency. This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.
To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added. All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened. This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.
Documentation for the new RPC server has been added to the
rpc/documentation directory. The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.
Some of the new RPC methods are not implementated exactly as described
by the specification. These are considered bugs with the
implementation, not the spec. Known bugs are commented as such.
2015-06-01 21:57:50 +02:00
|
|
|
}
|