This fixes a bug with the authentication handling for websocket
clients where it was possible that even after supplying bad
authentication using the HTTP Authorization header, the connection
would remain open and flagged as unauthenticated, and clients (if they
somehow knew auth failed, although btcwallet would never tell them
until after they failed their next request) could try their hand at
authorization again by issuing an authenticate request.
While I don't know of any reason the above described bug could result
in a security leak, it's better to fail the connection as soon as
possible if they failed their first authentication attempt.
While here, also set a read deadline of 10 seconds for the first
request. If the initial handshake cannot complete in this timeframe,
the connection is terminated. This matches the behavior in btcd, and
prevents websocket clients from connecting without the Authorization
header and never issuing their first authenticate request.
This change takes advantage of the RawMessage type in the
encoding/json package to defer unmarshaling of all JSON-RPC values
until absolutely necessary.
This is particularly important for request passthrough when btcwallet
must ask btcd to handle a chain request for a wallet client. In the
previous code, during the marshal and unmarshal dance to set the
original client's request id in the btcd response, large JSON numbers
were being mangled to use (scientific) E notation even when they could
be represented as a integer without any loss of precision.
The select statement does not guarantee selecting a better case if one
might panic for sending to a closed channel. This case was hit during
client disconnect due to having multiple senders on a single channel
with one of the senders closing the chan to notify the next goroutine
to finish. This change gives each writes its own unique channel to
prevent this error.
Just like btcd, this commit adds support for the authenticate request
allowing clients unable to set the HTTP Authorization header to
authenticate to use the RPC server. The rules for the authenticate
request are as follows:
1) Authentication make clients stateful. Clients may either be flagged
as authenticated or unauthenticated.
2) Clients may authenticate by exactly one of two possible ways,
either by setting the Authorization header or by sending a JSON-RPC
authenticate request as follows:
{
"jsonrpc":"1.0",
"id":0,
"method":"authenticate",
"params":["rpcuser", "rpcpass"]
}
3) When not authenticated by the Authorization header, the first request
must be an authenticate request.
4) Sending an authenticate request after a client has already
successfully authenticated (either by the Authorization header or a
previous authentication request) is invalid.
5) The result used in the response to a successful authenticate request
is a JSON null. For any unsuccessful or invalid authenticate
requests, the connection is terminated.
This change also orders all incoming requests for a client. This was
required to ensure that any authentication requests are processed
first.
This change replaces the old transaction store file format and
implementation. The most important change is how the full backing
transactions for any received or sent transaction are now saved,
rather than simply saving parsed-out details of the tx (tx shas, block
height/hash, pkScripts, etc.).
To support the change, notifications for received transaction outputs
and txs spending watched outpoints have been updated to use the new
redeemingtx and recvtx notifications as these contain the full tx,
which is deserializead and inserted into the store.
The old transaction store serialization code is completely removed, as
updating to the new format automatically cannot be done. Old wallets
first running past this change will error reading the file and start a
full rescan to rebuild the data. Unlike previous rescan code,
transactions spending outpoint managed by wallet are also included.
This results in recovering not just received history, but history for
sent transactions as well.
It appears that the websocket package will occasionally enter a
Codec's Send function and block forever, never erroring (presumably
due to a closed connection). This change adds a deadline for the send
of two seconds. If the send cannot complete before the deadline is
reached, the send is aborted and the connection is assumed to be lost.
A buffer should be added here as well, so even waiting max two seconds
for the send to error out won't cause wallet code to block.
At any instant when a duplicated notification must be sent, either one
of two channel sends/recvs must occur. The first possibility is that
the client is disconnected, in which case the disconnected channel
will be read, and then the context removed from the goroutine-managed
map. The second possibility is that the disconnect channel has not
yet been closed, in which case it must block on an actual message
send. This change moves the second case out of the default case of
the select statement to avoid a race where:
1) The client has not yet disconnected, and the disconnected chan is
not ready for reads.
2) Control switches to the default case.
3) The client disconnects, the goroutine reading the send channel
returns, closes the disconnected channel, and no more reads occur.
4) The notification duplicator blocks forever trying to send the
message even when the disconnected notification channel has
already been closed.
This shouldn't be nececssary (a TODO was added to remind me to make
clients explicitly ask for this info) but in the meantime this fixes
clients such as btcgui which otherwise wouldn't think btcwallet is
properly connected to btcd and will desensitise some widgets.
This change removes the three separate mutexes which used to lock an
account's wallet, tx store, and utxo store. Accounts no longer
contain any locking mechanism and rely on go's other synchronization
constructs (goroutines and channels) for correct access.
All accounts are now managed as a collection through the new
AccountManager, rather than the old AccountStore. AccountManager runs
as its own goroutine to provide access to accounts.
RPC requests are now queued for handling, being denied if the queue
buffer is exhausted. Notifications are also queued (instead of being
sent from their own goroutine after being received, in which order is
undefined), however, notifications are never dropped and will
potentially grow a queue of infinite size if unhandled.
This change greatly cleans up the RPC connection between btcwallet and
btcd. Proper (JSON-RPC spec-following) notifications are now expected
rather than Responses with a non-empty IDs.
A new RPCConn interface type has also been introduced with a
BtcdRPCConn concrete type for btcd RPC connections. Non-btcd-specific
code handles the RPCConn, while the btcd details have been abstracted
away to a handful of functions. This will make it easier to write
tests by creating a new fake RPC connection with hardcoded expected
replies.
This change saves (at most) the last 20 block hashes to disk. Upon
btcd connect, in the handshake, btcwallet checks whether btcd's best
chain still contains these blocks, starting from the most recently
added block and continuing until the earliest saved. If any blocks
are missing, Tx history and UTXOs from any blocks no longer in the
chain are removed, and a rescan is started from after the best block
still in the main chain.
If all previous block hashes are exhausted (either due to a large
reorg, or because not enough blocks have been seen), a full rescan is
triggered (full meaning from the earliest block that matters to this
wallet) since the last synced up to point is no longer available.
The previous 20 seen block hashes are saved to the wallet file, which
required bumping the file version. Older wallets written with lesser
versions will use the previous reading function, making this change
backwards compatible.
This change copies the listening behavior of btcd by replacing the
--serverport option with --listen. By default, btcwallet will only
listen for localhost connections, but with this change it will be
possible to add listeners for remote connections.
This was added due to finding a bug with updateConfigWithActiveParams.
After consulting the btcd source code, the bug was fixed by replacing
the function (as it was no longer needed) when the new listening code
was introduced.
While here, mask out the password flag from being shown in the help
message.
This changes the locations that account files (wallet.bin, utxo.bin,
and tx.bin) are searched for when opening or disk syncing accounts.
Previously, files were saved in the following layout:
~/.btcwallet/
- btcwallet/
- wallet.bin
- tx.bin
- utxo.bin
- btcwallet-AccountA/
- wallet.bin
- tx.bin
- utxo.bin
This format had two issues. First, each account would require its own
directory, causing a scalability issue on unix (and perhaps other)
platforms. Second, there was no distinction between testnet and
mainnet wallets, and if mainnet support was enabled, btcwallet would
attempt to open accounts with testnet wallets.
Instead, the following file structure is now used:
~/.btcwallet/
- testnet/
- wallet.bin
- tx.bin
- utxo.bin
- AccountA-wallet.bin
- AccountA-tx.bin
- AccountA-utxo.bin
This solves both of the previously-mentioned issues by requiring only
two subdirectories (one each for the testnet and mainnet bitcoin
networks), and by separating the locations to open and save testnet
and mainnet account files.
At startup, a check for the old account file structure is performed.
If found, files are moved to the new locations, and the old account
directories are removed. Account files are moved to the testnet
directory, as only testnet support is currently enabled.
The version has been bumped to 0.1.1 to reflect this change.
Fixes#16.
This change better organizes account handling by creating a new
AccountStore type and accountstore global variable, with receiver
funcs for all operations that require all accounts. More Account
funcs are also added to clean up account handling in the RPC code.
Intial work on this done by dhill.
This change adds support for the listtransactions RPC command. To
properly reply to this command, additonal information about received
transactions was added, and is now saved in an account's tx.bin file.
Additionally, when sending a transaction, a *tx.SendTx is now saved to
the Tx store, and is included in listtransactions replies under the
"send" category.
WARNING: All account's tx.bin and utxo.bin files should be removed
before running with this change, or else the files may not be read
correctly. Removing tx.bin is not an issue as it was not being used
before, and was being saved with incorrect data. Removing utxo.bin is
not an issue as it will just trigger a rescan on next start. File
format versions are now included in both files, so automatic updates
from previous file formats will be possible with future changes.
Fixes#12.
This adds the necessary bits for handling importing addresses for the
wallet file format, as well as implementing the importprivkey and
dumpprivkey RPC requests.
Initial code by dhill.
This adds an additional config option, -cafile, to specify the root
certificates checked when verifying a btcd TLC connection. btcd will
now automatically generate certs in
~/.btcd/data/{main,test}net/rpc.cert, and this file should be copied
to ~/.btcwallet/cert.pem.
The -btcdport option is also gone now, and replaced with -connect (or
-c), to specify both the hostname/ip and port of the server running
btcd.
With the exception of the createencryptedwallet extension (which is
required to make a wallet), all websocket-specific handlers are now
only available from a websocket connection, and standard RPC requests
are handled with a normal HTTP request and reply.
As an added bonus, listening on IPv6 now works.
This change modifies the order in which transaction to watched
addresses are processed and when frontend notifications occur. Due to
btcd notifying all transactions before sending the blockconnected
notification, the UTXO and transaction stores can be modified without
sending any frontend notifications, and then a single frontend
notification is sent when the blockconnected notification arrives.
The order in which each file is synced to disk was also changed to
write out the UTXO and transaction stores before writing the wallet.
This is to prevent a race where wallet closes after writing the dirty
wallet, but before the dirty UTXO store is written. In this
situation, newly added UTXOs will be missed and not found again on the
next wallet open during the rescan. Writing the wallet (which holds
the synced-to-block information) last prevents this.
An issue where the unconfirmed change UTXO created from a new
transaction never being properly notified to frontends is fixed now as
well.
This removes the enforced check for the spent field for tx-to-me
notifications, as this is no longer sent, and should be calculated by
wallet (not done yet). Additionally, the full CreatedTx information
is saved with the unmined tx map, so when a tx is mined, information
about which inputs and ouputs it creates that are relevant to the
wallet can be used.
When a wallet is opened, a rescan request will be sent to btcd with
all active addresses from the wallet, to rescan from the last synced
block (now saved to the wallet file) and the current best block.
As multi-account support is further explored, rescan requests should
be batched together to send a single request for all addresses from
all wallets.
This change introduces several changes to the wallet, tx, and utxo
files. Wallet files are still compatible, however, a rescan will try
to start at the genesis block since no correct "last synced to" or
"created at block X" was saved. The tx and utxo files, however, are
not compatible and should be deleted (or an error will occur on read).
If any errors occur opening the utxo file, a rescan will start
beginning at the creation block saved in the wallet.
