This change replaces the old transaction store file format and
implementation. The most important change is how the full backing
transactions for any received or sent transaction are now saved,
rather than simply saving parsed-out details of the tx (tx shas, block
height/hash, pkScripts, etc.).
To support the change, notifications for received transaction outputs
and txs spending watched outpoints have been updated to use the new
redeemingtx and recvtx notifications as these contain the full tx,
which is deserializead and inserted into the store.
The old transaction store serialization code is completely removed, as
updating to the new format automatically cannot be done. Old wallets
first running past this change will error reading the file and start a
full rescan to rebuild the data. Unlike previous rescan code,
transactions spending outpoint managed by wallet are also included.
This results in recovering not just received history, but history for
sent transactions as well.
It appears that the websocket package will occasionally enter a
Codec's Send function and block forever, never erroring (presumably
due to a closed connection). This change adds a deadline for the send
of two seconds. If the send cannot complete before the deadline is
reached, the send is aborted and the connection is assumed to be lost.
A buffer should be added here as well, so even waiting max two seconds
for the send to error out won't cause wallet code to block.
At any instant when a duplicated notification must be sent, either one
of two channel sends/recvs must occur. The first possibility is that
the client is disconnected, in which case the disconnected channel
will be read, and then the context removed from the goroutine-managed
map. The second possibility is that the disconnect channel has not
yet been closed, in which case it must block on an actual message
send. This change moves the second case out of the default case of
the select statement to avoid a race where:
1) The client has not yet disconnected, and the disconnected chan is
not ready for reads.
2) Control switches to the default case.
3) The client disconnects, the goroutine reading the send channel
returns, closes the disconnected channel, and no more reads occur.
4) The notification duplicator blocks forever trying to send the
message even when the disconnected notification channel has
already been closed.
This shouldn't be nececssary (a TODO was added to remind me to make
clients explicitly ask for this info) but in the meantime this fixes
clients such as btcgui which otherwise wouldn't think btcwallet is
properly connected to btcd and will desensitise some widgets.
This change removes the three separate mutexes which used to lock an
account's wallet, tx store, and utxo store. Accounts no longer
contain any locking mechanism and rely on go's other synchronization
constructs (goroutines and channels) for correct access.
All accounts are now managed as a collection through the new
AccountManager, rather than the old AccountStore. AccountManager runs
as its own goroutine to provide access to accounts.
RPC requests are now queued for handling, being denied if the queue
buffer is exhausted. Notifications are also queued (instead of being
sent from their own goroutine after being received, in which order is
undefined), however, notifications are never dropped and will
potentially grow a queue of infinite size if unhandled.
This change greatly cleans up the RPC connection between btcwallet and
btcd. Proper (JSON-RPC spec-following) notifications are now expected
rather than Responses with a non-empty IDs.
A new RPCConn interface type has also been introduced with a
BtcdRPCConn concrete type for btcd RPC connections. Non-btcd-specific
code handles the RPCConn, while the btcd details have been abstracted
away to a handful of functions. This will make it easier to write
tests by creating a new fake RPC connection with hardcoded expected
replies.
This change saves (at most) the last 20 block hashes to disk. Upon
btcd connect, in the handshake, btcwallet checks whether btcd's best
chain still contains these blocks, starting from the most recently
added block and continuing until the earliest saved. If any blocks
are missing, Tx history and UTXOs from any blocks no longer in the
chain are removed, and a rescan is started from after the best block
still in the main chain.
If all previous block hashes are exhausted (either due to a large
reorg, or because not enough blocks have been seen), a full rescan is
triggered (full meaning from the earliest block that matters to this
wallet) since the last synced up to point is no longer available.
The previous 20 seen block hashes are saved to the wallet file, which
required bumping the file version. Older wallets written with lesser
versions will use the previous reading function, making this change
backwards compatible.
This change copies the listening behavior of btcd by replacing the
--serverport option with --listen. By default, btcwallet will only
listen for localhost connections, but with this change it will be
possible to add listeners for remote connections.
This was added due to finding a bug with updateConfigWithActiveParams.
After consulting the btcd source code, the bug was fixed by replacing
the function (as it was no longer needed) when the new listening code
was introduced.
While here, mask out the password flag from being shown in the help
message.
This changes the locations that account files (wallet.bin, utxo.bin,
and tx.bin) are searched for when opening or disk syncing accounts.
Previously, files were saved in the following layout:
~/.btcwallet/
- btcwallet/
- wallet.bin
- tx.bin
- utxo.bin
- btcwallet-AccountA/
- wallet.bin
- tx.bin
- utxo.bin
This format had two issues. First, each account would require its own
directory, causing a scalability issue on unix (and perhaps other)
platforms. Second, there was no distinction between testnet and
mainnet wallets, and if mainnet support was enabled, btcwallet would
attempt to open accounts with testnet wallets.
Instead, the following file structure is now used:
~/.btcwallet/
- testnet/
- wallet.bin
- tx.bin
- utxo.bin
- AccountA-wallet.bin
- AccountA-tx.bin
- AccountA-utxo.bin
This solves both of the previously-mentioned issues by requiring only
two subdirectories (one each for the testnet and mainnet bitcoin
networks), and by separating the locations to open and save testnet
and mainnet account files.
At startup, a check for the old account file structure is performed.
If found, files are moved to the new locations, and the old account
directories are removed. Account files are moved to the testnet
directory, as only testnet support is currently enabled.
The version has been bumped to 0.1.1 to reflect this change.
Fixes#16.
This change better organizes account handling by creating a new
AccountStore type and accountstore global variable, with receiver
funcs for all operations that require all accounts. More Account
funcs are also added to clean up account handling in the RPC code.
Intial work on this done by dhill.
This change adds support for the listtransactions RPC command. To
properly reply to this command, additonal information about received
transactions was added, and is now saved in an account's tx.bin file.
Additionally, when sending a transaction, a *tx.SendTx is now saved to
the Tx store, and is included in listtransactions replies under the
"send" category.
WARNING: All account's tx.bin and utxo.bin files should be removed
before running with this change, or else the files may not be read
correctly. Removing tx.bin is not an issue as it was not being used
before, and was being saved with incorrect data. Removing utxo.bin is
not an issue as it will just trigger a rescan on next start. File
format versions are now included in both files, so automatic updates
from previous file formats will be possible with future changes.
Fixes#12.
This adds the necessary bits for handling importing addresses for the
wallet file format, as well as implementing the importprivkey and
dumpprivkey RPC requests.
Initial code by dhill.
This adds an additional config option, -cafile, to specify the root
certificates checked when verifying a btcd TLC connection. btcd will
now automatically generate certs in
~/.btcd/data/{main,test}net/rpc.cert, and this file should be copied
to ~/.btcwallet/cert.pem.
The -btcdport option is also gone now, and replaced with -connect (or
-c), to specify both the hostname/ip and port of the server running
btcd.
With the exception of the createencryptedwallet extension (which is
required to make a wallet), all websocket-specific handlers are now
only available from a websocket connection, and standard RPC requests
are handled with a normal HTTP request and reply.
As an added bonus, listening on IPv6 now works.
This change modifies the order in which transaction to watched
addresses are processed and when frontend notifications occur. Due to
btcd notifying all transactions before sending the blockconnected
notification, the UTXO and transaction stores can be modified without
sending any frontend notifications, and then a single frontend
notification is sent when the blockconnected notification arrives.
The order in which each file is synced to disk was also changed to
write out the UTXO and transaction stores before writing the wallet.
This is to prevent a race where wallet closes after writing the dirty
wallet, but before the dirty UTXO store is written. In this
situation, newly added UTXOs will be missed and not found again on the
next wallet open during the rescan. Writing the wallet (which holds
the synced-to-block information) last prevents this.
An issue where the unconfirmed change UTXO created from a new
transaction never being properly notified to frontends is fixed now as
well.
This removes the enforced check for the spent field for tx-to-me
notifications, as this is no longer sent, and should be calculated by
wallet (not done yet). Additionally, the full CreatedTx information
is saved with the unmined tx map, so when a tx is mined, information
about which inputs and ouputs it creates that are relevant to the
wallet can be used.
When a wallet is opened, a rescan request will be sent to btcd with
all active addresses from the wallet, to rescan from the last synced
block (now saved to the wallet file) and the current best block.
As multi-account support is further explored, rescan requests should
be batched together to send a single request for all addresses from
all wallets.
This change introduces several changes to the wallet, tx, and utxo
files. Wallet files are still compatible, however, a rescan will try
to start at the genesis block since no correct "last synced to" or
"created at block X" was saved. The tx and utxo files, however, are
not compatible and should be deleted (or an error will occur on read).
If any errors occur opening the utxo file, a rescan will start
beginning at the creation block saved in the wallet.
This change automatically sends the btcdconnected notification for all
frontends as they connect to btcwallet. The old btcdconnected command
has been removed, as it is no longer needed for clients to explicitly
request this information any longer.
This change removes a lot of unnecessary and complicated locking (if
serializing requests is needed in the future, a goroutine will be used
instead) and also shifts the heavy lifting from frontends to btcwallet
itself to handle any notifications when they can be properly handled.
Although it's still legal to, frontends no longer need to explicitly
request account balances as these are calculated and sent as an async
notification on frontend connect, and these notifications will only
occur if btcd is currently connected. Likewise, when btcd connects,
all frontends are immediately notified of all notifications that
require btcd information, such as the current block height for
calculating account balances.
CalculateBalance now works correctly: if confirmations is 0, all UTXOs
will be used for the balance. Otherwise, unconfirmed UTXOs will be
exclused. 1 confirmation will allow the UTXO height and current block
height to be equal. Even though the difference is zero, the
transaction including the UTXO has been mined into one block.
This change also remove extraneous account balance notifications for
connected and disconnected blocks.
Any and all resending should be handled directly by btcd, and btcd
ignores any duplicate transactions when adding to mempool anyways. A
set of unmined txs is still kept and send to btcd in case of btcd
restarting and losing wallet transactions from its mempool.
In case of a btcd restart, it is necessary to send unmined
transactions back to btcd so they can be added to the tx mempool.
btcd can make a best-try effort, but It is ultimately btcwallet's
responsibility that transactions appear in blocks even if btcd is
restarted.
