This change immediately writes a new empty transaction store out to
disk if the old one could not be read. Since old transaction store
versions are not read in at start, and were previously not written out
until new transaction history was received, it was possible that a
full rescan started and finished without ever marking a synced tx
history for the next wallet start.
If a rescan fails (for example, due to a disconnected btcd) in the
btcd handshake, the last block height from a rescanprogress
notification should be used for the next rescan job on next wallet
connect. Previously, this rescan would always start at the earliest
block height for any wallet address if the transaction store could not
be read at wallet startup. This change unsets the boolean flag which
would cause a full rescan at next connect when a rescan progress
notification is received and a partial sync height is written.
Fixes#87.
If an unexpected error is encounted when creating the encrypted
wallet, rather than using btcjson.ErrInternal, wrap the error message
using btcjson.ErrWallet.Code.
18555 is the network port, and wallet cannot listen on this port.
Instead, follow the pattern laid out by mainnet, testnet3, and regtest
by listening on 18554 (one less than the network port).
If the data directory is modified on the command line or from the
config file, all paths relative to the directory, if unmodified, must
be changed to reference it.
This change is the result of using the errcheck tool
(https://github.com/kisielk/errcheck) to find all unchecked errors,
both unassigned and those assigned to the blank identifier.
Every returned error is now handled in some manner. These include:
- Logging errors that would otherwise be missed
- Returning errors to the caller for further processing
- Checking error values to determine what to do next
- Panicking for truely exceptional "impossible" errors
On the subject of panics, they are a sharp tool and should be used
sparingly. That being said, I have added them to check errors that
were previously explicitly ignored, because they were expected to
always return without failure. This could be due to fake error paths
(i.e. writing to a bytes.Buffer panics for OOM and should never return
an error) or previous logic asserts that an error case is impossible.
Rather than leaving these unhandled and letting code fail later,
either with incorrect results or a nil pointer dereference, it now
produces a stack trace at the error emit site, which I find far more
useful when debugging.
While here, a bunch of dead code was removed, including code to move
pre-0.1.1 uxto and transaction history account files to the new
directory (as they would be unreadable anyways) and a big chunk of
commented out rpcclient code.
So (SigHashAll & SigHashSingle)!= 0, which is not the intention here. fix up
that check to only match SigHashSingle.
Found by drahn, debugged together, fix by me.
This is an intial pass at converting the btcwallet and deps codebases
to pass a network by their parameters, rather than by a magic number
to identify the network. The parameters in params.go have been
updated to embed a *btcnet.Params, and all previous uses of cfg.Net()
have been replaced with activeNet.{Params,Net} (where activeNet is
the global var for the active network).
Although dependancy packages have not yet been updated from using
btcwire.BitcoinNet to btcnet.Params, the parameters are now accessible
at all callsites, and individual packages can be updated to use btcnet
without requiring updates in each external btc* package at once.
While here, the exported API for btcwallet internal library packages
(txstore and wallet) have been updated to pass full network parameters
rather than the btcwire definition of a network.
This change adds the new btcdusername and btcdpassword options which,
if set, are used instead of the username and password when
authenticating to a btcd RPC server. If these new options are unset,
the btcd user and password settings are shared with the client auth
settings.
Instead of using 3 fallthroughs with obscure cases, use a single
switch statement with just a one case. This switch is only evaluated
if a previous if statement body is entered. Functionally no
different, but imo this much easier to read, and removes two uses of !
to negate bools.
The category for a received coinbase output should be "generate" for a
mature coinbase (one that has reached btcchain.CoinbaseMaturity
confirmations), or "immature" if the required number of confirmations
has not been reached yet. New Confirmed and Confirmations methods
have been added to the transaction store's TxRecord type to check if
the required number of confirmations have been met for coinbase
outputs.
While here, update the main package to use the new TxRecord methods,
rather than duplicating the confirmation checking code in two places.
The connect option is already used by btcd to force a connection to
other full node peers. Wallet does not talk directly with these
peers, so the connect option is being renamed to something unique for
an RPC client connection.
The last transaction store was a great example of how not to write
scalable software. For a variety of reasons, it was very slow at
processing transaction inserts. Among them:
1) Every single transaction record being saved in a linked list
(container/list), and inserting into this list would be an O(n)
operation so that records could be ordered by receive date.
2) Every single transaction in the above mentioned list was iterated
over in order to find double spends which must be removed. It is
silly to do this check for mined transactions, which already have
been checked for this by btcd. Worse yet, if double spends were
found, the list would be iterated a second (or third, or fourth)
time for each removed transaction.
3) All spend tracking for signed-by-wallet transactions was found on
each transaction insert, even if the now spent previous transaction
outputs were known by the caller.
This list could keep going on, but you get the idea. It was bad.
To resolve these issues a new transaction store had to be implemented.
The new implementation:
1) Tracks mined and unmined transactions in different data structures.
Mined transactions are cheap to track because the required double
spend checks have already been performed by the chain server, and
double spend checks are only required to be performed on
newly-inserted mined transactions which may conflict with previous
unmined transactions.
2) Saves mined transactions grouped by block first, and then by their
transaction index. Lookup keys for mined transactions are simply
the block height (in the best chain, that's all we save) and index
of the transaction in the block. This makes looking up any
arbitrary transaction almost an O(1) operation (almost, because
block height and block indexes are mapped to their slice indexes
with a Go map).
3) Saves records in each transaction for whether the outputs are
wallet credits (spendable by wallet) and for whether inputs debit
from previous credits. Both structures point back to the source
or spender (credits point to the transaction that spends them, or
nil for unspent credits, and debits include keys to lookup the
transaction credits they spent. While complicated to keep track
of, this greatly simplifies the spent tracking for transactions
across rollbacks and transaction removals.
4) Implements double spend checking as an almost O(1) operation. A
Go map is used to map each previous outpoint for all unconfirmed
transactions to the unconfirmed tx record itself. Checking for
double spends on confirmed transaction inserts only involves
looking up each previous outpoint of the inserted tx in this map.
If a double spend is found, removal is simplified by only
removing the transaction and its spend chain from store maps,
rather than iterating a linked list several times over to remove
each dead transaction in the spend chain.
5) Allows the caller to specify the previous credits which are spent
by a debiting transaction. When a transaction is created by
wallet, the previous outputs are already known, and by passing
their record types to the AddDebits method, lookups for each
previously unspent credit are omitted.
6) Bookkeeps all blocks with transactions with unspent credits, and
bookkeeps the transaction indexes of all transactions with unspent
outputs for a single block. For the case where the caller adding a
debit record does not know what credits a transaction debits from,
these bookkeeping structures allow the store to only consider known
unspent transactions, rather than searching through both spent and
unspents.
7) Saves amount deltas for the entire balance as a result of each
block, due to transactions within that block. This improves the
performance of calculating the full balance by not needing to
iterate over every transaction, and then every credit, to determine
if a credit is spent or unspent. When transactions are moved from
unconfirmed to a block structure, the amount deltas are incremented
by the amount of all transaction credits (both spent and unspent)
and debited by the total amount the transaction spends from
previous wallet credits. For the common case of calculating a
balance with just one confirmation, the only involves iterating
over each block structure and adding the (possibly negative)
amount delta. Coinbase rewards are saved similarly, but with a
different amount variable so they can be seperatly included or
excluded.
Due to all of the changes in how the store internally works, the
serialization format has changed. To simplify the serialization
logic, support for reading the last store file version has been
removed. Past this change, a rescan (run automatically) will be
required to rebuild the transaction history.
This fixes a bug with the authentication handling for websocket
clients where it was possible that even after supplying bad
authentication using the HTTP Authorization header, the connection
would remain open and flagged as unauthenticated, and clients (if they
somehow knew auth failed, although btcwallet would never tell them
until after they failed their next request) could try their hand at
authorization again by issuing an authenticate request.
While I don't know of any reason the above described bug could result
in a security leak, it's better to fail the connection as soon as
possible if they failed their first authentication attempt.
While here, also set a read deadline of 10 seconds for the first
request. If the initial handshake cannot complete in this timeframe,
the connection is terminated. This matches the behavior in btcd, and
prevents websocket clients from connecting without the Authorization
header and never issuing their first authenticate request.
The websocket extension command to register for notifications when an
address receives funds has been renamed. This commit catches up to the
change.
ok @jrick
