Recent btcd versions only allow one rescan to run at any given time
per websocket client. To better handle this, a new set of goroutines
are started by the account manager which batch and serialize rescan
jobs.
If no rescans are currently running, a new rescan starts. If a rescan
is already being processed, the request is queued and runs after the
current rescan finishes. For any additional incoming requests before
the current rescan finishes, the requests are merged with the
currently-waiting request so both can be handled with a single rescan.
This change also prepares for rescan progress notifications from btcd,
but are still unhandled until the necessary details for
partially-synced addresses are added to the wallet file format.
Calling the Bytes method for a big.Int does not pad the result to
required size for EncodePrivateKey. This change adds the leading
padding, preventing seemingly-random "malformed private key" errors
from being returned to users of dumpprivkey.
The select statement does not guarantee selecting a better case if one
might panic for sending to a closed channel. This case was hit during
client disconnect due to having multiple senders on a single channel
with one of the senders closing the chan to notify the next goroutine
to finish. This change gives each writes its own unique channel to
prevent this error.
Just like btcd, this commit adds support for the authenticate request
allowing clients unable to set the HTTP Authorization header to
authenticate to use the RPC server. The rules for the authenticate
request are as follows:
1) Authentication make clients stateful. Clients may either be flagged
as authenticated or unauthenticated.
2) Clients may authenticate by exactly one of two possible ways,
either by setting the Authorization header or by sending a JSON-RPC
authenticate request as follows:
{
"jsonrpc":"1.0",
"id":0,
"method":"authenticate",
"params":["rpcuser", "rpcpass"]
}
3) When not authenticated by the Authorization header, the first request
must be an authenticate request.
4) Sending an authenticate request after a client has already
successfully authenticated (either by the Authorization header or a
previous authentication request) is invalid.
5) The result used in the response to a successful authenticate request
is a JSON null. For any unsuccessful or invalid authenticate
requests, the connection is terminated.
This change also orders all incoming requests for a client. This was
required to ensure that any authentication requests are processed
first.
Now using w.IsLocked() for all instances of above.
Also changed one other place where the logic had to be reversed
in nextChainedAddress (len(w.secret) == 32 was the condition).
This change slightly improves the goroutines managing sends and
receives for a btcd connection by improving the logging (logging the
exact errors that caused the connection to be lost) as well as
cleaning up the shutdown handling by closing the websocket connection
for any fail and closing the response channel when no more responses
can be read.
The private key import codepath (called when handling the
importprivkey RPC method) was not triggering rescans for the imported
address. This change begins a new rescan for each import and adds
additional logic to the wallet file to keep track of unsynced imported
addresses. After a rescan on an imported address completes, the
address is marked as in sync with the rest of wallet and future
handshake rescans will start from the last seen block, rather than the
import height of the unsynced address.
While here, improve the logging for not just import rescans, but
rescanning on btcd connect (part of the handshake) as well.
Fixes#74.
Move the stuff that scripts can't possibly support out of the interface
and move about two type assertions so that everything still works. They
key-using interfaces can be made into a KeyedAddress itnerface if we add
any more.
Add a walletAddress interface to handle the differences betweent he
different types. Stop using btcutil.AddressPubKeyHash everywhere and just use
the standard address.
Shortly we will add new types of address, so make AddressInfo an
interface, with concrete types providing address-specific information.
Adapt existing code to this new status quo.
Notifications ariving from btcd were being reordered (each handled by
its own goroutine, rather then being always sent in the order they
originated). This was breaking the new transaction store by inserting
transaction records in an 'impossible' manner, that is, inserting txs
without block info after the store already held records of the same tx
with block info, without first performing a rollback.
This is handled by the transaction store insert methods by checking
for identical transactions (double spends with the same tx sha), but
where the block heights mismatch and the new record does not have a
block set. The error is returned all the way up to the goroutine
running each rpc request/notification handler, and if hit, the btcd
connection is closed and all accounts are reopened from disk. This is
not optimal, but it allows us to use the connect logic to correctly
catch us up to the best chain with the last good state of all accounts
while only rescanning a few blocks.
Fixes#72.
This change replaces the old transaction store file format and
implementation. The most important change is how the full backing
transactions for any received or sent transaction are now saved,
rather than simply saving parsed-out details of the tx (tx shas, block
height/hash, pkScripts, etc.).
To support the change, notifications for received transaction outputs
and txs spending watched outpoints have been updated to use the new
redeemingtx and recvtx notifications as these contain the full tx,
which is deserializead and inserted into the store.
The old transaction store serialization code is completely removed, as
updating to the new format automatically cannot be done. Old wallets
first running past this change will error reading the file and start a
full rescan to rebuild the data. Unlike previous rescan code,
transactions spending outpoint managed by wallet are also included.
This results in recovering not just received history, but history for
sent transactions as well.
It appears that the websocket package will occasionally enter a
Codec's Send function and block forever, never erroring (presumably
due to a closed connection). This change adds a deadline for the send
of two seconds. If the send cannot complete before the deadline is
reached, the send is aborted and the connection is assumed to be lost.
A buffer should be added here as well, so even waiting max two seconds
for the send to error out won't cause wallet code to block.
At any instant when a duplicated notification must be sent, either one
of two channel sends/recvs must occur. The first possibility is that
the client is disconnected, in which case the disconnected channel
will be read, and then the context removed from the goroutine-managed
map. The second possibility is that the disconnect channel has not
yet been closed, in which case it must block on an actual message
send. This change moves the second case out of the default case of
the select statement to avoid a race where:
1) The client has not yet disconnected, and the disconnected chan is
not ready for reads.
2) Control switches to the default case.
3) The client disconnects, the goroutine reading the send channel
returns, closes the disconnected channel, and no more reads occur.
4) The notification duplicator blocks forever trying to send the
message even when the disconnected notification channel has
already been closed.
This shouldn't be nececssary (a TODO was added to remind me to make
clients explicitly ask for this info) but in the meantime this fixes
clients such as btcgui which otherwise wouldn't think btcwallet is
properly connected to btcd and will desensitise some widgets.
The disk syncer now maintains its own countdown timer, creating a new
timer only when necessary (when there is no timer running, and
something is scheduled to be written). When the timer expires, the
select loop begins selecting on a grab of the account manager's binary
semaphore, and if read, performs the sync and nils the select channel
to prevent a future grab until a new timer has expired.
Tested with a race-enabled build on Windows. No lockups or races
related to the disk syncing experienced with constant client requests
and incoming btcd notifications, and scheduled writes run as expected
once the countdown timer expires, locking out all server request and
notifiation handling.
