Commit graph

383 commits

Author SHA1 Message Date
Josh Rickmar
04a6cb2176 go fmt 2014-05-06 13:22:57 -05:00
Josh Rickmar
361a74fcaf Rename listen option to rpclisten.
This matches the use of the rpclisten option in btcd to specify the
listening interfaces and ports for an RPC server.
2014-05-06 12:50:39 -05:00
Josh Rickmar
909091984b Update notifyspent requests to take multiple outpoints. 2014-05-06 08:38:23 -05:00
Josh Rickmar
437b4cbdbe Add missing license to top of source file. 2014-05-05 17:34:55 -05:00
Josh Rickmar
e9bdf2a094 Another day, another tx store implementation.
The last transaction store was a great example of how not to write
scalable software.  For a variety of reasons, it was very slow at
processing transaction inserts.  Among them:

1) Every single transaction record being saved in a linked list
   (container/list), and inserting into this list would be an O(n)
   operation so that records could be ordered by receive date.

2) Every single transaction in the above mentioned list was iterated
   over in order to find double spends which must be removed.  It is
   silly to do this check for mined transactions, which already have
   been checked for this by btcd.  Worse yet, if double spends were
   found, the list would be iterated a second (or third, or fourth)
   time for each removed transaction.

3) All spend tracking for signed-by-wallet transactions was found on
   each transaction insert, even if the now spent previous transaction
   outputs were known by the caller.

This list could keep going on, but you get the idea.  It was bad.

To resolve these issues a new transaction store had to be implemented.
The new implementation:

1) Tracks mined and unmined transactions in different data structures.
   Mined transactions are cheap to track because the required double
   spend checks have already been performed by the chain server, and
   double spend checks are only required to be performed on
   newly-inserted mined transactions which may conflict with previous
   unmined transactions.

2) Saves mined transactions grouped by block first, and then by their
   transaction index.  Lookup keys for mined transactions are simply
   the block height (in the best chain, that's all we save) and index
   of the transaction in the block.  This makes looking up any
   arbitrary transaction almost an O(1) operation (almost, because
   block height and block indexes are mapped to their slice indexes
   with a Go map).

3) Saves records in each transaction for whether the outputs are
   wallet credits (spendable by wallet) and for whether inputs debit
   from previous credits.  Both structures point back to the source
   or spender (credits point to the transaction that spends them, or
   nil for unspent credits, and debits include keys to lookup the
   transaction credits they spent.  While complicated to keep track
   of, this greatly simplifies the spent tracking for transactions
   across rollbacks and transaction removals.

4) Implements double spend checking as an almost O(1) operation.  A
   Go map is used to map each previous outpoint for all unconfirmed
   transactions to the unconfirmed tx record itself.  Checking for
   double spends on confirmed transaction inserts only involves
   looking up each previous outpoint of the inserted tx in this map.
   If a double spend is found, removal is simplified by only
   removing the transaction and its spend chain from store maps,
   rather than iterating a linked list several times over to remove
   each dead transaction in the spend chain.

5) Allows the caller to specify the previous credits which are spent
   by a debiting transaction.  When a transaction is created by
   wallet, the previous outputs are already known, and by passing
   their record types to the AddDebits method, lookups for each
   previously unspent credit are omitted.

6) Bookkeeps all blocks with transactions with unspent credits, and
   bookkeeps the transaction indexes of all transactions with unspent
   outputs for a single block.  For the case where the caller adding a
   debit record does not know what credits a transaction debits from,
   these bookkeeping structures allow the store to only consider known
   unspent transactions, rather than searching through both spent and
   unspents.

7) Saves amount deltas for the entire balance as a result of each
   block, due to transactions within that block.  This improves the
   performance of calculating the full balance by not needing to
   iterate over every transaction, and then every credit, to determine
   if a credit is spent or unspent.  When transactions are moved from
   unconfirmed to a block structure, the amount deltas are incremented
   by the amount of all transaction credits (both spent and unspent)
   and debited by the total amount the transaction spends from
   previous wallet credits.  For the common case of calculating a
   balance with just one confirmation, the only involves iterating
   over each block structure and adding the (possibly negative)
   amount delta.  Coinbase rewards are saved similarly, but with a
   different amount variable so they can be seperatly included or
   excluded.

Due to all of the changes in how the store internally works, the
serialization format has changed.  To simplify the serialization
logic, support for reading the last store file version has been
removed.  Past this change, a rescan (run automatically) will be
required to rebuild the transaction history.
2014-05-05 16:12:05 -05:00
Josh Rickmar
e956d0b290 Immediately terminate improperly auth'd ws conns.
This fixes a bug with the authentication handling for websocket
clients where it was possible that even after supplying bad
authentication using the HTTP Authorization header, the connection
would remain open and flagged as unauthenticated, and clients (if they
somehow knew auth failed, although btcwallet would never tell them
until after they failed their next request) could try their hand at
authorization again by issuing an authenticate request.

While I don't know of any reason the above described bug could result
in a security leak, it's better to fail the connection as soon as
possible if they failed their first authentication attempt.

While here, also set a read deadline of 10 seconds for the first
request.  If the initial handshake cannot complete in this timeframe,
the connection is terminated.  This matches the behavior in btcd, and
prevents websocket clients from connecting without the Authorization
header and never issuing their first authenticate request.
2014-04-28 18:19:45 -05:00
David Hill
6a908d63bc Handle disconnecting clients properly.
Also, add logging for connecting and disconnecting clients.
2014-04-22 16:03:17 -04:00
David Hill
6b24abfdad Code cleanup.
- Additional error checking
- Use the stack for small data sizes to avoid garbage collection
- Use io.ReadFull vs Read to detect underflows
2014-04-16 17:22:39 -04:00
Dave Collins
bbb24db42c Update for notifynewtxs to notifyreceived rename.
The websocket extension command to register for notifications when an
address receives funds has been renamed.  This commit catches up to the
change.

ok @jrick
2014-04-14 22:42:29 -05:00
Josh Rickmar
69dbad5999 Use btcchain constant for coinbase maturity. 2014-04-14 08:51:47 -05:00
Josh Rickmar
51fb9ad619 Use confirms func to find number of confirmations. 2014-04-13 23:06:25 -05:00
Josh Rickmar
2bb41582c9 Fix listsinceblock to consider target confirms.
Closes #80.
2014-04-12 12:27:59 -05:00
David Hill
d9432fe8ad Have listsinceblock return a btcjson result type.
ok @jrick
2014-04-11 20:33:15 -04:00
Josh Rickmar
1b69fd63ff Fix a test.
Calling Bytes() on a big.Int strips any leading padding zeros.  This
change fixes the test to always pad the byte slice for a private key
to a length of 32.
2014-04-11 15:18:54 -05:00
Josh Rickmar
00403c7839 Use []Type{} instead of make([]Type, 0). 2014-04-11 13:58:04 -05:00
David Hill
9cfa95d269 gofmt 2014-04-11 14:52:50 -04:00
David Hill
f08da980f0 Cleanup TxInfo 2014-04-11 14:51:41 -04:00
David Hill
880c1cbaeb Make gettransaction return a btcjson GetTransactionResult type 2014-04-11 14:10:27 -04:00
Dave Collins
677ec10ee7 Convert all tx results to use new concrete type.
This commit modifies all code paths which work with transaction result
objects to use the concrete ListTransactionsResult provided by the btcjson
package.  This provides nicer marshalling and unmarshalling as well as
access to properly typed fields.
2014-04-10 21:15:35 -05:00
Josh Rickmar
5194254715 Fixes for btcws API changes. 2014-04-10 21:12:19 -05:00
Josh Rickmar
66b02e7b77 Move GetBestBlockResult to btcws. 2014-04-10 16:43:11 -05:00
Owain G. Ainsworth
fba205f27e Use AccountByAddress() to save iteration. 2014-04-09 22:42:33 +01:00
Owain G. Ainsworth
674e9f2427 Rework wallet apis somewhat.
- Instead of returning a special constructed type whenever queries for an
address.  Return the internal object with an immutable external
interface.

- Make the private key gettable from PubKeyAddress to prevent having to look up
multiple times to get information from the same structure

- Enforce addresses always have public keys.
2014-04-09 22:40:28 +01:00
Josh Rickmar
da840f3855 Add a comment for FinishUnmarshal. 2014-04-09 13:02:12 -05:00
David Hill
936d922579 Return a btcjson result type for createmultisig.
ok @jrick
2014-04-09 13:57:28 -04:00
Josh Rickmar
b1a71d5f83 Delay JSON unmarshaling until needed.
This change takes advantage of the RawMessage type in the
encoding/json package to defer unmarshaling of all JSON-RPC values
until absolutely necessary.

This is particularly important for request passthrough when btcwallet
must ask btcd to handle a chain request for a wallet client.  In the
previous code, during the marshal and unmarshal dance to set the
original client's request id in the btcd response, large JSON numbers
were being mangled to use (scientific) E notation even when they could
be represented as a integer without any loss of precision.
2014-04-09 12:50:16 -05:00
Josh Rickmar
6216012aac Use btcjson result type for listunspent. 2014-04-08 22:35:39 -05:00
Josh Rickmar
a1b731f69e Remove old, commented-out code.
Found by @davecgh.
2014-04-08 22:09:22 -05:00
Josh Rickmar
abbe457ddc Kill last MarkAddressForAccount call and func. 2014-04-08 17:49:46 -05:00
Josh Rickmar
902bbd1111 Report correct change address after composing txs. 2014-04-08 17:49:02 -05:00
Josh Rickmar
eb567f2590 Remove some unused vars. 2014-04-07 20:04:39 -05:00
David Hill
08c23d858d fix up validateaddress to use a btcjson.ValidateAddressResult.
Also fix setting the account and hex fields.

ok @jrick
2014-04-07 14:21:20 -04:00
Owain G. Ainsworth
35bd7ef6d9 rework the account manager somewhat.
- Move the MarkAddresForAccount and LookupAccountByAddress functionality
into account maanger.

- Move the wallet opeing logic into account manager (the only place that calls
it) and unexport.

- Move accountHandler to using a single channel for commands. Many of
the commands have ordering restraints (add account, list all accounts,
remove account, access account, mark account for address) which are very
much undefined with the multi-channel model.

- Rework all callers of LookupAccountByAddress to get the account structure
directly.
2014-04-03 17:52:16 +01:00
Owain G. Ainsworth
391b269d06 Implement addmultisigaddress and createmultisig. 2014-04-03 17:52:15 +01:00
Josh Rickmar
3f40e256c2 Return correct JSON object for listunspent.
This change fixes the reply for listunspent to return a JSON object in
the same format as done by the reference implementation.  Previously,
listunspent would return an array of the same objects as returned for
listtransactions.
2014-03-31 10:11:37 -05:00
Josh Rickmar
189df5c535 Mark partially synced addresses.
This change adds a notification handler for the new rescanprogress
notification and takes advantage of the recent rescan manager and
partial syncing support to mark addresses as partially synced.  If the
network connection to btcd is lost or wallet is restarted during a
rescan, a new rescan will start at the earliest block height for any
wallet address, taking partial syncs into consideration.
2014-03-28 11:28:59 -05:00
Josh Rickmar
aa0980bfa7 Simplify logic in WalletRequestProcessor. 2014-03-27 22:18:23 -05:00
Josh Rickmar
53e4070a5a Support partial syncing of addresses in wallet format.
This change reappropriates the unused `last block` field from Armory's
wallet format to hold the block chain height for a partially synced
address, that is, an address that has been partially synced to
somewhere between its first seen block and the most recently seen
block.  The wallet's SyncHeight method has been updated to return
partial heights as well.

The actual marking of partially unsynced address from a rescan
progress update is not implemented yet.
2014-03-27 13:48:40 -05:00
Josh Rickmar
9444fdb985 Make a comment understandable. 2014-03-26 21:15:56 -05:00
Josh Rickmar
d4bee3656c Avoid unnecessarily calculating a rescan job twice. 2014-03-26 21:13:10 -05:00
Josh Rickmar
59845d9c21 Implement a batching rescan manager.
Recent btcd versions only allow one rescan to run at any given time
per websocket client.  To better handle this, a new set of goroutines
are started by the account manager which batch and serialize rescan
jobs.

If no rescans are currently running, a new rescan starts.  If a rescan
is already being processed, the request is queued and runs after the
current rescan finishes.  For any additional incoming requests before
the current rescan finishes, the requests are merged with the
currently-waiting request so both can be handled with a single rescan.

This change also prepares for rescan progress notifications from btcd,
but are still unhandled until the necessary details for
partially-synced addresses are added to the wallet file format.
2014-03-26 17:27:30 -05:00
Jimmy Song
e22d221ea8 Issue #65: Give the correct error when wallet is locked
When sending coins to an address with a wallet that's both
locked and has insufficient funds, the correct ErrWalletLocked
error will be returned.
2014-03-25 16:38:31 -05:00
Josh Rickmar
c9ff0531f9 Bootstrap rescan requests with utxo set. 2014-03-21 15:36:42 -05:00
Josh Rickmar
6024e0ecb6 Pad byte slice passed to btcutil.EncodePrivateKey.
Calling the Bytes method for a big.Int does not pad the result to
required size for EncodePrivateKey.  This change adds the leading
padding, preventing seemingly-random "malformed private key" errors
from being returned to users of dumpprivkey.
2014-03-21 12:47:10 -05:00
Josh Rickmar
19fd6406e8 Prevent a send on closed chan panic.
The select statement does not guarantee selecting a better case if one
might panic for sending to a closed channel.  This case was hit during
client disconnect due to having multiple senders on a single channel
with one of the senders closing the chan to notify the next goroutine
to finish.  This change gives each writes its own unique channel to
prevent this error.
2014-03-20 16:10:42 -05:00
Josh Rickmar
4f1d2e7121 Add support for the authenticate extension request.
Just like btcd, this commit adds support for the authenticate request
allowing clients unable to set the HTTP Authorization header to
authenticate to use the RPC server.  The rules for the authenticate
request are as follows:

1) Authentication make clients stateful.  Clients may either be flagged
   as authenticated or unauthenticated.

2) Clients may authenticate by exactly one of two possible ways,
   either by setting the Authorization header or by sending a JSON-RPC
   authenticate request as follows:

   {
     "jsonrpc":"1.0",
     "id":0,
     "method":"authenticate",
     "params":["rpcuser", "rpcpass"]
   }

3) When not authenticated by the Authorization header, the first request
   must be an authenticate request.

4) Sending an authenticate request after a client has already
   successfully authenticated (either by the Authorization header or a
   previous authentication request) is invalid.

5) The result used in the response to a successful authenticate request
   is a JSON null.  For any unsuccessful or invalid authenticate
   requests, the connection is terminated.

This change also orders all incoming requests for a client.  This was
required to ensure that any authentication requests are processed
first.
2014-03-20 14:27:52 -05:00
Jimmy Song
c51cbb3332 Refactor len(w.secret) != 32
Now using w.IsLocked() for all instances of above.
Also changed one other place where the logic had to be reversed
in nextChainedAddress (len(w.secret) == 32 was the condition).
2014-03-20 11:37:54 -05:00
Owain G. Ainsworth
6dea3789cb update for btcutil.DecodeAddress api change. 2014-03-19 01:47:12 +00:00
Josh Rickmar
ec0d84882a Make tests compile after API change. 2014-03-18 17:52:08 -05:00
Josh Rickmar
29fef9ffd2 Fix nil pointer dereference.
Ran into by @toddfries and brought up on IRC.
2014-03-18 16:41:01 -05:00