All rpc sockets now listen using TLS by default, and this can not be
turned off. The keys (defauling to the datadirectory) may be provided by
--rpccert and --rpckey. If the keys do not exist we will generate a new
self-signed keypair with some sane defaults (hostname and all current
interface addresses).
Additionally add tls capability to btcctl so that it can still be used.
The certificate to use for verify can be provided on the commandline or
verification can be turned off (this leaves you susceptible to MITM
attacks)
Initial code from dhill (rpc tls support) and jrick (key generation),
cleanup, debugging and polishing from me.
This allows the provision of address/port pairs to be listened on instead
of just providing the port. e.g.:
btcd --listen 1.2.3.4:4321 --listen 127.0.0.01 --listen [::1]:5432
When --proxy and --connect are used, we disable listening *unless* any --listen
arguments have been provided, when we will listen on those addresses as
requested.
Initial code by davec, integration by myself.
Closes#33
allow listens to fail, but warn. error if all failed
fmt
So far we only do level 0 and level 1 checks (precense and basic
sanity). The checks done at higher levels in bitcoind are closely
coupled with their database layout.
arguably Closes#13
Previously, RemoveMinedTxRequest was being run from a caller which
held a reader lock for the websocket request contexts. When
RemoveMinedTxRequest tried to grab a writer lock, it would block.
This change creates a new function, removeMinedTxRequest, that does
not grab any locks, and the caller (NotifyBlockConnected) grabs a
writer lock instead of a reader lock.
Previously, on a blockconnected notification, the websocket context
reader lock was not always being given up properly. This change
defers the unlock so it will always happen.
This fixes an issue where wallet will stop responding (due to not
being able to complete its handshake) on reconnect.
This change reworks where the command parsing occurs to be done before
handlers are checked. Before, the websocket extension handler called
the standard handler with the same message, and if it was unhandled,
would unmarshal it a second time for checking extension handlers.
Redo the datastructures we search so that we only do one lookup per txin and
txout instead of doing a loop per wallet connection.
Don't send spent data on tx notifications, this can be worked out in wallet and
it is expensiveish to calculate. However we DO check upon getting a notification
request if the output is already spent, and in which case we send an immediate
notification to force a rescan.
MinedTxNotfications are handled separately to the connected block messages
largely to enable this to scale rather better.
Tested by jrick (who found one bug i had introduced, thanks!)
Additionally (accidentally squashed in):
Add handlers for all known commands.
We have handlers for all wallet-requiring commands that will return a suitable
error.
Unimplemented commands temporarily return an error stating so.
This change allows map lookups using address hashes (which are
returned as []byte) instead of either copying the hash into an array,
or doing a bytes.Equal().
A stupid range over a map until the right key is found was also just
changed to a single map lookup.
This adds to the initial rescan implementation, but switches it to
rescan based on a group of addresses, rather than just one. Due to
how expensive database lookups are during a rescan, wallets should
take advantage of this to rescan once for all needed addresses for all
accounts.
This commit changes the various cases that were serializing transactions
into a buffer and taking the length to use the new faster SerializeSize
API. It also completes a TODO since the serialized size of a transaction
output is now available.
Rather than showing all errors from ProcessTransaction as an error, check
if the error is a TxRuleError meaning the transaction was rejected as
opposed to something actually going wrong and log it accordingly.
Looking up transactions from the database is an expensive operation.
This commit modifies the NotifyNewTxListener code to simply iterate the
transactions in the block instead of looking them up from the db.
Currently the wallet code needs a spent flag which ultimately shouldn't be
required. For now, the spent data is simply created on the fly which is
still significantly faster than doing database transaction lookups.
Closes#24.
This change unbreaks the case where an unknown command is sent to the
RPC server. Instead of replying back with a nil JSON id, if the
initial unmarshal was successful (and thus, the message was valid
JSON-RPC), the unmarshaled id will be used in the error reply.
We have a channel for queries and commands in server, where we pass in
args and the channel to reply from, let rpcserver use these interfaces
to provide the requistie information.
So far not all of the informaation is 100% correct, the syncpeer
information needs to be fetched from blockmanager, the subversion isn't
recorded and the number of bytes sent and recieved needs to be obtained
from btcwire. The rest should be correct.
This commit updates btcd to work with the new btcchain APIs which now
accept btcutil.Tx instead of raw btcwire.MsgTx. It also modifies the
transaction memory pool to store btcutil.Tx.
This is part of the ongoing transaction hash optimization effort noted in
conformal/btcd#25.
This change allows btcwallet to keep a pool of transactions that have
not yet been mined into a block, notifying wallet when transactions
are mined, as well as introducing a new way to send the
btcd:blockconnected notification with wallet-specific information as
part of the same notification. When a transaction is sent using the
RPC call 'sendrawtransaction', a notification request will be
automatically registered with the connected wallet (if using
websockets) to notify the wallet when the transaction first appears in
a block.
To perform this notification, and to avoid requiring wallets from
waiting for seperate mined tx notifications (and resend after a
timeout) or from sending an additional tx mined request for every tx
in the pool after each new block, the blockconnected notification is
now created seperately for each wallet. If the notified wallet has
sent a transaction, an additional JSON field "minedtxs" will include
an array of transaction IDs that the wallet has created and which are
included in the new block.
This new unique blockconnected notification can also be used for
additional notifications that may happen each new block in the future,
and to cut down on existing notification handlers in btcwallet, such
as for transactions to a watched address.
Rather than relying on the http package's DefaultServeMux for the RPC
server, create a unique mux specifically for the RPC server. This ensures
things, such as the http profiling handlers, do not commingle.
Results from FetchTxByShaList must each be checked for a nil Err and a
non-nil Tx. Fix this issue in two places where these conditions were
not being checked.
The latest websockets code added a quit channel to the RPC server, but did
not initialize it. This commit corrects that so shutdown works properly
again.
This refactors the wallet notification code to reverse the order of
how notification contexts are stored. Before, watched addresses and
outpoints were used as keys, with a special reply channel as the
value. This channel was read from and replies were marshalled and
sent to the main wallet notification chan, but the goroutine handling
this marshalling never exited because the reply channel was never
closed (and couldn't have been, because there was no way to tell it
was handling notifications for any particular wallet).
Notification contexts are now primarily mapped by wallet notification
channels, and code to send the notifications send directly to the
wallet channel, with the previous goroutine reading the reply chan
properly closing.
The RPC code is also refactored with this change as well, to separate
it more from websocket code. Websocket JSON extensions are no longer
available to RPC clients.
While here, unbreak RPC. Previously, replies were never sent back.
This broke when I merged in my websocket code, as sends for the reply
channel in jsonRead blocked before a reader for the channel was
opened. A 3 liner could have fixed this, but doing a proper fix
(changing jsonRead so it did not use the reply channel as it is
unneeded for the standard RPC API) is preferred.
This change adds additional http listeners for websocket connections
on "/wallet". Websockets are used to provide asynchronous messaging
between wallet daemons (i.e. btcwallet) and btcd as they allow an easy
way for btcd to provide instant notifications (instead of a wallet
polling for updates) and multiple replies to a single request.
Standard RPC commands sent over a websocket connection are handled
just like RPC, returning the same results, the only difference being
that the connection is async. In cases where the standard RPC
commands fall short of wallet daemons requests, and to request
notifications for addresses and events, extension JSON methods are
used.
Multiple wallets can be connected to the same btcd, and replies to
websocket requests and notifications are properly routed back to the
original requesting wallet.
Due to the nature of turning a synchronous protocol asynchronous, it
is highly recommended to use the JSON id field as a type of sequence
number, so replies from btcd can be routed back to the proper handler
in a wallet daemon.
This commit is a first pass at improving the logging. It changes a number
of things to improve the readability of the output. The biggest addition
is message summaries for each message type when using the debug logging
level.
There is sitll more to do here such as allowing the level of each
subsystem to be independently specified, syslog support, and allowing the
logging level to be changed run-time.
This commit changes the code so that all calls to .Add on waitgroups
happen before the associated goroutines are launched. Doing this after
the goroutine could technically cause a race where the goroutine started
and finished before the main goroutine has a chance to increment the
counter. In our particular case none of the goroutines exit quickly
enough for this to be an issue, but nevertheless the correct way should be
used.
