This commit improves how the headers-first mode works in several ways.
The previous headers-first code was an initial implementation that did not
have all of the bells and whistles and a few less than ideal
characteristics. This commit improves the heaers-first code to resolve
the issues discussed next.
- The previous code only used headers-first mode when starting out from
block height 0 rather than allowing it to work starting at any height
before the final checkpoint. This means if you stopped the chain
download at any point before the final checkpoint and restarted, it
would not resume and you therefore would not have the benefit of the
faster processing offered by headers-first mode.
- Previously all headers (even those after the final checkpoint) were
downloaded and only the final checkpoint was verified. This resulted in
the following issues:
- As the block chain grew, increasingly larger numbers of headers were
downloaded and kept in memory
- If the node the node serving up the headers was serving an invalid
chain, it wouldn't be detected until downloading a large number of
headers
- When an invalid checkpoint was detected, no action was taken to
recover which meant the chain download would essentially be stalled
- The headers were kept in memory even though they didn't need to be as
merely keeping track of the hashes and heights is enough to provde they
properly link together and checkpoints match
- There was no logging when headers were being downloaded so it could
appear like nothing was happening
- Duplicate requests for the same headers weren't being filtered which
meant is was possible to inadvertently download the same headers twice
only to throw them away.
This commit resolves these issues with the following changes:
- The current height is now examined at startup and prior each sync peer
selection to allow it to resume headers-first mode starting from the
known height to the next checkpoint
- All checkpoints are now verified and the headers are only downloaded
from the current known block height up to the next checkpoint. This has
several desirable properties:
- The amount of memory required is bounded by the maximum distance
between to checkpoints rather than the entire length of the chain
- A node serving up an invalid chain is detected very quickly and with
little work
- When an invalid checkpoint is detected, the headers are simply
discarded and the peer is disconnected for serving an invalid chain
- When the sync peer disconnets, all current headers are thrown away
and, due to the new aforementioned resume code, when a new sync peer
is selected, headers-first mode will continue from the last known good
block
- In addition to reduced memory usage from only keeping information about
headers between two checkpoints, the only information now kept in memory
about the headers is the hash and height rather than the entire header
- There is now logging information about what is happening with headers
- Duplicate header requests are now filtered
Previously the logging function which reports on progress was called for
every block, regardless of whether it was an orphan or not. This could be
confusing since it could show a different number of blocks processed as
compared to the old versus new heights reported (orphans do not add to the
block height since they aren't extending the main chain). Further, the
database had to be consulted for the latest block since the block we just
processed might not be the latest one if it was an orphan. This is quite
a bit more time conusming than it should've been for progress reporting.
This commit modifies that to only include non-orphan blocks. As a result,
the latest height shown will match the number of blocks processed (even
when there are orphans) and the additional block lookup from the database
is avoided.
The getinfo RPC method requires access to information only available in
the wallet. Therefore, it has been moved to the list of methods which
return an error information the caller to send the request to the wallet
instead.
The getnewaddress RPC method deals with wallet-related functionality and
therefore has been moved to the list of methods which return an error
information the caller to send the request to the wallet instead.
Both of these RPC methods require access to information ony available in
the wallet. Therefore they have been moved to the list of methods which
return an error information the caller to send the request to the wallet
instead.
Rather than having the iterator functions a separate entities that access
the state to iterate, just expose the iterators as receivers on the state
itself. This is more consistent with the style used throughout the code
and the other receivers on the state such as Count, OutboundCount, etc.
This commit adds full support for the getaddednodeinfo RPC command
including DNS lookups which abide by proxy/onion/tor rules when the DNS
flag is specified. Note that it returns an array of strings when the DNS
flag is not set which is different than the current version of bitcoind
which is bugged and scheduled to be fixed per issue 3581 on the bitcoind
issue tracker.
This commit improves how the legacy RPC server responds to authentication
failures so things like web browsers can react better. The following
changes have been made:
First, authentication failures were only printing the 401 error response
in the body instead of setting the http status code. This means the
response had a 200 OK header with a body of 401 Unauthorized. Therefore
the client would think everything was ok, but see the response as
malformed JSON.
Second, the spec for 401 Unauthorized responses state they must include a
WWW-Authenticate header to instruct the client how to authenticate.
Without this, browsers won't prompt the user for credentials.
The previous websocket code required HTTP auth headers to be sent in order
to use the websocket. While this makes sense for most applications, some
use cases such as javascript-based websockets from browsers do no have the
ability to send headers.
This commit modifies the authentication logic to allow an alternative
authentication mechanism when HTTP auth headers can't be sent. In
particular, it introduces a new JSON-RPC command named authenticate which
accepts the username and passphrase for the RPC server. The command is
only required if the connetion has not already been authenticated via HTTP
auth headers and it must be the first command to be received. Sending any
other command will immediately disconnect the websocket.
ok from @owainga and @jrick.
This closes#77.
This fixes two issues: first, the sendrawtransaction handler had an
extra character in the key in the websocket handler map, preventing
the handler from never running. Second, a nil pointer dereference was
removed from the handler.
This change fixes the minedtx notifications for btcwallet, since the
websocket-handler now runs instead of falling back to the legacy RPC
handler.
This commit adds the btcdb memdb backend as a supported database type.
Note that users will NOT want to run in this mode because, being memory
only, it obviously does not persist the database when shutdown.
It is being added for testing purposes to help prevent constant abuse to
developer's hard drive when churning the block database multiple times a
day.
This commit switches the handleGetHeadersMsg function to make use of the
new FetchBlockHeightBySha and FetchBlockHeaderBySha functions in btcdb.
Also, while here, nuke the header copy which is no longer required due to
the recent btcwire changes.
This commit reduces the initial idle timeout before version negotiation
has happened on a new peer to 30 seconds. Previously it could take 5
minutes due to the general idle timeout.
The websocket.Server used the by websocket.Handler type automatically adds
a handshake function which prevents connections when the Origin header is
not set. Not all clients send this information and we already require
authentication headers as the auth mechanism anyways.
