This refactors the RPC server to accept and take ownership of already
configured listeners and refactors the logic to setup those listeners to
the server. This mirrors the logic used by the connection manager and
is desirable since it is another step closer to being able to split the
RPC server code out into a separate package and will make it much easier
to internally test since it allows creating mock listeners.
This modifies all of the RPC code to use the chain parameters that are
associated with the RPC server instead of the global activeNetParams and
thus moves one step closer to being able to split the RPC server out
into a separate package.
This decouples the RPC server from the internal btcd server to move
closer to being able to split it out into a separate package.
In order to accomplish this, it introduces an rpcserverConfig type and
several new interfaces, named rpcserverPeer, rpcserverConnManager, and
rpcserverBlockManager, which are necessary to break the direct
dependencies on the main server and block manager instances.
It also adds concrete implementations of the new interfaces and uses
them to configure the RPC server.
Ultimately, the RPC server should ideally be decoupled even more such
that all of the types in the configuration struct use interfaces instead
of the concrete types. Doing this would make the RPC server much easier
to internally test since it would allow creating lightweight stubs for
the various pieces.
Instead of having the block manager notify the RPC server about
accepted, connected, and disconnected blocks, the RPC server will
directly listen for notifications from the blockchain.
This makes the code for getheaders more consistent with the rest of the
code in terms of making use of existing error functions and using the
same RPC error codes as other handlers.
While here, it also performs the fetching of headers directly instead of
using a function from server which makes it more tightly coupled.
This replaces the ErrDoubleSpend and ErrMissingTx error codes with a
single error code named ErrMissingTxOut and updates the relevant errors
and expected test results accordingly.
Once upon a time, the code relied on a transaction index, so it was able
to definitively differentiate between a transaction output that
legitimately did not exist and one that had already been spent.
However, since the code now uses a pruned utxoset, it is no longer
possible to reliably differentiate since once all outputs of a
transaction are spent, it is removed from the utxoset completely.
Consequently, a missing transaction could be either because the
transaction never existed or because it is fully spent.
This commit adds set of BIP0009 (Version Bits) deployment parameters
for all networks detailing the activation parameters for the segwit
soft-fork.
Additionally, the BIP0009 integration test has been updated to test for
the proper transitioning of version bits state for the segwit soft
fork. Finally, the `getblockchaininfo` test has also been updated to
properly display the state of segwit.
This commit implements the new “weight” metric introduced as part of
the segwit soft-fork. Post-fork activation, rather than limiting the
size of blocks and transactions based purely on serialized size, a new
metric “weight” will instead be used as a way to more accurately
reflect the costs of a tx/block on the system. With blocks constrained
by weight, the maximum block-size increases to ~4MB.
Version 0.15.0 of Bitcoin Core will include a new RPC command that will
allow us to obtain the amount of time (in seconds) that the server has
been running.
The btclog package has been changed to defining its own logging
interface (rather than seelog's) and provides a default implementation
for callers to use.
There are two primary advantages to the new logger implementation.
First, all log messages are created before the call returns. Compared
to seelog, this prevents data races when mutable variables are logged.
Second, the new logger does not implement any kind of artifical rate
limiting (what seelog refers to as "adaptive logging"). Log messages
are outputted as soon as possible and the application will appear to
perform much better when watching standard output.
Because log rotation is not a feature of the btclog logging
implementation, it is handled by the main package by importing a file
rotation package that provides an io.Reader interface for creating
output to a rotating file output. The rotator has been configured
with the same defaults that btcd previously used in the seelog config
(10MB file limits with maximum of 3 rolls) but now compresses newly
created roll files. Due to the high compressibility of log text, the
compressed files typically reduce to around 15-30% of the original
10MB file.
This modifies the blockNode and BestState structs in the blockchain
package to store hashes directly instead of pointers to them and updates
callers to deal with the API change in the exported BestState struct.
In general, the preferred approach for hashes moving forward is to store
hash values in complex data structures, particularly those that will be
used for cache entries, and accept pointers to hashes in arguments to
functions.
Some of the reasoning behind making this change is:
- It is generally preferred to avoid storing pointers to data in cache
objects since doing so can easily lead to storing interior pointers
into other structs that then can't be GC'd
- Keeping the hash values directly in the block node provides better
cache locality
This modifies the block node structure to include a couple of extra
fields needed to be able to reconstruct the block header from a node,
and exposes a new function from chain to fetch the block headers which
takes advantage of the new functionality to reconstruct the headers from
memory when possible. Finally, it updates both the p2p and RPC servers
to make use of the new function.
This is useful since many of the block header fields need to be kept in
order to form the block index anyways and storing the extra fields means
the database does not have to be consulted when headers are requested if
the associated node is still in memory.
The following timings show representative performance gains as measured
from one system:
new: Time to fetch 100000 headers: 59ms
old: Time to fetch 100000 headers: 4783ms
A DNS lookup was being attempted on onion addresses causing
connections to fail. This has been fixed by introducing type
onionAddr (which implements a net.Addr interface) and passing
it to btcdDial.
Also, the following onion related fixes have been made:
* getaddednodeinfo - updated to handle onion addrs.
* TorLookupIP - fixed err being shadowed.
* newServer - rename tcpAddr to netAddr
* addrStringToNetAddr - skip if host is already an IP addr.
* addrStringToNetAddr - err if tor is disabled
* getaddednodeinfo - check if host is already an IP addr.
This allows a caller-provided tag to be associated with orphan
transactions. This is useful since the caller can use the tag for
purposes such as keeping track of which peers orphans were first seen
from.
Also, since a parameter is required now anyways, it associates the peer
ID with processed transactions from remote peers.
This moves the error check for an attempt to call the generate RPC when
on a network that there is virtually no chance of mining a block with
the CPU into the RPC server where it more naturally belongs. The caller
of the CPU should be the one to determine if it wants to allow mining or
not. While here, use a more accurate RPC error code of ErrDifficulty
instead of ErrInternal.
This change is a step towards being able to separate the CPU mining code
into its own subpackage.
This modifies the NewMsgTx function to accept the transaction version as
a parameter and updates all callers.
The reason for this change is so the transaction version can be bumped
in wire without breaking existing tests and to provide the caller with
the flexibility to create the specific transaction version they desire.
This does the minimum work necessary to refactor the block template
generation code into the mining package. The idea is that separating
this code into the mining package will greatly improve its testability,
allow independent benchmarking and profiling, and open up some
interesting opportunities for future development related to mining.
There are some areas related to policy and other configuration that
could be further refactored, however it is better to do that in future
commits in order to keep the changeset as small as possible during this
refactor.
Overview of the major changes:
- Move mining.go -> mining/mining.go
- Move mining_test.go -> mining/mining_test.go
- Add logger to mining package
- Update the MINR subsystem to use the new mining package logger
- Export CoinbaseFlags from the mining package
- BlkTmplGenerator is now mining.BlkTmplGenerator
- Update all references to the mining code to use the package
This introduces a new type named BlkTmplGenerator which encapsulates the
various state needed to generate block templates.
This is useful since it means code that needs to generate block
templates can simply accept the generator rather than needing access to
all of the additional state which in turn will ultimately make it easier
to split the mining code into its own package.
Avoid compatibility issues with software that relies on the behavior of
bitcoind's JSON-RPC implementation.
The JSON-RPC 1.0 spec defines that notifications must have their "id"
set to null and states that notifications do not have a response.
A JSON-RPC 2.0 notification is a request with "json-rpc":"2.0", and
without an "id" member. The specification states that notifications
must not be responded to. JSON-RPC 2.0 permits the null value as a
valid request id, therefore such requests are not notifications.
Bitcoin Core serves requests with "id":null or even an absent "id", and
responds to such requests with "id":null in the response.
Btcd does not respond to any request without and "id" or with "id":null,
regardless the indicated JSON-RPC protocol version.
In order to avoid compatibility issues with software relying on
Core's behavior, this commit implements "quirks mode" as follows:
- quirks mode can be enabled via configuration (disabled by default)
- If no JSON-RPC version is indicated in the request, accept and
respond to request with "id":null
- If no JSON-RPC version is indicated in the request, accept and
respond to requests without an "id" member
- In both cases above, use "id":null in the response
- Do not respond to request without an "id" or with "id":null when
JSON-RPC version is indicated in the request (process as notification)
This does the minimum work necessary to refactor the mempool code into
its own package. The idea is that separating this code into its own
package will greatly improve its testability, allow independent
benchmarking and profiling, and open up some interesting opportunities
for future development related to the memory pool.
There are likely some areas related to policy that could be further
refactored, however it is better to do that in future commits in order
to keep the changeset as small as possible during this refactor.
Overview of the major changes:
- Create the new package
- Move several files into the new package:
- mempool.go -> mempool/mempool.go
- mempoolerror.go -> mempool/error.go
- policy.go -> mempool/policy.go
- policy_test.go -> mempool/policy_test.go
- Update mempool logging to use the new mempool package logger
- Rename mempoolPolicy to Policy (so it's now mempool.Policy)
- Rename mempoolConfig to Config (so it's now mempool.Config)
- Rename mempoolTxDesc to TxDesc (so it's now mempool.TxDesc)
- Rename txMemPool to TxPool (so it's now mempool.TxPool)
- Move defaultBlockPrioritySize to the new package and export it
- Export DefaultMinRelayTxFee from the mempool package
- Export the CalcPriority function from the mempool package
- Introduce a new RawMempoolVerbose function on the TxPool and update
the RPC server to use it
- Update all references to the mempool to use the package.
- Add a skeleton README.md
This rewrites the shutdown logic to simplify the shutdown signalling.
All cleanup is now run from deferred functions in the main function and
channels are used to signal shutdown either from OS signals or from
other subsystems such as the RPC server and windows service controller.
The RPC server has been modified to use a new channel for signalling
shutdown that is exposed via the RequestedProcessShutdown function
instead of directly calling Stop on the server as it previously did.
Finally, it adds a few checks for early termination during the main
start sequence so the process can be stopped without starting all the
subsystems if desired.
This is a backport of the equivalent logic from Decred with a few slight
modifications. Credits go to @jrick.
This moves several of the chain constants to the Params struct in the
chaincfg package which is intended for that purpose. This is mostly a
backport of the same modifications made in Decred along with a few
additional things cleaned up.
The following is an overview of the changes:
- Comment all fields in the Params struct definition
- Add locals to BlockChain instance for the calculated values based on
the provided chain params
- Rename the following param fields:
- SubsidyHalvingInterval -> SubsidyReductionInterval
- ResetMinDifficulty -> ReduceMinDifficulty
- Add new Param fields:
- CoinbaseMaturity
- TargetTimePerBlock
- TargetTimespan
- BlocksPerRetarget
- RetargetAdjustmentFactor
- MinDiffReductionTime
This is mostly a backport of some of the same modifications made in
Decred along with a few additional things cleaned up. In particular,
this updates the code to make use of the new chainhash package.
Also, since this required API changes anyways and the hash algorithm is
no longer tied specifically to SHA, all other functions throughout the
code base which had "Sha" in their name have been changed to Hash so
they are not incorrectly implying the hash algorithm.
The following is an overview of the changes:
- Remove the wire.ShaHash type
- Update all references to wire.ShaHash to the new chainhash.Hash type
- Rename the following functions and update all references:
- wire.BlockHeader.BlockSha -> BlockHash
- wire.MsgBlock.BlockSha -> BlockHash
- wire.MsgBlock.TxShas -> TxHashes
- wire.MsgTx.TxSha -> TxHash
- blockchain.ShaHashToBig -> HashToBig
- peer.ShaFunc -> peer.HashFunc
- Rename all variables that included sha in their name to include hash
instead
- Update for function name changes in other dependent packages such as
btcutil
- Update copyright dates on all modified files
- Update glide.lock file to use the required version of btcutil
It is not the responsibility of mempool to relay transactions, so
return a slice of transactions accepted to the mempool due to the
passed transaction to the caller.
This introduces a new indexing infrastructure for supporting optional
indexes using the new database and blockchain infrastructure along with
two concrete indexer implementations which provide both a
transaction-by-hash and a transaction-by-address index.
The new infrastructure is mostly separated into a package named indexers
which is housed under the blockchain package. In order to support this,
a new interface named IndexManager has been introduced in the blockchain
package which provides methods to be notified when the chain has been
initialized and when blocks are connected and disconnected from the main
chain. A concrete implementation of an index manager is provided by the
new indexers package.
The new indexers package also provides a new interface named Indexer
which allows the index manager to manage concrete index implementations
which conform to the interface.
The following is high level overview of the main index infrastructure
changes:
- Define a new IndexManager interface in the blockchain package and
modify the package to make use of the interface when specified
- Create a new indexers package
- Provides an Index interface which allows concrete indexes to plugin
to an index manager
- Provides a concrete IndexManager implementation
- Handles the lifecycle of all indexes it manages
- Tracks the index tips
- Handles catching up disabled indexes that have been reenabled
- Handles reorgs while the index was disabled
- Invokes the appropriate methods for all managed indexes to allow
them to index and deindex the blocks and transactions
- Implement a transaction-by-hash index
- Makes use of internal block IDs to save a significant amount of
space and indexing costs over the old transaction index format
- Implement a transaction-by-address index
- Makes use of a leveling scheme in order to provide a good tradeoff
between space required and indexing costs
- Supports enabling and disabling indexes at will
- Support the ability to drop indexes if they are no longer desired
The following is an overview of the btcd changes:
- Add a new index logging subsystem
- Add new options --txindex and --addrindex in order to enable the
optional indexes
- NOTE: The transaction index will automatically be enabled when the
address index is enabled because it depends on it
- Add new options --droptxindex and --dropaddrindex to allow the indexes
to be removed
- NOTE: The address index will also be removed when the transaction
index is dropped because it depends on it
- Update getrawtransactions RPC to make use of the transaction index
- Reimplement the searchrawtransaction RPC that makes use of the address
index
- Update sample-btcd.conf to include sample usage for the new optional
index flags
This commit is the first stage of several that are planned to convert
the blockchain package into a concurrent safe package that will
ultimately allow support for multi-peer download and concurrent chain
processing. The goal is to update btcd proper after each step so it can
take advantage of the enhancements as they are developed.
In addition to the aforementioned benefit, this staged approach has been
chosen since it is absolutely critical to maintain consensus.
Separating the changes into several stages makes it easier for reviewers
to logically follow what is happening and therefore helps prevent
consensus bugs. Naturally there are significant automated tests to help
prevent consensus issues as well.
The main focus of this stage is to convert the blockchain package to use
the new database interface and implement the chain-related functionality
which it no longer handles. It also aims to improve efficiency in
various areas by making use of the new database and chain capabilities.
The following is an overview of the chain changes:
- Update to use the new database interface
- Add chain-related functionality that the old database used to handle
- Main chain structure and state
- Transaction spend tracking
- Implement a new pruned unspent transaction output (utxo) set
- Provides efficient direct access to the unspent transaction outputs
- Uses a domain specific compression algorithm that understands the
standard transaction scripts in order to significantly compress them
- Removes reliance on the transaction index and paves the way toward
eventually enabling block pruning
- Modify the New function to accept a Config struct instead of
inidividual parameters
- Replace the old TxStore type with a new UtxoViewpoint type that makes
use of the new pruned utxo set
- Convert code to treat the new UtxoViewpoint as a rolling view that is
used between connects and disconnects to improve efficiency
- Make best chain state always set when the chain instance is created
- Remove now unnecessary logic for dealing with unset best state
- Make all exported functions concurrent safe
- Currently using a single chain state lock as it provides a straight
forward and easy to review path forward however this can be improved
with more fine grained locking
- Optimize various cases where full blocks were being loaded when only
the header is needed to help reduce the I/O load
- Add the ability for callers to get a snapshot of the current best
chain stats in a concurrent safe fashion
- Does not block callers while new blocks are being processed
- Make error messages that reference transaction outputs consistently
use <transaction hash>:<output index>
- Introduce a new AssertError type an convert internal consistency
checks to use it
- Update tests and examples to reflect the changes
- Add a full suite of tests to ensure correct functionality of the new
code
The following is an overview of the btcd changes:
- Update to use the new database and chain interfaces
- Temporarily remove all code related to the transaction index
- Temporarily remove all code related to the address index
- Convert all code that uses transaction stores to use the new utxo
view
- Rework several calls that required the block manager for safe
concurrency to use the chain package directly now that it is
concurrent safe
- Change all calls to obtain the best hash to use the new best state
snapshot capability from the chain package
- Remove workaround for limits on fetching height ranges since the new
database interface no longer imposes them
- Correct the gettxout RPC handler to return the best chain hash as
opposed the hash the txout was found in
- Optimize various RPC handlers:
- Change several of the RPC handlers to use the new chain snapshot
capability to avoid needlessly loading data
- Update several handlers to use new functionality to avoid accessing
the block manager so they are able to return the data without
blocking when the server is busy processing blocks
- Update non-verbose getblock to avoid deserialization and
serialization overhead
- Update getblockheader to request the block height directly from
chain and only load the header
- Update getdifficulty to use the new cached data from chain
- Update getmininginfo to use the new cached data from chain
- Update non-verbose getrawtransaction to avoid deserialization and
serialization overhead
- Update gettxout to use the new utxo store versus loading
full transactions using the transaction index
The following is an overview of the utility changes:
- Update addblock to use the new database and chain interfaces
- Update findcheckpoint to use the new database and chain interfaces
- Remove the dropafter utility which is no longer supported
NOTE: The transaction index and address index will be reimplemented in
another commit.
This simply exports and adds some comments to the fields of the
BlockTemplate struct.
This is primarily being done as a step toward being able to separate the
mining code into its own package, but also it makes sense on its own
because code that requests new block template necessarily examines the
returned fields which implies they should be exported.
Dynamic ban scores consist of a persistent and a decaying component. The
persistent score can be used to create simple additive banning policies
simlar to those found in other bitcoin node implementations. The
decaying score enables the creation of evasive logic which handles
misbehaving peers (especially application layer DoS attacks) gracefully
by disconnecting and banning peers attempting various kinds of flooding.
Dynamic ban scores allow these two approaches to be used in tandem.
This pull request includes the following:
- Dynamic ban score type & functions, with tests for core functionality
- Ban score of connected peers can be queried via rpc (getpeerinfo)
- Example policy with decaying score increments on mempool and getdata
- Logging of misbehavior once half of the ban threshold is reached
- Banning logic can be disabled via configuration (enabled by default)
- User defined ban threshold can be set via configuration
This optimizes the filter address handling code in the RPC server
handleSearchRawTransasctions path in a few ways:
- Only normalize the filter addresses provided in the command once
rather than for every transaction being returned
- Reset the passes filter flag just before it's used
- Use a local for the encoded address to avoid the bounds checking
overhead of accessing it from the slice
- Avoiding subsequent map lookups once the filter has already passed
Also, while here, modify a few of the related comments to match code
style and consistency.
This creates a skeleton mining package that simply contains a few of the
definitions used by the mining and mempool code.
This is a step towards decoupling the mining code from the internals of
btcd and ultimately will house all of the code related to creating block
templates and CPU mining.
The main reason a skeleton package is being created before the full
blown package is ready is to avoid blocking mempool separation which
relies on these type definitions.
This introduces the concept of a new interface named TxSource which aims
to generically provide a concurrent safe source of transactions to be
considered for inclusion in a new block. This is a step towards
decoupling the mining code from the internals of btcd. Ultimately the
intent is to create a separate mining package.
The new TxSource interface relies on a new struct named miningTxDesc,
which describes each entry in the transaction source. Once this code is
refactored into a separate mining package, the mining prefix can simply
be dropped leaving the type exported as mining.TxDesc.
To go along with this, the existing TxDesc type in the mempool has been
renamed to mempoolTxDesc and changed to embed the new miningTxDesc type.
This allows the mempool to efficiently implement the MiningTxDescs
method needed to satisfy the TxSource interface.
This approach effectively separates the direct reliance of the mining
code on the mempool and its data structures. Even though the memory
pool will still be the default concrete implementation of the interface,
making it an interface offers much more flexibility in terms of testing
and even provides the potential to allow more than one source (perhaps
multiple independent relay networks, for example).
Finally, the memory pool and all of the mining code has been updated to
implement and use the new interface.
This does three things:
- Splits the priority calculation logic from the TxDesc type
- Modifies the calcPriority function to perform the value age
calculation instead of accepting it as a parameter
- Changes the starting priority to be calculated when the transaction is
added to the pool
The first is useful as it is a step towards decoupling the mining code
from the internals of the memory pool. Also, the concept of priority is
related to mining policy, so it makes more sense to have the
calculations separate than being defined on the memory pool tx
descriptor.
The second change has been made because everywhere that uses the
calcPriority function first has to calculate the value age anyways and
by making it part of the function it can be avoided altogether in
certain circumstances thereby provided a bit of optimization.
The third change ensure the starting priority is safe for reentrancy
which will be important once the mempool is split into a separate
package.
This introduces the concept of a mining policy struct which is used to
control block template generation instead of directly accessing the
config struct. This is a step toward decoupling the mining code from
the internals of btcd. Ultimately the intent is to create a separate
mining package.
rpcserver:
If the locktime is given, the transaction inputs will be set to a
non-max value, activating the locktime. The locktime for the
new transaction will be set to the given value.
This mimics Bitcoin Core commit 212bcca92089f406d9313dbe6d0e1d25143d61ff
This commit introduces package peer which contains peer related features
refactored from peer.go.
The following is an overview of the features the package provides:
- Provides a basic concurrent safe bitcoin peer for handling bitcoin
communications via the peer-to-peer protocol
- Full duplex reading and writing of bitcoin protocol messages
- Automatic handling of the initial handshake process including protocol
version negotiation
- Automatic periodic keep-alive pinging and pong responses
- Asynchronous message queueing of outbound messages with optional
channel for notification when the message is actually sent
- Inventory message batching and send trickling with known inventory
detection and avoidance
- Ability to wait for shutdown/disconnect
- Flexible peer configuration
- Caller is responsible for creating outgoing connections and listening
for incoming connections so they have flexibility to establish
connections as they see fit (proxies, etc.)
- User agent name and version
- Bitcoin network
- Service support signalling (full nodes, bloom filters, etc.)
- Maximum supported protocol version
- Ability to register callbacks for handling bitcoin protocol messages
- Proper handling of bloom filter related commands when the caller does
not specify the related flag to signal support
- Disconnects the peer when the protocol version is high enough
- Does not invoke the related callbacks for older protocol versions
- Snapshottable peer statistics such as the total number of bytes read
and written, the remote address, user agent, and negotiated protocol
version
- Helper functions for pushing addresses, getblocks, getheaders, and
reject messages
- These could all be sent manually via the standard message output
function, but the helpers provide additional nice functionality such
as duplicate filtering and address randomization
- Full documentation with example usage
- Test coverage
In addition to the addition of the new package, btcd has been refactored
to make use of the new package by extending the basic peer it provides to
work with the blockmanager and server to act as a full node. The
following is a broad overview of the changes to integrate the package:
- The server is responsible for all connection management including
persistent peers and banning
- Callbacks for all messages that are required to implement a full node
are registered
- Logic necessary to serve data and behave as a full node is now in the
callback registered with the peer
Finally, the following peer-related things have been improved as a part
of this refactor:
- Don't log or send reject message due to peer disconnects
- Remove trace logs that aren't particularly helpful
- Finish an old TODO to switch the queue WaitGroup over to a channel
- Improve various comments and fix some code consistency cases
- Improve a few logging bits
- Implement a most-recently-used nonce tracking for detecting self
connections and generate a unique nonce for each peer
Also, update TravisCI goclean script to remove the special casing which
ignored 'Id' from the lint output since that exception is no longer
needed. It was previously required due to the old version of btcjson,
but that is no longer in the repo.
Introduce an ECDSA signature verification into btcd in order to
mitigate a certain DoS attack and as a performance optimization.
The benefits of SigCache are two fold. Firstly, usage of SigCache
mitigates a DoS attack wherein an attacker causes a victim's client to
hang due to worst-case behavior triggered while processing attacker
crafted invalid transactions. A detailed description of the mitigated
DoS attack can be found here: https://bitslog.wordpress.com/2013/01/23/fixed-bitcoin-vulnerability-explanation-why-the-signature-cache-is-a-dos-protection/
Secondly, usage of the SigCache introduces a signature verification
optimization which speeds up the validation of transactions within a
block, if they've already been seen and verified within the mempool.
The server itself manages the sigCache instance. The blockManager and
txMempool respectively now receive pointers to the created sigCache
instance. All read (sig triplet existence) operations on the sigCache
will not block unless a separate goroutine is adding an entry (writing)
to the sigCache. GetBlockTemplate generation now also utilizes the
sigCache in order to avoid unnecessarily double checking signatures
when generating a template after previously accepting a txn to the
mempool. Consequently, the CPU miner now also employs the same
optimization.
The maximum number of entries for the sigCache has been introduced as a
config parameter in order to allow users to configure the amount of
memory consumed by this new additional caching.
This commit optimizes the createVinList function which is used to
generate the JSON list of transaction inputs. It also makes it more
consistent with the createVinListPrevOut function.
In particular, it entails the following changes:
- Only do a single coinbase check and return right away instead of
checking multiple times inside the loop over the inputs
- Use a pointer for populating the details of each entry to avoid
multiple unnecessary array lookups and bounds checks
- Group all fields that populate the entry for better readability
This commit converts all block height references to int32 instead of
int64. The current target block production rate is 10 mins per block
which means it will take roughly 40,800 years to reach the maximum
height an int32 affords. Even if the target rate were lowered to one
block per minute, it would still take roughly another 4,080 years to
reach the maximum.
In the mean time, there is no reason to use a larger type which results
in higher memory and disk space usage. However, for now, in order to
avoid having to reserialize a bunch of database information, the heights
are still serialized to the database as 8-byte uint64s.
This is being mainly being done in preparation for further upcoming
infrastructure changes which will use the smaller and more efficient
4-byte serialization in the database as well.
This commit modifies the createTxRawResult code path along with callers
to work with block headers as opposed to btcutil.Blocks. This in turn
allows the code in handleGetRawTransaction and
handleSearchRawTransactions to perform a much cheaper block header load
as opposed to a full block load.
While here, also very slightly optimize the createVinList function to
avoid creating a util.Tx wrapper and to take advantage of the
btcutil.Amount type added after the function was originally written
Create GenerateCmd in btcjson v2. Update tests to check GenerateCmd.
Update chaincfg/params.go with a new bool in Params, GenerateSupported,
with true values in SimNetParams and RegressionNetParams and false in
the others.
Create new flag, discreteMining, in CPUMiner struct.
Add GenerateNBlocks function to cpuminer.go and handleGenerate
function to rpcserver.go.
Update documentation for the RPC calls.
* The cases for the 'addnode' command were previously
stacked on top the new cases for the 'node' command.
The intended behavior was to create a fall through and
handle both commands. However, trying to use this
syntax with a type switch caused the first case to be
ignored.
* addnode' specific functions and structs in the server
have been removed. Instead, the 'add' and 'del' subcommands
are now proxied to the matching 'node' cmd functions.
This commit removes the error returns from the BlockHeader.BlockSha,
MsgBlock.BlockSha, and MsgTx.TxSha functions since they can never fail and
end up causing a lot of unneeded error checking throughout the code base.
It also updates all call sites for the change.
* Gives node operators full control of peer connectivity
* RPC adds ability to disconnect all matching non-persistent peers,
remove persistent peers, and connect to peers making them either
temporary or persistent.
The limited user is specified with the --rpclimituser and
--rpclimitpass options (or the equivalent in the config file).
The config struct and loadConfig() are updated to take the
new options into account. The limited user can have neither
the same username nor the same password as the admin user.
The package-level rpcLimit map in rpcserver.go specifies
the RPC commands accessible by limited users. This map
includes both HTTP/S and websocket commands.
The checkAuth function gets a new return parameter to
signify whether the user is authorized to change server
state. The result is passed to the jsonRPCRead function and
to the WebsocketHandler function in rpcwebsocket.go.
The wsClient struct is updated with an "isAdmin" field
signifying that the client is authorized to change server
state, written by WebsocketHandler and handleMessage.
The handleMessage function also checks the field to
allow or disallow an RPC call.
The following documentation files are updated:
- doc.go
- sample-btcd.conf
- docs/README.md
- docs/json_rpc_api.md
- docs/configure_rpc_server_listen_interfaces.md
Fix#303 by changing the addrindex key prefix to 3 characters so that
it's easy to check length when dropping the index. To drop the old
index, check to make sure we aren't dropping any entries that end in
"sx" or "tx" as those aren't part of the addrindex. Update test to
deal with the new prefix length.
Fix#346 by changing the pointers in the mempool's addrindex map to
wire.ShaHash 32-byte values. This lets them be deleted even if the
transaction data changes places in memory upon expanding the maps.
Change the way addrindex uint32s are stored to big-endian in order to
sort the transactions on disk in chronological/dependency order.
Change the "searchrawtransactions" RPC call to return transactions
from the database before the memory pool so that they're returned in
order. This commit DOES NOT do topological sorting of the memory pool
transactions to ensure they're returned in dependency order. This may
be a good idea for a future enhancement.
Add addrindex versioning to automatically drop the old/incompatible
version of the index and rebuild with the new sort method and key
prefix.
When the fields in the command for the getnetworkhashps RPC don't have the
fields set, use the intended default values.
Since the btcjson package sets these fields to the default values when a
command is unmarshaled from the wire, this typically isn't necessary.
However, when the RPC server calls the handler internally with optional
command fields set to nil, as is the case in getmininginfo, the defaults
need to be set as well.
This commit updates the SearchRawTransactionsCmd verbose parameter in the
latest version of btcjson to an integer to match recent changes to the
previous version of btcjson.
This commit converts the RPC server over to use the new features available
in the latest version of btcjson and improve a few things along the way.
This following summarizes the changes:
- All btcjson imports have been updated to the latest package version
- The help has been significantly improved
- Invoking help with no command specified now provides an alphabetized
list of all supported commands along with one-line usage
- The help for each command is automatically generated and provides much
more explicit information such as the type of each parameter, whether
or not it's optional or required, etc
- The websocket-specific commands are now provided when accessing the
help when connected via websockets
- Help has been added for all websocket-specific commands and is only
accessible when connected via websockets
- The error returns and handling of both the standard and websocket
handlers has been made consistent
- All RPC errors have been converted to the new RPCError type
- Various variables have been renamed for consistency
- Several RPC errors have been improved
- The commands that are marked as unimplemented have been moved into the
separate map where they belong
- Several comments have been improved
- An unnecessary check has been removed from the createrawtransaction
handler
- The command parsing has been restructured a bit to pave the way for
JSON-RPC 2.0 batching support
- When invalid HTTP Basic Access Authentication details are provided to
the websocket endpoint, return a WWW-Authenticate header just like the
non-websocket endpoint
- When a connection to the websocket endpoint fails to properly upgrade,
return a 400 Bad Request HTTP error status code
This commit reorders the RPC server code to more closely match the
ordering used by the rest of the code base such that functions are
typically defined before they are used and near their call site.
* Address index is built up concurrently with the `--addrindex` flag.
* Entire index can be deleted with `--dropaddrindex`.
* New RPC call: `searchrawtransaction`
* Returns all transacitons related to a particular address
* Includes mempool transactions
* Requires `--addrindex` to be activated and fully caught up.
* New `blockLogger` struct has been added to factor our common logging
code
* Wiki and docs updated with new features.
