Commit graph

357 commits

Author SHA1 Message Date
David Hill
807d344fe9 Unassign some TODO's 2016-11-15 17:47:33 -06:00
David Hill
d9a674e1b7 btcjson: add ErrRPCClientNodeNotAdded 2016-11-14 18:44:38 -05:00
David Hill
2510baac35 btcd: support feefilter requests.
This only adds support for handling remote peer requests.
2016-11-03 14:47:30 -04:00
Dave Collins
915fa6639b
multi: Simplify code per gosimple linter.
This simplifies the code based on the recommendations of the gosimple
lint tool.
2016-11-03 13:00:35 -05:00
Dave Collins
af524fb3e7
multi: Remove unnecessary convs found by unconvert.
This removes all unnecessary typecast conversions as found by the
unconvert linter.
2016-11-03 11:59:38 -05:00
Dave Collins
e992d55822
mempool: Associated tag with orphan txns.
This allows a caller-provided tag to be associated with orphan
transactions.  This is useful since the caller can use the tag for
purposes such as keeping track of which peers orphans were first seen
from.

Also, since a parameter is required now anyways, it associates the peer
ID with processed transactions from remote peers.
2016-10-28 15:27:57 -05:00
Dave Collins
a755305a2e
rpcserver: Move error check for generate RPC.
This moves the error check for an attempt to call the generate RPC when
on a network that there is virtually no chance of mining a block with
the CPU into the RPC server where it more naturally belongs.  The caller
of the CPU should be the one to determine if it wants to allow mining or
not.  While here, use a more accurate RPC error code of ErrDifficulty
instead of ErrInternal.

This change is a step towards being able to separate the CPU mining code
into its own subpackage.
2016-10-28 10:58:51 -05:00
Dave Collins
14b51fc5f8
multi: Correct misspellings detected by misspell. 2016-10-28 09:43:38 -05:00
Dave Collins
f6ad7eb2c9
wire: Make NewMsgTx accept the tx version.
This modifies the NewMsgTx function to accept the transaction version as
a parameter and updates all callers.

The reason for this change is so the transaction version can be bumped
in wire without breaking existing tests and to provide the caller with
the flexibility to create the specific transaction version they desire.
2016-10-27 14:09:29 -05:00
Dave Collins
61ca40e0e9
mining: Refactor template code into mining package.
This does the minimum work necessary to refactor the block template
generation code into the mining package.  The idea is that separating
this code into the mining package will greatly improve its testability,
allow independent benchmarking and profiling, and open up some
interesting opportunities for future development related to mining.

There are some areas related to policy and other configuration that
could be further refactored, however it is better to do that in future
commits in order to keep the changeset as small as possible during this
refactor.

Overview of the major changes:

- Move mining.go -> mining/mining.go
- Move mining_test.go -> mining/mining_test.go
- Add logger to mining package
- Update the MINR subsystem to use the new mining package logger
- Export CoinbaseFlags from the mining package
- BlkTmplGenerator is now mining.BlkTmplGenerator
- Update all references to the mining code to use the package
2016-10-27 11:48:48 -05:00
Dave Collins
74fe2a4dfd
mining: Introduce a block template generator.
This introduces a new type named BlkTmplGenerator which encapsulates the
various state needed to generate block templates.

This is useful since it means code that needs to generate block
templates can simply accept the generator rather than needing access to
all of the additional state which in turn will ultimately make it easier
to split the mining code into its own package.
2016-10-26 15:17:21 -05:00
Dave Collins
2274d36333 bmgr: Remove block manager chain state.
This removes the block manager chain state in favor of using the
blockchain.BlockChain instance now that it is safe for concurrency.
2016-10-26 14:23:14 -05:00
Tibor Bősze
6b8a24918e rpcserver: Improve JSON-RPC compatibility
Avoid compatibility issues with software that relies on the behavior of
bitcoind's JSON-RPC implementation.

The JSON-RPC 1.0 spec defines that notifications must have their "id"
set to null and states that notifications do not have a response.

A JSON-RPC 2.0 notification is a request with "json-rpc":"2.0", and
without an "id" member. The specification states that notifications
must not be responded to. JSON-RPC 2.0 permits the null value as a
valid request id, therefore such requests are not notifications.

Bitcoin Core serves requests with "id":null or even an absent "id", and
responds to such requests with "id":null in the response.

Btcd does not respond to any request without and "id" or with "id":null,
regardless the indicated JSON-RPC protocol version.

In order to avoid compatibility issues with software relying on
Core's behavior, this commit implements "quirks mode" as follows:
 - quirks mode can be enabled via configuration (disabled by default)
 - If no JSON-RPC version is indicated in the request, accept and
respond to request with "id":null
 - If no JSON-RPC version is indicated in the request, accept and
respond to requests without an "id" member
 - In both cases above, use "id":null in the response
 - Do not respond to request without an "id" or with "id":null when
JSON-RPC version is indicated in the request (process as notification)
2016-10-24 13:24:18 -05:00
Tibor Bősze
9799f0e547 rpcserver: Improve JSON-RPC compatibility
In order to avoid compatibility issues with software relying on 
Core's behavior, terminate HTTP POST JSON-RPC responses with a newline.
2016-10-24 13:24:18 -05:00
David Hill
07e1e308f1 rpc: Add localaddr and relaytxes to getpeerinfo 2016-10-19 19:58:50 -04:00
David Hill
3b5bb9fd43 btcjson: Add preciousblock 2016-10-19 14:08:32 -05:00
David Hill
403aaf5cf3 rpcserver: avoid nested decodescript p2sh addrs 2016-10-19 13:59:50 -05:00
Dave Collins
7fac099bee mempool: Refactor mempool code to its own package. (#737)
This does the minimum work necessary to refactor the mempool code into
its own package.  The idea is that separating this code into its own
package will greatly improve its testability, allow independent
benchmarking and profiling, and open up some interesting opportunities
for future development related to the memory pool.

There are likely some areas related to policy that could be further
refactored, however it is better to do that in future commits in order
to keep the changeset as small as possible during this refactor.

Overview of the major changes:

- Create the new package
- Move several files into the new package:
  - mempool.go -> mempool/mempool.go
  - mempoolerror.go -> mempool/error.go
  - policy.go -> mempool/policy.go
  - policy_test.go -> mempool/policy_test.go
- Update mempool logging to use the new mempool package logger
- Rename mempoolPolicy to Policy (so it's now mempool.Policy)
- Rename mempoolConfig to Config (so it's now mempool.Config)
- Rename mempoolTxDesc to TxDesc (so it's now mempool.TxDesc)
- Rename txMemPool to TxPool (so it's now mempool.TxPool)
- Move defaultBlockPrioritySize to the new package and export it
- Export DefaultMinRelayTxFee from the mempool package
- Export the CalcPriority function from the mempool package
- Introduce a new RawMempoolVerbose function on the TxPool and update
  the RPC server to use it
- Update all references to the mempool to use the package.
- Add a skeleton README.md
2016-08-19 11:08:37 -05:00
Dave Collins
044a11c9fc btcd: Simplify shutdown signal handling logic. (#733)
This rewrites the shutdown logic to simplify the shutdown signalling.
All cleanup is now run from deferred functions in the main function and
channels are used to signal shutdown either from OS signals or from
other subsystems such as the RPC server and windows service controller.

The RPC server has been modified to use a new channel for signalling
shutdown that is exposed via the RequestedProcessShutdown function
instead of directly calling Stop on the server as it previously did.

Finally, it adds a few checks for early termination during the main
start sequence so the process can be stopped without starting all the
subsystems if desired.

This is a backport of the equivalent logic from Decred with a few slight
modifications.  Credits go to @jrick.
2016-08-11 13:39:23 -05:00
Dave Collins
a7b35d9f9e chaincfg/blockchain: Parameterize more chain consts. (#732)
This moves several of the chain constants to the Params struct in the
chaincfg package which is intended for that purpose.  This is mostly a
backport of the same modifications made in Decred along with a few
additional things cleaned up.

The following is an overview of the changes:

- Comment all fields in the Params struct definition
- Add locals to BlockChain instance for the calculated values based on
  the provided chain params
- Rename the following param fields:
  - SubsidyHalvingInterval -> SubsidyReductionInterval
  - ResetMinDifficulty -> ReduceMinDifficulty
- Add new Param fields:
  - CoinbaseMaturity
  - TargetTimePerBlock
  - TargetTimespan
  - BlocksPerRetarget
  - RetargetAdjustmentFactor
  - MinDiffReductionTime
2016-08-10 16:02:23 -05:00
Dave Collins
bd4e64d1d4 chainhash: Abstract hash logic to new package. (#729)
This is mostly a backport of some of the same modifications made in
Decred along with a few additional things cleaned up.  In particular,
this updates the code to make use of the new chainhash package.

Also, since this required API changes anyways and the hash algorithm is
no longer tied specifically to SHA, all other functions throughout the
code base which had "Sha" in their name have been changed to Hash so
they are not incorrectly implying the hash algorithm.

The following is an overview of the changes:

- Remove the wire.ShaHash type
- Update all references to wire.ShaHash to the new chainhash.Hash type
- Rename the following functions and update all references:
  - wire.BlockHeader.BlockSha -> BlockHash
  - wire.MsgBlock.BlockSha -> BlockHash
  - wire.MsgBlock.TxShas -> TxHashes
  - wire.MsgTx.TxSha -> TxHash
  - blockchain.ShaHashToBig -> HashToBig
  - peer.ShaFunc -> peer.HashFunc
- Rename all variables that included sha in their name to include hash
  instead
- Update for function name changes in other dependent packages such as
  btcutil
- Update copyright dates on all modified files
- Update glide.lock file to use the required version of btcutil
2016-08-08 14:04:33 -05:00
David Hill
a1bb291b28 mempool: Have ProcessTransaction return accepted transactions. (#547)
It is not the responsibility of mempool to relay transactions, so
return a slice of transactions accepted to the mempool due to the
passed transaction to the caller.
2016-04-14 12:58:09 -05:00
Dave Collins
b580cdb7d3 database: Replace with new version.
This commit removes the old database package, moves the new package into
its place, and updates all imports accordingly.
2016-04-12 14:55:15 -05:00
Dave Collins
7c174620f7 indexers: Implement optional tx/address indexes.
This introduces a new indexing infrastructure for supporting optional
indexes using the new database and blockchain infrastructure along with
two concrete indexer implementations which provide both a
transaction-by-hash and a transaction-by-address index.

The new infrastructure is mostly separated into a package named indexers
which is housed under the blockchain package.  In order to support this,
a new interface named IndexManager has been introduced in the blockchain
package which provides methods to be notified when the chain has been
initialized and when blocks are connected and disconnected from the main
chain.  A concrete implementation of an index manager is provided by the
new indexers package.

The new indexers package also provides a new interface named Indexer
which allows the index manager to manage concrete index implementations
which conform to the interface.

The following is high level overview of the main index infrastructure
changes:

- Define a new IndexManager interface in the blockchain package and
  modify the package to make use of the interface when specified
- Create a new indexers package
  - Provides an Index interface which allows concrete indexes to plugin
    to an index manager
  - Provides a concrete IndexManager implementation
    - Handles the lifecycle of all indexes it manages
    - Tracks the index tips
    - Handles catching up disabled indexes that have been reenabled
    - Handles reorgs while the index was disabled
    - Invokes the appropriate methods for all managed indexes to allow
      them to index and deindex the blocks and transactions
  - Implement a transaction-by-hash index
    - Makes use of internal block IDs to save a significant amount of
      space and indexing costs over the old transaction index format
  - Implement a transaction-by-address index
    - Makes use of a leveling scheme in order to provide a good tradeoff
      between space required and indexing costs
- Supports enabling and disabling indexes at will
- Support the ability to drop indexes if they are no longer desired

The following is an overview of the btcd changes:

- Add a new index logging subsystem
- Add new options --txindex and --addrindex in order to enable the
  optional indexes
  - NOTE: The transaction index will automatically be enabled when the
    address index is enabled because it depends on it
- Add new options --droptxindex and --dropaddrindex to allow the indexes
  to be removed
  - NOTE: The address index will also be removed when the transaction
    index is dropped because it depends on it
- Update getrawtransactions RPC to make use of the transaction index
- Reimplement the searchrawtransaction RPC that makes use of the address
  index
- Update sample-btcd.conf to include sample usage for the new optional
  index flags
2016-04-11 17:16:42 -05:00
Dave Collins
491acd4ca6 blockchain: Rework to use new db interface.
This commit is the first stage of several that are planned to convert
the blockchain package into a concurrent safe package that will
ultimately allow support for multi-peer download and concurrent chain
processing.  The goal is to update btcd proper after each step so it can
take advantage of the enhancements as they are developed.

In addition to the aforementioned benefit, this staged approach has been
chosen since it is absolutely critical to maintain consensus.
Separating the changes into several stages makes it easier for reviewers
to logically follow what is happening and therefore helps prevent
consensus bugs.  Naturally there are significant automated tests to help
prevent consensus issues as well.

The main focus of this stage is to convert the blockchain package to use
the new database interface and implement the chain-related functionality
which it no longer handles.  It also aims to improve efficiency in
various areas by making use of the new database and chain capabilities.

The following is an overview of the chain changes:

- Update to use the new database interface
- Add chain-related functionality that the old database used to handle
  - Main chain structure and state
  - Transaction spend tracking
- Implement a new pruned unspent transaction output (utxo) set
  - Provides efficient direct access to the unspent transaction outputs
  - Uses a domain specific compression algorithm that understands the
    standard transaction scripts in order to significantly compress them
  - Removes reliance on the transaction index and paves the way toward
    eventually enabling block pruning
- Modify the New function to accept a Config struct instead of
  inidividual parameters
- Replace the old TxStore type with a new UtxoViewpoint type that makes
  use of the new pruned utxo set
- Convert code to treat the new UtxoViewpoint as a rolling view that is
  used between connects and disconnects to improve efficiency
- Make best chain state always set when the chain instance is created
  - Remove now unnecessary logic for dealing with unset best state
- Make all exported functions concurrent safe
  - Currently using a single chain state lock as it provides a straight
    forward and easy to review path forward however this can be improved
    with more fine grained locking
- Optimize various cases where full blocks were being loaded when only
  the header is needed to help reduce the I/O load
- Add the ability for callers to get a snapshot of the current best
  chain stats in a concurrent safe fashion
  - Does not block callers while new blocks are being processed
- Make error messages that reference transaction outputs consistently
  use <transaction hash>:<output index>
- Introduce a new AssertError type an convert internal consistency
  checks to use it
- Update tests and examples to reflect the changes
- Add a full suite of tests to ensure correct functionality of the new
  code

The following is an overview of the btcd changes:

- Update to use the new database and chain interfaces
- Temporarily remove all code related to the transaction index
- Temporarily remove all code related to the address index
- Convert all code that uses transaction stores to use the new utxo
  view
- Rework several calls that required the block manager for safe
  concurrency to use the chain package directly now that it is
  concurrent safe
- Change all calls to obtain the best hash to use the new best state
  snapshot capability from the chain package
- Remove workaround for limits on fetching height ranges since the new
  database interface no longer imposes them
- Correct the gettxout RPC handler to return the best chain hash as
  opposed the hash the txout was found in
- Optimize various RPC handlers:
  - Change several of the RPC handlers to use the new chain snapshot
    capability to avoid needlessly loading data
  - Update several handlers to use new functionality to avoid accessing
    the block manager so they are able to return the data without
    blocking when the server is busy processing blocks
  - Update non-verbose getblock to avoid deserialization and
    serialization overhead
  - Update getblockheader to request the block height directly from
    chain and only load the header
  - Update getdifficulty to use the new cached data from chain
  - Update getmininginfo to use the new cached data from chain
  - Update non-verbose getrawtransaction to avoid deserialization and
    serialization overhead
  - Update gettxout to use the new utxo store versus loading
    full transactions using the transaction index

The following is an overview of the utility changes:
- Update addblock to use the new database and chain interfaces
- Update findcheckpoint to use the new database and chain interfaces
- Remove the dropafter utility which is no longer supported

NOTE: The transaction index and address index will be reimplemented in
another commit.
2016-04-11 16:47:27 -05:00
Dave Collins
a3fa066745 mining: Export block template fields. (#659)
This simply exports and adds some comments to the fields of the
BlockTemplate struct.

This is primarily being done as a step toward being able to separate the
mining code into its own package, but also it makes sense on its own
because code that requests new block template necessarily examines the
returned fields which implies they should be exported.
2016-04-11 10:27:29 -05:00
Dave Collins
f389742b39 multi: Update with result of gofmt -s.
This commit updates the code to make use of the most recent simplified
output from gofmt.
2016-02-25 13:02:54 -06:00
Dave Collins
eb882f39f8 multi: Fix several misspellings in the comments.
This commit corrects several typos in the comments found by misspell.
2016-02-25 11:17:12 -06:00
Tibor Bősze
ef9c50be57 Fix typos in comments 2016-02-22 19:57:29 +01:00
Tibor Bősze
c75fea9c94 Implement banning based on dynamic ban scores
Dynamic ban scores consist of a persistent and a decaying component. The
persistent score can be used to create simple additive banning policies
simlar to those found in other bitcoin node implementations. The
decaying score enables the creation of evasive logic which handles
misbehaving peers (especially application layer DoS attacks) gracefully
by disconnecting and banning peers attempting various kinds of flooding.
Dynamic ban scores allow these two approaches to be used in tandem.

This pull request includes the following:

 - Dynamic ban score type & functions, with tests for core functionality
 - Ban score of connected peers can be queried via rpc (getpeerinfo)
 - Example policy with decaying score increments on mempool and getdata
 - Logging of misbehavior once half of the ban threshold is reached
 - Banning logic can be disabled via configuration (enabled by default)
 - User defined ban threshold can be set via configuration
2016-02-16 10:10:29 +01:00
Dave Collins
16582789c3 rpcserver: Optimize filteraddr code.
This optimizes the filter address handling code in the RPC server
handleSearchRawTransasctions path in a few ways:

- Only normalize the filter addresses provided in the command once
  rather than for every transaction being returned
- Reset the passes filter flag just before it's used
- Use a local for the encoded address to avoid the bounds checking
  overhead of accessing it from the slice
- Avoiding subsequent map lookups once the filter has already passed

Also, while here, modify a few of the related comments to match code
style and consistency.
2016-02-03 20:38:45 -06:00
Dave Collins
ce981f45c2 mining: Create skeleton package.
This creates a skeleton mining package that simply contains a few of the
definitions used by the mining and mempool code.

This is a step towards decoupling the mining code from the internals of
btcd and ultimately will house all of the code related to creating block
templates and CPU mining.

The main reason a skeleton package is being created before the full
blown package is ready is to avoid blocking mempool separation which
relies on these type definitions.
2015-11-30 12:23:50 -06:00
Dave Collins
2b6a9a56e5 mempool/mining: Introduce TxSource interface.
This introduces the concept of a new interface named TxSource which aims
to generically provide a concurrent safe source of transactions to be
considered for inclusion in a new block.  This is a step towards
decoupling the mining code from the internals of btcd.  Ultimately the
intent is to create a separate mining package.

The new TxSource interface relies on a new struct named miningTxDesc,
which describes each entry in the transaction source.  Once this code is
refactored into a separate mining package, the mining prefix can simply
be dropped leaving the type exported as mining.TxDesc.

To go along with this, the existing TxDesc type in the mempool has been
renamed to mempoolTxDesc and changed to embed the new miningTxDesc type.
This allows the mempool to efficiently implement the MiningTxDescs
method needed to satisfy the TxSource interface.

This approach effectively separates the direct reliance of the mining
code on the mempool and its data structures.  Even though the memory
pool will still be the default concrete implementation of the interface,
making it an interface offers much more flexibility in terms of testing
and even provides the potential to allow more than one source (perhaps
multiple independent relay networks, for example).

Finally, the memory pool and all of the mining code has been updated to
implement and use the new interface.
2015-11-25 13:30:44 -06:00
Dave Collins
8ab565ce21 mempool/mining: Decouple and optimize priority calcs.
This does three things:

- Splits the priority calculation logic from the TxDesc type
- Modifies the calcPriority function to perform the value age
  calculation instead of accepting it as a parameter
- Changes the starting priority to be calculated when the transaction is
  added to the pool

The first is useful as it is a step towards decoupling the mining code
from the internals of the memory pool.  Also, the concept of priority is
related to mining policy, so it makes more sense to have the
calculations separate than being defined on the memory pool tx
descriptor.

The second change has been made because everywhere that uses the
calcPriority function first has to calculate the value age anyways and
by making it part of the function it can be avoided altogether in
certain circumstances thereby provided a bit of optimization.

The third change ensure the starting priority is safe for reentrancy
which will be important once the mempool is split into a separate
package.
2015-11-25 12:39:47 -06:00
Dave Collins
a4aa131dd5 mining: Refactor policy into its own struct.
This introduces the concept of a mining policy struct which is used to
control block template generation instead of directly accessing the
config struct.  This is a step toward decoupling the mining code from
the internals of btcd.  Ultimately the intent is to create a separate
mining package.
2015-11-23 22:02:14 -06:00
danda
c7eaee6020 adds filteraddrs param to searchrawtransactions API 2015-11-15 15:30:13 -08:00
David Hill
4b7206b54f btcjson: Add optional locktime to createrawtransaction
rpcserver:
If the locktime is given, the transaction inputs will be set to a
non-max value, activating the locktime.  The locktime for the
new transaction will be set to the given value.

This mimics Bitcoin Core commit 212bcca92089f406d9313dbe6d0e1d25143d61ff
2015-10-30 17:16:50 -04:00
Javed Khan
00bddf7540 peer: Refactor peer code into its own package.
This commit introduces package peer which contains peer related features
refactored from peer.go.

The following is an overview of the features the package provides:

- Provides a basic concurrent safe bitcoin peer for handling bitcoin
  communications via the peer-to-peer protocol
- Full duplex reading and writing of bitcoin protocol messages
- Automatic handling of the initial handshake process including protocol
  version negotiation
- Automatic periodic keep-alive pinging and pong responses
- Asynchronous message queueing of outbound messages with optional
  channel for notification when the message is actually sent
- Inventory message batching and send trickling with known inventory
  detection and avoidance
- Ability to wait for shutdown/disconnect
- Flexible peer configuration
  - Caller is responsible for creating outgoing connections and listening
    for incoming connections so they have flexibility to establish
    connections as they see fit (proxies, etc.)
  - User agent name and version
  - Bitcoin network
  - Service support signalling (full nodes, bloom filters, etc.)
  - Maximum supported protocol version
  - Ability to register callbacks for handling bitcoin protocol messages
- Proper handling of bloom filter related commands when the caller does
  not specify the related flag to signal support
  - Disconnects the peer when the protocol version is high enough
  - Does not invoke the related callbacks for older protocol versions
- Snapshottable peer statistics such as the total number of bytes read
  and written, the remote address, user agent, and negotiated protocol
  version
- Helper functions for pushing addresses, getblocks, getheaders, and
  reject messages
  - These could all be sent manually via the standard message output
    function, but the helpers provide additional nice functionality such
    as duplicate filtering and address randomization
- Full documentation with example usage
- Test coverage

In addition to the addition of the new package, btcd has been refactored
to make use of the new package by extending the basic peer it provides to
work with the blockmanager and server to act as a full node.  The
following is a broad overview of the changes to integrate the package:

- The server is responsible for all connection management including
  persistent peers and banning
- Callbacks for all messages that are required to implement a full node
  are registered
- Logic necessary to serve data and behave as a full node is now in the
  callback registered with the peer

Finally, the following peer-related things have been improved as a part
of this refactor:

- Don't log or send reject message due to peer disconnects
- Remove trace logs that aren't particularly helpful
- Finish an old TODO to switch the queue WaitGroup over to a channel
- Improve various comments and fix some code consistency cases
- Improve a few logging bits
- Implement a most-recently-used nonce tracking for detecting self
  connections and generate a unique nonce for each peer
2015-10-23 06:17:29 +05:30
David Hill
a56db22e9b config: New option --minrelaytxfee
--minrelaytxfee allows the user to specify the minimum transaction
fee in BTC/kB in which the fee is considered a non-zero fee.
2015-10-20 12:41:12 -04:00
Dave Collins
5a9bac9668 Correct a few style related issues found by golint.
Also, update TravisCI goclean script to remove the special casing which
ignored 'Id' from the lint output since that exception is no longer
needed.  It was previously required due to the old version of btcjson,
but that is no longer in the repo.
2015-10-20 10:34:14 -05:00
Josh Rickmar
07406791c9 rpcserver: Copy btcwallet fix for verifymessage. 2015-10-16 14:25:07 -04:00
Olaoluwa Osuntokun
0029905d43 Integrate a valid ECDSA signature cache into btcd
Introduce an ECDSA signature verification into btcd in order to
mitigate a certain DoS attack and as a performance optimization.

The benefits of SigCache are two fold. Firstly, usage of SigCache
mitigates a DoS attack wherein an attacker causes a victim's client to
hang due to worst-case behavior triggered while processing attacker
crafted invalid transactions. A detailed description of the mitigated
DoS attack can be found here: https://bitslog.wordpress.com/2013/01/23/fixed-bitcoin-vulnerability-explanation-why-the-signature-cache-is-a-dos-protection/
Secondly, usage of the SigCache introduces a signature verification
optimization which speeds up the validation of transactions within a
block, if they've already been seen and verified within the mempool.

The server itself manages the sigCache instance. The blockManager and
txMempool respectively now receive pointers to the created sigCache
instance. All read (sig triplet existence) operations on the sigCache
will not block unless a separate goroutine is adding an entry (writing)
to the sigCache. GetBlockTemplate generation now also utilizes the
sigCache in order to avoid unnecessarily double checking signatures
when generating a template after previously accepting a txn to the
mempool. Consequently, the CPU miner now also employs the same
optimization.

The maximum number of entries for the sigCache has been introduced as a
config parameter in order to allow users to configure the amount of
memory consumed by this new additional caching.
2015-10-08 17:31:42 -07:00
Dario Nieuwenhuis
0190c349aa Add reverse order option to searchrawtransactions rpc 2015-10-08 16:31:39 +02:00
Dave Collins
e4c053e504 rpcserver: Optimize JSON raw tx input list create.
This commit optimizes the createVinList function which is used to
generate the JSON list of transaction inputs.  It also makes it more
consistent with the createVinListPrevOut function.

In particular, it entails the following changes:
- Only do a single coinbase check and return right away instead of
  checking multiple times inside the loop over the inputs
- Use a pointer for populating the details of each entry to avoid
  multiple unnecessary array lookups and bounds checks
- Group all fields that populate the entry for better readability
2015-09-30 20:52:42 -05:00
Josh Rickmar
5983c9b98e Allow the session RPC for limited (read-only) clients. 2015-09-23 16:53:07 -04:00
Dario Nieuwenhuis
1806557d14 Fix skip not being applied to mempool txns in searchrawtransactions 2015-09-08 15:17:43 +02:00
danda
43774fe6bb adds optional prevOut section to vin for searchrawtransactions api. See https://github.com/btcsuite/btcd/issues/485 2015-08-23 09:58:03 -07:00
Dave Collins
0280fa0264 Convert block heights to int32.
This commit converts all block height references to int32 instead of
int64.  The current target block production rate is 10 mins per block
which means it will take roughly 40,800 years to reach the maximum
height an int32 affords.  Even if the target rate were lowered to one
block per minute, it would still take roughly another 4,080 years to
reach the maximum.

In the mean time, there is no reason to use a larger type which results
in higher memory and disk space usage.  However, for now, in order to
avoid having to reserialize a bunch of database information, the heights
are still serialized to the database as 8-byte uint64s.

This is being mainly being done in preparation for further upcoming
infrastructure changes which will use the smaller and more efficient
4-byte serialization in the database as well.
2015-08-11 11:13:17 -05:00
Dave Collins
506fc9fb94 rpcserver: Allow tx result creation without block.
This commit modifies the createTxRawResult code path along with callers
to work with block headers as opposed to btcutil.Blocks.  This in turn
allows the code in handleGetRawTransaction and
handleSearchRawTransactions to perform a much cheaper block header load
as opposed to a full block load.

While here, also very slightly optimize the createVinList function to
avoid creating a util.Tx wrapper and to take advantage of the
btcutil.Amount type added after the function was originally written
2015-07-28 18:44:30 -05:00
David Hill
a6c79c7a91 Implement getblockheader RPC.
This mimics Bitcoin Core commit 076badb60f33f0c32b035de220ca14c52a423a2a
2015-07-23 15:22:17 -04:00
David Hill
7cfa843832 Support getmempoolinfo. 2015-06-24 20:34:56 -04:00
Alex Akselrod
d26aaffb2b Create generate RPC command to close #362
Create GenerateCmd in btcjson v2. Update tests to check GenerateCmd.

Update chaincfg/params.go with a new bool in Params, GenerateSupported,
with true values in SimNetParams and RegressionNetParams and false in
the others.

Create new flag, discreteMining, in CPUMiner struct.

Add GenerateNBlocks function to cpuminer.go and handleGenerate
function to rpcserver.go.

Update documentation for the RPC calls.
2015-05-04 21:04:25 -04:00
Dave Collins
2dfb4be707 rpcserver: Increase getblock difficulty prec to 8.
This commit increases the precision of the difficulty field of the
getblock RPC to 8 to match Bitcoin Core.

Closes #414.
2015-05-04 03:53:39 -05:00
Dave Collins
6e402deb35 Relicense to the btcsuite developers.
This commit relicenses all code in this repository to the btcsuite
developers.
2015-05-01 12:00:56 -05:00
Dave Collins
d8a4423b90 btcjson: Replace btcjson with version 2.
This commit removes the old and deprecated btcjsonv1 package, moves the
new version 2 package into its place, and updates all imports accordingly.
2015-05-01 00:43:09 -05:00
Olaoluwa Osuntokun
ab2ed710cb Fix 'add/delnode' type switch evaluation in server
* The cases for the 'addnode' command were previously
  stacked on top the new cases for the 'node' command.
  The intended behavior was to create a fall through and
  handle both commands. However, trying to use this
  syntax with a type switch caused the first case to be
  ignored.
* addnode' specific functions and structs in the server
  have been removed. Instead, the 'add' and 'del' subcommands
  are now proxied to the matching 'node' cmd functions.
2015-04-24 13:53:34 -07:00
Dave Collins
a4a52ae24f wire: Remove errs from BlockHeader/MsgBlock/MsgTx.
This commit removes the error returns from the BlockHeader.BlockSha,
MsgBlock.BlockSha, and MsgTx.TxSha functions since they can never fail and
end up causing a lot of unneeded error checking throughout the code base.

It also updates all call sites for the change.
2015-04-17 01:27:12 -05:00
Dave Collins
750d657666 Update for recent btcutil Block.Sha API change. 2015-04-17 00:44:15 -05:00
Olaoluwa Osuntokun
65b044eea2 Fix #79 by adding a new node JSON-RPC command
* Gives node operators full control of peer connectivity
* RPC adds ability to disconnect all matching non-persistent peers,
  remove persistent peers, and connect to peers making them either
  temporary or persistent.
2015-04-14 23:07:51 -07:00
Alex Akselrod
4a1445a032 Create limited RPC user.
The limited user is specified with the --rpclimituser and
--rpclimitpass options (or the equivalent in the config file).
The config struct and loadConfig() are updated to take the
new options into account. The limited user can have neither
the same username nor the same password as the admin user.

The package-level rpcLimit map in rpcserver.go specifies
the RPC commands accessible by limited users. This map
includes both HTTP/S and websocket commands.

The checkAuth function gets a new return parameter to
signify whether the user is authorized to change server
state. The result is passed to the jsonRPCRead function and
to the WebsocketHandler function in rpcwebsocket.go.

The wsClient struct is updated with an "isAdmin" field
signifying that the client is authorized to change server
state, written by WebsocketHandler and handleMessage.
The handleMessage function also checks the field to
allow or disallow an RPC call.

The following documentation files are updated:
- doc.go
- sample-btcd.conf
- docs/README.md
- docs/json_rpc_api.md
- docs/configure_rpc_server_listen_interfaces.md
2015-04-13 14:14:52 -04:00
Alex Akselrod
ead39153af Fix #303 and #346 and change addrindex sort order
Fix #303 by changing the addrindex key prefix to 3 characters so that
it's easy to check length when dropping the index. To drop the old
index, check to make sure we aren't dropping any entries that end in
"sx" or "tx" as those aren't part of the addrindex. Update test to
deal with the new prefix length.

Fix #346 by changing the pointers in the mempool's addrindex map to
wire.ShaHash 32-byte values. This lets them be deleted even if the
transaction data changes places in memory upon expanding the maps.

Change the way addrindex uint32s are stored to big-endian in order to
sort the transactions on disk in chronological/dependency order.

Change the "searchrawtransactions" RPC call to return transactions
from the database before the memory pool so that they're returned in
order. This commit DOES NOT do topological sorting of the memory pool
transactions to ensure they're returned in dependency order. This may
be a good idea for a future enhancement.

Add addrindex versioning to automatically drop the old/incompatible
version of the index and rebuild with the new sort method and key
prefix.
2015-03-23 13:05:54 -04:00
Dave Collins
28269d2598 rpcserver: Use default values for getnetworkhashps.
When the fields in the command for the getnetworkhashps RPC don't have the
fields set, use the intended default values.

Since the btcjson package sets these fields to the default values when a
command is unmarshaled from the wire, this typically isn't necessary.
However, when the RPC server calls the handler internally with optional
command fields set to nil, as is the case in getmininginfo, the defaults
need to be set as well.
2015-02-28 20:49:37 -06:00
Josh Rickmar
5ac0f1247d Fix return types for handleGetNetworkHashPS. 2015-02-28 21:40:13 -05:00
Dave Collins
c0428f6f9f btcjson: Update searchrawtransactions verbose.
This commit updates the SearchRawTransactionsCmd verbose parameter in the
latest version of btcjson to an integer to match recent changes to the
previous version of btcjson.
2015-02-25 16:03:30 -06:00
Dave Collins
637fbcadec rpcserver: Convert to make use of new btcjson.
This commit converts the RPC server over to use the new features available
in the latest version of btcjson and improve a few things along the way.
This following summarizes the changes:

- All btcjson imports have been updated to the latest package version
- The help has been significantly improved
  - Invoking help with no command specified now provides an alphabetized
    list of all supported commands along with one-line usage
  - The help for each command is automatically generated and provides much
    more explicit information such as the type of each parameter, whether
    or not it's optional or required, etc
  - The websocket-specific commands are now provided when accessing the
    help when connected via websockets
  - Help has been added for all websocket-specific commands and is only
    accessible when connected via websockets
- The error returns and handling of both the standard and websocket
  handlers has been made consistent
- All RPC errors have been converted to the new RPCError type
- Various variables have been renamed for consistency
- Several RPC errors have been improved
- The commands that are marked as unimplemented have been moved into the
  separate map where they belong
- Several comments have been improved
- An unnecessary check has been removed from the createrawtransaction
  handler
- The command parsing has been restructured a bit to pave the way for
  JSON-RPC 2.0 batching support
2015-02-24 23:46:51 -06:00
Dave Collins
c0c48e0efd Improve a couple of JSON-RPC server err conditions.
- When invalid HTTP Basic Access Authentication details are provided to
  the websocket endpoint, return a WWW-Authenticate header just like the
  non-websocket endpoint
- When a connection to the websocket endpoint fails to properly upgrade,
  return a 400 Bad Request HTTP error status code
2015-02-24 20:09:15 -06:00
Dave Collins
7b849ef3eb Improve internal RPC server function order.
This commit reorders the RPC server code to more closely match the
ordering used by the rest of the code base such that functions are
typically defined before they are used and near their call site.
2015-02-24 20:09:14 -06:00
Dave Collins
467d44cfc3 Update btcws path import paths to new location. 2015-02-19 13:07:25 -06:00
Dave Collins
6d289f602a Update btcjson path import paths to new location. 2015-02-19 11:55:22 -06:00
Dave Collins
e7c7c3399f Update btcec path import paths to new location. 2015-02-06 10:54:32 -06:00
Dave Collins
c6bc8ac1eb Update btcnet path import paths to new location. 2015-02-05 23:24:53 -06:00
Olaoluwa Osuntokun
ecdffda748 Add support for an optional address-based transaction index.
* Address index is built up concurrently with the `--addrindex` flag.
* Entire index can be deleted with `--dropaddrindex`.
* New RPC call: `searchrawtransaction`
  * Returns all transacitons related to a particular address
  * Includes mempool transactions
  * Requires `--addrindex` to be activated and fully caught up.
* New `blockLogger` struct has been added to factor our common logging
  code
* Wiki and docs updated with new features.
2015-02-05 14:48:19 -08:00
Dave Collins
03433dad6a Update btcwire path import paths to new location. 2015-02-05 15:16:39 -06:00
Dave Collins
624bbb3216 Update btcchain import paths to new location. 2015-01-30 16:25:42 -06:00
Olaoluwa Osuntokun
c01d175fde Relay inv msgs now include underlying data object
* When an inv is to be sent to the server for relaying, the sender
already has access to the underlying data. So
instead of requiring the relay to look up the data by
hash, the data is now coupled in the request message.
2015-01-30 10:52:07 -08:00
Dave Collins
3b1a15d0d5 Update btcscript import paths to new location. 2015-01-30 12:14:33 -06:00
Dave Collins
d67a0e207c Update for recent ScriptBuilder API change. 2015-01-29 00:11:19 -06:00
Dave Collins
309a9ea31d Update database import paths to new location. 2015-01-27 15:38:23 -06:00
Dave Collins
66a17f8a47 Return time offset in getinfo RPC.
This was previously hard-coded to zero instead of using the offset
provided by the median time source which takes time samples from the other
connected nodes.
2015-01-24 11:24:53 -06:00
Dave Collins
919109f12c Update btcjson import paths to new location. 2015-01-16 23:58:09 -06:00
Dave Collins
10643379a9 Update btcws import paths to new location. 2015-01-16 23:37:26 -06:00
Dave Collins
a57505b7f0 Update btcscript import paths to new location. 2015-01-16 19:37:00 -06:00
Dave Collins
528622b259 Update btcchain import paths to new location. 2015-01-16 18:47:50 -06:00
Dave Collins
0b7a9074ef Update btcdb import paths to new location. 2015-01-16 18:30:32 -06:00
Dave Collins
44102d752a Update btcec import paths to new location. 2015-01-16 18:06:03 -06:00
Dave Collins
2bd21ead0b Update btcnet import paths to new location. 2015-01-16 17:31:08 -06:00
Dave Collins
54ccb83025 Update btcwire import paths to new location. 2015-01-16 15:13:21 -06:00
Dave Collins
3172e84bb3 Update fastsha256 import paths to new location. 2015-01-16 09:10:29 -06:00
Dave Collins
b732eac0be Update websocket import paths to new location. 2015-01-15 17:20:30 -06:00
Dave Collins
58db4a8b7e Update btcutil import paths to new location. 2015-01-15 10:30:38 -06:00
Dave Collins
a64ffb820a Correct RPC time handling for getblocktemplate.
This commit modifies the getblocktemplate RPC to correctly handle the
curtime field.

Currently, the RPC server will refuse to serve a block template if the
current time is before the minimum allowed time for a block even though
the real generated block template already accounted for it.

This consists of three changes:
- Remove the unnecessary and incorrect check in the RPC invocation for the
  current time against the min required time since it is already handled
  properly by the block template generation.
- Expose the network time source to the RPC work state and use the
  adjusted time for checking against the maximum allowed time instead of
  the current time.
- Set the returned RPC result curtime field to the time of the generated
  block header.

Fixes #209.
2015-01-08 01:41:50 -06:00
Dave Collins
4b727d2035 Ignore error in script disassembly for RPC results.
This commit modifies the creation of RPC results to ignore errors in
script disassembly since they already contain the error string inline and
the RPC results must still be generated regardless.

This was already done in the decodescript RPC, however the same thing was
not being done in the higher level verbose transaction results.  This
applies to both the createrawtransaction and decoderawtransaction RPCs.

Also, since this was the only thing that could error within those
functions, the error returns and caller checking of now non-existent
errors have been removed.

Fixes #210.
2015-01-07 16:23:23 -06:00
Dave Collins
04d47de262 Allow disabling RPC server TLS for localhost only.
This commit introduces a new flag, --notls, which can be used to disable
TLS for the RPC server.  However, the flag can only be used when the RPC
server is bound to localhost interfaces.  This is intended to prevent the
situation where someone decides they want to expose the RPC server to the
web for remote management/access, but forgot they have TLS disabled.
2015-01-02 11:17:23 -06:00
Dave Collins
33546047e3 Use correct error name in handleGetRawTransaction.
This commit corrects the error check from the createTxRawResult call in
handleGetRawTransaction.  It was previously given a different name which
resulted in the wrong error being checked.

Fixes #196.
2014-12-21 21:49:21 -06:00
Josh Rickmar
1d0c09a852 Require TLS 1.2 minimum.
This prevents a downgrade attack to the vulnerable SSLv3.  While here,
go ahead and require at least TLS 1.2 since TLS 1.0 and 1.1 have their
own set of issues and it's only a matter of time before those would
need to be completely avoided as well.

ok @davecgh
2014-12-15 13:53:14 -05:00
Dave Collins
888884a399 Use .haveTransaction in new getmempool RPC code.
The mempool is already locked and the exported version locks it, so the
unexported version is needed.
2014-11-17 15:46:42 -06:00
Dave Collins
1973aa5fd5 A few nitpicks with recent getrawmempool updates.
This commit addresses a few nitpicks in the recent getrawmppol update
which populates the starting and current priority fields.

In particular:

- Move the new calcInputValueAge function before the function which
  invokes it so it is consistent with the rest of the mempool code
- Double space after periods for consistency
- Correct the comments for calcInputValueAge to indiciate that inputs
  which are in the the memory pool count as zero toward the value age
  rather than the incorrect claim that that the overal input value age is
  zero when one of them does
- Rename endingPriority to currentPriority to match the RPC field and its
  actual function
- Make the comment about using zero when input transactions can't be found
  for some reason more accurate since there can be (and frequently is)
  more than one input transaction
2014-11-17 00:31:14 -06:00
Olaoluwa Osuntokun
a49b0d05b3 Add the starting+current priority to getrawmempool. 2014-11-16 23:58:33 -06:00
Dave Collins
c3065d32f4 Make use of the new btcchain MedianTimeSource.
This commit uses the new MedianTimeSource API in btcchain to create a
median time source which is stored in the server and is fed time samples
from all remote nodes that are connected.  It also modifies all call sites
which now require the the time source to pass it in.
2014-10-09 10:44:22 -05:00
Jonathan Gillham
1bbd1e9cba Changed TxIn.PreviousOutpoint to TxIn.PreviousOutPoint after btcwire API change. 2014-10-01 16:34:30 +01:00