Commit graph

209 commits

Author SHA1 Message Date
Dave Collins d06c0bb181
blockchain: Use hash values in structs.
This modifies the blockNode and BestState structs in the blockchain
package to store hashes directly instead of pointers to them and updates
callers to deal with the API change in the exported BestState struct.

In general, the preferred approach for hashes moving forward is to store
hash values in complex data structures, particularly those that will be
used for cache entries, and accept pointers to hashes in arguments to
functions.

Some of the reasoning behind making this change is:

- It is generally preferred to avoid storing pointers to data in cache
  objects since doing so can easily lead to storing interior pointers
  into other structs that then can't be GC'd
- Keeping the hash values directly in the block node provides better
  cache locality
2017-02-03 11:36:33 -06:00
Dave Collins 59169540c3
blockchain: Reconstruct headers from block nodes.
This modifies the block node structure to include a couple of extra
fields needed to be able to reconstruct the block header from a node,
and exposes a new function from chain to fetch the block headers which
takes advantage of the new functionality to reconstruct the headers from
memory when possible.  Finally, it updates both the p2p and RPC servers
to make use of the new function.

This is useful since many of the block header fields need to be kept in
order to form the block index anyways and storing the extra fields means
the database does not have to be consulted when headers are requested if
the associated node is still in memory.

The following timings show representative performance gains as measured
from one system:

new: Time to fetch 100000 headers:   59ms
old: Time to fetch 100000 headers: 4783ms
2017-02-01 12:27:56 -06:00
David Hill ecd348b2a7 btcd: disable mempool requests if bloom filtering is disabled 2017-01-27 20:51:25 -06:00
David Hill db5b9aef91 btcd: fix rebroadcasting of local txs. 2017-01-27 10:28:48 -05:00
David Hill 9bedd7720c btcd: only allow one getaddr request per peer 2017-01-25 12:05:55 -05:00
Javed Khan 8caa921ac3 server: Fix connections to onion nodes.
A DNS lookup was being attempted on onion addresses causing
connections to fail. This has been fixed by introducing type
onionAddr (which implements a net.Addr interface) and passing
it to btcdDial.

Also, the following onion related fixes have been made:

* getaddednodeinfo - updated to handle onion addrs.
* TorLookupIP - fixed err being shadowed.
* newServer - rename tcpAddr to netAddr
* addrStringToNetAddr - skip if host is already an IP addr.
* addrStringToNetAddr - err if tor is disabled
* getaddednodeinfo - check if host is already an IP addr.
2017-01-18 21:40:41 +05:30
Alex 7c44b6472f Port getheaders JSON-RPC command from dcrd 2017-01-10 19:11:18 -07:00
David de Kloet 4021ae2f6e server.go: Optimize newAddressFunc
Change the order of conditions to avoid calling fmt.Sprintf
unnecessarily.
2016-12-08 22:55:43 +01:00
Steven Roose 95e6de00b8 btcd: fix error in mempool response inv counting 2016-11-25 10:30:24 +01:00
dskloet e1b2ceca80 addrmgr: Remove unused param from GetAddress()
addrmgr.GetAddress() had a parameter `class string` originally intended
to support looking up addresses according to some type of filter such as
IPv4, IPv6, and only those which support specific wire.ServiceFlags
(full nodes, nodes that support bloom filters, nodes that support
segwit, etc). But currently the parameter is unused and also has an
inappropriate type `string`.

If it would ever be used, it's easy to add back and should then get an
appropriate type such as something that allows bitflags to be set so
that the caller could request combinations such as peers that support
IPv6, are full nodes, and support bloom filters.
2016-11-21 08:56:46 -06:00
David Hill 807d344fe9 Unassign some TODO's 2016-11-15 17:47:33 -06:00
Dave Collins 0d4546c965
server: Return error on address lookup fails.
This corrects an issue introduced by commit
e8f63bc295 where a failure to lookup a
hostname could lead to a panic in certain circumstances.  An error is
now returned in that case as expected.
2016-11-11 20:54:25 -06:00
Olaoluwa Osuntokun e8f63bc295
connmgr: switch to using net.Addr interface throughout for addresses
This commit modifies the `ConnManager` to use the `net.Add` interface
through the package instead of a plain string to represent and
manipulate addresses. This change makes the package much more general as
users of the package can possibly utilize custom implementations of the
`net.Addr` interface to establish connections.

More precisely, the `ConnReq` struct has been modified to use a net.Addr
instance explicitly, and the `DialFunc` type has also been modified to
take a `net.Addr` directly. This latter change gives functions that
adhere to the `DialFunc` type more flexibility as to exactly how the
connection is established.

Additionally, the `connmgr.Config.GetNewAddress` configuration option
now directly returns a `net.Addr. This change allows the `connmgr` to be
decoupled from all DNS queries which allows callers to preferentially
select more secure methods like performing DNS lookups over a Tor proxy.
2016-11-10 11:22:36 -08:00
Dave Collins df33d4340e
server: Ensure callbacks use the server peer.
This modifies the signatures of all serverPeer callbacks that are
provided as peer.Listeners to use _ for the first parameter name which
ensures the passed peer can't be used within the function and updates
all references to the server peer.

This helps ensure any overridden methods that might be defined on a
serverPeer will be invoked where directly calling methods on the passed
peer would not.

Also, while here, add a comment to the OnFeeFilter function.
2016-11-04 13:38:13 -05:00
Dave Collins d98430d8ca
connmgr: Implement inbound connection handling.
This modifies the connection manager to provide support for accepting
inbound connections on a caller-provided set of listeners and notify the
caller via a callback.

This is only the minimum work necessary to get inbound support into the
connection manager.  The intention for future commits is to move more
connection-related logic such as limiting the maximum number of overall
connections and banned peer tracking into the connection manager.
2016-11-04 13:14:44 -05:00
Dave Collins ea9bf748bb
connmgr: Remove type defs for callbacks.
This removes the type definitions for the callback functions in favor of
declaring them directly in the Config struct.  This is more consistent
with the rest of the code base and is preferred since it means callers
reviewing the documentation don't have to follow another level of
indirection to figure out the signature.
2016-11-04 13:14:39 -05:00
Javed Khan a041b4349b server: Use Disconnect in handleDonePeerMsg 2016-11-04 00:36:13 -05:00
Javed Khan b320129e9b server: Check max peers before connecting 2016-11-04 00:36:13 -05:00
Javed Khan aca9fc040c connmgr: Rename max outbound to target outbound 2016-11-04 00:36:13 -05:00
David Hill 2510baac35 btcd: support feefilter requests.
This only adds support for handling remote peer requests.
2016-11-03 14:47:30 -04:00
Dave Collins 915fa6639b
multi: Simplify code per gosimple linter.
This simplifies the code based on the recommendations of the gosimple
lint tool.
2016-11-03 13:00:35 -05:00
Javed Khan d1c39edee8 server: Cap max outbound in connmgr cfg at max peers 2016-11-02 15:10:14 -05:00
Dave Collins 6d5714e1b7
server/mempool: Evict orphans on peer disconnect.
This removes any remaining orphan transactions that were sent by a peer
when it disconnects since it is extremely unlikely that the missing
parents will ever materialize from elsewhere.
2016-10-28 15:27:57 -05:00
David Hill 2615fa0849 mempool: Return type TxDesc instead of type btcutil.Tx
This will provide callers more information on the accepted transaction.
2016-10-28 14:52:31 -04:00
Dave Collins 1a69eb0617
cpuminer: Refactor code to its own package.
This does the minimum work necessary to refactor the CPU miner code into
its own package.  The idea is that separating this code into its own
package will improve its testability and ultimately be useful to other
parts of the codebase such as the various tests which currently
effectively have their own stripped-down versions of this code.

The API will certainly need some additional cleanup and changes to make
it more usable outside of the specific circumstances it was originally
designed to support (namely the generate RPC), however it is better to
do that in future commits in order to keep the changeset as small as
possible during this refactor.

Overview of the major changes:

- Create the new package
- Move cpuminer.go -> cpuminer/cpuminer.go
- Update mining logging to use the new cpuminer package logger
- Rename cpuminerConfig to Config (so it's now cpuminer.Config)
- Rename newCPUMiner to New (so it's now cpuminer.New)
- Update all references to the cpuminer to use the package
- Add a skeleton README.md
2016-10-28 11:06:11 -05:00
Dave Collins 214d975adf
server: Notify connmgr if server peer assoc fails.
This corrects a few issues introduced with the connection manager where
the server was not notifying the connection manager when a connection
request is available again.

The cases resolved are:
- Unable to initialize a server peer instance in response to the connection
- Failure to associate the connection with the server peer instance
- Disconnection of a non-persistent outbound peer

It also changes the log message to a debug in the former case because
it's not something that should be shown to the user as an error given
it's not due to anything the user has misconfigured nor is it even
unexpected if an invalid address is provided.
2016-10-28 10:49:00 -05:00
Dave Collins 14b51fc5f8
multi: Correct misspellings detected by misspell. 2016-10-28 09:43:38 -05:00
Dave Collins f3c442deb2
server: Remove unused wakeup channel.
This channel is no longer used since the outgoing connection logic is
handled by the connmgr package now.
2016-10-27 20:04:58 -05:00
Dave Collins 61ca40e0e9
mining: Refactor template code into mining package.
This does the minimum work necessary to refactor the block template
generation code into the mining package.  The idea is that separating
this code into the mining package will greatly improve its testability,
allow independent benchmarking and profiling, and open up some
interesting opportunities for future development related to mining.

There are some areas related to policy and other configuration that
could be further refactored, however it is better to do that in future
commits in order to keep the changeset as small as possible during this
refactor.

Overview of the major changes:

- Move mining.go -> mining/mining.go
- Move mining_test.go -> mining/mining_test.go
- Add logger to mining package
- Update the MINR subsystem to use the new mining package logger
- Export CoinbaseFlags from the mining package
- BlkTmplGenerator is now mining.BlkTmplGenerator
- Update all references to the mining code to use the package
2016-10-27 11:48:48 -05:00
Dave Collins 660467259e
mining: Break dependency on block manager instance.
This modifies the block template generate for the mining code such that
it takes chain instance and params instead of requiring a fully
initialized blockManager instance.

Also, in preparation for being able to more easily separate the code, it
exposes and makes use of two new functions:
- BestSnapshot which returns the state snapshot from the underlying
  chain instance
- TxSource which returns the underlying transaction source

This is a step towards being able to separate the mining code into its
own package.  No functional change.
2016-10-27 11:48:32 -05:00
Dave Collins 671901486c
cpuminer: Introduce cpuminerConfig.
This introduces a cpuminerConfig type which contains the necessary
information to break the direct dependency on the main server instance.

This change is a step towards being able to separate the cpu miner into
its own subpackage.  No functional change.
2016-10-27 09:37:30 -05:00
Olaoluwa Osuntokun e3eeb4a34a
mempool: add policy config option for transaction version
This commit adds a new option to the mempool’s policy configuration
which determines which transaction versions should be accepted as
standard.

The default version set by the policy within the server is 2; this
allows accepting transactions which have version 2 enabled in order to
utilize the new sequence locks feature.
2016-10-26 21:49:14 -07:00
Olaoluwa Osuntokun 7eb0ab5f8d
mempool: add function to config for computing sequence locks 2016-10-26 21:49:04 -07:00
Dave Collins 74fe2a4dfd
mining: Introduce a block template generator.
This introduces a new type named BlkTmplGenerator which encapsulates the
various state needed to generate block templates.

This is useful since it means code that needs to generate block
templates can simply accept the generator rather than needing access to
all of the additional state which in turn will ultimately make it easier
to split the mining code into its own package.
2016-10-26 15:17:21 -05:00
Dave Collins 26e22790cd
mempool: Rename RelayNonStd config option.
This renames the mempool.Config.RelayNonStd option to AcceptNonStd which
more accurately describes its behavior since the mempool was refactored
into a separate package.

The reasoning for this change is that the mempool is not responsible for
relaying transactions (nor should it be).  Its job is to maintain a pool
of unmined transactions that are validated according to consensus and
policy configuration options which are then used to provide a source of
transactions that need to be mined.

Instead, it is the server that is responsible for relaying transactions.
While it is true that the current server code currently only relays txns
that were accepted to the mempool, this does not necessarily have to
be the case.  It would be entirely possible (and perhaps even a good
idea as something do in the future), to separate the relay policy from
the mempool acceptance policy (and thus indirectly the mining policy).
2016-10-23 20:41:54 -05:00
David Hill f161d6b69e chaincfg: Introduce new type DNSSeed
DNSSeed defines a DNS Seed with a hostname and whether it supports
filtering by service flag bits.
2016-10-23 15:59:15 -05:00
David Hill 0d508e6522 peer: BestLocalAddress config option is no longer used. 2016-10-23 16:38:45 -04:00
David Hill bca9877796 Do not add inbound peers to the address manager. 2016-10-23 16:37:44 -04:00
Javed Khan bff2ba70fd connmgr: Refactor connection management into pkg
This commit introduces package connmgr which contains connection
management related functionality.

The following is an overview of the features the package provides:

- Maintain fixed number of outbound connections
- Optional connect-only mode
- Retry persistent connections with increasing back-off
- Source peers from DNS seeds
- Use Tor to resolve DNS
- Dynamic ban scores
- Test coverage

In addition, btcd has been refactored to make use of the new package by
extending the connection manager to work with the server to source and
maintain peer connections. The following is a broad overview of the
changes to integrate the package:

- Simplify peer state by removing pending, retry peers
- Refactor to remove retries which are now handled by connmgr
- Use callback to add addresses sourced from the  DNS seed

Finally the following connection-related things have been improved as a
part of this refactor:

- Fixes 100% cpu usage when network is down (#129)
- Fixes issues with max peers (#577)
- Simplify outbound peer connections management
2016-10-22 01:11:57 -05:00
Javed Khan f6cd49ac51 peer: rename peer.Connect to AssociateConnection 2016-10-21 14:13:18 -05:00
Olaoluwa Osuntokun a82f67b538
mempool: add closure to compute median time past to config
This commit adds an additional closure function to the mempool’s config
which computes the median time past from the point of view of the best
node in the chain. The mempool test harness has also been updated to allow
setting a mock median time past for testing purposes.

In addition to increasing the testability of the mempool, this commit
should also speed up transaction and block validation for BIP 113 as
the MTP no longer needs to be re-calculated each time from scratch.
2016-10-19 11:13:25 -07:00
David Hill b1621332cc Optimize by removing defers
defer's are nice for readability but they do add overhead.  This
gets rid of defer's where it is just as easy as not to use one.
2016-10-18 17:56:51 -04:00
Dave Collins 8965d88893
peer: Strictly enforce bloom filter service bit.
This makes the enforcement of the bloom filter service bit much more
strict.  In particular, it does the following:

- Moves the enforcement of the bloom filter service bit out of the peer
  package and into the server so the server can ban as necessary
- Disconnect peers that send filter commands when the server is
  configured to disable them regardless of the protocol version
- Bans peers that are a high enough protocol version that they are
  supposed to observe the service bit is disabled, but ignore it and
  send filter commands regardless.

As an added bonus, this fixes the old logic which had a bug in that it
was examining the *remote* peer's supported services in order to choose
whether or not to disconnect instead of the *local* server's supported
services.
2016-10-16 02:19:28 -05:00
Olaoluwa Osuntokun 815ded348e
config: introduce new flags to accept/reject non-std transactions
This commit adds two new cli flags: one for accepting non-std
transactions, and the other for rejecting non-std transactions.

The two flag are rejected when using concurrently. Config parsing is
set up such that, the desired policy expressed via the config always
overrides the policy set by default for a particular chain.

The doc.go files and the sample-btcd.conf file have been updated to document
the new flags exposing further policy control.
2016-08-24 15:43:26 -07:00
Olaoluwa Osuntokun dc5486a579
mempool: add non-standard tx relaying to policy config 2016-08-24 15:43:22 -07:00
Dave Collins 641182b2ad
mempool: Break dependency on chain instance.
This modifies the config for the new mempool package such that it takes
a callback function to obtain the best chain height instead of requiring
a fully initialized blockchain.BlockChain instance.

This will make it much easier to test the mempool since the tests will
be able to provide their own height function to test various
functionality without having create and manipulate full blocks and chain
instances.
2016-08-23 12:29:45 -05:00
jadeblaquiere 47ced81d44 don't spin forever looking for peers (#724) 2016-08-20 22:11:12 -05:00
Dave Collins 7fac099bee mempool: Refactor mempool code to its own package. (#737)
This does the minimum work necessary to refactor the mempool code into
its own package.  The idea is that separating this code into its own
package will greatly improve its testability, allow independent
benchmarking and profiling, and open up some interesting opportunities
for future development related to the memory pool.

There are likely some areas related to policy that could be further
refactored, however it is better to do that in future commits in order
to keep the changeset as small as possible during this refactor.

Overview of the major changes:

- Create the new package
- Move several files into the new package:
  - mempool.go -> mempool/mempool.go
  - mempoolerror.go -> mempool/error.go
  - policy.go -> mempool/policy.go
  - policy_test.go -> mempool/policy_test.go
- Update mempool logging to use the new mempool package logger
- Rename mempoolPolicy to Policy (so it's now mempool.Policy)
- Rename mempoolConfig to Config (so it's now mempool.Config)
- Rename mempoolTxDesc to TxDesc (so it's now mempool.TxDesc)
- Rename txMemPool to TxPool (so it's now mempool.TxPool)
- Move defaultBlockPrioritySize to the new package and export it
- Export DefaultMinRelayTxFee from the mempool package
- Export the CalcPriority function from the mempool package
- Introduce a new RawMempoolVerbose function on the TxPool and update
  the RPC server to use it
- Update all references to the mempool to use the package.
- Add a skeleton README.md
2016-08-19 11:08:37 -05:00
Dave Collins 87b3756c8c server: Remove superfluous check in OnMemPool. (#736)
This reduces the mempool lock contention by removing an unnecessary
check when responding to a "mempool" request.

In particular, the code first gets a list of all transactions from the
mempool and then iterates them in order to construct the inventory
vectors and apply bloom filtering if it is enabled.  Since it is
possible that the transaction was removed from the mempool by another
thread while that list is being iterated, the code was checking if each
transaction was still in the mempool.  This is a pointless check because
the transaction might still be removed at any point after the check
anyways.  For example, it might be removed after the mempool response
has been sent to the remote peer or even while the loop is still
iterating.
2016-08-19 11:04:16 -05:00
Dave Collins 044a11c9fc btcd: Simplify shutdown signal handling logic. (#733)
This rewrites the shutdown logic to simplify the shutdown signalling.
All cleanup is now run from deferred functions in the main function and
channels are used to signal shutdown either from OS signals or from
other subsystems such as the RPC server and windows service controller.

The RPC server has been modified to use a new channel for signalling
shutdown that is exposed via the RequestedProcessShutdown function
instead of directly calling Stop on the server as it previously did.

Finally, it adds a few checks for early termination during the main
start sequence so the process can be stopped without starting all the
subsystems if desired.

This is a backport of the equivalent logic from Decred with a few slight
modifications.  Credits go to @jrick.
2016-08-11 13:39:23 -05:00