Commit graph

3459 commits

Author SHA1 Message Date
David Hill
a09d052f96 peer: Stop setting AddrMe in the version message.
Older nodes previously added the IP and port information to the address
manager which proved to be unreliable as an inbound connection from a
peer didn't necessarily mean the peer itself accepted inbound
connections.

This also fixes a bug where the peer package was incorrectly sending
the peer's services as its own.
2016-10-23 16:37:44 -04:00
David Hill
bca9877796 Do not add inbound peers to the address manager. 2016-10-23 16:37:44 -04:00
Dave Collins
b8df516b4b
docs: Add fullblocktests entry and make consistent. 2016-10-23 13:34:14 -05:00
Dave Collins
9634a8cb0d
fullblocktests: Add missing doc.go file. 2016-10-23 13:31:26 -05:00
Javed Khan
bff2ba70fd connmgr: Refactor connection management into pkg
This commit introduces package connmgr which contains connection
management related functionality.

The following is an overview of the features the package provides:

- Maintain fixed number of outbound connections
- Optional connect-only mode
- Retry persistent connections with increasing back-off
- Source peers from DNS seeds
- Use Tor to resolve DNS
- Dynamic ban scores
- Test coverage

In addition, btcd has been refactored to make use of the new package by
extending the connection manager to work with the server to source and
maintain peer connections. The following is a broad overview of the
changes to integrate the package:

- Simplify peer state by removing pending, retry peers
- Refactor to remove retries which are now handled by connmgr
- Use callback to add addresses sourced from the  DNS seed

Finally the following connection-related things have been improved as a
part of this refactor:

- Fixes 100% cpu usage when network is down (#129)
- Fixes issues with max peers (#577)
- Simplify outbound peer connections management
2016-10-22 01:11:57 -05:00
Marco Peereboom
69fca4d9b1 Reconcile differences between btcd/dcrd.
Fixes #793
2016-10-21 16:37:30 -05:00
David Hill
4494f0f852 txscript: Remove OP_SMALLDATA 2016-10-21 15:18:42 -04:00
Javed Khan
f6cd49ac51 peer: rename peer.Connect to AssociateConnection 2016-10-21 14:13:18 -05:00
Dave Collins
0731f2ddc9 txscript: Cleanup and improve NullDataScript tests.
This modifies the recently-added NullDataScript function in several
ways in an effort to make them more consistent with the tests in the
rest of the code base and improve/correct the logic:

- Use the hexToBytes and mustParseShortForm functions
- Consistently format the test errors
- Replace the valid bool flag with an expected error and test against it
- Ensure the returned script type is the expected type in all cases
2016-10-21 10:26:09 -05:00
DanielKrawisz
b77654f8d4 txscript: Add null data script creator
This adds a new function named NullDataScript to the txscript package that returns a provably-pruneable  OP_RETURN script with the provided data.  The function will return an error if the provided data is larger than the maximum allowed length for a nulldata script to be be considered standard.
2016-10-21 09:37:48 -05:00
Josh Rickmar
d0a9c03844 Concurrently handle websocket client JSON-RPC requests. 2016-10-20 19:55:58 -04:00
Dave Collins
da04285e0d
rpctest: Choose flags based on provided params.
This modifies the rpctest framework to start btcd with the appropriate
network flags depending on the provided parameters.

Previously, it always started btcd with --simnet even if other
parameters, such as those for the regression test network, were
provided.
2016-10-20 12:19:58 -05:00
Dave Collins
49cbaf23dd
blockchain: Support small coinbase block size
This modifies the ExtractCoinbaseHeight function to recognize small
canonically serialized block heights in coinbase scripts of blocks
higher than version 2.

This allows regression test chains in which blocks encode the serialized
height in the coinbase starting from block 1.
2016-10-20 12:09:09 -05:00
Dave Collins
59a3fc2f66 txscript: Consolidate tests into txscript package.
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.

Also, do some light cleanup on a few tests while here.
2016-10-20 09:28:33 -05:00
Dave Collins
b60e3547d2 txscript: Correct nulldata standardness check.
This corrects the isNullData standard transaction type test to work
properly with canonically-encoded data pushes.  In particular, single
byte data pushes that are small integers (0-16) are converted to the
equivalent numeric opcodes when canonically encoded and the code failed
to detect them properly.

It also adds several tests to ensure that both canonical and
non-canonical nulldata scripts are recognized properly and modifies the
test failure print to include the script that failed.

This does not affect consensus since it is just a standardness check.
2016-10-20 01:44:58 -05:00
David Hill
07e1e308f1 rpc: Add localaddr and relaytxes to getpeerinfo 2016-10-19 19:58:50 -04:00
David Hill
3b5bb9fd43 btcjson: Add preciousblock 2016-10-19 14:08:32 -05:00
David Hill
403aaf5cf3 rpcserver: avoid nested decodescript p2sh addrs 2016-10-19 13:59:50 -05:00
Olaoluwa Osuntokun
e7caccc866
mempool: transaction finality checks now use median-time-past
This coincides with the mempool only, policy change which enforces
transaction finality according to the median-time-past rather than
blockheader timestamps. The behavior is pre-cursor to full blown BIP
113 consensus deployment, and subsequent activation.

As a result, the TimeSource field in the mempoolConfig is no longer
needed so it has been removed. Additionally, checkTransactionStandard has been
modified to instead take a time.Time as the mempool is no longer explicitly
dependant on a Chain instance.
2016-10-19 11:13:34 -07:00
Olaoluwa Osuntokun
a82f67b538
mempool: add closure to compute median time past to config
This commit adds an additional closure function to the mempool’s config
which computes the median time past from the point of view of the best
node in the chain. The mempool test harness has also been updated to allow
setting a mock median time past for testing purposes.

In addition to increasing the testability of the mempool, this commit
should also speed up transaction and block validation for BIP 113 as
the MTP no longer needs to be re-calculated each time from scratch.
2016-10-19 11:13:25 -07:00
David Hill
a6bf1d9850 txscript: Implement CheckSequenceVerify (BIP0112) 2016-10-19 12:06:44 -04:00
Dave Collins
fdfa07b0be
btcec: Consolidate tests into the btcec package.
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.

Also, remove the exception from the lint checks about returning the
unexported type since it is no longer required.
2016-10-19 00:55:23 -05:00
David Hill
b1621332cc Optimize by removing defers
defer's are nice for readability but they do add overhead.  This
gets rid of defer's where it is just as easy as not to use one.
2016-10-18 17:56:51 -04:00
Jimmy Song
294b5d46da btcec: Add regression tests for field.go.
This adds new tests to the TestNormalize, TestMul, TestAdd2 functions
which trigger an issue with modular reduction that was fixed in the
prevous commit to prevent regressions.
2016-10-18 16:21:45 -05:00
Dave Collins
a52eb04aaa
btcec: Ensure reduction when > P in all cases.
As noted in issue #706, the existing code had an issue where the
normalized result was > P when both the first and second words of the
field representation being normalized were BOTH greater than or equal to
the first and second words of P.  Although this condition is rare in
practice, it needs to be handled properly.

This resolves the issue by comparing the low words in the final
reduction step against the normalized low order prime bits to ensure the
final subtraction occurs correctly any time they're > P.  This approach
retains the constant time property as well.
2016-10-18 16:21:36 -05:00
David Hill
d009185a56 peer: Implement feefilter p2p message (bip0133) 2016-10-17 15:45:56 -04:00
David Hill
9935fe5dba wire: Bump minor due to feefilter addition 2016-10-17 13:45:12 -04:00
David Hill
ca4e9b82d6 wire: implement feefilter message (bip0133)
feefilter is used to request the receiving peer does not announce any
transactions below the specified minimum fee rate.
2016-10-17 13:33:16 -04:00
Dave Collins
f21410e47c
blockchain: Add block validation infrastructure.
This adds a full-blown testing infrastructure in order to test consensus
validation rules.  It is built around the idea of dynamically generating
full blocks that target specific rules linked together to form a block
chain.  In order to properly test the rules, each test instance starts
with a valid block that is then modified in the specific way needed to
test a specific rule.

Blocks which exercise following rules have been added for this initial
version.  These tests were largely ported from the original Java-based
'official' block acceptance tests as well as some additional tests
available in the Core python port.  It is expected that further tests
can be added over time as consensus rules change.

* Enough valid blocks to have a stable base of mature coinbases to spend
  for futher tests
* Basic forking and chain reorganization
* Double spends on forks
* Too much proof-of-work coinbase (extending main chain, in block that
  forces a reorg, and in a valid fork)
* Max and too many signature operations via various combinations of
  OP_CHECKSIG, OP_MULTISIG, OP_CHECKSIGVERIFY, and OP_MULTISIGVERIFY
* Too many and max signature operations with offending sigop after
  invalid data push
* Max and too many signature operations via pay-to-script-hash redeem
  scripts
* Attempt to spend tx created on a different fork
* Attempt to spend immature coinbase (on main chain and fork)
* Max size block and block that exceeds the max size
* Children of rejected blocks are either orphans or rejected
* Coinbase script too small and too large
* Max length coinbase script
* Attempt to spend tx in blocks that failed to connect
* Valid non-coinbase tx in place of coinbase
* Block with no transactions
* Invalid proof-of-work
* Block with a timestamp too far in the future
* Invalid merkle root
* Invalid proof-of-work limit (bits header field)
* Negative proof-of-work limit (bits header field)
* Two coinbase transactions
* Duplicate transactions
* Spend from transaction that does not exist
* Timestamp exactly at and one second after the median time
* Blocks with same hash via merkle root tricks
* Spend from transaction index that is out of range
* Transaction that spends more that its inputs provide
* Transaction with same hash as an existing tx that has not been
  fully spent (BIP0030)
* Non-final coinbase and non-coinbase txns
* Max size block with canonical encoding which exceeds max size with
  non-canonical encoding
* Spend from transaction earlier in same block
* Spend from transaction later in same block
* Double spend transaction from earlier in same block
* Coinbase that pays more than subsidy + fees
* Coinbase that includes subsidy + fees
* Invalid opcode in dead execution path
* Reorganization of txns with OP_RETURN outputs
* Spend of an OP_RETURN output
* Transaction with multiple OP_RETURN outputs
* Large max-sized block reorganization test (disabled by default since
  it takes a long time and a lot of memory to run)

Finally, the README.md files in the main and docs directories have been
updated to reflect the use of the new testing framework.
2016-10-17 12:16:53 -05:00
Dave Collins
1cba5c8fc0
blockchain: Remove exported CalcPastTimeMedian func.
This removes the exported CalcPastTimeMedian function from the
blockchain package as it is no longer needed since the information is
now available via the BestState snapshot.

Also, update the only known caller of this, which is the chain state in
block manager, to use the snapshot instead.  In reality, now that
everything the block manager chain state provides is available via the
blockchain BestState snapshot, the entire thing can be removed, however
that will be done in a separate to commit to keep the changes targeted.
2016-10-17 10:41:25 -05:00
Dave Collins
e88f2d7bf4
mempool: Add test for max orphan entry eviction.
This adds a test to the mempool for ensuring that orphans are evicted
when exceeding the max orphan policy setting as expected.
2016-10-17 10:33:09 -05:00
Dave Collins
8965d88893
peer: Strictly enforce bloom filter service bit.
This makes the enforcement of the bloom filter service bit much more
strict.  In particular, it does the following:

- Moves the enforcement of the bloom filter service bit out of the peer
  package and into the server so the server can ban as necessary
- Disconnect peers that send filter commands when the server is
  configured to disable them regardless of the protocol version
- Bans peers that are a high enough protocol version that they are
  supposed to observe the service bit is disabled, but ignore it and
  send filter commands regardless.

As an added bonus, this fixes the old logic which had a bug in that it
was examining the *remote* peer's supported services in order to choose
whether or not to disconnect instead of the *local* server's supported
services.
2016-10-16 02:19:28 -05:00
Dave Collins
77913ad2e8
blockchain: Expose main chain flag on ProcessBlock.
This modifies the blockchain.ProcessBlock function to return an
additional boolean as the first parameter which indicates whether or not
the block ended up on the main chain.

This is primarily useful for upcoming test code that needs to be able to
tell the difference between a block accepted to a side chain and a block
that either extends the main chain or causes a reorganize that causes it
to become the main chain.  However, it is also useful for the addblock
utility since it allows a better error in the case a file with out of
order blocks is provided.
2016-10-13 16:47:50 -05:00
David Hill
42a4366ba8 addrmgr: Fix AddressCache to not include nils
allAddr was being allocated with counters instead of the actual size
of the address map.  This led to the possibility of including nils
in the returned slice, which resulted in a panic.
2016-09-26 12:16:40 -05:00
Dave Collins
754c4fbe0c
rpctest: Gate rpctest-based behind a build tag.
This adds a new build tag named rpctest which must be set in order for
rpctest-based tests to be executed.  The new build tag is also added to
the goclean.sh script which is executed by Travis during continuous
integration builds.

This change is being made because the rpctest framework requires
additional careful user configuration to ensure the version of btcd
under test can be programmatically launched from the system path with
all of the necessary ports open whereas all of the other tests are
self-contained within the test binary itself.

Since said additional configuration is typically not done, it leads to a
lot of false positives.  Putting the tests behind a build tag allows
them to remain to be available and run during continuous integration
without imposing the additional configuration requirements on users.
2016-09-26 01:20:31 -05:00
David Hill
99165eb558 rpctest: Fix typo
Use os.Getpid() to get the process ID, not os.Getppid(), which returns
the parent process ID.  This resulted in multiple calls to
generateListeningAddresses() getting the same listening ports.
2016-09-22 00:14:51 -04:00
David Hill
5ec83d23f3 Update dependencies and API usage. 2016-09-21 20:50:55 -04:00
Dave Collins
7cf9ec8190
rpctest: Use ports based on the process id.
This modifies the ports that are selected for use for the p2p and rpc
ports to start with a port that is based on the process id instead of a
hard-coded value.  The chosen ports are incremented for each running
instance similar to the previous code except the p2p and rpc ports and
now split into ranges instead of being 2 apart.

This is being done because the previous code only worked for a single
process which means it prevented the ability to run tests in parallel.

The new approach will work with multiple processes, however it must be
stated that there is still a very small probability that the stars could
align resulting in the same ports being selected.

Finally, this also reverts the recent change to run tests serially since
this fixes the underlying cause for that change.
2016-09-20 16:59:37 -05:00
Olaoluwa Osuntokun
daac24626e build: execute tests across all packages serially
This modifies the goclean.sh to execute all the tests amongst
the packages serially. The default behavior of the `go test` command is
to execute all tests in parallel amongst the listed packages. This
behavior can at times cause tests which use the `rpctest` package to
fail due to multiple `btcd` nodes attempting to bind to the same port
simultaneously. As only one node can successfully bind to the port, the
btcd processes for the other concurrent harness instances exit silently
causing the RPC clients to fail with connection timeouts as their
target process no longer exists. Executing all tests serially
eliminates such a race condition which can cause non-deterministic test
failures.
2016-09-20 01:16:08 -05:00
Dave Collins
2b780d16b0 Improve HTTP error handling.
This modifies the code which handles failed server responses to attempt
to unmarshal the response bytes as a regular JSON-RPC response
regardless of the HTTP status code and, if that fails, return an error
that includes the status code as well as the raw response bytes.

This is being done because some JSON-RPC servers, such as Bitcoin Core,
don't follow the intended HTTP status code meanings.  For example, if
you request a block that doesn't exist, it returns a status code of 500
which is supposed to mean internal server error instead of a standard
200 status code (since the HTTP request itself was successful) along
with the JSON-RPC error response.

The result is that errors from Core will now show up as an actual
RPCError instead of an error with the raw JSON bytes.

This also has the benefit of returning the HTTP status code in the
error for real HTTP failure cases such as 401 authentication failures,
which previously would just be an empty error when used against Core
since it doesn't return the actual response along with the status code
as it should.
2016-09-16 14:52:50 -05:00
Chris Martin
c20db1cf14 blockchain: make scriptval_test fail more nicely
Return early when failing on len(blocks) > 1.

Check for len(blocks) == 0 so we can fail on that condition with an
error message instead of a panic.
2016-09-16 15:44:27 -04:00
David Hill
6e5c0a7904 Fix GetBlockVerbose API.
This changes the GetBlockVersion API to not send a third parameter.
The third parameter is a boolean that expands the transaction data
structures as well.  However, Bitcore Core does not recognize this
feature.

GetBlockVerbose now only takes a hash value.

Users of the GetBlockVerbose(hash, true) must now use
GetBlockVerboseTx(hash).
2016-09-16 12:35:52 -04:00
Dave Collins
982229439a Return wire.MsgBlock from GetBlock(Async).
This modifies the return value of the GetBlock and GetBlockAsync
functions to a raw *wire.MsgBlock instead of *btcutil.Block.

This is being done because a btcutil.Block assumes there is a height
associated with the block and the block height is not part of a raw
serialized block as returned by the underlying non-verbose RPC API
getblock call and thus is not available to populate a btcutil.Block
correctly.

A raw wire.Block correctly matches what the underlying RPC call is
actually returning and thus avoids the confusion that could occur.
2016-09-15 14:06:36 -05:00
David Hill
96ed4cb5b0 Update installation section in README.md
Have users use -u along with go get to pull the latest
dependencies.
2016-09-15 14:25:23 -04:00
David Hill
8ff9623a8c Add API for getblockheader
GetBlockHeader returns the blockheader of the specified blockhash.
GetBlockHeaderVerbose returns the data structure with information about
the blockheader of the specified blockhash.

Fixes #89
2016-09-15 13:39:27 -04:00
Dave Collins
fb90c334df
rpctest: Cleanup resources on failed setup.
This modifies the rpctest harness to call the TearDown function in the
SetUp error path.  This ensures any resources, such as temp directories,
that were created during setup before whatever the failure that caused
the error are properly cleaned up.
2016-09-13 17:19:32 -05:00
Chris Martin
0ddd10add6 Remove extra backtick in README.md 2016-09-13 15:10:48 -04:00
Marco Peereboom
5e93b1664e config: Replace log outputs with fmt.Fprintln.
This corrects the case where any errors that might have happened when
creating the config file were not being displayed due to the logging
system not being initialized yet.

While here, consistently use fmt.Fprint{f,ln} throughout the loadConfig
function even after the logging system is initialized since it will
help prevent copy/paste errors such as the one that induced this change
to begin with.
2016-09-12 02:21:37 -05:00
Dave Collins
2ef82e7db3
mempool: Improve tx input standard checks.
This changes the transaction input standardness checks as follows:

- Allow any script in a pay-to-script-hash transaction to be relayed and
  mined so long as it has no more than 15 signature operations
- Remove the obsolete checks which naively calculated the number of
  expected inputs in favor of the more accurate ScriptVerifyCleanStack
  and ScriptVerifySigPushOnly functionality of the script engine that was
  added after these checks.
2016-08-24 23:16:23 -05:00
Olaoluwa Osuntokun
c6d50b7abf
rpctest: ensure the main harness rejects non-standard transactions
This commit modifies the current set of integration tests to ensure
that that the main harness always rejects non-standard transactions.

With this in place, even though we’re using simnet parameters, we
ensure that transaction acceptance/validation is identical to that of
main net.
2016-08-24 15:43:35 -07:00