Commit graph

2420 commits

Author SHA1 Message Date
David Hill
4494f0f852 txscript: Remove OP_SMALLDATA 2016-10-21 15:18:42 -04:00
Javed Khan
f6cd49ac51 peer: rename peer.Connect to AssociateConnection 2016-10-21 14:13:18 -05:00
Dave Collins
0731f2ddc9 txscript: Cleanup and improve NullDataScript tests.
This modifies the recently-added NullDataScript function in several
ways in an effort to make them more consistent with the tests in the
rest of the code base and improve/correct the logic:

- Use the hexToBytes and mustParseShortForm functions
- Consistently format the test errors
- Replace the valid bool flag with an expected error and test against it
- Ensure the returned script type is the expected type in all cases
2016-10-21 10:26:09 -05:00
DanielKrawisz
b77654f8d4 txscript: Add null data script creator
This adds a new function named NullDataScript to the txscript package that returns a provably-pruneable  OP_RETURN script with the provided data.  The function will return an error if the provided data is larger than the maximum allowed length for a nulldata script to be be considered standard.
2016-10-21 09:37:48 -05:00
Josh Rickmar
d0a9c03844 Concurrently handle websocket client JSON-RPC requests. 2016-10-20 19:55:58 -04:00
Dave Collins
da04285e0d
rpctest: Choose flags based on provided params.
This modifies the rpctest framework to start btcd with the appropriate
network flags depending on the provided parameters.

Previously, it always started btcd with --simnet even if other
parameters, such as those for the regression test network, were
provided.
2016-10-20 12:19:58 -05:00
Dave Collins
49cbaf23dd
blockchain: Support small coinbase block size
This modifies the ExtractCoinbaseHeight function to recognize small
canonically serialized block heights in coinbase scripts of blocks
higher than version 2.

This allows regression test chains in which blocks encode the serialized
height in the coinbase starting from block 1.
2016-10-20 12:09:09 -05:00
Dave Collins
59a3fc2f66 txscript: Consolidate tests into txscript package.
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.

Also, do some light cleanup on a few tests while here.
2016-10-20 09:28:33 -05:00
Dave Collins
b60e3547d2 txscript: Correct nulldata standardness check.
This corrects the isNullData standard transaction type test to work
properly with canonically-encoded data pushes.  In particular, single
byte data pushes that are small integers (0-16) are converted to the
equivalent numeric opcodes when canonically encoded and the code failed
to detect them properly.

It also adds several tests to ensure that both canonical and
non-canonical nulldata scripts are recognized properly and modifies the
test failure print to include the script that failed.

This does not affect consensus since it is just a standardness check.
2016-10-20 01:44:58 -05:00
David Hill
07e1e308f1 rpc: Add localaddr and relaytxes to getpeerinfo 2016-10-19 19:58:50 -04:00
David Hill
3b5bb9fd43 btcjson: Add preciousblock 2016-10-19 14:08:32 -05:00
David Hill
403aaf5cf3 rpcserver: avoid nested decodescript p2sh addrs 2016-10-19 13:59:50 -05:00
Olaoluwa Osuntokun
e7caccc866
mempool: transaction finality checks now use median-time-past
This coincides with the mempool only, policy change which enforces
transaction finality according to the median-time-past rather than
blockheader timestamps. The behavior is pre-cursor to full blown BIP
113 consensus deployment, and subsequent activation.

As a result, the TimeSource field in the mempoolConfig is no longer
needed so it has been removed. Additionally, checkTransactionStandard has been
modified to instead take a time.Time as the mempool is no longer explicitly
dependant on a Chain instance.
2016-10-19 11:13:34 -07:00
Olaoluwa Osuntokun
a82f67b538
mempool: add closure to compute median time past to config
This commit adds an additional closure function to the mempool’s config
which computes the median time past from the point of view of the best
node in the chain. The mempool test harness has also been updated to allow
setting a mock median time past for testing purposes.

In addition to increasing the testability of the mempool, this commit
should also speed up transaction and block validation for BIP 113 as
the MTP no longer needs to be re-calculated each time from scratch.
2016-10-19 11:13:25 -07:00
David Hill
a6bf1d9850 txscript: Implement CheckSequenceVerify (BIP0112) 2016-10-19 12:06:44 -04:00
Dave Collins
fdfa07b0be
btcec: Consolidate tests into the btcec package.
Putting the test code in the same package makes it easier for forks
since they don't have to change the import paths as much and it also
gets rid of the need for internal_test.go to bridge.

Also, remove the exception from the lint checks about returning the
unexported type since it is no longer required.
2016-10-19 00:55:23 -05:00
David Hill
b1621332cc Optimize by removing defers
defer's are nice for readability but they do add overhead.  This
gets rid of defer's where it is just as easy as not to use one.
2016-10-18 17:56:51 -04:00
Jimmy Song
294b5d46da btcec: Add regression tests for field.go.
This adds new tests to the TestNormalize, TestMul, TestAdd2 functions
which trigger an issue with modular reduction that was fixed in the
prevous commit to prevent regressions.
2016-10-18 16:21:45 -05:00
Dave Collins
a52eb04aaa
btcec: Ensure reduction when > P in all cases.
As noted in issue #706, the existing code had an issue where the
normalized result was > P when both the first and second words of the
field representation being normalized were BOTH greater than or equal to
the first and second words of P.  Although this condition is rare in
practice, it needs to be handled properly.

This resolves the issue by comparing the low words in the final
reduction step against the normalized low order prime bits to ensure the
final subtraction occurs correctly any time they're > P.  This approach
retains the constant time property as well.
2016-10-18 16:21:36 -05:00
David Hill
d009185a56 peer: Implement feefilter p2p message (bip0133) 2016-10-17 15:45:56 -04:00
David Hill
9935fe5dba wire: Bump minor due to feefilter addition 2016-10-17 13:45:12 -04:00
David Hill
ca4e9b82d6 wire: implement feefilter message (bip0133)
feefilter is used to request the receiving peer does not announce any
transactions below the specified minimum fee rate.
2016-10-17 13:33:16 -04:00
Dave Collins
f21410e47c
blockchain: Add block validation infrastructure.
This adds a full-blown testing infrastructure in order to test consensus
validation rules.  It is built around the idea of dynamically generating
full blocks that target specific rules linked together to form a block
chain.  In order to properly test the rules, each test instance starts
with a valid block that is then modified in the specific way needed to
test a specific rule.

Blocks which exercise following rules have been added for this initial
version.  These tests were largely ported from the original Java-based
'official' block acceptance tests as well as some additional tests
available in the Core python port.  It is expected that further tests
can be added over time as consensus rules change.

* Enough valid blocks to have a stable base of mature coinbases to spend
  for futher tests
* Basic forking and chain reorganization
* Double spends on forks
* Too much proof-of-work coinbase (extending main chain, in block that
  forces a reorg, and in a valid fork)
* Max and too many signature operations via various combinations of
  OP_CHECKSIG, OP_MULTISIG, OP_CHECKSIGVERIFY, and OP_MULTISIGVERIFY
* Too many and max signature operations with offending sigop after
  invalid data push
* Max and too many signature operations via pay-to-script-hash redeem
  scripts
* Attempt to spend tx created on a different fork
* Attempt to spend immature coinbase (on main chain and fork)
* Max size block and block that exceeds the max size
* Children of rejected blocks are either orphans or rejected
* Coinbase script too small and too large
* Max length coinbase script
* Attempt to spend tx in blocks that failed to connect
* Valid non-coinbase tx in place of coinbase
* Block with no transactions
* Invalid proof-of-work
* Block with a timestamp too far in the future
* Invalid merkle root
* Invalid proof-of-work limit (bits header field)
* Negative proof-of-work limit (bits header field)
* Two coinbase transactions
* Duplicate transactions
* Spend from transaction that does not exist
* Timestamp exactly at and one second after the median time
* Blocks with same hash via merkle root tricks
* Spend from transaction index that is out of range
* Transaction that spends more that its inputs provide
* Transaction with same hash as an existing tx that has not been
  fully spent (BIP0030)
* Non-final coinbase and non-coinbase txns
* Max size block with canonical encoding which exceeds max size with
  non-canonical encoding
* Spend from transaction earlier in same block
* Spend from transaction later in same block
* Double spend transaction from earlier in same block
* Coinbase that pays more than subsidy + fees
* Coinbase that includes subsidy + fees
* Invalid opcode in dead execution path
* Reorganization of txns with OP_RETURN outputs
* Spend of an OP_RETURN output
* Transaction with multiple OP_RETURN outputs
* Large max-sized block reorganization test (disabled by default since
  it takes a long time and a lot of memory to run)

Finally, the README.md files in the main and docs directories have been
updated to reflect the use of the new testing framework.
2016-10-17 12:16:53 -05:00
Dave Collins
1cba5c8fc0
blockchain: Remove exported CalcPastTimeMedian func.
This removes the exported CalcPastTimeMedian function from the
blockchain package as it is no longer needed since the information is
now available via the BestState snapshot.

Also, update the only known caller of this, which is the chain state in
block manager, to use the snapshot instead.  In reality, now that
everything the block manager chain state provides is available via the
blockchain BestState snapshot, the entire thing can be removed, however
that will be done in a separate to commit to keep the changes targeted.
2016-10-17 10:41:25 -05:00
Dave Collins
e88f2d7bf4
mempool: Add test for max orphan entry eviction.
This adds a test to the mempool for ensuring that orphans are evicted
when exceeding the max orphan policy setting as expected.
2016-10-17 10:33:09 -05:00
Dave Collins
8965d88893
peer: Strictly enforce bloom filter service bit.
This makes the enforcement of the bloom filter service bit much more
strict.  In particular, it does the following:

- Moves the enforcement of the bloom filter service bit out of the peer
  package and into the server so the server can ban as necessary
- Disconnect peers that send filter commands when the server is
  configured to disable them regardless of the protocol version
- Bans peers that are a high enough protocol version that they are
  supposed to observe the service bit is disabled, but ignore it and
  send filter commands regardless.

As an added bonus, this fixes the old logic which had a bug in that it
was examining the *remote* peer's supported services in order to choose
whether or not to disconnect instead of the *local* server's supported
services.
2016-10-16 02:19:28 -05:00
Dave Collins
77913ad2e8
blockchain: Expose main chain flag on ProcessBlock.
This modifies the blockchain.ProcessBlock function to return an
additional boolean as the first parameter which indicates whether or not
the block ended up on the main chain.

This is primarily useful for upcoming test code that needs to be able to
tell the difference between a block accepted to a side chain and a block
that either extends the main chain or causes a reorganize that causes it
to become the main chain.  However, it is also useful for the addblock
utility since it allows a better error in the case a file with out of
order blocks is provided.
2016-10-13 16:47:50 -05:00
David Hill
42a4366ba8 addrmgr: Fix AddressCache to not include nils
allAddr was being allocated with counters instead of the actual size
of the address map.  This led to the possibility of including nils
in the returned slice, which resulted in a panic.
2016-09-26 12:16:40 -05:00
Dave Collins
754c4fbe0c
rpctest: Gate rpctest-based behind a build tag.
This adds a new build tag named rpctest which must be set in order for
rpctest-based tests to be executed.  The new build tag is also added to
the goclean.sh script which is executed by Travis during continuous
integration builds.

This change is being made because the rpctest framework requires
additional careful user configuration to ensure the version of btcd
under test can be programmatically launched from the system path with
all of the necessary ports open whereas all of the other tests are
self-contained within the test binary itself.

Since said additional configuration is typically not done, it leads to a
lot of false positives.  Putting the tests behind a build tag allows
them to remain to be available and run during continuous integration
without imposing the additional configuration requirements on users.
2016-09-26 01:20:31 -05:00
David Hill
99165eb558 rpctest: Fix typo
Use os.Getpid() to get the process ID, not os.Getppid(), which returns
the parent process ID.  This resulted in multiple calls to
generateListeningAddresses() getting the same listening ports.
2016-09-22 00:14:51 -04:00
David Hill
5ec83d23f3 Update dependencies and API usage. 2016-09-21 20:50:55 -04:00
Dave Collins
7cf9ec8190
rpctest: Use ports based on the process id.
This modifies the ports that are selected for use for the p2p and rpc
ports to start with a port that is based on the process id instead of a
hard-coded value.  The chosen ports are incremented for each running
instance similar to the previous code except the p2p and rpc ports and
now split into ranges instead of being 2 apart.

This is being done because the previous code only worked for a single
process which means it prevented the ability to run tests in parallel.

The new approach will work with multiple processes, however it must be
stated that there is still a very small probability that the stars could
align resulting in the same ports being selected.

Finally, this also reverts the recent change to run tests serially since
this fixes the underlying cause for that change.
2016-09-20 16:59:37 -05:00
Olaoluwa Osuntokun
daac24626e build: execute tests across all packages serially
This modifies the goclean.sh to execute all the tests amongst
the packages serially. The default behavior of the `go test` command is
to execute all tests in parallel amongst the listed packages. This
behavior can at times cause tests which use the `rpctest` package to
fail due to multiple `btcd` nodes attempting to bind to the same port
simultaneously. As only one node can successfully bind to the port, the
btcd processes for the other concurrent harness instances exit silently
causing the RPC clients to fail with connection timeouts as their
target process no longer exists. Executing all tests serially
eliminates such a race condition which can cause non-deterministic test
failures.
2016-09-20 01:16:08 -05:00
Chris Martin
c20db1cf14 blockchain: make scriptval_test fail more nicely
Return early when failing on len(blocks) > 1.

Check for len(blocks) == 0 so we can fail on that condition with an
error message instead of a panic.
2016-09-16 15:44:27 -04:00
Dave Collins
fb90c334df
rpctest: Cleanup resources on failed setup.
This modifies the rpctest harness to call the TearDown function in the
SetUp error path.  This ensures any resources, such as temp directories,
that were created during setup before whatever the failure that caused
the error are properly cleaned up.
2016-09-13 17:19:32 -05:00
Chris Martin
0ddd10add6 Remove extra backtick in README.md 2016-09-13 15:10:48 -04:00
Marco Peereboom
5e93b1664e config: Replace log outputs with fmt.Fprintln.
This corrects the case where any errors that might have happened when
creating the config file were not being displayed due to the logging
system not being initialized yet.

While here, consistently use fmt.Fprint{f,ln} throughout the loadConfig
function even after the logging system is initialized since it will
help prevent copy/paste errors such as the one that induced this change
to begin with.
2016-09-12 02:21:37 -05:00
Dave Collins
2ef82e7db3
mempool: Improve tx input standard checks.
This changes the transaction input standardness checks as follows:

- Allow any script in a pay-to-script-hash transaction to be relayed and
  mined so long as it has no more than 15 signature operations
- Remove the obsolete checks which naively calculated the number of
  expected inputs in favor of the more accurate ScriptVerifyCleanStack
  and ScriptVerifySigPushOnly functionality of the script engine that was
  added after these checks.
2016-08-24 23:16:23 -05:00
Olaoluwa Osuntokun
c6d50b7abf
rpctest: ensure the main harness rejects non-standard transactions
This commit modifies the current set of integration tests to ensure
that that the main harness always rejects non-standard transactions.

With this in place, even though we’re using simnet parameters, we
ensure that transaction acceptance/validation is identical to that of
main net.
2016-08-24 15:43:35 -07:00
Olaoluwa Osuntokun
815ded348e
config: introduce new flags to accept/reject non-std transactions
This commit adds two new cli flags: one for accepting non-std
transactions, and the other for rejecting non-std transactions.

The two flag are rejected when using concurrently. Config parsing is
set up such that, the desired policy expressed via the config always
overrides the policy set by default for a particular chain.

The doc.go files and the sample-btcd.conf file have been updated to document
the new flags exposing further policy control.
2016-08-24 15:43:26 -07:00
Olaoluwa Osuntokun
dc5486a579
mempool: add non-standard tx relaying to policy config 2016-08-24 15:43:22 -07:00
Dave Collins
15bace88dc
mempool: Add basic test harness infrastructure.
This adds a basic test harness infrastructure for the mempool package
which aims to make writing tests for it much easier.

The harness provides functionality for creating and signing transactions
as well as a fake chain that provides utxos for use in generating valid
transactions and allows an arbitrary chain height to be set.  In order
to simplify transaction creation, a single signing key and payment
address is used throughout and a convenience type for spendable outputs
is provided.

The harness is initialized with a spendable coinbase output by default
and the fake chain height set to the maturity height needed to ensure
the provided output is in fact spendable as well as a policy that is
suitable for testing.

Since tests are in the same package and each harness provides a unique
pool and fake chain instance, the tests can safely reach into the pool
policy, or any other state, and change it for a given harness without
affecting the others.

In order to be able to make use of the existing blockchain.Viewpoint
type, a Clone method has been to the UtxoEntry type which allows the
fake chain instance to keep a single view with the actual available
unspent utxos while the mempool ends up fetching a subset of the view
with the specifically requested entries cloned.

To demo the harness, this also contains a couple of tests which make use
of it:

- TestSimpleOrphanChain -- Ensures an entire chain of orphans is
  properly accepted and connects up when the missing parent transaction
  is added
- TestOrphanRejects -- Ensure orphans are actually rejected when the
  flag on ProcessTransactions is set to reject them
2016-08-24 10:04:40 -05:00
David Hill
a109bea3f1 mempool: unexport the mutex
callers should not need to lock/unlock the mempool themselves.
2016-08-23 14:59:48 -04:00
Dave Collins
641182b2ad
mempool: Break dependency on chain instance.
This modifies the config for the new mempool package such that it takes
a callback function to obtain the best chain height instead of requiring
a fully initialized blockchain.BlockChain instance.

This will make it much easier to test the mempool since the tests will
be able to provide their own height function to test various
functionality without having create and manipulate full blocks and chain
instances.
2016-08-23 12:29:45 -05:00
Dave Collins
c57c18c8e2
blockchain: Add median time to state snapshot.
This adds a new field to the best chain state snapshot for the
calculated past median time as returned by the calcPastMedianTime
function.  This is useful since it provides fast access to it without
having to acquire the chain lock which is needed to recalculate it.

This will ultimately allow the associated exported function to be
removed since it only exists to be able to calculate this exact value,
however this commit only introduces the new field in order to keep the
changes minimal.
2016-08-23 01:29:11 -05:00
Olaoluwa Osuntokun
bfe2ba4191
rpctest: prevent leaking processes in tests due to panics
This commit adds a `defer` statement at the top of  `TestRpcServer`
which will attempt a `recover` which tears down all active harnesses in
the event that one of the tests causes a panic in the main goroutine.

Before this commit, if a buggy test caused a panic while all integration
tests were being executed, then any active harnesses would fail to be
properly torn down. This would cause the running btcd processes to be
leaked, possibly interfering with future test runs until the process was
manually killed. This commit fixes such behavior.

In order to aide in debugging, when a test panics, the test number is
printed out along with a full stack-trace from the start of the test to
the panic point.
2016-08-22 17:09:18 -07:00
Dave Collins
6cf60b5fae
rpctest: Correct several issues in tests and joins.
This corrects several issues with the new rpctest package.

The following is an overview of the issues that have been corrected:

- The JoinNodes code was incorrect for both mempool and blocks.
  - For mempool it was only checking the first node against itself
  - For blocks it was only testing the height instead of hash and height
    which means the chains could be different and it would claim they're
    synced
- The test for mempool joins was inaccurate in a few ways:
  - It was generating separate chains such that the generated
    transaction was invalid (an orphan) on one chain, but not
    the other
  - Mempools are not automatically synced when nodes are connected and
    there is a large random delay before any transaction rebroadcast
    happens, so it can't be relied on for the purposes of this test
- The test for block joins was generating two independent chains of the
  same height with the same difficulty and was only passing due to the
  aforementioned bug in JoinNodes
- All of the ConnectNode calls were connecting the main harness outbound
  to the local test harness instances
  - This is not correct because ConnectNode makes the outbound
    connection persistent, which means once the local test harness is
    gone, it would keep trying to connect for the remainder of the tests
    to a node that is never coming back and instead ends up connecting to
    an independent test harness.
2016-08-21 15:10:23 -05:00
jadeblaquiere
47ced81d44 don't spin forever looking for peers (#724) 2016-08-20 22:11:12 -05:00
Olaoluwa Osuntokun
7c4b169faa
build: install btcd binary within Travis container
This commit modifies the existing Travis configuration to also install
the btcd binary within the container’s PATH.

This change is necessary in order for tests written against the new
rpctest package to work properly within the Travis testing environment.
2016-08-19 17:43:42 -05:00
Olaoluwa Osuntokun
c3d5371615
build: update dependancies required by rpctest 2016-08-19 17:42:25 -05:00