Commit graph

3469 commits

Author SHA1 Message Date
Dave Collins
1b54badde1 Improve efficiency of checkpoint tracking.
Previously the code performed a database query for every checkpoint (going
backwards) to find the latest known checkpoint on every block.  This was
particularly noticabled near the beginning of the block chain when there
are still several checkpoints that haven't been reached yet.

This commit changes the logic to cache the latest known checkpoint while
keeping track of when it needs to be updated once a new later known
checkpoint has been reached.

While here, also add a log message when a checkpoint has been reached and
verified.
2014-01-17 09:59:52 -06:00
Dave Collins
b11f1620e2 Remove warnings for errors that are returned.
ok @owainga.
2014-01-17 09:40:34 -06:00
Dave Collins
4c13410e62 Update README.md TODO. 2014-01-17 00:51:42 -06:00
Dave Collins
03394d6309 Bring README.md and doc.go examples up to date.
Since sqlite is now deprecated, update the examples to use levelb.  Also,
add false to the ProcessBlock calls for the fastAdd parameter.
2014-01-17 00:48:32 -06:00
Dave Collins
84f6089bc9 Rework and improve async script validation logic.
The previous script validation logic entailed starting up a hard-coded
number of goroutines to process the transaction scripts in parallel.  In
particular, one goroutine (up to 8 max) was started per transaction in a
block and another one was started for each input script pair in the
each transaction.  This resulted in 64 goroutines simultaneously running
scripts and verifying cryptographic signatures.  This could easily lead to
the overall system feeling sluggish.

Further the previous design could also result in bursty behavior since the
number of inputs to a transaction as well as its complexity can vary
widely between transactions.  For example, starting 2 goroutines (one to
process the transaction and one for actual script pair validation) to
verify a transaction with a single input was not desirable.

Finally, the previous design validated all transactions and inputs
regardless of a failure in one of the other scripts.  This really didn't
have a big impact since it's quite rare that blocks with invalid
verifications are being processed, but it was a potential way DoS vector.

This commit changes the logic in a few ways to improve things:

- The max number of validation goroutines is now based on the number of
  cores in the system
- All transaction inputs from all transactions in the block are collated
  into a single list which is fed through the aforementioned validation
  goroutines
- The validation CPU usage is much more consistent due to the collation of
  inputs
- A validation error in any goroutine immediately stops validation of all
  remaining inputs
- The errors have been improved to include context about what tx script
  pair failed as opposed to showing the information as a warning

This closes conformal/btcd#59.
2014-01-16 20:02:30 -06:00
David Hill
871481ce1b Implement submitblock.
Closes #61.
2014-01-15 15:07:43 -06:00
Dave Collins
e3122c1b1d Remove extra parnethesis in previous. 2014-01-15 10:19:51 -06:00
Dave Collins
fcdddd499f Use a more specific license adjective in README.md.
This was proposed by @apotheon.
2014-01-15 10:17:22 -06:00
Josh Rickmar
bd98836a2b Fix several bugs in the RPC server shutdown.
The RPC server was performing some of the shutdown logic in the wrong
order, that is, logging the the server has shut down, waiting for all
server goroutines to finish, and then closing a channel to notify
server goroutines to stop.  These three items have been reversed to
fix a hang where goroutines currently being waited on had not shut
down because they did not receive the notification.

While here, the server waitgroup was incremented for a goroutine that
was running without it, another select statement was added to stop a
duplicate close (which never occured last commit when I added the
select statements), and the "stopping rescan" logging was moved to
debug to make the ^C shutdown logging nicer.
2014-01-14 22:53:07 -05:00
Josh Rickmar
d8227c2751 Only close websocket disconnected chan if still open.
A channel cannot be closed multiple times, so use a select statement
to only close the channel if it can not be read from.
2014-01-14 21:47:27 -05:00
Josh Rickmar
86600a0356 Remove punctuation from end of logging for consistency. 2014-01-14 16:41:32 -05:00
Dave Collins
477b733ed9 Make Go 1.2 requirement more noticeable.
It was pointed out in #76 that if you arrived to the Update section
of the README without seeing the Installation section, the requirement for
Go 1.2 is easy to miss.  This commit builds the requirement in the
Installation section and adds it to the Updating section as well to
hopefully make it more noticable.
2014-01-14 15:28:02 -06:00
Josh Rickmar
051a9013ce Stop rescans when a websocket client disconnects.
Fixes #66.
2014-01-14 15:59:31 -05:00
Josh Rickmar
6abad1d8ac Change websocket handlers to return results.
This is the first step to fixing #66.
2014-01-14 13:15:22 -05:00
Dave Collins
6222b1d8cc Rework and Improve addblock utility.
The addblock utility was originally written as a quick debug tool during
initial development to populate blocks into the database.  However, now
that it has been designated as the standard way to import bootstrap.dat
(and indeed block data files in general), it was lacking a few features
such as properly checking against the chain rules and known good
checkpoints.

This commit reworks and improves the utility in several ways:

- Imported blocks are now checked against the chain rules including
  checkpoints to ensure they match the known good chain
- The utility now properly shuts down after processing all blocks
- Attempting to import orphan blocks (blocks which build off a block you
  don't yet have in the database) returns an error
- Blocks that are already known are now skipped instead of causing an
  error which means you can stop and restart the import mid-way through
  without issues or start it after you've already downloaded a
  portion of the chain
- The block height is no longer assumed to start at 0 which means input
  files that start later in the chain will work properly so long as you
  already have the chain at least up to the point of the block just before
  the first one in the input file
- Improved error handling and reporting
- How often the progress display is shown is now configurable
- Statistics about how many blocks were processed, imported, and already
  known are now displayed after the input file has been fully processed

This resolves comments made in #60.
2014-01-14 01:18:52 -06:00
Dave Collins
d8ec5bd33c Prepare for release 0.5.0. 2014-01-13 17:47:21 -06:00
Dave Collins
8b479794ab Add 0.5.0 deps to deps.txt. 2014-01-13 17:45:15 -06:00
Josh Rickmar
15f2ce4abd Fix a couple incorrect comments. 2014-01-13 17:16:58 -05:00
Dave Collins
c99a227df2 Fix a couple of comment typos. 2014-01-10 22:32:05 -06:00
Josh Rickmar
427fb3cd94 gencerts: Print error lowercased for consistency. 2014-01-10 17:57:14 -05:00
Josh Rickmar
aec17304a0 Add standalone gencerts utility to create RPC TLS certificates. 2014-01-10 16:59:54 -05:00
Josh Rickmar
035f8f82b7 Switch to btcutil for certificate generation. 2014-01-10 15:41:57 -05:00
David Hill
3a59e4d064 Set the Connection header to close 2014-01-10 10:57:05 -05:00
Dave Collins
dacf9d77c5 Comment the onion/proxy dial/lookup configuration. 2014-01-10 01:52:15 -06:00
Dave Collins
4f8c2a3aaf Move missing config file warn just before return.
The warning about a missing config file should only be shown after all
other configuration has succeeded so it's not shown when there are invalid
options specified.  Also add a comment about it where its intended
placement is for the future.
2014-01-10 01:34:50 -06:00
Dave Collins
d33e9b4165 Unexport and comment btcDial and btcLookup func.
These functions are at the package level and only apply within btcd, so
unexport them to be consistent.
2014-01-10 01:31:20 -06:00
Marco Peereboom
6578e7345f catch up to new goleveldb 2014-01-09 13:11:37 -06:00
Josh Rickmar
cd3084afcd Rework the btcwallet connection.
This changes the protocol between btcd and btcwallet to follow
JSON-RPC specifications sending notifications as requests with an
empty ID.

The notification request context handling has been greatly cleaned up
now that IDs no longer need to be saved when sending notifications.
2014-01-09 14:03:00 -05:00
Dale Rahn
a5cc3196b4 Clear fetchheaders fields if in that mode and syncpeer detaches. 2014-01-09 13:22:42 -05:00
Dave Collins
f93c8a4af1 Don't display btcctl getconnectioncount as float. 2014-01-09 12:17:26 -06:00
Dave Collins
af0750dd76 Don't display getblockccount from btcctl as float. 2014-01-09 12:15:46 -06:00
Josh Rickmar
05a0ba18b4 Kill GetBalancesCmd. 2014-01-09 13:14:53 -05:00
Dave Collins
b3f63cf35e Fix typo in fetch blocks log message. 2014-01-09 12:13:14 -06:00
Dave Collins
28f485a1d1 Fix a couple of style nits. 2014-01-09 11:36:50 -06:00
Josh Rickmar
33890d49e9 Fix a typo. 2014-01-09 11:51:51 -05:00
Marco Peereboom
371dfdd76a Kill last sqlite3 bits.
This was driving me insane when building with -a -v and it slowed down
the process dramatically.

ok davec
2014-01-09 10:26:17 -06:00
Dave Collins
6b18796af7 Add 2014 to copyright dates. 2014-01-08 23:54:52 -06:00
Dave Collins
18ac5c848a Add 2014 to copyright dates. 2014-01-08 23:52:54 -06:00
Dave Collins
58cab817f0 Add 2014 to copyright dates. 2014-01-08 23:51:37 -06:00
Dave Collins
bf90ed2142 Add 2014 to copyright dates. 2014-01-08 23:49:06 -06:00
Dave Collins
565f11409c Add 2014 to copyright dates. 2014-01-08 23:47:47 -06:00
Dave Collins
6c7f45fdb7 Add 2014 to copyright dates. 2014-01-08 23:44:08 -06:00
Josh Rickmar
d493181886 Add TxSpentNtfn to notify wallets spent outpoints. 2014-01-08 21:29:19 -05:00
Dave Collins
3946d84887 Make use of the new size hint functions in btcwire.
This commit changes a couple of sections which deal with large lists of
inventory vectors to use the new size hint functions recently added to
btcwire.  This allows a bit more efficiency since the size of the list is
known up front and we can therefore avoid dynamically growing the backing
array several times.  This also helps avoid a Go bug that leaks memory on
appends and GC churn.
2014-01-08 17:46:59 -06:00
Dave Collins
2458841855 Add tests for new size hint functions.
This brings the test coverage back up to 100%.
2014-01-08 17:43:18 -06:00
Dave Collins
497aac6d4a Add new funcs to allow size hints on getdata/inv.
This commit adds two new functions named NewMsgGetDataSizeHint and
NewMsgInvSizeHint.  These are intended to allow callers which know in
advance how large the inventory lists will grow the ability to provides
that information when creating the message.  This in turn provides a
mechanism to avoid the need to perform several grow operations of the
backing array when adding large number of inventory vectors.
2014-01-08 17:21:51 -06:00
Josh Rickmar
6e3f9e451b Rename RescanResultNtfn to ProcessedTxNtfn.
This notification should be generic enough to be used by both rescan
and normal block processing code, so give it a generic name too.
2014-01-08 15:23:59 -05:00
Dave Collins
462bc5a031 Update for recent btcscript API changes.
This commit changes all code which deals with extracting addresses from
scripts to use the btcscript API ExtractPkScriptAddrs which in turn makes
use of the new btcutil.Address interface.

This provides much cleaner code for dealing with arbitrary script
destinations which is extensible without having to churn the APIs if new
destination types are added.
2014-01-08 11:22:29 -06:00
Josh Rickmar
9ea77c00f2 Add RescanResultNtfn to notify rescan results. 2014-01-08 11:28:38 -05:00
John C. Vernaleo
6747e36f51 Slight increase in test coverage. 2014-01-08 10:31:54 -05:00