Commit graph

3546 commits

Author SHA1 Message Date
Marco Peereboom
ada49f1413 Add a flag to enable live profiling.
The profile information can be seen with a browser on e.g.
http://localhost:6060/debug/pprof/
Alternatively, one can use the pprof tool as described at
http://golang.org/pkg/net/http/pprof/

ok davec
2013-09-17 17:40:27 -04:00
Dave Collins
7609ff396a Comment multi-peer connect while code is under dev. 2013-09-17 10:58:16 -05:00
Dave Collins
92a8605b24 Continue work on addrmgr and multi-peer.
- Remove leftover debug log prints
- Increment waitgroup outside of goroutine
- Various comment and log message consistency
- Combine peer setup and newPeer -> newInboundPeer
- Save and load peers.json to/from cfg.DataDir
- Only claim addrmgr needs more addresses when it has less than 1000
- Add warning if unkown peer on orphan block.
2013-09-17 10:48:47 -05:00
Owain G. Ainsworth
6c05e9d475 Flesh out addrmanger with some basic functionality.
Use it to add multiple peer support. We try and keep 8 outbound peers
active at all times.

This address manager is not as complete as the one in bitcoind yet, but
additional functionality is being worked on.

We currently handle (in a similar manner to bitcoind):

- biasing between new and already tried addresses based on number of connected
  peers.
- rejection of non-default ports until desparate
- address selection probabilities based on last successful connection and number
  of failures.
- routability checks based on known unroutable subnets.
- only connecting to each network `group' once at any one time.

We currently lack support for:
- tor ``addresses'' (an .onion address encoded in 64 bytes of ip address)
- full state save and restore (we just save a json with the list of known
  addresses in it)
- multiple buckets for new and tried addresses selected by a hash of address and
 source.  The current algorithm functions the same as bitcoind would with only
 one bucket for new and tried (making the address cache rather smaller than it
 otherwise would be).
2013-09-15 20:25:55 -05:00
Dave Collins
79bb334b9a Remove old regression test db in regtest mode.
The regression test mode is special in that the 'official' block test
suite requires an empty database to work properly.  Rather than having to
manual go delete it before each test, add code to automatically delete the
old regression test database when in regression test mode.
2013-09-15 16:58:02 -05:00
Dave Collins
f3cd8174bb Use the new btcdb.SupportedDBs for known db types. 2013-09-15 15:21:46 -05:00
Dave Collins
62e38e29e5 Add a new function named SupportedDBs.
This function allows the callers to programatically ascertain which
database backend drivers are registered and therefore supported.
2013-09-15 15:18:46 -05:00
Dave Collins
629a1c9d06 Rework the data path and db type handling.
This commit modifies the way the data paths are handled.  Since there will
ultimately be more data associated with each network than just the block
database, the data path has been modified to be "namespaced" based on the
network.  This allows all data associated with a specific network to
simply use the data path without having to worry about conflicts with data
from other networks.

In addition, this commit renames the block database to "blocks" plus a
suffix which denotes the database type.  This prevents issues that would
otherwise arise if the user decides to use a different database type and
a file/folder with the same name already eixsts but is of the old database
type.  For most users this won't matter, but it does provide nice
properties for testing and development as well since it makes it easy to
go back and forth between database types.

This commit also includes code to upgrade the old database paths to the
new ones so the change is seamless for the user.

Finally, bump the version to 0.2.0.
2013-09-15 14:25:32 -05:00
Dave Collins
252ecf8b00 Add basic infrastructure for upgrading btcd.
This commit adds a basic infrastructure to allow upgrades to happen to
btcd as needed.  This paves the way for the upcoming data path changes to
be automatically updated for the user as needed and also ensures any
future changes that might require upgrades already have an established
way of performing the needed upgrades.
2013-09-15 13:40:26 -05:00
Dave Collins
1c9a7095b3 Change --dbdir flag to --datadir.
This change paves the way for saving more than just the block database to
the filesystem (such as address manager data, index data, etc) where the
name "dbdir" no longer makes sense.
2013-09-13 18:02:10 -05:00
Dave Collins
a2946ea14b Don't add "btcd.ldb" directory to provided path. 2013-09-13 17:32:04 -05:00
Dave Collins
be3dc1837c Gofmt latest dbtype config additions. 2013-09-13 14:52:34 -05:00
Dale Rahn
1530141ba0 This query should be performed as part of the transaction, not outside it. 2013-09-13 11:52:07 -04:00
Dale Rahn
9e27c82a23 Fix error messages (pointed out by go vet) 2013-09-13 11:49:30 -04:00
Dale Rahn
5e4c78a1b7 Fix a shadowed return parameter. 2013-09-13 10:54:45 -04:00
Dale Rahn
af7aa39624 Allow config file and command line to select between sqlite and levdldb 2013-09-13 10:08:19 -04:00
Dave Collins
33b65f943f Increment waitgroups before goroutines are started.
This commit changes the code so that all calls to .Add on waitgroups
happen before the associated goroutines are launched.   Doing this after
the goroutine could technically cause a race where the goroutine started
and finished before the main goroutine has a chance to increment the
counter.  In our particular case none of the goroutines exit quickly
enough for this to be an issue, but nevertheless the correct way should be
used.
2013-09-12 17:24:37 -05:00
Dave Collins
69efe46eb5 Fix a comment typo and comment outputBufferSize. 2013-09-12 12:11:46 -05:00
Dave Collins
e4577ab5f4 Syncrhonize access to the blockPeer map. 2013-09-12 12:10:48 -05:00
Dave Collins
f5c03696f7 Remove relay log message and update comment. 2013-09-12 09:29:42 -05:00
Dave Collins
b58787f87a Remove block relay from TODO list. 2013-09-12 09:15:38 -05:00
Dave Collins
121f7a47d4 Add block inventory relay.
This commit adds support for relaying blocks between peers.  It keeps
track of inventory that has either already been advertised to remote peers
or advertised by remote peers using a size-limited most recently used
cache.  This helps avoid relaying inventory the peer already knows as
much as possible while not allowing rogue peers to eat up arbitrary
amounts of memory with bogus inventory.
2013-09-11 20:14:13 -05:00
Dale Rahn
7d679a6228 Adjust Block height indicators properly when database is created. 2013-09-10 14:17:24 -04:00
Dave Collins
9049bcacec Update error message for zero hash change. 2013-09-10 10:47:49 -05:00
Dave Collins
d9404fe880 Correct test for zero hash versus nil on empty db. 2013-09-10 10:45:02 -05:00
Dale Rahn
adc73cbc9d quiet. 2013-09-10 11:25:13 -04:00
Dave Collins
85553133ce gofmt. 2013-09-10 10:09:44 -05:00
Dave Collins
30f2003301 Remove spurious log.Info prints. 2013-09-10 10:09:43 -05:00
Dave Collins
66d6f10d5d Fix NewestSha. 2013-09-10 10:09:42 -05:00
Dale Rahn
96f7305c29 One DB instad of many.
improved error handling

Fix testing, no point in running tests multiple times

Fix error when no blocks in database, reopen would misreport NewstSha() return
2013-09-10 10:05:26 -04:00
Dave Collins
adf7149838 Update for recent btcchain orphan notify change. 2013-09-09 19:58:38 -05:00
Dave Collins
b557a33f7c Send orphan hash in notifications instead of root.
Rather than sending the root of the an orphan chain when sending an orphan
notification, send the hash of the orphan block itself.  The caller can
then call the GetOrphanRoot function with the hash to get the root of the
orphan chain as needed.  This is being changed since it's cleaner and more
detministic sending the hash for the orphan block that was just processed
rather than sending a possibly different hash depending on whether there
is an orphan chain or not.
2013-09-09 19:35:36 -05:00
Dave Collins
3525552140 Update TODO list in README.md. 2013-09-09 11:01:05 -05:00
Dave Collins
afddca6870 Use log.Errorf for error messages with params. 2013-09-09 10:59:31 -05:00
Dave Collins
0ba8cb9187 Send notification in their own goroutine.
Since the notification channel is provided by the caller and it may or may
not be buffered, send notifications in their own goroutine so the chain
processing code does not have to wait around on the caller to process the
notification before continuing.
2013-09-06 11:05:04 -05:00
Dave Collins
cf7438a646 Rework getblocks handling.
This commit reworks the getblocks handling a bit to clean it up and match
the reference implementation handling.  In particular, it adds monitoring
for when peers request the final block advertised from a previous
getblocks message and automatically avertises the latest known block
inventory to trigger the peer to send another getblocks message.
2013-09-05 01:25:52 -05:00
Dave Collins
83a9bbd4dd Update getheaders unknown block locator handling.
When no blocks in the block locator are found, start with the block after
the genesis block.  This means the client will start over with the genesis
block if unknown block locators are provided.  This mirrors the behavior
in the reference implementation.
2013-09-04 20:46:44 -05:00
Dave Collins
f07d427837 Rework and correct getheaders handling.
This commit reworks the getheaders handling a bit to clean it up and match
the reference implementation handling.  In particular, in addition to the
normal handling where headers starting after the block locator up to the
stop hash are served, when no locator hashes are provided, the stop hash
acts as a way to specifically request that header.  Next, an empty headers
message is sent when no hashes provided by the block locator can be
found.  Finally, there was a bug that was limiting the number of headers
that could requested at once to 500 instead of the expected 2000.
2013-09-04 18:17:42 -05:00
Dave Collins
cbfee93b74 Modify syncing code to support multiple peers.
Previously the code was only designed to work with a single peer.  This
commit modifies the syncing code to deal with multiple peers.
2013-09-04 10:10:00 -05:00
Dave Collins
a69ba92006 Set services to remote peer services for all peers.
Rather than only setting the services field for inbound peers, set it for
all peers.  This field referes to the remote peer's services regardless of
inbound or outbound.
2013-09-03 13:34:27 -05:00
Dave Collins
83407ade61 Rework block fetching code.
This commit significantly reworks the fetching code to interop better with
bitcoind.  In particular, when an inventory message is sent, and the
remote peer requests the final block, the remote peer sends the current
end of the main chain to signal that there are more blocks to get.

Previously this code was automatically requesting more blocks when the
number of in-flight blocks was under a certain threshold.  The original
approach does help alleviate delays in the "request final, wait for
orphan, request more" round trip, but due to the aforementioned mechanism,
it leads to double requests and other subtle issues.
2013-08-30 13:04:50 -05:00
Dave Collins
49f6b7d35d Add checkpoint at block 250000. 2013-08-29 12:01:19 -05:00
Dave Collins
4ac899e26f Clear disconnected transactions.
Rather than removing disconnected transactions from the node viewpoint
transaction store, clear the entry instead.  This is needed for the
connect code to update transactions that were removed on the other side of
a chain fork.
2013-08-29 08:56:27 -05:00
Dave Collins
117765ba7c Correct error message in other input tx lookup. 2013-08-28 13:50:47 -05:00
Dave Collins
ffa56b437c Correct error msg for missing input transaction.
The error message for missing input transaction had the referenced and
referencing transactions backwards.
2013-08-28 13:46:17 -05:00
Dave Collins
fc804aaec0 Add block locator infastructure.
This commit adds a new type, BlockLocator, and supporting infrastructure
to support generating block locators from a provided hash and getting
a full block locator for the latest known block.  This will likely be
expanded in the future to provide the opposite functionality as well.
That is to say the ability to lookup a block using a block locator.
2013-08-28 11:45:21 -05:00
Dave Collins
fc1b2e54d3 Set best chain during GenerateInitialIndex.
This commit modifies the GenerateInitialIndex function to update the best
chain with each node as it is loaded.  This allows the best chain to be
set immediately upon generating the initial index which is a slight
performance optimization.
2013-08-28 10:42:38 -05:00
Dave Collins
d0be0ed5ed Add HaveInventory function.
This commit provides a new exported function, HaveInventory, which can
be used to determine if the already has the item referenced by passed
inventory vector.  Currently it only provides support for blocks and
cursory support for transactions in the main chain.  Types that are
unrecognized are returned as if they unknown as expected.
2013-08-22 16:14:10 -05:00
Dave Collins
5c6911c775 Add IsKnownOrphan function.
This commit provides a new exported function, IsKnownOrphan, which can
be used to determine if the passed hash is currently already known to the
chain as an orphan.
2013-08-22 14:47:58 -05:00
Dave Collins
9d5f855580 Export GetOrphanRoot.
This commit exposes GetOrphanRoot to callers.  It also modifies the code
to allow the function to be called in a concurrent safe manner.  This
allows the information to be safely obtained from multiple goroutines.
2013-08-22 14:47:24 -05:00