Commit graph

43 commits

Author SHA1 Message Date
Olaoluwa Osuntokun
9b9ef42f8a blockchain/indexers: proper handling of empty filters 2018-05-23 16:46:15 -07:00
Olaoluwa Osuntokun
e0943a84bd blockchain/indexers: add a bit more line spacing to cfindex.go 2018-05-23 16:46:15 -07:00
Alex
1c5f25bbf2 blockchain/cfindex: Factor out filter/header calc. 2018-05-23 16:46:15 -07:00
Alex
860100019f blockchain: fix --dropcfindex 2018-05-23 16:46:15 -07:00
Alex
ba7b5f3308 Use NBytes() instead of Bytes() in CFIndex 2018-05-23 16:46:15 -07:00
Alex
9780ef5997 Skip TxIn on coinbase transaction when indexing 2018-05-23 16:46:15 -07:00
Alex
6102e129c5 Fixed a couple of bugs and added --dropcfindex option 2018-05-23 16:46:15 -07:00
pedro martelletto
a5bf8941d5 Define FilterHeaderByBlockHash() 2018-05-23 16:46:15 -07:00
pedro martelletto
e620538343 Generate and store filter headers 2018-05-23 16:46:15 -07:00
pedro martelletto
b53c42f5dc Define dbFetch{Basic,Extended}Header() 2018-05-23 16:46:15 -07:00
pedro martelletto
7673859108 Create db buckets for committed filter hashes 2018-05-23 16:46:15 -07:00
pedro martelletto
ddfaed7f6f Delete extended filters in DisconnectBlock() 2018-05-23 16:46:15 -07:00
pedro martelletto
0a841fefcf Generate and store extended filters in ConnectBlock() 2018-05-23 16:46:15 -07:00
pedro martelletto
71c421db66 Differentiate between basic/extended filters in p2p/RPC 2018-05-23 16:46:15 -07:00
pedro martelletto
f703e18652 Populate filter #1 (extended) 2018-05-23 16:46:15 -07:00
pedro martelletto
f16da156c9 Include data pushes from pkscripts 2018-05-23 16:46:15 -07:00
pedro martelletto
396d28955c better separation between filters; comments 2018-05-23 16:46:15 -07:00
pedro martelletto
57995fd111 Start preparing the ground for layer {0,1} filters 2018-05-23 16:46:15 -07:00
pedro martelletto
472141f88d Switch to new GCS builder interface 2018-05-23 16:46:15 -07:00
pedro martelletto
6d6677b797 zap errCFEntry 2018-05-23 16:46:15 -07:00
pedro martelletto
92c8ec4094 cbfindex.go -> cfindex.go 2018-05-23 16:46:15 -07:00
pedro martelletto
6e5f650be9 CBFilter -> CFilter, discussed with davec@ 2018-05-23 16:46:15 -07:00
pedro martelletto
a77b1e00d5 FilterByBlockHash(), pointed out by davec@ 2018-05-23 16:46:15 -07:00
pedro martelletto
9f02951b0e CloneBytes() -> [:], pointed out by davec@ 2018-05-23 16:46:15 -07:00
pedro martelletto
8b8c7bcf05 Implement DisconnectBlock() for the CBF Indexer 2018-05-23 16:46:15 -07:00
pedro martelletto
43bf8db793 Look up filters from p2p GetCBF message 2018-05-23 16:46:15 -07:00
pedro martelletto
76926f8904 Roll in a dbFetchCBFIndexEntry() 2018-05-23 16:46:15 -07:00
pedro martelletto
9809f4ffdd Instrument basic logging 2018-05-23 16:46:15 -07:00
pedro martelletto
cf74c8c3ca Abstract filter generation to a separate function 2018-05-23 16:46:15 -07:00
pedro martelletto
05d2fdeb11 Rename a variable in CBF's ConnectBlock() 2018-05-23 16:46:15 -07:00
pedro martelletto
620ad5b6fb Store filter in a db block index 2018-05-23 16:46:15 -07:00
pedro martelletto
b2990e7999 Move filter generating code to CBF indexer 2018-05-23 16:46:15 -07:00
pedro martelletto
95cbe2a911 Add the skeleton of a CBF indexer 2018-05-23 16:46:15 -07:00
Nicola 'tekNico' Larosa
11fcd83963 btcd/multi: fix a number of typos in comments. 2018-01-25 23:23:59 -06:00
Dave Collins
8c883d1fca
blockchain/indexers: Allow interrupts.
This propagates the interrupt channel through to blockchain and the
indexers so that it is possible to interrupt long-running operations
such as catching up indexes.
2017-09-05 11:02:46 -05:00
Dave Collins
20910511e9
blockchain: Refactor to use new chain view.
- Remove inMainChain from block nodes since that can now be efficiently
  determined by using the chain view
- Track the best chain via a chain view instead of a single block node
  - Use the tip of the best chain view everywhere bestNode was used
  - Update chain view tip instead of updating best node
- Change reorg logic to use more efficient chain view fork finding logic
- Change block locator code over to use more efficient chain view logic
  - Remove now unused block-index-based block locator code
  - Move BlockLocator definition to chain.go
  - Move BlockLocatorFromHash and LatestBlockLocator to chain.go
    - Update both to use more efficient chain view logic
- Rework IsCheckpointCandidate to use block index and chain view
- Optimize MainChainHasBlock to use chain view instead of hitting db
  - Move to chain.go since it no longer involves database I/O
  - Removed error return since it can no longer fail
- Optimize BlockHeightByHash to use chain view instead of hitting db
  - Move to chain.go since it no longer involves database I/O
  - Removed error return since it can no longer fail
- Optimize BlockHashByHeight to use chain view instead of hitting db
  - Move to chain.go since it no longer involves database I/O
  - Removed error return since it can no longer fail
- Optimize HeightRange to use chain view instead of hitting db
  - Move to chain.go since it no longer involves database I/O
- Optimize BlockByHeight to use chain view for main chain check
- Optimize BlockByHash to use chain view for main chain check
2017-08-23 23:43:37 -05:00
Olaoluwa Osuntokun
137aabd631 blockchain/indexers: add addrindex support for p2wkh and p2wsh 2017-08-13 23:17:40 -05:00
Dave Collins
9918e2a561
multi: Update markdown files for GFM changes.
The github markdown interpreter has been changed such that it no longer
allows spaces in between the brackets and parenthesis of links and now
requires a newline in between anchors and other formatting.  This
updates all of the markdown files accordingly.

While here, it also corrects a couple of inconsistencies in some of the
README.md files.
2017-05-25 12:06:16 -05:00
Dave Collins
915fa6639b
multi: Simplify code per gosimple linter.
This simplifies the code based on the recommendations of the gosimple
lint tool.
2016-11-03 13:00:35 -05:00
Dave Collins
14b51fc5f8
multi: Correct misspellings detected by misspell. 2016-10-28 09:43:38 -05:00
Dave Collins
bd4e64d1d4 chainhash: Abstract hash logic to new package. (#729)
This is mostly a backport of some of the same modifications made in
Decred along with a few additional things cleaned up.  In particular,
this updates the code to make use of the new chainhash package.

Also, since this required API changes anyways and the hash algorithm is
no longer tied specifically to SHA, all other functions throughout the
code base which had "Sha" in their name have been changed to Hash so
they are not incorrectly implying the hash algorithm.

The following is an overview of the changes:

- Remove the wire.ShaHash type
- Update all references to wire.ShaHash to the new chainhash.Hash type
- Rename the following functions and update all references:
  - wire.BlockHeader.BlockSha -> BlockHash
  - wire.MsgBlock.BlockSha -> BlockHash
  - wire.MsgBlock.TxShas -> TxHashes
  - wire.MsgTx.TxSha -> TxHash
  - blockchain.ShaHashToBig -> HashToBig
  - peer.ShaFunc -> peer.HashFunc
- Rename all variables that included sha in their name to include hash
  instead
- Update for function name changes in other dependent packages such as
  btcutil
- Update copyright dates on all modified files
- Update glide.lock file to use the required version of btcutil
2016-08-08 14:04:33 -05:00
Dave Collins
b580cdb7d3 database: Replace with new version.
This commit removes the old database package, moves the new package into
its place, and updates all imports accordingly.
2016-04-12 14:55:15 -05:00
Dave Collins
7c174620f7 indexers: Implement optional tx/address indexes.
This introduces a new indexing infrastructure for supporting optional
indexes using the new database and blockchain infrastructure along with
two concrete indexer implementations which provide both a
transaction-by-hash and a transaction-by-address index.

The new infrastructure is mostly separated into a package named indexers
which is housed under the blockchain package.  In order to support this,
a new interface named IndexManager has been introduced in the blockchain
package which provides methods to be notified when the chain has been
initialized and when blocks are connected and disconnected from the main
chain.  A concrete implementation of an index manager is provided by the
new indexers package.

The new indexers package also provides a new interface named Indexer
which allows the index manager to manage concrete index implementations
which conform to the interface.

The following is high level overview of the main index infrastructure
changes:

- Define a new IndexManager interface in the blockchain package and
  modify the package to make use of the interface when specified
- Create a new indexers package
  - Provides an Index interface which allows concrete indexes to plugin
    to an index manager
  - Provides a concrete IndexManager implementation
    - Handles the lifecycle of all indexes it manages
    - Tracks the index tips
    - Handles catching up disabled indexes that have been reenabled
    - Handles reorgs while the index was disabled
    - Invokes the appropriate methods for all managed indexes to allow
      them to index and deindex the blocks and transactions
  - Implement a transaction-by-hash index
    - Makes use of internal block IDs to save a significant amount of
      space and indexing costs over the old transaction index format
  - Implement a transaction-by-address index
    - Makes use of a leveling scheme in order to provide a good tradeoff
      between space required and indexing costs
- Supports enabling and disabling indexes at will
- Support the ability to drop indexes if they are no longer desired

The following is an overview of the btcd changes:

- Add a new index logging subsystem
- Add new options --txindex and --addrindex in order to enable the
  optional indexes
  - NOTE: The transaction index will automatically be enabled when the
    address index is enabled because it depends on it
- Add new options --droptxindex and --dropaddrindex to allow the indexes
  to be removed
  - NOTE: The address index will also be removed when the transaction
    index is dropped because it depends on it
- Update getrawtransactions RPC to make use of the transaction index
- Reimplement the searchrawtransaction RPC that makes use of the address
  index
- Update sample-btcd.conf to include sample usage for the new optional
  index flags
2016-04-11 17:16:42 -05:00