Expose `--index_address_statuses` setting and improve first sync performance #35

Merged

jackrobison merged 15 commits from optional-address-history-status into master

2022-05-18 02:14:59 +02:00

jackrobison commented

2022-05-11 19:01:48 +02:00

(Migrated from github.com)

adds --index_address_statuses flag, off by default. This is used to turn on/off the index of status hashes for addresses added in https://github.com/lbryio/scribe/pull/17 to allow scribe to run with fewer resources necessary. The index adds ~10gb to the database, it makes processing new blocks slower but makes handling the notifications for the addresses touched by transactions in the block much faster.
add internal caches to handle updating address statuses and reading address histories faster without needing --cache_all_tx_hashes to be on
sync the address status index ( --index_address_statuses ) when caught up after startup, including after first initial-sync.
refactors and combines updating and compacting the address history to be more performant, running in one pass, and use the rocksdb multi_get api

- adds `--index_address_statuses` flag, off by default. This is used to turn on/off the index of status hashes for addresses added in https://github.com/lbryio/scribe/pull/17 to allow scribe to run with fewer resources necessary. The index adds ~10gb to the database, it makes processing new blocks slower but makes handling the notifications for the addresses touched by transactions in the block much faster. - add internal caches to handle updating address statuses and reading address histories faster without needing `--cache_all_tx_hashes` to be on - sync the address status index ( `--index_address_statuses` ) when caught up after startup, including after first initial-sync. - refactors and combines updating and compacting the address history to be more performant, running in one pass, and use the rocksdb multi_get api

shyba (Migrated from github.com) reviewed 2022-05-11 19:01:48 +02:00

No reviewers

shyba

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/hub#35

No description provided.

Expose --index_address_statuses setting and improve first sync performance #35

Expose `--index_address_statuses` setting and improve first sync performance #35