LBRYCommunity/hub
1
0
Fork 0
Code Issues 35 Pull requests 6 Projects Releases Packages Wiki Activity

Improve blockchain.address.get_history performance #40

Merged
jackrobison merged 6 commits from improve-history-cache into master 2022-05-22 06:00:39 +02:00
Conversation 0 Commits 6 Files changed 5 +163 -27
jackrobison commented 2022-05-19 22:55:55 +02:00 (Migrated from github.com)
Copy link

This updates the caching of address histories to match the pattern used in scribe, which is much more performant. In addition to caching the final address histories, the tx_hash and heights are also cached in a smaller (but more items) cache that allows for a high hit rate on blocks containing transactions that touch many addresses (where each touched address has a fairly large history).

Additionally, the history cache is now updated in place using a minimal set of batched lookups upon a new block instead of being cleared.

This updates the caching of address histories to match the pattern used in `scribe`, which is much more performant. In addition to caching the final address histories, the tx_hash and heights are also cached in a smaller (but more items) cache that allows for a high hit rate on blocks containing transactions that touch many addresses (where each touched address has a fairly large history). Additionally, the history cache is now updated in place using a minimal set of batched lookups upon a new block instead of being cleared.
shyba (Migrated from github.com) reviewed 2022-05-19 22:55:55 +02:00
Sign in to join this conversation.
Reviewers
No reviewers
shyba
Labels
Clear labels   
area: database

   
area: documentation

Improvements or additions to documentation

  
area: elasticsearch

   
area: herald

   
area: packaging

   
area: scribe

   
consider soon

Discuss this issue at the next planning meeting, then remove this label

  
critical

   
dependencies

Pull requests that update a dependency file

  
good first issue

Good for newcomers

  
hacktoberfest

Welcome to Hacktoberfest

  
help wanted

Extra attention is needed

  
improvement

   
needs: repro

Needs a way to reproduce

  
new feature

   
priority: blocker

Issue is blocking release, do ASAP

  
priority: high

Work needs to be moved into sprint ASAP

  
priority: low

Work should be done but can stay in the backlog for now

  
priority: medium

Work needs to be done within 2-3 sprints

  
type: bug

Existing functionality is wrong or broken

  
type: bug-fix

   
type: discussion

A conversation. No specific changes requested

  
type: feature request

   
type: improvement

Existing (or partially existing) functionality needs to be changed

  
type: new feature

New functionality that does not exist yet

  
type: refactor

Minimal user-visible changes, but significant internal work

  
type: task

Either work that's not related to the code, or a small chore that does not fit into other categories

No labels
area: database
area: documentation
area: elasticsearch
area: herald
area: packaging
area: scribe
consider soon
critical
dependencies
good first issue
hacktoberfest
help wanted
improvement
needs: repro
new feature
priority: blocker
priority: high
priority: low
priority: medium
type: bug
type: bug-fix
type: discussion
type: feature request
type: improvement
type: new feature
type: refactor
type: task
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/hub#40
No description provided.
Delete branch "improve-history-cache"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?