LBRYCommunity/lbrycrd
1
0
Fork 0
Code Issues 29 Pull requests 1 Projects Releases 32 Packages Wiki Activity

Add progress logging to claimtrie loading and validation. #408

Open
pseudoscalar wants to merge 1 commit from pseudoscalar/issue/363 into master
pull from: pseudoscalar/issue/363
merge into: LBRYCommunity:master
Conversation 2 Commits 1 Files changed 2 +51 -4
pseudoscalar commented 2021-09-15 07:51:32 +02:00 (Migrated from github.com)
Copy link

To resolve 363

To resolve [363](https://github.com/lbryio/lbrycrd/issues/363)
BrannonKing (Migrated from github.com) reviewed 2021-09-15 15:34:08 +02:00
src/claimtrie.cpp
BrannonKing (Migrated from github.com) commented 2021-09-15 15:33:19 +02:00
Copy link

This pre-count loop could be very expensive, as the disk cache won't be warm. How long is this loop taking compared to the lower loop?

This pre-count loop could be very expensive, as the disk cache won't be warm. How long is this loop taking compared to the lower loop?
pseudoscalar (Migrated from github.com) reviewed 2021-09-15 16:10:47 +02:00
src/claimtrie.cpp
pseudoscalar (Migrated from github.com) commented 2021-09-15 16:10:47 +02:00
Copy link

I was seeing 1-3 seconds in the precount loop vs ~2 minutes for the lower loop.

I was seeing 1-3 seconds in the precount loop vs ~2 minutes for the lower loop.
pseudoscalar (Migrated from github.com) reviewed 2021-09-15 16:22:25 +02:00
src/claimtrie.cpp
pseudoscalar (Migrated from github.com) commented 2021-09-15 16:22:25 +02:00
Copy link

Sorry, I should clarify. ~2 minutes for the whole load from disk. It was split pretty evenly between the 2 lower loops (so ~1 minute each). That's why I decided to do a precount loop: counting the total in the first lower loop would leave the operation already half done (time-wise). I couldn't see another way to estimate progress, and the added time seemed negligible next to the main operation.

Sorry, I should clarify. ~2 minutes for the whole load from disk. It was split pretty evenly between the 2 lower loops (so ~1 minute each). That's why I decided to do a precount loop: counting the total in the first lower loop would leave the operation already half done (time-wise). I couldn't see another way to estimate progress, and the added time seemed negligible next to the main operation.
pseudoscalar (Migrated from github.com) reviewed 2021-09-17 16:03:32 +02:00
src/claimtrie.cpp
pseudoscalar (Migrated from github.com) commented 2021-09-17 16:03:32 +02:00
Copy link

Did you want me to find a way to eliminate the precount, or is this good?

Did you want me to find a way to eliminate the precount, or is this good?
BrannonKing commented 2021-09-17 16:13:47 +02:00 (Migrated from github.com)
Copy link

Have you looked at adding the progress notifiers to the v17_master and v19_master branches?

Have you looked at adding the progress notifiers to the v17_master and v19_master branches?
pseudoscalar commented 2021-09-17 16:48:28 +02:00 (Migrated from github.com)
Copy link

I haven't yet. I start putting together some PRs for those.

I haven't yet. I start putting together some PRs for those.
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin pseudoscalar/issue/363:pseudoscalar/issue/363
git checkout pseudoscalar/issue/363

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git checkout master
git merge --no-ff pseudoscalar/issue/363
git checkout pseudoscalar/issue/363
git rebase master
git checkout master
git merge --ff-only pseudoscalar/issue/363
git checkout pseudoscalar/issue/363
git rebase master
git checkout master
git merge --no-ff pseudoscalar/issue/363
git checkout master
git merge --squash pseudoscalar/issue/363
git checkout master
git merge --ff-only pseudoscalar/issue/363
git checkout master
git merge pseudoscalar/issue/363
git push origin master
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
area: devops

   
area: discovery

   
area: docs

   
area: livestream

   
area: proposal

Work is part of a proposal

  
consider soon

Discuss this issue at the next planning meeting, then remove this label

  
Epic

   
good first issue

   
hacktoberfest

Welcome to Hacktoberfest

  
hard fork

   
help wanted

   
icebox

Long-term storage

  
Invalid

   
level: 0

   
level: 1

No knowledge of the existing code required

  
level: 2

Some knowledge of the existing code is recommended

  
level: 3

Significant knowledge of the existing code is recommended

  
level: 4

Intimate knowledge of the existing code is recommended

  
needs: exploration

Solution unclear, needs research

  
needs: grooming

Issue needs to be groomed before work can start

  
needs: priority

Priority level needs to be defined

  
needs: repro

Issue needs step-by-step instructions on how to reproduce in latest code

  
needs: tech design

Needs technical design signoff before implementation can begin

  
on hold

Temporarily paused

  
priority: blocker

Issue is blocking release, do ASAP

  
priority: high

Work needs to be moved into sprint ASAP

  
priority: low

Work should be done but can stay in the backlog for now

  
priority: medium

Work needs to be done within 2-3 sprints

  
resilience

   
soft fork

   
Tom's Wishlist

   
type: bug

Existing functionality is wrong or broken

  
type: discussion

A conversation. No specific changes requested

  
type: improvement

Existing (or partially existing) functionality needs to be changed

  
type: new feature

New functionality that does not exist yet

  
type: refactor

Minimal user-visible changes, but significant internal work

  
type: task

Either work that's not related to the code, or a small chore that does not fit into other categories

  
type: testing

Solution needs additional user testing

  
unplanned

Work that was not planned into the spirnt, took priority over planned work

  
work in progress

The proposed solution is still incomplete.

No labels
area: devops
area: discovery
area: docs
area: livestream
area: proposal
consider soon
Epic
good first issue
hacktoberfest
hard fork
help wanted
icebox
Invalid
level: 0
level: 1
level: 2
level: 3
level: 4
needs: exploration
needs: grooming
needs: priority
needs: repro
needs: tech design
on hold
priority: blocker
priority: high
priority: low
priority: medium
resilience
soft fork
Tom's Wishlist
type: bug
type: discussion
type: improvement
type: new feature
type: refactor
type: task
type: testing
unplanned
work in progress
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/lbrycrd#408
No description provided.
Delete branch "pseudoscalar/issue/363"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?