LBRYCommunity/lbrycrd
1
0
Fork 0
Code Issues 29 Pull requests 1 Projects Releases 32 Packages Wiki Activity

Increase default dbcache size #238

Merged
bvbfan merged 1 commit from increase_dbcache_size into master 2018-11-29 10:27:33 +01:00
Conversation 14 Commits 1 Files changed 4 +48 -62
bvbfan commented 2018-11-23 13:42:33 +01:00 (Migrated from github.com)
Copy link

Signed-off-by: Anthony Fieroni bvbfan@abv.bg

Signed-off-by: Anthony Fieroni <bvbfan@abv.bg>
kaykurokawa (Migrated from github.com) reviewed 2018-11-23 13:42:33 +01:00
BrannonKing (Migrated from github.com) reviewed 2018-11-23 14:11:46 +01:00
src/rpc/claimtrie.cpp
BrannonKing (Migrated from github.com) commented 2018-11-23 14:06:21 +01:00
Copy link

Why not throw here as well? and make the method void return?

Why not throw here as well? and make the method void return?
BrannonKing (Migrated from github.com) commented 2018-11-23 14:11:33 +01:00
Copy link

I like the idea of limiting decrements on RAM. However, I don't think this line accomplishes that. You can't make an assumption that each block uses the same amount of coin cache. I think this check of MAX decrements could probably go away. We just use available memory then give up.

I like the idea of limiting decrements on RAM. However, I don't think this line accomplishes that. You can't make an assumption that each block uses the same amount of coin cache. I think this check of MAX decrements could probably go away. We just use available memory then give up.
src/txdb.h
BrannonKing (Migrated from github.com) commented 2018-11-23 14:07:33 +01:00
Copy link

I'm worried about this default on Windows. It's currently running in 32bit and using 1.2GB of RAM. This will bump it up against the 2GB barrier on Windows.

I'm worried about this default on Windows. It's currently running in 32bit and using 1.2GB of RAM. This will bump it up against the 2GB barrier on Windows.
bvbfan (Migrated from github.com) reviewed 2018-11-23 14:21:29 +01:00
src/rpc/claimtrie.cpp
bvbfan (Migrated from github.com) commented 2018-11-23 14:21:29 +01:00
Copy link

It's relaxing check that deeper block can fail on rollback.
E.g. MAX_RPC_BLOCK_DECREMENTS is 50 but request wants 100 so does not need to call rollback - we know that it'll fail.

It's *relaxing* check that deeper block can fail on rollback. E.g. MAX_RPC_BLOCK_DECREMENTS is 50 but request wants 100 so does not need to call rollback - we know that it'll fail.
bvbfan (Migrated from github.com) reviewed 2018-11-23 14:21:58 +01:00
src/txdb.h
bvbfan (Migrated from github.com) commented 2018-11-23 14:21:58 +01:00
Copy link

So 450 or 500 as Kay suggest?

So 450 or 500 as Kay suggest?
bvbfan (Migrated from github.com) reviewed 2018-11-23 14:23:38 +01:00
src/rpc/claimtrie.cpp
bvbfan (Migrated from github.com) commented 2018-11-23 14:23:38 +01:00
Copy link

Ok.

Ok.
BrannonKing (Migrated from github.com) reviewed 2018-11-23 14:58:26 +01:00
src/txdb.h
BrannonKing (Migrated from github.com) commented 2018-11-23 14:58:26 +01:00
Copy link

What about sizeof(void*) >= 8 ? 700 : 400.

What about `sizeof(void*) >= 8 ? 700 : 400`.
BrannonKing (Migrated from github.com) reviewed 2018-11-23 14:59:22 +01:00
src/rpc/claimtrie.cpp
BrannonKing (Migrated from github.com) commented 2018-11-23 14:59:22 +01:00
Copy link

We don't know that 100 blocks back will fail. It's dependent upon the number of TXOs in each block.

We don't know that 100 blocks back will fail. It's dependent upon the number of TXOs in each block.
bvbfan (Migrated from github.com) reviewed 2018-11-23 15:00:12 +01:00
src/txdb.h
bvbfan (Migrated from github.com) commented 2018-11-23 15:00:12 +01:00
Copy link

That's good.

That's good.
bvbfan (Migrated from github.com) reviewed 2018-11-23 15:18:57 +01:00
src/rpc/claimtrie.cpp
bvbfan (Migrated from github.com) commented 2018-11-23 15:18:57 +01:00
Copy link

Yep but most likely will fail, i think we need some value that will prevent rollback to take time for nothing. nCoinCacheUsage / currentMemoryUsage * 1.5 or something ?

Yep but most likely will fail, i think we need some value that will prevent rollback to take time for nothing. nCoinCacheUsage / currentMemoryUsage * 1.5 or something ?
bvbfan (Migrated from github.com) reviewed 2018-11-23 15:42:15 +01:00
src/rpc/claimtrie.cpp
bvbfan (Migrated from github.com) commented 2018-11-23 15:42:15 +01:00
Copy link

Looks like state is totally unused in DisconnectBlock i'll remove that check entirely.

Looks like state is totally unused in DisconnectBlock i'll remove that check entirely.
kaykurokawa (Migrated from github.com) reviewed 2018-11-25 22:57:38 +01:00
src/rpc/claimtrie.cpp
kaykurokawa (Migrated from github.com) commented 2018-11-25 22:57:38 +01:00
Copy link

I agree with Brannon that this approach

'MAX_RPC_BLOCK_DECREMENTS = nCoinCacheUsage / currentMemoryUsage' is not correct.

because current memory usage is not predictive of what memory usage a roll back will cost.

I'm in favor of keeping MAX_RPC_BLOCK_DECREMENTS to a high constant. I think the previous max of 50 is fine, or we could divide nCoinCacheUsage by 20 mb (in my tests I saw that a 1mb block used about 10 mb of cache, and our max block size is 2mb). This is in case someone accidentally sets a very high roll back, we don't have to waste time or resource trying to roll back, we will know immediately that the roll back is too high.

I agree with Brannon that this approach 'MAX_RPC_BLOCK_DECREMENTS = nCoinCacheUsage / currentMemoryUsage' is not correct. because current memory usage is not predictive of what memory usage a roll back will cost. I'm in favor of keeping MAX_RPC_BLOCK_DECREMENTS to a high constant. I think the previous max of 50 is fine, or we could divide nCoinCacheUsage by 20 mb (in my tests I saw that a 1mb block used about 10 mb of cache, and our max block size is 2mb). This is in case someone accidentally sets a very high roll back, we don't have to waste time or resource trying to roll back, we will know immediately that the roll back is too high.
bvbfan commented 2018-11-26 08:53:06 +01:00 (Migrated from github.com)
Copy link

Give it try to another approach.

Give it try to another approach.
BrannonKing (Migrated from github.com) reviewed 2018-11-28 09:26:46 +01:00
src/rpc/claimtrie.cpp
BrannonKing (Migrated from github.com) commented 2018-11-28 09:26:46 +01:00
Copy link

I still don't think this approach will work. Too often we will have empty blocks. If you use one of those as a template your numbers here will be meaningless.

I still don't think this approach will work. Too often we will have empty blocks. If you use one of those as a template your numbers here will be meaningless.
bvbfan (Migrated from github.com) reviewed 2018-11-28 09:32:39 +01:00
src/rpc/claimtrie.cpp
bvbfan (Migrated from github.com) commented 2018-11-28 09:32:39 +01:00
Copy link

On empty block we can fallback to @kaykurokawa suggest

think the previous max of 50 is fine, or we could divide nCoinCacheUsage by 20 mb

What you think?

On empty block we can fallback to @kaykurokawa suggest > think the previous max of 50 is fine, or we could divide nCoinCacheUsage by 20 mb What you think?
Sign in to join this conversation.
Reviewers
No reviewers
kaykurokawa
Labels
Clear labels   
area: devops

   
area: discovery

   
area: docs

   
area: livestream

   
area: proposal

Work is part of a proposal

  
consider soon

Discuss this issue at the next planning meeting, then remove this label

  
Epic

   
good first issue

   
hacktoberfest

Welcome to Hacktoberfest

  
hard fork

   
help wanted

   
icebox

Long-term storage

  
Invalid

   
level: 0

   
level: 1

No knowledge of the existing code required

  
level: 2

Some knowledge of the existing code is recommended

  
level: 3

Significant knowledge of the existing code is recommended

  
level: 4

Intimate knowledge of the existing code is recommended

  
needs: exploration

Solution unclear, needs research

  
needs: grooming

Issue needs to be groomed before work can start

  
needs: priority

Priority level needs to be defined

  
needs: repro

Issue needs step-by-step instructions on how to reproduce in latest code

  
needs: tech design

Needs technical design signoff before implementation can begin

  
on hold

Temporarily paused

  
priority: blocker

Issue is blocking release, do ASAP

  
priority: high

Work needs to be moved into sprint ASAP

  
priority: low

Work should be done but can stay in the backlog for now

  
priority: medium

Work needs to be done within 2-3 sprints

  
resilience

   
soft fork

   
Tom's Wishlist

   
type: bug

Existing functionality is wrong or broken

  
type: discussion

A conversation. No specific changes requested

  
type: improvement

Existing (or partially existing) functionality needs to be changed

  
type: new feature

New functionality that does not exist yet

  
type: refactor

Minimal user-visible changes, but significant internal work

  
type: task

Either work that's not related to the code, or a small chore that does not fit into other categories

  
type: testing

Solution needs additional user testing

  
unplanned

Work that was not planned into the spirnt, took priority over planned work

  
work in progress

The proposed solution is still incomplete.

No labels
area: devops
area: discovery
area: docs
area: livestream
area: proposal
consider soon
Epic
good first issue
hacktoberfest
hard fork
help wanted
icebox
Invalid
level: 0
level: 1
level: 2
level: 3
level: 4
needs: exploration
needs: grooming
needs: priority
needs: repro
needs: tech design
on hold
priority: blocker
priority: high
priority: low
priority: medium
resilience
soft fork
Tom's Wishlist
type: bug
type: discussion
type: improvement
type: new feature
type: refactor
type: task
type: testing
unplanned
work in progress
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/lbrycrd#238
No description provided.
Delete branch "increase_dbcache_size"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?