LBRYCommunity/herald.go
1
0
Fork 0
Code Issues 16 Pull requests Projects Releases 14 Packages Wiki Activity

Query caching #15

Merged
jeffreypicard merged 6 commits from query-caching into master 2021-10-05 00:05:44 +02:00
Conversation 1 Commits 6 Files changed 8 +227 -95
jeffreypicard commented 2021-10-02 23:58:25 +02:00 (Migrated from github.com)
Copy link

related to https://github.com/lbryio/lbry-sdk/issues/3393

related to https://github.com/lbryio/lbry-sdk/issues/3393
shyba (Migrated from github.com) reviewed 2021-10-02 23:58:25 +02:00
jeffreypicard commented 2021-10-04 18:50:15 +02:00 (Migrated from github.com)
Copy link

N.B. This works against the leveldb-resolve branch in lbry-sdk https://github.com/lbryio/lbry-sdk/tree/leveldb-resolve so deployment needs to wait until those changes are merged.

N.B. This works against the leveldb-resolve branch in lbry-sdk https://github.com/lbryio/lbry-sdk/tree/leveldb-resolve so deployment needs to wait until those changes are merged.
lyoshenka (Migrated from github.com) approved these changes 2021-10-04 20:59:47 +02:00
server/search.go
lyoshenka (Migrated from github.com) commented 2021-10-04 20:53:46 +02:00
Copy link

does this need fixing?

does this need fixing?
lyoshenka (Migrated from github.com) commented 2021-10-04 20:55:16 +02:00
Copy link

i generally don't like swallowing errors. at the very least you should log it

i generally don't like swallowing errors. at the very least you should log it
server/search.go
@ -203,1 +208,4 @@
_ = s.QueryCache.Purge()
s.NumESRefreshes = numRefreshes
}
}
lyoshenka (Migrated from github.com) commented 2021-10-04 20:57:26 +02:00
Copy link

ES data should only change if a new block comes in. Maybe in the future we can use that info to purge the cache so we avoid checking the index every 2 seconds

ES data should only change if a new block comes in. Maybe in the future we can use that info to purge the cache so we avoid checking the index every 2 seconds
jeffreypicard (Migrated from github.com) reviewed 2021-10-04 21:06:07 +02:00
server/search.go
@ -203,1 +208,4 @@
_ = s.QueryCache.Purge()
s.NumESRefreshes = numRefreshes
}
}
jeffreypicard (Migrated from github.com) commented 2021-10-04 21:06:06 +02:00
Copy link

This check shouldn't cost much, it's essentially just pinging the ES server, but we could definitely set the refresh delta higher to like 5 or maybe 10 seconds. @shyba we were talking about this over the weekend, any thoughts?

This check shouldn't cost much, it's essentially just pinging the ES server, but we could definitely set the refresh delta higher to like 5 or maybe 10 seconds. @shyba we were talking about this over the weekend, any thoughts?
lyoshenka (Migrated from github.com) reviewed 2021-10-04 21:13:45 +02:00
server/search.go
@ -203,1 +208,4 @@
_ = s.QueryCache.Purge()
s.NumESRefreshes = numRefreshes
}
}
lyoshenka (Migrated from github.com) commented 2021-10-04 21:13:45 +02:00
Copy link

this is fine. we can always bump it up later if its an issue. no need to worry about i tmuch now

this is fine. we can always bump it up later if its an issue. no need to worry about i tmuch now
jeffreypicard (Migrated from github.com) reviewed 2021-10-04 21:59:06 +02:00
server/search.go
jeffreypicard (Migrated from github.com) commented 2021-10-04 21:59:06 +02:00
Copy link

No, on seconds though, I don't think so. We accept a single ES index as an argument, and in all our use cases I believe we only ever have to deal with one.

No, on seconds though, I don't think so. We accept a single ES index as an argument, and in all our use cases I believe we only ever have to deal with one.
jeffreypicard (Migrated from github.com) reviewed 2021-10-05 00:02:26 +02:00
server/search.go
jeffreypicard (Migrated from github.com) commented 2021-10-05 00:02:25 +02:00
Copy link

Done.

Done.
jeffreypicard (Migrated from github.com) reviewed 2021-10-05 00:03:26 +02:00
server/search.go
@ -203,1 +208,4 @@
_ = s.QueryCache.Purge()
s.NumESRefreshes = numRefreshes
}
}
jeffreypicard (Migrated from github.com) commented 2021-10-05 00:03:25 +02:00
Copy link

Cool, I parameterized it at least along with the cache ttl.

Cool, I parameterized it at least along with the cache ttl.
Sign in to join this conversation.
Reviewers
No reviewers
shyba
lyoshenka
Labels
Clear labels   
consider soon

Discuss this issue at the next planning meeting, then remove this label

  
documentation

Improvements or additions to documentation

  
good first issue

Good for newcomers

  
hacktoberfest

Welcome to Hacktoberfest

  
help wanted

Extra attention is needed

  
priority: blocker

Issue is blocking release, do ASAP

  
priority: high

Work needs to be moved into sprint ASAP

  
priority: low

Work should be done but can stay in the backlog for now

  
priority: medium

Work needs to be done within 2-3 sprints

  
type: bug

Existing functionality is wrong or broken

  
type: discussion

A conversation. No specific changes requested

  
type: improvement

Existing (or partially existing) functionality needs to be changed

  
type: new feature

New functionality that does not exist yet

  
type: refactor

Minimal user-visible changes, but significant internal work

  
type: task

Either work that's not related to the code, or a small chore that does not fit into other categories

No labels
consider soon
documentation
good first issue
hacktoberfest
help wanted
priority: blocker
priority: high
priority: low
priority: medium
type: bug
type: discussion
type: improvement
type: new feature
type: refactor
type: task
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/herald.go#15
No description provided.
Delete branch "query-caching"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?