LBRYCommunity/hub

Improve address status index performance #120

Merged

jackrobison merged 5 commits from resumable-hash into master

2023-01-07 02:32:46 +01:00

jackrobison commented

2022-12-29 16:06:39 +01:00

(Migrated from github.com)

This adds a resumable sha256 digester, which allows the status hash to be mutated from where it's left off instead of rebuilding it from scratch each time a new element needs to be added.

This adds a resumable sha256 digester, which allows the status hash to be mutated from where it's left off instead of rebuilding it from scratch each time a new element needs to be added.

moodyjon (Migrated from github.com) requested changes

2022-12-29 18:04:46 +01:00

moodyjon (Migrated from github.com) left a comment

Main comment: You should check len(state) >= ctx_size and raise an error if input size is not OK.

Main comment: You should check len(state) >= ctx_size and raise an error if input size is not OK.

moodyjon (Migrated from github.com) commented

2022-12-29 17:40:32 +01:00

You should check len(state) >= ctx_size and raise an error if input size is not OK. Actually handling the exception is probably not necessary, but we should get some predictable failure when size is wrong.

You should check `len(state) >= ctx_size` and raise an error if input size is not OK. Actually handling the exception is probably not necessary, but we should get some predictable failure when size is wrong.

moodyjon (Migrated from github.com) commented

2022-12-29 17:42:41 +01:00

I don't see this being used.

I don't see this being used.

moodyjon (Migrated from github.com) commented

2022-12-29 17:45:50 +01:00

Should be __copy__ if the intention is to hook into the copy() function.

https://docs.python.org/3.9/library/copy.html?highlight=copy

Should be `__copy__` if the intention is to hook into the copy() function. https://docs.python.org/3.9/library/copy.html?highlight=copy

hub/scribe/service.py

moodyjon (Migrated from github.com) commented

2022-12-29 16:49:46 +01:00

Not needed? It's encapsulated in ResumableSH256.

Not needed? It's encapsulated in ResumableSH256.

hub/scribe/service.py

moodyjon (Migrated from github.com) commented

2022-12-29 18:03:11 +01:00

Seems like it might be possible to reduce the number of gets if this is delayed until after:

            if not new_history:
                 continue

If it is uncommon to have empty new_history then never mind.

Seems like it might be possible to reduce the number of gets if this is delayed until after: ``` if not new_history: continue ``` If it is uncommon to have empty `new_history` then never mind.

jackrobison (Migrated from github.com) reviewed

2022-12-30 17:05:56 +01:00

hub/scribe/service.py

jackrobison (Migrated from github.com) commented

2022-12-30 17:05:56 +01:00

I think it's impossible for there to not be a new history, and this is meant to allow compaction independent of advancing a block. It should be ok to delete but I'd need to test this.

I think it's impossible for there to not be a new history, and this is meant to allow compaction independent of advancing a block. It should be ok to delete but I'd need to test this.

Sign in to join this conversation.

No reviewers

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/hub#120

No description provided.