LBRYCommunity/hub

Fork 0

hub changes for stream/repost with extensions #113

Open

moodyjon wants to merge 13 commits from moodyjon/limited_repost into master

moodyjon commented

2022-11-15 22:49:56 +01:00

(Migrated from github.com)

Context: https://github.com/lbryio/lbry-sdk/pull/3706

Not sure that the query for extensions works as well as I would like.

Also I plan to look into some of the options for limiting how many fields, and how deep the indexing of fields within extensions goes:
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-settings-limit.html

Context: https://github.com/lbryio/lbry-sdk/pull/3706 Not sure that the query for extensions works as well as I would like. Also I plan to look into some of the options for limiting how many fields, and how deep the indexing of fields within extensions goes: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-settings-limit.html

jackrobison (Migrated from github.com) reviewed

2022-11-15 22:49:56 +01:00

moodyjon (Migrated from github.com) reviewed

2022-11-15 22:53:06 +01:00

hub/elastic_sync/service.py

moodyjon (Migrated from github.com) commented

2022-11-15 22:53:06 +01:00

This surprised me, but it appears that the old way of doing this did not handle deletions of extensions. It's like it was applying doc as a set of changes to the existing doc.

The new way is intended to completely replace the doc with new contents.

This surprised me, but it appears that the old way of doing this did not handle deletions of extensions. It's like it was applying `doc` as a set of changes to the existing doc. The new way is intended to completely replace the `doc` with new contents.

moodyjon commented

2022-11-17 05:54:10 +01:00

(Migrated from github.com)

The point of commit https://github.com/lbryio/hub/pull/113/commits/5b1d4bff5c7c57b8613a147fc4d2a89bb9e0d4cb

Is to cut off indexing of all the nested things inside extensions. The fields which are “runtime” can be queried, but not as time-efficiently as others. So you can find extension “cad” quickly because they are indexed, but query on fields like “cad.foo” are evaluated more slowly.

The point of commit https://github.com/lbryio/hub/pull/113/commits/5b1d4bff5c7c57b8613a147fc4d2a89bb9e0d4cb Is to cut off indexing of all the nested things inside extensions. The fields which are “runtime” can be queried, but not as time-efficiently as others. So you can find extension “cad” quickly because they are indexed, but query on fields like “cad.foo” are evaluated more slowly.

moodyjon (Migrated from github.com) reviewed

2022-11-29 18:37:39 +01:00

hub/elastic_sync/service.py

moodyjon (Migrated from github.com) commented

2022-11-29 18:37:39 +01:00

It appears this was not necessary. I must have had some other problem involving the ALL_FIELDS or FIELDS definitions or the prepare_claim_metadata_batch() definition. Reverted.

It appears this was not necessary. I must have had some other problem involving the `ALL_FIELDS` or `FIELDS` definitions or the `prepare_claim_metadata_batch() definition`. Reverted.

This pull request has changes conflicting with the target branch.

setup.py

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin moodyjon/limited_repost:moodyjon/limited_repost

git switch moodyjon/limited_repost

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch master

git merge --no-ff moodyjon/limited_repost

git switch moodyjon/limited_repost

git rebase master

git switch master

git merge --ff-only moodyjon/limited_repost

git switch moodyjon/limited_repost

git rebase master

git switch master

git merge --no-ff moodyjon/limited_repost

git switch master

git merge --squash moodyjon/limited_repost

git switch master

git merge --ff-only moodyjon/limited_repost

git switch master

git merge moodyjon/limited_repost

git push origin master

No reviewers

jackrobison

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/hub#113

No description provided.

Rows
Columns