## Issue
Apparently, a user is experiencing 423 locked errors from the server, which should not happen given the locking mechanism plus the user wasn't trying to do concurrent uploads.
## Fix
Anyway, fix the Resume button so that at least they can try to resume.
* Add ability to have claim searches auto-fetch up to 3 pages.
* make total_items and total_pages optional
* use auto pagination strategy when determining live claim
* Bump page size back to 50
* Upload: fix redux key clash
## Issue
`params` is the "final" value that will be passed to the SDK and `channel` is not a valid argument (it should be `channel_name`). Also, it seems like we only pass the channel ID now and skip the channel name entirely.
For the anonymous case, a clash will still happen when since the channel part is hardcoded to `anonymous`.
## Approach
Generate a guid in `params` and use that as the key to handle all the cases above. We couldn't use the `uploadUrl` because v1 doesn't have it.
The old formula is retained to allow users to retry or cancel their existing uploads one last time (otherwise it will persist forever). The next upload will be using the new key.
* Upload: add tab-locking
## Issue
- The previous code does detect uploads from multiple tabs, but it was done by handling the CONFLICT error message from the backend. At certain corner-cases, this does not work well. A better way is to not allow resumption while the same file is being uploading from another tab.
- When an upload from 1 tab finishes, the GUI on the other tab does not remove the completed item. User either have to refresh or click Cancel. Clicking Cancel results in the 404 backend error. This should be avoided.
## Approach
- Added tab synchronization and locking by passing the "locked" and "removed" information through `localStorage`.
## Other considered approaches
- Wallet sync -- but decided not to pollute the wallet.
- 3rd-party redux tab syncing -- but decided it's not worth adding another module for 1 usage.
* Upload: check if locked before confirming delete
## Reproduce
Have 2 tabs + paused upload
Open "cancel" dialog in one of the tabs.
Continue upload in other tab
Confirm cancellation in first tab
Upload disappears from both tabs, but based on network traffic the upload keeps happening.
(If upload finishes the claim seems to get created)
I yanked out the parseURI part in a prior commit ... the comment was misleading me to think it was redundant. But it had another hidden function, which is to handle abandoned claims which `claim` will be `null`.
* Route recommendation search to recsys 5% of the time + add `user_id`
## Ticket
334 send some recommended requests to recsys
## Approach
`doSearch`:
- If the search options include `related_to`, route that to the new `searchRecommendations` which performs the 5% check + appends `user_id` at the end. This way, we don't need to alter the function signature of `doSearch`.
- Else, run proceed as normal.
* Always go to alt provider
f
Co-authored-by: Thomas Zarebczan <thomas.zarebczan@gmail.com>
* Add remove_duplicates to tile/list claim_search except for Channel Page
This removes the any duplicates from reposts.
* Re-activate the "Hide reposts" setting
* Category Rows: default to ['stream', 'repost'] unless specified otherwise.
* apiCall: add option to not send the auth header
## Why
Want an option to make un-authenticated `resolve` calls where appropriate, to improve caching.
## How
All `apiCall`s are authenticated by default, but when clients add NO_AUTH to the params, `apiCall` will exclude the X_LBRY_AUTH_TOKEN. It will also strip NO_AUTH from the param object before sending it out.
* Add hook for 'resolve' and 'claim_search' to check and skip auth...
... if the params does not contain anything that requires the wallet.
* doResolveUri, doClaimSearch: let clients decide when to include_my_output
- No more hardcoding 'include_purchase_receipt' and 'include_is_my_output'
- doResolveUri: include these params when opening a file page. This was the only place that was doing that prior to this PR.
* is_my_output: use the signing_channel as alternative
## Notes
`is_my_output` is more expensive to resolve, so it is not being requested all the time.
## Change
Looking at the signing channel as the additional fallback, on top of `myClaimIds`.
## Aside
I think using `myClaimIds` here is redundant, as it is usually populated from `is_my_ouput`. But leaving as is for now...
## Why
- No memo required (no transformation).
- `makeSelect*` is an incorrect pattern.
## Changes
- Replaced makeSelectClientSetting with selectClientSetting.
- Remove unused selectShowRepostedContent.
## Issue
- Large resolve count (albeit batched) on bootup.
## Changes
- Skip the call on bootup. The same call will happen when you click the notification bell, so it's not too late to resolve at that time.
- Added `true` to `doResolveUris` to return cached results, otherwise it will keep resolving the same channels every time we enter Notifications Page.
## Issue
One of the bottlenecks of livestream page.
The component probably needs a re-design:
- Don't perpetually mount -- only mount when activated by the user through "@". This would avoid the heavy processing entirely.
- Better way of resolving uris (too many arrays, too many loops).
- Tom also mentioned that we should not be resolving every commenter as we see encounter them in a livestream. This is currently the case because the component is always mounted.
## Changes
Until the re-design occurs, attempt to cache the heavy processing. Also, trimmed down the amount of loops.
For the case of livestreams, the comments are added incrementally via websocket. The selector returns everything, which grows as a user watches the livestream.
We could even make it a bit more efficient by passing in `maxCount` to `filterComments`, and do a `for` loop there, but decided to keep things readable by not changing the `filter` usage.
- Memo not required. `resolvingUris` is very dynamic and is a short array anyways.
- Changeg from using `indexOf` to `includes`, which is more concise.
## Changes
- doHandleSyncComplete: only call doGetAndPopulatePreferences when there is new data.
- But for that to work, we'll need to populate preferences at least once. We'll do that in doSignIn.
- We can also remove the "sync/prefs ready" mechanism that was mainly meant for Desktop.
Then came another problem: while trying to spark changes between 2 tabs, `sync/get` was saying "no change" despite the local and server hash being different. I think it is because the both `sync_hash + sync/get` combo is operating on server data, so the hash is the same. I'm guessing this is why we ended up just running doGetAndPopulatePreferences every time before PR, since this flag wasn't correct in this scenario.
- Updated `data.changed` to consider both API results and comparison with local hash.
## Issue
When the muted list was being cleared from another app, the web version ended up restoring the previous muted list.
## Change
- As long as `blocked` is defined, return that since an empty array is a valid result.
- If undefined, something went wrong when calling the reducer, so retain the muted list. I believe this was the original intention of that line.
* Exclude default homepage data at compile time
The youtuber IDs alone is pretty huge, and is unused in the `CUSTOM_HOMEPAGE=true` configuration.
* Remove Desktop items and other cleanup
- Moved constants out of the component.
- Remove SIMPLE_SITE check.
- Remove Desktop-only items
* Sidebar: limit subscription and tag section
## Issue
Too slow for huge lists
## Change
Limit to 10 initially, and load everything on "Show more"
* Fix makeSelectThumbnailForUri
- Fix memo
- Expose function to extract directly from claim if client already have it.
## Issues with `makeSelectIsSubscribed`
- It will not return true if the uri provided is canonical, because the compared subscription uri is in permanent form. This was causing certain elements like the Heart to not appear in claim tiles.
- It is super slow for large subscriptions not just because of the array size + being a hot selector, but also because it is looking up the claim twice (not memo'd) and also calling `parseURI` to determine if it's a channel, which is unnecessary if you already have the claim.
## Changes
- Optimize the selector to only look up the claim once, and make operations using already-obtained info.
## Issue
If you make 2 claims from the same source file, the second upload thinks it's trying to resume from the first one. They should be unique uploads.
## Approach
Stash the upload url for comparison when looking up existing uploads to resume.
Stash that in `params` to minimize code changes. We'll just need to ensure it is cleared before we generate the SDK payload.
## Why
Frequently used; top in perf profile
## Changes
Most of the time, you already have the claim object in the current context. `selectClaimIsMineForUri` will retrieve the claim again, which is wasteful, even if it is memoized (looking up the cache still takes time).
Break apart the logic and added the alternative `selectClaimIsMine` for faster lookup.
* Publish button: use spinner instead of "Publishing..."
Looks better, plus the preview could take a while sometimes.
* Refactor `doPublish`. No functional change
This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data.
* Add doPublishResume
* Support resume-able upload via tus
## Issue
38 Handle resumable file upload
## Notes
Since we can't serialize a File object, we'll need to the user to re-select the file to resume.
* Exclude "modified date" for Firefox/Android
## Issue
It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload.
## Change
Exclude modification date for now. Let's assume a smart user.
* Move 'currentUploads' to 'publish' reducer
`publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app.
We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me).
This change is mostly moving items between files, with the exception of
1. An additional REHYDRATE in the publish reducer to clean up the tusUploader.
2. Not clearing `currentUploads` in CLEAR_PUBLISH.
* Restore v1 code for livestream replay, etc.
v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
- memo not required.
- start to not use the confusing and wrongly-named 'selectCommentsByUri' (per comment from Sean); use the existing 'selectClaimIdForUri' instead. This works because currently we do fetch any comments without first visiting a claim/uri, so we'll always have fetched the required claim, and can be queried in 'selectClaimIdForUri'.
It's technically incorrect and was causing the GUI to not update sometimes because the reference did not change, despite the array contents did. The GUI just happens to update most of the time due to other state changes.
## Issue
`normalizeUri` | `parseURI` is expensive and has been causing sluggish operations when called repeatedly or within a loop.
## Change
Since I'm not confident enough to remove the call entirely from makeSelectClaimIsMine (although I've yet to find a scenario that the uri is not already normalized), we'll try caching the calls instead.
## Results
- in a simple test of toggling between 2 category pages, we saved 20ms from `parseURI` calls alone.
- in a test of opening all categories one time, the memory usage remained similar. This makes sense since we removed a `makeSelect*` (which creates a selector for each call + not memoizing), and replaced that with a cached selector that's actually memoizing.
## Issue
- selectMyActiveClaims memo problem -- being recalculated on every click -- high workload for wallet with large uploads.
- Mistake in handling abandoning IDs (it was trying to extract keys from an array)
## Changes
- selectAbandoningIds: never use `state` as an input selector. Breaks memo.
- Don't use selectMyClaimsRaw and then reduce it back to IDs. Use selectMyClaimIdsRaw instead.
- selectAbandoningIds is already in array form, so don't run Object.keys.
- Fix abandoningById never clearing when succeeded.
* Simplify dispatch map
Since none of dispatches are doing any custom transformation, just use a direct map. The number of arguments for the comment function are getting crazy.
* Block: pass comment ID for deletion when being blocked.
* Don't update 'pendingById' if no changes.
'pendingById' isn't frequently updated, but using it as a proof-of-concept to fix how reducers should be written to avoid unnecessary updates.
ImmutableJS apparently does all of this for us, but there are cons to using it as well, so using own wrappers for now.
* Don't update 'byId' if no changes + add 'selectClaimWithId'
## Ticket
116 Claim store optimization ideas (reducing unnecessary renders)
## Changes
- Ignore things like `confirmations` so that already-fetched claims aren't invalidated and causes re-rendering. The `stringify` might look expensive, but the amount of avoided re-renders outweighs it. There might be faster ways to compare, though.
- With `byId[claimId]` references more stable now, memoized selectors can now use 'selectClaimWithId' to pick a specific claim to depend on, instead of 'byId' which changes on every update.
* Fix memo: selectMyChannelClaims, selectActiveChannelClaim
## Issue
These should never recalculate after `channel_list` has been fetched, but they do because of poor selector dependency.
## Change
With the `byId` changes from the previous commit, we are now able to memoize these selectors correctly.
## Ticket
223 Add ability for delegated moderators to delete comments
## Changes
- Refactored doCommentAbandon's signature so we don't end up converting between "uri" and "Claim" several times and needing to lookup redux when the client can already provide us the exact values that we need.
- Pass the new moderator fields to the API.
- Remove the need to call 'makeSelectChannelPermUrlForClaimUri' since it's a simple field query when we already have the claim.
- selectMyChannelClaims depends on `byId`, which currently is always invalidated per update, so it is not memoized.
- Most of the use-cases just needs the ID or the length of the array anyways, so avoid generating a Claim array (in selectMyChannelClaims) unnecessarily -- the client need to reduce it back down to IDs again :/
- The simpler boolean also removes the need to memoize the selector, which saves a bit of memory.
`selectMyActiveClaims` includes `byId`, which gets invalidated on each resolve. Having this as an input selector breaks memoization.
For the case of comment-filtering, we don't really care about pending or abandoned own claims (I think), so just grab the raw IDs.
* Prevent multiple parseURI calls
## Ticket
129
## Issue
Code was shortened to use `isURIValid` during the consolidation. `isURIValid` calls `normalizeURI`, which calls another `parseURI`.
`parseURI` is pretty expensive.
## Approach
- Add optional parameter to `isURIValid` to skip the normalization.
- Set those that were converted during the consolidation to skip the normalization. Also covered a few other instances where it is obvious to me that normalization is not required.
- For the rest, I can't tell for sure if it's safe to remove the normalization, so the default `normalize=true` will leave things as is.
The whole `parseURI` probably needs a refactoring, or a few lighter version for specific needs.
* Simplify isURIEqual
## Issue
`parseURI` is too expensive to be used in a loop, plus `normalizeURI` itself is calling `parseURI`.
## Approach
Not sure if it covers all cases, but just try convert colons to hashes before comparing.
* Refactor filePrice
* Refactor Wallet Tip Components
* Add backend sticker support for comments
* Add stickers
* Refactor commentCreate
* Add Sticker Selector and sticker comment creation
* Add stickers display to comments and hyperchats
* Fix wrong checks for total Super Chats
## Issue
We previously automatically reload when there is a chunk error. This works fine if it's the case of new code was pushed recently while the user was active. But if the failure was caused by other things like network problems or the file IS actually missing, we end up in an infinite loop of refreshes.
## New approach
Tell the user to reload instead of automatically doing it.
* fix type error
fix is subscribed check
- Persist subscription data locally
- add / remove subscription during log in / out
- Use store directly in hook
Add toast error if subscription fails
Revert removal of v2
hotfix linting issue
Add custom notification handler
- fix isSupported flag
- make icon color compatible with light/dark theme
- fix icon on notifications blocked banner
wip: add push notification banner to notifications page.
- ignore failed deletions via internal API
- add ua parsing package
- add more robust meta data to token save
refactor naming + add push toggle to notification button
shift some code around
update css naming o proper BEM notation
update notifications UI
remove now unneeded util function
Update push notification system to sue firebase sdk
separate service worker webpack bundling
update service worker to use firebase sdk
Add firebase config
Add firebase and remove filemanager
Stub out the basics for browser push notifications.
* fix safari
* try smaller image for badge
* add token validation with server, refactor code
* remove param
* add special icon for web notification badge
* add translations
* add missing trans for toast error
* add pushRequest method that will not prompt users who have subscribed but since disabled notifications in the settings.
## Issue
85 Add additional GA events
## Approach
Instead of placing analytic calls all over the GUI code (no separation of concerns), try to do it through a redux middleware instead.
## Changes
- Updated GA event and parameter naming after understanding how reporting works.
- Removed unused analytics.
followedTags:
- Moved the filtering to the reducer side, so that we don't do it every time. We can't rely on `createSelector` because the store will be invalidated on each `USER_STATE_POPULATE`, unfortunately.
tags:
- Memoize via re-reselect for the "ForUri" selector.
## Issue
`makeSelectDataForUri` always returns a new reference, so `ClaimPreview` was constantly being rendered. It's pretty expensive since `ClaimPreview`'s rendering checks against a huge blocklist, which is another issue on it's own.
## Changes
- This commit tests the usage of `re-reselect` as the solution to the multi-instance memoization problem (https://github.com/toomuchdesign/re-reselect/blob/master/examples/1-join-selectors.md)
## Issue
When comments are refreshed, each `Comment` gets rendered 4-5 times due to reference invalidation for `othersReacts` (the data didn't actually change).
## Change
For selectors without transformation, there is no need to memoize using `createSelector` -- just access it directly. Also, don't do things like `return a[id] || {}` in a reducer, because the reference to the empty object will be different on each call.
Always return directly from the state so that the same reference is returned.
This simple change avoided the wasted resources needed for `createSelector`, and reduced to render to just 2 (initial render, and when reactions are fetched).
## Issue
Components render unnecessarily due to reference invalidation from `selectMutedChannels` selector.
## Notes
`selectMutedChannels` run and return a new reference each time the app gains focus. `createSelector` will not help in this case, because we are indeed invalidating the data in the store in `USER_STATE_POPULATE`.
## Changes
- Don't update the state if the array is identical in content.
- Fixed `selectMutedChannels` to return the reference from the store, so `createSelector` is not needed.
- Also, the filtering is not needed because we've already done it in the reducer.
## Comments
I've done some profiling on large blocklists. The time needed for the array comparison is still an order magnitude lower than the time needed to render all the Components that got incorrectly marked by this.
The ideal solution is for the sync code to return a hash or timestamp of the array, so that we can compare that instead of the array.
* adding functionality to detect user download speed
* calculating bandwidth speed more intelligently
* saving download speed and updating it every 30s
* all the functionality should be done needs testing
* fix linting
* use a 1mb file for calculating bandwidth
* add optional chaining plugin to babel and get bitrate from texttrack
* allow optional chaining for flow
* ignore flow error
* disable bandwidth checking functionality
* fix flow error
* Add option to pass in url-search params.
Impetus: allow linked comment ID and setting the discussion tab when clicking on the `ClaimPreview`.
* comment.list: fix typos and renamed variables
- Switch from 'author' to 'creator' to disambiguate between comment author and content author. For comment author, we'll use 'commenter' from now on.
- Corrected 'commenterClaimId' to 'creatorClaimId' (just a typo, no functional change).
* doCommentReset: change param from uri to claimId
This reduces one lookup as clients will always have the claimID ready, but might not have the full URI.
It was using URI previously just to match the other APIs.
* Add doCommentListOwn -- command to fetch own comments
Since the redux slice is set up based on content or channel ID (for Channel Discussion page), re-use the channel ID for the case of "own comments". We always clear each ID when fetching page-0, so no worries of conflict when actually browsing the Channel Discussion page.
* Comment: add option to hide the actions section
* Implement own-comments page
* Use new param to remove sort-pins-first.
comment.List currently always pushes pins to the top to support pagination. This new param removes this behavior.
I think this the best solution so far, at the expense of a slight delay in scrolling if the network call stalls.
- Added "fetching by ID" state so that we don't need to use the ugly N-retries method.
- `scrollIntoView` doesn't work if the element is already in the viewport, and the `scrollBy` adjustment doesn't take into account the y-position restoration that we perform on certain type of pages. Use `window.scrollTo` instead and taking into account current scroll position.
## Issue
.../archives/C02FQBM00Q0/p1633044695010600
## Changes
When querying a search key, it has to be an exact match. This was broken by the insertion of `free_only` in the fetch.
Added a function to generate the options, so that all clients stay in sync.
It was being recalculated repeatedly.
This memoizes it, although it still re-calculates occasionally despite none of the source arrays changed. I think it is due to the state change in the Preference Sync.
Note: input selectors to `createSelector` needs to be extractions-only (i.e. must not have transformations). I think most of our `makeSelect*` selectors violate this and broke memoization.
* ❌ Remove old method of displaying active livestreams
Completely remove it for now to make the commit deltas clearer.
We'll replace it with the new method at the end.
* Fetch and store active-livestream info in redux
* Tiles can now query active-livestream state from redux instead of getting from parent.
* ⏪ ClaimTilesDiscover: revert and cleanup
## Simplify
- Simplify to just `uris` instead of having multiple arrays (`uris`, `modifiedUris`, `prevUris`)
- The `prevUris` is for CLS prevention. With this removal, the CLS issue is back, but we'll handle it differently later.
- Temporarily disable the view-count fetching. Code is left there so that I don't forget.
## Fix
- `shouldPerformSearch` was never true when `prefixUris` is present. Corrected the logic.
- Aside: prefix and pin is so similar in function. Hm ....
* ClaimTilesDiscover: factor out options
## Change
Move the `option` code outside and passed in as a pre-calculated prop.
## Reason
To skip rendering while waiting for `claim_search`, we need to add `React.memo(areEqual)`. However, the flag that determines if we are fetching `claim_search` (fetchingClaimSearchByQuery[]) depends on the derived options as the key.
Instead of calculating `options` twice, we moved it to the props so both sides can use it.
It also makes the component a bit more readable.
The downside is that the prop-passing might not be clear.
* ClaimTilesDiscover: reduce ~17 renders at startup to just 2.
* ClaimTilesDiscover: fill with placeholder while waiting for claim_search
## Issue
Livestream claims are fetched seperately, so they might already exists. While claim_search is running, the list only consists of livestreams (collapsed).
## Fix
Fill up the space with placeholders to prevent layout shift.
* Add 'useFetchViewCount' to handle fetching from lists
This effect also stashes fetched uris, so that we won't re-fetch the same uris during the same instance (e.g. during infinite scroll).
* ⏪ ClaimListDiscover: revert and cleanup
## Revert
- Removed the 'finalUris' stuff that was meant to "pause" visual changes when fetching. I think it'll be cleaner to use React.memo to achieve that.
## Alterations
- Added `renderUri` to make it clear which array that this component will render.
- Re-do the way we fetch view counts now that 'finalUris' is gone. Not the best method, but at least correct for now.
* ClaimListDiscover: add prefixUris, similar to ClaimTilesDiscover
This will be initially used to append livestreams at the top.
* ✅ Re-enable active livestream tiles using the new method
* doFetchActiveLivestreams: add interval check
- Added a default minimum of 5 minutes between fetches. Clients can bypass this through `forceFetch` if needed.
* doFetchActiveLivestreams: add option check
We'll need to support different 'orderBy', so adding an "options check" when determining if we just made the same fetch.
* WildWest: limit livestream tiles + add ability to show more
Most likely this behavior will change in the future, so we'll leave `ClaimListDiscover` untouched and handle the logic at the page level.
This solution uses 2 `ClaimListDiscover` -- if the reduced livestream list is visible, it handles the header; else the normal list handles the header.
* Use better tile-count on larger screens.
Used the same method as how the homepage does it.
## Issue
Closes 7121 Missing mod block option (large number of moderated channels?)
## Notes
It was bad to assume `channelSignatures` would be the same length as the promise result -- any failure in signing would cause a misalignment.
For my case, an accidentally-merged channel couldn't be signed due to a missing private key. I think it's the same for Drew.
* Fix Floating Player stopping on certain files
* Dont show additional player buttons from markdown and comments
* Fix markdown resizing for the same video playing
* Update changelog
## Issue
7003 Can't unblock if delegator deleted their channel
## Changes
- Changed the function parameter from 'creatorId' to 'creatorUri'
- It got short-circuited because we don't resolve deleted channels. But the client already have the full creator URI (containing the needed 'name' and 'id'), so there is no need to actually look at the resolved list -- just pass the uri like all the other functions.
## Preamble
- The app-side uses a cached blocklist, so when a Timeout expires, it doesn't know that the ban has been lifted.
- Meanwhile, we are doing extra comment filtering using this blocklist (so that we don't see comments that we have blocked, regardless of whose claim we are viewing).
## Issue
In a livestream, if a new message from an ex-offender comes in after their ban has been lifted, we do get the websocket message but it's being filtered out locally as mentioned above. So, the msg ended up being visible for everyone except the owner.
## Fix (band aid)
- Don't run the extra filter if the claim we are viewing is ours -- commentron would have filtered it for us anyways, and is the right logic to use even before this Timeout feature is introduced.
- For the case of Timeout, this only serves as a band-aid until Commentron Issue 80 is available for us to detect the ban has been lifted. This is because it doesn't handle the case where I am a viewer and I decided to timeout someone for a few minutes. Because I am not the owner of the claim, the offender will continue to be blocked due to the same issue mentioned above.
