lbry-desktop/web/setup/publish-v2.js

154 lines
5.3 KiB
JavaScript
Raw Normal View History

Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
// @flow
import * as tus from 'tus-js-client';
2021-11-12 09:30:33 +01:00
import analytics from '../../ui/analytics';
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
import { X_LBRY_AUTH_TOKEN } from '../../ui/constants/token';
import { doUpdateUploadAdd, doUpdateUploadProgress, doUpdateUploadRemove } from '../../ui/redux/actions/publish';
import { LBRY_WEB_PUBLISH_API_V2 } from 'config';
const RESUMABLE_ENDPOINT = LBRY_WEB_PUBLISH_API_V2;
const RESUMABLE_ENDPOINT_METHOD = 'publish';
const UPLOAD_CHUNK_SIZE_BYTE = 10 * 1024 * 1024;
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
const STATUS_CONFLICT = 409;
const STATUS_LOCKED = 423;
/**
* Checks whether a given status is in the range of the expected category.
*
* @param status
* @param category
* @returns {boolean}
*/
function inStatusCategory(status, category) {
return status >= category && status < category + 100;
}
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
export function makeResumableUploadRequest(
token: string,
params: FileUploadSdkParams,
file: File | string,
isPreview?: boolean
) {
return new Promise<any>((resolve, reject) => {
if (!RESUMABLE_ENDPOINT) {
reject(new Error('Publish: endpoint undefined'));
}
if (params.remote_url) {
reject(new Error('Publish: v2 does not support remote_url'));
}
Upload: tab sync and various fixes (#428) * Upload: fix redux key clash ## Issue `params` is the "final" value that will be passed to the SDK and `channel` is not a valid argument (it should be `channel_name`). Also, it seems like we only pass the channel ID now and skip the channel name entirely. For the anonymous case, a clash will still happen when since the channel part is hardcoded to `anonymous`. ## Approach Generate a guid in `params` and use that as the key to handle all the cases above. We couldn't use the `uploadUrl` because v1 doesn't have it. The old formula is retained to allow users to retry or cancel their existing uploads one last time (otherwise it will persist forever). The next upload will be using the new key. * Upload: add tab-locking ## Issue - The previous code does detect uploads from multiple tabs, but it was done by handling the CONFLICT error message from the backend. At certain corner-cases, this does not work well. A better way is to not allow resumption while the same file is being uploading from another tab. - When an upload from 1 tab finishes, the GUI on the other tab does not remove the completed item. User either have to refresh or click Cancel. Clicking Cancel results in the 404 backend error. This should be avoided. ## Approach - Added tab synchronization and locking by passing the "locked" and "removed" information through `localStorage`. ## Other considered approaches - Wallet sync -- but decided not to pollute the wallet. - 3rd-party redux tab syncing -- but decided it's not worth adding another module for 1 usage. * Upload: check if locked before confirming delete ## Reproduce Have 2 tabs + paused upload Open "cancel" dialog in one of the tabs. Continue upload in other tab Confirm cancellation in first tab Upload disappears from both tabs, but based on network traffic the upload keeps happening. (If upload finishes the claim seems to get created)
2021-12-07 15:48:09 +01:00
const { uploadUrl, guid, ...sdkParams } = params;
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
const jsonPayload = JSON.stringify({
jsonrpc: '2.0',
method: RESUMABLE_ENDPOINT_METHOD,
Upload: tab sync and various fixes (#428) * Upload: fix redux key clash ## Issue `params` is the "final" value that will be passed to the SDK and `channel` is not a valid argument (it should be `channel_name`). Also, it seems like we only pass the channel ID now and skip the channel name entirely. For the anonymous case, a clash will still happen when since the channel part is hardcoded to `anonymous`. ## Approach Generate a guid in `params` and use that as the key to handle all the cases above. We couldn't use the `uploadUrl` because v1 doesn't have it. The old formula is retained to allow users to retry or cancel their existing uploads one last time (otherwise it will persist forever). The next upload will be using the new key. * Upload: add tab-locking ## Issue - The previous code does detect uploads from multiple tabs, but it was done by handling the CONFLICT error message from the backend. At certain corner-cases, this does not work well. A better way is to not allow resumption while the same file is being uploading from another tab. - When an upload from 1 tab finishes, the GUI on the other tab does not remove the completed item. User either have to refresh or click Cancel. Clicking Cancel results in the 404 backend error. This should be avoided. ## Approach - Added tab synchronization and locking by passing the "locked" and "removed" information through `localStorage`. ## Other considered approaches - Wallet sync -- but decided not to pollute the wallet. - 3rd-party redux tab syncing -- but decided it's not worth adding another module for 1 usage. * Upload: check if locked before confirming delete ## Reproduce Have 2 tabs + paused upload Open "cancel" dialog in one of the tabs. Continue upload in other tab Confirm cancellation in first tab Upload disappears from both tabs, but based on network traffic the upload keeps happening. (If upload finishes the claim seems to get created)
2021-12-07 15:48:09 +01:00
params: sdkParams,
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
id: new Date().getTime(),
});
const urlOptions = {};
if (params.uploadUrl) {
// Resuming from previous upload. TUS clears the resume fingerprint on any
// 4xx error, so we need to use the fixed URL mode instead.
urlOptions.uploadUrl = params.uploadUrl;
} else {
// New upload, so use `endpoint`.
urlOptions.endpoint = RESUMABLE_ENDPOINT;
}
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
const uploader = new tus.Upload(file, {
...urlOptions,
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
chunkSize: UPLOAD_CHUNK_SIZE_BYTE,
retryDelays: [5000, 10000, 30000],
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
parallelUploads: 1,
storeFingerprintForResuming: false,
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
removeFingerprintOnSuccess: true,
headers: { [X_LBRY_AUTH_TOKEN]: token },
metadata: {
filename: file instanceof File ? file.name : file,
filetype: file instanceof File ? file.type : undefined,
},
onShouldRetry: (err, retryAttempt, options) => {
Upload: tab sync and various fixes (#428) * Upload: fix redux key clash ## Issue `params` is the "final" value that will be passed to the SDK and `channel` is not a valid argument (it should be `channel_name`). Also, it seems like we only pass the channel ID now and skip the channel name entirely. For the anonymous case, a clash will still happen when since the channel part is hardcoded to `anonymous`. ## Approach Generate a guid in `params` and use that as the key to handle all the cases above. We couldn't use the `uploadUrl` because v1 doesn't have it. The old formula is retained to allow users to retry or cancel their existing uploads one last time (otherwise it will persist forever). The next upload will be using the new key. * Upload: add tab-locking ## Issue - The previous code does detect uploads from multiple tabs, but it was done by handling the CONFLICT error message from the backend. At certain corner-cases, this does not work well. A better way is to not allow resumption while the same file is being uploading from another tab. - When an upload from 1 tab finishes, the GUI on the other tab does not remove the completed item. User either have to refresh or click Cancel. Clicking Cancel results in the 404 backend error. This should be avoided. ## Approach - Added tab synchronization and locking by passing the "locked" and "removed" information through `localStorage`. ## Other considered approaches - Wallet sync -- but decided not to pollute the wallet. - 3rd-party redux tab syncing -- but decided it's not worth adding another module for 1 usage. * Upload: check if locked before confirming delete ## Reproduce Have 2 tabs + paused upload Open "cancel" dialog in one of the tabs. Continue upload in other tab Confirm cancellation in first tab Upload disappears from both tabs, but based on network traffic the upload keeps happening. (If upload finishes the claim seems to get created)
2021-12-07 15:48:09 +01:00
window.store.dispatch(doUpdateUploadProgress({ guid, status: 'retry' }));
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
const status = err.originalResponse ? err.originalResponse.getStatus() : 0;
return !inStatusCategory(status, 400) || status === STATUS_CONFLICT || status === STATUS_LOCKED;
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
},
2021-11-22 09:06:28 +01:00
onError: (err) => {
const status = err.originalResponse ? err.originalResponse.getStatus() : 0;
const errMsg = typeof err === 'string' ? err : err.message;
let customErr;
if (status === STATUS_LOCKED || errMsg === 'file currently locked') {
customErr = 'File is locked. Try resuming after waiting a few minutes';
2021-11-22 09:06:28 +01:00
}
window.store.dispatch(doUpdateUploadProgress({ guid, status: 'error' }));
reject(
// $FlowFixMe - flow's constructor for Error is incorrect.
new Error(customErr || err, {
cause: {
url: uploader.url,
status,
...(uploader._fingerprint ? { fingerprint: uploader._fingerprint } : {}),
...(uploader._retryAttempt ? { retryAttempt: uploader._retryAttempt } : {}),
...(uploader._offsetBeforeRetry ? { offsetBeforeRetry: uploader._offsetBeforeRetry } : {}),
...(customErr ? { original: errMsg } : {}),
},
})
);
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
},
onProgress: (bytesUploaded, bytesTotal) => {
const percentage = ((bytesUploaded / bytesTotal) * 100).toFixed(2);
Upload: tab sync and various fixes (#428) * Upload: fix redux key clash ## Issue `params` is the "final" value that will be passed to the SDK and `channel` is not a valid argument (it should be `channel_name`). Also, it seems like we only pass the channel ID now and skip the channel name entirely. For the anonymous case, a clash will still happen when since the channel part is hardcoded to `anonymous`. ## Approach Generate a guid in `params` and use that as the key to handle all the cases above. We couldn't use the `uploadUrl` because v1 doesn't have it. The old formula is retained to allow users to retry or cancel their existing uploads one last time (otherwise it will persist forever). The next upload will be using the new key. * Upload: add tab-locking ## Issue - The previous code does detect uploads from multiple tabs, but it was done by handling the CONFLICT error message from the backend. At certain corner-cases, this does not work well. A better way is to not allow resumption while the same file is being uploading from another tab. - When an upload from 1 tab finishes, the GUI on the other tab does not remove the completed item. User either have to refresh or click Cancel. Clicking Cancel results in the 404 backend error. This should be avoided. ## Approach - Added tab synchronization and locking by passing the "locked" and "removed" information through `localStorage`. ## Other considered approaches - Wallet sync -- but decided not to pollute the wallet. - 3rd-party redux tab syncing -- but decided it's not worth adding another module for 1 usage. * Upload: check if locked before confirming delete ## Reproduce Have 2 tabs + paused upload Open "cancel" dialog in one of the tabs. Continue upload in other tab Confirm cancellation in first tab Upload disappears from both tabs, but based on network traffic the upload keeps happening. (If upload finishes the claim seems to get created)
2021-12-07 15:48:09 +01:00
window.store.dispatch(doUpdateUploadProgress({ guid, progress: percentage }));
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
},
onSuccess: () => {
let retries = 1;
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
function makeNotifyRequest() {
const xhr = new XMLHttpRequest();
xhr.open('POST', `${uploader.url}/notify`);
xhr.setRequestHeader('Content-Type', 'application/json');
xhr.setRequestHeader('Tus-Resumable', '1.0.0');
xhr.setRequestHeader(X_LBRY_AUTH_TOKEN, token);
xhr.responseType = 'json';
xhr.onload = () => {
Upload: tab sync and various fixes (#428) * Upload: fix redux key clash ## Issue `params` is the "final" value that will be passed to the SDK and `channel` is not a valid argument (it should be `channel_name`). Also, it seems like we only pass the channel ID now and skip the channel name entirely. For the anonymous case, a clash will still happen when since the channel part is hardcoded to `anonymous`. ## Approach Generate a guid in `params` and use that as the key to handle all the cases above. We couldn't use the `uploadUrl` because v1 doesn't have it. The old formula is retained to allow users to retry or cancel their existing uploads one last time (otherwise it will persist forever). The next upload will be using the new key. * Upload: add tab-locking ## Issue - The previous code does detect uploads from multiple tabs, but it was done by handling the CONFLICT error message from the backend. At certain corner-cases, this does not work well. A better way is to not allow resumption while the same file is being uploading from another tab. - When an upload from 1 tab finishes, the GUI on the other tab does not remove the completed item. User either have to refresh or click Cancel. Clicking Cancel results in the 404 backend error. This should be avoided. ## Approach - Added tab synchronization and locking by passing the "locked" and "removed" information through `localStorage`. ## Other considered approaches - Wallet sync -- but decided not to pollute the wallet. - 3rd-party redux tab syncing -- but decided it's not worth adding another module for 1 usage. * Upload: check if locked before confirming delete ## Reproduce Have 2 tabs + paused upload Open "cancel" dialog in one of the tabs. Continue upload in other tab Confirm cancellation in first tab Upload disappears from both tabs, but based on network traffic the upload keeps happening. (If upload finishes the claim seems to get created)
2021-12-07 15:48:09 +01:00
window.store.dispatch(doUpdateUploadRemove(guid));
resolve(xhr);
};
xhr.onerror = () => {
if (retries > 0 && xhr.status === 0) {
--retries;
2021-11-12 09:30:33 +01:00
analytics.error('notify: first attempt failed (status=0). Retrying after 10s...');
setTimeout(() => makeNotifyRequest(), 10000); // Auto-retry after 10s delay.
} else {
Upload: tab sync and various fixes (#428) * Upload: fix redux key clash ## Issue `params` is the "final" value that will be passed to the SDK and `channel` is not a valid argument (it should be `channel_name`). Also, it seems like we only pass the channel ID now and skip the channel name entirely. For the anonymous case, a clash will still happen when since the channel part is hardcoded to `anonymous`. ## Approach Generate a guid in `params` and use that as the key to handle all the cases above. We couldn't use the `uploadUrl` because v1 doesn't have it. The old formula is retained to allow users to retry or cancel their existing uploads one last time (otherwise it will persist forever). The next upload will be using the new key. * Upload: add tab-locking ## Issue - The previous code does detect uploads from multiple tabs, but it was done by handling the CONFLICT error message from the backend. At certain corner-cases, this does not work well. A better way is to not allow resumption while the same file is being uploading from another tab. - When an upload from 1 tab finishes, the GUI on the other tab does not remove the completed item. User either have to refresh or click Cancel. Clicking Cancel results in the 404 backend error. This should be avoided. ## Approach - Added tab synchronization and locking by passing the "locked" and "removed" information through `localStorage`. ## Other considered approaches - Wallet sync -- but decided not to pollute the wallet. - 3rd-party redux tab syncing -- but decided it's not worth adding another module for 1 usage. * Upload: check if locked before confirming delete ## Reproduce Have 2 tabs + paused upload Open "cancel" dialog in one of the tabs. Continue upload in other tab Confirm cancellation in first tab Upload disappears from both tabs, but based on network traffic the upload keeps happening. (If upload finishes the claim seems to get created)
2021-12-07 15:48:09 +01:00
window.store.dispatch(doUpdateUploadProgress({ guid, status: 'error' }));
reject(new Error(`There was a problem in the processing. Please retry. (${xhr.status})`));
}
};
xhr.onabort = () => {
Upload: tab sync and various fixes (#428) * Upload: fix redux key clash ## Issue `params` is the "final" value that will be passed to the SDK and `channel` is not a valid argument (it should be `channel_name`). Also, it seems like we only pass the channel ID now and skip the channel name entirely. For the anonymous case, a clash will still happen when since the channel part is hardcoded to `anonymous`. ## Approach Generate a guid in `params` and use that as the key to handle all the cases above. We couldn't use the `uploadUrl` because v1 doesn't have it. The old formula is retained to allow users to retry or cancel their existing uploads one last time (otherwise it will persist forever). The next upload will be using the new key. * Upload: add tab-locking ## Issue - The previous code does detect uploads from multiple tabs, but it was done by handling the CONFLICT error message from the backend. At certain corner-cases, this does not work well. A better way is to not allow resumption while the same file is being uploading from another tab. - When an upload from 1 tab finishes, the GUI on the other tab does not remove the completed item. User either have to refresh or click Cancel. Clicking Cancel results in the 404 backend error. This should be avoided. ## Approach - Added tab synchronization and locking by passing the "locked" and "removed" information through `localStorage`. ## Other considered approaches - Wallet sync -- but decided not to pollute the wallet. - 3rd-party redux tab syncing -- but decided it's not worth adding another module for 1 usage. * Upload: check if locked before confirming delete ## Reproduce Have 2 tabs + paused upload Open "cancel" dialog in one of the tabs. Continue upload in other tab Confirm cancellation in first tab Upload disappears from both tabs, but based on network traffic the upload keeps happening. (If upload finishes the claim seems to get created)
2021-12-07 15:48:09 +01:00
window.store.dispatch(doUpdateUploadRemove(guid));
};
xhr.send(jsonPayload);
}
setTimeout(() => makeNotifyRequest(), 15000);
Support resume-able upload via tus (#186) * Publish button: use spinner instead of "Publishing..." Looks better, plus the preview could take a while sometimes. * Refactor `doPublish`. No functional change This is to allow `doPublish` to accept a custom payload as an input (for resuming uploads), instead of always resolving it from the redux data. * Add doPublishResume * Support resume-able upload via tus ## Issue 38 Handle resumable file upload ## Notes Since we can't serialize a File object, we'll need to the user to re-select the file to resume. * Exclude "modified date" for Firefox/Android ## Issue It appears that the modification date of the Android file changes when selected, so that file was deemed "different" when trying to resume upload. ## Change Exclude modification date for now. Let's assume a smart user. * Move 'currentUploads' to 'publish' reducer `publish` is currently rehydrated, so we can ride on that and don't need to store the `currentUploads` in `localStorage` for persistence. This would allow us to store Markdown Post data too, as `localStorage` has a 5MB limit per app. We could have also made `webReducer` rehydrate, but in this repo, there is no need to split it to another reducer. It also makes more sense to be part of publish anyway (at least to me). This change is mostly moving items between files, with the exception of 1. An additional REHYDRATE in the publish reducer to clean up the tusUploader. 2. Not clearing `currentUploads` in CLEAR_PUBLISH. * Restore v1 code for livestream replay, etc. v2 (tus) does not handle `remote_url`, so the app still needs v1 for that. Since we'll still have v1 code, use v1 for previews as well.
2021-11-10 19:16:16 +01:00
},
});
uploader
.findPreviousUploads()
.then((previousUploads) => {
if (!isPreview) {
window.store.dispatch(doUpdateUploadAdd(file, params, uploader));
}
uploader.start();
})
.catch((err) => {
reject(new Error(__('Failed to initiate upload (%err%)', { err })));
});
});
}