handle throttling automatically #98
Labels
No labels
area: devops
area: discovery
area: docs
area: livestream
area: proposal
consider soon
dependencies
Epic
good first issue
hacktoberfest
help wanted
icebox
level: 1
level: 2
level: 3
level: 4
needs: exploration
needs: grooming
needs: priority
needs: repro
needs: tech design
on hold
priority: blocker
priority: high
priority: low
priority: medium
resilience
sdk dependent
Tom's Wishlist
type: bug
type: discussion
type: improvement
type: new feature
type: refactor
type: task
type: testing
unplanned
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: LBRYCommunity/ytsync#98
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Today we need to shutdown servers, wait, fix yt sync records so they are picked up by other channels, etc.
23 recently got stuck at
eleting original file /tmp/ytsync318973938/QTWjcTZI5ko/jennifer-love-hewitt-stories.f140 (pass -k to keep) ERRO[694297] 2604:2dc0:101:200::e9f set to throttled DEBU[694297] slack: ytsync-status: :sos: 2604:2dc0:101:200::e9f set to throttled INFO[694297] Command finished with error: WARNING: Unable to download webpage: HTTP Error 429: Too Many Requests INFO[694297] /tmp/ytsync318973938/QTWjcTZI5ko DEBU[694297] QTWjcTZI5ko deleted from disk for 'due to error' (/tmp/ytsync318973938/QTWjcTZI5ko/jennifer-love-hewitt-stories.mp4) INFO[694312] Running command youtube-dl --no-progress -o/tmp/ytsync318973938/NE6xHQGFJSc/jennifer-love-hewitt-stories --merge-output-format mp4 --rm-cache-dir --postprocessor-args -movflags faststart --abort-on-unavailable-fragment --fragment-retries 1 --cookies cookies.txt --max-filesize 2048M --match-filter duration <= 7200 --source-address 2604:2dc0:101:200::e9f https://www.youtube.com/watch?v=NE6xHQGFJSc -fbestvideo[ext=mp4][height<=1080]+bestaudio[ext!=webm] --user-agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36 DEBU[694315] Removing cache dir /home/lbry/.cache/youtube-dl .. [youtube] NE6xHQGFJSc: Downloading webpage [youtube] NE6xHQGFJSc: Downloading API JSON [youtube] NE6xHQGFJSc: Downloading MPD manifest [youtube] NE6xHQGFJSc: Downloading API JSON [download] Destination: /tmp/ytsync318973938/NE6xHQGFJSc/jennifer-love-hewitt-stories.f135 [download] Download completed [download] Destination: /tmp/ytsync318973938/NE6xHQGFJSc/jennifer-love-hewitt-stories.f140 [download] Download completed [ffmpeg] Merging formats into "/tmp/ytsync318973938/NE6xHQGFJSc/jennifer-love-hewitt-stories.mp4" Deleting original file /tmp/ytsync318973938/NE6xHQGFJSc/jennifer-love-hewitt-stories.f135 (pass -k to keep) Deleting original file /tmp/ytsync318973938/NE6xHQGFJSc/jennifer-love-hewitt-stories.f140 (pass -k to keep) INFO[694315] Command finished with error: WARNING: Unable to download webpage: HTTP Error 429: Too Many Requests INFO[694315] /tmp/ytsync318973938/NE6xHQGFJSc DEBU[694315] NE6xHQGFJSc deleted from disk for 'due to error' (/tmp/ytsync318973938/NE6xHQGFJSc/jennifer-love-hewitt-stories.mp4)
it detects it - it should stop syncing so that another server can pick it up and then wait a day.