The download directory gets deleted before the publish finishes #6

New issue

Closed

opened 2018-04-20 21:25:23 +02:00 by nikooo777 · 2 comments

nikooo777 commented

2018-04-20 21:25:23 +02:00

(Migrated from github.com)

*/10 * * * * (/bin/ls /tmp/ | /bin/grep -q ytsync && /usr/bin/find /tmp/ytsync* -mmin +20 -delete) || true
^ the above command might have something to do with this error.

At a certain point the download might fail and from there on all publishes fail.

example: ERRO[62269] error processing video: download error: open /tmp/ytsync259862098/iMvNYejP8xY.mp4: no such file or directory

this is probably also related to what i experienced a couple of days ago

INFO[4719] Processing 9rscUMFPQ8c (14237 in channel)
DEBU[4721] Downloaded 9rscUMFPQ8c
DEBU[4722] Created thumbnail for 9rscUMFPQ8c
DEBU[4722] jsonrpc: publish author=David Pakman Show bid=0.01 channel_name=@davidpakman claim_address=bZxmbHGKxuo5ewDP8Cedkyt5ippCxHHC3i description=--Mitt Romney's mobile app misspells America as "Amercia" and we wonder if that's actually the country that will benefit if Romney becomes President.

--On the Bonus Show: Misleading Drudge headline, TX honor student jailed for missing class, Bilderberg US meeting upcoming, more.

The David Pakman Show is an internationally syndicated talk radio and television program hosted by David Pakman

http://www.davidpakman.com
http://www.davidpakman.com/membership
http://www.davidpakman.com/gear
http://www.facebook.com/davidpakmanshow
...
https://www.youtube.com/watch?v=9rscUMFPQ8c file_path=/tmp/ytsync782624140/9rscUMFPQ8c.mp4 language=en license=Copyrighted (contact author) name=romney-campaign-misspells-america-on thumbnail=https://berk.ninja/thumbnails/9rscUMFPQ8c title=Romney Campaign Misspells AMERICA on Mobile App
INFO[5029] 9rscUMFPQ8c took 5m8.70042584s
ERRO[5030] error processing video: publish error: Post http://localhost:5279: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
INFO[5033] Retrying
INFO[5034] Processing 9rscUMFPQ8c (14237 in channel)
DEBU[5039] 9rscUMFPQ8c already exists at /tmp/ytsync782624140/9rscUMFPQ8c.mp4
DEBU[5040] Downloaded 9rscUMFPQ8c
INFO[5080] 9rscUMFPQ8c took 41.630574618s
ERRO[5085] error processing video: thumbnail error: Put https://jgp4g1qoud.execute-api.us-east-1.amazonaws.com/prod/thumbnail: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
INFO[5089] Retrying
INFO[5090] Processing 9rscUMFPQ8c (14237 in channel)
DEBU[5092] 9rscUMFPQ8c already exists at /tmp/ytsync782624140/9rscUMFPQ8c.mp4
DEBU[5094] Downloaded 9rscUMFPQ8c
INFO[5128] 9rscUMFPQ8c took 37.483586185s
ERRO[5129] error processing video: thumbnail error: Put https://jgp4g1qoud.execute-api.us-east-1.amazonaws.com/prod/thumbnail: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
INFO[5129] Video failed after 3 retries, skipping

so either the download error is not handled correctly or the cronjob deletes videos when it's not supposed to do so.

`*/10 * * * * (/bin/ls /tmp/ | /bin/grep -q ytsync && /usr/bin/find /tmp/ytsync* -mmin +20 -delete) || true` ^ the above command might have something to do with this error. At a certain point the download might fail and from there on all publishes fail. example: `ERRO[62269] error processing video: download error: open /tmp/ytsync259862098/iMvNYejP8xY.mp4: no such file or directory` this is probably also related to what i experienced a couple of days ago ``` INFO[4719] Processing 9rscUMFPQ8c (14237 in channel) DEBU[4721] Downloaded 9rscUMFPQ8c DEBU[4722] Created thumbnail for 9rscUMFPQ8c DEBU[4722] jsonrpc: publish author=David Pakman Show bid=0.01 channel_name=@davidpakman claim_address=bZxmbHGKxuo5ewDP8Cedkyt5ippCxHHC3i description=--Mitt Romney's mobile app misspells America as "Amercia" and we wonder if that's actually the country that will benefit if Romney becomes President. --On the Bonus Show: Misleading Drudge headline, TX honor student jailed for missing class, Bilderberg US meeting upcoming, more. The David Pakman Show is an internationally syndicated talk radio and television program hosted by David Pakman http://www.davidpakman.com http://www.davidpakman.com/membership http://www.davidpakman.com/gear http://www.facebook.com/davidpakmanshow ... https://www.youtube.com/watch?v=9rscUMFPQ8c file_path=/tmp/ytsync782624140/9rscUMFPQ8c.mp4 language=en license=Copyrighted (contact author) name=romney-campaign-misspells-america-on thumbnail=https://berk.ninja/thumbnails/9rscUMFPQ8c title=Romney Campaign Misspells AMERICA on Mobile App INFO[5029] 9rscUMFPQ8c took 5m8.70042584s ERRO[5030] error processing video: publish error: Post http://localhost:5279: net/http: request canceled (Client.Timeout exceeded while awaiting headers) INFO[5033] Retrying INFO[5034] Processing 9rscUMFPQ8c (14237 in channel) DEBU[5039] 9rscUMFPQ8c already exists at /tmp/ytsync782624140/9rscUMFPQ8c.mp4 DEBU[5040] Downloaded 9rscUMFPQ8c INFO[5080] 9rscUMFPQ8c took 41.630574618s ERRO[5085] error processing video: thumbnail error: Put https://jgp4g1qoud.execute-api.us-east-1.amazonaws.com/prod/thumbnail: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) INFO[5089] Retrying INFO[5090] Processing 9rscUMFPQ8c (14237 in channel) DEBU[5092] 9rscUMFPQ8c already exists at /tmp/ytsync782624140/9rscUMFPQ8c.mp4 DEBU[5094] Downloaded 9rscUMFPQ8c INFO[5128] 9rscUMFPQ8c took 37.483586185s ERRO[5129] error processing video: thumbnail error: Put https://jgp4g1qoud.execute-api.us-east-1.amazonaws.com/prod/thumbnail: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) INFO[5129] Video failed after 3 retries, skipping ``` so either the download error is not handled correctly or the cronjob deletes videos when it's not supposed to do so.

nikooo777 commented

2018-04-25 02:22:56 +02:00

(Migrated from github.com)

i deactivated the cron job and this didn't happen anymore.
I'll resolve this by deleting the file from the code instead. Makes more sense to do.

i deactivated the cron job and this didn't happen anymore. I'll resolve this by deleting the file from the code instead. Makes more sense to do.

nikooo777 commented

2018-04-25 23:13:50 +02:00

(Migrated from github.com)

the cron job was the issue. I implemented it in the code and it works flawlessly

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/lbry.go#6

No description provided.