Commit graph

33 commits

Author SHA1 Message Date
Niko Storni
ea3315d1d6 this is actually necessary 2022-08-16 02:14:33 +02:00
Niko Storni
57e017ec8f fix some logic
reduce verbosity of debug logs
2022-08-15 21:19:20 +02:00
Niko Storni
7ff1a009da remove unused params 2022-01-14 18:49:02 +01:00
Niko Storni
17944fa46a refactor get video time
remove broken time lookup
refactor quite some code
2021-12-30 13:17:11 -05:00
Niko Storni
070287716b switch from env vars to a config file
get rid of stupid stuff
simplify S3 configuration
split wallets from blockchain.db and use separate S3 store
fix bugs
2021-11-24 05:54:08 +01:00
Niko Storni
519e1e4648 switch to yt-dlp
add progressbars
avoid unnecessary calls to youtube
update user agents
cookies fixes
bug fixes
introduction of new bugs
2021-06-17 17:51:21 +02:00
Niko Storni
d7e194cb5c update chrome UA
change throttle timeout to 48 hours
2021-02-11 23:44:48 +01:00
Niko Storni
6a4093f3f8
fix youtube channel info parsing 2020-11-27 13:12:01 -05:00
Niko Storni
bac2969295 prevent panic failure 2020-11-19 15:58:31 +01:00
Niko Storni
beade71aa6 fix bugs 2020-11-03 02:14:01 +01:00
Niko Storni
c53cf4c1b3 prevent channels from failing when manually killed 2020-08-26 19:05:16 +02:00
Niko Storni
ecda80b02d bypass socialblade block
stop using socialblade for now
2020-08-21 00:15:14 +02:00
Niko Storni
0b002c8228 lots of bug fixes and improvements 2020-08-18 00:03:38 +02:00
Niko Storni
fb0e567caf be more specific on video failures 2020-08-08 03:29:57 +02:00
Niko Storni
7c02c5b92d refactor structures
fix various bugs
2020-08-08 01:12:55 +02:00
Niko Storni
5d230a6b54 potentially fix looping
add limits by db
2020-08-06 20:32:49 +02:00
Niko Storni
f0280b51b4 fix time parsing for real 2020-08-06 02:12:05 +02:00
Niko Storni
ec9f46f552 fix release date bug 2020-08-06 00:25:54 +02:00
Niko Storni
751bc84ce5
grab channel info without APIs
fix new channel syncs
fix e2e tests
2020-08-05 15:47:53 -04:00
Mark Beamer Jr
5be3551abe
Mark video failed if we cannot get video info instead of failing the sync
Turn on error log after fixing lockup
Try different user agents if calls fail
Prevent double processing a channel in a row, just fail the sync instead
2020-08-05 15:47:53 -04:00
Mark Beamer Jr
a1caea4a28
added logging 2020-08-05 15:47:53 -04:00
Mark Beamer Jr
64040ea67a
Add calls to internal-apis and add get released date if available. 2020-08-05 15:47:47 -04:00
Mark Beamer Jr
d99e200178
Add check for already published videos using new video-state api. 2020-07-29 00:12:23 -04:00
Alex Grintsvayg
0eef62b5fd
i made a huge mess. sorry future me, when you're digging through this 2020-07-28 21:34:08 -04:00
Alex Grintsvayg
8600077caa
no panics. thanks Beam-master 2020-07-27 17:27:50 -04:00
Mark Beamer Jr
0bb6b6d833
commit my shit 2020-07-27 17:14:18 -04:00
Alex Grintsvayg
b59ef28267
punting on ChannelInfo for now 2020-07-27 16:02:52 -04:00
Mark Beamer Jr
e926a2c1f6
Fix quickSync && maxVideos 2020-07-27 15:59:14 -04:00
Alex Grintsvayg
1369ed0b48
replace yt channel video count with a scrape of socialblade.com 2020-07-27 15:51:06 -04:00
Mark Beamer Jr
f39fc11697
Add downloader to create api
adjust GetVideosToSync
2020-07-27 15:42:45 -04:00
Alex Grintsvayg
bffc0823be
more 2020-07-27 14:52:21 -04:00
Alex Grintsvayg
edbb22fcf0
rename 2020-07-27 14:49:37 -04:00
Alex Grintsvayg
843303301a
move all youtube api calls into a single place 2020-07-27 14:48:05 -04:00