Commit graph

411 commits

Author SHA1 Message Date
Niko Storni 29d1ccf68c add singleflight to web requests 2022-05-02 23:07:22 +02:00
Niko Storni 2f7d67794f remove debug leftover 2022-02-10 00:57:13 +01:00
Niko Storni 4d8e7739d7 Merge branch 'fix-ci' 2021-12-14 22:36:01 +01:00
Niko Storni 6fc0ceea2a adjust travis vars
fix build script


fix more scripts


adapt script to scale
2021-12-14 22:35:44 +01:00
Niko Storni ae0c7dd2bb upgrade quic-go for go1.17 2021-12-14 20:49:50 +01:00
Niko Storni 4af5c2f4c6 make slack channel configurable 2021-10-30 00:21:58 +02:00
Alex Grin def0a97f49
Update readme.md 2021-09-28 10:15:24 -04:00
Niko Storni 6dde793745 do not delete blobs for blocked content 2021-09-21 18:00:55 +02:00
Niko Storni 654cc44935 Merge branch 'hash_twice' 2021-09-21 16:21:25 +02:00
Victor Shyba 90d6d29452 remove all hashing on the download path 2021-09-21 16:19:49 +02:00
Niko Storni b2272fef3a delete overflowing blobs from underlying cache 2021-08-18 19:36:47 +02:00
Niko Storni 86f3e62aa8 fix a panic error
update gin-go
2021-08-05 17:47:20 +02:00
Niko e1b4f21e00
Merge pull request #52 from lbryio/ittt
Merge months of work including all sort of caches and O_DIRECT optimizations
2021-07-23 20:35:22 -04:00
Niko Storni b4913ecedf cleanup 2021-07-24 01:03:51 +02:00
Niko Storni 63a574ec2f unify caches
fix tests
2021-07-24 01:03:51 +02:00
Niko Storni b8af3408e0 move server packages 2021-07-24 01:03:51 +02:00
Niko Storni 847089d0d6 fix error propagation
update readme
2021-07-24 01:03:51 +02:00
Niko Storni 170dfef3a8 fix copy pasta mistake 2021-07-24 01:03:51 +02:00
Niko Storni 2b458a6bd0 fix params
more cleanups
2021-07-24 01:03:51 +02:00
Niko Storni febfc51cb0 refactor refactor refactor 2021-07-24 01:03:51 +02:00
Andrey Beletsky 72be487262 Fix broken import 2021-07-24 01:03:51 +02:00
Andrey Beletsky 94e7d81bd3 Fix OpenFile call flags for macos 2021-07-24 01:03:51 +02:00
Niko Storni c6c779da39 fix panic
fix counter leak
2021-07-24 01:03:51 +02:00
Niko Storni 2e101083e6 write blobs to tmp dir to avoid corruption 2021-07-24 01:03:51 +02:00
Niko Storni 63aacd8a69 use O_DIRECT to write to disk (fixes everything)
add queue back to serving blobs
improve a lot of things
upgrade modules
2021-07-24 01:03:51 +02:00
Niko Storni c03ae6487d fix unsafe dereference 2021-07-24 01:03:51 +02:00
Niko Storni 0c4f455f0c add metrics 2021-07-24 01:03:51 +02:00
Niko Storni af3e08c446 update lbry.go dep 2021-07-24 01:03:51 +02:00
Niko Storni 975bfe7fac upgrade singleflight
http store fix
2021-07-24 01:03:51 +02:00
Niko Storni b075d948bb remove locks causing deadlocks 2021-07-24 01:03:51 +02:00
Niko Storni 2651a64dbb add http server/client 2021-07-24 01:03:51 +02:00
Mark Beamer Jr fa7150cf2b Add queue to prevent writing too many files at once. 2021-07-24 01:03:51 +02:00
Mark Beamer Jr 6c4db980c9 Add queue to prevent writing too many files at once. 2021-07-24 01:03:51 +02:00
Mark Beamer Jr 7adaa510fd Add locks to disk store. 2021-07-24 01:03:51 +02:00
Niko Storni 64ed7304f6 add a lot of extra heavy debugging 2021-07-24 01:03:51 +02:00
Mark Beamer Jr 5aefaf061e Add single flight for cache not just origin 2021-07-24 01:03:51 +02:00
Mark Beamer Jr 724ee47c8b add metric calls for other packages 2021-07-24 01:03:51 +02:00
Mark Beamer Jr caaec6fcb1 add guage metrics for go routines in reflector package 2021-07-24 01:03:51 +02:00
Mark Beamer Jr 15984b8fd9 add gops to reflector server 2021-07-24 01:03:51 +02:00
Niko Storni 2be913b077 request queue size param 2021-07-24 01:03:51 +02:00
Niko Storni 34c11b0a0e increase window size 2021-07-24 01:03:51 +02:00
Niko Storni 64acdc29c3 improve disk cleanup
add index to is_stored
fix test
replace LRU cache
2021-07-24 01:03:51 +02:00
Niko Storni 598773c90d fix mess with lbry.go 2021-07-24 01:03:47 +02:00
Niko Storni 766238fd7e add if this than that store
switch to wasabi for uploads
2021-07-24 01:03:47 +02:00
Niko Storni ac5242f173 add integrity check cmd
throttle live integrity checks
bug fixes
2021-07-24 01:03:47 +02:00
Mark Beamer Jr 215103cb33 use wait group not stopper 2021-07-24 01:03:47 +02:00
Mark Beamer Jr ed3622d0a6 Wait for request to be handled before returning 2021-07-24 01:03:47 +02:00
Mark Beamer Jr 848fce5afa Add request queue for blob cache 2021-07-24 01:03:47 +02:00
Niko Storni e37eeba0c9 check blobs when reading them 2021-07-24 01:03:47 +02:00
Niko Storni 7da49a4ccb upgrade quic-go
add cache for blobs not found
2021-07-24 01:03:47 +02:00