Niko Storni
|
975bfe7fac
|
upgrade singleflight
http store fix
|
2021-07-24 01:03:51 +02:00 |
|
Niko Storni
|
b075d948bb
|
remove locks causing deadlocks
|
2021-07-24 01:03:51 +02:00 |
|
Niko Storni
|
2651a64dbb
|
add http server/client
|
2021-07-24 01:03:51 +02:00 |
|
Mark Beamer Jr
|
fa7150cf2b
|
Add queue to prevent writing too many files at once.
|
2021-07-24 01:03:51 +02:00 |
|
Mark Beamer Jr
|
6c4db980c9
|
Add queue to prevent writing too many files at once.
|
2021-07-24 01:03:51 +02:00 |
|
Mark Beamer Jr
|
7adaa510fd
|
Add locks to disk store.
|
2021-07-24 01:03:51 +02:00 |
|
Niko Storni
|
64ed7304f6
|
add a lot of extra heavy debugging
|
2021-07-24 01:03:51 +02:00 |
|
Mark Beamer Jr
|
5aefaf061e
|
Add single flight for cache not just origin
|
2021-07-24 01:03:51 +02:00 |
|
Mark Beamer Jr
|
724ee47c8b
|
add metric calls for other packages
|
2021-07-24 01:03:51 +02:00 |
|
Mark Beamer Jr
|
caaec6fcb1
|
add guage metrics for go routines in reflector package
|
2021-07-24 01:03:51 +02:00 |
|
Mark Beamer Jr
|
15984b8fd9
|
add gops to reflector server
|
2021-07-24 01:03:51 +02:00 |
|
Niko Storni
|
2be913b077
|
request queue size param
|
2021-07-24 01:03:51 +02:00 |
|
Niko Storni
|
34c11b0a0e
|
increase window size
|
2021-07-24 01:03:51 +02:00 |
|
Niko Storni
|
64acdc29c3
|
improve disk cleanup
add index to is_stored
fix test
replace LRU cache
|
2021-07-24 01:03:51 +02:00 |
|
Niko Storni
|
598773c90d
|
fix mess with lbry.go
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
766238fd7e
|
add if this than that store
switch to wasabi for uploads
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
ac5242f173
|
add integrity check cmd
throttle live integrity checks
bug fixes
|
2021-07-24 01:03:47 +02:00 |
|
Mark Beamer Jr
|
215103cb33
|
use wait group not stopper
|
2021-07-24 01:03:47 +02:00 |
|
Mark Beamer Jr
|
ed3622d0a6
|
Wait for request to be handled before returning
|
2021-07-24 01:03:47 +02:00 |
|
Mark Beamer Jr
|
848fce5afa
|
Add request queue for blob cache
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
e37eeba0c9
|
check blobs when reading them
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
7da49a4ccb
|
upgrade quic-go
add cache for blobs not found
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
7b02ace5e2
|
fix issues caused by beamer's renaming
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
5fb67b32db
|
run go mod tidy
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
a0c9ed2ace
|
make it simpler
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
998b082a06
|
remove panics
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
36d4156e2a
|
add tracing to blobs
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
74925ebba2
|
optimize batch insertions
reduce touch time to every 6 hours
|
2021-07-24 01:03:47 +02:00 |
|
Alex Grintsvayg
|
6f95b3395f
|
avoid heavy interpolateparams call
|
2021-07-24 01:03:47 +02:00 |
|
Alex Grintsvayg
|
dff00e2317
|
fix long query
|
2021-07-24 01:03:47 +02:00 |
|
Alex Grintsvayg
|
9a5d9d7ff5
|
only touch blobs when you get them
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
5794c57898
|
save uploaded blobs and work around the blocklist issue
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
35c713a26e
|
add cmd to populate db
fix store init
try fixing unreasonable db bottleneck
|
2021-07-24 01:03:47 +02:00 |
|
Alex Grintsvayg
|
6fb0620091
|
something like this
|
2021-07-24 01:03:38 +02:00 |
|
Niko Storni
|
03df751bc7
|
add PoC for litedb to avoid all the overhead
|
2021-07-24 01:03:16 +02:00 |
|
Niko Storni
|
c902858958
|
address some review comments
|
2021-07-24 01:03:16 +02:00 |
|
Niko Storni
|
84fabdd5f4
|
add option to run with RO-CF only as upstream
increase idle timeout to avoid errors downstream
add option to delete blobs from DB if storage doesn't have it (for future local tracking)
|
2021-07-24 01:03:16 +02:00 |
|
Niko Storni
|
f5cad15f84
|
upgrade quic
|
2021-07-24 01:03:10 +02:00 |
|
Niko Storni
|
dd3d0ae42c
|
update lfuda library
|
2021-07-24 01:03:10 +02:00 |
|
Niko Storni
|
0b565852b8
|
only store the blobs in the underlying storage if LFUDA accepted them
|
2021-07-24 01:03:10 +02:00 |
|
Niko Storni
|
ff13d7b2f7
|
fix cache size mess
|
2021-07-24 01:03:10 +02:00 |
|
Niko Storni
|
7f5a89fa5a
|
fix buffer cache running out of space
|
2021-07-24 01:03:10 +02:00 |
|
Niko Storni
|
704e15f8c1
|
use LFUDA store
swap size to bytes
|
2021-07-24 01:03:10 +02:00 |
|
Niko Storni
|
5eb1f13b54
|
add LFUDA store
update quic
fix tests
|
2021-07-24 01:03:10 +02:00 |
|
Alex Grintsvayg
|
176e05714e
|
rename cahces
|
2021-07-24 01:03:10 +02:00 |
|
Niko Storni
|
eefd84b02d
|
add buffer cache for nvme drive
|
2021-07-24 01:03:10 +02:00 |
|
Niko Storni
|
af2742c34f
|
update quic
don't wait for a blob to be written to disk before sending it downstream
don't wait for the disk store to be walked before starting everything up
|
2021-07-24 01:03:10 +02:00 |
|
Alex Grintsvayg
|
2cf4acdb59
|
add 'send' command to efficiently send a file to reflector
|
2021-04-02 14:30:36 -04:00 |
|
Alex Grintsvayg
|
044e2fe5d7
|
Merge branch 'fix_mem_leak'
* fix_mem_leak:
dont over-allocate ram when reading blobs via QUIC
dont overallocate ram when reading blobs from disk
|
2020-11-27 16:18:44 -05:00 |
|
Alex Grintsvayg
|
fb77bf621e
|
dont over-allocate ram when reading blobs via QUIC
|
2020-11-27 16:18:26 -05:00 |
|