Niko Storni
|
63aacd8a69
|
use O_DIRECT to write to disk (fixes everything)
add queue back to serving blobs
improve a lot of things
upgrade modules
|
2021-07-24 01:03:51 +02:00 |
|
Niko Storni
|
2651a64dbb
|
add http server/client
|
2021-07-24 01:03:51 +02:00 |
|
Niko Storni
|
2be913b077
|
request queue size param
|
2021-07-24 01:03:51 +02:00 |
|
Niko Storni
|
64acdc29c3
|
improve disk cleanup
add index to is_stored
fix test
replace LRU cache
|
2021-07-24 01:03:51 +02:00 |
|
Niko Storni
|
766238fd7e
|
add if this than that store
switch to wasabi for uploads
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
ac5242f173
|
add integrity check cmd
throttle live integrity checks
bug fixes
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
36d4156e2a
|
add tracing to blobs
|
2021-07-24 01:03:47 +02:00 |
|
Alex Grintsvayg
|
6f95b3395f
|
avoid heavy interpolateparams call
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
5794c57898
|
save uploaded blobs and work around the blocklist issue
|
2021-07-24 01:03:47 +02:00 |
|
Niko Storni
|
35c713a26e
|
add cmd to populate db
fix store init
try fixing unreasonable db bottleneck
|
2021-07-24 01:03:47 +02:00 |
|
Alex Grintsvayg
|
6fb0620091
|
something like this
|
2021-07-24 01:03:38 +02:00 |
|
Niko Storni
|
03df751bc7
|
add PoC for litedb to avoid all the overhead
|
2021-07-24 01:03:16 +02:00 |
|
Niko Storni
|
84fabdd5f4
|
add option to run with RO-CF only as upstream
increase idle timeout to avoid errors downstream
add option to delete blobs from DB if storage doesn't have it (for future local tracking)
|
2021-07-24 01:03:16 +02:00 |
|
Niko Storni
|
ff13d7b2f7
|
fix cache size mess
|
2021-07-24 01:03:10 +02:00 |
|
Niko Storni
|
7f5a89fa5a
|
fix buffer cache running out of space
|
2021-07-24 01:03:10 +02:00 |
|
Niko Storni
|
704e15f8c1
|
use LFUDA store
swap size to bytes
|
2021-07-24 01:03:10 +02:00 |
|
Alex Grintsvayg
|
176e05714e
|
rename cahces
|
2021-07-24 01:03:10 +02:00 |
|
Niko Storni
|
eefd84b02d
|
add buffer cache for nvme drive
|
2021-07-24 01:03:10 +02:00 |
|
Alex Grintsvayg
|
2cf4acdb59
|
add 'send' command to efficiently send a file to reflector
|
2021-04-02 14:30:36 -04:00 |
|
Alex Grintsvayg
|
560e180e36
|
separate singleflight cache wrapper, component names for cache metrics
|
2020-11-04 16:03:46 -05:00 |
|
Alex Grintsvayg
|
124d4065c2
|
split cloudfront into RO and RW stores
|
2020-11-04 16:03:45 -05:00 |
|
Alex Grintsvayg
|
3608971f0b
|
add lru cache eviction metric
|
2020-11-04 16:03:44 -05:00 |
|
Alex Grintsvayg
|
c9fa04043c
|
rename the stores, add caching to reflector cmd
|
2020-11-04 16:03:44 -05:00 |
|
Alex Grintsvayg
|
69fa06420b
|
LRU cache for disk store, abstract fs in disk store for testing
|
2020-11-04 16:03:43 -05:00 |
|
Niko Storni
|
f043516a14
|
revert upload changes
|
2020-10-15 03:08:27 +02:00 |
|
Niko Storni
|
f1875454cc
|
store uploaded blobs
|
2020-10-15 00:12:18 +02:00 |
|
Alex Grintsvayg
|
8364d3fc54
|
small changes for easier testing
|
2020-10-05 18:08:54 -04:00 |
|
Alex Grintsvayg
|
de0ccd4da7
|
track approximate access time for blobs
|
2020-10-05 18:08:53 -04:00 |
|
Alex Grintsvayg
|
5658fe4607
|
add publish command
|
2020-10-05 16:30:57 -04:00 |
|
Niko Storni
|
fc5f5ff7d3
|
add cloudfront support
|
2020-09-09 00:18:07 +02:00 |
|
Niko Storni
|
e8b98bc862
|
update dependencies
|
2020-09-01 20:15:34 +02:00 |
|
Niko Storni
|
09c7718f30
|
refactor code
|
2020-07-09 15:02:32 +02:00 |
|
Niko Storni
|
8a5f57b14f
|
remove QUIC protocol
add HTTP3/QUIC protocol
|
2020-07-09 15:02:32 +02:00 |
|
Niko Storni
|
90997b9918
|
use new connections for each blob
|
2020-07-09 15:02:32 +02:00 |
|
Niko Storni
|
e0da2674a1
|
use new connections for each action
reduce timeout
|
2020-07-09 15:02:32 +02:00 |
|
Niko Storni
|
a80599413c
|
improve params description
|
2020-07-09 15:02:32 +02:00 |
|
Niko Storni
|
3ffe7a10c7
|
add other reflector store
add flags
improve disk cleanup
|
2020-07-09 15:02:32 +02:00 |
|
Niko Storni
|
fb0004bac4
|
increase packet size
add retro-compatibility
remove unused SPV servers
remove travis test failure
|
2020-07-09 15:02:32 +02:00 |
|
Alex Grintsvayg
|
be69c2f05c
|
add flag to delete blobs after upload
|
2020-05-02 14:31:10 -04:00 |
|
Alex Grintsvayg
|
dde93a1fe6
|
make cache dir a cli flag
|
2020-02-27 14:53:33 -05:00 |
|
Niko Storni
|
4a5a148843
|
implement disk cleanup
|
2020-02-27 14:53:33 -05:00 |
|
Alex Grintsvayg
|
d291c063ec
|
add traces to unmarshall errors
|
2020-02-25 15:49:51 -05:00 |
|
Alex Grintsvayg
|
50089481fb
|
switch to prometheus for metrics
|
2020-01-02 13:12:33 -05:00 |
|
Alex Grintsvayg
|
dc6dd8d12b
|
fix blocklist, log future blocklist errors, add resolve wallet server method
|
2019-12-28 19:17:52 -05:00 |
|
Andrey Beletsky
|
834733b675
|
Add options for peer.NewStore to allow for setting TCP timeout
|
2019-11-22 18:56:20 +07:00 |
|
Mark Beamer Jr
|
86a553b876
|
upgrade to lbry.go v2.4.0
|
2019-11-13 19:11:35 -05:00 |
|
Alex Grintsvayg
|
661c20a21d
|
make db-backed store more generic (not specific to s3)
|
2019-10-03 16:58:17 -04:00 |
|
Alex Grintsvayg
|
69f1e0f4ca
|
make MemoryStore consistent with the New...() pattern
|
2019-10-03 16:49:28 -04:00 |
|
Alex Grintsvayg
|
2ca83139df
|
use stream.Blob for BlobStore interface
|
2019-10-03 16:34:57 -04:00 |
|
Alex Grintsvayg
|
0af6d65d40
|
rename FileStore -> DiskStore
|
2019-10-03 16:24:59 -04:00 |
|