Commit graph

138 commits

Author SHA1 Message Date
Niko Storni
c4084eeb68 improve disk cleanup
add index to is_stored
fix test
replace LRU cache
2021-04-29 03:41:18 +02:00
Niko Storni
4392c97242 fix mess with lbry.go 2021-04-13 00:52:56 +02:00
Niko Storni
ec3aae33ba add if this than that store
switch to wasabi for uploads
2021-04-12 23:05:50 +02:00
Niko Storni
dc95351cf3 add integrity check cmd
throttle live integrity checks
bug fixes
2021-04-07 04:46:18 +02:00
Niko Storni
3a441aed3a fix issues caused by beamer's renaming 2021-03-29 19:56:18 +02:00
Niko Storni
6291e33ee1 add tracing to blobs 2021-01-14 20:38:04 +01:00
Alex Grintsvayg
c4504631bc
avoid heavy interpolateparams call 2021-01-06 10:43:35 -05:00
Niko Storni
b33651ae26 save uploaded blobs and work around the blocklist issue 2021-01-05 05:09:55 +01:00
Niko Storni
0d5004a83b add cmd to populate db
fix store init
try fixing unreasonable db bottleneck
2020-12-30 04:24:11 +01:00
Niko Storni
04f6859c74 Merge branch 'grin' into litedb 2020-12-24 23:13:31 +01:00
Alex Grintsvayg
3a1d9d3304
something like this 2020-12-23 17:08:13 -05:00
Niko Storni
03304312e8 add PoC for litedb to avoid all the overhead 2020-12-23 06:04:42 +01:00
Niko Storni
def551cc89 add option to run with RO-CF only as upstream
increase idle timeout to avoid errors downstream
add option to delete blobs from DB if storage doesn't have it (for future local tracking)
2020-12-22 20:53:48 +01:00
Niko Storni
ff9b61b034
fix cache size mess 2020-11-27 16:20:49 -05:00
Niko Storni
7b80b2d4d2
fix buffer cache running out of space 2020-11-27 16:20:49 -05:00
Niko Storni
d45abdbdb0
use LFUDA store
swap size to bytes
2020-11-27 16:19:46 -05:00
Alex Grintsvayg
bb41a84bb7
rename cahces 2020-11-27 16:19:45 -05:00
Niko Storni
a574fecf4e
add buffer cache for nvme drive 2020-11-27 16:19:45 -05:00
Alex Grintsvayg
560e180e36
separate singleflight cache wrapper, component names for cache metrics 2020-11-04 16:03:46 -05:00
Alex Grintsvayg
124d4065c2
split cloudfront into RO and RW stores 2020-11-04 16:03:45 -05:00
Alex Grintsvayg
3608971f0b
add lru cache eviction metric 2020-11-04 16:03:44 -05:00
Alex Grintsvayg
c9fa04043c
rename the stores, add caching to reflector cmd 2020-11-04 16:03:44 -05:00
Alex Grintsvayg
69fa06420b
LRU cache for disk store, abstract fs in disk store for testing 2020-11-04 16:03:43 -05:00
Niko Storni
f043516a14 revert upload changes 2020-10-15 03:08:27 +02:00
Niko Storni
f1875454cc store uploaded blobs 2020-10-15 00:12:18 +02:00
Alex Grintsvayg
8364d3fc54
small changes for easier testing 2020-10-05 18:08:54 -04:00
Alex Grintsvayg
de0ccd4da7
track approximate access time for blobs 2020-10-05 18:08:53 -04:00
Alex Grintsvayg
5658fe4607
add publish command 2020-10-05 16:30:57 -04:00
Niko Storni
fc5f5ff7d3 add cloudfront support 2020-09-09 00:18:07 +02:00
Niko Storni
e8b98bc862 update dependencies 2020-09-01 20:15:34 +02:00
Niko Storni
09c7718f30 refactor code 2020-07-09 15:02:32 +02:00
Niko Storni
8a5f57b14f remove QUIC protocol
add HTTP3/QUIC protocol
2020-07-09 15:02:32 +02:00
Niko Storni
90997b9918 use new connections for each blob 2020-07-09 15:02:32 +02:00
Niko Storni
e0da2674a1 use new connections for each action
reduce timeout
2020-07-09 15:02:32 +02:00
Niko Storni
a80599413c improve params description 2020-07-09 15:02:32 +02:00
Niko Storni
3ffe7a10c7 add other reflector store
add flags
improve disk cleanup
2020-07-09 15:02:32 +02:00
Niko Storni
fb0004bac4 increase packet size
add retro-compatibility
remove unused SPV servers
remove travis test failure
2020-07-09 15:02:32 +02:00
Alex Grintsvayg
be69c2f05c
add flag to delete blobs after upload 2020-05-02 14:31:10 -04:00
Alex Grintsvayg
dde93a1fe6
make cache dir a cli flag 2020-02-27 14:53:33 -05:00
Niko Storni
4a5a148843
implement disk cleanup 2020-02-27 14:53:33 -05:00
Alex Grintsvayg
d291c063ec
add traces to unmarshall errors 2020-02-25 15:49:51 -05:00
Alex Grintsvayg
50089481fb
switch to prometheus for metrics 2020-01-02 13:12:33 -05:00
Alex Grintsvayg
dc6dd8d12b
fix blocklist, log future blocklist errors, add resolve wallet server method 2019-12-28 19:17:52 -05:00
Andrey Beletsky
834733b675 Add options for peer.NewStore to allow for setting TCP timeout 2019-11-22 18:56:20 +07:00
Mark Beamer Jr
86a553b876
upgrade to lbry.go v2.4.0 2019-11-13 19:11:35 -05:00
Alex Grintsvayg
661c20a21d
make db-backed store more generic (not specific to s3) 2019-10-03 16:58:17 -04:00
Alex Grintsvayg
69f1e0f4ca
make MemoryStore consistent with the New...() pattern 2019-10-03 16:49:28 -04:00
Alex Grintsvayg
2ca83139df
use stream.Blob for BlobStore interface 2019-10-03 16:34:57 -04:00
Alex Grintsvayg
0af6d65d40
rename FileStore -> DiskStore 2019-10-03 16:24:59 -04:00
Alex Grintsvayg
a8230db802
rewrite getstream command using caching store 2019-10-03 16:13:08 -04:00