Commit graph

8744 commits

Author SHA1 Message Date
Victor Shyba
48bb84fc1e make sure the downloader always stops gracefully 2022-03-15 15:34:09 -03:00
Victor Shyba
b055c25156 cache space stats from running components so status is instant 2022-03-15 15:34:09 -03:00
Victor Shyba
35905b99ff add index for blob table so size summaries are faster 2022-03-15 15:34:09 -03:00
Victor Shyba
806857c066 download from stored announcements and dont reannounce 2022-03-15 15:34:09 -03:00
Victor Shyba
ba8a7fc351 improve disk space manager status, include more info and unify space queries 2022-03-15 15:34:09 -03:00
Victor Shyba
a36fd76eb1 normal_blobs->stream_blobs, proactive->background 2022-03-15 15:34:09 -03:00
Victor Shyba
02994164be fix free space calculation, test it and give a margin of 10mb before starting so it doesnt insist when full 2022-03-15 15:34:09 -03:00
Victor Shyba
c3b89a896b move more logic out of the downloader component 2022-03-15 15:34:09 -03:00
Victor Shyba
b0bf6eff16 extract background downloader to its own class 2022-03-15 15:34:09 -03:00
Victor Shyba
47c4cb800d add analytics event for network disk space 2022-03-15 15:34:09 -03:00
Victor Shyba
c5f6a4ca1c announce orphan blobs manually, as that was done when save stream 2022-03-15 15:34:09 -03:00
Victor Shyba
813108b9d8 cleanup background downloader blobs from conf 2022-03-15 15:34:09 -03:00
Victor Shyba
95703b3af5 separated network seeding space metrics 2022-03-15 15:34:09 -03:00
Victor Shyba
2ceec7c3d4 don't save streams for network blobs and bypass disk space manager 2022-03-15 15:34:09 -03:00
Victor Shyba
3985f1578e add conf for network seeding space limit 2022-03-15 15:34:09 -03:00
Victor Shyba
355ccb1e32 schedule the download task instead 2022-03-15 15:34:09 -03:00
Victor Shyba
04d2c6f013 fix unit tests from component dependency chain changes 2022-03-15 15:34:09 -03:00
Victor Shyba
8d6a7101f6 download from DHT 2022-03-15 15:34:09 -03:00
Victor Shyba
abbd8473bb drop channel support, prepare to hook into DHT 2022-03-15 15:34:09 -03:00
Victor Shyba
02b2103d94 handle case where something that isn't a sd blob gets hit 2022-03-15 15:34:09 -03:00
Victor Shyba
85474ae381 no api yet 2022-03-15 15:34:09 -03:00
Victor Shyba
9d3c401abb download only blobs 2022-03-15 15:34:09 -03:00
Victor Shyba
32a7d1a4a0 fix tests 2022-03-15 15:34:09 -03:00
Victor Shyba
602cd5bd2e fix exception arguments 2022-03-15 15:34:09 -03:00
Victor Shyba
4b88b191e7 test add/remove/list subscriptions 2022-03-15 15:34:09 -03:00
Victor Shyba
e449cd01ba fix and test main api 2022-03-15 15:34:08 -03:00
Victor Shyba
b1db42acf4 download all blobs and check that on tests 2022-03-15 15:34:08 -03:00
Victor Shyba
fcbe8cf00b create downloader component and initial tests 2022-03-15 15:34:08 -03:00
Victor Shyba
a1866c40f5 with the fix we no longer need to restart the stream 2022-03-15 15:34:08 -03:00
Victor Shyba
d78540f3cc fix tests by checking there are actual blobs being deleted 2022-03-15 15:34:08 -03:00
Victor Shyba
7c1c04674c dont lose results on duplicates, just warn 2022-03-15 15:34:08 -03:00
Victor Shyba
694aeab75c remove tried_for_this_blob so banned peers are retried for same blob 2022-03-15 15:34:08 -03:00
Cristian Vicas
90aaf64b77 Bug [#2070] where blob_get RPC timed out.
Both stream.downloader and blob_exchange.downloader paths are adding the fixed_peers list to the DHT node.
Tested jsonrpc_blob_get daemon call.

Bug [#2070] where blob_get RPC timed out.

Both stream.downloader and blob_exchange.downloader paths are adding the fixed_peers list to the DHT node.
Tested jsonrpc_blob_get daemon call.
2022-03-15 15:34:08 -03:00
Jack Robison
6eeabb1a1a use mempool cache in transaction_get_batch 2022-03-15 15:34:08 -03:00
FemtosecondLaser
77d58b82a0 Added an integration test covering the following scenario:
On start, if download dir is non-writable - daemon terminates with a helpful message.
2022-03-15 15:34:08 -03:00
FemtosecondLaser
a7af877e9e Improved the readability of the tests. 2022-03-15 15:34:08 -03:00
FemtosecondLaser
25092f56be Changed the tests to execute against a real file system instead of a fake one. 2022-03-15 15:34:08 -03:00
FemtosecondLaser
e55f9dd21e renamed the test class to be more specific about the sut 2022-03-15 15:34:08 -03:00
FemtosecondLaser
6c10509705 removed redundant tests
renamed a test to be more specific about the kind of the precondition
2022-03-15 15:34:08 -03:00
FemtosecondLaser
bed2e253b2 Modified ensure_directory_exists() to check if the directory is writable by the process. 2022-03-15 15:34:08 -03:00
Jack Robison
c683ae9ed6 Update docker-compose-wallet-server.yml 2022-03-15 15:34:08 -03:00
Jack Robison
8ce1cebd07 add script to setup docker volumes from snapshots 2022-03-15 15:34:08 -03:00
Jack Robison
9029b4b5c3 fix attempting to update trending on abandoned claims 2022-03-15 15:34:08 -03:00
Jack Robison
db0831d55b update docs 2022-03-15 15:34:08 -03:00
Jack Robison
b3705073a3 skip loading tx/claim caches in the elastic sync script when not needed 2022-03-15 15:34:08 -03:00
Jack Robison
755e8ce101 update trending with help from @eggplantbren 2022-03-15 15:34:08 -03:00
Jack Robison
fd2ab47a16 update sync script to handle ES falling behind leveldb on shutdown 2022-03-15 15:34:08 -03:00
Jack Robison
9aafb7a743 logging 2022-03-15 15:34:08 -03:00
Jack Robison
aa1b20cf7b update docker 2022-03-15 15:34:08 -03:00
Jack Robison
3cd2227c29 update Env to accept parameters from cli args 2022-03-15 15:34:08 -03:00