Commit graph

1829 commits

Author SHA1 Message Date
Victor Shyba
02b2103d94 handle case where something that isn't a sd blob gets hit 2022-03-15 15:34:09 -03:00
Victor Shyba
85474ae381 no api yet 2022-03-15 15:34:09 -03:00
Victor Shyba
9d3c401abb download only blobs 2022-03-15 15:34:09 -03:00
Victor Shyba
602cd5bd2e fix exception arguments 2022-03-15 15:34:09 -03:00
Victor Shyba
4b88b191e7 test add/remove/list subscriptions 2022-03-15 15:34:09 -03:00
Victor Shyba
e449cd01ba fix and test main api 2022-03-15 15:34:08 -03:00
Victor Shyba
b1db42acf4 download all blobs and check that on tests 2022-03-15 15:34:08 -03:00
Victor Shyba
fcbe8cf00b create downloader component and initial tests 2022-03-15 15:34:08 -03:00
Victor Shyba
7c1c04674c dont lose results on duplicates, just warn 2022-03-15 15:34:08 -03:00
Victor Shyba
694aeab75c remove tried_for_this_blob so banned peers are retried for same blob 2022-03-15 15:34:08 -03:00
Cristian Vicas
90aaf64b77 Bug [#2070] where blob_get RPC timed out.
Both stream.downloader and blob_exchange.downloader paths are adding the fixed_peers list to the DHT node.
Tested jsonrpc_blob_get daemon call.

Bug [#2070] where blob_get RPC timed out.

Both stream.downloader and blob_exchange.downloader paths are adding the fixed_peers list to the DHT node.
Tested jsonrpc_blob_get daemon call.
2022-03-15 15:34:08 -03:00
Jack Robison
6eeabb1a1a use mempool cache in transaction_get_batch 2022-03-15 15:34:08 -03:00
FemtosecondLaser
bed2e253b2 Modified ensure_directory_exists() to check if the directory is writable by the process. 2022-03-15 15:34:08 -03:00
Jack Robison
9029b4b5c3 fix attempting to update trending on abandoned claims 2022-03-15 15:34:08 -03:00
Jack Robison
b3705073a3 skip loading tx/claim caches in the elastic sync script when not needed 2022-03-15 15:34:08 -03:00
Jack Robison
755e8ce101 update trending with help from @eggplantbren 2022-03-15 15:34:08 -03:00
Jack Robison
fd2ab47a16 update sync script to handle ES falling behind leveldb on shutdown 2022-03-15 15:34:08 -03:00
Jack Robison
9aafb7a743 logging 2022-03-15 15:34:08 -03:00
Jack Robison
3cd2227c29 update Env to accept parameters from cli args 2022-03-15 15:34:08 -03:00
Jack Robison
27cc83c03b set default CACHE_MB to 1024mb and the default QUERY_TIMEOUT_MS to 10s 2022-03-15 15:34:08 -03:00
Jack Robison
cb6db3f3d8 remove unused hub env settings 2022-03-15 15:34:08 -03:00
Jack Robison
9020e39a83 update lbry-hub-elastic-sync to support resyncing recent blocks 2022-03-15 15:34:08 -03:00
Jack Robison
64509ca95d add CACHE_ALL_CLAIM_TXOS hub setting 2022-03-15 15:34:08 -03:00
Jack Robison
dafd62104b remove dead code 2022-03-15 15:34:08 -03:00
Jack Robison
e09588e433 add CACHE_ALL_TX_HASHES setting to optionally use more memory to save i/o 2022-03-15 15:34:08 -03:00
Jack Robison
50e00192e8 sleeps 2022-03-15 15:34:08 -03:00
Jack Robison
e6d470f110 improve resolve caching 2022-03-15 15:34:08 -03:00
Jack Robison
eeaf9a72e2 threadpools for block processor and es sync reader 2022-03-15 15:34:08 -03:00
Jack Robison
a4ad1bb0a9 doc strings 2022-03-15 15:34:08 -03:00
Jack Robison
e0086682b9 improve claims_producer performance 2022-03-15 15:34:08 -03:00
Jack Robison
15ac2ade59 resolve lru cache 2022-03-15 15:34:08 -03:00
Jack Robison
09e0d5c55e add block_txs index 2022-03-15 15:34:08 -03:00
Jack Robison
dfe855c0c9 smaller caches 2022-03-15 15:34:08 -03:00
Jack Robison
a53b454d86 Update daemon.py
docstring
2022-03-15 15:34:08 -03:00
Cristian Vicas
01557b599a Updated documentation for RPC calls: status, blob_list. 2022-03-15 15:34:08 -03:00
belikor
65323b4169 file_manager: raise new InvalidStreamURLError if the URL is invalid
When using `lbrynet get URL`, if the URL is not a valid URL
the function `url.URL.parse` will raise a `ValueError` exception
which will produce a whole backtrace.

For example, this is the case if we provide a channel name
with a forward slash but without a stream name.
```
lbrynet get @Non-existing/
```

```
Traceback (most recent call last):
  File "/opt/git/lbry-sdk/lbry/file/file_manager.py", line 84, in download_from_uri
    if not URL.parse(uri).has_stream:
  File "/opt/git/lbry-sdk/lbry/schema/url.py", line 114, in parse
    raise ValueError('Invalid LBRY URL')
ValueError: Invalid LBRY URL
WARNING  lbry.extras.daemon.daemon:1110: Error downloading Non-existing/: Invalid LBRY URL
```

Now we raise a new `InvalidStreamURLError` which can be trapped in the upper functions
that use `url.URL.parse` such as `FileManager.download_from_uri`.
If we do this the traceback won't be shown.
```
WARNING  lbry.file.file_manager:252:
Failed to download Non-existing/: Invalid LBRY stream URL: '@Non-existing/'
WARNING  lbry.extras.daemon.daemon:1110:
Error downloading Non-existing/: Invalid LBRY stream URL: '@Non-existing/'
```

This handles the case when trying to download only "channel" parts
without the claim part.
```
lbrynet get @Non-existing
lbrynet get @Non-existing/
lbrynet get Non-existing/
```
2022-03-15 15:34:08 -03:00
Jack Robison
cdef8b4852 clear es attributes during initial sync 2022-03-15 15:34:08 -03:00
Jack Robison
8d72142390 fix setting references on txos in extra_txos 2022-03-15 15:34:08 -03:00
Jack Robison
172a0f0ac2 fix reposted channel being missing from resolve result
-improve names of the resolve related methods in `LevelDB`
2022-03-15 15:34:08 -03:00
Jack Robison
1022080be6 fix compactify script 2022-03-15 15:34:08 -03:00
Jack Robison
8031a55dbb fix test 2022-03-15 15:34:08 -03:00
Jack Robison
cb5c39a159 only save undo info for blocks within reorg limit 2022-03-15 15:34:08 -03:00
Jack Robison
9ec510c742 fix channel count 2022-03-15 15:34:08 -03:00
Jack Robison
ccc9a900ef sort touched or deleted claim hashes 2022-03-15 15:34:08 -03:00
Jack Robison
c2a5ff0ae3 block processor db refactoring
-access db through HubDB class, don't use plyvel.DB directly
-add channel count and support amount prefixes
2022-03-15 15:34:08 -03:00
Jack Robison
afa710dcb5 fix and add test for abandoning a controlling in the same block a new claim is made 2022-03-15 15:34:08 -03:00
Jack Robison
830fc7f8cc fix ES index name so it stays the same within a test case 2022-03-15 15:34:08 -03:00
Jack Robison
88e8926a59 fix bug with early takeover by an update 2022-03-15 15:34:08 -03:00
Jack Robison
acd0c2188f fix activating non existent claim 2022-03-15 15:34:08 -03:00
Jack Robison
6155cda66f add tests for takeovers from amount changes in updates before/on/after activation 2022-03-15 15:34:08 -03:00