Commit graph

8700 commits

Author SHA1 Message Date
Jack Robison
b3705073a3 skip loading tx/claim caches in the elastic sync script when not needed 2022-03-15 15:34:08 -03:00
Jack Robison
755e8ce101 update trending with help from @eggplantbren 2022-03-15 15:34:08 -03:00
Jack Robison
fd2ab47a16 update sync script to handle ES falling behind leveldb on shutdown 2022-03-15 15:34:08 -03:00
Jack Robison
9aafb7a743 logging 2022-03-15 15:34:08 -03:00
Jack Robison
aa1b20cf7b update docker 2022-03-15 15:34:08 -03:00
Jack Robison
3cd2227c29 update Env to accept parameters from cli args 2022-03-15 15:34:08 -03:00
Jack Robison
27cc83c03b set default CACHE_MB to 1024mb and the default QUERY_TIMEOUT_MS to 10s 2022-03-15 15:34:08 -03:00
Jack Robison
cb6db3f3d8 remove unused hub env settings 2022-03-15 15:34:08 -03:00
Jack Robison
9020e39a83 update lbry-hub-elastic-sync to support resyncing recent blocks 2022-03-15 15:34:08 -03:00
Jack Robison
64509ca95d add CACHE_ALL_CLAIM_TXOS hub setting 2022-03-15 15:34:08 -03:00
Jack Robison
dafd62104b remove dead code 2022-03-15 15:34:08 -03:00
Jack Robison
e09588e433 add CACHE_ALL_TX_HASHES setting to optionally use more memory to save i/o 2022-03-15 15:34:08 -03:00
Jack Robison
50e00192e8 sleeps 2022-03-15 15:34:08 -03:00
Jack Robison
e6d470f110 improve resolve caching 2022-03-15 15:34:08 -03:00
Jack Robison
eeaf9a72e2 threadpools for block processor and es sync reader 2022-03-15 15:34:08 -03:00
Jack Robison
a4ad1bb0a9 doc strings 2022-03-15 15:34:08 -03:00
Jack Robison
e0086682b9 improve claims_producer performance 2022-03-15 15:34:08 -03:00
Jack Robison
15ac2ade59 resolve lru cache 2022-03-15 15:34:08 -03:00
Jack Robison
09e0d5c55e add block_txs index 2022-03-15 15:34:08 -03:00
Jack Robison
dfe855c0c9 smaller caches 2022-03-15 15:34:08 -03:00
Victor Shyba
a2996768fe add optional prometheus to dht_node script 2022-03-15 15:34:08 -03:00
Jack Robison
a53b454d86 Update daemon.py
docstring
2022-03-15 15:34:08 -03:00
Cristian Vicas
01557b599a Updated documentation for RPC calls: status, blob_list. 2022-03-15 15:34:08 -03:00
belikor
65323b4169 file_manager: raise new InvalidStreamURLError if the URL is invalid
When using `lbrynet get URL`, if the URL is not a valid URL
the function `url.URL.parse` will raise a `ValueError` exception
which will produce a whole backtrace.

For example, this is the case if we provide a channel name
with a forward slash but without a stream name.
```
lbrynet get @Non-existing/
```

```
Traceback (most recent call last):
  File "/opt/git/lbry-sdk/lbry/file/file_manager.py", line 84, in download_from_uri
    if not URL.parse(uri).has_stream:
  File "/opt/git/lbry-sdk/lbry/schema/url.py", line 114, in parse
    raise ValueError('Invalid LBRY URL')
ValueError: Invalid LBRY URL
WARNING  lbry.extras.daemon.daemon:1110: Error downloading Non-existing/: Invalid LBRY URL
```

Now we raise a new `InvalidStreamURLError` which can be trapped in the upper functions
that use `url.URL.parse` such as `FileManager.download_from_uri`.
If we do this the traceback won't be shown.
```
WARNING  lbry.file.file_manager:252:
Failed to download Non-existing/: Invalid LBRY stream URL: '@Non-existing/'
WARNING  lbry.extras.daemon.daemon:1110:
Error downloading Non-existing/: Invalid LBRY stream URL: '@Non-existing/'
```

This handles the case when trying to download only "channel" parts
without the claim part.
```
lbrynet get @Non-existing
lbrynet get @Non-existing/
lbrynet get Non-existing/
```
2022-03-15 15:34:08 -03:00
Jack Robison
cdef8b4852 clear es attributes during initial sync 2022-03-15 15:34:08 -03:00
Jack Robison
8d72142390 fix setting references on txos in extra_txos 2022-03-15 15:34:08 -03:00
Jack Robison
172a0f0ac2 fix reposted channel being missing from resolve result
-improve names of the resolve related methods in `LevelDB`
2022-03-15 15:34:08 -03:00
Jack Robison
1022080be6 fix compactify script 2022-03-15 15:34:08 -03:00
Jack Robison
8031a55dbb fix test 2022-03-15 15:34:08 -03:00
Jack Robison
1ddd29656e move test_transaction_commands, test_internal_transaction_api , and test_transactions into their own runner
-move test_resolve_command to its own runner
2022-03-15 15:34:08 -03:00
Jack Robison
cb5c39a159 only save undo info for blocks within reorg limit 2022-03-15 15:34:08 -03:00
Jack Robison
9ec510c742 fix channel count 2022-03-15 15:34:08 -03:00
Jack Robison
aa80cf47b9 try to fix test_sqlite_coin_chooser 2022-03-15 15:34:08 -03:00
Jack Robison
e4e1d42e09 move test_claim_commands and test_resolve_command into new directory 2022-03-15 15:34:08 -03:00
Jack Robison
ccc9a900ef sort touched or deleted claim hashes 2022-03-15 15:34:08 -03:00
Jack Robison
c2a5ff0ae3 block processor db refactoring
-access db through HubDB class, don't use plyvel.DB directly
-add channel count and support amount prefixes
2022-03-15 15:34:08 -03:00
Jack Robison
afa710dcb5 fix and add test for abandoning a controlling in the same block a new claim is made 2022-03-15 15:34:08 -03:00
Jack Robison
d0ed7593de fix test_sqlite_coin_chooser 2022-03-15 15:34:08 -03:00
Jack Robison
830fc7f8cc fix ES index name so it stays the same within a test case 2022-03-15 15:34:08 -03:00
Jack Robison
88e8926a59 fix bug with early takeover by an update 2022-03-15 15:34:08 -03:00
Jack Robison
65e39b8e76 fix test 2022-03-15 15:34:08 -03:00
Jack Robison
acd0c2188f fix activating non existent claim 2022-03-15 15:34:08 -03:00
Jack Robison
6155cda66f add tests for takeovers from amount changes in updates before/on/after activation 2022-03-15 15:34:08 -03:00
Jack Robison
f77f2f6e80 batch address history notifications 2022-03-15 15:34:08 -03:00
Jack Robison
68d397a269 improve leveldb caching 2022-03-15 15:34:08 -03:00
Jack Robison
66ed57c834 refactor reload_blocking_filtering_streams 2022-03-15 15:34:08 -03:00
Jack Robison
8b42475d59 fix filtering error upon abandon 2022-03-15 15:34:08 -03:00
Jack Robison
de9159d83c faster read_claim_txos 2022-03-15 15:34:08 -03:00
Jack Robison
b66adfdf78 remove unused executor 2022-03-15 15:34:08 -03:00
Victor Shyba
a4be5eb1d8 fix blocking and filtering 2022-03-15 15:34:08 -03:00