LBRYCommunity/lbry-sdk
1
0
Fork 1
Code Issues Pull requests Projects Releases 282 Packages Wiki Activity
8890 commits 110 branches 641 tags 31 MiB
Commit graph

1942 commits

Author SHA1 Message Date
Victor Shyba
f89c75e642 bump hub version to latest supporting sd_hash search 2022-01-05 17:10:31 -03:00
Victor Shyba
d2c1961101 update hub protobuf including sd_hash field 2022-01-05 17:10:31 -03:00
Victor Shyba
2a4c5a48bf increase indexed sd_hash prefix to 4 chars 2022-01-05 17:10:31 -03:00
Victor Shyba
5f5f39a4aa enable and test prefix search for sd hash 2022-01-05 17:10:31 -03:00
Victor Shyba
df54cc04af sync and search sd_hash 2022-01-05 17:10:31 -03:00
Victor Shyba
19fa274227 add sd hash to API 2022-01-05 17:10:31 -03:00
Lex Berezhny
4822792ee2 create nondetermnistic channel in test to replicate old test behavior 2021-12-22 18:29:46 -05:00
Lex Berezhny
23c10faff5 lint 2021-12-22 18:29:46 -05:00
Lex Berezhny
1eaa195363 reduced crypto dependency in wallet to coincurve 2021-12-22 18:29:46 -05:00
Lex Berezhny
fb57cfa5d8 moved imports for lint 2021-12-22 18:29:46 -05:00
Lex Berezhny
d815a6f02c use ecdsa for signing/veryfing instead of coincurve due to compatibility issues 2021-12-22 18:29:46 -05:00
Lex Berezhny
8216f4a873 work in progress 2021-12-22 18:29:46 -05:00
Lex Berezhny
e4cc4521d9 channel key generation no longer arbitrarily bounded 2021-12-22 18:29:46 -05:00
Lex Berezhny
6bd9b3744d progress, channel keys generate deterministically now 2021-12-22 18:29:46 -05:00
Lex Berezhny
f741b00768 progress on deterministic channel keys 2021-12-22 18:29:46 -05:00
Lex Berezhny
e5268f43e7 changes default coin selection strategy from standard to prefer_confirmed 2021-12-21 10:22:09 -05:00
Victor Shyba
54d6fb9da4 do not limit DHT results by K, respect max_results 2021-12-09 14:34:55 -03:00
Victor Shyba
3d5c9cc1c2 clarify DHT debug logging on key and operation 2021-12-09 14:32:30 -03:00
Jeffrey Picard
d66f46e07b Switch RangeField back to ints 2021-12-03 18:12:38 -05:00
Jeffrey Picard
3436965b33 Debugging 2021-12-03 17:22:52 -05:00
Jeffrey Picard
1b322dc404
Update protobufs, go hub shim, and claim test. 2021-12-03 13:03:00 -05:00
Jack Robison
58341f4ff1
remove unused ES fields 2021-12-03 13:03:00 -05:00
Jack Robison
0d3ca80008
support lists of constraints for all range fields 2021-12-03 13:03:00 -05:00
Lex Berezhny
63437712cd
Merge pull request #3490 from ghost/integration_test_setup_cleanup_timeouts 
added timeout of async operations to integration test setup/teardown
 2021-12-02 19:52:44 -05:00
Jack Robison
26d0e87f46 v0.106.0 2021-12-02 17:17:00 -05:00
Jack Robison
7bb293e5d6 update claim_search doc 
backward compatibility for `trending_mixed`, `trending_local`, `trending_global`, and `trending_group` args to `claim_search`
 2021-12-02 14:51:52 -05:00
Lex Berezhny
e4777f9314
Merge branch 'master' into integration_test_setup_cleanup_timeouts 2021-12-01 22:08:18 -05:00
Jack Robison
1aa66c6038
update header checkpoints 2021-12-01 18:46:24 -05:00
Victor Shyba
e7458edb72 test case for stream_type search on claims missing source + fix 2021-12-01 18:42:47 -05:00
Lex Berezhny
9e43060d41 fee per name env var 2021-12-01 10:22:34 -05:00
FemtosecondLaser
d69486fb6e returned conditional check in add_timeout() as it was making test_node.py tests unhappy 2021-11-30 01:01:35 +00:00
FemtosecondLaser
d4ebfdbc3c removed conditional check in add_timeout() 2021-11-29 22:56:50 +00:00
FemtosecondLaser
e00c3db71a
Merge branch 'master' into integration_test_setup_cleanup_timeouts 2021-11-29 21:50:05 +00:00
Jack Robison
7531401623
keep touched_or_deleted records 2021-11-21 13:52:03 -05:00
FemtosecondLaser
e6c1dc251e changed addTimeout to add_timeout for lint compliance 2021-11-20 00:47:46 +00:00
FemtosecondLaser
dca7977051 added timeout of async operations to integration test setup/teardown 2021-11-20 00:22:25 +00:00
Brendon J. Brewer
3f6fe995b8 Rename trending 2021-11-16 10:59:10 -05:00
Jack Robison
1e00fb369d fix missing es notification for support amount changing 2021-11-15 00:58:18 -05:00
Jack Robison
54b522383a improve tests 2021-11-15 00:58:18 -05:00
Jack Robison
3fe1582432 fix duplicate trending notification to ES 2021-11-15 00:58:18 -05:00
Jack Robison
85eddd2100 fix effective amount for resolve/ES being off while claims/supports are unactivated 2021-11-15 00:58:18 -05:00
Jack Robison
0ca98678f7 update default tcp/blob port to be the same as the default udp/dht port (4444) 2021-11-10 13:02:28 -03:00
Victor Shyba
a19060c08d log unexpected errors, rename task/loop 2021-11-09 14:27:06 -05:00
Victor Shyba
fa2ad88cc4 clear cache on test assertions 2021-11-09 14:27:06 -05:00
Victor Shyba
63cbcd0956 make sure the downloader always stops gracefully 2021-11-09 14:27:06 -05:00
Victor Shyba
d6d0ebf8f4 cache space stats from running components so status is instant 2021-11-09 14:27:06 -05:00
Victor Shyba
0d810d92ca add index for blob table so size summaries are faster 2021-11-09 14:27:06 -05:00
Victor Shyba
1ff914a6f4 download from stored announcements and dont reannounce 2021-11-09 14:27:06 -05:00
Victor Shyba
5959b1be72 improve disk space manager status, include more info and unify space queries 2021-11-09 14:27:06 -05:00
Victor Shyba
d12a214c05 normal_blobs->stream_blobs, proactive->background 2021-11-09 14:27:06 -05:00
Victor Shyba
3a83052f2e fix free space calculation, test it and give a margin of 10mb before starting so it doesnt insist when full 2021-11-09 14:27:06 -05:00
Victor Shyba
510b44ca92 move more logic out of the downloader component 2021-11-09 14:27:06 -05:00
Victor Shyba
15edb6756d extract background downloader to its own class 2021-11-09 14:27:06 -05:00
Victor Shyba
fbfd02b08b add analytics event for network disk space 2021-11-09 14:27:06 -05:00
Victor Shyba
b39c26fc86 announce orphan blobs manually, as that was done when save stream 2021-11-09 14:27:06 -05:00
Victor Shyba
95b2c8d175 cleanup background downloader blobs from conf 2021-11-09 14:27:06 -05:00
Victor Shyba
d52748b09f separated network seeding space metrics 2021-11-09 14:27:06 -05:00
Victor Shyba
34d18a3a9a don't save streams for network blobs and bypass disk space manager 2021-11-09 14:27:06 -05:00
Victor Shyba
3b27d6a9b5 add conf for network seeding space limit 2021-11-09 14:27:06 -05:00
Victor Shyba
703c391f99 schedule the download task instead 2021-11-09 14:27:06 -05:00
Victor Shyba
13667df374 download from DHT 2021-11-09 14:27:06 -05:00
Victor Shyba
8800d6985f drop channel support, prepare to hook into DHT 2021-11-09 14:27:06 -05:00
Victor Shyba
364b8f2605 handle case where something that isn't a sd blob gets hit 2021-11-09 14:27:06 -05:00
Victor Shyba
67b9ea9deb no api yet 2021-11-09 14:27:06 -05:00
Victor Shyba
b78f2336a7 download only blobs 2021-11-09 14:27:06 -05:00
Victor Shyba
23a5ce3df7 fix exception arguments 2021-11-09 14:27:06 -05:00
Victor Shyba
8f88e28e50 test add/remove/list subscriptions 2021-11-09 14:27:06 -05:00
Victor Shyba
9cf6139557 fix and test main api 2021-11-09 14:27:06 -05:00
Victor Shyba
d556065a8b download all blobs and check that on tests 2021-11-09 14:27:06 -05:00
Victor Shyba
951716f7dc create downloader component and initial tests 2021-11-09 14:27:06 -05:00
Victor Shyba
c42b76dcb8 dont lose results on duplicates, just warn 2021-11-08 10:50:47 -05:00
Victor Shyba
a73582d9ae remove tried_for_this_blob so banned peers are retried for same blob 2021-11-08 10:50:47 -05:00
Cristian Vicas
42c4fc7557 Bug [#2070] where blob_get RPC timed out. 
Both stream.downloader and blob_exchange.downloader paths are adding the fixed_peers list to the DHT node.
Tested jsonrpc_blob_get daemon call.

Bug [#2070] where blob_get RPC timed out.

Both stream.downloader and blob_exchange.downloader paths are adding the fixed_peers list to the DHT node.
Tested jsonrpc_blob_get daemon call.
 2021-11-08 10:49:48 -05:00
Jack Robison
ddbbb6f1dd
use mempool cache in transaction_get_batch 2021-10-27 20:19:08 -04:00
Lex Berezhny
ff21a92330
Merge pull request #3457 from FemtosecondLaser/feature/3270-check-default-download-dir-writable 
Modified ensure_directory_exists() to check if the directory is writable by the process.
 2021-10-27 11:00:13 -04:00
Jack Robison
1e391d211b
fix attempting to update trending on abandoned claims 2021-10-23 18:39:04 -04:00
Jack Robison
1a74d6604d skip loading tx/claim caches in the elastic sync script when not needed 2021-10-22 15:10:35 -04:00
Jack Robison
48505c2968 update trending with help from @eggplantbren 2021-10-21 00:17:12 -04:00
Jack Robison
a98ea1e66a update sync script to handle ES falling behind leveldb on shutdown 2021-10-20 23:41:11 -04:00
Jack Robison
3dec697816 logging 2021-10-20 23:41:11 -04:00
Jack Robison
b05d071a1c update Env to accept parameters from cli args 2021-10-20 23:41:11 -04:00
Jack Robison
a27d3b9689 set default CACHE_MB to 1024mb and the default QUERY_TIMEOUT_MS to 10s 2021-10-20 23:41:11 -04:00
Jack Robison
1facc0cd01 remove unused hub env settings 2021-10-20 23:41:11 -04:00
FemtosecondLaser
6b8d4a444b Modified ensure_directory_exists() to check if the directory is writable by the process. 2021-10-20 15:26:16 +01:00
Jack Robison
6bef09a3b1 update lbry-hub-elastic-sync to support resyncing recent blocks 2021-10-19 15:53:20 -04:00
Jack Robison
e35319e5a2 add CACHE_ALL_CLAIM_TXOS hub setting 2021-10-19 15:53:20 -04:00
Jack Robison
0e548b3812 remove dead code 2021-10-19 15:53:20 -04:00
Jack Robison
bfac02ccab add CACHE_ALL_TX_HASHES setting to optionally use more memory to save i/o 2021-10-19 15:53:20 -04:00
Jack Robison
7ea1a2b361 sleeps 2021-10-19 15:53:20 -04:00
Jack Robison
99df418f1d improve resolve caching 2021-10-19 15:53:20 -04:00
Jack Robison
6416d8ce9c threadpools for block processor and es sync reader 2021-10-19 15:53:20 -04:00
Jack Robison
22b43a2b01 doc strings 2021-10-19 15:53:20 -04:00
Jack Robison
05e5d24c5e improve claims_producer performance 2021-10-19 15:53:20 -04:00
Jack Robison
eabcc30367 resolve lru cache 2021-10-19 15:53:20 -04:00
Jack Robison
f5e0ef5223 add block_txs index 2021-10-19 15:53:20 -04:00
Jack Robison
f46d9330b0 smaller caches 2021-10-19 15:53:20 -04:00
Jack Robison
b62a0b4607 Update daemon.py 
docstring
 2021-10-15 09:40:15 -04:00
Cristian Vicas
1f044321fb Updated documentation for RPC calls: status, blob_list. 2021-10-15 09:40:15 -04:00
belikor
9509acc490
file_manager: raise new InvalidStreamURLError if the URL is invalid 
When using `lbrynet get URL`, if the URL is not a valid URL
the function `url.URL.parse` will raise a `ValueError` exception
which will produce a whole backtrace.

For example, this is the case if we provide a channel name
with a forward slash but without a stream name.
```
lbrynet get @Non-existing/
```

```
Traceback (most recent call last):
  File "/opt/git/lbry-sdk/lbry/file/file_manager.py", line 84, in download_from_uri
    if not URL.parse(uri).has_stream:
  File "/opt/git/lbry-sdk/lbry/schema/url.py", line 114, in parse
    raise ValueError('Invalid LBRY URL')
ValueError: Invalid LBRY URL
WARNING  lbry.extras.daemon.daemon:1110: Error downloading Non-existing/: Invalid LBRY URL
```

Now we raise a new `InvalidStreamURLError` which can be trapped in the upper functions
that use `url.URL.parse` such as `FileManager.download_from_uri`.
If we do this the traceback won't be shown.
```
WARNING  lbry.file.file_manager:252:
Failed to download Non-existing/: Invalid LBRY stream URL: '@Non-existing/'
WARNING  lbry.extras.daemon.daemon:1110:
Error downloading Non-existing/: Invalid LBRY stream URL: '@Non-existing/'
```

This handles the case when trying to download only "channel" parts
without the claim part.
```
lbrynet get @Non-existing
lbrynet get @Non-existing/
lbrynet get Non-existing/
```
 2021-10-15 08:59:37 -04:00
Jack Robison
d3516f299e
clear es attributes during initial sync 2021-10-08 16:34:48 -04:00