Madiator2011/lbry-sdk
1
0
Fork 0
forked from LBRYCommunity/lbry-sdk
Code Pull requests Activity
8279 commits 111 branches 641 tags 30 MiB
Commit graph

1491 commits

Author SHA1 Message Date
Victor Shyba 19494088bd generate from queue 2021-03-19 19:58:13 -03:00
Victor Shyba 920dad524a simplify sync and use asyncio Queue instead 2021-03-19 19:58:13 -03:00
Victor Shyba ec89bcac8e improve sync script for no-downtime maintenance 2021-03-19 19:58:13 -03:00
Victor Shyba a916c1f4ad check if db file exists before sync 2021-03-19 19:58:13 -03:00
Victor Shyba a9a0ac92d7 ignore unset flag 2021-03-19 19:58:13 -03:00
Victor Shyba da8a8bd1ef filter+fts and tests for edge cases 2021-03-19 19:58:13 -03:00
Victor Shyba 67817005b5 check ES synced without a process and wait for ES 2021-03-19 19:58:13 -03:00
Jack Robison 24d11de5a7 torba-elastic-sync 2021-03-19 19:58:13 -03:00
Victor Shyba e12fab90d1 docker compose update 2021-03-19 19:58:13 -03:00
Victor Shyba 0a194b5b01 claim_ids query 2021-03-19 19:58:13 -03:00
Victor Shyba 8d028adc53 be a writer by default 2021-03-19 19:58:13 -03:00
Victor Shyba dfca15395e claim id is also a keyword 2021-03-19 19:58:13 -03:00
Victor Shyba e21f2362fe apply reorg deletion as well 2021-03-19 19:58:13 -03:00
Victor Shyba 1ce328e8a9 cache signature inspection 2021-03-19 19:58:13 -03:00
Victor Shyba 038a5f999f cache encoded headers 2021-03-19 19:58:13 -03:00
Victor Shyba 5d3704c7ea reader mode 2021-03-19 19:58:13 -03:00
Victor Shyba 87037c06c9 remove reader code 2021-03-19 19:58:13 -03:00
Victor Shyba dd412c0f50 delete sqlite fts 2021-03-19 19:58:13 -03:00
Victor Shyba bf44befff6 backport fixes from server 2021-03-19 19:58:13 -03:00
Victor Shyba e61874bb6f only repeat search if it has blocked items 2021-03-19 19:58:13 -03:00
Victor Shyba 1e5331768f fix some of the tests 2021-03-19 19:58:13 -03:00
Victor Shyba ec9a3a4f7c do not page filtered 2021-03-19 19:58:13 -03:00
Victor Shyba e439a3a8dc advanced resolve 2021-03-19 19:58:13 -03:00
Victor Shyba 19f70d7a11 create changelog trigger 2021-03-19 19:58:13 -03:00
Victor Shyba afe7ed5b05 adjust size 2021-03-19 19:58:13 -03:00
Victor Shyba d4bf004d74 use a thread pool to sync changes 2021-03-19 19:58:13 -03:00
Victor Shyba e4d06a088b include the channel being filtered/blocked 2021-03-19 19:58:13 -03:00
Victor Shyba 0929088b12 missing refresh step 2021-03-19 19:58:13 -03:00
Victor Shyba 7b4838fc9b dont update more than 400 items a time 2021-03-19 19:58:13 -03:00
Victor Shyba 0cf9533248 narrow update by query 2021-03-19 19:58:13 -03:00
Victor Shyba 84ff0b8a9f general timeout 2021-03-19 19:58:13 -03:00
Victor Shyba d467dcfeaf increase sync queue 2021-03-19 19:58:13 -03:00
Victor Shyba 8e68ba4751 fix join, refresh before update 2021-03-19 19:58:13 -03:00
Victor Shyba 0f2a85ba9f simplify sync 2021-03-19 19:58:13 -03:00
Victor Shyba 7674a0a91e backport fixes from testing server 2021-03-19 19:58:13 -03:00
Victor Shyba 5bc1a66572 32 slices and add censor type to fields 2021-03-19 19:58:13 -03:00
Victor Shyba 9b56067213 raise request timeout for content filtering 2021-03-19 19:58:13 -03:00
Victor Shyba 9a9df2fc3c apply filtering only to whats unfiltered 2021-03-19 19:58:13 -03:00
Victor Shyba 9989d8d1d4 refresh after delete 2021-03-19 19:58:13 -03:00
Victor Shyba f9471f297e apply filter and block from ES script lang 2021-03-19 19:58:13 -03:00
Victor Shyba 146b693e4a exclude title and description 2021-03-19 19:58:13 -03:00
Victor Shyba e2441ea3e7 use prefix from ES docs 2021-03-19 19:58:13 -03:00
Victor Shyba 119e51912e fix partial id 2021-03-19 19:58:13 -03:00
Victor Shyba dd950f5b0d tag can have empty space 2021-03-19 19:58:13 -03:00
Victor Shyba 78a9bad1e1 no indexer_task 2021-03-19 19:58:13 -03:00
Victor Shyba 0c6eaf5484 fix resolve partial id 2021-03-19 19:58:13 -03:00
Victor Shyba 1010068ddb disable refresh interval. start with 3 shards 2021-03-19 19:58:13 -03:00
Victor Shyba ee7b37d3f3 also normalize the name supplied by user 2021-03-19 19:58:13 -03:00
Victor Shyba 143d82d242 normalized, not normalized_name 2021-03-19 19:58:13 -03:00
Victor Shyba 8b91b38855 update winners in one go 2021-03-19 19:58:13 -03:00
Victor Shyba 1098f0d2a3 use normalized name instead 2021-03-19 19:58:13 -03:00
Victor Shyba ab53cec022 fix is_controlling sync 2021-03-19 19:58:13 -03:00
Victor Shyba 90106f5f08 all test_claim_commands tests green 2021-03-19 19:58:13 -03:00
Victor Shyba 9924b7b438 reposts and tag inheritance 2021-03-19 19:58:13 -03:00
Victor Shyba aa37faab0a use porter analyzer with weights on full text search 2021-03-19 19:58:13 -03:00
Victor Shyba dc10f8ce72 ignore errors when deleting 2021-03-19 19:58:13 -03:00
Victor Shyba 996686c1da claim search and resolve translated to ES queries 2021-03-19 19:58:13 -03:00
Victor Shyba 488785d013 add indexer task 2021-03-19 19:58:13 -03:00
Victor Shyba 3abdc01230 index ES during sync 2021-03-19 19:58:13 -03:00
Victor Shyba 8da04a584f start waiting before generate 2021-03-19 18:01:29 -03:00
Victor Shyba 27cc61d45e limit test time to 2 minutes, then consider it a failure and log what was running 2021-03-19 18:01:29 -03:00
Lex Berezhny 7371c30064 v0.92.0 2021-03-15 13:07:30 -04:00
Victor Shyba 74df4fab83 change column to has_source and document both flags 2021-03-14 10:11:42 -04:00
Victor Shyba 1e5cd3d7a1 typo, fix tests 2021-03-14 10:11:42 -04:00
Victor Shyba a54e9b64aa add no_source claim_search filter 2021-03-14 10:11:42 -04:00
Victor Shyba 74660704e3 fix update 2021-03-14 10:11:42 -04:00
Victor Shyba 7439893a2a fix get for sourceless claims 2021-03-14 10:11:42 -04:00
Victor Shyba e27e49e9dc call update only once 2021-03-14 10:11:42 -04:00
Victor Shyba 34ed729c59 there is no 'sd_hash' parameter for this API 2021-03-14 10:11:42 -04:00
Victor Shyba adaeeca3fd let file_path be optional 2021-03-14 10:11:42 -04:00
Jack Robison dac75563d3 add --no_file_path param to publish, stream_create, and stream_update 2021-03-14 10:11:42 -04:00
Alex Grintsvayg cbc76adcaa only return unspent txos if is_spent flag is not used. fixes #2923 2021-03-13 06:44:20 -05:00
Lex Berezhny 4343073c00 clients can connect to wallet server even when they are not reachable by UDP 2021-03-12 13:29:55 -05:00
Victor Shyba ae337807f5 get merkles outside thread cooperatively 2021-03-10 13:05:17 -03:00
Victor Shyba 9ae30ac08e during subscribe, hash address only when its time 2021-03-10 12:51:58 -03:00
Lex Berezhny 7bb873dad9 removed connection_status field from the status command, use wallet.connected instead to determine if SDK is connected 2021-03-09 13:27:36 -05:00
Lex Berezhny 5f6c1c14cb v0.91.0 2021-03-04 00:04:25 -05:00
Lex Berezhny fcad76fc51 lint 2021-03-04 00:03:16 -05:00
Lex Berezhny 97e6e1684e simplifying 2021-03-04 00:03:16 -05:00
zeppi 183fb9f9ff provide --resolve tag for collection claim, separate from resolving its contents 
bugfix and docs generation

review changes
 2021-03-04 00:03:16 -05:00
Lex Berezhny 9815ddef1f fixes stalling client reconnect issue 2021-03-03 23:31:59 -05:00
Lex Berezhny f6d0847453 v0.90.1 2021-03-01 11:57:40 -05:00
Lex Berezhny 0cec80f676 fixes transaction signing bug when tx had no change outputs 2021-03-01 11:20:08 -05:00
Patrick Keane 48c64143e3 Add no_totals argument to claim search 2021-02-26 10:41:15 -05:00
Patrick Keane a8712422bc Update daemon.py 2021-02-21 23:45:18 -05:00
Lex Berezhny fd3c1c50f1 v0.90.0 2021-02-16 11:13:14 -05:00
Lex Berezhny b153e4bb9f added support to claim_search for filtering collections via --claim_type 2021-02-16 11:08:54 -05:00
Lex Berezhny db9856a8db use median exchange rate when several exchange rates are available 2021-02-15 14:09:32 -05:00
Lex Berezhny be8751cb73 integration test fixes 2021-02-10 22:14:13 -05:00
Lex Berezhny f1cb7d27ac lint 2021-02-10 22:14:13 -05:00
Lex Berezhny dee494e12f converting from USD, BTC to LBC is now done via several exchange rate providers: Bittrex, Cryptonator, CoinEx, hotbit and UPbit 2021-02-10 22:14:13 -05:00
Lex Berezhny b13a121915 v0.89.0 2021-02-09 10:12:05 -05:00
Lex Berezhny 7486ee9537 do not tx._reset() in generate_channel_private_key 2021-02-08 09:56:03 -05:00
Lex Berezhny 4a20ccc28e tx._reset() the transaction after more internal changes 2021-02-08 09:56:03 -05:00
Lex Berezhny f80dd2b307 tx._reset() the transaction after internal changes 2021-02-08 09:56:03 -05:00
Lex Berezhny b208cf6d32 significant performance improvement when creating transactions and with txo spend specifically 2021-02-08 09:56:03 -05:00
Lex Berezhny 39e78ff17e updated exchange rate manager to use v3 bittrex API (old one is deprecated) 2021-02-03 14:15:19 -05:00
Alex Grin bedcfc154b
Merge branch 'master' into fix-collectionChannel 2021-02-02 11:25:52 -05:00
Victor Shyba f6cfe266e0 specify Returns correctly for docs gen 2021-02-01 19:19:46 -05:00
Lex Berezhny ccb250b410 txo_list --resolve now works for signed supports by resolving the signing channel 2021-01-26 20:53:36 -05:00
zeppi aca57ffc62 allows cli to accept channel_id for collections 2021-01-26 10:02:01 -05:00
Dispatch 7f375f42d8 Fix / Add correct mimetypes for HLS / DASH 2021-01-25 15:55:56 -05:00
Lex Berezhny eedcc2034d fixes #3166 2021-01-25 10:56:48 -05:00
Jack Robison 3b27cb3671 v0.88.0 2021-01-22 10:41:06 -05:00
Lex Berezhny 8b55814ab2 fixes #2943 bug where get forced user to purchase their own priced stream 2021-01-22 07:51:13 -05:00
Jack Robison 0f5f1aebed
accept connections from ledger clients in spv server 2021-01-21 21:37:39 -05:00
Jack Robison 1f9d0f4582
pylint 2021-01-21 16:30:54 -05:00
Jack Robison a6d65233f1
fallback to getting external ip from spv servers instead of internal apis 2021-01-21 16:20:53 -05:00
Jack Robison 20efdc70b3
use UDP ping for wallet server selection 
-only connect to one spv server at a time
-remove session pool
 2021-01-21 16:15:42 -05:00
Jack Robison f0d8fb8f1a
add UDP based ping protocol for spv servers 2021-01-21 16:08:33 -05:00
Jack Robison f7a380e9b7
start prometheus before block processing 2021-01-21 16:01:33 -05:00
Jack Robison e9c7cf6f63
logging 2021-01-21 14:51:59 -05:00
Jack Robison 68f1661452
add LRUCache with no prometheus metrics 2021-01-21 14:37:08 -05:00
Jack Robison 36fd1b91ae
cache server features 2021-01-21 14:37:08 -05:00
Jack Robison a4ec430ac0
improve subscription performance 2021-01-21 14:37:08 -05:00
Jack Robison 519614b2fd
skip libtorrent component in tests 2021-01-21 14:37:08 -05:00
Lex Berezhny bf0118c8ef added support for --not_channel_id to txo_list commands 2021-01-21 14:03:59 -05:00
Jack Robison 7952fc8324
fix reflector loop propagating cancelled error 2021-01-08 09:53:12 -05:00
Victor Shyba 2a17787242 fix test 2021-01-04 18:04:06 -03:00
Victor Shyba 0a53ad5721 use the comment api signing implementation 2021-01-04 18:04:06 -03:00
Victor Shyba 6da6bdc863 adds jsonrpc_channel_sign signing api 2021-01-04 18:04:06 -03:00
Jack Robison 77e19ab1a4
prometheus metrics for asyncio loop responsiveness 2021-01-01 15:25:31 -05:00
Victor Shyba b882f1a010 log checkpoint finishing for gathering information on wal/shm file bug 2020-12-31 16:11:22 -03:00
Jack Robison 0758b85179
skip null args given to channel_ids in claim search 2020-12-29 21:41:07 -05:00
Jack Robison ab3d9bd080
block cache metrics 2020-12-29 21:40:39 -05:00
Jack Robison 66029e60d3 v0.87.0 2020-12-23 21:51:13 -05:00
Jack Robison 9dd5159414
increase server cache sizes 2020-12-23 20:47:05 -05:00
Jack Robison 13e38d6fd8
use LRUCache instead of pylru in wallet server 2020-12-23 20:47:05 -05:00
Jack Robison 10dcb64715
lru cache metrics 2020-12-23 20:47:03 -05:00
Victor Shyba adb418aafc group cache hits in a single dict 2020-12-23 21:37:20 -03:00
Victor Shyba 270da80d64 return whole batches 2020-12-23 19:10:19 -03:00
Victor Shyba b2027cfd66 sync in batches of 10, clearing after 2020-12-23 18:30:52 -03:00
Jack Robison 7a7446c8bd
force resync blockchain.db 2020-12-22 20:08:26 -05:00
Jack Robison 8c71b744f3
fix request_synced_transactions edge cases 2020-12-22 15:19:08 -05:00
Jack Robison 4cbf4230e8
fix txi.txo_ref 2020-12-22 14:05:37 -05:00
Victor Shyba 6a610187e0 cache bypass 2020-12-22 13:01:30 -03:00
Victor Shyba eb2a4aebba unrestricted and reusing verified cache hits 2020-12-21 16:49:08 -03:00
Victor Shyba 21a2e67755 fix rebase 2020-12-20 21:39:36 -03:00
Victor Shyba 3b9e312615 fix verification 2020-12-20 21:25:26 -03:00
Victor Shyba 26dab04c9e checkpoint 2020-12-20 21:25:26 -03:00
Victor Shyba 00713c0d11 asyncgens 2020-12-20 21:25:26 -03:00
Victor Shyba 751b5f3027 refactor duplicate code 2020-12-20 21:25:26 -03:00
Victor Shyba e8261b000e wip, see jack 2020-12-20 21:23:58 -03:00
Victor Shyba 41ecb70297 join network can only happen after initial header sync returns 2020-12-20 21:15:11 -03:00
Jack Robison e3a4964787
catch any remaining index errors in fs_tx_hash 2020-12-16 01:28:30 -05:00
Jack Robison 9bf72910a4
fix off by one error in reorg 2020-12-16 01:27:03 -05:00
Jack Robison ee39e20e6d
logging 2020-12-16 01:26:19 -05:00
Jack Robison 0821ce44b5
remove unused heavy call 2020-12-15 19:22:12 -05:00
Jack Robison 674ce02e58
logging 2020-12-14 14:38:36 -05:00
Jack Robison 8dfa2767ec
new_touched 2020-12-14 13:52:26 -05:00
Jack Robison 20dad7f07f
only notify hashxs touched since last notification 2020-12-14 13:42:20 -05:00
Jack Robison 751cc4c44d
don't deserialize mempool in a thread 2020-12-13 13:48:50 -05:00
Jack Robison 2318e6d8e9
faster fs_transactions 2020-12-13 13:48:02 -05:00
Jack Robison 61b4a492c3 v0.86.1 2020-12-08 16:41:46 -05:00
Jack Robison 8da73ad3dd
improve hash_to_hex_str performance 2020-12-08 15:39:03 -05:00
Jack Robison b8c16d8ac5
fix mempool txs being re-requested with each address notification 2020-12-08 15:39:03 -05:00
Jack Robison 429c0951f3 v0.86.0 2020-12-07 21:19:24 -05:00
Jack Robison 74e103c791
version 2020-12-07 21:15:15 -05:00
Jack Robison f941950ee2
fix comment_create docs 2020-12-07 21:13:43 -05:00
Jack Robison 846df2eef1 v0.86.0 2020-12-07 15:58:05 -05:00
Jack Robison eae0290978
fix mempool notification bloat 2020-12-07 13:16:37 -05:00
Jack Robison 3467d1fed0
fix tests 2020-12-03 22:16:28 -05:00
Jack Robison d02ff232e5
_single_batch 2020-12-03 20:50:25 -05:00
Jack Robison 2d1c6a5402
fix 2020-12-03 19:15:15 -05:00
Jack Robison eab3b65629
calculate notifications for all subscriptions instead of per session 2020-12-03 19:15:15 -05:00
Jack Robison 929617273d
resync blockchain.db 2020-12-03 15:56:35 -05:00
Jack Robison 2717bf7d49
increase default tx_cache_size setting 2020-12-03 15:56:35 -05:00
Jack Robison 9b4afe9816
store txids in memory, faster address subscription/history 2020-11-25 16:30:11 -05:00
Jack Robison af1d7813e9
ensure transactions are returned in the order they're requested 2020-11-24 11:52:07 -05:00
Jack Robison c02750edbd
fix tx_merkle 2020-11-23 16:48:24 -05:00
Jack Robison faeba9a7e4
faster blockchain.transaction.get_batch 2020-11-23 12:05:47 -05:00
Jack Robison 190d238a1f
faster read_headers 2020-11-23 12:03:11 -05:00
Jack Robison 715451b5fb v0.85.0 2020-11-21 00:14:20 -05:00
Jack Robison 923d817751
store headers in memory 2020-11-20 22:37:42 -05:00
Victor Shyba b8b9dcc2ee WAL_CHECKPOINT on close 2020-11-20 20:16:03 -03:00
Jack Robison 34f4f12eb9
pylint 2020-11-20 16:15:57 -05:00
Jack Robison fa63bf758d
delete single_call_context, use session pool 2020-11-20 15:52:11 -05:00
Jack Robison 2c7fd58e34
threaded compress headers 2020-11-20 13:09:54 -05:00
Jack Robison f2fd42b47a
use write_batch 2020-11-19 18:37:49 -05:00
Jack Robison 1b4ccad938
allow settings TRENDING_ALGORITHMS to nothing 2020-11-19 18:36:34 -05:00
Jack Robison a9de1ce8e0
fix fs_transactions 2020-11-19 12:40:46 -05:00
Jack Robison 632d8d02d2
remove dead code 2020-11-17 16:17:24 -05:00
Jack Robison 48aeb26e02
threaded_get_merkle 
-remove dead code
 2020-11-17 16:05:08 -05:00
Jack Robison 1694af8b5e
get transactions from leveldb instead of lbrycrd rpc 2020-11-17 15:30:48 -05:00
Jack Robison 83bcab9cd2
cleanup 2020-11-17 15:30:48 -05:00
Jack Robison bdc7f4b3f5
combine tx_count_db and hashes_db, add tx db 2020-11-17 15:30:48 -05:00
Lex Berezhny 912065a121 remove url sequence feature from code and tests 2020-11-17 11:24:29 -05:00
Thomas Zarebczan c8466afac2
Remove * from invalid characters 
This is already part of other URLs and channels, will be removed from spec as it won't be used for sequencing.
 2020-11-16 14:09:04 -05:00
Victor Shyba 2619e162c1 recover invalid state described on #3026 2020-11-09 16:03:38 -05:00
Lex Berezhny e1112e17f8 v0.84.0 2020-11-08 19:42:33 -05:00
Jack Robison bbed9b94c1
null language filtering for claim search 2020-11-05 21:23:49 -05:00
Brendon J. Brewer 1cdff47477
Rewrite of variable_decay.py for speed improvements 2020-11-05 21:15:55 -05:00
Jack Robison d83936a66a
fix uncaught error 2020-11-04 22:21:35 -05:00
Jack Robison 5517d2bf56
fix new_sdk_server arg for resolve 2020-11-03 16:23:31 -05:00
Jack Robison 925a458abe
tags 2020-11-03 10:39:08 -05:00
Jack Robison 76946c447f
use single_call_context for claim_search and resolve 2020-11-03 10:39:08 -05:00
Lex Berezhny 5e17ce0a0b minor fixup 2020-10-30 11:37:54 -04:00
Kevin Raoofi e8d299d3b6 Allow : or # for claim_id 
This removes the code for trying multiple patterns and the setup for it

Added a few unit tests to check that the parsed URL is as expected
 2020-10-30 11:37:54 -04:00
Kevin Raoofi 7637aa2ab6 Added support for spec compliant URL parsing 
Legacy URLs are preserved by attempting to parse the new URL format and,
on failing that, it'll attempt the legacy one.

Tests had to be updated such that the correct things are asserted
against each other.
 2020-10-30 11:37:54 -04:00
Jack Robison ab067d1d3a fix unclean closing of blockchain.db 2020-10-26 16:23:28 -04:00