LBRYCommunity/lbry-sdk
1
0
Fork 1
Code Issues Pull requests Projects Releases 282 Packages Wiki Activity
8515 commits 110 branches 641 tags 31 MiB
Commit graph

604 commits

Author SHA1 Message Date
shubhendra
be0ba22222 Remove unnecessary comprehension 
Signed-off-by: shubhendra <withshubh@gmail.com>
 2021-04-06 21:22:27 -04:00
Victor Shyba
46dd389d0d add elasticsearch service to gitlab 2021-03-30 17:00:15 -04:00
Jack Robison
7832c62c5d
add ELASTIC_HOST and ELASTIC_PORT settings to hub 2021-03-30 12:48:13 -04:00
Victor Shyba
aa727cb9b1 show channels regardless of no_source 2021-03-30 09:47:08 -04:00
Victor Shyba
b8c9a99f20 fix no_source for reposts 2021-03-30 09:47:08 -04:00
Lex Berezhny
aff995b0d0 temporary fix for mempool sync failing during reorg 2021-03-29 16:11:03 -04:00
Victor Shyba
5235a150b1 add prog name to sync arg parser 2021-03-24 17:07:17 -03:00
Victor Shyba
7df4cc44c4 fixes from review 2021-03-24 16:30:33 -03:00
Victor Shyba
7f5d88e95c remove dead/broken/unused API 2021-03-19 19:58:13 -03:00
Victor Shyba
d09663c066 remove flush call 2021-03-19 19:58:13 -03:00
Victor Shyba
ef97c9b69f torba-server -> hub 2021-03-19 19:58:13 -03:00
Victor Shyba
d855e6c8b1 move elasticsearch things into its own module 2021-03-19 19:58:13 -03:00
Victor Shyba
cd66f7eb43 if not no_totals, use default page size 2021-03-19 19:58:13 -03:00
Victor Shyba
a3e146dc68 sort on index time 2021-03-19 19:58:13 -03:00
Victor Shyba
b81305a4a9 index and allow has_source 2021-03-19 19:58:13 -03:00
Victor Shyba
73884b34bc apply no_totals 2021-03-19 19:58:13 -03:00
Victor Shyba
6166a34db2 check cache item before locking 2021-03-19 19:58:13 -03:00
Victor Shyba
6fa7da4b1c less slices 2021-03-19 19:58:13 -03:00
Victor Shyba
c3e426c491 fix search by channel for invalid channel 2021-03-19 19:58:13 -03:00
Victor Shyba
21e023f0db fix search by channel 2021-03-19 19:58:13 -03:00
Victor Shyba
063be001b3 cache inner parsing 2021-03-19 19:58:13 -03:00
Victor Shyba
5dff02e8bc on resolve, get all claims at once 2021-03-19 19:58:13 -03:00
Victor Shyba
60a59407d8 cache the encoded output instead 2021-03-19 19:58:13 -03:00
Victor Shyba
20a5aecfca fix lib exception to asyncio TimeoutError 2021-03-19 19:58:13 -03:00
Victor Shyba
c2e7b5a67d restore some of the interrupt metrics 2021-03-19 19:58:13 -03:00
Victor Shyba
8f32303d07 apply search timeout 2021-03-19 19:58:13 -03:00
Victor Shyba
891b1e7782 track results up to 200 2021-03-19 19:58:13 -03:00
Victor Shyba
f26394fd3b report deletions on docs that doesnt exist, but dont raise 2021-03-19 19:58:13 -03:00
Victor Shyba
4d83d42b4c fix equality instead of mod 2021-03-19 19:58:13 -03:00
Victor Shyba
57f1108df2 fix query being json serializable 2021-03-19 19:58:13 -03:00
Victor Shyba
2641a9abe5 make better resolve cache 2021-03-19 19:58:13 -03:00
Victor Shyba
6b193ab350 make indexing cooperative 2021-03-19 19:58:13 -03:00
Victor Shyba
b1bb37511c use right key on cache 2021-03-19 19:58:13 -03:00
Victor Shyba
319187d6d6 log mempool task exceptions 2021-03-19 19:58:13 -03:00
Victor Shyba
02eb789f84 caching for resolve 2021-03-19 19:58:13 -03:00
Victor Shyba
eb6924277f round time to 10 minutes and fetch referenced by id 2021-03-19 19:58:13 -03:00
Victor Shyba
bd8f371fd5 bump referenced rows query limit up 2021-03-19 19:58:13 -03:00
Victor Shyba
1783ff2845 dont delete claims on reorg 2021-03-19 19:58:13 -03:00
Victor Shyba
d388527ffa log indexing errors 2021-03-19 19:58:13 -03:00
Victor Shyba
19494088bd generate from queue 2021-03-19 19:58:13 -03:00
Victor Shyba
920dad524a simplify sync and use asyncio Queue instead 2021-03-19 19:58:13 -03:00
Victor Shyba
ec89bcac8e improve sync script for no-downtime maintenance 2021-03-19 19:58:13 -03:00
Victor Shyba
a916c1f4ad check if db file exists before sync 2021-03-19 19:58:13 -03:00
Victor Shyba
a9a0ac92d7 ignore unset flag 2021-03-19 19:58:13 -03:00
Victor Shyba
da8a8bd1ef filter+fts and tests for edge cases 2021-03-19 19:58:13 -03:00
Victor Shyba
67817005b5 check ES synced without a process and wait for ES 2021-03-19 19:58:13 -03:00
Jack Robison
24d11de5a7 torba-elastic-sync 2021-03-19 19:58:13 -03:00
Victor Shyba
e12fab90d1 docker compose update 2021-03-19 19:58:13 -03:00
Victor Shyba
0a194b5b01 claim_ids query 2021-03-19 19:58:13 -03:00
Victor Shyba
8d028adc53 be a writer by default 2021-03-19 19:58:13 -03:00
Victor Shyba
dfca15395e claim id is also a keyword 2021-03-19 19:58:13 -03:00
Victor Shyba
e21f2362fe apply reorg deletion as well 2021-03-19 19:58:13 -03:00
Victor Shyba
1ce328e8a9 cache signature inspection 2021-03-19 19:58:13 -03:00
Victor Shyba
038a5f999f cache encoded headers 2021-03-19 19:58:13 -03:00
Victor Shyba
5d3704c7ea reader mode 2021-03-19 19:58:13 -03:00
Victor Shyba
87037c06c9 remove reader code 2021-03-19 19:58:13 -03:00
Victor Shyba
dd412c0f50 delete sqlite fts 2021-03-19 19:58:13 -03:00
Victor Shyba
bf44befff6 backport fixes from server 2021-03-19 19:58:13 -03:00
Victor Shyba
e61874bb6f only repeat search if it has blocked items 2021-03-19 19:58:13 -03:00
Victor Shyba
ec9a3a4f7c do not page filtered 2021-03-19 19:58:13 -03:00
Victor Shyba
e439a3a8dc advanced resolve 2021-03-19 19:58:13 -03:00
Victor Shyba
19f70d7a11 create changelog trigger 2021-03-19 19:58:13 -03:00
Victor Shyba
afe7ed5b05 adjust size 2021-03-19 19:58:13 -03:00
Victor Shyba
d4bf004d74 use a thread pool to sync changes 2021-03-19 19:58:13 -03:00
Victor Shyba
e4d06a088b include the channel being filtered/blocked 2021-03-19 19:58:13 -03:00
Victor Shyba
0929088b12 missing refresh step 2021-03-19 19:58:13 -03:00
Victor Shyba
7b4838fc9b dont update more than 400 items a time 2021-03-19 19:58:13 -03:00
Victor Shyba
0cf9533248 narrow update by query 2021-03-19 19:58:13 -03:00
Victor Shyba
84ff0b8a9f general timeout 2021-03-19 19:58:13 -03:00
Victor Shyba
d467dcfeaf increase sync queue 2021-03-19 19:58:13 -03:00
Victor Shyba
8e68ba4751 fix join, refresh before update 2021-03-19 19:58:13 -03:00
Victor Shyba
0f2a85ba9f simplify sync 2021-03-19 19:58:13 -03:00
Victor Shyba
7674a0a91e backport fixes from testing server 2021-03-19 19:58:13 -03:00
Victor Shyba
5bc1a66572 32 slices and add censor type to fields 2021-03-19 19:58:13 -03:00
Victor Shyba
9b56067213 raise request timeout for content filtering 2021-03-19 19:58:13 -03:00
Victor Shyba
9a9df2fc3c apply filtering only to whats unfiltered 2021-03-19 19:58:13 -03:00
Victor Shyba
9989d8d1d4 refresh after delete 2021-03-19 19:58:13 -03:00
Victor Shyba
f9471f297e apply filter and block from ES script lang 2021-03-19 19:58:13 -03:00
Victor Shyba
146b693e4a exclude title and description 2021-03-19 19:58:13 -03:00
Victor Shyba
e2441ea3e7 use prefix from ES docs 2021-03-19 19:58:13 -03:00
Victor Shyba
119e51912e fix partial id 2021-03-19 19:58:13 -03:00
Victor Shyba
dd950f5b0d tag can have empty space 2021-03-19 19:58:13 -03:00
Victor Shyba
78a9bad1e1 no indexer_task 2021-03-19 19:58:13 -03:00
Victor Shyba
0c6eaf5484 fix resolve partial id 2021-03-19 19:58:13 -03:00
Victor Shyba
1010068ddb disable refresh interval. start with 3 shards 2021-03-19 19:58:13 -03:00
Victor Shyba
ee7b37d3f3 also normalize the name supplied by user 2021-03-19 19:58:13 -03:00
Victor Shyba
143d82d242 normalized, not normalized_name 2021-03-19 19:58:13 -03:00
Victor Shyba
8b91b38855 update winners in one go 2021-03-19 19:58:13 -03:00
Victor Shyba
1098f0d2a3 use normalized name instead 2021-03-19 19:58:13 -03:00
Victor Shyba
ab53cec022 fix is_controlling sync 2021-03-19 19:58:13 -03:00
Victor Shyba
90106f5f08 all test_claim_commands tests green 2021-03-19 19:58:13 -03:00
Victor Shyba
9924b7b438 reposts and tag inheritance 2021-03-19 19:58:13 -03:00
Victor Shyba
aa37faab0a use porter analyzer with weights on full text search 2021-03-19 19:58:13 -03:00
Victor Shyba
dc10f8ce72 ignore errors when deleting 2021-03-19 19:58:13 -03:00
Victor Shyba
996686c1da claim search and resolve translated to ES queries 2021-03-19 19:58:13 -03:00
Victor Shyba
488785d013 add indexer task 2021-03-19 19:58:13 -03:00
Victor Shyba
3abdc01230 index ES during sync 2021-03-19 19:58:13 -03:00
Victor Shyba
74df4fab83 change column to has_source and document both flags 2021-03-14 10:11:42 -04:00
Victor Shyba
1e5cd3d7a1 typo, fix tests 2021-03-14 10:11:42 -04:00
Victor Shyba
a54e9b64aa add no_source claim_search filter 2021-03-14 10:11:42 -04:00
Lex Berezhny
4343073c00 clients can connect to wallet server even when they are not reachable by UDP 2021-03-12 13:29:55 -05:00
Victor Shyba
ae337807f5 get merkles outside thread cooperatively 2021-03-10 13:05:17 -03:00
Victor Shyba
9ae30ac08e during subscribe, hash address only when its time 2021-03-10 12:51:58 -03:00
zeppi
183fb9f9ff provide --resolve tag for collection claim, separate from resolving its contents 
bugfix and docs generation

review changes
 2021-03-04 00:03:16 -05:00
Lex Berezhny
9815ddef1f fixes stalling client reconnect issue 2021-03-03 23:31:59 -05:00
Lex Berezhny
0cec80f676 fixes transaction signing bug when tx had no change outputs 2021-03-01 11:20:08 -05:00
Lex Berezhny
b153e4bb9f added support to claim_search for filtering collections via --claim_type 2021-02-16 11:08:54 -05:00
Lex Berezhny
7486ee9537 do not tx._reset() in generate_channel_private_key 2021-02-08 09:56:03 -05:00
Lex Berezhny
f80dd2b307 tx._reset() the transaction after internal changes 2021-02-08 09:56:03 -05:00
Lex Berezhny
b208cf6d32 significant performance improvement when creating transactions and with txo spend specifically 2021-02-08 09:56:03 -05:00
Lex Berezhny
ccb250b410 txo_list --resolve now works for signed supports by resolving the signing channel 2021-01-26 20:53:36 -05:00
Lex Berezhny
eedcc2034d fixes #3166 2021-01-25 10:56:48 -05:00
Jack Robison
0f5f1aebed
accept connections from ledger clients in spv server 2021-01-21 21:37:39 -05:00
Jack Robison
1f9d0f4582
pylint 2021-01-21 16:30:54 -05:00
Jack Robison
20efdc70b3
use UDP ping for wallet server selection 
-only connect to one spv server at a time
-remove session pool
 2021-01-21 16:15:42 -05:00
Jack Robison
f0d8fb8f1a
add UDP based ping protocol for spv servers 2021-01-21 16:08:33 -05:00
Jack Robison
f7a380e9b7
start prometheus before block processing 2021-01-21 16:01:33 -05:00
Jack Robison
e9c7cf6f63
logging 2021-01-21 14:51:59 -05:00
Jack Robison
68f1661452
add LRUCache with no prometheus metrics 2021-01-21 14:37:08 -05:00
Jack Robison
36fd1b91ae
cache server features 2021-01-21 14:37:08 -05:00
Jack Robison
a4ec430ac0
improve subscription performance 2021-01-21 14:37:08 -05:00
Lex Berezhny
bf0118c8ef added support for --not_channel_id to txo_list commands 2021-01-21 14:03:59 -05:00
Victor Shyba
b882f1a010 log checkpoint finishing for gathering information on wal/shm file bug 2020-12-31 16:11:22 -03:00
Jack Robison
0758b85179
skip null args given to channel_ids in claim search 2020-12-29 21:41:07 -05:00
Jack Robison
ab3d9bd080
block cache metrics 2020-12-29 21:40:39 -05:00
Jack Robison
9dd5159414
increase server cache sizes 2020-12-23 20:47:05 -05:00
Jack Robison
13e38d6fd8
use LRUCache instead of pylru in wallet server 2020-12-23 20:47:05 -05:00
Jack Robison
10dcb64715
lru cache metrics 2020-12-23 20:47:03 -05:00
Victor Shyba
adb418aafc group cache hits in a single dict 2020-12-23 21:37:20 -03:00
Victor Shyba
270da80d64 return whole batches 2020-12-23 19:10:19 -03:00
Victor Shyba
b2027cfd66 sync in batches of 10, clearing after 2020-12-23 18:30:52 -03:00
Jack Robison
7a7446c8bd
force resync blockchain.db 2020-12-22 20:08:26 -05:00
Jack Robison
8c71b744f3
fix request_synced_transactions edge cases 2020-12-22 15:19:08 -05:00
Jack Robison
4cbf4230e8
fix txi.txo_ref 2020-12-22 14:05:37 -05:00
Victor Shyba
6a610187e0 cache bypass 2020-12-22 13:01:30 -03:00
Victor Shyba
eb2a4aebba unrestricted and reusing verified cache hits 2020-12-21 16:49:08 -03:00
Victor Shyba
21a2e67755 fix rebase 2020-12-20 21:39:36 -03:00
Victor Shyba
3b9e312615 fix verification 2020-12-20 21:25:26 -03:00
Victor Shyba
26dab04c9e checkpoint 2020-12-20 21:25:26 -03:00
Victor Shyba
00713c0d11 asyncgens 2020-12-20 21:25:26 -03:00
Victor Shyba
751b5f3027 refactor duplicate code 2020-12-20 21:25:26 -03:00
Victor Shyba
e8261b000e wip, see jack 2020-12-20 21:23:58 -03:00
Victor Shyba
41ecb70297 join network can only happen after initial header sync returns 2020-12-20 21:15:11 -03:00
Jack Robison
e3a4964787
catch any remaining index errors in fs_tx_hash 2020-12-16 01:28:30 -05:00
Jack Robison
9bf72910a4
fix off by one error in reorg 2020-12-16 01:27:03 -05:00
Jack Robison
ee39e20e6d
logging 2020-12-16 01:26:19 -05:00
Jack Robison
0821ce44b5
remove unused heavy call 2020-12-15 19:22:12 -05:00
Jack Robison
674ce02e58
logging 2020-12-14 14:38:36 -05:00
Jack Robison
8dfa2767ec
new_touched 2020-12-14 13:52:26 -05:00
Jack Robison
20dad7f07f
only notify hashxs touched since last notification 2020-12-14 13:42:20 -05:00
Jack Robison
751cc4c44d
don't deserialize mempool in a thread 2020-12-13 13:48:50 -05:00
Jack Robison
2318e6d8e9
faster fs_transactions 2020-12-13 13:48:02 -05:00
Jack Robison
8da73ad3dd
improve hash_to_hex_str performance 2020-12-08 15:39:03 -05:00
Jack Robison
b8c16d8ac5
fix mempool txs being re-requested with each address notification 2020-12-08 15:39:03 -05:00
Jack Robison
eae0290978
fix mempool notification bloat 2020-12-07 13:16:37 -05:00
Jack Robison
3467d1fed0
fix tests 2020-12-03 22:16:28 -05:00
Jack Robison
d02ff232e5
_single_batch 2020-12-03 20:50:25 -05:00
Jack Robison
2d1c6a5402
fix 2020-12-03 19:15:15 -05:00
Jack Robison
eab3b65629
calculate notifications for all subscriptions instead of per session 2020-12-03 19:15:15 -05:00
Jack Robison
929617273d
resync blockchain.db 2020-12-03 15:56:35 -05:00
Jack Robison
2717bf7d49
increase default tx_cache_size setting 2020-12-03 15:56:35 -05:00
Jack Robison
9b4afe9816
store txids in memory, faster address subscription/history 2020-11-25 16:30:11 -05:00
Jack Robison
af1d7813e9
ensure transactions are returned in the order they're requested 2020-11-24 11:52:07 -05:00
Jack Robison
c02750edbd
fix tx_merkle 2020-11-23 16:48:24 -05:00
Jack Robison
faeba9a7e4
faster blockchain.transaction.get_batch 2020-11-23 12:05:47 -05:00
Jack Robison
190d238a1f
faster read_headers 2020-11-23 12:03:11 -05:00
Jack Robison
923d817751
store headers in memory 2020-11-20 22:37:42 -05:00
Victor Shyba
b8b9dcc2ee WAL_CHECKPOINT on close 2020-11-20 20:16:03 -03:00
Jack Robison
34f4f12eb9
pylint 2020-11-20 16:15:57 -05:00
Jack Robison
fa63bf758d
delete single_call_context, use session pool 2020-11-20 15:52:11 -05:00
Jack Robison
2c7fd58e34
threaded compress headers 2020-11-20 13:09:54 -05:00
Jack Robison
f2fd42b47a
use write_batch 2020-11-19 18:37:49 -05:00
Jack Robison
1b4ccad938
allow settings TRENDING_ALGORITHMS to nothing 2020-11-19 18:36:34 -05:00
Jack Robison
a9de1ce8e0
fix fs_transactions 2020-11-19 12:40:46 -05:00
Jack Robison
632d8d02d2
remove dead code 2020-11-17 16:17:24 -05:00
Jack Robison
48aeb26e02
threaded_get_merkle 
-remove dead code
 2020-11-17 16:05:08 -05:00
Jack Robison
1694af8b5e
get transactions from leveldb instead of lbrycrd rpc 2020-11-17 15:30:48 -05:00
Jack Robison
83bcab9cd2
cleanup 2020-11-17 15:30:48 -05:00
Jack Robison
bdc7f4b3f5
combine tx_count_db and hashes_db, add tx db 2020-11-17 15:30:48 -05:00
Victor Shyba
2619e162c1 recover invalid state described on #3026 2020-11-09 16:03:38 -05:00
Jack Robison
bbed9b94c1
null language filtering for claim search 2020-11-05 21:23:49 -05:00
Brendon J. Brewer
1cdff47477
Rewrite of variable_decay.py for speed improvements 2020-11-05 21:15:55 -05:00
Jack Robison
d83936a66a
fix uncaught error 2020-11-04 22:21:35 -05:00
Jack Robison
5517d2bf56
fix new_sdk_server arg for resolve 2020-11-03 16:23:31 -05:00
Jack Robison
925a458abe
tags 2020-11-03 10:39:08 -05:00
Jack Robison
76946c447f
use single_call_context for claim_search and resolve 2020-11-03 10:39:08 -05:00
Jack Robison
ab067d1d3a fix unclean closing of blockchain.db 2020-10-26 16:23:28 -04:00
Alex Grintsvayg
0a0ac3b7c9 pass-through for new support_sum api 2020-10-16 12:03:01 -04:00
Lex Berezhny
ec2f6c6b80 lint 2020-09-29 07:52:33 -04:00
Akinwale Ariwodola
838cc60161 conditional import 2020-09-29 07:52:33 -04:00
Akinwale Ariwodola
310c61a5cc check KIVY_BUILD env variable 2020-09-29 07:52:33 -04:00
Jack Robison
46dc15dd29
use language indexes 2020-09-28 14:45:51 -04:00
Jack Robison
8dc654b513
add language table and indexes 2020-09-28 14:45:51 -04:00
Jack Robison
7000ac3f3f
update indexed tags 2020-09-28 14:45:51 -04:00
Lex Berezhny
7b666efcf8 lbrycrd download path 2020-09-24 20:11:40 -04:00
Lex Berezhny
ec24ebf2cf added --limit_claims_per_channel argument to claim_search to only return up to the specified number of claims per channel 2020-08-19 10:51:31 -04:00
Lex Berezhny
9f1ed6e8c3 release_all_outputs on startup 2020-08-04 15:03:58 -04:00
Lex Berezhny
9c5f940b00 claim search forwarding to new sdk 2020-08-04 12:11:02 -04:00
Lex Berezhny
455b4043b8 new resolve 2020-08-04 11:33:39 -04:00
Lex Berezhny
8996aafe0d always cast modified_on to int() 2020-07-21 19:14:57 -04:00