LBRYCommunity/lbry-sdk
1
0
Fork 1
Code Issues Pull requests Projects Releases 282 Packages Wiki Activity
8468 commits 110 branches 641 tags 31 MiB
Commit graph

1600 commits

Author SHA1 Message Date
Victor Shyba
8c4b0037f5 API: add --remove_duplicates to claim_search 2021-05-27 20:14:12 -03:00
Victor Shyba
ca28de02d8 test and implementation for remove_duplicates on post-search filtering 2021-05-27 20:14:12 -03:00
Victor Shyba
bfc15ea029 handle limit being 0 and skip reordering if 0/none 2021-05-27 20:14:12 -03:00
Victor Shyba
6e8b8a5920 always call search_ahead 2021-05-27 20:14:12 -03:00
Victor Shyba
142d182bc1 if progress was made, retry without a delay 2021-05-27 18:24:58 -03:00
Victor Shyba
1437871d88 fix reflector client: only set completed when server says so 2021-05-27 18:24:58 -03:00
Victor Shyba
352bf69409 improve test 2021-05-27 18:24:58 -03:00
Victor Shyba
9bdf3d23e1 test bug 3296, failing 2021-05-27 18:24:58 -03:00
Victor Shyba
be8ecfa707 sort keys so helper scripts can send blobs using send_request 2021-05-27 18:24:58 -03:00
Lex Berezhny
51da0d0259 v0.98.0 2021-05-26 09:23:19 -04:00
Alex Grin
e1a44c93f8
Merge branch 'master' into fix-collectionUpdateWithReplace 2021-05-18 15:25:40 -04:00
Alex Grin
2c79c7e2f6
Merge branch 'master' into patch-1 2021-05-18 15:23:08 -04:00
Victor Shyba
09f6637fe0 remove unused multiprocessin.Manager 2021-05-17 15:07:32 -03:00
zeppi
2b950ff5dd fix bug in collection_update --replace 2021-05-15 03:27:33 -03:00
Victor Shyba
ccadd88af5 fix cache call 2021-05-13 22:40:21 -03:00
Victor Shyba
cc02a0efc2 fix es migration bug, expand test case 2021-05-13 19:00:53 -03:00
Victor Shyba
43a1385b79 test sync helper 2021-05-13 19:00:53 -03:00
keikari
d9413039ec
Fix suggestion for issue #3240 
L135: If `getattr()` returns `None`, use `""` instead to avoid error in issue #3240
 2021-05-12 18:30:38 +03:00
Jack Robison
eba0c9be34
fix typo 2021-05-07 16:51:19 -04:00
Jack Robison
81ebde88db
resync ES search index on version bumps 
-bump ES search index to version 1
 2021-05-07 14:36:53 -04:00
Victor Shyba
a4058b84ce clean out unused sharding 2021-05-07 15:03:37 -03:00
Victor Shyba
7bf211a52b apply reposted_claim_type on es sync 2021-05-07 15:03:37 -03:00
Victor Shyba
d5f722792f fix and test has_source for channel reposts 2021-05-07 15:03:37 -03:00
Victor Shyba
0f02906c9b fix has_source for reposted channels 2021-05-07 15:03:37 -03:00
Victor Shyba
7554e6d7f9 remove dead code 2021-05-07 15:02:31 -03:00
Victor Shyba
cb8f26f177 remove broken feed 2021-05-07 15:02:31 -03:00
Jack Robison
b5dfce7861
Revert "finished switch from using hash # in URLs to colon :" 
This reverts commit 888aa558
 2021-05-07 11:31:28 -04:00
Jack Robison
2ca5a65544
Revert "FindShortestID updated" 
This reverts commit 8f04a50c
 2021-05-07 11:30:31 -04:00
Jack Robison
32f7ecb261
fix invalid claim fees breaking the block processor 2021-05-06 11:18:58 -04:00
Victor Shyba
fb77fde710 for debug, it is always whole page 2021-05-04 22:22:07 -03:00
Victor Shyba
3c67bb90d7 don't fail when a single one go on maintenance and set completion event regardless of failures 2021-05-04 22:22:07 -03:00
Victor Shyba
dabb168853 dont log full exceptions on simple connection errors 2021-05-04 22:22:07 -03:00
Victor Shyba
45e5b3b219 dont log full pages 2021-05-04 22:22:07 -03:00
Jack Robison
21d0038ff2
add timestamps to hub log 2021-04-28 16:47:00 -04:00
Jack Robison
c094d8f2e8
add ALLOW_LAN_UDP hub setting 2021-04-28 16:47:00 -04:00
Jack Robison
c465d6a6c2
ignore udp packets with low source ports 2021-04-28 16:47:00 -04:00
Lex Berezhny
73d35bc985 v0.97.0 2021-04-28 16:23:46 -04:00
Lex Berezhny
8f04a50ce1 FindShortestID updated 2021-04-28 16:21:01 -04:00
Lex Berezhny
888aa5586b finished switch from using hash # in URLs to colon : 2021-04-28 16:21:01 -04:00
Lex Berezhny
99f56f5d22 v0.96.0 2021-04-28 15:26:58 -04:00
Victor Shyba
f0d334d3e2 refactor from review 2021-04-28 13:28:38 -03:00
Victor Shyba
5f829b048f use separator to avoid cache key conflicts 2021-04-27 22:57:04 -03:00
Victor Shyba
1a961e66ff invalidate short_id cache on new block 2021-04-27 22:57:04 -03:00
Victor Shyba
fdb0e22656 cache search_ahead 2021-04-27 22:57:04 -03:00
Victor Shyba
44bf4f3c8f fix if statement from always evaluating a string 2021-04-27 17:10:04 -03:00
Lex Berezhny
b167c87267 v0.95.0 2021-04-23 14:55:38 -04:00
Lex Berezhny
2280fe8e8e default has_source to 1 2021-04-23 14:54:51 -04:00
Lex Berezhny
575d6dcd2d migration specifically for upgrading from client db v1.5 to v1.6 2021-04-23 14:54:51 -04:00
Lex Berezhny
f729490c6b pending claims ordered towards top in claim_list 2021-04-23 11:00:58 -04:00
Lex Berezhny
3d4321ee38 added --has_source/--has_no_source filters to claim_list 2021-04-23 10:24:48 -04:00
Lex Berezhny
5e3a234cbe v0.94.1 2021-04-16 11:18:24 -04:00
Lex Berezhny
e54c31d2d5 fix bug in how reserved balance is calculated 2021-04-16 11:17:51 -04:00
Lex Berezhny
3d241500cf v0.94.0 2021-04-14 19:55:35 -04:00
Lex Berezhny
f8814881a1 ability to set sd_hash, file_name and file_hash when updating a stream claim 2021-04-14 19:52:50 -04:00
Victor Shyba
cc2852cd48 new implementation for limit_claims_per_channel 2021-04-14 18:32:16 -04:00
Lex Berezhny
3cfc292d84 lint 2021-04-14 11:24:58 -04:00
Lex Berezhny
6acf94a810 moved balance calculation to SQL 2021-04-14 11:24:58 -04:00
Jack Robison
31367fb4c4 show hostnames of spvs 2021-04-13 11:51:27 -04:00
Jack Robison
12d6074e3b fix typing 2021-04-13 11:51:27 -04:00
Lex Berezhny
ff30386051 lint 2021-04-06 21:22:27 -04:00
shubhendra
601f99ac16 Remove unnecessary generator 
Signed-off-by: shubhendra <withshubh@gmail.com>
 2021-04-06 21:22:27 -04:00
shubhendra
68399ca31c Iterate dictionary directly 
Signed-off-by: shubhendra <withshubh@gmail.com>
 2021-04-06 21:22:27 -04:00
shubhendra
2a6d7fd80f Remove methods with unnecessary super delegation. 
Signed-off-by: shubhendra <withshubh@gmail.com>
 2021-04-06 21:22:27 -04:00
shubhendra
4725f510d8 Remove unnecessary use of comprehension 
Signed-off-by: shubhendra <withshubh@gmail.com>
 2021-04-06 21:22:27 -04:00
shubhendra
be0ba22222 Remove unnecessary comprehension 
Signed-off-by: shubhendra <withshubh@gmail.com>
 2021-04-06 21:22:27 -04:00
Lex Berezhny
c8781392be added unit test for Access-Control HTTP headers 2021-04-06 17:12:05 -04:00
John Leith
b97164fcfb adding access control headers 2021-04-06 17:12:05 -04:00
Lex Berezhny
0dfb92281b v0.93.0 2021-03-30 20:59:47 -04:00
Victor Shyba
46dd389d0d add elasticsearch service to gitlab 2021-03-30 17:00:15 -04:00
Jack Robison
7832c62c5d
add ELASTIC_HOST and ELASTIC_PORT settings to hub 2021-03-30 12:48:13 -04:00
Lex Berezhny
d025ee9dbe revert release 2021-03-30 11:29:17 -04:00
Lex Berezhny
a9a9cb4319 v0.93.0 2021-03-30 10:15:31 -04:00
Victor Shyba
aa727cb9b1 show channels regardless of no_source 2021-03-30 09:47:08 -04:00
Victor Shyba
b8c9a99f20 fix no_source for reposts 2021-03-30 09:47:08 -04:00
Lex Berezhny
aff995b0d0 temporary fix for mempool sync failing during reorg 2021-03-29 16:11:03 -04:00
Victor Shyba
5235a150b1 add prog name to sync arg parser 2021-03-24 17:07:17 -03:00
Victor Shyba
7df4cc44c4 fixes from review 2021-03-24 16:30:33 -03:00
Victor Shyba
7f5d88e95c remove dead/broken/unused API 2021-03-19 19:58:13 -03:00
Victor Shyba
d09663c066 remove flush call 2021-03-19 19:58:13 -03:00
Victor Shyba
ef97c9b69f torba-server -> hub 2021-03-19 19:58:13 -03:00
Victor Shyba
d855e6c8b1 move elasticsearch things into its own module 2021-03-19 19:58:13 -03:00
Victor Shyba
cd66f7eb43 if not no_totals, use default page size 2021-03-19 19:58:13 -03:00
Victor Shyba
a3e146dc68 sort on index time 2021-03-19 19:58:13 -03:00
Victor Shyba
b81305a4a9 index and allow has_source 2021-03-19 19:58:13 -03:00
Victor Shyba
73884b34bc apply no_totals 2021-03-19 19:58:13 -03:00
Victor Shyba
6166a34db2 check cache item before locking 2021-03-19 19:58:13 -03:00
Victor Shyba
6fa7da4b1c less slices 2021-03-19 19:58:13 -03:00
Victor Shyba
c3e426c491 fix search by channel for invalid channel 2021-03-19 19:58:13 -03:00
Victor Shyba
21e023f0db fix search by channel 2021-03-19 19:58:13 -03:00
Victor Shyba
063be001b3 cache inner parsing 2021-03-19 19:58:13 -03:00
Victor Shyba
5dff02e8bc on resolve, get all claims at once 2021-03-19 19:58:13 -03:00
Victor Shyba
60a59407d8 cache the encoded output instead 2021-03-19 19:58:13 -03:00
Victor Shyba
20a5aecfca fix lib exception to asyncio TimeoutError 2021-03-19 19:58:13 -03:00
Victor Shyba
c2e7b5a67d restore some of the interrupt metrics 2021-03-19 19:58:13 -03:00
Victor Shyba
8f32303d07 apply search timeout 2021-03-19 19:58:13 -03:00
Victor Shyba
891b1e7782 track results up to 200 2021-03-19 19:58:13 -03:00
Victor Shyba
f26394fd3b report deletions on docs that doesnt exist, but dont raise 2021-03-19 19:58:13 -03:00
Victor Shyba
4d83d42b4c fix equality instead of mod 2021-03-19 19:58:13 -03:00
Victor Shyba
57f1108df2 fix query being json serializable 2021-03-19 19:58:13 -03:00
Victor Shyba
2641a9abe5 make better resolve cache 2021-03-19 19:58:13 -03:00
Victor Shyba
6b193ab350 make indexing cooperative 2021-03-19 19:58:13 -03:00
Victor Shyba
b1bb37511c use right key on cache 2021-03-19 19:58:13 -03:00
Victor Shyba
319187d6d6 log mempool task exceptions 2021-03-19 19:58:13 -03:00
Victor Shyba
02eb789f84 caching for resolve 2021-03-19 19:58:13 -03:00
Victor Shyba
5a9338a27f use a dict on set_reference 2021-03-19 19:58:13 -03:00
Victor Shyba
eb6924277f round time to 10 minutes and fetch referenced by id 2021-03-19 19:58:13 -03:00
Victor Shyba
bd8f371fd5 bump referenced rows query limit up 2021-03-19 19:58:13 -03:00
Victor Shyba
1783ff2845 dont delete claims on reorg 2021-03-19 19:58:13 -03:00
Victor Shyba
d388527ffa log indexing errors 2021-03-19 19:58:13 -03:00
Victor Shyba
19494088bd generate from queue 2021-03-19 19:58:13 -03:00
Victor Shyba
920dad524a simplify sync and use asyncio Queue instead 2021-03-19 19:58:13 -03:00
Victor Shyba
ec89bcac8e improve sync script for no-downtime maintenance 2021-03-19 19:58:13 -03:00
Victor Shyba
a916c1f4ad check if db file exists before sync 2021-03-19 19:58:13 -03:00
Victor Shyba
a9a0ac92d7 ignore unset flag 2021-03-19 19:58:13 -03:00
Victor Shyba
da8a8bd1ef filter+fts and tests for edge cases 2021-03-19 19:58:13 -03:00
Victor Shyba
67817005b5 check ES synced without a process and wait for ES 2021-03-19 19:58:13 -03:00
Jack Robison
24d11de5a7 torba-elastic-sync 2021-03-19 19:58:13 -03:00
Victor Shyba
e12fab90d1 docker compose update 2021-03-19 19:58:13 -03:00
Victor Shyba
0a194b5b01 claim_ids query 2021-03-19 19:58:13 -03:00
Victor Shyba
8d028adc53 be a writer by default 2021-03-19 19:58:13 -03:00
Victor Shyba
dfca15395e claim id is also a keyword 2021-03-19 19:58:13 -03:00
Victor Shyba
e21f2362fe apply reorg deletion as well 2021-03-19 19:58:13 -03:00
Victor Shyba
1ce328e8a9 cache signature inspection 2021-03-19 19:58:13 -03:00
Victor Shyba
038a5f999f cache encoded headers 2021-03-19 19:58:13 -03:00
Victor Shyba
5d3704c7ea reader mode 2021-03-19 19:58:13 -03:00
Victor Shyba
87037c06c9 remove reader code 2021-03-19 19:58:13 -03:00
Victor Shyba
dd412c0f50 delete sqlite fts 2021-03-19 19:58:13 -03:00
Victor Shyba
bf44befff6 backport fixes from server 2021-03-19 19:58:13 -03:00
Victor Shyba
e61874bb6f only repeat search if it has blocked items 2021-03-19 19:58:13 -03:00
Victor Shyba
1e5331768f fix some of the tests 2021-03-19 19:58:13 -03:00
Victor Shyba
ec9a3a4f7c do not page filtered 2021-03-19 19:58:13 -03:00
Victor Shyba
e439a3a8dc advanced resolve 2021-03-19 19:58:13 -03:00
Victor Shyba
19f70d7a11 create changelog trigger 2021-03-19 19:58:13 -03:00
Victor Shyba
afe7ed5b05 adjust size 2021-03-19 19:58:13 -03:00
Victor Shyba
d4bf004d74 use a thread pool to sync changes 2021-03-19 19:58:13 -03:00
Victor Shyba
e4d06a088b include the channel being filtered/blocked 2021-03-19 19:58:13 -03:00
Victor Shyba
0929088b12 missing refresh step 2021-03-19 19:58:13 -03:00
Victor Shyba
7b4838fc9b dont update more than 400 items a time 2021-03-19 19:58:13 -03:00
Victor Shyba
0cf9533248 narrow update by query 2021-03-19 19:58:13 -03:00
Victor Shyba
84ff0b8a9f general timeout 2021-03-19 19:58:13 -03:00
Victor Shyba
d467dcfeaf increase sync queue 2021-03-19 19:58:13 -03:00
Victor Shyba
8e68ba4751 fix join, refresh before update 2021-03-19 19:58:13 -03:00
Victor Shyba
0f2a85ba9f simplify sync 2021-03-19 19:58:13 -03:00
Victor Shyba
7674a0a91e backport fixes from testing server 2021-03-19 19:58:13 -03:00
Victor Shyba
5bc1a66572 32 slices and add censor type to fields 2021-03-19 19:58:13 -03:00
Victor Shyba
9b56067213 raise request timeout for content filtering 2021-03-19 19:58:13 -03:00
Victor Shyba
9a9df2fc3c apply filtering only to whats unfiltered 2021-03-19 19:58:13 -03:00
Victor Shyba
9989d8d1d4 refresh after delete 2021-03-19 19:58:13 -03:00
Victor Shyba
f9471f297e apply filter and block from ES script lang 2021-03-19 19:58:13 -03:00
Victor Shyba
146b693e4a exclude title and description 2021-03-19 19:58:13 -03:00
Victor Shyba
e2441ea3e7 use prefix from ES docs 2021-03-19 19:58:13 -03:00
Victor Shyba
119e51912e fix partial id 2021-03-19 19:58:13 -03:00
Victor Shyba
dd950f5b0d tag can have empty space 2021-03-19 19:58:13 -03:00
Victor Shyba
78a9bad1e1 no indexer_task 2021-03-19 19:58:13 -03:00
Victor Shyba
0c6eaf5484 fix resolve partial id 2021-03-19 19:58:13 -03:00
Victor Shyba
1010068ddb disable refresh interval. start with 3 shards 2021-03-19 19:58:13 -03:00
Victor Shyba
ee7b37d3f3 also normalize the name supplied by user 2021-03-19 19:58:13 -03:00
Victor Shyba
143d82d242 normalized, not normalized_name 2021-03-19 19:58:13 -03:00
Victor Shyba
8b91b38855 update winners in one go 2021-03-19 19:58:13 -03:00
Victor Shyba
1098f0d2a3 use normalized name instead 2021-03-19 19:58:13 -03:00
Victor Shyba
ab53cec022 fix is_controlling sync 2021-03-19 19:58:13 -03:00
Victor Shyba
90106f5f08 all test_claim_commands tests green 2021-03-19 19:58:13 -03:00
Victor Shyba
9924b7b438 reposts and tag inheritance 2021-03-19 19:58:13 -03:00
Victor Shyba
aa37faab0a use porter analyzer with weights on full text search 2021-03-19 19:58:13 -03:00
Victor Shyba
dc10f8ce72 ignore errors when deleting 2021-03-19 19:58:13 -03:00
Victor Shyba
996686c1da claim search and resolve translated to ES queries 2021-03-19 19:58:13 -03:00
Victor Shyba
488785d013 add indexer task 2021-03-19 19:58:13 -03:00
Victor Shyba
3abdc01230 index ES during sync 2021-03-19 19:58:13 -03:00
Victor Shyba
8da04a584f start waiting before generate 2021-03-19 18:01:29 -03:00
Victor Shyba
27cc61d45e limit test time to 2 minutes, then consider it a failure and log what was running 2021-03-19 18:01:29 -03:00
Lex Berezhny
7371c30064 v0.92.0 2021-03-15 13:07:30 -04:00
Victor Shyba
74df4fab83 change column to has_source and document both flags 2021-03-14 10:11:42 -04:00
Victor Shyba
1e5cd3d7a1 typo, fix tests 2021-03-14 10:11:42 -04:00
Victor Shyba
a54e9b64aa add no_source claim_search filter 2021-03-14 10:11:42 -04:00
Victor Shyba
74660704e3 fix update 2021-03-14 10:11:42 -04:00
Victor Shyba
7439893a2a fix get for sourceless claims 2021-03-14 10:11:42 -04:00
Victor Shyba
e27e49e9dc call update only once 2021-03-14 10:11:42 -04:00
Victor Shyba
34ed729c59 there is no 'sd_hash' parameter for this API 2021-03-14 10:11:42 -04:00
Victor Shyba
adaeeca3fd let file_path be optional 2021-03-14 10:11:42 -04:00
Jack Robison
dac75563d3 add --no_file_path param to publish, stream_create, and stream_update 2021-03-14 10:11:42 -04:00
Alex Grintsvayg
cbc76adcaa only return unspent txos if is_spent flag is not used. fixes #2923 2021-03-13 06:44:20 -05:00
Lex Berezhny
4343073c00 clients can connect to wallet server even when they are not reachable by UDP 2021-03-12 13:29:55 -05:00
Victor Shyba
ae337807f5 get merkles outside thread cooperatively 2021-03-10 13:05:17 -03:00
Victor Shyba
9ae30ac08e during subscribe, hash address only when its time 2021-03-10 12:51:58 -03:00
Lex Berezhny
7bb873dad9 removed connection_status field from the status command, use wallet.connected instead to determine if SDK is connected 2021-03-09 13:27:36 -05:00
Lex Berezhny
5f6c1c14cb v0.91.0 2021-03-04 00:04:25 -05:00
Lex Berezhny
fcad76fc51 lint 2021-03-04 00:03:16 -05:00
Lex Berezhny
97e6e1684e simplifying 2021-03-04 00:03:16 -05:00
zeppi
183fb9f9ff provide --resolve tag for collection claim, separate from resolving its contents 
bugfix and docs generation

review changes
 2021-03-04 00:03:16 -05:00
Lex Berezhny
9815ddef1f fixes stalling client reconnect issue 2021-03-03 23:31:59 -05:00
Lex Berezhny
f6d0847453 v0.90.1 2021-03-01 11:57:40 -05:00
Lex Berezhny
0cec80f676 fixes transaction signing bug when tx had no change outputs 2021-03-01 11:20:08 -05:00
Patrick Keane
48c64143e3 Add no_totals argument to claim search 2021-02-26 10:41:15 -05:00
Patrick Keane
a8712422bc Update daemon.py 2021-02-21 23:45:18 -05:00
Lex Berezhny
fd3c1c50f1 v0.90.0 2021-02-16 11:13:14 -05:00
Lex Berezhny
b153e4bb9f added support to claim_search for filtering collections via --claim_type 2021-02-16 11:08:54 -05:00
Lex Berezhny
db9856a8db use median exchange rate when several exchange rates are available 2021-02-15 14:09:32 -05:00
Lex Berezhny
be8751cb73 integration test fixes 2021-02-10 22:14:13 -05:00
Lex Berezhny
f1cb7d27ac lint 2021-02-10 22:14:13 -05:00
Lex Berezhny
dee494e12f converting from USD, BTC to LBC is now done via several exchange rate providers: Bittrex, Cryptonator, CoinEx, hotbit and UPbit 2021-02-10 22:14:13 -05:00