Commit graph

841 commits

Author SHA1 Message Date
Lex Berezhny 8216f4a873 work in progress 2021-12-22 18:29:46 -05:00
Lex Berezhny e4cc4521d9 channel key generation no longer arbitrarily bounded 2021-12-22 18:29:46 -05:00
Lex Berezhny 6bd9b3744d progress, channel keys generate deterministically now 2021-12-22 18:29:46 -05:00
Lex Berezhny f741b00768 progress on deterministic channel keys 2021-12-22 18:29:46 -05:00
Jeffrey Picard 3436965b33 Debugging 2021-12-03 17:22:52 -05:00
Jeffrey Picard 1b322dc404
Update protobufs, go hub shim, and claim test. 2021-12-03 13:03:00 -05:00
Jack Robison 58341f4ff1
remove unused ES fields 2021-12-03 13:03:00 -05:00
Jack Robison 0d3ca80008
support lists of constraints for all range fields 2021-12-03 13:03:00 -05:00
Jack Robison 7bb293e5d6 update claim_search doc
backward compatibility for `trending_mixed`, `trending_local`, `trending_global`, and `trending_group` args to `claim_search`
2021-12-02 14:51:52 -05:00
Jack Robison 1aa66c6038
update header checkpoints 2021-12-01 18:46:24 -05:00
Victor Shyba e7458edb72 test case for stream_type search on claims missing source + fix 2021-12-01 18:42:47 -05:00
Lex Berezhny 9e43060d41 fee per name env var 2021-12-01 10:22:34 -05:00
Jack Robison 7531401623
keep touched_or_deleted records 2021-11-21 13:52:03 -05:00
Brendon J. Brewer 3f6fe995b8 Rename trending 2021-11-16 10:59:10 -05:00
Jack Robison 1e00fb369d fix missing es notification for support amount changing 2021-11-15 00:58:18 -05:00
Jack Robison 54b522383a improve tests 2021-11-15 00:58:18 -05:00
Jack Robison 3fe1582432 fix duplicate trending notification to ES 2021-11-15 00:58:18 -05:00
Jack Robison 85eddd2100 fix effective amount for resolve/ES being off while claims/supports are unactivated 2021-11-15 00:58:18 -05:00
Victor Shyba fbfd02b08b add analytics event for network disk space 2021-11-09 14:27:06 -05:00
Jack Robison ddbbb6f1dd
use mempool cache in transaction_get_batch 2021-10-27 20:19:08 -04:00
Jack Robison 1e391d211b
fix attempting to update trending on abandoned claims 2021-10-23 18:39:04 -04:00
Jack Robison 1a74d6604d skip loading tx/claim caches in the elastic sync script when not needed 2021-10-22 15:10:35 -04:00
Jack Robison 48505c2968 update trending with help from @eggplantbren 2021-10-21 00:17:12 -04:00
Jack Robison a98ea1e66a update sync script to handle ES falling behind leveldb on shutdown 2021-10-20 23:41:11 -04:00
Jack Robison 3dec697816 logging 2021-10-20 23:41:11 -04:00
Jack Robison b05d071a1c update Env to accept parameters from cli args 2021-10-20 23:41:11 -04:00
Jack Robison a27d3b9689 set default CACHE_MB to 1024mb and the default QUERY_TIMEOUT_MS to 10s 2021-10-20 23:41:11 -04:00
Jack Robison 1facc0cd01 remove unused hub env settings 2021-10-20 23:41:11 -04:00
Jack Robison 6bef09a3b1 update lbry-hub-elastic-sync to support resyncing recent blocks 2021-10-19 15:53:20 -04:00
Jack Robison e35319e5a2 add CACHE_ALL_CLAIM_TXOS hub setting 2021-10-19 15:53:20 -04:00
Jack Robison 0e548b3812 remove dead code 2021-10-19 15:53:20 -04:00
Jack Robison bfac02ccab add CACHE_ALL_TX_HASHES setting to optionally use more memory to save i/o 2021-10-19 15:53:20 -04:00
Jack Robison 7ea1a2b361 sleeps 2021-10-19 15:53:20 -04:00
Jack Robison 99df418f1d improve resolve caching 2021-10-19 15:53:20 -04:00
Jack Robison 6416d8ce9c threadpools for block processor and es sync reader 2021-10-19 15:53:20 -04:00
Jack Robison 22b43a2b01 doc strings 2021-10-19 15:53:20 -04:00
Jack Robison 05e5d24c5e improve claims_producer performance 2021-10-19 15:53:20 -04:00
Jack Robison eabcc30367 resolve lru cache 2021-10-19 15:53:20 -04:00
Jack Robison f5e0ef5223 add block_txs index 2021-10-19 15:53:20 -04:00
Jack Robison f46d9330b0 smaller caches 2021-10-19 15:53:20 -04:00
Jack Robison d3516f299e
clear es attributes during initial sync 2021-10-08 16:34:48 -04:00
Jack Robison 084a76d075
fix reposted channel being missing from resolve result
-improve names of the resolve related methods in `LevelDB`
2021-10-07 15:09:13 -04:00
Jack Robison 43432a9e48
fix compactify script 2021-10-07 00:37:55 -04:00
Jack Robison d64a5bc12f
fix test 2021-10-06 23:53:17 -04:00
Jack Robison ccf03fc07b
only save undo info for blocks within reorg limit 2021-10-06 12:07:42 -04:00
Jack Robison a7c45da10c
fix channel count 2021-10-06 00:02:16 -04:00
Jack Robison 8167af9b4a
sort touched or deleted claim hashes 2021-10-05 16:44:49 -04:00
Jack Robison 4cf76123e5
block processor db refactoring
-access db through HubDB class, don't use plyvel.DB directly
-add channel count and support amount prefixes
2021-10-05 16:44:49 -04:00
Jack Robison 01ee4b23e6
fix and add test for abandoning a controlling in the same block a new claim is made 2021-10-05 16:44:49 -04:00
Jack Robison 09db868a28
fix ES index name so it stays the same within a test case 2021-10-05 16:44:49 -04:00
Jack Robison 33e8ef75ff
fix bug with early takeover by an update 2021-10-05 16:44:49 -04:00
Jack Robison 86f21da28b
fix activating non existent claim 2021-10-05 16:44:49 -04:00
Jack Robison 89cd6a9aa4
add tests for takeovers from amount changes in updates before/on/after activation 2021-10-05 16:44:49 -04:00
Jack Robison 18e1256037
batch address history notifications 2021-10-05 16:44:49 -04:00
Jack Robison 02cf478d91
improve leveldb caching 2021-10-05 16:44:49 -04:00
Jack Robison 6ec70192fe
refactor reload_blocking_filtering_streams 2021-10-05 16:44:49 -04:00
Jack Robison 8c75098a9a
fix filtering error upon abandon 2021-10-05 16:44:49 -04:00
Jack Robison 72500f6948
faster read_claim_txos 2021-10-05 16:44:49 -04:00
Jack Robison 37ec9ab464
remove unused executor 2021-10-05 16:44:49 -04:00
Victor Shyba 82fe2a4c8d
fix blocking and filtering 2021-10-05 16:44:49 -04:00
Jack Robison 91a07cfaee
fix logging number of notified sessions 2021-10-05 16:44:49 -04:00
Jack Robison 709f5e9a65
fix update that initiates takeover not being delayed 2021-10-05 16:44:49 -04:00
Jack Robison b2f9ef21cc
use hub binary from https://github.com/lbryio/hub/pull/13 2021-10-05 16:44:49 -04:00
Jack Robison be6b72edcd
handle invalid release time 2021-10-05 16:44:49 -04:00
Jack Robison ece2d1e78a
name and normalized -> claim_name and normalized_name
-update generated pb files
2021-10-05 16:44:49 -04:00
Jack Robison 1ee1a5f2a1
fix es sync.py 2021-10-05 16:44:49 -04:00
Jack Robison a567326853
fix all_claims_producer 2021-10-05 16:44:49 -04:00
Jack Robison 6231861dd6
merge conflicts 2021-10-05 16:44:49 -04:00
Jack Robison 1ff7b77ee0
claim search fixes 2021-10-05 16:44:49 -04:00
Jack Robison 9365708bb2
fix release_time and creation_timestamp 2021-10-05 16:44:49 -04:00
Jack Robison d23a0a8589
delete unused code 2021-10-05 16:44:49 -04:00
Jack Robison 58ad1f3876
non blocking claim producer 2021-10-05 16:44:49 -04:00
Jack Robison 2138e7ea33
fix tests 2021-10-05 16:44:49 -04:00
Jack Robison 32f8c9e59f
renormalization 2021-10-05 16:44:49 -04:00
Jack Robison 3a16edd8a6
fix trending overflow 2021-10-05 16:44:49 -04:00
Jack Robison 165f3bb270
refactor trending 2021-10-05 16:44:49 -04:00
Jack Robison 0ba75153f3
trending fixes 2021-10-05 16:44:49 -04:00
Jack Robison db2789990f
make app backward compatible with trending_score
-update trending decay function to zero out low trending score values faster
2021-10-05 16:44:49 -04:00
Jack Robison acaf299bcb
log time to update and decay trending in elasticsearch 2021-10-05 16:44:49 -04:00
Jack Robison 1940301824
skip integrity errors for trending spikes 2021-10-05 16:44:49 -04:00
Jack Robison 34576e880d
update trending in elasticsearch
-add TrendingPrefixSpike to leveldb
-expose `TRENDING_HALF_LIFE`, `TRENDING_WHALE_HALF_LIFE` and `TRENDING_WHALE_THRESHOLD` hub settings
2021-10-05 16:44:49 -04:00
Brendon J. Brewer 65c0668d40
constants 2021-10-05 16:44:49 -04:00
Brendon J. Brewer 53bd2bcbfe
Put trending score into ES 2021-10-05 16:44:49 -04:00
Brendon J. Brewer 388724fccb
Mark claims as touched 2021-10-05 16:44:49 -04:00
Jack Robison 231eabb013
fix non normalized canonical urls 2021-10-05 16:44:49 -04:00
Jack Robison 54903fc2ea
handle unicode error for unnormalized names 2021-10-05 16:44:49 -04:00
Jack Robison 3a1baf0700
prefix db 2021-10-05 16:44:49 -04:00
Brendon J. Brewer 0c0e36b6f8
trending 2021-10-05 16:44:49 -04:00
Jack Robison 234c03db09
fix claims not having non-normalized names 2021-10-05 16:44:49 -04:00
Jack Robison 28aa7da349
merge conflicts 2021-10-05 16:44:49 -04:00
Jack Robison c51e344b87
fix missing fields in reposts 2021-10-05 16:44:49 -04:00
Jack Robison 54461dfa75
fix merge conflicts and simplify extract_doc 2021-10-05 16:44:49 -04:00
Jack Robison 2d48e93f74
fix bulk es sync 2021-10-05 16:44:49 -04:00
Jack Robison af22646322
fix tests 2021-10-05 16:44:49 -04:00
Jack Robison 722b42a93e
fix tests 2021-10-05 16:44:49 -04:00
Jack Robison 8f9e7f77a7
handle invalid claim update 2021-10-05 16:44:49 -04:00
Jack Robison 09bb1ba494
fix keeping claim_hash_to_txo and txo_to_claim in sync 2021-10-05 16:44:49 -04:00
Victor Shyba d4137428ff
implement blocking and filtering 2021-10-05 16:44:49 -04:00
Jack Robison b4d6c4f5b7
fix _get_pending_claim_name 2021-10-05 16:44:49 -04:00
Jack Robison ffbe59ece5
fix applying expiration fork 2021-10-05 16:44:49 -04:00