883 commits

Author	SHA1	Message	Date
Jack Robison	7bb293e5d6	update claim_search doc ... backward compatibility for `trending_mixed`, `trending_local`, `trending_global`, and `trending_group` args to `claim_search`	2021-12-02 14:51:52 -05:00
Jack Robison	1aa66c6038	update header checkpoints	2021-12-01 18:46:24 -05:00
Victor Shyba	e7458edb72	test case for stream_type search on claims missing source + fix	2021-12-01 18:42:47 -05:00
Lex Berezhny	9e43060d41	fee per name env var	2021-12-01 10:22:34 -05:00
Jack Robison	7531401623	keep touched_or_deleted records	2021-11-21 13:52:03 -05:00
Brendon J. Brewer	3f6fe995b8	Rename trending	2021-11-16 10:59:10 -05:00
Jack Robison	1e00fb369d	fix missing es notification for support amount changing	2021-11-15 00:58:18 -05:00
Jack Robison	54b522383a	improve tests	2021-11-15 00:58:18 -05:00
Jack Robison	3fe1582432	fix duplicate trending notification to ES	2021-11-15 00:58:18 -05:00
Jack Robison	85eddd2100	fix effective amount for resolve/ES being off while claims/supports are unactivated	2021-11-15 00:58:18 -05:00
Victor Shyba	fbfd02b08b	add analytics event for network disk space	2021-11-09 14:27:06 -05:00
Jack Robison	ddbbb6f1dd	use mempool cache in transaction_get_batch	2021-10-27 20:19:08 -04:00
Jack Robison	1e391d211b	fix attempting to update trending on abandoned claims	2021-10-23 18:39:04 -04:00
Jack Robison	1a74d6604d	skip loading tx/claim caches in the elastic sync script when not needed	2021-10-22 15:10:35 -04:00
Jack Robison	48505c2968	update trending with help from @eggplantbren	2021-10-21 00:17:12 -04:00
Jack Robison	a98ea1e66a	update sync script to handle ES falling behind leveldb on shutdown	2021-10-20 23:41:11 -04:00
Jack Robison	3dec697816	logging	2021-10-20 23:41:11 -04:00
Jack Robison	b05d071a1c	update Env to accept parameters from cli args	2021-10-20 23:41:11 -04:00
Jack Robison	a27d3b9689	set default CACHE_MB to 1024mb and the default QUERY_TIMEOUT_MS to 10s	2021-10-20 23:41:11 -04:00
Jack Robison	1facc0cd01	remove unused hub env settings	2021-10-20 23:41:11 -04:00
Jack Robison	6bef09a3b1	update lbry-hub-elastic-sync to support resyncing recent blocks	2021-10-19 15:53:20 -04:00
Jack Robison	e35319e5a2	add CACHE_ALL_CLAIM_TXOS hub setting	2021-10-19 15:53:20 -04:00
Jack Robison	0e548b3812	remove dead code	2021-10-19 15:53:20 -04:00
Jack Robison	bfac02ccab	add CACHE_ALL_TX_HASHES setting to optionally use more memory to save i/o	2021-10-19 15:53:20 -04:00
Jack Robison	7ea1a2b361	sleeps	2021-10-19 15:53:20 -04:00
Jack Robison	99df418f1d	improve resolve caching	2021-10-19 15:53:20 -04:00
Jack Robison	6416d8ce9c	threadpools for block processor and es sync reader	2021-10-19 15:53:20 -04:00
Jack Robison	22b43a2b01	doc strings	2021-10-19 15:53:20 -04:00
Jack Robison	05e5d24c5e	improve claims_producer performance	2021-10-19 15:53:20 -04:00
Jack Robison	eabcc30367	resolve lru cache	2021-10-19 15:53:20 -04:00
Jack Robison	f5e0ef5223	add block_txs index	2021-10-19 15:53:20 -04:00
Jack Robison	f46d9330b0	smaller caches	2021-10-19 15:53:20 -04:00
Jack Robison	d3516f299e	clear es attributes during initial sync	2021-10-08 16:34:48 -04:00
Jack Robison	084a76d075	fix reposted channel being missing from resolve result ... -improve names of the resolve related methods in `LevelDB`	2021-10-07 15:09:13 -04:00
Jack Robison	43432a9e48	fix compactify script	2021-10-07 00:37:55 -04:00
Jack Robison	d64a5bc12f	fix test	2021-10-06 23:53:17 -04:00
Jack Robison	ccf03fc07b	only save undo info for blocks within reorg limit	2021-10-06 12:07:42 -04:00
Jack Robison	a7c45da10c	fix channel count	2021-10-06 00:02:16 -04:00
Jack Robison	8167af9b4a	sort touched or deleted claim hashes	2021-10-05 16:44:49 -04:00
Jack Robison	4cf76123e5	block processor db refactoring ... -access db through HubDB class, don't use plyvel.DB directly -add channel count and support amount prefixes	2021-10-05 16:44:49 -04:00
Jack Robison	01ee4b23e6	fix and add test for abandoning a controlling in the same block a new claim is made	2021-10-05 16:44:49 -04:00
Jack Robison	09db868a28	fix ES index name so it stays the same within a test case	2021-10-05 16:44:49 -04:00
Jack Robison	33e8ef75ff	fix bug with early takeover by an update	2021-10-05 16:44:49 -04:00
Jack Robison	86f21da28b	fix activating non existent claim	2021-10-05 16:44:49 -04:00
Jack Robison	89cd6a9aa4	add tests for takeovers from amount changes in updates before/on/after activation	2021-10-05 16:44:49 -04:00
Jack Robison	18e1256037	batch address history notifications	2021-10-05 16:44:49 -04:00
Jack Robison	02cf478d91	improve leveldb caching	2021-10-05 16:44:49 -04:00
Jack Robison	6ec70192fe	refactor reload_blocking_filtering_streams	2021-10-05 16:44:49 -04:00
Jack Robison	8c75098a9a	fix filtering error upon abandon	2021-10-05 16:44:49 -04:00
Jack Robison	72500f6948	faster read_claim_txos	2021-10-05 16:44:49 -04:00
Jack Robison	37ec9ab464	remove unused executor	2021-10-05 16:44:49 -04:00
Victor Shyba	82fe2a4c8d	fix blocking and filtering	2021-10-05 16:44:49 -04:00
Jack Robison	91a07cfaee	fix logging number of notified sessions	2021-10-05 16:44:49 -04:00
Jack Robison	709f5e9a65	fix update that initiates takeover not being delayed	2021-10-05 16:44:49 -04:00
Jack Robison	b2f9ef21cc	use hub binary from https://github.com/lbryio/hub/pull/13	2021-10-05 16:44:49 -04:00
Jack Robison	be6b72edcd	handle invalid release time	2021-10-05 16:44:49 -04:00
Jack Robison	ece2d1e78a	name and normalized -> claim_name and normalized_name ... -update generated pb files	2021-10-05 16:44:49 -04:00
Jack Robison	1ee1a5f2a1	fix es sync.py	2021-10-05 16:44:49 -04:00
Jack Robison	a567326853	fix all_claims_producer	2021-10-05 16:44:49 -04:00
Jack Robison	6231861dd6	merge conflicts	2021-10-05 16:44:49 -04:00
Jack Robison	1ff7b77ee0	claim search fixes	2021-10-05 16:44:49 -04:00
Jack Robison	9365708bb2	fix release_time and creation_timestamp	2021-10-05 16:44:49 -04:00
Jack Robison	d23a0a8589	delete unused code	2021-10-05 16:44:49 -04:00
Jack Robison	58ad1f3876	non blocking claim producer	2021-10-05 16:44:49 -04:00
Jack Robison	2138e7ea33	fix tests	2021-10-05 16:44:49 -04:00
Jack Robison	32f8c9e59f	renormalization	2021-10-05 16:44:49 -04:00
Jack Robison	3a16edd8a6	fix trending overflow	2021-10-05 16:44:49 -04:00
Jack Robison	165f3bb270	refactor trending	2021-10-05 16:44:49 -04:00
Jack Robison	0ba75153f3	trending fixes	2021-10-05 16:44:49 -04:00
Jack Robison	db2789990f	make app backward compatible with trending_score ... -update trending decay function to zero out low trending score values faster	2021-10-05 16:44:49 -04:00
Jack Robison	acaf299bcb	log time to update and decay trending in elasticsearch	2021-10-05 16:44:49 -04:00
Jack Robison	1940301824	skip integrity errors for trending spikes	2021-10-05 16:44:49 -04:00
Jack Robison	34576e880d	update trending in elasticsearch ... -add TrendingPrefixSpike to leveldb -expose `TRENDING_HALF_LIFE`, `TRENDING_WHALE_HALF_LIFE` and `TRENDING_WHALE_THRESHOLD` hub settings	2021-10-05 16:44:49 -04:00
Brendon J. Brewer	65c0668d40	constants	2021-10-05 16:44:49 -04:00
Brendon J. Brewer	53bd2bcbfe	Put trending score into ES	2021-10-05 16:44:49 -04:00
Brendon J. Brewer	388724fccb	Mark claims as touched	2021-10-05 16:44:49 -04:00
Jack Robison	231eabb013	fix non normalized canonical urls	2021-10-05 16:44:49 -04:00
Jack Robison	54903fc2ea	handle unicode error for unnormalized names	2021-10-05 16:44:49 -04:00
Jack Robison	3a1baf0700	prefix db	2021-10-05 16:44:49 -04:00
Brendon J. Brewer	0c0e36b6f8	trending	2021-10-05 16:44:49 -04:00
Jack Robison	234c03db09	fix claims not having non-normalized names	2021-10-05 16:44:49 -04:00
Jack Robison	28aa7da349	merge conflicts	2021-10-05 16:44:49 -04:00
Jack Robison	c51e344b87	fix missing fields in reposts	2021-10-05 16:44:49 -04:00
Jack Robison	54461dfa75	fix merge conflicts and simplify extract_doc	2021-10-05 16:44:49 -04:00
Jack Robison	2d48e93f74	fix bulk es sync	2021-10-05 16:44:49 -04:00
Jack Robison	af22646322	fix tests	2021-10-05 16:44:49 -04:00
Jack Robison	722b42a93e	fix tests	2021-10-05 16:44:49 -04:00
Jack Robison	8f9e7f77a7	handle invalid claim update	2021-10-05 16:44:49 -04:00
Jack Robison	09bb1ba494	fix keeping claim_hash_to_txo and txo_to_claim in sync	2021-10-05 16:44:49 -04:00
Victor Shyba	d4137428ff	implement blocking and filtering	2021-10-05 16:44:49 -04:00
Jack Robison	b4d6c4f5b7	fix _get_pending_claim_name	2021-10-05 16:44:49 -04:00
Jack Robison	ffbe59ece5	fix applying expiration fork	2021-10-05 16:44:49 -04:00
Jack Robison	fab9c90ccb	update iterators to use pack_partial_key	2021-10-05 16:44:49 -04:00
Jack Robison	fb1a774bc4	delete lbry/wallet/server/storage.py ... -expose leveldb lru cache size as `CACHE_MB` hub param	2021-10-05 16:44:49 -04:00
Jack Robison	98bc7d1e0e	remove dead code	2021-10-05 16:44:49 -04:00
Jack Robison	f7622f24b2	non blocking mempool loop	2021-10-05 16:44:49 -04:00
Jack Robison	f0a195a6d4	faster es sync	2021-10-05 16:44:49 -04:00
Jack Robison	180ba27d84	run advance_block in threadpool	2021-10-05 16:44:49 -04:00
Jack Robison	f944671f86	use claim_to_txo cache	2021-10-05 16:44:49 -04:00
Jack Robison	def2903f7d	faster _cached_get_active_amount for claims ... -remove dead code	2021-10-05 16:44:49 -04:00
Jack Robison	0273a4e839	fix claim search by fee for claims without fees	2021-10-05 16:44:49 -04:00
Jack Robison	f8d2f02c5d	clear claim_to_txo cache before reading	2021-10-05 16:44:49 -04:00
Jack Robison	25147d8897	handle claims that dont exist in ES sync	2021-10-05 16:44:49 -04:00
Jack Robison	0fb6f05fba	in memory claim_to_txo and txo_to_claim dictionaries	2021-10-05 16:44:49 -04:00
Jack Robison	4e4e899356	fix spend_utxo	2021-10-05 16:44:49 -04:00
Jack Robison	5a01dbf269	split flush from advance_block	2021-10-05 16:44:49 -04:00
Jack Robison	30b923b283	rename extend_ops	2021-10-05 16:44:49 -04:00
Jack Robison	73ba381d20	faster spend_utxo	2021-10-05 16:44:49 -04:00
Jack Robison	1a5912877e	faster get_future_activated	2021-10-05 16:44:49 -04:00
Jack Robison	813e506b68	threadpool	2021-10-05 16:44:49 -04:00
Jack Robison	077ca987f7	cleanup	2021-10-05 16:44:49 -04:00
Jack Robison	c632a7a6a5	fix getting block hash during reorg	2021-10-05 16:44:49 -04:00
Jack Robison	e33e767510	fix test	2021-10-05 16:44:49 -04:00
Jack Robison	ac82617aa9	fix spends in address histories	2021-10-05 16:44:49 -04:00
Jack Robison	a35dfd1fd1	faster es sync	2021-10-05 16:44:49 -04:00
Jack Robison	c28aae9913	fix expiring channels	2021-10-05 16:44:49 -04:00
Jack Robison	c26a99e65c	fix abandoning signed claims in the same tx as their channel ... -fix canonical/short url in es	2021-10-05 16:44:49 -04:00
Jack Robison	ca57dcfc2f	handle failure to generate a short id	2021-10-05 16:44:49 -04:00
Jack Robison	df5662dd69	fix resolve by short id	2021-10-05 16:44:49 -04:00
Jack Robison	1ac7831f3c	move MemPool into BlockProcessor	2021-10-05 16:44:49 -04:00
Jack Robison	292d272a94	combine MemPool and Notifications classes	2021-10-05 16:44:49 -04:00
Jack Robison	a6ee8dc66e	fix touched hashXs notifications	2021-10-05 16:44:49 -04:00
Jack Robison	496f89f184	reorg claims in the search index	2021-10-05 16:44:49 -04:00
Jack Robison	7a56eff1ac	small fixes	2021-10-05 16:44:49 -04:00
Jack Robison	07e182aa16	rename	2021-10-05 16:44:49 -04:00
Jack Robison	7de06aa1e0	delete stale code	2021-10-05 16:44:49 -04:00
Jack Robison	3955b64405	simplify advance and reorg	2021-10-05 16:44:49 -04:00
Jack Robison	2bb55d681d	update limited_history	2021-10-05 16:44:49 -04:00
Jack Robison	f94e6ac527	update lookup_utxos	2021-10-05 16:44:49 -04:00
Jack Robison	b344f17b86	update RevertableOpStack	2021-10-05 16:44:49 -04:00
Jack Robison	677b8cb633	add remaining db prefixes	2021-10-05 16:44:49 -04:00
Jack Robison	a1ddd762e0	cleanup	2021-10-05 16:44:49 -04:00
Jack Robison	68474e4057	skip es sync during initial hub sync, halt the hub upon finishing initial sync	2021-10-05 16:44:49 -04:00
Jack Robison	a84b9ee396	fix es sync	2021-10-05 16:44:49 -04:00
Jack Robison	b9c2ee745a	fix non localhost elasticsearch	2021-10-05 16:44:49 -04:00
Jack Robison	c68f9f6f16	fix signed claim invalidation corner cases	2021-10-05 16:44:49 -04:00
Jack Robison	229cb85a6a	extra deletes ... -the channel_to_claim/claim_to_channel entries already get deleted when the claim txo is spent	2021-10-05 16:44:49 -04:00
Jack Robison	e5c22fa665	fix has_no_source for reposts	2021-10-05 16:44:49 -04:00
Jack Robison	8bcfff05d7	update channel_to_claim and claim_to_channel at the same time	2021-10-05 16:44:49 -04:00
Jack Robison	6416ee8151	typing and fix error string	2021-10-05 16:44:49 -04:00
Jack Robison	f8eceb48e6	update staged txo_to_claim after invalidating channel sig ... -fixes abandon of claim with invalidated signature and an update in same block	2021-10-05 16:44:49 -04:00
Jack Robison	310c483bfa	missing channel_to_claim delete	2021-10-05 16:44:49 -04:00
Jack Robison	a8f20361aa	fix RepostKey	2021-10-05 16:44:49 -04:00
Jack Robison	290be69d99	typing	2021-10-05 16:44:49 -04:00
Jack Robison	3b96bd7ea0	fix	2021-10-05 16:44:49 -04:00
Jack Robison	dc2f22f5fa	cleanup	2021-10-05 16:44:49 -04:00
Jack Robison	821be29f41	rename effective_amount prefix	2021-10-05 16:44:49 -04:00
Jack Robison	52ff1a12ff	fix undeleted claim_to_channel record	2021-10-05 16:44:49 -04:00
Jack Robison	814699ef11	cleanup	2021-10-05 16:44:49 -04:00
Jack Robison	0c30838b25	fix mismatch in claim_to_txo<->txo_to_claim	2021-10-05 16:44:49 -04:00