510 commits

Author	SHA1	Message	Date
Jack Robison	54461dfa75	fix merge conflicts and simplify extract_doc	2021-10-05 16:44:49 -04:00
Jack Robison	2d48e93f74	fix bulk es sync	2021-10-05 16:44:49 -04:00
Jack Robison	af22646322	fix tests	2021-10-05 16:44:49 -04:00
Jack Robison	722b42a93e	fix tests	2021-10-05 16:44:49 -04:00
Jack Robison	8f9e7f77a7	handle invalid claim update	2021-10-05 16:44:49 -04:00
Jack Robison	09bb1ba494	fix keeping claim_hash_to_txo and txo_to_claim in sync	2021-10-05 16:44:49 -04:00
Victor Shyba	d4137428ff	implement blocking and filtering	2021-10-05 16:44:49 -04:00
Jack Robison	b4d6c4f5b7	fix _get_pending_claim_name	2021-10-05 16:44:49 -04:00
Jack Robison	ffbe59ece5	fix applying expiration fork	2021-10-05 16:44:49 -04:00
Jack Robison	fab9c90ccb	update iterators to use pack_partial_key	2021-10-05 16:44:49 -04:00
Jack Robison	fb1a774bc4	delete lbry/wallet/server/storage.py ... -expose leveldb lru cache size as `CACHE_MB` hub param	2021-10-05 16:44:49 -04:00
Jack Robison	98bc7d1e0e	remove dead code	2021-10-05 16:44:49 -04:00
Jack Robison	f7622f24b2	non blocking mempool loop	2021-10-05 16:44:49 -04:00
Jack Robison	f0a195a6d4	faster es sync	2021-10-05 16:44:49 -04:00
Jack Robison	180ba27d84	run advance_block in threadpool	2021-10-05 16:44:49 -04:00
Jack Robison	f944671f86	use claim_to_txo cache	2021-10-05 16:44:49 -04:00
Jack Robison	def2903f7d	faster _cached_get_active_amount for claims ... -remove dead code	2021-10-05 16:44:49 -04:00
Jack Robison	0273a4e839	fix claim search by fee for claims without fees	2021-10-05 16:44:49 -04:00
Jack Robison	f8d2f02c5d	clear claim_to_txo cache before reading	2021-10-05 16:44:49 -04:00
Jack Robison	25147d8897	handle claims that dont exist in ES sync	2021-10-05 16:44:49 -04:00
Jack Robison	0fb6f05fba	in memory claim_to_txo and txo_to_claim dictionaries	2021-10-05 16:44:49 -04:00
Jack Robison	4e4e899356	fix spend_utxo	2021-10-05 16:44:49 -04:00
Jack Robison	5a01dbf269	split flush from advance_block	2021-10-05 16:44:49 -04:00
Jack Robison	30b923b283	rename extend_ops	2021-10-05 16:44:49 -04:00
Jack Robison	73ba381d20	faster spend_utxo	2021-10-05 16:44:49 -04:00
Jack Robison	1a5912877e	faster get_future_activated	2021-10-05 16:44:49 -04:00
Jack Robison	813e506b68	threadpool	2021-10-05 16:44:49 -04:00
Jack Robison	077ca987f7	cleanup	2021-10-05 16:44:49 -04:00
Jack Robison	c632a7a6a5	fix getting block hash during reorg	2021-10-05 16:44:49 -04:00
Jack Robison	e33e767510	fix test	2021-10-05 16:44:49 -04:00
Jack Robison	ac82617aa9	fix spends in address histories	2021-10-05 16:44:49 -04:00
Jack Robison	a35dfd1fd1	faster es sync	2021-10-05 16:44:49 -04:00
Jack Robison	c28aae9913	fix expiring channels	2021-10-05 16:44:49 -04:00
Jack Robison	c26a99e65c	fix abandoning signed claims in the same tx as their channel ... -fix canonical/short url in es	2021-10-05 16:44:49 -04:00
Jack Robison	ca57dcfc2f	handle failure to generate a short id	2021-10-05 16:44:49 -04:00
Jack Robison	df5662dd69	fix resolve by short id	2021-10-05 16:44:49 -04:00
Jack Robison	1ac7831f3c	move MemPool into BlockProcessor	2021-10-05 16:44:49 -04:00
Jack Robison	292d272a94	combine MemPool and Notifications classes	2021-10-05 16:44:49 -04:00
Jack Robison	a6ee8dc66e	fix touched hashXs notifications	2021-10-05 16:44:49 -04:00
Jack Robison	496f89f184	reorg claims in the search index	2021-10-05 16:44:49 -04:00
Jack Robison	7a56eff1ac	small fixes	2021-10-05 16:44:49 -04:00
Jack Robison	07e182aa16	rename	2021-10-05 16:44:49 -04:00
Jack Robison	7de06aa1e0	delete stale code	2021-10-05 16:44:49 -04:00
Jack Robison	3955b64405	simplify advance and reorg	2021-10-05 16:44:49 -04:00
Jack Robison	2bb55d681d	update limited_history	2021-10-05 16:44:49 -04:00
Jack Robison	f94e6ac527	update lookup_utxos	2021-10-05 16:44:49 -04:00
Jack Robison	b344f17b86	update RevertableOpStack	2021-10-05 16:44:49 -04:00
Jack Robison	677b8cb633	add remaining db prefixes	2021-10-05 16:44:49 -04:00
Jack Robison	a1ddd762e0	cleanup	2021-10-05 16:44:49 -04:00
Jack Robison	68474e4057	skip es sync during initial hub sync, halt the hub upon finishing initial sync	2021-10-05 16:44:49 -04:00
Jack Robison	a84b9ee396	fix es sync	2021-10-05 16:44:49 -04:00
Jack Robison	b9c2ee745a	fix non localhost elasticsearch	2021-10-05 16:44:49 -04:00
Jack Robison	c68f9f6f16	fix signed claim invalidation corner cases	2021-10-05 16:44:49 -04:00
Jack Robison	229cb85a6a	extra deletes ... -the channel_to_claim/claim_to_channel entries already get deleted when the claim txo is spent	2021-10-05 16:44:49 -04:00
Jack Robison	e5c22fa665	fix has_no_source for reposts	2021-10-05 16:44:49 -04:00
Jack Robison	8bcfff05d7	update channel_to_claim and claim_to_channel at the same time	2021-10-05 16:44:49 -04:00
Jack Robison	6416ee8151	typing and fix error string	2021-10-05 16:44:49 -04:00
Jack Robison	f8eceb48e6	update staged txo_to_claim after invalidating channel sig ... -fixes abandon of claim with invalidated signature and an update in same block	2021-10-05 16:44:49 -04:00
Jack Robison	310c483bfa	missing channel_to_claim delete	2021-10-05 16:44:49 -04:00
Jack Robison	a8f20361aa	fix RepostKey	2021-10-05 16:44:49 -04:00
Jack Robison	290be69d99	typing	2021-10-05 16:44:49 -04:00
Jack Robison	3b96bd7ea0	fix	2021-10-05 16:44:49 -04:00
Jack Robison	dc2f22f5fa	cleanup	2021-10-05 16:44:49 -04:00
Jack Robison	821be29f41	rename effective_amount prefix	2021-10-05 16:44:49 -04:00
Jack Robison	52ff1a12ff	fix undeleted claim_to_channel record	2021-10-05 16:44:49 -04:00
Jack Robison	814699ef11	cleanup	2021-10-05 16:44:49 -04:00
Jack Robison	0c30838b25	fix mismatch in claim_to_txo<->txo_to_claim	2021-10-05 16:44:49 -04:00
Jack Robison	cf66c2a1ee	rename things ... -fix effective amount integrity error	2021-10-05 16:44:49 -04:00
Jack Robison	2ee419ffca	fix	2021-10-05 16:44:49 -04:00
Jack Robison	bfb9d696d7	pretty print	2021-10-05 16:44:49 -04:00
Jack Robison	bb2a34dd6b	fix duplicate activate	2021-10-05 16:44:49 -04:00
Jack Robison	ed652c0c56	fix updating resolve by effective amount after abandoning support	2021-10-05 16:44:49 -04:00
Jack Robison	1dc961d6eb	use RevertableOpStack in _get_takeover_ops	2021-10-05 16:44:49 -04:00
Jack Robison	d119fcfc98	remove debug prints	2021-10-05 16:44:49 -04:00
Jack Robison	4d3573724a	add RevertableOpStack to verify consistency of ops as they're staged	2021-10-05 16:44:49 -04:00
Jack Robison	8b37a66075	fix fee amount overflow in es	2021-10-05 16:44:49 -04:00
Jack Robison	ba4f32075a	faster claim producer ... -make batches of claim txos from the iterator, and sort by tx hash before fetching to maximize cache and read ahead hits	2021-10-05 16:44:49 -04:00
Jack Robison	218be22576	imports	2021-10-05 16:44:49 -04:00
Jack Robison	7688293716	close db in sync script	2021-10-05 16:44:49 -04:00
Jack Robison	458f8533c4	try default block size	2021-10-05 16:44:49 -04:00
Jack Robison	34502752fc	update elastic sync	2021-10-05 16:44:49 -04:00
Jack Robison	d6758fd823	invalidate channel signatures upon channel abandon	2021-10-05 16:44:49 -04:00
Jack Robison	65700e790e	_prepare_claim_for_sync generators	2021-10-05 16:44:49 -04:00
Jack Robison	7c34e4bb96	logging	2021-10-05 16:44:49 -04:00
Jack Robison	d0d6e3563b	use default sync=False during write_batch	2021-10-05 16:44:49 -04:00
Jack Robison	a2619f8c78	genesis_bytes attribute	2021-10-05 16:44:49 -04:00
Jack Robison	42d07fd2f0	fix	2021-10-05 16:44:49 -04:00
Jack Robison	8bea10960f	disable es (revert)	2021-10-05 16:44:49 -04:00
Jack Robison	9cbb19c304	_cached_get_active_amount	2021-10-05 16:44:49 -04:00
Jack Robison	1b94dfd712	fix removing unactivated support	2021-10-05 16:44:49 -04:00
Jack Robison	9f3604d739	debug	2021-10-05 16:44:49 -04:00
Jack Robison	4a1b2be269	leveldb tuning	2021-10-05 16:44:49 -04:00
Jack Robison	962dc1b55b	debug	2021-10-05 16:44:49 -04:00
Jack Robison	07c86502f6	refactor ClaimToTXO prefix	2021-10-05 16:44:49 -04:00
Jack Robison	adb188e5d0	filter abandoned claims from those considered for early activation	2021-10-05 16:44:49 -04:00
Jack Robison	ce031dc6b8	only do early takeover on a larger amount (fix case where they're equal)	2021-10-05 16:44:49 -04:00
Jack Robison	18b5f03247	filter supported claim hashes for claims that dont exist from early takeover/activations	2021-10-05 16:44:49 -04:00
Jack Robison	8a555ecf1c	remove extra open functions	2021-10-05 16:44:49 -04:00
Jack Robison	1b325b9acd	fix flush id	2021-10-05 16:44:49 -04:00
Jack Robison	1bdaddb319	fix clearing pending_support caches upon abandon	2021-10-05 16:44:49 -04:00