8208 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Victor Shyba	920dad524a	simplify sync and use asyncio Queue instead	2021-03-19 19:58:13 -03:00
Victor Shyba	ec89bcac8e	improve sync script for no-downtime maintenance	2021-03-19 19:58:13 -03:00
Victor Shyba	a916c1f4ad	check if db file exists before sync	2021-03-19 19:58:13 -03:00
Victor Shyba	a9a0ac92d7	ignore unset flag	2021-03-19 19:58:13 -03:00
Victor Shyba	da8a8bd1ef	filter+fts and tests for edge cases	2021-03-19 19:58:13 -03:00
Victor Shyba	d9c746891d	pin python3.7	2021-03-19 19:58:13 -03:00
Victor Shyba	67817005b5	check ES synced without a process and wait for ES	2021-03-19 19:58:13 -03:00
Jack Robison	24d11de5a7	torba-elastic-sync	2021-03-19 19:58:13 -03:00
Victor Shyba	9251c87323	refresh after sync	2021-03-19 19:58:13 -03:00
Victor Shyba	e12fab90d1	docker compose update	2021-03-19 19:58:13 -03:00
Victor Shyba	0a194b5b01	claim_ids query	2021-03-19 19:58:13 -03:00
Victor Shyba	8d028adc53	be a writer by default	2021-03-19 19:58:13 -03:00
Victor Shyba	dfca15395e	claim id is also a keyword	2021-03-19 19:58:13 -03:00
Victor Shyba	e21f2362fe	apply reorg deletion as well	2021-03-19 19:58:13 -03:00
Victor Shyba	1ce328e8a9	cache signature inspection	2021-03-19 19:58:13 -03:00
Victor Shyba	038a5f999f	cache encoded headers	2021-03-19 19:58:13 -03:00
Victor Shyba	5d3704c7ea	reader mode	2021-03-19 19:58:13 -03:00
Victor Shyba	87037c06c9	remove reader code	2021-03-19 19:58:13 -03:00
Victor Shyba	dd412c0f50	delete sqlite fts	2021-03-19 19:58:13 -03:00
Victor Shyba	bf44befff6	backport fixes from server	2021-03-19 19:58:13 -03:00
Victor Shyba	e61874bb6f	only repeat search if it has blocked items	2021-03-19 19:58:13 -03:00
Victor Shyba	1e5331768f	fix some of the tests	2021-03-19 19:58:13 -03:00
Victor Shyba	ec9a3a4f7c	do not page filtered	2021-03-19 19:58:13 -03:00
Victor Shyba	e439a3a8dc	advanced resolve	2021-03-19 19:58:13 -03:00
Victor Shyba	19f70d7a11	create changelog trigger	2021-03-19 19:58:13 -03:00
Victor Shyba	afe7ed5b05	adjust size	2021-03-19 19:58:13 -03:00
Victor Shyba	d4bf004d74	use a thread pool to sync changes	2021-03-19 19:58:13 -03:00
Victor Shyba	e4d06a088b	include the channel being filtered/blocked	2021-03-19 19:58:13 -03:00
Victor Shyba	0929088b12	missing refresh step	2021-03-19 19:58:13 -03:00
Victor Shyba	7b4838fc9b	dont update more than 400 items a time	2021-03-19 19:58:13 -03:00
Victor Shyba	0cf9533248	narrow update by query	2021-03-19 19:58:13 -03:00
Victor Shyba	84ff0b8a9f	general timeout	2021-03-19 19:58:13 -03:00
Victor Shyba	d467dcfeaf	increase sync queue	2021-03-19 19:58:13 -03:00
Victor Shyba	8e68ba4751	fix join, refresh before update	2021-03-19 19:58:13 -03:00
Victor Shyba	0f2a85ba9f	simplify sync	2021-03-19 19:58:13 -03:00
Victor Shyba	7674a0a91e	backport fixes from testing server	2021-03-19 19:58:13 -03:00
Victor Shyba	5bc1a66572	32 slices and add censor type to fields	2021-03-19 19:58:13 -03:00
Victor Shyba	9b56067213	raise request timeout for content filtering	2021-03-19 19:58:13 -03:00
Victor Shyba	9a9df2fc3c	apply filtering only to whats unfiltered	2021-03-19 19:58:13 -03:00
Victor Shyba	9989d8d1d4	refresh after delete	2021-03-19 19:58:13 -03:00
Victor Shyba	f9471f297e	apply filter and block from ES script lang	2021-03-19 19:58:13 -03:00
Victor Shyba	146b693e4a	exclude title and description	2021-03-19 19:58:13 -03:00
Victor Shyba	7295b7e329	make sync parallel	2021-03-19 19:58:13 -03:00
Victor Shyba	e2441ea3e7	use prefix from ES docs	2021-03-19 19:58:13 -03:00
Victor Shyba	119e51912e	fix partial id	2021-03-19 19:58:13 -03:00
Victor Shyba	dd950f5b0d	tag can have empty space	2021-03-19 19:58:13 -03:00
Victor Shyba	78a9bad1e1	no indexer_task	2021-03-19 19:58:13 -03:00
Victor Shyba	0c6eaf5484	fix resolve partial id	2021-03-19 19:58:13 -03:00
Victor Shyba	1010068ddb	disable refresh interval. start with 3 shards	2021-03-19 19:58:13 -03:00
Victor Shyba	82eec3d8d7	use multiple clients on sync script indexing	2021-03-19 19:58:13 -03:00