Commit graph

841 commits

Author SHA1 Message Date
Jack Robison a8f20361aa
fix RepostKey 2021-10-05 16:44:49 -04:00
Jack Robison 290be69d99
typing 2021-10-05 16:44:49 -04:00
Jack Robison 3b96bd7ea0
fix 2021-10-05 16:44:49 -04:00
Jack Robison dc2f22f5fa
cleanup 2021-10-05 16:44:49 -04:00
Jack Robison 821be29f41
rename effective_amount prefix 2021-10-05 16:44:49 -04:00
Jack Robison 52ff1a12ff
fix undeleted claim_to_channel record 2021-10-05 16:44:49 -04:00
Jack Robison 814699ef11
cleanup 2021-10-05 16:44:49 -04:00
Jack Robison 0c30838b25
fix mismatch in claim_to_txo<->txo_to_claim 2021-10-05 16:44:49 -04:00
Jack Robison cf66c2a1ee
rename things
-fix effective amount integrity error
2021-10-05 16:44:49 -04:00
Jack Robison 2ee419ffca
fix 2021-10-05 16:44:49 -04:00
Jack Robison bfb9d696d7
pretty print 2021-10-05 16:44:49 -04:00
Jack Robison bb2a34dd6b
fix duplicate activate 2021-10-05 16:44:49 -04:00
Jack Robison ed652c0c56
fix updating resolve by effective amount after abandoning support 2021-10-05 16:44:49 -04:00
Jack Robison 1dc961d6eb
use RevertableOpStack in _get_takeover_ops 2021-10-05 16:44:49 -04:00
Jack Robison d119fcfc98
remove debug prints 2021-10-05 16:44:49 -04:00
Jack Robison 4d3573724a
add RevertableOpStack to verify consistency of ops as they're staged 2021-10-05 16:44:49 -04:00
Jack Robison 8b37a66075
fix fee amount overflow in es 2021-10-05 16:44:49 -04:00
Jack Robison ba4f32075a
faster claim producer
-make batches of claim txos from the iterator, and sort by tx hash before fetching to maximize cache and read ahead hits
2021-10-05 16:44:49 -04:00
Jack Robison 218be22576
imports 2021-10-05 16:44:49 -04:00
Jack Robison 7688293716
close db in sync script 2021-10-05 16:44:49 -04:00
Jack Robison 458f8533c4
try default block size 2021-10-05 16:44:49 -04:00
Jack Robison 34502752fc
update elastic sync 2021-10-05 16:44:49 -04:00
Jack Robison d6758fd823
invalidate channel signatures upon channel abandon 2021-10-05 16:44:49 -04:00
Jack Robison 65700e790e
_prepare_claim_for_sync generators 2021-10-05 16:44:49 -04:00
Jack Robison 7c34e4bb96
logging 2021-10-05 16:44:49 -04:00
Jack Robison d0d6e3563b
use default sync=False during write_batch 2021-10-05 16:44:49 -04:00
Jack Robison a2619f8c78
genesis_bytes attribute 2021-10-05 16:44:49 -04:00
Jack Robison 42d07fd2f0
fix 2021-10-05 16:44:49 -04:00
Jack Robison 8bea10960f
disable es (revert) 2021-10-05 16:44:49 -04:00
Jack Robison 9cbb19c304
_cached_get_active_amount 2021-10-05 16:44:49 -04:00
Jack Robison 1b94dfd712
fix removing unactivated support 2021-10-05 16:44:49 -04:00
Jack Robison 9f3604d739
debug 2021-10-05 16:44:49 -04:00
Jack Robison 4a1b2be269
leveldb tuning 2021-10-05 16:44:49 -04:00
Jack Robison 962dc1b55b
debug 2021-10-05 16:44:49 -04:00
Jack Robison 07c86502f6
refactor ClaimToTXO prefix 2021-10-05 16:44:49 -04:00
Jack Robison adb188e5d0
filter abandoned claims from those considered for early activation 2021-10-05 16:44:49 -04:00
Jack Robison ce031dc6b8
only do early takeover on a larger amount (fix case where they're equal) 2021-10-05 16:44:49 -04:00
Jack Robison 18b5f03247
filter supported claim hashes for claims that dont exist from early takeover/activations 2021-10-05 16:44:49 -04:00
Jack Robison 8a555ecf1c
remove extra open functions 2021-10-05 16:44:49 -04:00
Jack Robison 1b325b9acd
fix flush id 2021-10-05 16:44:49 -04:00
Jack Robison 1bdaddb319
fix clearing pending_support caches upon abandon 2021-10-05 16:44:49 -04:00
Jack Robison 7896e177ef
fix putting spent unactivated supports in removed_active_support 2021-10-05 16:44:49 -04:00
Jack Robison ce8e659008
fix syncing claim to es where channel is in the same block 2021-10-05 16:44:49 -04:00
Jack Robison 27be5deeb2
ignore activation for headless supports 2021-10-05 16:44:49 -04:00
Jack Robison 515f270c3a
faster get_future_activated 2021-10-05 16:44:49 -04:00
Jack Robison ffff3bd334
debugging 2021-10-05 16:44:49 -04:00
Jack Robison f493f13b25
prints 2021-10-05 16:44:49 -04:00
Jack Robison e605c14b13
flush count 2021-10-05 16:44:49 -04:00
Jack Robison 338488f16d
tests 2021-10-05 16:44:49 -04:00
Jack Robison 2abc67c3e8
reposts 2021-10-05 16:44:49 -04:00
Jack Robison eb1ba143ec
fix updating the ES search index
-update search index to use ResolveResult tuples
2021-10-05 16:44:49 -04:00
Jack Robison 6f5bca0f67
bid ordered resolve, feed ES claim data from block processor 2021-10-05 16:44:49 -04:00
Jack Robison 407cd8dd4b
fix duplicate update op for early activating claim 2021-10-05 16:44:49 -04:00
Jack Robison 62a4f0fc04
fix early takeovers by not-yet activated claims 2021-10-05 16:44:49 -04:00
Jack Robison 3eb9d23108
require previous_winning arg for get_takeover_name_ops 2021-10-05 16:44:49 -04:00
Jack Robison 410d4aeb21
fix takeover edge case
if a claim with a higher value than that of a claim taking over a name exists but isn't yet activated, activate it early and have it take over the name
2021-10-05 16:44:49 -04:00
Jack Robison 0a28d216fd
comments 2021-10-05 16:44:49 -04:00
Jack Robison b69faf6920
bid ordered resolve (WIP) 2021-10-05 16:44:49 -04:00
Jack Robison e77f9981df
DBError 2021-10-05 16:44:49 -04:00
Jack Robison d27c2cc1e9
remove unused COIN file 2021-10-05 16:44:49 -04:00
Jack Robison 586b19675e
claim takeovers 2021-10-05 16:44:49 -04:00
Jack Robison f2907536b4
move get_expiration_height and claimtrie constants to Coin class 2021-10-05 16:44:49 -04:00
Jack Robison 9a11ac06bf
claim activations and takeovers (WIP) 2021-10-05 16:44:49 -04:00
Jack Robison aa3b18f848
advance_blocks -> advance_block 2021-10-05 16:44:49 -04:00
Jack Robison 103bdc151f
dead code 2021-10-05 16:44:49 -04:00
Jack Robison 6d4c1cd879
LBRYBlockProcessor -> BlockProcessor
- temporarily disable claim_search
2021-10-05 16:44:49 -04:00
Jack Robison cacbe30871
rebase 2021-10-05 16:44:49 -04:00
Jack Robison 04bb7b4919
add wrapper for getnamesintrie
-used for verifying db state against lbrycrd
2021-10-05 16:44:49 -04:00
Jack Robison b7df277a5c
db state struct
-remove dead code
2021-10-05 16:44:49 -04:00
Jack Robison c681041b48
claim expiration 2021-10-05 16:44:49 -04:00
Jack Robison 923834c784
get_claim_by_claim_id 2021-10-05 16:44:49 -04:00
Jack Robison 588edf98be
claims db
-move all leveldb prefixes to DB_PREFIXES enum
-add serializable RevertableOp interface for key/value puts and deletes
-resolve urls from leveldb
2021-10-05 16:44:49 -04:00
Jack Robison 28c603ad5f
transaction_num_mapping 2021-10-05 16:44:49 -04:00
Jack Robison 6988a47e02
disable sqlite in block processor 2021-10-05 16:44:49 -04:00
Jack Robison 2c8ceb1217
named tuples 2021-10-05 16:44:49 -04:00
Jack Robison ccac4ffa24
consolidate flush_backup 2021-10-05 16:44:49 -04:00
Jack Robison 4258cef9bd
remove lbry.wallet.server.history 2021-10-05 16:44:49 -04:00
Jack Robison 62cc6dfe76
consolidate leveldb block advance/reorg
-move methods from History to LevelDB
2021-10-05 16:44:49 -04:00
Jack Robison 9f224a971b
atomic flush_dbs 2021-10-05 16:44:49 -04:00
Jack Robison cf5dba9157
combine leveldb databases 2021-10-05 16:44:49 -04:00
Jack Robison 23035b9aa0
Merkle staticmethods 2021-10-05 16:44:49 -04:00
Victor Shyba c2dcc4c898 avoid [''] on peers list 2021-09-13 15:57:21 -03:00
Victor Shyba 5061a35e66 remove ignored output from hub node 2021-09-10 18:57:20 -03:00
Victor Shyba cd9a1e8c9e default to legacy search for this release 2021-09-10 18:57:20 -03:00
Victor Shyba b64aa51c0c fix stream_types being an integer 2021-09-10 18:57:20 -03:00
Victor Shyba 8206441834 run CI for old and new setups 2021-09-10 18:57:20 -03:00
Victor Shyba 57dffaa2ce update hub to beta release 2021-09-10 18:57:20 -03:00
Victor Shyba 9e81dd2360 refactor arguments fixup 2021-09-10 18:57:20 -03:00
Victor Shyba 1c31ec66f2 simplify operator handling 2021-09-10 18:57:20 -03:00
Victor Shyba 270192486a translate grpc errors to RPCError 2021-09-10 18:57:20 -03:00
Victor Shyba a799503c97 update fields from hub 2021-09-10 18:57:20 -03:00
Victor Shyba 9685928087 there is no first_search 2021-09-10 18:57:20 -03:00
Victor Shyba 0e4b2fad99 specify index name 2021-09-10 18:57:20 -03:00
Jeffrey Picard 046147eb1d updates for fields 2021-09-10 18:57:20 -03:00
Jeffrey Picard 4544a074d9 Move the go hub settings from network to ledger config and hook reset
correctly.
2021-09-10 18:57:20 -03:00
Jeffrey Picard 9b78501392 Set default server to the networks default and use go hub by default 2021-09-10 18:57:20 -03:00
Jeffrey Picard a4955a2b79 remove uneeded prints 2021-09-10 18:57:20 -03:00
Jeffrey Picard 15a56ca25e tons of small changes squashed together 2021-09-10 18:57:20 -03:00
Jeffrey Picard 9f65799a3d uncomment tests, add remove_duplicates param
Cleanup prints and commented out code

remove print

don't do list claims

cleanup
2021-09-10 18:57:20 -03:00
Jeffrey Picard a97fc6dba8 cleanup and reorgnazing some stuff
Fixing tests

relabel failing tests properly

run all the tests for the hub

cleanup HubNode
2021-09-10 18:57:20 -03:00
Jeffrey Picard c124e88d12 grpc client for python 2021-09-10 18:57:20 -03:00
Jeffrey Picard 17f3870296 Add tests for hub
Have the basic starting /stopping / querying. Still don't have the hub
jsonrpc stuff working right and from the looks of it I need to clearify
some of the logic in the claim search function itself because it's not
returning the correct number of claims anyways.

get the integration working with grpcurl

Got tests working, still need to port the rest of them

ported all of the claim search tests

still a few failing due to not having inflation working, and there's something weird
with limit_claims_per_channel that needs to be fixed.
2021-09-10 18:57:20 -03:00
Cristian Vicas 45bf6c3bf3 Drop comment_* apis.
Refactored dangling functions.
Added unit test.
2021-09-02 11:38:29 +03:00
Lex Berezhny dc1d9e1c84 upgrade pylint and fix lint errors 2021-08-20 22:36:35 -04:00
Jack Robison 2445c00c7e
raise WalletNotLoadedError in get_wallet_or_error instead of ValueError 2021-08-17 11:30:58 -04:00
Victor Shyba 21c71bfac1 update sync utility 2021-08-09 18:33:47 -03:00
Victor Shyba 6baaed3581 refactor query with new fields 2021-08-09 18:33:47 -03:00
Victor Shyba 152dbfd5d1 reflect fee_currency, fee_amount and duration on repost searches 2021-08-09 18:33:47 -03:00
Victor Shyba a56d14086b reflect media_type on repost searches 2021-08-09 18:33:47 -03:00
Victor Shyba aee87693f8 reflect stream_type on repost searches 2021-08-09 18:33:47 -03:00
Victor Shyba e222b6ad9c log that a invalid query happened 2021-08-09 15:07:44 -03:00
Victor Shyba 19b17374e8 throttle instead of disconnecting 2021-08-09 15:07:44 -03:00
Victor Shyba 43989122bb add error type and message to error readme and update code 2021-08-09 15:07:44 -03:00
Victor Shyba 72712d6047 raise and disconnect if too many parameters are used on search 2021-08-09 15:07:44 -03:00
Victor Shyba c591792de9 has_source is a special case 2021-07-22 16:25:55 -03:00
Victor Shyba 3108543ae5 3 missing fields 2021-07-22 16:25:55 -03:00
Victor Shyba 1eb221c743 translate reposted, signature_valid and normalized 2021-07-22 16:25:55 -03:00
Alex Grin bebf6bc2e7 Update constants.py 2021-07-22 16:25:55 -03:00
Alex Grin 9e91cc2138 Update constants.py 2021-07-22 16:25:55 -03:00
Victor Shyba 5bd411ca27 filtering hash->id 2021-07-22 16:25:55 -03:00
Victor Shyba a533cda6f0 ES: all _hash to _id 2021-07-22 16:25:55 -03:00
Jack Robison 9a8f9f0a94
fix stuck notification due to mempool/notification race 2021-07-20 15:14:10 -04:00
Victor Shyba 0ccafd5b53 make get_or_create_usable_address respect the generator lock 2021-07-19 14:09:52 -04:00
Victor Shyba 3de0982a4a limit request error logging to 16k 2021-07-07 18:39:38 -03:00
Victor Shyba c2184fb3bf run migration on history db open 2021-07-07 18:39:38 -03:00
Victor Shyba 919c09fcb0 add migration 2021-07-07 18:39:38 -03:00
Victor Shyba 1d9dbd40ec increase flush counter to 32 bits 2021-07-07 18:39:38 -03:00
Victor Shyba 309e957a85 add concurrent_hub_requests conf 2021-06-24 21:21:19 -03:00
Victor Shyba d7007e402e move request semaphore to session and apply to all requests 2021-06-24 21:02:41 -03:00
Victor Shyba 91323a21cf add hub_timeout and propagate it to network code 2021-06-24 21:02:41 -03:00
Lex Berezhny 485b958599 revert release 2021-06-22 12:50:11 -04:00
Lex Berezhny c39195488a bug fix 2021-06-22 11:07:58 -04:00
Lex Berezhny 0946c72b88 lint 2021-06-22 11:07:58 -04:00
Lex Berezhny 7d49b046d4 added support to config for determining if value is set and implemented hub selection logic 2021-06-22 11:07:58 -04:00
Lex Berezhny 5f0426c840 country jurisdiction added to hub UDP protocol 2021-06-22 11:07:58 -04:00
Lex Berezhny 73e239cc5f client side hub discovery pub/sub and hub metadata stored, removed old peers implementation 2021-06-22 11:07:58 -04:00
Lex Berezhny ad670f721a working client peer hub 2021-06-22 11:07:58 -04:00
Lex Berezhny 028a4a70cf wallet server federation, client portion 2021-06-22 11:07:58 -04:00
Lex Berezhny e457b2f0d6 fix trending to use built-in sqlite instead of apsw 2021-06-15 18:22:42 -04:00
Lex Berezhny c9cf7fd4d4 drop apsw in wallet.server.db.elasticsearch.sync 2021-06-15 18:22:42 -04:00
Lex Berezhny 25e16c3565 dropping apsw 2021-06-15 18:22:42 -04:00
Victor Shyba a638aa9d53 add and test support for support_create anonymous --comment 2021-06-02 12:05:36 -04:00
Victor Shyba 67cce0ef7e test+implement --comment for support_create 2021-06-02 12:05:36 -04:00
Lex Berezhny 464cfd475e properly format scripthash address on output 2021-06-02 11:34:21 -04:00
Lex Berezhny fe469ae57f create appropriate script for scripthash address 2021-06-02 11:34:21 -04:00
Lex Berezhny 550ef9a1c4 allows script addresses (beginning with r) to be used 2021-06-02 11:34:21 -04:00
Victor Shyba 3974df4a62 fix interaction between two modes 2021-05-27 20:14:12 -03:00
Victor Shyba ca28de02d8 test and implementation for remove_duplicates on post-search filtering 2021-05-27 20:14:12 -03:00
Victor Shyba bfc15ea029 handle limit being 0 and skip reordering if 0/none 2021-05-27 20:14:12 -03:00
Victor Shyba 6e8b8a5920 always call search_ahead 2021-05-27 20:14:12 -03:00
Victor Shyba 09f6637fe0 remove unused multiprocessin.Manager 2021-05-17 15:07:32 -03:00
Victor Shyba ccadd88af5 fix cache call 2021-05-13 22:40:21 -03:00
Victor Shyba cc02a0efc2 fix es migration bug, expand test case 2021-05-13 19:00:53 -03:00
Victor Shyba 43a1385b79 test sync helper 2021-05-13 19:00:53 -03:00
Jack Robison eba0c9be34
fix typo 2021-05-07 16:51:19 -04:00
Jack Robison 81ebde88db
resync ES search index on version bumps
-bump ES search index to version 1
2021-05-07 14:36:53 -04:00
Victor Shyba a4058b84ce clean out unused sharding 2021-05-07 15:03:37 -03:00
Victor Shyba 7bf211a52b apply reposted_claim_type on es sync 2021-05-07 15:03:37 -03:00
Victor Shyba d5f722792f fix and test has_source for channel reposts 2021-05-07 15:03:37 -03:00
Victor Shyba 0f02906c9b fix has_source for reposted channels 2021-05-07 15:03:37 -03:00
Jack Robison b5dfce7861
Revert "finished switch from using hash # in URLs to colon :"
This reverts commit 888aa558
2021-05-07 11:31:28 -04:00
Jack Robison 2ca5a65544
Revert "FindShortestID updated"
This reverts commit 8f04a50c
2021-05-07 11:30:31 -04:00
Jack Robison 32f7ecb261
fix invalid claim fees breaking the block processor 2021-05-06 11:18:58 -04:00
Jack Robison 21d0038ff2
add timestamps to hub log 2021-04-28 16:47:00 -04:00
Jack Robison c094d8f2e8
add ALLOW_LAN_UDP hub setting 2021-04-28 16:47:00 -04:00
Jack Robison c465d6a6c2
ignore udp packets with low source ports 2021-04-28 16:47:00 -04:00
Lex Berezhny 8f04a50ce1 FindShortestID updated 2021-04-28 16:21:01 -04:00
Lex Berezhny 888aa5586b finished switch from using hash # in URLs to colon : 2021-04-28 16:21:01 -04:00
Victor Shyba f0d334d3e2 refactor from review 2021-04-28 13:28:38 -03:00
Victor Shyba 5f829b048f use separator to avoid cache key conflicts 2021-04-27 22:57:04 -03:00
Victor Shyba 1a961e66ff invalidate short_id cache on new block 2021-04-27 22:57:04 -03:00
Victor Shyba fdb0e22656 cache search_ahead 2021-04-27 22:57:04 -03:00
Victor Shyba 44bf4f3c8f fix if statement from always evaluating a string 2021-04-27 17:10:04 -03:00
Lex Berezhny 2280fe8e8e default has_source to 1 2021-04-23 14:54:51 -04:00
Lex Berezhny 575d6dcd2d migration specifically for upgrading from client db v1.5 to v1.6 2021-04-23 14:54:51 -04:00
Lex Berezhny f729490c6b pending claims ordered towards top in claim_list 2021-04-23 11:00:58 -04:00
Lex Berezhny 3d4321ee38 added --has_source/--has_no_source filters to claim_list 2021-04-23 10:24:48 -04:00
Lex Berezhny e54c31d2d5 fix bug in how reserved balance is calculated 2021-04-16 11:17:51 -04:00
Victor Shyba cc2852cd48 new implementation for limit_claims_per_channel 2021-04-14 18:32:16 -04:00
Lex Berezhny 3cfc292d84 lint 2021-04-14 11:24:58 -04:00
Lex Berezhny 6acf94a810 moved balance calculation to SQL 2021-04-14 11:24:58 -04:00
Jack Robison 31367fb4c4 show hostnames of spvs 2021-04-13 11:51:27 -04:00
Jack Robison 12d6074e3b fix typing 2021-04-13 11:51:27 -04:00
shubhendra 601f99ac16 Remove unnecessary generator
Signed-off-by: shubhendra <withshubh@gmail.com>
2021-04-06 21:22:27 -04:00
shubhendra 68399ca31c Iterate dictionary directly
Signed-off-by: shubhendra <withshubh@gmail.com>
2021-04-06 21:22:27 -04:00
shubhendra 2a6d7fd80f Remove methods with unnecessary super delegation.
Signed-off-by: shubhendra <withshubh@gmail.com>
2021-04-06 21:22:27 -04:00
shubhendra 4725f510d8 Remove unnecessary use of comprehension
Signed-off-by: shubhendra <withshubh@gmail.com>
2021-04-06 21:22:27 -04:00
shubhendra be0ba22222 Remove unnecessary comprehension
Signed-off-by: shubhendra <withshubh@gmail.com>
2021-04-06 21:22:27 -04:00
Victor Shyba 46dd389d0d add elasticsearch service to gitlab 2021-03-30 17:00:15 -04:00
Jack Robison 7832c62c5d
add ELASTIC_HOST and ELASTIC_PORT settings to hub 2021-03-30 12:48:13 -04:00
Victor Shyba aa727cb9b1 show channels regardless of no_source 2021-03-30 09:47:08 -04:00
Victor Shyba b8c9a99f20 fix no_source for reposts 2021-03-30 09:47:08 -04:00
Lex Berezhny aff995b0d0 temporary fix for mempool sync failing during reorg 2021-03-29 16:11:03 -04:00
Victor Shyba 5235a150b1 add prog name to sync arg parser 2021-03-24 17:07:17 -03:00
Victor Shyba 7df4cc44c4 fixes from review 2021-03-24 16:30:33 -03:00
Victor Shyba 7f5d88e95c remove dead/broken/unused API 2021-03-19 19:58:13 -03:00
Victor Shyba d09663c066 remove flush call 2021-03-19 19:58:13 -03:00
Victor Shyba ef97c9b69f torba-server -> hub 2021-03-19 19:58:13 -03:00
Victor Shyba d855e6c8b1 move elasticsearch things into its own module 2021-03-19 19:58:13 -03:00
Victor Shyba cd66f7eb43 if not no_totals, use default page size 2021-03-19 19:58:13 -03:00
Victor Shyba a3e146dc68 sort on index time 2021-03-19 19:58:13 -03:00
Victor Shyba b81305a4a9 index and allow has_source 2021-03-19 19:58:13 -03:00
Victor Shyba 73884b34bc apply no_totals 2021-03-19 19:58:13 -03:00
Victor Shyba 6166a34db2 check cache item before locking 2021-03-19 19:58:13 -03:00
Victor Shyba 6fa7da4b1c less slices 2021-03-19 19:58:13 -03:00
Victor Shyba c3e426c491 fix search by channel for invalid channel 2021-03-19 19:58:13 -03:00
Victor Shyba 21e023f0db fix search by channel 2021-03-19 19:58:13 -03:00
Victor Shyba 063be001b3 cache inner parsing 2021-03-19 19:58:13 -03:00
Victor Shyba 5dff02e8bc on resolve, get all claims at once 2021-03-19 19:58:13 -03:00
Victor Shyba 60a59407d8 cache the encoded output instead 2021-03-19 19:58:13 -03:00
Victor Shyba 20a5aecfca fix lib exception to asyncio TimeoutError 2021-03-19 19:58:13 -03:00
Victor Shyba c2e7b5a67d restore some of the interrupt metrics 2021-03-19 19:58:13 -03:00
Victor Shyba 8f32303d07 apply search timeout 2021-03-19 19:58:13 -03:00
Victor Shyba 891b1e7782 track results up to 200 2021-03-19 19:58:13 -03:00
Victor Shyba f26394fd3b report deletions on docs that doesnt exist, but dont raise 2021-03-19 19:58:13 -03:00
Victor Shyba 4d83d42b4c fix equality instead of mod 2021-03-19 19:58:13 -03:00
Victor Shyba 57f1108df2 fix query being json serializable 2021-03-19 19:58:13 -03:00
Victor Shyba 2641a9abe5 make better resolve cache 2021-03-19 19:58:13 -03:00
Victor Shyba 6b193ab350 make indexing cooperative 2021-03-19 19:58:13 -03:00
Victor Shyba b1bb37511c use right key on cache 2021-03-19 19:58:13 -03:00
Victor Shyba 319187d6d6 log mempool task exceptions 2021-03-19 19:58:13 -03:00
Victor Shyba 02eb789f84 caching for resolve 2021-03-19 19:58:13 -03:00
Victor Shyba eb6924277f round time to 10 minutes and fetch referenced by id 2021-03-19 19:58:13 -03:00
Victor Shyba bd8f371fd5 bump referenced rows query limit up 2021-03-19 19:58:13 -03:00
Victor Shyba 1783ff2845 dont delete claims on reorg 2021-03-19 19:58:13 -03:00
Victor Shyba d388527ffa log indexing errors 2021-03-19 19:58:13 -03:00
Victor Shyba 19494088bd generate from queue 2021-03-19 19:58:13 -03:00
Victor Shyba 920dad524a simplify sync and use asyncio Queue instead 2021-03-19 19:58:13 -03:00
Victor Shyba ec89bcac8e improve sync script for no-downtime maintenance 2021-03-19 19:58:13 -03:00
Victor Shyba a916c1f4ad check if db file exists before sync 2021-03-19 19:58:13 -03:00
Victor Shyba a9a0ac92d7 ignore unset flag 2021-03-19 19:58:13 -03:00
Victor Shyba da8a8bd1ef filter+fts and tests for edge cases 2021-03-19 19:58:13 -03:00
Victor Shyba 67817005b5 check ES synced without a process and wait for ES 2021-03-19 19:58:13 -03:00
Jack Robison 24d11de5a7 torba-elastic-sync 2021-03-19 19:58:13 -03:00
Victor Shyba e12fab90d1 docker compose update 2021-03-19 19:58:13 -03:00
Victor Shyba 0a194b5b01 claim_ids query 2021-03-19 19:58:13 -03:00
Victor Shyba 8d028adc53 be a writer by default 2021-03-19 19:58:13 -03:00
Victor Shyba dfca15395e claim id is also a keyword 2021-03-19 19:58:13 -03:00
Victor Shyba e21f2362fe apply reorg deletion as well 2021-03-19 19:58:13 -03:00
Victor Shyba 1ce328e8a9 cache signature inspection 2021-03-19 19:58:13 -03:00
Victor Shyba 038a5f999f cache encoded headers 2021-03-19 19:58:13 -03:00
Victor Shyba 5d3704c7ea reader mode 2021-03-19 19:58:13 -03:00
Victor Shyba 87037c06c9 remove reader code 2021-03-19 19:58:13 -03:00
Victor Shyba dd412c0f50 delete sqlite fts 2021-03-19 19:58:13 -03:00
Victor Shyba bf44befff6 backport fixes from server 2021-03-19 19:58:13 -03:00
Victor Shyba e61874bb6f only repeat search if it has blocked items 2021-03-19 19:58:13 -03:00
Victor Shyba ec9a3a4f7c do not page filtered 2021-03-19 19:58:13 -03:00
Victor Shyba e439a3a8dc advanced resolve 2021-03-19 19:58:13 -03:00
Victor Shyba 19f70d7a11 create changelog trigger 2021-03-19 19:58:13 -03:00
Victor Shyba afe7ed5b05 adjust size 2021-03-19 19:58:13 -03:00