Commit graph

794 commits

Author SHA1 Message Date
Victor Shyba
08d635322e implement blocking and filtering 2022-03-15 15:34:08 -03:00
Jack Robison
bc0c1b9a3e fix _get_pending_claim_name 2022-03-15 15:34:08 -03:00
Jack Robison
c42ee926da fix applying expiration fork 2022-03-15 15:34:08 -03:00
Jack Robison
c59d08080e update iterators to use pack_partial_key 2022-03-15 15:34:08 -03:00
Jack Robison
94e0624024 delete lbry/wallet/server/storage.py
-expose leveldb lru cache size as `CACHE_MB` hub param
2022-03-15 15:34:08 -03:00
Jack Robison
e94a2c7c94 remove dead code 2022-03-15 15:34:07 -03:00
Jack Robison
cd1ce32377 non blocking mempool loop 2022-03-15 15:34:07 -03:00
Jack Robison
d1560ef09b faster es sync 2022-03-15 15:34:07 -03:00
Jack Robison
6699d1e2f8 run advance_block in threadpool 2022-03-15 15:34:07 -03:00
Jack Robison
f1fbfa1b39 use claim_to_txo cache 2022-03-15 15:34:07 -03:00
Jack Robison
0d86717a9a faster _cached_get_active_amount for claims
-remove dead code
2022-03-15 15:34:07 -03:00
Jack Robison
528af27e4a fix claim search by fee for claims without fees 2022-03-15 15:34:07 -03:00
Jack Robison
fe69afaa56 clear claim_to_txo cache before reading 2022-03-15 15:34:07 -03:00
Jack Robison
31d7823498 handle claims that dont exist in ES sync 2022-03-15 15:34:07 -03:00
Jack Robison
ef6ec03161 in memory claim_to_txo and txo_to_claim dictionaries 2022-03-15 15:34:07 -03:00
Jack Robison
68596be1b9 fix spend_utxo 2022-03-15 15:34:07 -03:00
Jack Robison
a48fe84971 split flush from advance_block 2022-03-15 15:34:07 -03:00
Jack Robison
bfbe7c1bf5 rename extend_ops 2022-03-15 15:34:07 -03:00
Jack Robison
085ce1ff16 faster spend_utxo 2022-03-15 15:34:07 -03:00
Jack Robison
c81c0d9480 faster get_future_activated 2022-03-15 15:34:07 -03:00
Jack Robison
05b6bdb8f6 threadpool 2022-03-15 15:34:07 -03:00
Jack Robison
991d4f8859 cleanup 2022-03-15 15:34:07 -03:00
Jack Robison
22c75605ee fix getting block hash during reorg 2022-03-15 15:34:07 -03:00
Jack Robison
f52faa8d14 fix test 2022-03-15 15:34:07 -03:00
Jack Robison
8dd88a2780 fix spends in address histories 2022-03-15 15:34:07 -03:00
Jack Robison
7ad2234983 faster es sync 2022-03-15 15:34:07 -03:00
Jack Robison
fb5c008fc5 fix expiring channels 2022-03-15 15:34:07 -03:00
Jack Robison
f55b6bdc71 fix abandoning signed claims in the same tx as their channel
-fix canonical/short url in es
2022-03-15 15:34:07 -03:00
Jack Robison
de9edb0695 handle failure to generate a short id 2022-03-15 15:34:07 -03:00
Jack Robison
2180e24bc1 fix resolve by short id 2022-03-15 15:34:07 -03:00
Jack Robison
b4eaa5f918 move MemPool into BlockProcessor 2022-03-15 15:34:07 -03:00
Jack Robison
d4194954d3 combine MemPool and Notifications classes 2022-03-15 15:34:07 -03:00
Jack Robison
6e221fc7d9 fix touched hashXs notifications 2022-03-15 15:34:07 -03:00
Jack Robison
ea1285cd9f reorg claims in the search index 2022-03-15 15:34:07 -03:00
Jack Robison
4e77fa100b small fixes 2022-03-15 15:34:07 -03:00
Jack Robison
babb76d90d rename 2022-03-15 15:34:07 -03:00
Jack Robison
f73153ed8d delete stale code 2022-03-15 15:34:07 -03:00
Jack Robison
acfc1f56ee simplify advance and reorg 2022-03-15 15:34:07 -03:00
Jack Robison
81773a6497 update limited_history 2022-03-15 15:34:07 -03:00
Jack Robison
25cf751158 update lookup_utxos 2022-03-15 15:34:07 -03:00
Jack Robison
a9c8061c0c update RevertableOpStack 2022-03-15 15:34:07 -03:00
Jack Robison
f53b1ee290 add remaining db prefixes 2022-03-15 15:34:07 -03:00
Jack Robison
fd2753b95a cleanup 2022-03-15 15:34:07 -03:00
Jack Robison
768934e1cc skip es sync during initial hub sync, halt the hub upon finishing initial sync 2022-03-15 15:34:07 -03:00
Jack Robison
ad7dee3e7f fix es sync 2022-03-15 15:34:07 -03:00
Jack Robison
354c69bf4f fix non localhost elasticsearch 2022-03-15 15:34:07 -03:00
Jack Robison
0c85de7839 fix signed claim invalidation corner cases 2022-03-15 15:34:07 -03:00
Jack Robison
d74d06d97b extra deletes
-the channel_to_claim/claim_to_channel entries already get deleted when the claim txo is spent
2022-03-15 15:34:07 -03:00
Jack Robison
55351c5842 fix has_no_source for reposts 2022-03-15 15:34:07 -03:00
Jack Robison
e67152ec14 update channel_to_claim and claim_to_channel at the same time 2022-03-15 15:34:07 -03:00
Jack Robison
287ff0a557 typing and fix error string 2022-03-15 15:34:07 -03:00
Jack Robison
bf539d67ea update staged txo_to_claim after invalidating channel sig
-fixes abandon of claim with invalidated signature and an update in same block
2022-03-15 15:34:07 -03:00
Jack Robison
69651453dd missing channel_to_claim delete 2022-03-15 15:34:07 -03:00
Jack Robison
7f743ff3f1 fix RepostKey 2022-03-15 15:34:07 -03:00
Jack Robison
a1b7c61b56 typing 2022-03-15 15:34:07 -03:00
Jack Robison
fbd1d53542 fix 2022-03-15 15:34:07 -03:00
Jack Robison
02adc74e2c cleanup 2022-03-15 15:34:07 -03:00
Jack Robison
3a452a3b2a rename effective_amount prefix 2022-03-15 15:34:07 -03:00
Jack Robison
91f8b3b505 fix undeleted claim_to_channel record 2022-03-15 15:34:07 -03:00
Jack Robison
ce09f2eb0f cleanup 2022-03-15 15:34:07 -03:00
Jack Robison
02d479b92d fix mismatch in claim_to_txo<->txo_to_claim 2022-03-15 15:34:07 -03:00
Jack Robison
bce14c3d79 rename things
-fix effective amount integrity error
2022-03-15 15:34:07 -03:00
Jack Robison
90cfcaac40 fix 2022-03-15 15:34:07 -03:00
Jack Robison
02563db2c7 pretty print 2022-03-15 15:34:07 -03:00
Jack Robison
cfa32a3986 fix duplicate activate 2022-03-15 15:34:07 -03:00
Jack Robison
ab4a6bc5b8 fix updating resolve by effective amount after abandoning support 2022-03-15 15:34:07 -03:00
Jack Robison
f01b8c849d use RevertableOpStack in _get_takeover_ops 2022-03-15 15:34:07 -03:00
Jack Robison
6a46f50a35 remove debug prints 2022-03-15 15:34:07 -03:00
Jack Robison
468ed91ee3 add RevertableOpStack to verify consistency of ops as they're staged 2022-03-15 15:34:07 -03:00
Jack Robison
b856e2120a fix fee amount overflow in es 2022-03-15 15:34:07 -03:00
Jack Robison
fd7bfbea78 faster claim producer
-make batches of claim txos from the iterator, and sort by tx hash before fetching to maximize cache and read ahead hits
2022-03-15 15:34:07 -03:00
Jack Robison
a74434e269 imports 2022-03-15 15:34:07 -03:00
Jack Robison
92be63ff5b close db in sync script 2022-03-15 15:34:07 -03:00
Jack Robison
4ce16b6509 try default block size 2022-03-15 15:34:07 -03:00
Jack Robison
2341667d04 update elastic sync 2022-03-15 15:34:07 -03:00
Jack Robison
89e7c8582e invalidate channel signatures upon channel abandon 2022-03-15 15:34:07 -03:00
Jack Robison
da4e4ecd23 _prepare_claim_for_sync generators 2022-03-15 15:34:07 -03:00
Jack Robison
c85648d43b logging 2022-03-15 15:34:07 -03:00
Jack Robison
c90331af98 use default sync=False during write_batch 2022-03-15 15:34:07 -03:00
Jack Robison
e5461f6d4f genesis_bytes attribute 2022-03-15 15:34:07 -03:00
Jack Robison
ac82d6b27b fix 2022-03-15 15:34:07 -03:00
Jack Robison
01db974040 disable es (revert) 2022-03-15 15:34:07 -03:00
Jack Robison
6ae6cf733d _cached_get_active_amount 2022-03-15 15:34:07 -03:00
Jack Robison
ad1d25d945 fix removing unactivated support 2022-03-15 15:34:07 -03:00
Jack Robison
69ed47fc22 debug 2022-03-15 15:34:07 -03:00
Jack Robison
7c62654534 leveldb tuning 2022-03-15 15:34:07 -03:00
Jack Robison
531e6c1a61 debug 2022-03-15 15:34:07 -03:00
Jack Robison
066f797ad4 refactor ClaimToTXO prefix 2022-03-15 15:34:07 -03:00
Jack Robison
9f0611f3d9 filter abandoned claims from those considered for early activation 2022-03-15 15:34:07 -03:00
Jack Robison
b0d2efd613 only do early takeover on a larger amount (fix case where they're equal) 2022-03-15 15:34:07 -03:00
Jack Robison
073283a433 filter supported claim hashes for claims that dont exist from early takeover/activations 2022-03-15 15:34:07 -03:00
Jack Robison
5be04448ea remove extra open functions 2022-03-15 15:34:07 -03:00
Jack Robison
5541b80179 fix flush id 2022-03-15 15:34:07 -03:00
Jack Robison
35232b3650 fix clearing pending_support caches upon abandon 2022-03-15 15:34:07 -03:00
Jack Robison
f4ca3ea66b fix putting spent unactivated supports in removed_active_support 2022-03-15 15:34:07 -03:00
Jack Robison
306efa17cc fix syncing claim to es where channel is in the same block 2022-03-15 15:34:07 -03:00
Jack Robison
83107ad877 ignore activation for headless supports 2022-03-15 15:34:07 -03:00
Jack Robison
b0cf25bf5c faster get_future_activated 2022-03-15 15:34:07 -03:00
Jack Robison
6330424bcd debugging 2022-03-15 15:34:07 -03:00
Jack Robison
5f3850bfa2 prints 2022-03-15 15:34:07 -03:00
Jack Robison
82e6658483 flush count 2022-03-15 15:34:07 -03:00
Jack Robison
49f4add8d1 tests 2022-03-15 15:34:07 -03:00
Jack Robison
6ea96e79bd reposts 2022-03-15 15:34:07 -03:00
Jack Robison
9ad31008a5 fix updating the ES search index
-update search index to use ResolveResult tuples
2022-03-15 15:34:07 -03:00
Jack Robison
966f47a5b1 bid ordered resolve, feed ES claim data from block processor 2022-03-15 15:34:07 -03:00
Jack Robison
8711ece274 fix duplicate update op for early activating claim 2022-03-15 15:34:07 -03:00
Jack Robison
c0e2d56f55 fix early takeovers by not-yet activated claims 2022-03-15 15:34:07 -03:00
Jack Robison
2ba044ee4e require previous_winning arg for get_takeover_name_ops 2022-03-15 15:34:07 -03:00
Jack Robison
d69180da71 fix takeover edge case
if a claim with a higher value than that of a claim taking over a name exists but isn't yet activated, activate it early and have it take over the name
2022-03-15 15:34:07 -03:00
Jack Robison
b6e4cb9102 comments 2022-03-15 15:34:07 -03:00
Jack Robison
99d16fcb5a bid ordered resolve (WIP) 2022-03-15 15:34:07 -03:00
Jack Robison
96c318ee88 DBError 2022-03-15 15:34:07 -03:00
Jack Robison
c1ac4d8261 remove unused COIN file 2022-03-15 15:34:07 -03:00
Jack Robison
e678df86e0 claim takeovers 2022-03-15 15:34:07 -03:00
Jack Robison
6aa124592d move get_expiration_height and claimtrie constants to Coin class 2022-03-15 15:34:07 -03:00
Jack Robison
53ee3a5f80 claim activations and takeovers (WIP) 2022-03-15 15:34:07 -03:00
Jack Robison
8af410b184 advance_blocks -> advance_block 2022-03-15 15:34:07 -03:00
Jack Robison
8cb3fe8831 dead code 2022-03-15 15:34:07 -03:00
Jack Robison
2e92f3acad LBRYBlockProcessor -> BlockProcessor
- temporarily disable claim_search
2022-03-15 15:34:07 -03:00
Jack Robison
4e58094e4b rebase 2022-03-15 15:34:07 -03:00
Jack Robison
eb0eacd404 add wrapper for getnamesintrie
-used for verifying db state against lbrycrd
2022-03-15 15:34:07 -03:00
Jack Robison
1ff3ab4b83 db state struct
-remove dead code
2022-03-15 15:34:07 -03:00
Jack Robison
06841a4fde claim expiration 2022-03-15 15:34:07 -03:00
Jack Robison
d57cd5acd7 get_claim_by_claim_id 2022-03-15 15:34:07 -03:00
Jack Robison
b40cda78ee claims db
-move all leveldb prefixes to DB_PREFIXES enum
-add serializable RevertableOp interface for key/value puts and deletes
-resolve urls from leveldb
2022-03-15 15:34:07 -03:00
Jack Robison
0a833f5f83 transaction_num_mapping 2022-03-15 15:34:07 -03:00
Jack Robison
ef3bab16d3 disable sqlite in block processor 2022-03-15 15:34:07 -03:00
Jack Robison
3e826d0a5d named tuples 2022-03-15 15:34:07 -03:00
Jack Robison
bcd2c7d90b consolidate flush_backup 2022-03-15 15:34:07 -03:00
Jack Robison
eda1b0b3fc remove lbry.wallet.server.history 2022-03-15 15:34:07 -03:00
Jack Robison
bc7fe680c0 consolidate leveldb block advance/reorg
-move methods from History to LevelDB
2022-03-15 15:34:07 -03:00
Jack Robison
3ed748f2fd atomic flush_dbs 2022-03-15 15:34:07 -03:00
Jack Robison
9634753efd combine leveldb databases 2022-03-15 15:34:07 -03:00
Jack Robison
31df4f0bb5 Merkle staticmethods 2022-03-15 15:34:07 -03:00
Victor Shyba
c2dcc4c898 avoid [''] on peers list 2021-09-13 15:57:21 -03:00
Victor Shyba
5061a35e66 remove ignored output from hub node 2021-09-10 18:57:20 -03:00
Victor Shyba
cd9a1e8c9e default to legacy search for this release 2021-09-10 18:57:20 -03:00
Victor Shyba
b64aa51c0c fix stream_types being an integer 2021-09-10 18:57:20 -03:00
Victor Shyba
8206441834 run CI for old and new setups 2021-09-10 18:57:20 -03:00
Victor Shyba
57dffaa2ce update hub to beta release 2021-09-10 18:57:20 -03:00
Victor Shyba
9e81dd2360 refactor arguments fixup 2021-09-10 18:57:20 -03:00
Victor Shyba
1c31ec66f2 simplify operator handling 2021-09-10 18:57:20 -03:00
Victor Shyba
270192486a translate grpc errors to RPCError 2021-09-10 18:57:20 -03:00
Victor Shyba
a799503c97 update fields from hub 2021-09-10 18:57:20 -03:00
Victor Shyba
9685928087 there is no first_search 2021-09-10 18:57:20 -03:00
Victor Shyba
0e4b2fad99 specify index name 2021-09-10 18:57:20 -03:00
Jeffrey Picard
046147eb1d updates for fields 2021-09-10 18:57:20 -03:00
Jeffrey Picard
4544a074d9 Move the go hub settings from network to ledger config and hook reset
correctly.
2021-09-10 18:57:20 -03:00
Jeffrey Picard
9b78501392 Set default server to the networks default and use go hub by default 2021-09-10 18:57:20 -03:00
Jeffrey Picard
a4955a2b79 remove uneeded prints 2021-09-10 18:57:20 -03:00