LBRYCommunity/lbry-sdk
1
0
Fork 1
Code Issues Pull requests Projects Releases 282 Packages Wiki Activity
8759 commits 110 branches 641 tags 31 MiB
Commit graph

1852 commits

Author SHA1 Message Date
Jack Robison
34576e880d
update trending in elasticsearch 
-add TrendingPrefixSpike to leveldb
-expose `TRENDING_HALF_LIFE`, `TRENDING_WHALE_HALF_LIFE` and `TRENDING_WHALE_THRESHOLD` hub settings
 2021-10-05 16:44:49 -04:00
Brendon J. Brewer
65c0668d40
constants 2021-10-05 16:44:49 -04:00
Brendon J. Brewer
53bd2bcbfe
Put trending score into ES 2021-10-05 16:44:49 -04:00
Brendon J. Brewer
388724fccb
Mark claims as touched 2021-10-05 16:44:49 -04:00
Jack Robison
231eabb013
fix non normalized canonical urls 2021-10-05 16:44:49 -04:00
Jack Robison
54903fc2ea
handle unicode error for unnormalized names 2021-10-05 16:44:49 -04:00
Jack Robison
3a1baf0700
prefix db 2021-10-05 16:44:49 -04:00
Brendon J. Brewer
0c0e36b6f8
trending 2021-10-05 16:44:49 -04:00
Jack Robison
234c03db09
fix claims not having non-normalized names 2021-10-05 16:44:49 -04:00
Jack Robison
28aa7da349
merge conflicts 2021-10-05 16:44:49 -04:00
Jack Robison
c51e344b87
fix missing fields in reposts 2021-10-05 16:44:49 -04:00
Jack Robison
54461dfa75
fix merge conflicts and simplify extract_doc 2021-10-05 16:44:49 -04:00
Jack Robison
2d48e93f74
fix bulk es sync 2021-10-05 16:44:49 -04:00
Jack Robison
af22646322
fix tests 2021-10-05 16:44:49 -04:00
Jack Robison
722b42a93e
fix tests 2021-10-05 16:44:49 -04:00
Jack Robison
8f9e7f77a7
handle invalid claim update 2021-10-05 16:44:49 -04:00
Jack Robison
09bb1ba494
fix keeping claim_hash_to_txo and txo_to_claim in sync 2021-10-05 16:44:49 -04:00
Victor Shyba
d4137428ff
implement blocking and filtering 2021-10-05 16:44:49 -04:00
Jack Robison
b4d6c4f5b7
fix _get_pending_claim_name 2021-10-05 16:44:49 -04:00
Jack Robison
ffbe59ece5
fix applying expiration fork 2021-10-05 16:44:49 -04:00
Jack Robison
fab9c90ccb
update iterators to use pack_partial_key 2021-10-05 16:44:49 -04:00
Jack Robison
fb1a774bc4
delete lbry/wallet/server/storage.py 
-expose leveldb lru cache size as `CACHE_MB` hub param
 2021-10-05 16:44:49 -04:00
Jack Robison
98bc7d1e0e
remove dead code 2021-10-05 16:44:49 -04:00
Jack Robison
f7622f24b2
non blocking mempool loop 2021-10-05 16:44:49 -04:00
Jack Robison
f0a195a6d4
faster es sync 2021-10-05 16:44:49 -04:00
Jack Robison
180ba27d84
run advance_block in threadpool 2021-10-05 16:44:49 -04:00
Jack Robison
f944671f86
use claim_to_txo cache 2021-10-05 16:44:49 -04:00
Jack Robison
def2903f7d
faster _cached_get_active_amount for claims 
-remove dead code
 2021-10-05 16:44:49 -04:00
Jack Robison
0273a4e839
fix claim search by fee for claims without fees 2021-10-05 16:44:49 -04:00
Jack Robison
f8d2f02c5d
clear claim_to_txo cache before reading 2021-10-05 16:44:49 -04:00
Jack Robison
25147d8897
handle claims that dont exist in ES sync 2021-10-05 16:44:49 -04:00
Jack Robison
0fb6f05fba
in memory claim_to_txo and txo_to_claim dictionaries 2021-10-05 16:44:49 -04:00
Jack Robison
4e4e899356
fix spend_utxo 2021-10-05 16:44:49 -04:00
Jack Robison
5a01dbf269
split flush from advance_block 2021-10-05 16:44:49 -04:00
Jack Robison
30b923b283
rename extend_ops 2021-10-05 16:44:49 -04:00
Jack Robison
73ba381d20
faster spend_utxo 2021-10-05 16:44:49 -04:00
Jack Robison
1a5912877e
faster get_future_activated 2021-10-05 16:44:49 -04:00
Jack Robison
813e506b68
threadpool 2021-10-05 16:44:49 -04:00
Jack Robison
077ca987f7
cleanup 2021-10-05 16:44:49 -04:00
Jack Robison
c632a7a6a5
fix getting block hash during reorg 2021-10-05 16:44:49 -04:00
Jack Robison
e33e767510
fix test 2021-10-05 16:44:49 -04:00
Jack Robison
ac82617aa9
fix spends in address histories 2021-10-05 16:44:49 -04:00
Jack Robison
a35dfd1fd1
faster es sync 2021-10-05 16:44:49 -04:00
Jack Robison
c28aae9913
fix expiring channels 2021-10-05 16:44:49 -04:00
Jack Robison
c26a99e65c
fix abandoning signed claims in the same tx as their channel 
-fix canonical/short url in es
 2021-10-05 16:44:49 -04:00
Jack Robison
ca57dcfc2f
handle failure to generate a short id 2021-10-05 16:44:49 -04:00
Jack Robison
df5662dd69
fix resolve by short id 2021-10-05 16:44:49 -04:00
Jack Robison
1ac7831f3c
move MemPool into BlockProcessor 2021-10-05 16:44:49 -04:00
Jack Robison
292d272a94
combine MemPool and Notifications classes 2021-10-05 16:44:49 -04:00
Jack Robison
a6ee8dc66e
fix touched hashXs notifications 2021-10-05 16:44:49 -04:00
Jack Robison
496f89f184
reorg claims in the search index 2021-10-05 16:44:49 -04:00
Jack Robison
7a56eff1ac
small fixes 2021-10-05 16:44:49 -04:00
Jack Robison
07e182aa16
rename 2021-10-05 16:44:49 -04:00
Jack Robison
7de06aa1e0
delete stale code 2021-10-05 16:44:49 -04:00
Jack Robison
3955b64405
simplify advance and reorg 2021-10-05 16:44:49 -04:00
Jack Robison
2bb55d681d
update limited_history 2021-10-05 16:44:49 -04:00
Jack Robison
f94e6ac527
update lookup_utxos 2021-10-05 16:44:49 -04:00
Jack Robison
b344f17b86
update RevertableOpStack 2021-10-05 16:44:49 -04:00
Jack Robison
677b8cb633
add remaining db prefixes 2021-10-05 16:44:49 -04:00
Jack Robison
a1ddd762e0
cleanup 2021-10-05 16:44:49 -04:00
Jack Robison
68474e4057
skip es sync during initial hub sync, halt the hub upon finishing initial sync 2021-10-05 16:44:49 -04:00
Jack Robison
a84b9ee396
fix es sync 2021-10-05 16:44:49 -04:00
Jack Robison
b9c2ee745a
fix non localhost elasticsearch 2021-10-05 16:44:49 -04:00
Jack Robison
615e489d8d
fix stream_update --clear_channel flag 2021-10-05 16:44:49 -04:00
Jack Robison
c68f9f6f16
fix signed claim invalidation corner cases 2021-10-05 16:44:49 -04:00
Jack Robison
229cb85a6a
extra deletes 
-the channel_to_claim/claim_to_channel entries already get deleted when the claim txo is spent
 2021-10-05 16:44:49 -04:00
Jack Robison
e5c22fa665
fix has_no_source for reposts 2021-10-05 16:44:49 -04:00
Jack Robison
8bcfff05d7
update channel_to_claim and claim_to_channel at the same time 2021-10-05 16:44:49 -04:00
Jack Robison
6416ee8151
typing and fix error string 2021-10-05 16:44:49 -04:00
Jack Robison
f8eceb48e6
update staged txo_to_claim after invalidating channel sig 
-fixes abandon of claim with invalidated signature and an update in same block
 2021-10-05 16:44:49 -04:00
Jack Robison
310c483bfa
missing channel_to_claim delete 2021-10-05 16:44:49 -04:00
Jack Robison
a8f20361aa
fix RepostKey 2021-10-05 16:44:49 -04:00
Jack Robison
290be69d99
typing 2021-10-05 16:44:49 -04:00
Jack Robison
3b96bd7ea0
fix 2021-10-05 16:44:49 -04:00
Jack Robison
dc2f22f5fa
cleanup 2021-10-05 16:44:49 -04:00
Jack Robison
821be29f41
rename effective_amount prefix 2021-10-05 16:44:49 -04:00
Jack Robison
52ff1a12ff
fix undeleted claim_to_channel record 2021-10-05 16:44:49 -04:00
Jack Robison
814699ef11
cleanup 2021-10-05 16:44:49 -04:00
Jack Robison
0c30838b25
fix mismatch in claim_to_txo<->txo_to_claim 2021-10-05 16:44:49 -04:00
Jack Robison
cf66c2a1ee
rename things 
-fix effective amount integrity error
 2021-10-05 16:44:49 -04:00
Jack Robison
2ee419ffca
fix 2021-10-05 16:44:49 -04:00
Jack Robison
bfb9d696d7
pretty print 2021-10-05 16:44:49 -04:00
Jack Robison
bb2a34dd6b
fix duplicate activate 2021-10-05 16:44:49 -04:00
Jack Robison
ed652c0c56
fix updating resolve by effective amount after abandoning support 2021-10-05 16:44:49 -04:00
Jack Robison
1dc961d6eb
use RevertableOpStack in _get_takeover_ops 2021-10-05 16:44:49 -04:00
Jack Robison
d119fcfc98
remove debug prints 2021-10-05 16:44:49 -04:00
Jack Robison
4d3573724a
add RevertableOpStack to verify consistency of ops as they're staged 2021-10-05 16:44:49 -04:00
Jack Robison
8b37a66075
fix fee amount overflow in es 2021-10-05 16:44:49 -04:00
Jack Robison
ba4f32075a
faster claim producer 
-make batches of claim txos from the iterator, and sort by tx hash before fetching to maximize cache and read ahead hits
 2021-10-05 16:44:49 -04:00
Jack Robison
218be22576
imports 2021-10-05 16:44:49 -04:00
Jack Robison
7688293716
close db in sync script 2021-10-05 16:44:49 -04:00
Jack Robison
458f8533c4
try default block size 2021-10-05 16:44:49 -04:00
Jack Robison
34502752fc
update elastic sync 2021-10-05 16:44:49 -04:00
Jack Robison
d6758fd823
invalidate channel signatures upon channel abandon 2021-10-05 16:44:49 -04:00
Jack Robison
65700e790e
_prepare_claim_for_sync generators 2021-10-05 16:44:49 -04:00
Jack Robison
7c34e4bb96
logging 2021-10-05 16:44:49 -04:00
Jack Robison
d0d6e3563b
use default sync=False during write_batch 2021-10-05 16:44:49 -04:00
Jack Robison
a2619f8c78
genesis_bytes attribute 2021-10-05 16:44:49 -04:00
Jack Robison
42d07fd2f0
fix 2021-10-05 16:44:49 -04:00
Jack Robison
8bea10960f
disable es (revert) 2021-10-05 16:44:49 -04:00
Jack Robison
9cbb19c304
_cached_get_active_amount 2021-10-05 16:44:49 -04:00
Jack Robison
1b94dfd712
fix removing unactivated support 2021-10-05 16:44:49 -04:00
Jack Robison
9f3604d739
debug 2021-10-05 16:44:49 -04:00
Jack Robison
4a1b2be269
leveldb tuning 2021-10-05 16:44:49 -04:00
Jack Robison
962dc1b55b
debug 2021-10-05 16:44:49 -04:00
Jack Robison
07c86502f6
refactor ClaimToTXO prefix 2021-10-05 16:44:49 -04:00
Jack Robison
adb188e5d0
filter abandoned claims from those considered for early activation 2021-10-05 16:44:49 -04:00
Jack Robison
ce031dc6b8
only do early takeover on a larger amount (fix case where they're equal) 2021-10-05 16:44:49 -04:00
Jack Robison
18b5f03247
filter supported claim hashes for claims that dont exist from early takeover/activations 2021-10-05 16:44:49 -04:00
Jack Robison
8a555ecf1c
remove extra open functions 2021-10-05 16:44:49 -04:00
Jack Robison
1b325b9acd
fix flush id 2021-10-05 16:44:49 -04:00
Jack Robison
1bdaddb319
fix clearing pending_support caches upon abandon 2021-10-05 16:44:49 -04:00
Jack Robison
7896e177ef
fix putting spent unactivated supports in removed_active_support 2021-10-05 16:44:49 -04:00
Jack Robison
ce8e659008
fix syncing claim to es where channel is in the same block 2021-10-05 16:44:49 -04:00
Jack Robison
27be5deeb2
ignore activation for headless supports 2021-10-05 16:44:49 -04:00
Jack Robison
515f270c3a
faster get_future_activated 2021-10-05 16:44:49 -04:00
Jack Robison
ffff3bd334
debugging 2021-10-05 16:44:49 -04:00
Jack Robison
f493f13b25
prints 2021-10-05 16:44:49 -04:00
Jack Robison
e605c14b13
flush count 2021-10-05 16:44:49 -04:00
Jack Robison
338488f16d
tests 2021-10-05 16:44:49 -04:00
Jack Robison
2abc67c3e8
reposts 2021-10-05 16:44:49 -04:00
Jack Robison
eb1ba143ec
fix updating the ES search index 
-update search index to use ResolveResult tuples
 2021-10-05 16:44:49 -04:00
Jack Robison
6f5bca0f67
bid ordered resolve, feed ES claim data from block processor 2021-10-05 16:44:49 -04:00
Jack Robison
407cd8dd4b
fix duplicate update op for early activating claim 2021-10-05 16:44:49 -04:00
Jack Robison
62a4f0fc04
fix early takeovers by not-yet activated claims 2021-10-05 16:44:49 -04:00
Jack Robison
3eb9d23108
require previous_winning arg for get_takeover_name_ops 2021-10-05 16:44:49 -04:00
Jack Robison
410d4aeb21
fix takeover edge case 
if a claim with a higher value than that of a claim taking over a name exists but isn't yet activated, activate it early and have it take over the name
 2021-10-05 16:44:49 -04:00
Jack Robison
0a28d216fd
comments 2021-10-05 16:44:49 -04:00
Jack Robison
b69faf6920
bid ordered resolve (WIP) 2021-10-05 16:44:49 -04:00
Jack Robison
e77f9981df
DBError 2021-10-05 16:44:49 -04:00
Jack Robison
d27c2cc1e9
remove unused COIN file 2021-10-05 16:44:49 -04:00
Jack Robison
586b19675e
claim takeovers 2021-10-05 16:44:49 -04:00
Jack Robison
f2907536b4
move get_expiration_height and claimtrie constants to Coin class 2021-10-05 16:44:49 -04:00
Jack Robison
9a11ac06bf
claim activations and takeovers (WIP) 2021-10-05 16:44:49 -04:00
Jack Robison
aa3b18f848
advance_blocks -> advance_block 2021-10-05 16:44:49 -04:00
Jack Robison
103bdc151f
dead code 2021-10-05 16:44:49 -04:00
Jack Robison
6d4c1cd879
LBRYBlockProcessor -> BlockProcessor 
- temporarily disable claim_search
 2021-10-05 16:44:49 -04:00
Jack Robison
cacbe30871
rebase 2021-10-05 16:44:49 -04:00
Jack Robison
bfeeacb230
tests 2021-10-05 16:44:49 -04:00
Jack Robison
04bb7b4919
add wrapper for getnamesintrie 
-used for verifying db state against lbrycrd
 2021-10-05 16:44:49 -04:00
Jack Robison
b7df277a5c
db state struct 
-remove dead code
 2021-10-05 16:44:49 -04:00
Jack Robison
c681041b48
claim expiration 2021-10-05 16:44:49 -04:00
Jack Robison
923834c784
get_claim_by_claim_id 2021-10-05 16:44:49 -04:00
Jack Robison
588edf98be
claims db 
-move all leveldb prefixes to DB_PREFIXES enum
-add serializable RevertableOp interface for key/value puts and deletes
-resolve urls from leveldb
 2021-10-05 16:44:49 -04:00
Jack Robison
28c603ad5f
transaction_num_mapping 2021-10-05 16:44:49 -04:00
Jack Robison
6988a47e02
disable sqlite in block processor 2021-10-05 16:44:49 -04:00
Jack Robison
2c8ceb1217
named tuples 2021-10-05 16:44:49 -04:00
Jack Robison
ccac4ffa24
consolidate flush_backup 2021-10-05 16:44:49 -04:00
Jack Robison
4258cef9bd
remove lbry.wallet.server.history 2021-10-05 16:44:49 -04:00
Jack Robison
62cc6dfe76
consolidate leveldb block advance/reorg 
-move methods from History to LevelDB
 2021-10-05 16:44:49 -04:00