lbrycrdd crashes on restart and needs to be reindexed to recover #134

Closed
opened 2018-05-07 02:42:58 +02:00 by kaykurokawa · 4 comments
kaykurokawa commented 2018-05-07 02:42:58 +02:00 (Migrated from github.com)

When restarting lbrycrdd, there has been reports of it crashing

a) https://github.com/lbryio/lbrycrd/pull/109

b) Grin reported this error when restarting a lbrycrd node: lbrycrdd: main.cpp:2112: bool DisconnectBlock(const CBlock&, CValidationState&, const CBlockIndex*, CCoinsViewCache&, CClaimTrieCache&, bool*): Assertion `pindex->GetBlockHash() == trieCache.getBestBlock()' failed

Happens at block 359949

slack-imgs com

This is a possible cause of this issue but not sure... https://github.com/lbryio/lbrycrd/issues/133

When restarting lbrycrdd, there has been reports of it crashing a) https://github.com/lbryio/lbrycrd/pull/109 b) Grin reported this error when restarting a lbrycrd node: lbrycrdd: main.cpp:2112: bool DisconnectBlock(const CBlock&, CValidationState&, const CBlockIndex*, CCoinsViewCache&, CClaimTrieCache&, bool*): Assertion `pindex->GetBlockHash() == trieCache.getBestBlock()' failed Happens at block 359949 ![slack-imgs com](https://user-images.githubusercontent.com/7075051/39683228-6acb307c-5182-11e8-8731-af99f903a98c.png) This is a possible cause of this issue but not sure... https://github.com/lbryio/lbrycrd/issues/133
kaykurokawa commented 2018-07-09 02:00:47 +02:00 (Migrated from github.com)

Just had this happen to my node:

lbrycrdd: claimtrie.cpp:2416: bool CClaimTrieCache::decrementBlock(insertUndoType&, claimQueueRowType&, insertUndoType&, supportQueueRowType&, std::vector<std::pair<std::__cxx11::basic_string, int> >&) const: Assertion `removeClaimFromTrie(itInsertUndo->name, itInsertUndo->outPoint, claim, false)' failed.
Aborted (core dumped)

Below is the log

2018-07-08 22:25:30 Bitcoin version v0.12.2.0-9c1654b (2018-06-30 17:39:51 -0400)
2018-07-08 22:25:30 InitParameterInteraction: parameter interaction: -whitelistforcerelay=1 -> setting -whitelistrelay=1
2018-07-08 22:25:30 Using BerkeleyDB version Berkeley DB 4.8.30: (April 9, 2010)
2018-07-08 22:25:30 Default data directory /home/treantprotector/.lbrycrd
2018-07-08 22:25:30 Using data directory /home/treantprotector/.lbrycrd
2018-07-08 22:25:30 Using config file /home/treantprotector/.lbrycrd/lbrycrd.conf
2018-07-08 22:25:30 Using at most 125 connections (1024 file descriptors available)
2018-07-08 22:25:30 Using 2 threads for script verification
2018-07-08 22:25:30 scheduler thread start
2018-07-08 22:25:30 HTTP: creating work queue of depth 16
2018-07-08 22:25:30 Config options rpcuser and rpcpassword will soon be deprecated. Locally-run instances may remove rpcuser to use cookie-based auth, or may be replaced with rpcauth. Please see share/rpcuser for rpcauth auth generation.
2018-07-08 22:25:30 HTTP: starting 4 worker threads
2018-07-08 22:25:30 Using wallet wallet.dat
2018-07-08 22:25:30 init message: Verifying wallet...
2018-07-08 22:25:30 CDBEnv::Open: LogDir=/home/treantprotector/.lbrycrd/database ErrorFile=/home/treantprotector/.lbrycrd/db.log
2018-07-08 22:25:30 Bound to [::]:9246
2018-07-08 22:25:30 Bound to 0.0.0.0:9246
2018-07-08 22:25:30 Cache configuration:
2018-07-08 22:25:30 * Using 2.0MiB for block index database
2018-07-08 22:25:30 * Using 32.5MiB for chain state database
2018-07-08 22:25:30 * Using 65.5MiB for in-memory UTXO set
2018-07-08 22:25:30 init message: Loading block index...
2018-07-08 22:25:30 Opening LevelDB in /home/treantprotector/.lbrycrd/blocks/index
2018-07-08 22:25:30 Opened LevelDB successfully
2018-07-08 22:25:30 Using obfuscation key for /home/treantprotector/.lbrycrd/blocks/index: 0000000000000000
2018-07-08 22:25:30 Opening LevelDB in /home/treantprotector/.lbrycrd/chainstate
2018-07-08 22:25:30 Opened LevelDB successfully
2018-07-08 22:25:30 Using obfuscation key for /home/treantprotector/.lbrycrd/chainstate: 669fe790e31967ae
2018-07-08 22:25:30 Opening LevelDB in /home/treantprotector/.lbrycrd/claimtrie
2018-07-08 22:25:30 Opened LevelDB successfully
2018-07-08 22:25:30 Using obfuscation key for /home/treantprotector/.lbrycrd/claimtrie: 0000000000000000
2018-07-08 22:25:34 LoadBlockIndexDB: last block file = 9
2018-07-08 22:25:34 LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=48504, size=129979567, heights=344312...392893, time=2018-03-27...2018-06-25)
2018-07-08 22:25:34 Checking all blk files are present...
2018-07-08 22:25:34 LoadBlockIndexDB: transaction index disabled
2018-07-08 22:25:34 LoadBlockIndexDB: hashBestChain=d032d4fe29595376e41bfaaab3a957c1437bb971523cbe2328bd2a3231c65435 height=392892 date=2018-06-25 21:01:04 progress=0.997355
2018-07-08 22:25:34 setExpirationTime: Expiration time is now 262974
2018-07-08 22:25:44 Checking Claim trie consistency...consistent
2018-07-08 22:25:44 init message: Verifying blocks...
2018-07-08 22:25:44 Verifying last 288 blocks at level 3
2018-07-08 22:25:44 decrementBlock: nCurrentHeight (before decrement): 392893
2018-07-08 22:25:44 removeClaimFromTrie: The name buffer-itstime-baraomcdonald does not exist in the trie

Just had this happen to my node: lbrycrdd: claimtrie.cpp:2416: bool CClaimTrieCache::decrementBlock(insertUndoType&, claimQueueRowType&, insertUndoType&, supportQueueRowType&, std::vector<std::pair<std::__cxx11::basic_string<char>, int> >&) const: Assertion `removeClaimFromTrie(itInsertUndo->name, itInsertUndo->outPoint, claim, false)' failed. Aborted (core dumped) Below is the log 2018-07-08 22:25:30 Bitcoin version v0.12.2.0-9c1654b (2018-06-30 17:39:51 -0400) 2018-07-08 22:25:30 InitParameterInteraction: parameter interaction: -whitelistforcerelay=1 -> setting -whitelistrelay=1 2018-07-08 22:25:30 Using BerkeleyDB version Berkeley DB 4.8.30: (April 9, 2010) 2018-07-08 22:25:30 Default data directory /home/treantprotector/.lbrycrd 2018-07-08 22:25:30 Using data directory /home/treantprotector/.lbrycrd 2018-07-08 22:25:30 Using config file /home/treantprotector/.lbrycrd/lbrycrd.conf 2018-07-08 22:25:30 Using at most 125 connections (1024 file descriptors available) 2018-07-08 22:25:30 Using 2 threads for script verification 2018-07-08 22:25:30 scheduler thread start 2018-07-08 22:25:30 HTTP: creating work queue of depth 16 2018-07-08 22:25:30 Config options rpcuser and rpcpassword will soon be deprecated. Locally-run instances may remove rpcuser to use cookie-based auth, or may be replaced with rpcauth. Please see share/rpcuser for rpcauth auth generation. 2018-07-08 22:25:30 HTTP: starting 4 worker threads 2018-07-08 22:25:30 Using wallet wallet.dat 2018-07-08 22:25:30 init message: Verifying wallet... 2018-07-08 22:25:30 CDBEnv::Open: LogDir=/home/treantprotector/.lbrycrd/database ErrorFile=/home/treantprotector/.lbrycrd/db.log 2018-07-08 22:25:30 Bound to [::]:9246 2018-07-08 22:25:30 Bound to 0.0.0.0:9246 2018-07-08 22:25:30 Cache configuration: 2018-07-08 22:25:30 * Using 2.0MiB for block index database 2018-07-08 22:25:30 * Using 32.5MiB for chain state database 2018-07-08 22:25:30 * Using 65.5MiB for in-memory UTXO set 2018-07-08 22:25:30 init message: Loading block index... 2018-07-08 22:25:30 Opening LevelDB in /home/treantprotector/.lbrycrd/blocks/index 2018-07-08 22:25:30 Opened LevelDB successfully 2018-07-08 22:25:30 Using obfuscation key for /home/treantprotector/.lbrycrd/blocks/index: 0000000000000000 2018-07-08 22:25:30 Opening LevelDB in /home/treantprotector/.lbrycrd/chainstate 2018-07-08 22:25:30 Opened LevelDB successfully 2018-07-08 22:25:30 Using obfuscation key for /home/treantprotector/.lbrycrd/chainstate: 669fe790e31967ae 2018-07-08 22:25:30 Opening LevelDB in /home/treantprotector/.lbrycrd/claimtrie 2018-07-08 22:25:30 Opened LevelDB successfully 2018-07-08 22:25:30 Using obfuscation key for /home/treantprotector/.lbrycrd/claimtrie: 0000000000000000 2018-07-08 22:25:34 LoadBlockIndexDB: last block file = 9 2018-07-08 22:25:34 LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=48504, size=129979567, heights=344312...392893, time=2018-03-27...2018-06-25) 2018-07-08 22:25:34 Checking all blk files are present... 2018-07-08 22:25:34 LoadBlockIndexDB: transaction index disabled 2018-07-08 22:25:34 LoadBlockIndexDB: hashBestChain=d032d4fe29595376e41bfaaab3a957c1437bb971523cbe2328bd2a3231c65435 height=392892 date=2018-06-25 21:01:04 progress=0.997355 2018-07-08 22:25:34 setExpirationTime: Expiration time is now 262974 2018-07-08 22:25:44 Checking Claim trie consistency...consistent 2018-07-08 22:25:44 init message: Verifying blocks... 2018-07-08 22:25:44 Verifying last 288 blocks at level 3 2018-07-08 22:25:44 decrementBlock: nCurrentHeight (before decrement): 392893 2018-07-08 22:25:44 removeClaimFromTrie: The name buffer-itstime-baraomcdonald does not exist in the trie
kaykurokawa commented 2018-07-11 19:58:56 +02:00 (Migrated from github.com)

The post directly above was a false alarm, due to https://github.com/lbryio/lbrycrd/pull/160 not being downgrade compatible

The post directly above was a false alarm, due to https://github.com/lbryio/lbrycrd/pull/160 not being downgrade compatible
alyssaoc commented 2018-08-21 22:44:54 +02:00 (Migrated from github.com)

@kaykurokawa Can this be closed? Or is this bug still happening?

@kaykurokawa Can this be closed? Or is this bug still happening?
alyssaoc commented 2018-10-17 19:53:23 +02:00 (Migrated from github.com)

reopen if necessary

reopen if necessary
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/lbrycrd#134
No description provided.