LBRYCommunity/lbrycrd
1
0
Fork 0
Code Issues 29 Pull requests 1 Projects Releases 32 Packages Wiki Activity

Add a claim index for fast reference by id #110

Merged
lbrynaut merged 3 commits from claimindex into master 2018-05-18 19:15:45 +02:00
Conversation 26 Commits 3 Files changed 4 +180 -83
lbrynaut commented 2018-03-26 23:37:52 +02:00 (Migrated from github.com)
Copy link
No description provided.
kaykurokawa (Migrated from github.com) reviewed 2018-03-26 23:37:52 +02:00
lyoshenka (Migrated from github.com) reviewed 2018-03-27 03:18:18 +02:00
src/claimtrie.cpp
lyoshenka (Migrated from github.com) commented 2018-03-27 03:18:18 +02:00
Copy link

all claims should go into the index, even nonwinning ones. When I run getclaimbyid, it doesn't care about whether that claim is winning.

all claims should go into the index, even nonwinning ones. When I run `getclaimbyid`, it doesn't care about whether that claim is winning.
lbrynaut commented 2018-03-27 18:53:51 +02:00 (Migrated from github.com)
Copy link

Updated

Updated
tzarebczan commented 2018-04-03 15:11:18 +02:00 (Migrated from github.com)
Copy link

@kaykurokawa can you please review at your earliest convenience, thanks! (Reached out on DM via Slack too)

@kaykurokawa can you please review at your earliest convenience, thanks! (Reached out on DM via Slack too)
kaykurokawa commented 2018-04-05 03:11:19 +02:00 (Migrated from github.com)
Copy link

Hmm, kind of getting bizarre results , for example winning claim for "test" is a607349ce83d3a86bac87a967ce7f9647e1ba736 , try: ./lbrycrd-cli getvalueforname test

And when I do : ./lbrycrd-cli getclaimbyid a607349ce83d3a86bac87a967ce7f9647e1ba736

{
"name": "testy",
"value": "{"ver": "0.0.3", "description": "test", "license": "Creative Commons Attribution 4.0 International", "author": "test", "title": "test", "language": "en", "sources": {"lbry_sd_hash": "29ad218c61c599499b22c17228371b5fe9a6e725edc9ef691a7819b3e7406500467852920629fbcddbcacf13d31579d4"}, "content_type": "image/png", "nsfw": false}",
"claimId": "a607349ce83d3a86bac87a967ce7f9647e1ba736",
"txid": "a84cfc89c6169da6e0b63fe2b0adeed71464566bb1dab71f737a78d7711b2526",
"n": 1,
"amount": 1000000000,
"effective amount": 0,
"height": 94633
}

Observer that the name above is "testy" not "test"

Also I checked the debug log for "addToClaimIndex:" and the name that's being added seems to be off ? Seems like some names have an extra junk letter added to the end..

The code looks correct so maybe the serialization of CClaimIndexElement.name isn't handled properly? Not sure.

Hmm, kind of getting bizarre results , for example winning claim for "test" is a607349ce83d3a86bac87a967ce7f9647e1ba736 , try: ./lbrycrd-cli getvalueforname test And when I do : ./lbrycrd-cli getclaimbyid a607349ce83d3a86bac87a967ce7f9647e1ba736 { "name": "testy", "value": "{\"ver\": \"0.0.3\", \"description\": \"test\", \"license\": \"Creative Commons Attribution 4.0 International\", \"author\": \"test\", \"title\": \"test\", \"language\": \"en\", \"sources\": {\"lbry_sd_hash\": \"29ad218c61c599499b22c17228371b5fe9a6e725edc9ef691a7819b3e7406500467852920629fbcddbcacf13d31579d4\"}, \"content_type\": \"image/png\", \"nsfw\": false}", "claimId": "a607349ce83d3a86bac87a967ce7f9647e1ba736", "txid": "a84cfc89c6169da6e0b63fe2b0adeed71464566bb1dab71f737a78d7711b2526", "n": 1, "amount": 1000000000, "effective amount": 0, "height": 94633 } Observer that the name above is "testy" not "test" Also I checked the debug log for "addToClaimIndex:" and the name that's being added seems to be off ? Seems like some names have an extra junk letter added to the end.. The code looks correct so maybe the serialization of CClaimIndexElement.name isn't handled properly? Not sure.
lbrynaut commented 2018-04-05 03:14:57 +02:00 (Migrated from github.com)
Copy link

Interesting, will take a look.

Interesting, will take a look.
lbrynaut commented 2018-04-05 03:49:33 +02:00 (Migrated from github.com)
Copy link

@kaykurokawa Can you give another example? I don't see this issue and the above passes correctly here. For now, do a fresh sync for testing (i.e. rm -rf ~/.lbrycrd, assuming no meaningful wallet) to make sure the index is built properly.

@kaykurokawa Can you give another example? I don't see this issue and the above passes correctly here. For now, do a fresh sync for testing (i.e. rm -rf ~/.lbrycrd, assuming no meaningful wallet) to make sure the index is built properly.
lbrynaut commented 2018-04-05 03:55:33 +02:00 (Migrated from github.com)
Copy link

@kaykurokawa Ok, scratch that. I can now see the issue, after a restart. I see why you suspect the serialization.

@kaykurokawa Ok, scratch that. I can now see the issue, after a restart. I see why you suspect the serialization.
kaykurokawa commented 2018-04-05 04:16:09 +02:00 (Migrated from github.com)
Copy link

Ok cool, I tried the command again after reindex-ing and the above problem went away btw.

Ok cool, I tried the command again after reindex-ing and the above problem went away btw.
lbrynaut commented 2018-04-05 16:04:49 +02:00 (Migrated from github.com)
Copy link

@kaykurokawa Fixed. Please re-test, across restarts as well.

@kaykurokawa Fixed. Please re-test, across restarts as well.
kaykurokawa commented 2018-04-09 17:31:29 +02:00 (Migrated from github.com)
Copy link

I see the fix you made ( by getting the substring of name..) but I don't understand why that is necessary to fix it , do you have any insight into this?

I see the fix you made ( by getting the substring of name..) but I don't understand why that is necessary to fix it , do you have any insight into this?
lbrynaut commented 2018-04-10 20:21:41 +02:00 (Migrated from github.com)
Copy link

If you look at that method, everything referring to the name truncates/ignores the last character, including the main iteration loop. Why that is, I'm not sure yet, but that's how it comes back from disk. Whatever it is (intentional or not), it appears to have been there a long time.

If you look at that method, everything referring to the name truncates/ignores the last character, including the main iteration loop. Why that is, I'm not sure yet, but that's how it comes back from disk. Whatever it is (intentional or not), it appears to have been there a long time.
kaykurokawa commented 2018-04-13 16:24:37 +02:00 (Migrated from github.com)
Copy link

I will look over this again and merge

I will look over this again and merge
shyba commented 2018-04-20 21:45:13 +02:00 (Migrated from github.com)
Copy link

I think this solves #74

I think this solves #74
lyoshenka commented 2018-04-21 02:20:23 +02:00 (Migrated from github.com)
Copy link

it does. @kaykurokawa please take a look and merge :-)

it does. @kaykurokawa please take a look and merge :-)
shyba commented 2018-04-21 19:03:49 +02:00 (Migrated from github.com)
Copy link

I think (not yet sure) I've found another bug:
This is how dd622f534fb79af737e4edb8737a732ce74c45ff appears on getclaimsforname test:

    {                                                                                                                                                                                                              
      "claimId": "dd622f534fb79af737e4edb8737a732ce74c45ff",                                                                                                                                                       
      "txid": "e7857e5b96ae3ed836b7616dc8f454a4dc07bed5f64fd0eb939ebaa4bafe51ad",                                                                                                                                  
      "n": 0,                                                                                                                                                                                                      
      "nHeight": 193440,                                                                                                                                                                                           
      "nValidAtHeight": 196523,                                                                                                                                                                                    
      "nAmount": 10000000,                                                                                                                                                                                         
      "value": "\b\u0001\u0010\u0001\u001ae\b\u0001\u0012\u001e\b\u0004\u0010\u0001\u001a\nTEST VIDEO\"\u0000*\u00002\u00008\u0000J\u0000R\u0000Z\u0000\u001aA\b\u0001\u0010\u0001\u001a0\u00c3\u00b51d\tG\u00c2\u00be7\u00c2\u00af\u00c2\u00a5\u00c3\u008e\u00c3\u00b5\u00c2\u008a`\u00c3\u0081\u00c3\u00ae\u00c3\u009c\u00c2\u00bf\u0013\u0017\u00c3\u00abQ\u00c2\u00b6\u00c2\u009a\u00c2\u0092\u0013E\u00c3\u0093)\u00c3\u009a\u00c3\u0092I\u0018q\u00c2\u00ab\u00c3\u009a\u00c3\u0081\b\u001b\u0003g\u00c3\u00b6\u00c3\u00a1\u00c3\u0086\u00c3\u009f\u0001Y\u001e\u0016\"\tvideo/mp4*\\\b\u0001\u0010\u0003\u001a@0\u00c3\u00adz\u00c3\u009c\u00c2\u00beh2\u00c3\u008c\u00c3\u0086\u00c2\u009d\u00c2\u00b4\u000e\u00c2\u0097`\u00c3\u0098dX=Ho\u00c3\u0086<U\u00c2\u00bdDZ\u00c2\u0085\b\u00c2\u0080\u00c2\u008e=\u00c2\u0092\u00c3\u00ba\u00c3\u00ab\u00c2\u008b%\u00c3\u00a6%\u0011\u00c2\u0095\u00c2\u009a\u00c3\u008a\u00c2\u0092\u00c3\u0099\u00c2\u00a7\u00c3\u00a9\u00c3\u009d\u00c2\u0089\u0015\u00c2\u00b51\u00c3\u008e2\u0003\u00c3\u00aeWm\u00c3\u0096\u0013O\u00c3\u008c\u00c2\u009c\b\u00c2\u008b\"\u0014F\u00c3\u00ba\u00c3\u009c\u00c2\u00a3H\u0000\u00c3\u00aa<L\u00c2\u009a1CC\u00c2\u00a6\u00c3\u00a1\u00c2\u00bf\u00c2\u009f\u0019\u00c2\u0096\u00c2\u00a3",                               
      "nEffectiveAmount": 10000000,                                                                                                                                                                                
      "supports": [                                                                                                                                                                                                
      ]                                                                                                                                                                                                            
    }

Now being returned by getclaimbyid on this branch rebased on master:

{
  "name": "test",
  "value": "\b\u0001\u0010\u0001\u001ae\b\u0001\u0012\u001e\b\u0004\u0010\u0001\u001a\nTEST VIDEO\"\u0000*\u00002\u00008\u0000J\u0000R\u0000Z\u0000\u001aA\b\u0001\u0010\u0001\u001a0\u00c3\u00b51d\tG\u00c2\u00be7\u00c2\u00af\u00c2\u00a5\u00c3\u008e\u00c3\u00b5\u00c2\u008a`\u00c3\u0081\u00c3\u00ae\u00c3\u009c\u00c2\u00bf\u0013\u0017\u00c3\u00abQ\u00c2\u00b6\u00c2\u009a\u00c2\u0092\u0013E\u00c3\u0093)\u00c3\u009a\u00c3\u0092I\u0018q\u00c2\u00ab\u00c3\u009a\u00c3\u0081\b\u001b\u0003g\u00c3\u00b6\u00c3\u00a1\u00c3\u0086\u00c3\u009f\u0001Y\u001e\u0016\"\tvideo/mp4*\\\b\u0001\u0010\u0003\u001a@0\u00c3\u00adz\u00c3\u009c\u00c2\u00beh2\u00c3\u008c\u00c3\u0086\u00c2\u009d\u00c2\u00b4\u000e\u00c2\u0097`\u00c3\u0098dX=Ho\u00c3\u0086<U\u00c2\u00bdDZ\u00c2\u0085\b\u00c2\u0080\u00c2\u008e=\u00c2\u0092\u00c3\u00ba\u00c3\u00ab\u00c2\u008b%\u00c3\u00a6%\u0011\u00c2\u0095\u00c2\u009a\u00c3\u008a\u00c2\u0092\u00c3\u0099\u00c2\u00a7\u00c3\u00a9\u00c3\u009d\u00c2\u0089\u0015\u00c2\u00b51\u00c3\u008e2\u0003\u00c3\u00aeWm\u00c3\u0096\u0013O\u00c3\u008c\u00c2\u009c\b\u00c2\u008b\"\u0014F\u00c3\u00ba\u00c3\u009c\u00c2\u00a3H\u0000\u00c3\u00aa<L\u00c2\u009a1CC\u00c2\u00a6\u00c3\u00a1\u00c2\u00bf\u00c2\u009f\u0019\u00c2\u0096\u00c2\u00a3",
  "claimId": "dd622f534fb79af737e4edb8737a732ce74c45ff",
  "txid": "e7857e5b96ae3ed836b7616dc8f454a4dc07bed5f64fd0eb939ebaa4bafe51ad",
  "n": 0,
  "amount": 10000000,
  "effective amount": 0,
  "height": 193440
}

As you can see, effective amount isn't considering the claim amount itself. While getclaimsforname considers it.

However, this also happens on the current getclaimbyid implementation:

{
  "name": "test",
  "value": "\b\u0001\u0010\u0001\u001ae\b\u0001\u0012\u001e\b\u0004\u0010\u0001\u001a\nTEST VIDEO\"\u0000*\u00002\u00008\u0000J\u0000R\u0000Z\u0000\u001aA\b\u0001\u0010\u0001\u001a0\u00c3\u00b51d\tG\u00c2\u00be7\u00c2\u00af\u00c2\u00a5\u00c3\u008e\u00c3\u00b5\u00c2\u008a`\u00c3\u0081\u00c3\u00ae\u00c3\u009c\u00c2\u00bf\u0013\u0017\u00c3\u00abQ\u00c2\u00b6\u00c2\u009a\u00c2\u0092\u0013E\u00c3\u0093)\u00c3\u009a\u00c3\u0092I\u0018q\u00c2\u00ab\u00c3\u009a\u00c3\u0081\b\u001b\u0003g\u00c3\u00b6\u00c3\u00a1\u00c3\u0086\u00c3\u009f\u0001Y\u001e\u0016\"\tvideo/mp4*\\\b\u0001\u0010\u0003\u001a@0\u00c3\u00adz\u00c3\u009c\u00c2\u00beh2\u00c3\u008c\u00c3\u0086\u00c2\u009d\u00c2\u00b4\u000e\u00c2\u0097`\u00c3\u0098dX=Ho\u00c3\u0086<U\u00c2\u00bdDZ\u00c2\u0085\b\u00c2\u0080\u00c2\u008e=\u00c2\u0092\u00c3\u00ba\u00c3\u00ab\u00c2\u008b%\u00c3\u00a6%\u0011\u00c2\u0095\u00c2\u009a\u00c3\u008a\u00c2\u0092\u00c3\u0099\u00c2\u00a7\u00c3\u00a9\u00c3\u009d\u00c2\u0089\u0015\u00c2\u00b51\u00c3\u008e2\u0003\u00c3\u00aeWm\u00c3\u0096\u0013O\u00c3\u008c\u00c2\u009c\b\u00c2\u008b\"\u0014F\u00c3\u00ba\u00c3\u009c\u00c2\u00a3H\u0000\u00c3\u00aa<L\u00c2\u009a1CC\u00c2\u00a6\u00c3\u00a1\u00c2\u00bf\u00c2\u009f\u0019\u00c2\u0096\u00c2\u00a3",
  "claimId": "dd622f534fb79af737e4edb8737a732ce74c45ff",
  "txid": "e7857e5b96ae3ed836b7616dc8f454a4dc07bed5f64fd0eb939ebaa4bafe51ad",
  "n": 0,
  "amount": 10000000,
  "effective amount": 0,
  "height": 193440
}

Maybe the fix for #33 wasn't applied here?

I think (not yet sure) I've found another bug: This is how `dd622f534fb79af737e4edb8737a732ce74c45ff` appears on `getclaimsforname test`: ``` { "claimId": "dd622f534fb79af737e4edb8737a732ce74c45ff", "txid": "e7857e5b96ae3ed836b7616dc8f454a4dc07bed5f64fd0eb939ebaa4bafe51ad", "n": 0, "nHeight": 193440, "nValidAtHeight": 196523, "nAmount": 10000000, "value": "\b\u0001\u0010\u0001\u001ae\b\u0001\u0012\u001e\b\u0004\u0010\u0001\u001a\nTEST VIDEO\"\u0000*\u00002\u00008\u0000J\u0000R\u0000Z\u0000\u001aA\b\u0001\u0010\u0001\u001a0\u00c3\u00b51d\tG\u00c2\u00be7\u00c2\u00af\u00c2\u00a5\u00c3\u008e\u00c3\u00b5\u00c2\u008a`\u00c3\u0081\u00c3\u00ae\u00c3\u009c\u00c2\u00bf\u0013\u0017\u00c3\u00abQ\u00c2\u00b6\u00c2\u009a\u00c2\u0092\u0013E\u00c3\u0093)\u00c3\u009a\u00c3\u0092I\u0018q\u00c2\u00ab\u00c3\u009a\u00c3\u0081\b\u001b\u0003g\u00c3\u00b6\u00c3\u00a1\u00c3\u0086\u00c3\u009f\u0001Y\u001e\u0016\"\tvideo/mp4*\\\b\u0001\u0010\u0003\u001a@0\u00c3\u00adz\u00c3\u009c\u00c2\u00beh2\u00c3\u008c\u00c3\u0086\u00c2\u009d\u00c2\u00b4\u000e\u00c2\u0097`\u00c3\u0098dX=Ho\u00c3\u0086<U\u00c2\u00bdDZ\u00c2\u0085\b\u00c2\u0080\u00c2\u008e=\u00c2\u0092\u00c3\u00ba\u00c3\u00ab\u00c2\u008b%\u00c3\u00a6%\u0011\u00c2\u0095\u00c2\u009a\u00c3\u008a\u00c2\u0092\u00c3\u0099\u00c2\u00a7\u00c3\u00a9\u00c3\u009d\u00c2\u0089\u0015\u00c2\u00b51\u00c3\u008e2\u0003\u00c3\u00aeWm\u00c3\u0096\u0013O\u00c3\u008c\u00c2\u009c\b\u00c2\u008b\"\u0014F\u00c3\u00ba\u00c3\u009c\u00c2\u00a3H\u0000\u00c3\u00aa<L\u00c2\u009a1CC\u00c2\u00a6\u00c3\u00a1\u00c2\u00bf\u00c2\u009f\u0019\u00c2\u0096\u00c2\u00a3", "nEffectiveAmount": 10000000, "supports": [ ] } ``` Now being returned by `getclaimbyid` on this branch rebased on master: ``` { "name": "test", "value": "\b\u0001\u0010\u0001\u001ae\b\u0001\u0012\u001e\b\u0004\u0010\u0001\u001a\nTEST VIDEO\"\u0000*\u00002\u00008\u0000J\u0000R\u0000Z\u0000\u001aA\b\u0001\u0010\u0001\u001a0\u00c3\u00b51d\tG\u00c2\u00be7\u00c2\u00af\u00c2\u00a5\u00c3\u008e\u00c3\u00b5\u00c2\u008a`\u00c3\u0081\u00c3\u00ae\u00c3\u009c\u00c2\u00bf\u0013\u0017\u00c3\u00abQ\u00c2\u00b6\u00c2\u009a\u00c2\u0092\u0013E\u00c3\u0093)\u00c3\u009a\u00c3\u0092I\u0018q\u00c2\u00ab\u00c3\u009a\u00c3\u0081\b\u001b\u0003g\u00c3\u00b6\u00c3\u00a1\u00c3\u0086\u00c3\u009f\u0001Y\u001e\u0016\"\tvideo/mp4*\\\b\u0001\u0010\u0003\u001a@0\u00c3\u00adz\u00c3\u009c\u00c2\u00beh2\u00c3\u008c\u00c3\u0086\u00c2\u009d\u00c2\u00b4\u000e\u00c2\u0097`\u00c3\u0098dX=Ho\u00c3\u0086<U\u00c2\u00bdDZ\u00c2\u0085\b\u00c2\u0080\u00c2\u008e=\u00c2\u0092\u00c3\u00ba\u00c3\u00ab\u00c2\u008b%\u00c3\u00a6%\u0011\u00c2\u0095\u00c2\u009a\u00c3\u008a\u00c2\u0092\u00c3\u0099\u00c2\u00a7\u00c3\u00a9\u00c3\u009d\u00c2\u0089\u0015\u00c2\u00b51\u00c3\u008e2\u0003\u00c3\u00aeWm\u00c3\u0096\u0013O\u00c3\u008c\u00c2\u009c\b\u00c2\u008b\"\u0014F\u00c3\u00ba\u00c3\u009c\u00c2\u00a3H\u0000\u00c3\u00aa<L\u00c2\u009a1CC\u00c2\u00a6\u00c3\u00a1\u00c2\u00bf\u00c2\u009f\u0019\u00c2\u0096\u00c2\u00a3", "claimId": "dd622f534fb79af737e4edb8737a732ce74c45ff", "txid": "e7857e5b96ae3ed836b7616dc8f454a4dc07bed5f64fd0eb939ebaa4bafe51ad", "n": 0, "amount": 10000000, "effective amount": 0, "height": 193440 } ``` As you can see, effective amount isn't considering the claim amount itself. While `getclaimsforname` considers it. However, this **also** happens on the current `getclaimbyid` implementation: ``` { "name": "test", "value": "\b\u0001\u0010\u0001\u001ae\b\u0001\u0012\u001e\b\u0004\u0010\u0001\u001a\nTEST VIDEO\"\u0000*\u00002\u00008\u0000J\u0000R\u0000Z\u0000\u001aA\b\u0001\u0010\u0001\u001a0\u00c3\u00b51d\tG\u00c2\u00be7\u00c2\u00af\u00c2\u00a5\u00c3\u008e\u00c3\u00b5\u00c2\u008a`\u00c3\u0081\u00c3\u00ae\u00c3\u009c\u00c2\u00bf\u0013\u0017\u00c3\u00abQ\u00c2\u00b6\u00c2\u009a\u00c2\u0092\u0013E\u00c3\u0093)\u00c3\u009a\u00c3\u0092I\u0018q\u00c2\u00ab\u00c3\u009a\u00c3\u0081\b\u001b\u0003g\u00c3\u00b6\u00c3\u00a1\u00c3\u0086\u00c3\u009f\u0001Y\u001e\u0016\"\tvideo/mp4*\\\b\u0001\u0010\u0003\u001a@0\u00c3\u00adz\u00c3\u009c\u00c2\u00beh2\u00c3\u008c\u00c3\u0086\u00c2\u009d\u00c2\u00b4\u000e\u00c2\u0097`\u00c3\u0098dX=Ho\u00c3\u0086<U\u00c2\u00bdDZ\u00c2\u0085\b\u00c2\u0080\u00c2\u008e=\u00c2\u0092\u00c3\u00ba\u00c3\u00ab\u00c2\u008b%\u00c3\u00a6%\u0011\u00c2\u0095\u00c2\u009a\u00c3\u008a\u00c2\u0092\u00c3\u0099\u00c2\u00a7\u00c3\u00a9\u00c3\u009d\u00c2\u0089\u0015\u00c2\u00b51\u00c3\u008e2\u0003\u00c3\u00aeWm\u00c3\u0096\u0013O\u00c3\u008c\u00c2\u009c\b\u00c2\u008b\"\u0014F\u00c3\u00ba\u00c3\u009c\u00c2\u00a3H\u0000\u00c3\u00aa<L\u00c2\u009a1CC\u00c2\u00a6\u00c3\u00a1\u00c2\u00bf\u00c2\u009f\u0019\u00c2\u0096\u00c2\u00a3", "claimId": "dd622f534fb79af737e4edb8737a732ce74c45ff", "txid": "e7857e5b96ae3ed836b7616dc8f454a4dc07bed5f64fd0eb939ebaa4bafe51ad", "n": 0, "amount": 10000000, "effective amount": 0, "height": 193440 } ``` Maybe the fix for #33 wasn't applied here?
shyba commented 2018-04-21 23:00:54 +02:00 (Migrated from github.com)
Copy link

Should be fixed in #123 and isn't related to code touched by this PR.

Should be fixed in #123 and isn't related to code touched by this PR.
lbrynaut commented 2018-04-23 03:33:34 +02:00 (Migrated from github.com)
Copy link

@shyba Not sure I'm following. This code replaces the existing getclaimbyid in a way that improves performance by using a claim index that's built and maintained by this (PR) code. Are you seeing a discrepancy between the master branch and this one?

@shyba Not sure I'm following. This code replaces the existing getclaimbyid in a way that improves performance by using a claim index that's built and maintained by this (PR) code. Are you seeing a discrepancy between the master branch and this one?
shyba commented 2018-04-23 16:09:26 +02:00 (Migrated from github.com)
Copy link

@lbrynaut No. At first I thought it was a discrepancy, then I found a bug and fixed in #123 but it turned out to be unrelated to this PR. Sorry for the confusion.

@lbrynaut No. At first I thought it was a discrepancy, then I found a bug and fixed in #123 but it turned out to be unrelated to this PR. Sorry for the confusion.
kaykurokawa (Migrated from github.com) reviewed 2018-04-26 01:50:07 +02:00
src/claimtrie.cpp
kaykurokawa (Migrated from github.com) commented 2018-04-26 01:50:07 +02:00
Copy link

Hmm, the part that comes after this, where it checks the db does not seem necessary..

Correct me if I'm wrong but there should never be a case where if it's not found in claimIndex but its found in leveldb.

Hmm, the part that comes after this, where it checks the db does not seem necessary.. Correct me if I'm wrong but there should never be a case where if it's not found in claimIndex but its found in leveldb.
kaykurokawa commented 2018-04-26 03:14:03 +02:00 (Migrated from github.com)
Copy link

Other than the one comment I left above, I think this is good to merge.

Also, reminder that this feature will require a reindex for it to work, this should be noted once a release is made.

Other than the one comment I left above, I think this is good to merge. Also, reminder that this feature will require a reindex for it to work, this should be noted once a release is made.
kaykurokawa commented 2018-04-26 04:38:06 +02:00 (Migrated from github.com)
Copy link

Actually two more thing:

Is it even necessary to keep and maintain claimIndex ? We could just query leveldb directly and we wouldn't have to worry about claimIndex growing unbounded. Since its just used for RPC calls, maybe its ok that its slow?

When we call BatchWriteClaimIndex, we are not deleting abandoned claims from leveldb even though we are deleting them from claimIndex using removeFromClaimIndex. Is that the behavior we want? Victor pointed out that before this PR, getclaimbyid would not return abandoned claims, but now it does, because it is stored in leveldb.

Actually two more thing: Is it even necessary to keep and maintain claimIndex ? We could just query leveldb directly and we wouldn't have to worry about claimIndex growing unbounded. Since its just used for RPC calls, maybe its ok that its slow? When we call BatchWriteClaimIndex, we are not deleting abandoned claims from leveldb even though we are deleting them from claimIndex using removeFromClaimIndex. Is that the behavior we want? Victor pointed out that before this PR, getclaimbyid would not return abandoned claims, but now it does, because it is stored in leveldb.
lbrynaut commented 2018-04-26 19:48:36 +02:00 (Migrated from github.com)
Copy link

@kaykurokawa Both good points. I'll compare the performance and see if it's acceptable. As for the abandoned claims, either way. I'll remove since that's how the previous code worked.

@kaykurokawa Both good points. I'll compare the performance and see if it's acceptable. As for the abandoned claims, either way. I'll remove since that's how the previous code worked.
lbrynaut commented 2018-05-07 16:49:09 +02:00 (Migrated from github.com)
Copy link

Updated

Updated
lbrynaut commented 2018-05-15 15:28:44 +02:00 (Migrated from github.com)
Copy link

Rebased, updated, added unit test. @kaykurokawa Merge if ready.

Rebased, updated, added unit test. @kaykurokawa Merge if ready.
kaykurokawa commented 2018-05-18 19:15:34 +02:00 (Migrated from github.com)
Copy link

This LGTM

This LGTM
Sign in to join this conversation.
Reviewers
No reviewers
kaykurokawa
Labels
Clear labels   
area: devops

   
area: discovery

   
area: docs

   
area: livestream

   
area: proposal

Work is part of a proposal

  
consider soon

Discuss this issue at the next planning meeting, then remove this label

  
Epic

   
good first issue

   
hacktoberfest

Welcome to Hacktoberfest

  
hard fork

   
help wanted

   
icebox

Long-term storage

  
Invalid

   
level: 0

   
level: 1

No knowledge of the existing code required

  
level: 2

Some knowledge of the existing code is recommended

  
level: 3

Significant knowledge of the existing code is recommended

  
level: 4

Intimate knowledge of the existing code is recommended

  
needs: exploration

Solution unclear, needs research

  
needs: grooming

Issue needs to be groomed before work can start

  
needs: priority

Priority level needs to be defined

  
needs: repro

Issue needs step-by-step instructions on how to reproduce in latest code

  
needs: tech design

Needs technical design signoff before implementation can begin

  
on hold

Temporarily paused

  
priority: blocker

Issue is blocking release, do ASAP

  
priority: high

Work needs to be moved into sprint ASAP

  
priority: low

Work should be done but can stay in the backlog for now

  
priority: medium

Work needs to be done within 2-3 sprints

  
resilience

   
soft fork

   
Tom's Wishlist

   
type: bug

Existing functionality is wrong or broken

  
type: discussion

A conversation. No specific changes requested

  
type: improvement

Existing (or partially existing) functionality needs to be changed

  
type: new feature

New functionality that does not exist yet

  
type: refactor

Minimal user-visible changes, but significant internal work

  
type: task

Either work that's not related to the code, or a small chore that does not fit into other categories

  
type: testing

Solution needs additional user testing

  
unplanned

Work that was not planned into the spirnt, took priority over planned work

  
work in progress

The proposed solution is still incomplete.

No labels
area: devops
area: discovery
area: docs
area: livestream
area: proposal
consider soon
Epic
good first issue
hacktoberfest
hard fork
help wanted
icebox
Invalid
level: 0
level: 1
level: 2
level: 3
level: 4
needs: exploration
needs: grooming
needs: priority
needs: repro
needs: tech design
on hold
priority: blocker
priority: high
priority: low
priority: medium
resilience
soft fork
Tom's Wishlist
type: bug
type: discussion
type: improvement
type: new feature
type: refactor
type: task
type: testing
unplanned
work in progress
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
LBRYCommunity/lbrycrd!110
No description provided.