Commit graph

67 commits

Author SHA1 Message Date
Jeffrey Picard
b301fb953e switch herald to herald.go 2022-08-03 22:37:07 +03:00
Jonathan Moody
49e5d7b8c1
Fix release_time query to match python hub. (#41)
* Revise error handling and cleanup in GetProdDB(). Avoids panic when DB not found.

* Fix release_time query to match python hub. Claims with unset release_time match the query.
2022-08-03 19:19:25 +03:00
Jeffrey Picard
b2dd70bc7c
Jeffreypicard/feature/move to herald (#37)
* changes for repo move

* Go back to protobuf 1.27.1, update go to 1.18

* update readme for new rocksdb version
2022-07-11 20:31:27 +03:00
Jeffrey Picard
28a299efa8
rocksdb (#29)
* Initial rocksdb commit

Basic reading from rocksdb works

* Try github action thing

* try local dockerfile

* asdf

* qwer

* asdf

* Try adding test db with git-lfs

* update action

* cleanup

* Don't hardcode stop on read

* Progress of reading rocksdb

* fixes and arg test

* asdf

* Fix rocksdb iterator and tests

* update script

* asdf

* Better iterator. Need to implement a lot of keys next, and tests, maybe
tests needed.

* asdf

* asdf

* asdf

* Implementation, testing, and cleanup.

Implemented more prefixes. Figured out a good test that should work for
all prefixes. Removed binary databases so we can just store human
readable csv files.

* more tests, prefixes and small refactor

* Another prefix

* EffectiveAmount

* ActiveAmount

* ActivatedClaimAndSupport

* PendingActivation

* ClaimTakeover

* ClaimExpiration

* SupportToClaim

* ClaimToSupport

* Fix bug with variable length keys

* ChannelToClaim

* ClaimToChannel

* ClaimShortID

* TXOToClaim

* ClaimToTXO

* BlockHeader

* BlockHash

* Undo

* HashXHistory

* Tx and big refactor

* rest the the keys

* Refactor and starting to add resolve

* asdf

* Refactor tests and add column families

* changes

* more work on implementing resolve

* code cleanup, function tests

* small code refactoring

* start building pieces of the test data set for full resolve.

* Export constant, add test

* another test

* TestGetTxHash

* more tests

* more tests

* More tests

* Refactor db functions into three files

* added slice backed stack, need to fix tests

* fix some issues with test suite

* some cleanup and adding arguments and db load / refresh to server command

* fix some bugs, start using logrus for leveled logging, upgrade to go 1.17, run go mod tidy

* logrus, protobuf updates, resolve grpc endpoint

* don't run integration test with unit tests

* signal handling and cleanup functions

* signal handling code files

* Unit tests for db stack

* reorganize bisect function so we lock it properly

* fix txcounts loading

* cleanup some logic around iterators and fix a bug where I was running two detect changes threads

* add some metrics

* cleanup

* blocking and filtering implemented

* add params for blocking and filtering channels and streams

* updates and fixes for integration tests

* use newer version of lbry.go when possible

* Add height endpoint and move string functions internal

* remove gitattributes, unused

* some cleanup

* more cleanup / refactor. almost ready for another review

* More cleanup

* use chainhash.Hash types from lbcd where appropriate

* update github action to go-1.17.8

* update go version needed

* trying to fix these builds

* cleanup

* trying to fix memory leak

* fix memory leak (iterator never finished so cleanup didn't run)

* changes per code review

* remove lbry.go v2

* rename sort.go search.go

* fix test
2022-04-29 11:04:01 -04:00
Jeffrey Picard
d51f5ca064 Fix behavior when the index isn't there.
One of the tests expects an empty list returned, we're currently
returning an error.
2022-01-05 10:07:15 -05:00
Victor Shyba
8e99bc56d0
add sd_hash prefix search (#30)
* add sd_hash to search request

* search sd_hash by prefix
2021-12-17 09:37:32 -05:00
Jeffrey Picard
72ea236d86 Run gofmt 2021-12-06 11:31:51 -05:00
Jeffrey Picard
d511b08736 Switch RangeField to int32 in the protobuf, as none of those fields use
strings.
2021-12-03 19:39:40 -05:00
Jeffrey Picard
2537a84b86 Update RangeField
Allow repeated range fields per term (for i.e. 0 < x < 100)
2021-12-03 13:02:06 -05:00
Jeffrey Picard
5387aeeebe Most of federation is written, need to finish udp and test
Cleanup, more reorg, more arguments, started adding tests
Comments and another test
Simplify writing of peers and add unit test
2021-10-29 16:56:36 -04:00
Jeffrey Picard
45e9817ced Cleanup metrics
Consolidated error metrics into a vector counter with a label,
removed other uneeded counters.
2021-10-29 16:56:35 -04:00
Jeffrey Picard
a27b6e730d Record metrics
There are a couple metrics that try to mirror the ones from python,
many of the python ones don't apply, and vice-versa.
2021-10-29 16:56:35 -04:00
Jeffrey Picard
a6d47e662a Don't swallow error, cachettl and refresh delta as params 2021-10-04 18:04:26 -04:00
Jeffrey Picard
220a42984e put jack's changes back 2021-10-04 12:44:55 -04:00
Jeffrey Picard
a0de5164ba cleanup 2021-10-02 23:13:47 -04:00
Jeffrey Picard
0d59480f3c bug fixes and cache purging 2021-10-02 22:49:49 -04:00
Jeffrey Picard
36b4a3cdd9 implement caching 2021-10-02 17:56:12 -04:00
Jeffrey Picard
076adcca39 Initial caching 2021-09-30 15:03:00 -04:00
Jeffrey Picard
e0141ae5ca Merge branch 'master' of github.com:lbryio/hub into prometheus-and-versioning 2021-09-21 14:58:24 -04:00
Jeffrey Picard
fdbd7ec509 proto fixes and cleanup 2021-09-18 13:36:49 -04:00
Jeffrey Picard
81c3de3bfd Merge remote-tracking branch 'origin/master' into prometheus-and-versioning 2021-09-18 13:19:47 -04:00
Jeffrey Picard
9c89907bb8 versioning and metrics 2021-09-18 12:06:56 -04:00
Jack Robison
fc57c177e2
handle error 2021-09-15 13:25:19 -04:00
Jack Robison
7c9d773ff0
remove old trending fields
-rename `Reposted` -> `RepostCount`
-add `TrendingScore`
-rename `Name` -> `ClaimName`
-round up `ReleaseTime` params the same way as the python hub
2021-09-15 13:12:59 -04:00
Ivan Hranat
64870c54bc linitng 2021-08-24 11:45:30 +03:00
Victor Shyba
8a2a98726d initialize the server when making it instead of during request 2021-08-13 19:16:30 -03:00
Victor Shyba
b4782ce6ac add --debug to show queries and error details 2021-08-13 18:36:43 -03:00
Victor Shyba
d71e91c58e throttle flood errors like previous hub did 2021-08-13 16:04:19 -03:00
Victor Shyba
9edba6109f check repeatable fields for being less than 2048 items 2021-08-13 16:02:45 -03:00
Victor Shyba
7a8e8b9b47 return empty if 404 and log errors 2021-08-13 14:08:25 -03:00
Victor Shyba
d45f215789 field changes 2021-08-12 04:19:25 -03:00
Victor Shyba
80ab0e7cd3 censoring_channel_hash->censoring_channel_id 2021-08-11 01:41:27 -03:00
Victor Shyba
b857eb38b2 reposted->repost_count 2021-08-11 00:59:46 -03:00
Victor Shyba
87b3db5d98 signature_valid->is_signature_valid 2021-08-11 00:53:16 -03:00
Victor Shyba
5757edfb9e normalized->normalized_name, public_key_hash->public_key_id, gofmt 2021-08-11 00:40:42 -03:00
Victor Shyba
d2d17bee3b specify index, remove --dev 2021-08-11 00:39:37 -03:00
Jeffrey Picard
9498257c69 Use util functions from lbry.go 2021-06-28 13:54:49 -04:00
Jeffrey Picard
122224b0ad mixed up parameter order 2021-06-18 00:34:22 -04:00
Jeffrey Picard
3f64655a26 Updates from review feedback 2021-06-18 00:01:47 -04:00
Jeffrey Picard
9b1cb354d4 fix more index problems 2021-06-17 13:06:33 -04:00
Jeffrey Picard
f04af2b12e Fix search indices issue 2021-06-17 12:15:05 -04:00
Jeffrey Picard
c70467335b Try adding github action 2021-06-16 19:42:11 -04:00
Jeffrey Picard
60a38f80d6 extra_txo / channel stuff works now 2021-06-11 18:18:17 -04:00
Jeffrey Picard
af9b5de49d get claims for reposts and channels with mget 2021-06-11 08:34:27 -04:00
Jeffrey Picard
acf7ce671d make record -> output it's own function 2021-06-09 22:41:01 -04:00
Jeffrey Picard
4280edc465 cleanup and updates 2021-06-09 20:04:06 -04:00
Jeffrey Picard
ad21f22f8e unused var 2021-06-08 23:02:22 -04:00
Jeffrey Picard
bb9eae2fc6 remove unneeded prints 2021-06-08 22:59:13 -04:00
Jeffrey Picard
f98738647f cleanup and delete unused code 2021-06-08 22:57:41 -04:00
Jeffrey Picard
79db1be087 implement remove duplicates 2021-06-08 22:07:59 -04:00