Commit graph

38 commits

Author SHA1 Message Date
Mark Beamer Jr 162107239b
Add capability to restart sync where it left off by storing state of the sync. Set to update
Signed-off-by: Mark Beamer Jr <markbeamerjr@gmail.com>
2019-09-21 17:25:18 -04:00
Mark Beamer Jr c67f298ee6
Add channel id to claim if exists for more precise filtering 2019-09-19 22:15:52 -04:00
Thomas Zarebczan c002a09d7e fix: eslint problems 2019-09-18 12:47:21 -04:00
Niko Storni b873d93bf6 .io to .com 2019-07-08 20:00:10 +02:00
Mark Beamer Jr db39a6c47f
skip bad claim when processing to ensure all future claims make it into elastic search. 2019-06-22 16:37:39 -04:00
Mark Beamer Jr 11d000119b
check for null value to notify lbry team. 2019-06-22 16:17:40 -04:00
Mark Beamer Jr ea2fcb56ab
log error when step in sync process fails. 2019-06-21 00:11:16 -04:00
Mark Beamer Jr 4f5206d18e
Check for Claim field 2019-06-20 23:26:36 -04:00
Mark Beamer Jr 5a9ce272b9
Added reverted changes back in 2019-02-26 08:50:26 -05:00
Mark 79a2470504
Revert "added channel effective amount to the claim weight." 2019-02-24 22:05:39 -05:00
Mark Beamer Jr 101e55d924
added channel effective amount to the claim weight. 2019-02-24 17:03:50 -05:00
Mark Beamer Jr 81986315cb
Batched the chainquery calls for lighthouse in case it needs to grab all 700K in a redo. 2019-02-24 16:57:56 -05:00
Mark Beamer Jr 965746b9d2
added slack integration 2018-11-04 01:01:23 -04:00
filipnyquist 6b3269bfe3 Added the breaking change of #93. Added the change, do not merge until the chainquery change has been merged and deployed. 2018-07-30 15:19:29 +02:00
filipnyquist 4423fc2cfc This PR adds channels into the index, this combined with the added channel filtering parameter allows us to search specific channels for content.
While adding the code for filtering I split the search query into individual parts so it easier can be edited and read.
2018-07-23 12:06:47 +02:00
Mark Beamer Jr cd0f980904 -Refactored handling of blocked claims.
-Changed deletes to be queued and refactored.
-Added logging.
-Removed commented debug code.
-Added Spent and Expired bid_state handling.
2018-05-26 22:56:09 -04:00
Mark Beamer Jr 78e26b2e6e -Parameterized elastic search logging to a const. 2018-05-26 22:49:29 -04:00
Mark Beamer Jr bb8525f69e -Fixed bug that causes completionmapper parse error in elastic search. This causes an entire batch to fail. Batches are up to 500 so if 1 fails they all fail. The impact is that many claims were not indexed in elastic search. 2018-05-26 22:47:26 -04:00
Mark Beamer Jr 4e1808c585 -fixed util import for chainquery sync
-added https for blocked files api
-added comma to commented section so query is intact if uncommented.
2018-05-15 00:03:07 -04:00
Mark Beamer Jr c6893991c3 -removed comma on last column of query.
-removed useless console log.
-added more understandable logging for which claims are pushed to elastic search.
2018-05-14 21:28:14 -04:00
Mark Beamer Jr b1c7f8b8ca -chainquery integration. Instead of using lbrycrd directly, use the chainquery app as the intermediary. 2018-05-12 19:35:56 -04:00
Mark Beamer Jr 04956a9e2c -export getvalue to be used for chainquery integration 2018-05-12 19:21:40 -04:00
Mark Beamer Jr 59af51ccec -Added call to sync if sync fails. 2018-05-09 21:41:14 -04:00
Mark Beamer Jr 7d55981ad8 Added functionality to delete any potentially blocked claims (sourced from internal-apis) from showing up in search results. 2018-04-07 21:53:15 -04:00
BelfordZ d285e4e485 fix(search) Update getAutoComplete 2017-10-19 20:33:07 -07:00
BelfordZ eb5c3acf7c fix(eslint) Run lint-fix 2017-10-12 20:56:21 -07:00
Fillerino d2bc8f2220 Bugfixes...
Fixed status bug and lbrycrdd rpc client not working properly which forced syncing to stop.
2017-09-26 20:27:44 +02:00
Fillerino 4b05a76e28 Added basic status API.
Added a basic status API, will be upgraded later on.
2017-09-26 19:39:03 +02:00
Fillerino b55d9b5619 Forgot to add os as import. 2017-09-26 19:09:23 +02:00
Fillerino 58ca2ebc23 Added so lighthouse gets the connection details from ~ config.
Added so lighthouse get the configuration for lbrycrdd from the ~/.lbrycrd/lbrycrd.conf file.
2017-09-26 19:08:42 +02:00
Fillerino fd7f751523 Fixed creation of claimTrieCache.json.
Fixes #22
2017-09-26 18:11:40 +02:00
Fillerino 57b4e7cad0 Added apiDoc and eslinted some code
Added basic apiDoc generation and eslinted some code.
2017-09-20 19:47:25 +02:00
Wallermadev 58c0fa6aa6 Improvements to search relevancy and seperation of search/autocomplete feature.
-- Removed `getSuggestion`

-- Added `getResults`
-- Added `getAutoComplete`

Restructured to search query to bool query that searches on a claims name, title, author and keywords in descripton -- `it's a disaster` now shows in a search for `disaster` autocomplete also returned correctly for the same example.
2017-09-20 02:49:42 +01:00
Fillerino 5c929f343c Added synctoolv2, claimTrie resolving instead of full chain!
Added synctoolv2, claimTrie resolving instead of full chain and some small patches!
2017-09-03 14:47:40 +02:00
Fillerino d64cf79f5d Fixed logging for elastic 2017-08-18 10:16:29 +02:00
Fillerino dd74ca7c0a Added the basic search, no sorting, no extra stuff.
Added the basic search, no sorting, no extra stuff, raw from elasticsearch.
2017-08-17 18:11:53 +02:00
Fillerino e9ff812273 Added importer as a module of the API server, importing now works.
Added importer as a module of the API server which cleans up some mess.
Fixes #6
2017-08-17 12:06:24 +02:00
Fillerino 1bcc272e0b Added the basic structure and fixed some issues!
Added the basic structure and fixed some issues!
Fixes #2, fixes # 3
2017-08-16 19:49:27 +02:00