Commit graph

40 commits

Author SHA1 Message Date
Mark Beamer Jr
4b88db1f55
Cleanup the sync
Add syncstate to status call
Remove old sync logic

Signed-off-by: Mark Beamer Jr <markbeamerjr@gmail.com>
2019-09-21 18:47:23 -04:00
Mark Beamer Jr
42bec75ef9
Set last sync time to start of sync when doing a long running sync.
Signed-off-by: Mark Beamer Jr <markbeamerjr@gmail.com>
2019-09-21 18:09:11 -04:00
Mark Beamer Jr
162107239b
Add capability to restart sync where it left off by storing state of the sync. Set to update
Signed-off-by: Mark Beamer Jr <markbeamerjr@gmail.com>
2019-09-21 17:25:18 -04:00
Mark Beamer Jr
c67f298ee6
Add channel id to claim if exists for more precise filtering 2019-09-19 22:15:52 -04:00
Thomas Zarebczan
c002a09d7e fix: eslint problems 2019-09-18 12:47:21 -04:00
Niko Storni
b873d93bf6 .io to .com 2019-07-08 20:00:10 +02:00
Mark Beamer Jr
db39a6c47f
skip bad claim when processing to ensure all future claims make it into elastic search. 2019-06-22 16:37:39 -04:00
Mark Beamer Jr
11d000119b
check for null value to notify lbry team. 2019-06-22 16:17:40 -04:00
Mark Beamer Jr
ea2fcb56ab
log error when step in sync process fails. 2019-06-21 00:11:16 -04:00
Mark Beamer Jr
4f5206d18e
Check for Claim field 2019-06-20 23:26:36 -04:00
Mark Beamer Jr
5a9ce272b9
Added reverted changes back in 2019-02-26 08:50:26 -05:00
Mark
79a2470504
Revert "added channel effective amount to the claim weight." 2019-02-24 22:05:39 -05:00
Mark Beamer Jr
101e55d924
added channel effective amount to the claim weight. 2019-02-24 17:03:50 -05:00
Mark Beamer Jr
81986315cb
Batched the chainquery calls for lighthouse in case it needs to grab all 700K in a redo. 2019-02-24 16:57:56 -05:00
Mark Beamer Jr
965746b9d2
added slack integration 2018-11-04 01:01:23 -04:00
filipnyquist
6b3269bfe3 Added the breaking change of #93. Added the change, do not merge until the chainquery change has been merged and deployed. 2018-07-30 15:19:29 +02:00
filipnyquist
4423fc2cfc This PR adds channels into the index, this combined with the added channel filtering parameter allows us to search specific channels for content.
While adding the code for filtering I split the search query into individual parts so it easier can be edited and read.
2018-07-23 12:06:47 +02:00
Mark Beamer Jr
cd0f980904 -Refactored handling of blocked claims.
-Changed deletes to be queued and refactored.
-Added logging.
-Removed commented debug code.
-Added Spent and Expired bid_state handling.
2018-05-26 22:56:09 -04:00
Mark Beamer Jr
78e26b2e6e -Parameterized elastic search logging to a const. 2018-05-26 22:49:29 -04:00
Mark Beamer Jr
bb8525f69e -Fixed bug that causes completionmapper parse error in elastic search. This causes an entire batch to fail. Batches are up to 500 so if 1 fails they all fail. The impact is that many claims were not indexed in elastic search. 2018-05-26 22:47:26 -04:00
Mark Beamer Jr
4e1808c585 -fixed util import for chainquery sync
-added https for blocked files api
-added comma to commented section so query is intact if uncommented.
2018-05-15 00:03:07 -04:00
Mark Beamer Jr
c6893991c3 -removed comma on last column of query.
-removed useless console log.
-added more understandable logging for which claims are pushed to elastic search.
2018-05-14 21:28:14 -04:00
Mark Beamer Jr
b1c7f8b8ca -chainquery integration. Instead of using lbrycrd directly, use the chainquery app as the intermediary. 2018-05-12 19:35:56 -04:00
Mark Beamer Jr
04956a9e2c -export getvalue to be used for chainquery integration 2018-05-12 19:21:40 -04:00
Mark Beamer Jr
59af51ccec -Added call to sync if sync fails. 2018-05-09 21:41:14 -04:00
Mark Beamer Jr
7d55981ad8 Added functionality to delete any potentially blocked claims (sourced from internal-apis) from showing up in search results. 2018-04-07 21:53:15 -04:00
BelfordZ
d285e4e485 fix(search) Update getAutoComplete 2017-10-19 20:33:07 -07:00
BelfordZ
eb5c3acf7c fix(eslint) Run lint-fix 2017-10-12 20:56:21 -07:00
Fillerino
d2bc8f2220 Bugfixes...
Fixed status bug and lbrycrdd rpc client not working properly which forced syncing to stop.
2017-09-26 20:27:44 +02:00
Fillerino
4b05a76e28 Added basic status API.
Added a basic status API, will be upgraded later on.
2017-09-26 19:39:03 +02:00
Fillerino
b55d9b5619 Forgot to add os as import. 2017-09-26 19:09:23 +02:00
Fillerino
58ca2ebc23 Added so lighthouse gets the connection details from ~ config.
Added so lighthouse get the configuration for lbrycrdd from the ~/.lbrycrd/lbrycrd.conf file.
2017-09-26 19:08:42 +02:00
Fillerino
fd7f751523 Fixed creation of claimTrieCache.json.
Fixes #22
2017-09-26 18:11:40 +02:00
Fillerino
57b4e7cad0 Added apiDoc and eslinted some code
Added basic apiDoc generation and eslinted some code.
2017-09-20 19:47:25 +02:00
Wallermadev
58c0fa6aa6 Improvements to search relevancy and seperation of search/autocomplete feature.
-- Removed `getSuggestion`

-- Added `getResults`
-- Added `getAutoComplete`

Restructured to search query to bool query that searches on a claims name, title, author and keywords in descripton -- `it's a disaster` now shows in a search for `disaster` autocomplete also returned correctly for the same example.
2017-09-20 02:49:42 +01:00
Fillerino
5c929f343c Added synctoolv2, claimTrie resolving instead of full chain!
Added synctoolv2, claimTrie resolving instead of full chain and some small patches!
2017-09-03 14:47:40 +02:00
Fillerino
d64cf79f5d Fixed logging for elastic 2017-08-18 10:16:29 +02:00
Fillerino
dd74ca7c0a Added the basic search, no sorting, no extra stuff.
Added the basic search, no sorting, no extra stuff, raw from elasticsearch.
2017-08-17 18:11:53 +02:00
Fillerino
e9ff812273 Added importer as a module of the API server, importing now works.
Added importer as a module of the API server which cleans up some mess.
Fixes #6
2017-08-17 12:06:24 +02:00
Fillerino
1bcc272e0b Added the basic structure and fixed some issues!
Added the basic structure and fixed some issues!
Fixes #2, fixes # 3
2017-08-16 19:49:27 +02:00