LBRYCommunity/lighthouse.js
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
228 commits 4 branches 0 tags 3.6 MiB
Commit graph

118 commits

Author SHA1 Message Date
marcdeb1
44ea0093ef Added NSFW filter to query 2018-10-17 11:42:25 +02:00
Mark Beamer Jr
5b2b274423
added blacklist for words. 
don't search words less than 3 characters
 2018-09-26 21:32:42 -04:00
Mark
6d52314f16
Merge pull request #114 from lbryio/trim_white_space 
Trim white space
 2018-09-23 10:35:37 -04:00
Mark
6e07477689
Merge pull request #115 from lbryio/addition_reserved_characters 
I was missing some additional special characters
 2018-09-23 10:35:12 -04:00
Mark Beamer Jr
56545cfed2
Fixed the auto-complete query to leverage the nested structure and to return only relevant claims and the text that triggered the auto-complete. 2018-09-16 01:52:19 -04:00
Mark Beamer Jr
6960a0084e
I was missing some additional special characters that needed to be escaped. 2018-09-15 22:13:01 -04:00
Mark Beamer Jr
ad533e3989
trimmed whitespace for search terms. 2018-09-15 21:44:15 -04:00
filipnyquist
fa18e29a8a
Update lighthouse.js 2018-08-22 19:56:09 +02:00
filipnyquist
b64ef9f97a
Added back #81 
Added back: 
Updated search query to include controlling and effective amount weights. Also made sure the results are sorted properly so highest score is at the top.
 2018-08-22 16:38:15 +02:00
Mark Beamer Jr
9469f953d6 added escape function to escape special characters for the query_string elastic search query. 2018-08-14 20:58:53 -04:00
Mark
aa4558bb57
Merge pull request #99 from lbryio/chainquery-fix 
Added the breaking change of #93. Added the change, do not merge unti…
 2018-08-12 13:53:24 -04:00
filipnyquist
7adbb67a2b Added swagger.json documentation instead of jsDoc. This PR also adds ReDoc to our documentation instad of apiDoc. Fixes #97 2018-08-05 15:19:11 +02:00
filipnyquist
6b3269bfe3 Added the breaking change of #93. Added the change, do not merge until the chainquery change has been merged and deployed. 2018-07-30 15:19:29 +02:00
filipnyquist
4423fc2cfc This PR adds channels into the index, this combined with the added channel filtering parameter allows us to search specific channels for content. 
While adding the code for filtering I split the search query into individual parts so it easier can be edited and read.
 2018-07-23 12:06:47 +02:00
Mark Beamer Jr
d9a4af9636 removed effective amount function score from query. 2018-07-04 00:00:18 -04:00
Mark Beamer Jr
49a7c42e73 removed logging reference. 2018-07-03 23:51:39 -04:00
filipnyquist
22bf2b8a5d Added better winston logging, this saves logs up to 14days, then rotates them. This allows for easier debugging over time. 2018-06-28 14:24:00 +02:00
filipnyquist
d418179feb Added a ENV variable for the elasticsearch url for easier debugging. 2018-06-27 15:04:41 +02:00
filipnyquist
0557855fab
Revert "Revert "Updated search query"" 2018-06-23 18:23:15 +02:00
filipnyquist
13a465fce3
Revert "Updated search query" 2018-06-21 18:24:57 +02:00
filipnyquist
e9d7d75da6
Merge pull request #81 from lbryio/controlling_weight 
Updated search query
 2018-06-21 17:07:43 +02:00
Mark Beamer Jr
1b041f8e87 Updated search query to include controlling and effective amount weights. Also made sure the results are sorted properly so highest score is at the top. 2018-06-20 22:33:58 -04:00
Mark Beamer Jr
dfcc5e3ef4 check against value not just _source 2018-06-20 22:17:59 -04:00
Mark Beamer Jr
20eedbc660 Fixed bug caused by blank claims ( deleted ) which resulted in internal server error. 2018-06-20 19:18:09 -04:00
Mark Beamer Jr
cd0f980904 -Refactored handling of blocked claims. 
-Changed deletes to be queued and refactored.
-Added logging.
-Removed commented debug code.
-Added Spent and Expired bid_state handling.
 2018-05-26 22:56:09 -04:00
Mark Beamer Jr
78e26b2e6e -Parameterized elastic search logging to a const. 2018-05-26 22:49:29 -04:00
Mark Beamer Jr
bb8525f69e -Fixed bug that causes completionmapper parse error in elastic search. This causes an entire batch to fail. Batches are up to 500 so if 1 fails they all fail. The impact is that many claims were not indexed in elastic search. 2018-05-26 22:47:26 -04:00
Mark Beamer Jr
4e1808c585 -fixed util import for chainquery sync 
-added https for blocked files api
-added comma to commented section so query is intact if uncommented.
 2018-05-15 00:03:07 -04:00
Mark Beamer Jr
c6893991c3 -removed comma on last column of query. 
-removed useless console log.
-added more understandable logging for which claims are pushed to elastic search.
 2018-05-14 21:28:14 -04:00
Mark Beamer Jr
b1c7f8b8ca -chainquery integration. Instead of using lbrycrd directly, use the chainquery app as the intermediary. 2018-05-12 19:35:56 -04:00
Mark Beamer Jr
04956a9e2c -export getvalue to be used for chainquery integration 2018-05-12 19:21:40 -04:00
Mark Beamer Jr
59af51ccec -Added call to sync if sync fails. 2018-05-09 21:41:14 -04:00
Mark Beamer Jr
b7e0cb327a -added check to make sure size + from stays under 10,000 2018-04-29 22:09:28 -04:00
Mark Beamer Jr
69d770b032 Added size for number of results to get and from for pagination. 2018-04-26 20:11:18 -04:00
filipnyquist
b76b7b7a58
Merge pull request #61 from lbryio/autocomplete_fix 
The internal server error was because we were checking for a null val…
 2018-04-08 09:17:02 +02:00
Mark Beamer Jr
482efbdb5f The internal server error was because we were checking for a null value for a claim. This is already checked and should never be null. The intention is to make sure we can get the right information out which is only available in a stream. However, there is no stream for channel claims, only certificates. So the fix is to check for null on the stream not the value. 2018-04-08 02:36:57 -04:00
Mark Beamer Jr
47890ad56e Updated API docs to fix documentation errors on github pages which is the target of the documentation. The main culprit was not having cors enabled to handle the same origin policy of the browsers. With the koa-cors package now enabled this will work perfectly. 2018-04-08 02:13:27 -04:00
Mark Beamer Jr
7d55981ad8 Added functionality to delete any potentially blocked claims (sourced from internal-apis) from showing up in search results. 2018-04-07 21:53:15 -04:00
Mark Beamer Jr
c32f8e9367 I modified the query to be more complex so that relevancy is considered. 
Features of the query are below in order of importance(weight):

1) Phrase Match on Name
2) Term Match on Name
3) Contains Term on Name
4) Phrase Match on Author,Title,Description
5) Term Match on Author,Title,Description
6) Contains Term on Author,Title,Description
 2018-03-14 23:18:15 -04:00
Mark Beamer Jr
90e2d3f64b changed weight to 5 from 100. I made the weight 5 based on tests around "super" which is a query that returns 1800 hits. The intention is to make sure that channels show up first in the results. 2018-03-13 21:43:08 -04:00
Mark Beamer Jr
00a41e9e19 using ^n on a field is only valid syntax for lycene. This caused the name field to never be search which resulted in channels not being searchable. I added the boost parameter which is what is supposed to be used. I made the weight 5 based tests around "super" which is a query that returns 1800 hits. The intention is to make sure that channels show up first in the results. 2018-03-13 21:37:57 -04:00
Mark Beamer Jr
674dddca8a updated search query for elasticsearch. value is stored as a nested object in the elastic document. Therefore to query fields at that level or lower, a nested query is required. 2018-03-08 22:09:48 -05:00
BelfordZ
d285e4e485 fix(search) Update getAutoComplete 2017-10-19 20:33:07 -07:00
BelfordZ
eb5c3acf7c fix(eslint) Run lint-fix 2017-10-12 20:56:21 -07:00
filipnyquist
9c7cf63d13 Changed so root path will redir to github. 
Changed so root path will redir to github project.
 2017-10-03 22:41:53 +02:00
Fillerino
55037a2cd6 Updated example. 2017-09-28 22:34:04 +02:00
Fillerino
27749a76d0 Updated port 
Lets not roll with 80....
 2017-09-26 20:30:29 +02:00
Fillerino
d2bc8f2220 Bugfixes... 
Fixed status bug and lbrycrdd rpc client not working properly which forced syncing to stop.
 2017-09-26 20:27:44 +02:00
Fillerino
4b05a76e28 Added basic status API. 
Added a basic status API, will be upgraded later on.
 2017-09-26 19:39:03 +02:00
Fillerino
b55d9b5619 Forgot to add os as import. 2017-09-26 19:09:23 +02:00
Fillerino
58ca2ebc23 Added so lighthouse gets the connection details from ~ config. 
Added so lighthouse get the configuration for lbrycrdd from the ~/.lbrycrd/lbrycrd.conf file.
 2017-09-26 19:08:42 +02:00
Fillerino
c026893c32 Lets stick to base-path for API. 
Removed /api/lighthouse from path, updated API doc.
 2017-09-26 18:40:33 +02:00
Fillerino
fd7f751523 Fixed creation of claimTrieCache.json. 
Fixes #22
 2017-09-26 18:11:40 +02:00
Fillerino
8d364be153 Updated api documentation 
Updated api documentation to match the api
 2017-09-20 22:41:11 +02:00
Fillerino
72c16eb656 Merge branch 'master' into master 2017-09-20 22:17:09 +02:00
Wallermadev
40a5ef384a now only return feilds needed in application, 
Wildcard searches added to allow partial matches.
 2017-09-20 19:34:06 +00:00
Fillerino
57b4e7cad0 Added apiDoc and eslinted some code 
Added basic apiDoc generation and eslinted some code.
 2017-09-20 19:47:25 +02:00
Wallermadev
66fcf4c833 Adding Basic Pagination support using size/from params. 2017-09-20 04:01:06 +01:00
Wallermadev
c6fab81e61 Improvements to autocomplete functionality 
-- Array is now clensed of null and duplicate values and returned in a flat array for ease of use.
 2017-09-20 03:17:11 +01:00
Wallermadev
17ee066134 Removing console.log from lighthouse.js 2017-09-20 02:52:11 +01:00
Wallermadev
58c0fa6aa6 Improvements to search relevancy and seperation of search/autocomplete feature. 
-- Removed `getSuggestion`

-- Added `getResults`
-- Added `getAutoComplete`

Restructured to search query to bool query that searches on a claims name, title, author and keywords in descripton -- `it's a disaster` now shows in a search for `disaster` autocomplete also returned correctly for the same example.
 2017-09-20 02:49:42 +01:00
Fillerino
d5470ce0e3 Fixed the search api endpoint to work for now. 2017-09-12 19:25:01 +02:00
Fillerino
229e4fa8c8 Added ansible and decoder 
Added ansible playbook and decoder into this repo.
 2017-09-05 20:11:29 +02:00
Fillerino
5c929f343c Added synctoolv2, claimTrie resolving instead of full chain! 
Added synctoolv2, claimTrie resolving instead of full chain and some small patches!
 2017-09-03 14:47:40 +02:00
Fillerino
d64cf79f5d Fixed logging for elastic 2017-08-18 10:16:29 +02:00
Fillerino
dd74ca7c0a Added the basic search, no sorting, no extra stuff. 
Added the basic search, no sorting, no extra stuff, raw from elasticsearch.
 2017-08-17 18:11:53 +02:00
Fillerino
e9ff812273 Added importer as a module of the API server, importing now works. 
Added importer as a module of the API server which cleans up some mess.
Fixes #6
 2017-08-17 12:06:24 +02:00
Fillerino
1bcc272e0b Added the basic structure and fixed some issues! 
Added the basic structure and fixed some issues!
Fixes #2, fixes # 3
 2017-08-16 19:49:27 +02:00