LBRYCommunity/lighthouse.js
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Added channels to the index, added channel filtering, cleaned doc #92

Closed
filipnyquist wants to merge 1 commit from add-channel-clean-doc into master
Conversation 9 Commits 1 Files changed 110 +2249 -5613
filipnyquist commented 2018-07-20 13:31:11 +02:00 (Migrated from github.com)

This PR adds channels into the index, this combined with the added channel filtering parameter allows us to search specific channels for content.
While adding the code for filtering I split the search query into individual parts so it easier can be edited and read.
The documentation has been changed to swagger, which now means we follow the OpenAPI spec.
We now use the swagger spec to generate API documentation with the help of the yarn run gendoc command, which gets the latest API documentation from in line and adds it to the swagger file located in ./docs/swagger.json

This PR has been tested and is working locally. I still want someone else to test it to be sure that the indexing is working as it should.

This PR adds channels into the index, this combined with the added channel filtering parameter allows us to search specific channels for content. While adding the code for filtering I split the search query into individual parts so it easier can be edited and read. The documentation has been changed to swagger, which now means we follow the OpenAPI spec. We now use the swagger spec to generate API documentation with the help of the `yarn run gendoc` command, which gets the latest API documentation from in line and adds it to the swagger file located in ./docs/swagger.json This PR has been tested and is working locally. I still want someone else to test it to be sure that the indexing is working as it should.
kauffj commented 2018-07-20 16:29:11 +02:00 (Migrated from github.com)

@seanyesmunt and @akinwale can you create issues in your apps if there's anything to change or add on your side for this?

(I'm actually a bit unclear what channel searching means from an end-user UX perspective. Is the intention that someone would type channel: foo to add a filter to search only for content in a channel? If so, why would someone want to perform this search as opposed to viewing the channel?)

@seanyesmunt and @akinwale can you create issues in your apps if there's anything to change or add on your side for this? (I'm actually a bit unclear what channel searching means from an end-user UX perspective. Is the intention that someone would type `channel: foo` to add a filter to search only for content in a channel? If so, why would someone want to perform this search as opposed to viewing the channel?)
neb-b commented 2018-07-20 16:33:20 +02:00 (Migrated from github.com)

@kauffj Now we can add a search bar on the channel page to search for content specific to a channel

@kauffj Now we can add a search bar on the channel page to search for content specific to a channel
neb-b commented 2018-07-20 16:34:32 +02:00 (Migrated from github.com)

https://github.com/lbryio/lbry-desktop/issues/1735

https://github.com/lbryio/lbry-desktop/issues/1735
tiger5226 commented 2018-07-20 17:41:38 +02:00 (Migrated from github.com)

I will test this soon

I will test this soon
tiger5226 (Migrated from github.com) reviewed 2018-07-22 07:03:56 +02:00
tiger5226 (Migrated from github.com) left a comment

you switched to apidoc but its still in the package.json

you switched to apidoc but its still in the package.json
tiger5226 (Migrated from github.com) requested changes 2018-07-22 07:34:48 +02:00
docs/swagger.json
@ -0,0 +1 @@
{"info":{"title":"Lighthouse","version":"1.0.0","description":"This is the documentation for Lighthouse, LBRYs search engine server which handles searching on the LBRY blockchain.","contact":{"name":"Lighthouse","url":"https://github.com/lbryio/lighthouse"}},"swagger":"2.0","paths":{"/search":{"get":{"tags":["Lighthouse API"],"description":"Make a lighthouse search.","produces":["application/json"],"parameters":[{"name":"s","description":"The search text.","in":"query","required":true,"type":"string","example":"Why crypto is good!"},{"name":"channel","description":"The specific channel you want to search.","in":"query","required":false,"type":"string","example":"CryptoLovers"},{"name":"size","description":"The amount of items to get back at a maximum.","in":"query","required":false,"type":"integer","example":10},{"name":"from","description":"The number to start from, good for pagination.","in":"query","required":false,"type":"integer","example":10}],"responses":{"200":{"description":"The search returns all the found matching items.","schema":{"type":"array","items":{"$ref":"#/definitions/SearchItem"}}}}}},"/autocomplete":{"get":{"tags":["Lighthouse API"],"description":"The autocomplete API.","produces":["application/json"],"parameters":[{"name":"s","description":"The string to be autocompleted.","in":"query","required":true,"type":"string","example":"LBRY is"}],"responses":{"200":{"description":"The search returns all the found matching items.","schema":{"type":"array","items":{"$ref":"#/definitions/AutocompleteItem"}}}}}},"/status":{"get":{"tags":["Lighthouse API"],"description":"Gets the status of lighthouse.","produces":["application/json"],"responses":{"200":{"description":"Returns the current status of lighthouse","schema":{"type":"object","required":["spaceUsed","claimsInIndex","totSearches"],"properties":{"spaceUsed":{"type":"string","description":"The amount of space used for the search db."},"claimsInIndex":{"type":"integer","description":"The amount of claims in the index."},"totSearches":{"type":"integer","description":"The total amount of searches since the start."}}}}}}}},"definitions":{"SearchItem":{"type":"object","required":["name"],"properties":{"name":{"type":"string","description":"The name of the claim.","example":"lbry-testclaim"},"channel":{"type":"string","description":"The name of the channel that claimed this item.","example":"@lbrytestclaim"},"value":{"type":"object","description":"The decoded value of the metadata for the item.","example":"Here is the decoded claimdata/metadata from the claim in an object."}}},"AutocompleteItem":{"type":"string","description":"A matching search string.","example":"LBRY is amazing"}},"responses":{},"parameters":{},"securityDefinitions":{},"tags":[]}
tiger5226 (Migrated from github.com) commented 2018-07-22 07:34:39 +02:00

so plugging this into the swagger.io editor there are 12 errors. Can you take a look at those? I don't think this is a showstopper.

so plugging this into the swagger.io editor there are 12 errors. Can you take a look at those? I don't think this is a showstopper.
package.json
@ -31,3 +31,3 @@
"prod": "npm run build && node dist/",
"mocha": "./node_modules/.bin/mocha --compilers js:babel-register --require babel-polyfill",
"gendoc": "apidoc -i server/ -o docs/"
"gendoc": "node ./docs/genswagger.js"
tiger5226 (Migrated from github.com) commented 2018-07-22 07:05:02 +02:00

nothing else is removed from here?

nothing else is removed from here?
server/controllers/lighthouse.js
@ -41,0 +46,4 @@
},
},
},
};
tiger5226 (Migrated from github.com) commented 2018-07-22 07:16:17 +02:00

why do we have @ here? this should come through the api and it comes from chainquery too already in this format. It is actually a requirement of a channel. We could test for it to see if we were actually passed a channel, however, I don't think that is the job of the api anyway. The channel parameter is a text that we match against in the elastic search database.

why do we have `@` here? this should come through the api and it comes from chainquery too already in this format. It is actually a requirement of a channel. We could test for it to see if we were actually passed a channel, however, I don't think that is the job of the api anyway. The channel parameter is a text that we match against in the elastic search database.
server/controllers/lighthouse.js
@ -54,2 +163,4 @@
],
'must': [
channelSearch,
{
tiger5226 (Migrated from github.com) commented 2018-07-22 07:09:36 +02:00

why did you add channel to the source? This is not needed.

why did you add channel to the source? This is not needed.
tiger5226 commented 2018-07-22 07:37:07 +02:00 (Migrated from github.com)

@filipnyquist Can we make this 2 pull requests? the swagger stuff is flooding things and I am not sure if I missed anything important. Also the api docs are not really required for the functioning of the api. So worse case scenario we get the channel parameter and a little later we get the docs for example.

Overall great work man! how long did that swagger file take to create!? I know those things are tedious.

@filipnyquist Can we make this 2 pull requests? the swagger stuff is flooding things and I am not sure if I missed anything important. Also the api docs are not really required for the functioning of the api. So worse case scenario we get the channel parameter and a little later we get the docs for example. Overall great work man! how long did that swagger file take to create!? I know those things are tedious.
filipnyquist commented 2018-07-23 11:58:50 +02:00 (Migrated from github.com)

The swagger file was a pain in the ass, they take ages to fix. I´ll split this into two PRs. Closing this for now.

The swagger file was a pain in the ass, they take ages to fix. I´ll split this into two PRs. Closing this for now.
filipnyquist commented 2018-07-23 12:10:33 +02:00 (Migrated from github.com)

Moved the channel stuff to , doc will be in its own PR soon.

Moved the channel stuff to #92, doc will be in its own PR soon.

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
No reviewers
tiger5226
No labels
area: app c
area: app d
area: devops
area: discovery
area: docs
area: proposal
area: X-device Sync
Chainquery
consider soon
dependencies
Epic
Fix till next release
good first issue
hacktoberfest
help wanted
icebox
Invalid
level: 1
level: 2
level: 3
level: 4
needs: exploration
needs: grooming
needs: priority
needs: repro
needs: tech design
on hold
Parked
priority: blocker
priority: high
priority: low
priority: medium
Tom's Wishlist
type: bug
type: discussion
type: improvement
type: new feature
type: refactor
type: task
type: testing
unplanned
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/lighthouse.js#92
No description provided.
Delete branch "add-channel-clean-doc"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?