Examples for using this repo to download LBRY content?

gmlewis commented

2020-06-07 17:47:00 +02:00

(Migrated from github.com)

I would like to write a client program that, given a LBRY URI (e.g. lbry://@irmf#a/irmf-logo-model-1.irmf#4),
the client downloads the data (provided that either the user has already paid for the content or the content is free).

Eventually, it would be nice to allow the user to optionally send credits if the content is gated and they have not already purchased it. But for starting out, I would just like to download the content that is either free or previously-paid-for.

I'm not finding any examples, and searching through the subdirectories, it is not obvious to me how to get started.

If I were to take a random stab-in-the-dark, for example, I might try doing the following:

lbrycrd.New(lbrycrdURL, chainParams) (What should lbrycrdURL and chainParams be?)

but then I get stuck, as none of the methods appear to be what I want.

Alternatively, maybe I need to go this route:

blobex.NewBlobExchangeClient(cc) (How should I get cc correctly?)
blobex.Download(ctx) (How do I specify the URI for the content?)

Can you please provide some guidance on the steps that need to be taken?

Thank you!

I would like to write a client program that, given a LBRY URI (e.g. `lbry://@irmf#a/irmf-logo-model-1.irmf#4`), the client downloads the data (provided that either the user has already paid for the content or the content is free). Eventually, it would be nice to allow the user to optionally send credits if the content is gated and they have not already purchased it. But for starting out, I would just like to download the content that is either free or previously-paid-for. I'm not finding any examples, and searching through the subdirectories, it is not obvious to me how to get started. If I were to take a random stab-in-the-dark, for example, I might try doing the following: * `lbrycrd.New(lbrycrdURL, chainParams)` (What should `lbrycrdURL` and `chainParams` be?) but then I get stuck, as none of the methods appear to be what I want. Alternatively, maybe I need to go this route: * `blobex.NewBlobExchangeClient(cc)` (How should I get `cc` correctly?) * `blobex.Download(ctx)` (How do I specify the URI for the content?) Can you please provide some guidance on the steps that need to be taken? Thank you!

lyoshenka commented

2020-06-08 21:51:50 +02:00

(Migrated from github.com)

@gmlewis thanks for your interest. Our Go codebase is a bit scattered right now because we've been focused on our primary Python implementation. We have this repo, and https://github.com/lbryio/reflector.go. What you're looking for is spread across both.

Here are some pointers to get you started on downloading something that's free. I'll omit the payment details for now.

the first step is to "resolve" a URL to get the sd hash. here's an example of how to do that. You'll need to interact with a wallet server to do this. We run a bunch for public use. Here's a list of some you can use.
after you get the sd hash, you have to look it up in the DHT to find peers that have that content. This is done using the DHT. Our Go implementation of the DHT lags behind the Python implementation, so it may not work quite right. That said, here's how to start up a dht node. Once its started, you can call dht.Get(sdHash) to get a list of peers for that hash.
once you know the SD hash and the peer you wish to download from, you can look at this code to see how to download the stream. That code assumes that a single peer has all the data. If it doesn't, you you can do the DHT lookup and the blob download for each individual blob.

I hope this is enough to get you started. I'm planning to unify our Go code into a single repo in the near future and provide clearer examples of the steps. If you want more nitty-gritty details, check out the LBRY spec.

What's your LBRY address? I'll send you some LBC to get you started.

@gmlewis thanks for your interest. Our Go codebase is a bit scattered right now because we've been focused on our [primary Python implementation](https://github.com/lbryio/lbry-sdk). We have this repo, and https://github.com/lbryio/reflector.go. What you're looking for is spread across both. Here are some pointers to get you started on downloading something that's free. I'll omit the payment details for now. - the first step is to "resolve" a URL to get the sd hash. here's [an example](https://github.com/lbryio/reflector.go/blob/master/cmd/resolve.go) of how to do that. You'll need to interact with a wallet server to do this. We run a bunch for public use. Here's [a list](https://github.com/lbryio/lbry-sdk/blob/3c8bec61d3267df28c191aa8323c1794cf933417/lbry/conf.py#L597) of some you can use. - after you get the sd hash, you have to look it up in the DHT to find peers that have that content. This is done using the DHT. Our Go implementation of the DHT lags behind the Python implementation, so it may not work quite right. That said, here's how to [start up a dht node](https://github.com/lbryio/reflector.go/blob/master/cmd/dht.go). Once its started, you can call `dht.Get(sdHash)` to get a list of peers for that hash. - once you know the SD hash and the peer you wish to download from, you can look at [this code](https://github.com/lbryio/reflector.go/blob/master/cmd/getstream.go) to see how to download the stream. That code assumes that a single peer has all the data. If it doesn't, you you can do the DHT lookup and the blob download for each individual blob. I hope this is enough to get you started. I'm planning to unify our Go code into a single repo in the near future and provide clearer examples of the steps. If you want more nitty-gritty details, check out the [LBRY spec](https://spec.lbry.com/). What's your LBRY address? I'll send you some LBC to get you started.

gmlewis commented

2020-06-09 01:02:27 +02:00

(Migrated from github.com)

Awesome! Thank you, @lyoshenka ... this is extremely helpful!
My LBRY address is: bQWkPubxVt9JdPuR2iZPcJrDX3BJcHncwG

Awesome! Thank you, @lyoshenka ... this is extremely helpful! My LBRY address is: bQWkPubxVt9JdPuR2iZPcJrDX3BJcHncwG

kauffj commented

2020-06-09 01:06:19 +02:00

(Migrated from github.com)

Welcome @gmlewis! Would love to know what you're working on, when you're ready :)

gmlewis commented

2020-06-09 01:16:17 +02:00

(Migrated from github.com)

Thank you, @kauffj!

My latest obsession is: http://irmf.io and specifically, I would like to deliver IRMF models via the LBRY protocol from within two of my programs so that the digital files are treated as first-class citizens and gated appropriately based on purchase history:

Both programs are written in Go, and the first one compiles to WebAssembly, which has been lots of fun to write. 😄

Thank you, @kauffj! My latest obsession is: http://irmf.io and specifically, I would like to deliver IRMF models via the LBRY protocol from within two of my programs so that the digital files are treated as first-class citizens and gated appropriately based on purchase history: * [irmf-editor](https://gmlewis.github.io/irmf-editor/?s=github.com/gmlewis/irmf/blob/master/examples/017-nostalgia/cos125.irmf) * [irmf-slicer](https://github.com/gmlewis/irmf-slicer) Both programs are written in Go, and the first one compiles to WebAssembly, which has been lots of fun to write. :smile:

gmlewis commented

2020-06-09 01:18:52 +02:00

(Migrated from github.com)

Oh, and on LBRY, I now have two channels: @irmf and @gmlewis.
The first one has a free IRMF test model, but I haven't published anything from the second channel yet. 😄

Oh, and on LBRY, I now have two channels: @irmf and @gmlewis. The first one has a free IRMF test model, but I haven't published anything from the second channel yet. :smile:

gmlewis commented

2020-06-09 01:25:57 +02:00

(Migrated from github.com)

Since irmf-editor is run in the web browser, starting up a dht node server is a non-starter, but if I can use a public wallet server as a launch-point without running a dht node, and still download the content, then I think this just might work!

lyoshenka commented

2020-06-15 15:57:14 +02:00

(Migrated from github.com)

@gmlewis how did it go?

gmlewis commented

2020-06-16 03:13:34 +02:00

(Migrated from github.com)

@lyoshenka - I actually hit a roadblock that I've been trying to solve.

As far as I can tell, github.com/spf13/viper is not WASM-friendly, and it is unfortunately a dependency of github.com/lbryio/reflector.go:

~/go/src/github.com/lbryio/reflector.go (master) $ go mod why github.com/spf13/viper
# github.com/spf13/viper
github.com/lbryio/reflector.go/wallet
github.com/lbryio/chainquery/lbrycrd
github.com/lbryio/chainquery/util
github.com/spf13/viper

The error I'm getting is:

~/go/src/github.com/gmlewis/irmf-editor (master) $ PATH=$(go env GOROOT)/misc/wasm:${PATH} GOARCH=wasm GOOS=js go test
# github.com/spf13/viper
../../../../pkg/mod/github.com/spf13/viper@v1.0.2/viper.go:264:19: undefined: fsnotify.NewWatcher
FAIL	github.com/gmlewis/irmf-editor [build failed]

Which appears to be this issue: https://github.com/desmos-labs/desmos/issues/2

So it looks like I'm currently stuck. 😞

@lyoshenka - I actually hit a roadblock that I've been trying to solve. As far as I can tell, `github.com/spf13/viper` is not WASM-friendly, and it is unfortunately a dependency of `github.com/lbryio/reflector.go`: ``` ~/go/src/github.com/lbryio/reflector.go (master) $ go mod why github.com/spf13/viper # github.com/spf13/viper github.com/lbryio/reflector.go/wallet github.com/lbryio/chainquery/lbrycrd github.com/lbryio/chainquery/util github.com/spf13/viper ``` The error I'm getting is: ``` ~/go/src/github.com/gmlewis/irmf-editor (master) $ PATH=$(go env GOROOT)/misc/wasm:${PATH} GOARCH=wasm GOOS=js go test # github.com/spf13/viper ../../../../pkg/mod/github.com/spf13/viper@v1.0.2/viper.go:264:19: undefined: fsnotify.NewWatcher FAIL github.com/gmlewis/irmf-editor [build failed] ``` Which appears to be this issue: https://github.com/desmos-labs/desmos/issues/2 So it looks like I'm currently stuck. :disappointed:

dpanic commented

2021-02-08 09:55:21 +01:00

(Migrated from github.com)

github.com/spf13/viper

Hey @gmlewis any progress on spf13/viper issue? I am facing the same.
Did you managed somehow to bypass it?

> github.com/spf13/viper Hey @gmlewis any progress on spf13/viper issue? I am facing the same. Did you managed somehow to bypass it?

gmlewis commented

2021-02-08 13:36:47 +01:00

(Migrated from github.com)

Hey @gmlewis any progress on spf13/viper issue? I am facing the same.
Did you managed somehow to bypass it?

No, @dpanic , unfortunately I'm still stuck. Please report back here if you find a solution or workaround! Thanks.

> Hey @gmlewis any progress on spf13/viper issue? I am facing the same. > Did you managed somehow to bypass it? No, @dpanic , unfortunately I'm still stuck. Please report back here if you find a solution or workaround! Thanks.

lyoshenka commented

2021-02-10 17:48:38 +01:00

(Migrated from github.com)

@gmlewis you could fork the repo and remove the dependency. you may have to change the way we load configs in 1 or 2 places, but feel free to hack around any roadblocks.

if that works for you, I can look into removing it from the main repo as part of a refactor we're planning. let me know how it goes

@gmlewis you could fork the repo and remove the dependency. you may have to change the way we load configs in 1 or 2 places, but feel free to hack around any roadblocks. if that works for you, I can look into removing it from the main repo as part of a refactor we're planning. let me know how it goes

👍 1

gmlewis commented

2021-02-13 03:36:00 +01:00

(Migrated from github.com)

I made some good progress by forking:

github.com/lbryio/reflector.go (to github.com/irmf/reflector.go)
github.com/lbryio/chainquery (to github.com/irmf/chainquery)

and then removing the dependency upon github.com/spf13/viper by modifying a single file...

but then was surprised by an unexpected error message:

../../../../pkg/mod/github.com/lbryio/lbry.go/v2@v2.6.1-0.20200901175808-73382bb02128/lbrycrd/client.go:120:3: 
cannot use chain (type chaincfg.Params) as type string in field value

I must have some version mismatches now that will take some more investigation.

I made some good progress by forking: * github.com/lbryio/reflector.go (to github.com/irmf/reflector.go) * github.com/lbryio/chainquery (to github.com/irmf/chainquery) and then removing the dependency upon github.com/spf13/viper by modifying a single file... but then was surprised by an unexpected error message: ``` ../../../../pkg/mod/github.com/lbryio/lbry.go/v2@v2.6.1-0.20200901175808-73382bb02128/lbrycrd/client.go:120:3: cannot use chain (type chaincfg.Params) as type string in field value ``` I must have some version mismatches now that will take some more investigation.

lyoshenka commented

2021-02-15 16:14:57 +01:00

(Migrated from github.com)

try putting this in your go.mod, above the require line

replace github.com/btcsuite/btcd => github.com/lbryio/lbrycrd.go v0.0.0-20200203050410-e1076f12bf19

@tiger5226 may know more about this error too

try putting this in your go.mod, above the require line ``` replace github.com/btcsuite/btcd => github.com/lbryio/lbrycrd.go v0.0.0-20200203050410-e1076f12bf19 ``` @tiger5226 may know more about this error too

gmlewis commented

2021-02-15 16:26:46 +01:00

(Migrated from github.com)

@lyoshenka - Sweet! I now have a successful build. Thank you!

I have to work now, but after work I'll see if I can download the (free) content from the lbry URI from my Go-based WebAssembly client.
The next step will be to figure out how to download paid content.

Have you guys implemented any kind of OpenID/OAuth2 "Log in with lbry.tv" or "Log in with Odysee.com" so that our client programs never have to deal with usernames and passwords even for paid content?

@lyoshenka - Sweet! I now have a successful build. Thank you! I have to work now, but after work I'll see if I can download the (free) content from the lbry URI from my Go-based WebAssembly client. The next step will be to figure out how to download paid content. Have you guys implemented any kind of OpenID/OAuth2 "Log in with lbry.tv" or "Log in with Odysee.com" so that our client programs never have to deal with usernames and passwords even for paid content?

gmlewis commented

2021-02-16 01:15:59 +01:00

(Migrated from github.com)

I just hit the next snag:

node.Connect: dial tcp: Protocol not available

Apparently, Go in WebAssembly doesn't support the tcp procotol:
https://stackoverflow.com/questions/55880920/dial-tcp-protocol-not-available-go-webassembly-test

I just hit the next snag: ``` node.Connect: dial tcp: Protocol not available ``` Apparently, Go in WebAssembly doesn't support the tcp procotol: https://stackoverflow.com/questions/55880920/dial-tcp-protocol-not-available-go-webassembly-test

tiger5226 commented

2021-02-16 17:32:01 +01:00

(Migrated from github.com)

This is true. Anything to be used in a browser is limited by browser APIs. We have the https://github.com/lbryio/lbrytv-player repository. This exposes a web server to download lbry content from a webpage which would be WASM compatible for you since its over http.

Since this is a native application that leverage WASM, it might be interesting to try an alternative route. I am not sure of your web development skills, but one thing I have done in the past with Go is packaging an app into a single binary that contains 2 pieces. A website, so that you can build a UI with a web language. It launches the webpage on startup. Then as part of the binary a web server/app is launched which is what the web page opens to. So that contains the APIs for the webpage to use locally.

It leverages some great Go Libs to make it all come together for a fast POC. Namely packr2. The interesting part is that if you had something similar you could use all the native go codebases without change and be able to handle your frontend and backend in the same app, while also leveraging a more native web experience like angular.

Just an idea to investigate.

This is true. Anything to be used in a browser is limited by browser APIs. We have the https://github.com/lbryio/lbrytv-player repository. This exposes a web server to download lbry content from a webpage which would be WASM compatible for you since its over http. Since this is a native application that leverage WASM, it might be interesting to try an alternative route. I am not sure of your web development skills, but one thing I have done in the past with Go is packaging an app into a single binary that contains 2 pieces. A website, so that you can build a UI with a web language. It launches the webpage on startup. Then as part of the binary a web server/app is launched which is what the web page opens to. So that contains the APIs for the webpage to use locally. It leverages some great Go Libs to make it all come together for a fast POC. Namely packr2. The interesting part is that if you had something similar you could use all the native go codebases without change and be able to handle your frontend and backend in the same app, while also leveraging a more native web experience like angular. Just an idea to investigate.

Examples for using this repo to download LBRY content? #83