reflector.go/readme.md

# Reflector

Reflector is a central piece of software that providers LBRY with the following features:
- Blobs reflection: when something is published, we capture the data and store it on our servers for quicker retrieval
- Blobs distribution: when a piece of content is requested and the LBRY network doesn't have it, reflector will retrieve it from its storage and distribute it
- Blobs caching: reflectors can be chained together in multiple regions or servers to form a chain of cached content. We call those "blobcaches". They are layered so that content distribution is favorable in all the regions we deploy it to

There are a few other features embedded in reflector.go including publishing streams from Go, downloading or upload blobs, resolving content and more unfinished tools.

This code includes a Go implementations of the LBRY peer protocol, reflector protocol, and DHT.

## Installation

- Install mysql 8 (5.7 might work too)
- add a reflector user and database with password `reflector` with localhost access only
- Create the tables as described [here](https://github.com/lbryio/reflector.go/blob/ittt/db/db.go#L735) (the link might not update as the code does so just look for the schema in that file)

#### We do not support running reflector.go as a blob receiver, however if you want to run it as a private blobcache you may compile it yourself and run it as following:
```bash
./prism-bin reflector \
--conf="none" \
--disable-uploads=true \
--use-db=false \
--upstream-reflector="reflector.lbry.com" \
--upstream-protocol="http" \
--request-queue-size=200 \
--disk-cache="2GB:/path/to/your/storage/:localdb" \
```

Create a systemd script if you want to run it automatically on startup or as a service.

## Usage

Usage as reflector/blobcache:
```bash
Run reflector server

Usage:
  prism reflector [flags]

Flags:
      --disable-blocklist                 Disable blocklist watching/updating
      --disable-uploads                   Disable uploads to this reflector server
      --disk-cache string                 Where to cache blobs on the file system. format is 'sizeGB:CACHE_PATH:cachemanager' (cachemanagers: localdb/lfuda/lru) (default "100GB:/tmp/downloaded_blobs:localdb")
  -h, --help                              help for reflector
      --http-peer-port int                The port reflector will distribute content from over HTTP protocol (default 5569)
      --http3-peer-port int               The port reflector will distribute content from over HTTP3 protocol (default 5568)
      --mem-cache int                     enable in-memory cache with a max size of this many blobs
      --metrics-port int                  The port reflector will use for prometheus metrics (default 2112)
      --optional-disk-cache string        Optional secondary file system cache for blobs. format is 'sizeGB:CACHE_PATH:cachemanager' (cachemanagers: localdb/lfuda/lru) (this would get hit before the one specified in disk-cache)
      --origin-endpoint string            HTTP edge endpoint for standard HTTP retrieval
      --origin-endpoint-fallback string   HTTP edge endpoint for standard HTTP retrieval if first origin fails
      --receiver-port int                 The port reflector will receive content from (default 5566)
      --request-queue-size int            How many concurrent requests from downstream should be handled at once (the rest will wait) (default 200)
      --tcp-peer-port int                 The port reflector will distribute content from for the TCP (LBRY) protocol (default 5567)
      --upstream-protocol string          protocol used to fetch blobs from another upstream reflector server (tcp/http3/http) (default "http")
      --upstream-reflector string         host:port of a reflector server where blobs are fetched from
      --use-db                            Whether to connect to the reflector db or not (default true)

Global Flags:
      --conf string       Path to config. Use 'none' to disable (default "config.json")
  -v, --verbose strings   Verbose logging for specific components
```

Other uses:

```bash
Prism is a single entry point application with multiple sub modules which can be leveraged individually or together

Usage:
  prism [command]

Available Commands:
  check-integrity check blobs integrity for a given path
  cluster         Start(join) to or Start a new cluster
  decode          Decode a claim value
  dht             Run dht node
  getstream       Get a stream from a reflector server
  help            Help about any command
  peer            Run peer server
  populate-db     populate local database with blobs from a disk storage
  publish         Publish a file
  reflector       Run reflector server
  resolve         Resolve a URL
  send            Send a file to a reflector
  sendblob        Send a random blob to a reflector server
  start           Runs full prism application with cluster, dht, peer server, and reflector server.
  test            Test things
  upload          Upload blobs to S3
  version         Print the version

Flags:
      --conf string       Path to config. Use 'none' to disable (default "config.json")
  -h, --help              help for prism
  -v, --verbose strings   Verbose logging for specific components
```
## Running from Source

This project requires [Go v1.16](https://golang.org/doc/install).

On Ubuntu you can install it with `sudo snap install go --classic`

```
git clone git@github.com:lbryio/reflector.go.git
cd reflector.go
make
./bin/prism-bin
```

## Contributing

coming soon

## License

This project is MIT licensed.

## Security

We take security seriously. Please contact security@lbry.com regarding any security issues.
Our PGP key is [here](https://keybase.io/lbry/key.asc) if you need it.

## Contact
The primary contact for this project is [@Nikooo777](https://github.com/Nikooo777) (niko-at-lbry.com)