LBRYCommunity/lbry-docker
1
0
Fork 0
Code Issues 2 Pull requests 2 Projects Releases 1 Packages Wiki Activity

Cleaned up documentation

Browse source 
Closes #49
This commit is contained in:
Leopere 2019-04-27 17:09:49 -04:00
parent 7c6f0b58a1
commit e8f6fa30d9
2 changed files with 43 additions and 40 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

45
chainquery/README.md
View file

@ -1,39 +1,42 @@
# Installation instructions for Chainquery and dependencies # Installation instructions for Chainquery and dependencies
## Preface ## Preface
I have set up this tutorial so that you should be able to simply copy and paste the commands into your linux console. However if you run into issues please be sure to read the detailed expansions on the current and prior steps to ensure you didn't miss a detail. I have set up this tutorial so that you should be able to copy and paste the commands into your Linux console. However, if you run into issues, please be sure to read the detailed expansions on the current and prior steps to ensure you didn't miss a detail.
## Caveats to installing Chainquery
Chainquery is an interface to an extensive index of data which has to parse an entire blockchain's history into a SQL database in a way that is easily query-able. The process of syncing your lbrycrd level.db files can take a long time depending on your deployment and then once your instance is synced you need to index the blockchain with Chainquery. All of this takes a fair bit of time or resources at first it is possible to deploy this stack without the shell scripts, and I will eventually include a docker-compose.yml file which gives you a baseline for deploying this to k8's and other platforms but for now single host deployment via shell script will be what's supported.
## Goals of this README.md ## Goals of this README.md
This readme is targeted towards people who need a copy pasta recipe to get Chainquery online and running asap and with some degree of repeatability. Once this is up and running on your machine you should be able to see what a running instance of Chainquery looks like and start writing apps against it's API calls etc. This readme is targeted towards people who need a copy paste recipe to get Chainquery online and running asap and with some degree of repeatability. Once this is up and running on your machine, you should be able to see what a running instance of Chainquery looks like and start writing apps against its API.
## Dependencies ## Dependencies
My goal is to avoid as many dependencies as possible so for now the only pre-requisites you are required to have will be `git`, `docker`, and `docker-compose` for the most part your understanding of the technologies can be superficial so long as you can follow commands and are open to reading a bit you should be fine. My goal is to avoid as many dependencies as possible so for now the only pre-requisites you are required to have are `git`, `docker`, and `docker-compose` for the most part your understanding of the technologies can be superficial so long as you can follow commands and are open to reading a bit you should be fine.
### Docker ### Docker
Docker
Docker is effectively the cross platform software package repository it allows you to ship an entire environment in what's referred to as a container. Implying that containers hold everything that is needed to ship what's inside effectively from one place to another with a degree of standard that makes it easy for everyone along the way to replicate what is needed for their step in the chain.
The other side of docker is it brings everything that we would normally have to teach you about the individual components of your soon to be installed Chainquery, Lbrycrd, and MySQL stack and wraps it up nicely in a handfull of easy to follow steps that should result in the same environment on everyones host. Docker is effectively the cross-platform software package repository it allows you to ship an entire environment in what's referred to as a container. Containers are intended to hold everything that is needed to ship what's required to run an application from one place to another with a degree of a standard that makes it easy for everyone along the way to reproduce the environment for their step in the chain.
The installation method we recommend is via the `docker.com` website however if your specific operating system's repository versions are at the latest along with docker you should be good to launch with you using whatever instructions you wish. The version of Docker we used in our deployment for testing this process was `Docker version 17.05.0-ce, build 89658be` however any versions later than this will suffice. At the writing of this tutorial this was not the latest version of Docker. The other side of docker is it brings everything that we would typically have to teach you about the individual components of your soon to be installed Chainquery, Lbrycrd, and MySQL stack and wraps it up nicely in a handful of easy to follow steps that should result in the same environment on everyone's host.
#### Just the link to installation instructions please The installation method we recommend is via the `docker.com` website however if your specific operating system's repository versions are at the latest along with docker you should be good to launch with you using whatever instructions you wish. The version of Docker we used in our deployment for testing this process was `Docker version 17.05.0-ce, build 89658be` however any versions later than this is sufficient. At the writing of this tutorial, this was not the latest version of Docker.
#### Just the link to installation instructions, please
Instructions for installing on Ubuntu are at the link that follows: Instructions for installing on Ubuntu are at the link that follows:
https://docs.docker.com/install/linux/docker-ce/ubuntu/ https://docs.docker.com/install/linux/docker-ce/ubuntu/
If you plan on using other versions of OS you should at least aim to be a linux base with a x64 CPU and the appropriate minimum version of the dependencies. If you plan on using other versions of OS you should at least aim to be a Linux base with an x86_64 CPU and the appropriate minimum version of the dependencies.
### Docker-compose ### Docker-compose
Docker Compose's role in this deployment is to get you a fully working cluster of microservices in containers that will deploy Chainquery, MySQL, and LBRYCrd exactly as you would need it to have a reasonable instance for testing / developing on. You are encouraged to learn how to use the Docker-Compose format but it's not a required prerequisite for getting this running you just need to have it installed successfully. Docker Compose's role in this deployment is to get you a fully working cluster of microservices in containers that will deploy Chainquery, MySQL, and LBRYCrd exactly as you would need it to have a reasonable instance for testing / developing on. You are encouraged to learn how to use the Docker-Compose format, but it's not a required prerequisite for getting this running you need to have it installed successfully.
Install Docker Compose via this guide below, its important if you're using an older version of linux to use the official documentation from Docker.com because you will need the more recent version of docker-compose at least version 3.4 aka 1.22.0 or newer. Install Docker Compose via this guide below, and it is essential if you're using an older version of Linux to use the official documentation from Docker.com because you require the more recent version of docker-compose at least version 3.4 aka 1.22.0 or newer.
#### Just the link to installation instructions please #### Just the link to installation instructions, please
https://docs.docker.com/compose/install/ https://docs.docker.com/compose/install/
### Git ### Git
For now the recommended install includes grabbing the latest git repository from https://github.com/lbryio/lbry-docker for this you're going to need to install git with the following command. The amount of git knowledge required for this is ideally fairly minimal. For now, the recommended install includes grabbing the latest git repository from https://github.com/lbryio/lbry-docker for this you're going to need to install git with the following command. The amount of git knowledge required for this is ideally reasonably minimal.
#### Just the instructions please #### Just the instructions, please
`apt-get install git -y` `apt-get install git -y`
## Get the lbry-docker repository ## Get the lbry-docker repository
@ -43,14 +46,14 @@ For now the recommended install includes grabbing the latest git repository from
## Setup networking ## Setup networking
You only need external networking if you plan on keeping your docker-compose files separate. You only need external networking if you plan on keeping your docker-compose files separate.
For the sake of modularity in the design of this git repository the plan is to give you examples to try and then you're supposed to move towards your own custom docker-compose configuration. We're going to create For the sake of modularity in the design of this git repository, the plan is to give you examples to try and then you're supposed to move towards your custom docker-compose configuration. We're going to create
a docker bridge network that is going to be managed externally to your usual docker-compose networks which are compose internal. a docker bridge network that is going to be managed externally to your usual docker-compose networks which are docker-compose internal.
`docker network create -d bridge --subnet=10.6.1.0/16 lbry-network` `docker network create -d bridge --subnet=10.6.1.0/16 lbry-network`
## Make directories and set permissions ## Make directories and set permissions
This is only required on host mounted volumes. (the default settings) Setting permissions is only typically required on host mounted volumes. (the default settings)
`sudo mkdir -p ./lbry-docker/lbrycrd/data` `sudo mkdir -p ./lbry-docker/lbrycrd/data`
@ -66,14 +69,14 @@ This is only required on host mounted volumes. (the default settings)
Wait for lbrycrd to reach the top of the blockchain (console output should noticeably slow down) Wait for lbrycrd to reach the top of the blockchain (console output should noticeably slow down)
Once you've reached the top of the blockchain you can press `CTRL+C` to exit back to the linux shell. Once you've reached the top of the blockchain, you can press `CTRL+C` to exit back to the Linux shell.
## Setup chainquery ## Setup chainquery
Now that you're done syncing your own copy of the lbry blockchain into the `lbrycrd` instance you can start spinning up Chainquery and it's dependencies. Since Chainquery is parsing copious amounts of unstructured data stored in the `lbry` blockchain you have two routes to go. Now you're finished syncing your copy of the lbry blockchain into the `lbrycrd` instance you can start spinning up Chainquery, and it's dependencies since Chainquery is parsing copious amounts of unstructured data stored in the `lbry` blockchain you have two routes to go.
1. Route number one [Recommended]: You use some variant of this README.md's instructions to start your chainquery instance using a LBRY.io provided database checkpoint snapshot. This should be basically copy paste-able series of commands to get your own staging instance up asap if you want Route number one, follow along with the rest of this README.md 1. Route number one [Recommended]: You use some variant of this README.md's instructions to start your chainquery instance using an LBRY.io provided database checkpoint snapshot. The checkpoint should be copy-paste-able series of commands to get your staging instance up asap if you want Route number one, follow along with the rest of this README.md
2. Route number two: You can your own fresh copy of the chainquery database indexing times may vary from hours to days depending on your hardware. 2. Route number two: You can your fresh copy of the chainquery database indexing times may vary from hours to days depending on your hardware.
`cd ../chainquery/` `cd ../chainquery/`

34
contrib/systemd-cloud-init.md
View file

@ -4,18 +4,18 @@ Contributing Author: [EnigmaCurry](https://www.enigmacurry.com)
Last Update: April 18 2019 Last Update: April 18 2019
This is meant to be easy instructions for running a lbrycrd and chainquery This tutorial meant to be easy instructions for running a lbrycrd and chainquery
service on DigitalOcean. It's pretty much just copy-and-paste. service on DigitalOcean. It's pretty much just copy-and-paste.
This should also work on any host that supports This tutorial should also work on any host that supports
[cloud-init](https://cloud-init.io/), but I've not tested it anywhere except for [cloud-init](https://cloud-init.io/), but I've not tested it anywhere except for
DigitalOcean. DigitalOcean.
If you wish to use docker-compose, there is an [alternative If you wish to use docker-compose, there is an [alternative
configuration](https://github.com/lbryio/lbry-docker/tree/master/lbrycrd) configuration](https://github.com/lbryio/lbry-docker/tree/master/lbrycrd)
for that. This tutorial will use cloud-init and systemd to control docker. for that. This tutorial uses cloud-init and systemd to control docker.
## It's easy to run your own full lbrycrd node ## It's easy to run your full lbrycrd node
[![Video of creating lbrycrd droplet on DigitalOcean](https://spee.ch/@EnigmaCurry:d/lbrycrd-video-thumb.jpg)](https://spee.ch/@EnigmaCurry:d/lbrycrd-docker-cloud-init.mp4) [![Video of creating lbrycrd droplet on DigitalOcean](https://spee.ch/@EnigmaCurry:d/lbrycrd-video-thumb.jpg)](https://spee.ch/@EnigmaCurry:d/lbrycrd-docker-cloud-init.mp4)
@ -33,7 +33,7 @@ for that. This tutorial will use cloud-init and systemd to control docker.
## DigitalOcean user-data for Ubuntu 18.04 droplet ## DigitalOcean user-data for Ubuntu 18.04 droplet
## Installs docker ## Installs docker
## Setup systemd service for lbrycrd ## Setup systemd service for lbrycrd
## (This config just runs docker on vanilla Ubuntu, ## (This config runs docker on vanilla Ubuntu,
## it uses systemd in place of docker-compose or kubernetes.) ## it uses systemd in place of docker-compose or kubernetes.)
write_files: write_files:
@ -83,7 +83,7 @@ write_files:
- path: "/etc/mysql/conf.d/chainquery.cnf" - path: "/etc/mysql/conf.d/chainquery.cnf"
content: | content: |
# Put mysql optimizations specific to chainquery here # Put MySQL optimizations specific to chainquery here
- path: "/etc/systemd/system/mysql.service" - path: "/etc/systemd/system/mysql.service"
content: | content: |
@ -171,7 +171,7 @@ runcmd:
- echo "Good to go." - echo "Good to go."
``` ```
* You can leave everything above as it is, to use the default configuration, OR * You can leave everything above as it is, to use the default configuration, OR
you may edit the config in the box to your own liking. you may edit the config in the box to your liking.
* For instance, if you wanted to run in [regtest * For instance, if you wanted to run in [regtest
mode](https://lbry.tech/resources/regtest-setup), you would set mode](https://lbry.tech/resources/regtest-setup), you would set
`regtest=1` in the first section under `write_files`. `regtest=1` in the first section under `write_files`.
@ -186,7 +186,7 @@ runcmd:
### How to administer the system ### How to administer the system
Copy the IP address from the droplet status page, SSH into the droplet as root Copy the IP address from the droplet status page, SSH into the droplet as root
using the same SSH key you configured for the droplet. using the same SSH key, you configured for the droplet.
The config file is in `/etc/lbry/lbrycrd.conf` on the host. The config file is in `/etc/lbry/lbrycrd.conf` on the host.
@ -208,7 +208,7 @@ Wait for the final `Good to go` message to know that the installer has finished.
#### Check the status of the systemd service #### Check the status of the systemd service
You can interact with systemd using `systemctl` (status, start, stop, restart, You can interact with systemd using `systemctl` (status, start, stop, restart,
etc.) and `journalctl` (logging) tools. and more.) and `journalctl` (logging) tools.
``` ```
systemctl status lbrycrd systemctl status lbrycrd
@ -235,7 +235,7 @@ docker logs lbrycrd
### Utilize lbrycrd-cli ### Utilize lbrycrd-cli
You can use lbrycrd-cli from the host console. A bash alias has been added to You can use lbrycrd-cli from the host console. A bash alias gets added to
`/root/.bash_aliases` that invokes lbrycrd-cli in its own container. `/root/.bash_aliases` that invokes lbrycrd-cli in its own container.
``` ```
@ -261,28 +261,28 @@ $ lbrycrd-cli getinfo
### Chainquery (optional) ### Chainquery (optional)
The chainquery service is pre-installed, but it is not enabled by default. The chainquery service is pre-installed but not enabled by default.
#### Enable and start the mysql service #### Enable and start the MySQL service
``` ```
systemctl enable --now mysql systemctl enable --now mysql
``` ```
In case you need it, there is a bash alias called `mysql` In case you need it, there is a bash alias called `mysql`.
(`/root/.bash_aliases`) for the mysql client that allows you to login to the (`/root/.bash_aliases`) for the MySQL client that allows you to log in to the
chainquery database. chainquery database.
#### Enable and start the chainquery service #### Enable and start the chainquery service
The chainquery config file is located on the host: `/etc/lbry/chainqueryconfig.toml` The chainquery config file located on the host at: `/etc/lbry/chainqueryconfig.toml`
``` ```
systemctl enable --now chainquery systemctl enable --now chainquery
``` ```
In systemd, when you enable a service, it means to always start the service at In systemd, when you enable a service, it means always to start the service at
system boot. (`--now` just means you also want to start the service right away.) system boot. (`--now` means you also want to start the service right away.)
As with any service, you can control chainquery with `systemctl` and get logs As with any service, you can control chainquery with `systemctl` and get logs
with `journalctl`: with `journalctl`: