Updated documentation
This commit is contained in:
Leopere
parent
832c2f6b1b
commit
0e537ef3e0
|
|
@ -1,39 +1,42 @@
|
|||
# Installation instructions for Chainquery and dependencies
|
||||
## Preface
|
||||
I have set up this tutorial so that you should be able to simply copy and paste the commands into your linux console. However if you run into issues please be sure to read the detailed expansions on the current and prior steps to ensure you didn't miss a detail.
|
||||
I have set up this tutorial so that you should be able to copy and paste the commands into your Linux console. However, if you run into issues, please be sure to read the detailed expansions on the current and prior steps to ensure you didn't miss a detail.
|
||||
|
||||
## Caveats to installing Chainquery
|
||||
Chainquery is an interface to an extensive index of data which has to parse an entire blockchain's history into a SQL database in a way that is easily query-able. The process of syncing your lbrycrd level.db files can take a long time depending on your deployment and then once your instance is synced you need to index the blockchain with Chainquery. All of this takes a fair bit of time or resources at first it is possible to deploy this stack without the shell scripts, and I will eventually include a docker-compose.yml file which gives you a baseline for deploying this to k8's and other platforms but for now single host deployment via shell script will be what's supported.
|
||||
|
||||
## Goals of this README.md
|
||||
This readme is targeted towards people who need a copy pasta recipe to get Chainquery online and running asap and with some degree of repeatability. Once this is up and running on your machine you should be able to see what a running instance of Chainquery looks like and start writing apps against it's API calls etc.
|
||||
This readme is targeted towards people who need a copy paste recipe to get Chainquery online and running asap and with some degree of repeatability. Once this is up and running on your machine, you should be able to see what a running instance of Chainquery looks like and start writing apps against its API.
|
||||
|
||||
## Dependencies
|
||||
My goal is to avoid as many dependencies as possible so for now the only pre-requisites you are required to have will be `git`, `docker`, and `docker-compose` for the most part your understanding of the technologies can be superficial so long as you can follow commands and are open to reading a bit you should be fine.
|
||||
My goal is to avoid as many dependencies as possible so for now the only pre-requisites you are required to have are `git`, `docker`, and `docker-compose` for the most part your understanding of the technologies can be superficial so long as you can follow commands and are open to reading a bit you should be fine.
|
||||
|
||||
### Docker
|
||||
Docker
|
||||
Docker is effectively the cross platform software package repository it allows you to ship an entire environment in what's referred to as a container. Implying that containers hold everything that is needed to ship what's inside effectively from one place to another with a degree of standard that makes it easy for everyone along the way to replicate what is needed for their step in the chain.
|
||||
|
||||
The other side of docker is it brings everything that we would normally have to teach you about the individual components of your soon to be installed Chainquery, Lbrycrd, and MySQL stack and wraps it up nicely in a handfull of easy to follow steps that should result in the same environment on everyones host.
|
||||
Docker is effectively the cross-platform software package repository it allows you to ship an entire environment in what's referred to as a container. Containers are intended to hold everything that is needed to ship what's required to run an application from one place to another with a degree of a standard that makes it easy for everyone along the way to reproduce the environment for their step in the chain.
|
||||
|
||||
The installation method we recommend is via the `docker.com` website however if your specific operating system's repository versions are at the latest along with docker you should be good to launch with you using whatever instructions you wish. The version of Docker we used in our deployment for testing this process was `Docker version 17.05.0-ce, build 89658be` however any versions later than this will suffice. At the writing of this tutorial this was not the latest version of Docker.
|
||||
The other side of docker is it brings everything that we would typically have to teach you about the individual components of your soon to be installed Chainquery, Lbrycrd, and MySQL stack and wraps it up nicely in a handful of easy to follow steps that should result in the same environment on everyone's host.
|
||||
|
||||
#### Just the link to installation instructions please
|
||||
The installation method we recommend is via the `docker.com` website however if your specific operating system's repository versions are at the latest along with docker you should be good to launch with you using whatever instructions you wish. The version of Docker we used in our deployment for testing this process was `Docker version 17.05.0-ce, build 89658be` however any versions later than this is sufficient. At the writing of this tutorial, this was not the latest version of Docker.
|
||||
|
||||
#### Just the link to installation instructions, please
|
||||
Instructions for installing on Ubuntu are at the link that follows:
|
||||
https://docs.docker.com/install/linux/docker-ce/ubuntu/
|
||||
|
||||
If you plan on using other versions of OS you should at least aim to be a linux base with a x64 CPU and the appropriate minimum version of the dependencies.
|
||||
If you plan on using other versions of OS you should at least aim to be a Linux base with an x86_64 CPU and the appropriate minimum version of the dependencies.
|
||||
|
||||
### Docker-compose
|
||||
Docker Compose's role in this deployment is to get you a fully working cluster of microservices in containers that will deploy Chainquery, MySQL, and LBRYCrd exactly as you would need it to have a reasonable instance for testing / developing on. You are encouraged to learn how to use the Docker-Compose format but it's not a required prerequisite for getting this running you just need to have it installed successfully.
|
||||
Docker Compose's role in this deployment is to get you a fully working cluster of microservices in containers that will deploy Chainquery, MySQL, and LBRYCrd exactly as you would need it to have a reasonable instance for testing / developing on. You are encouraged to learn how to use the Docker-Compose format, but it's not a required prerequisite for getting this running you need to have it installed successfully.
|
||||
|
||||
Install Docker Compose via this guide below, its important if you're using an older version of linux to use the official documentation from Docker.com because you will need the more recent version of docker-compose at least version 3.4 aka 1.22.0 or newer.
|
||||
Install Docker Compose via this guide below, and it is essential if you're using an older version of Linux to use the official documentation from Docker.com because you require the more recent version of docker-compose at least version 3.4 aka 1.22.0 or newer.
|
||||
|
||||
#### Just the link to installation instructions please
|
||||
#### Just the link to installation instructions, please
|
||||
https://docs.docker.com/compose/install/
|
||||
|
||||
### Git
|
||||
For now the recommended install includes grabbing the latest git repository from https://github.com/lbryio/lbry-docker for this you're going to need to install git with the following command. The amount of git knowledge required for this is ideally fairly minimal.
|
||||
For now, the recommended install includes grabbing the latest git repository from https://github.com/lbryio/lbry-docker for this you're going to need to install git with the following command. The amount of git knowledge required for this is ideally reasonably minimal.
|
||||
|
||||
#### Just the instructions please
|
||||
#### Just the instructions, please
|
||||
`apt-get install git -y`
|
||||
|
||||
## Get the lbry-docker repository
|
||||
|
|
@ -43,14 +46,14 @@ For now the recommended install includes grabbing the latest git repository from
|
|||
## Setup networking
|
||||
|
||||
You only need external networking if you plan on keeping your docker-compose files separate.
|
||||
For the sake of modularity in the design of this git repository the plan is to give you examples to try and then you're supposed to move towards your own custom docker-compose configuration. We're going to create
|
||||
a docker bridge network that is going to be managed externally to your usual docker-compose networks which are compose internal.
|
||||
For the sake of modularity in the design of this git repository, the plan is to give you examples to try and then you're supposed to move towards your custom docker-compose configuration. We're going to create
|
||||
a docker bridge network that is going to be managed externally to your usual docker-compose networks which are docker-compose internal.
|
||||
|
||||
`docker network create -d bridge --subnet=10.6.1.0/16 lbry-network`
|
||||
|
||||
## Make directories and set permissions
|
||||
|
||||
This is only required on host mounted volumes. (the default settings)
|
||||
Setting permissions is only typically required on host mounted volumes. (the default settings)
|
||||
|
||||
`sudo mkdir -p ./lbry-docker/lbrycrd/data`
|
||||
|
||||
|
|
@ -66,17 +69,19 @@ This is only required on host mounted volumes. (the default settings)
|
|||
|
||||
Wait for lbrycrd to reach the top of the blockchain (console output should noticeably slow down)
|
||||
|
||||
Once you've reached the top of the blockchain you can press `CTRL+C` to exit back to the linux shell.
|
||||
Once you've reached the top of the blockchain, you can press `CTRL+C` to exit back to the Linux shell.
|
||||
|
||||
## Setup chainquery
|
||||
|
||||
Now that you're done syncing your own copy of the lbry blockchain into the `lbrycrd` instance you can start spinning up Chainquery and it's dependencies. Since Chainquery is parsing copious amounts of unstructured data stored in the `lbry` blockchain you have two routes to go.
|
||||
Now you're finished syncing your copy of the lbry blockchain into the `lbrycrd` instance you can start spinning up Chainquery, and it's dependencies since Chainquery is parsing copious amounts of unstructured data stored in the `lbry` blockchain you have two routes to go.
|
||||
|
||||
1. Route number one [Recommended]: You use some variant of this README.md's instructions to start your chainquery instance using a LBRY.io provided database checkpoint snapshot. This should be basically copy paste-able series of commands to get your own staging instance up asap if you want Route number one, follow along with the rest of this README.md
|
||||
2. Route number two: You can your own fresh copy of the chainquery database indexing times may vary from hours to days depending on your hardware.
|
||||
1. Route number one [Recommended]: You use some variant of this README.md's instructions to start your chainquery instance using an LBRY.io provided database checkpoint snapshot. The checkpoint should be copy-paste-able series of commands to get your staging instance up asap if you want Route number one, follow along with the rest of this README.md
|
||||
2. Route number two: You can your fresh copy of the chainquery database indexing times may vary from hours to days depending on your hardware.
|
||||
|
||||
`cd ../chainquery/`
|
||||
|
||||
`cat ./compose/docker-compose.yml-prod-example > docker-compose.yml`
|
||||
|
||||
`./quick-bootstrap.sh getdata`
|
||||
|
||||
`./quick-bootstrap.sh extract`
|
||||
|
|
|
|||
|
|
@ -4,18 +4,18 @@ Contributing Author: [EnigmaCurry](https://www.enigmacurry.com)
|
|||
|
||||
Last Update: April 18 2019
|
||||
|
||||
This is meant to be easy instructions for running a lbrycrd and chainquery
|
||||
This tutorial meant to be easy instructions for running a lbrycrd and chainquery
|
||||
service on DigitalOcean. It's pretty much just copy-and-paste.
|
||||
|
||||
This should also work on any host that supports
|
||||
This tutorial should also work on any host that supports
|
||||
[cloud-init](https://cloud-init.io/), but I've not tested it anywhere except for
|
||||
DigitalOcean.
|
||||
|
||||
If you wish to use docker-compose, there is an [alternative
|
||||
configuration](https://github.com/lbryio/lbry-docker/tree/master/lbrycrd)
|
||||
for that. This tutorial will use cloud-init and systemd to control docker.
|
||||
for that. This tutorial uses cloud-init and systemd to control docker.
|
||||
|
||||
## It's easy to run your own full lbrycrd node
|
||||
## It's easy to run your full lbrycrd node
|
||||
|
||||
[](https://spee.ch/@EnigmaCurry:d/lbrycrd-docker-cloud-init.mp4)
|
||||
|
||||
|
|
@ -26,15 +26,15 @@ for that. This tutorial will use cloud-init and systemd to control docker.
|
|||
* Select a Standard droplet with 8GB of memory ($40 per month in 2019.)
|
||||
* Select whatever datacenter you want.
|
||||
* Mark the checkbox called `User data`, and paste the following into the box:
|
||||
|
||||
|
||||
```
|
||||
#cloud-config
|
||||
|
||||
## DigitalOcean user-data for Ubuntu 18.04 droplet
|
||||
## Installs docker
|
||||
## Setup systemd service for lbrycrd
|
||||
## (This config just runs docker on vanilla Ubuntu,
|
||||
## it uses systemd inplace of docker-compose or kubernetes.)
|
||||
## (This config runs docker on vanilla Ubuntu,
|
||||
## it uses systemd in place of docker-compose or kubernetes.)
|
||||
|
||||
write_files:
|
||||
- path: "/etc/lbry/lbrycrd.conf"
|
||||
|
|
@ -57,10 +57,10 @@ write_files:
|
|||
Description=lbrycrd docker container
|
||||
After=snap.docker.dockerd.service
|
||||
Requires=snap.docker.dockerd.service
|
||||
|
||||
|
||||
[Service]
|
||||
Environment=SERVICE=lbrycrd
|
||||
Environment=IMAGE=lbry/lbry-docker:lbrycrd-production
|
||||
Environment=IMAGE=lbry/lbrycrd:linux-x86-64-production
|
||||
TimeoutStartSec=0
|
||||
ExecStartPre=-/snap/bin/docker stop $SERVICE
|
||||
ExecStartPre=-/snap/bin/docker rm -f $SERVICE
|
||||
|
|
@ -77,14 +77,14 @@ write_files:
|
|||
ExecStop=/snap/bin/docker stop $SERVICE
|
||||
Restart=always
|
||||
RestartSec=60
|
||||
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
- path: "/etc/mysql/conf.d/chainquery.cnf"
|
||||
content: |
|
||||
# Put mysql optimizations specific to chainquery here
|
||||
|
||||
# Put MySQL optimizations specific to chainquery here
|
||||
|
||||
- path: "/etc/systemd/system/mysql.service"
|
||||
content: |
|
||||
[Unit]
|
||||
|
|
@ -112,7 +112,7 @@ write_files:
|
|||
ExecStop=/snap/bin/docker stop $SERVICE
|
||||
Restart=always
|
||||
RestartSec=60
|
||||
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
|
|
@ -122,7 +122,7 @@ write_files:
|
|||
lbrycrdurl="rpc://lbry:lbry@lbrycrd:9245"
|
||||
mysqldsn="chainquery:chainquery@tcp(mysql:3306)/chainquery"
|
||||
apimysqldsn="chainquery:chainquery@tcp(mysql:3306)/chainquery"
|
||||
|
||||
|
||||
- path: "/etc/systemd/system/chainquery.service"
|
||||
content: |
|
||||
[Unit]
|
||||
|
|
@ -133,7 +133,7 @@ write_files:
|
|||
|
||||
[Service]
|
||||
Environment=SERVICE=chainquery
|
||||
Environment=IMAGE=lbry/lbry-docker:chainquery-production
|
||||
Environment=IMAGE=lbry/chainquery:linux-x86-64-production
|
||||
TimeoutStartSec=0
|
||||
ExecStartPre=-/snap/bin/docker stop $SERVICE
|
||||
ExecStartPre=-/snap/bin/docker rm -f $SERVICE
|
||||
|
|
@ -149,7 +149,7 @@ write_files:
|
|||
ExecStop=/snap/bin/docker stop $SERVICE
|
||||
Restart=always
|
||||
RestartSec=60
|
||||
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
||||
|
|
@ -157,7 +157,7 @@ write_files:
|
|||
- path: "/root/.bash_aliases"
|
||||
content: |
|
||||
alias lbrycrd-cli="docker run --rm -it --link lbrycrd:lbrycrd --mount type=bind,source=/etc/lbry/lbrycrd.conf,target=/etc/lbry/lbrycrd.conf \
|
||||
lbry/lbry-docker:lbrycrd-production lbrycrd-cli -conf=/etc/lbry/lbrycrd.conf -rpcconnect=lbrycrd"
|
||||
lbry/lbrycrd:linux-x86-64-production lbrycrd-cli -conf=/etc/lbry/lbrycrd.conf -rpcconnect=lbrycrd"
|
||||
alias mysql="docker run --rm -it --link mysql:mysql mysql:5 mysql -hmysql -u chainquery --password=chainquery"
|
||||
|
||||
runcmd:
|
||||
|
|
@ -169,15 +169,15 @@ runcmd:
|
|||
- /snap/bin/docker volume create mysql-data
|
||||
- systemctl enable --now lbrycrd
|
||||
- echo "Good to go."
|
||||
```
|
||||
```
|
||||
* You can leave everything above as it is, to use the default configuration, OR
|
||||
you may edit the config in the box to your own liking.
|
||||
you may edit the config in the box to your liking.
|
||||
* For instance, if you wanted to run in [regtest
|
||||
mode](https://lbry.tech/resources/regtest-setup), you would set
|
||||
`regtest=1` in the first section under `write_files`.
|
||||
* You can also edit the config files at any later point in `/etc/lbry`,
|
||||
after you create the droplet.
|
||||
* Select your SSH key so you can login.
|
||||
* Select your SSH key so you can log in.
|
||||
* Give it a good hostname.
|
||||
* Click Create.
|
||||
|
||||
|
|
@ -186,7 +186,7 @@ runcmd:
|
|||
### How to administer the system
|
||||
|
||||
Copy the IP address from the droplet status page, SSH into the droplet as root
|
||||
using the same SSH key you configured for the droplet.
|
||||
using the same SSH key, you configured for the droplet.
|
||||
|
||||
The config file is in `/etc/lbry/lbrycrd.conf` on the host.
|
||||
|
||||
|
|
@ -199,7 +199,7 @@ startup.
|
|||
You can tail the log to monitor the install progress:
|
||||
|
||||
```
|
||||
tail -f /var/log/cloud-init-output.log
|
||||
tail -f /var/log/cloud-init-output.log
|
||||
```
|
||||
|
||||
Wait for the final `Good to go` message to know that the installer has finished.
|
||||
|
|
@ -208,7 +208,7 @@ Wait for the final `Good to go` message to know that the installer has finished.
|
|||
#### Check the status of the systemd service
|
||||
|
||||
You can interact with systemd using `systemctl` (status, start, stop, restart,
|
||||
etc.) and `journalctl` (logging) tools.
|
||||
and more.) and `journalctl` (logging) tools.
|
||||
|
||||
```
|
||||
systemctl status lbrycrd
|
||||
|
|
@ -235,7 +235,7 @@ docker logs lbrycrd
|
|||
|
||||
### Utilize lbrycrd-cli
|
||||
|
||||
You can use lbrycrd-cli from the host console. A bash alias has been added to
|
||||
You can use lbrycrd-cli from the host console. A bash alias gets added to
|
||||
`/root/.bash_aliases` that invokes lbrycrd-cli in its own container.
|
||||
|
||||
```
|
||||
|
|
@ -261,28 +261,28 @@ $ lbrycrd-cli getinfo
|
|||
|
||||
### Chainquery (optional)
|
||||
|
||||
The chainquery service is pre-installed, but it is not enabled by default.
|
||||
The chainquery service is pre-installed but not enabled by default.
|
||||
|
||||
#### Enable and start the mysql service
|
||||
#### Enable and start the MySQL service
|
||||
|
||||
```
|
||||
systemctl enable --now mysql
|
||||
```
|
||||
|
||||
In case you need it, there is a bash alias called `mysql`
|
||||
(`/root/.bash_aliases`) for the mysql client that allows you to login to the
|
||||
In case you need it, there is a bash alias called `mysql`.
|
||||
(`/root/.bash_aliases`) for the MySQL client that allows you to log in to the
|
||||
chainquery database.
|
||||
|
||||
#### Enable and start the chainquery service
|
||||
|
||||
The chainquery config file is located on the host: `/etc/lbry/chainqueryconfig.toml`
|
||||
The chainquery config file located on the host at: `/etc/lbry/chainqueryconfig.toml`
|
||||
|
||||
```
|
||||
systemctl enable --now chainquery
|
||||
```
|
||||
|
||||
In systemd, when you enable a service, it means to always start the service at
|
||||
system boot. (`--now` just means you also want to start the service right away.)
|
||||
In systemd, when you enable a service, it means always to start the service at
|
||||
system boot. (`--now` means you also want to start the service right away.)
|
||||
|
||||
As with any service, you can control chainquery with `systemctl` and get logs
|
||||
with `journalctl`:
|
||||
|
|
@ -304,7 +304,7 @@ journalctl --unit chainquery
|
|||
|
||||
(optionally use `-f` if you want to tail/follow the logs)
|
||||
|
||||
##### Disabling chainquery service
|
||||
##### Disabling chainquery service
|
||||
|
||||
```
|
||||
systemctl disable --now chainquery
|
||||
|
|
@ -319,4 +319,3 @@ In the future, this tutorial may replace the snap version of docker with the
|
|||
regular PPA version of docker-ce, which has a more predictable update strategy
|
||||
(apt-get) rather than auto-updates. More long term testing is needed to know
|
||||
which way is better.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue