More removals

Renamed to fit with new nomenclature

Removed directory and renamed/moved Dockerfile

Updated COPY path

Moved docker-compose.yml for Lbrynet
Cleaned up comments a bit

Important notice to users in new comment

Updated compose example for image instead of build

updated paths

Added a step for chainquery-bootstrap setup

Moved and updated lbrycrd's env_file

Created lbrynet env_file

Moved and updated chainquery's env_file

Moved and updated lbrynet's compose file

More Cleanup

Added TODO's for checkmount.sh

Renamed compiler

Renamed production container

Final rename

fixup

fixup

Updated README refs

Just readability improvements

Packing directories with the repo

Git won't track empty directories but if you take the path of running these containers locally from within this repo you should have this directory.

Updated image refs to final production locations
This commit is contained in:
Leopere 2019-04-25 20:41:42 -04:00 committed by Ryan McGuire
parent 44f8f86308
commit 93ea84bb81
18 changed files with 50 additions and 49 deletions

View file

@ -4,16 +4,16 @@
Document tags and link to their Dockerfiles here. Document tags and link to their Dockerfiles here.
## Scope ## Scope
This repository is in heavy flux as it travels towards [version 1.0](https://github.com/lbryio/lbry-docker/projects/1) however its goal is to make development for and adoption of any of the LBRY appliances trivial. You should be able to clone pull fork your way to a better LBRY without having to do much more than some light reading of a README to get started. This repository is in flux as it travels towards [version 1.0](https://github.com/lbryio/lbry-docker/projects/1); however its goal is to make development for and adoption of any of the LBRY appliances trivial. You should be able to clone pull fork your way to a better LBRY without having to do much more than some light reading of a README to get started.
#### Documentation is WIP #### Documentation is WIP
Currently this repository is a WIP and is under heavy construction, use at your own risk make sure you keep regular backups of your wallets. Your milage may vary as how far this will work for you be sure to file good and concise issues if you plan to and keep in mind we're allergic to regressions when filing PR's. Currently, this repository is a WIP and is under heavy construction, use at your own risk make sure you keep regular backups of your wallets. Your mileage may vary as to how far this will work for you be sure to file good and concise issues if you plan to and be mindful of we're allergic to regressions when filing PRs.
#### Goals #### Goals
This repository aims for [Docker Best Practices](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/) minimalism, ephemerality, and statelessness. It will use well commented **`Shell scripts`**, Dockerfiles, and Docker-Compose which is a template that is a baseline for many other container management services such as [RancherOS](https://rancher.com/rancher-os/) and [Kubernetes](https://kubernetes.io/). This repository aims for [Docker Best Practices](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/) minimalism, ephemerality, and statelessness. It will use well commented **`Shell scripts`**, Dockerfiles, and Docker-Compose which is a template that is a baseline for many other container management services such as [RancherOS](https://rancher.com/rancher-os/) and [Kubernetes](https://kubernetes.io/).
#### Beautiful Screenshot/Gif #### Beautiful Screenshot/Gif
Since there is nothing to see here just yet I'll entertain you with the current state of affairs with this repository. Since there is nothing to see here just yet, I'll entertain you with the current state of affairs with this repository.
![image](https://spee.ch/855d1958650b850b249b9ee592ba2f4c6fc7eeec/container-unloading-gone-wrong-151175.gif) ![image](https://spee.ch/855d1958650b850b249b9ee592ba2f4c6fc7eeec/container-unloading-gone-wrong-151175.gif)
## Installation ## Installation
@ -23,7 +23,7 @@ See [Running from source](##Running-from-source) for the current instructions on
#### Currently supported platforms #### Currently supported platforms
**X86_64 cpu architecture** **X86_64 CPU architecture**
Lbrynet, Lbrycrd, Chainqery are currently supported. Lbrynet, Lbrycrd, Chainqery are currently supported.
**Some arm architectures** **Some arm architectures**
@ -31,25 +31,25 @@ Lbrynet, Lbrycrd, Chainqery are currently supported.
**More will be added on request and over time** **More will be added on request and over time**
## Usage ## Usage
For now I don't recommend using this container cluster however you're welcomed to [contribute](#contributing) if you feel up to the task. For now, I don't recommend using this container cluster however you're welcomed to [contribute](#contributing) if you feel up to the task.
## Running from source ## Running from source
Running this stuff from source should be possible if you have both [Docker](https://docs.docker.com/install/) and [docker-compose](https://docs.docker.com/compose/install/) both installed. If these are both installed you can proceed to run the following from within your development directory. Running this stuff from source should be possible if you have both [Docker](https://docs.docker.com/install/) and [docker-compose](https://docs.docker.com/compose/install/) both installed. If these are both installed, you can proceed to run the following from within your development directory.
``` ```
git clone https://github.com/lbryio/lbry-docker.git git clone https://github.com/lbryio/lbry-docker.git
``` ```
Once you have a local copy of the recent source you will want to consider what containers/applications you require in your environment. At the moment since at the writing of this documentation this comes with an assertion of [YMMV](https://dictionary.cambridge.org/dictionary/english/ymmv) so if something isn't working feel free to make suggestions in the form of a PR for how we should do this better. The beauty of Open Source is learning better ways to do things as well as contributing to the world so I'm always going to be welcoming to contributions. Once you have a local copy of the recent source, you will want to consider what containers/applications you require in your environment. At the moment since at the writing of this documentation, this comes with an assertion of [YMMV](https://dictionary.cambridge.org/dictionary/english/ymmv) so if something isn't working feel free to make suggestions in the form of a PR for how we should do this better. The beauty of Open Source is learning better ways to do things as well as contributing to the world, so I'm always going to be welcoming to contributions.
#### From Source for Contributions #### From Source for Contributions
Running from source for contributing and Merge/Pull requests. Running from source for contributing and Merge/Pull requests.
My goal is to make contributing to this possible using Docker and also GitLab CI/CD time. My goal is to make contributing to this possible using Docker and also GitLab CI/CD time.
## [Contributing](CONTRIBUTING.md) ## [Contributing](CONTRIBUTING.md)
Keep in mind [I am](https://github.com/leopere/) preferential to receiving patches over rule following as we can always nudge you in the right direction to get things more compatible with the project ethos if it's not. Never be afraid to file a PR no one should be offended. This said following the next two guides will greatly improve the speed at which we can integrate your improvements. Keep in mind [I am](https://github.com/leopere/) preferential to receiving patches over rule-following as we can always nudge you in the right direction to get things more compatible with the project ethos if it's not. Never be afraid to file a PR no one should be offended. Having said this following the next two guides will greatly improve the speed at which we can integrate your improvements.
* [Repository Standards]( https://lbry.tech/resources/repository-standards) * [Repository Standards]( https://lbry.tech/resources/repository-standards)
* [Contribute](https://lbry.tech/contribute) * [Contribute](https://lbry.tech/contribute)
* Have a LBC wallet ready as we want you to have some for the help! Hell why not post it in your Commit or Merge Request for all I care but take your tips! * Have an LBC wallet ready as we want you to have some for the help! Hell, why not post it in your Commit or Merge Request for all I care but take your tips!
## Getting Support ## Getting Support

View file

@ -8,7 +8,7 @@ RUN apt-get update && \
rm -rf /var/lib/apt/lists/* rm -rf /var/lib/apt/lists/*
WORKDIR / WORKDIR /
SHELL ["/bin/bash", "-o", "pipefail", "-c"] SHELL ["/bin/bash", "-o", "pipefail", "-c"]
COPY ../stuff/start.sh start COPY ./stuff/start.sh start
RUN curl -L -o /chainquery.zip $(curl -s https://api.github.com/repos/lbryio/chainquery/releases | grep -F 'Linux_x86_64.zip' | grep download | head -n 1 | cut -d'"' -f4) && \ RUN curl -L -o /chainquery.zip $(curl -s https://api.github.com/repos/lbryio/chainquery/releases | grep -F 'Linux_x86_64.zip' | grep download | head -n 1 | cut -d'"' -f4) && \
unzip /chainquery.zip && \ unzip /chainquery.zip && \
rm /chainquery.zip rm /chainquery.zip

View file

@ -77,6 +77,8 @@ Now that you're done syncing your own copy of the lbry blockchain into the `lbry
`cd ../chainquery/` `cd ../chainquery/`
`cat ./compose/docker-compose.yml-prod-example > docker-compose.yml`
`./quick-bootstrap.sh getdata` `./quick-bootstrap.sh getdata`
`./quick-bootstrap.sh extract` `./quick-bootstrap.sh extract`

View file

@ -19,7 +19,7 @@ services:
aliases: aliases:
- mysql - mysql
env_file: env_file:
- env - ../environment/chainquery.env
expose: expose:
- 3306 - 3306
## TODO: I want to find a way that is acceptable to everyone to lock this up ## TODO: I want to find a way that is acceptable to everyone to lock this up
@ -42,7 +42,7 @@ services:
ipv4_address: 10.6.1.3 ipv4_address: 10.6.1.3
env_file: env_file:
- env - env
- ../lbrycrd/env - ../environment/lbrycrd.env
labels: labels:
- "traefik.expose=false" - "traefik.expose=false"
expose: expose:
@ -52,5 +52,5 @@ services:
depends_on: depends_on:
- mysql - mysql
## TODO: Uncomment this in a docker-compose.override.yml to allow for external configurations. ## TODO: Uncomment this in a docker-compose.override.yml to allow for external configurations.
# volumes: volumes:
# - ../persist/chainquery/config/chainqueryconfig.toml:/etc/chainquery/chainqueryconfig.toml - ../persist/chainquery/config/chainqueryconfig.toml:/etc/chainquery/chainqueryconfig.toml

View file

@ -26,7 +26,7 @@ for that. This tutorial will use cloud-init and systemd to control docker.
* Select a Standard droplet with 8GB of memory ($40 per month in 2019.) * Select a Standard droplet with 8GB of memory ($40 per month in 2019.)
* Select whatever datacenter you want. * Select whatever datacenter you want.
* Mark the checkbox called `User data`, and paste the following into the box: * Mark the checkbox called `User data`, and paste the following into the box:
``` ```
#cloud-config #cloud-config
@ -57,10 +57,10 @@ write_files:
Description=lbrycrd docker container Description=lbrycrd docker container
After=snap.docker.dockerd.service After=snap.docker.dockerd.service
Requires=snap.docker.dockerd.service Requires=snap.docker.dockerd.service
[Service] [Service]
Environment=SERVICE=lbrycrd Environment=SERVICE=lbrycrd
Environment=IMAGE=lbry/lbry-docker:lbrycrd-production Environment=IMAGE=lbry/lbrycrd:linux-x86-64-production
TimeoutStartSec=0 TimeoutStartSec=0
ExecStartPre=-/snap/bin/docker stop $SERVICE ExecStartPre=-/snap/bin/docker stop $SERVICE
ExecStartPre=-/snap/bin/docker rm -f $SERVICE ExecStartPre=-/snap/bin/docker rm -f $SERVICE
@ -77,14 +77,14 @@ write_files:
ExecStop=/snap/bin/docker stop $SERVICE ExecStop=/snap/bin/docker stop $SERVICE
Restart=always Restart=always
RestartSec=60 RestartSec=60
[Install] [Install]
WantedBy=multi-user.target WantedBy=multi-user.target
- path: "/etc/mysql/conf.d/chainquery.cnf" - path: "/etc/mysql/conf.d/chainquery.cnf"
content: | content: |
# Put mysql optimizations specific to chainquery here # Put mysql optimizations specific to chainquery here
- path: "/etc/systemd/system/mysql.service" - path: "/etc/systemd/system/mysql.service"
content: | content: |
[Unit] [Unit]
@ -112,7 +112,7 @@ write_files:
ExecStop=/snap/bin/docker stop $SERVICE ExecStop=/snap/bin/docker stop $SERVICE
Restart=always Restart=always
RestartSec=60 RestartSec=60
[Install] [Install]
WantedBy=multi-user.target WantedBy=multi-user.target
@ -122,7 +122,7 @@ write_files:
lbrycrdurl="rpc://lbry:lbry@lbrycrd:9245" lbrycrdurl="rpc://lbry:lbry@lbrycrd:9245"
mysqldsn="chainquery:chainquery@tcp(mysql:3306)/chainquery" mysqldsn="chainquery:chainquery@tcp(mysql:3306)/chainquery"
apimysqldsn="chainquery:chainquery@tcp(mysql:3306)/chainquery" apimysqldsn="chainquery:chainquery@tcp(mysql:3306)/chainquery"
- path: "/etc/systemd/system/chainquery.service" - path: "/etc/systemd/system/chainquery.service"
content: | content: |
[Unit] [Unit]
@ -133,7 +133,7 @@ write_files:
[Service] [Service]
Environment=SERVICE=chainquery Environment=SERVICE=chainquery
Environment=IMAGE=lbry/lbry-docker:chainquery-production Environment=IMAGE=lbry/chainquery:linux-x86-64-production
TimeoutStartSec=0 TimeoutStartSec=0
ExecStartPre=-/snap/bin/docker stop $SERVICE ExecStartPre=-/snap/bin/docker stop $SERVICE
ExecStartPre=-/snap/bin/docker rm -f $SERVICE ExecStartPre=-/snap/bin/docker rm -f $SERVICE
@ -149,7 +149,7 @@ write_files:
ExecStop=/snap/bin/docker stop $SERVICE ExecStop=/snap/bin/docker stop $SERVICE
Restart=always Restart=always
RestartSec=60 RestartSec=60
[Install] [Install]
WantedBy=multi-user.target WantedBy=multi-user.target
@ -157,7 +157,7 @@ write_files:
- path: "/root/.bash_aliases" - path: "/root/.bash_aliases"
content: | content: |
alias lbrycrd-cli="docker run --rm -it --link lbrycrd:lbrycrd --mount type=bind,source=/etc/lbry/lbrycrd.conf,target=/etc/lbry/lbrycrd.conf \ alias lbrycrd-cli="docker run --rm -it --link lbrycrd:lbrycrd --mount type=bind,source=/etc/lbry/lbrycrd.conf,target=/etc/lbry/lbrycrd.conf \
lbry/lbry-docker:lbrycrd-production lbrycrd-cli -conf=/etc/lbry/lbrycrd.conf -rpcconnect=lbrycrd" lbry/lbrycrd:linux-x86-64-production lbrycrd-cli -conf=/etc/lbry/lbrycrd.conf -rpcconnect=lbrycrd"
alias mysql="docker run --rm -it --link mysql:mysql mysql:5 mysql -hmysql -u chainquery --password=chainquery" alias mysql="docker run --rm -it --link mysql:mysql mysql:5 mysql -hmysql -u chainquery --password=chainquery"
runcmd: runcmd:
@ -169,7 +169,7 @@ runcmd:
- /snap/bin/docker volume create mysql-data - /snap/bin/docker volume create mysql-data
- systemctl enable --now lbrycrd - systemctl enable --now lbrycrd
- echo "Good to go." - echo "Good to go."
``` ```
* You can leave everything above as it is, to use the default configuration, OR * You can leave everything above as it is, to use the default configuration, OR
you may edit the config in the box to your own liking. you may edit the config in the box to your own liking.
* For instance, if you wanted to run in [regtest * For instance, if you wanted to run in [regtest
@ -199,7 +199,7 @@ startup.
You can tail the log to monitor the install progress: You can tail the log to monitor the install progress:
``` ```
tail -f /var/log/cloud-init-output.log tail -f /var/log/cloud-init-output.log
``` ```
Wait for the final `Good to go` message to know that the installer has finished. Wait for the final `Good to go` message to know that the installer has finished.
@ -304,7 +304,7 @@ journalctl --unit chainquery
(optionally use `-f` if you want to tail/follow the logs) (optionally use `-f` if you want to tail/follow the logs)
##### Disabling chainquery service ##### Disabling chainquery service
``` ```
systemctl disable --now chainquery systemctl disable --now chainquery
@ -319,4 +319,3 @@ In the future, this tutorial may replace the snap version of docker with the
regular PPA version of docker-ce, which has a more predictable update strategy regular PPA version of docker-ce, which has a more predictable update strategy
(apt-get) rather than auto-updates. More long term testing is needed to know (apt-get) rather than auto-updates. More long term testing is needed to know
which way is better. which way is better.

View file

@ -1,4 +1,4 @@
COMPOSE_PROJECT_NAME=chainquery ## Contains environment variables for chainquery containers
######################### #########################
## Chainquery Settings ## ## Chainquery Settings ##

View file

@ -1,5 +1,4 @@
COMPOSE_PROJECT_NAME=lbrycrd ## Contains environment variables for lbrycrd containers
############# #############
## Lbrycrd ## ## Lbrycrd ##
############# #############

2
environment/lbrynet.env Normal file
View file

@ -0,0 +1,2 @@
## Contains environment variables for lbrynet containers
RUN_MODE=default

View file

@ -7,10 +7,10 @@ RUN apt-get update && \
rm -rf /var/lib/apt/lists/* rm -rf /var/lib/apt/lists/*
WORKDIR / WORKDIR /
SHELL ["/bin/bash", "-o", "pipefail", "-c"] SHELL ["/bin/bash", "-o", "pipefail", "-c"]
COPY ../stuff/start.sh start COPY ./stuff/start.sh start
COPY ../stuff/healthcheck.sh healthcheck COPY ./stuff/healthcheck.sh healthcheck
COPY ../stuff/advance_blocks.sh advance COPY ./stuff/advance_blocks.sh advance
COPY ../stuff/fix-permissions.c fix-permissions.c COPY ./stuff/fix-permissions.c fix-permissions.c
RUN curl -L -o ./lbrycrd-linux.zip $(curl -s https://api.github.com/repos/lbryio/lbrycrd/releases | grep -F 'lbrycrd-linux.zip' | grep download | head -n 1 | cut -d'"' -f4) && \ RUN curl -L -o ./lbrycrd-linux.zip $(curl -s https://api.github.com/repos/lbryio/lbrycrd/releases | grep -F 'lbrycrd-linux.zip' | grep download | head -n 1 | cut -d'"' -f4) && \
unzip ./lbrycrd-linux.zip && \ unzip ./lbrycrd-linux.zip && \
gcc fix-permissions.c -o fix-permissions && \ gcc fix-permissions.c -o fix-permissions && \

View file

@ -5,7 +5,7 @@ services:
## Lbrycrd ## ## Lbrycrd ##
############# #############
lbrycrd: lbrycrd:
image: lbry/lbry-docker:lbrycrd-linux-x86_64-production image: lbry/lbrycrd:linux-x86_64-production
restart: always restart: always
ports: ports:
- "11336:9246" - "11336:9246"

View file

@ -5,7 +5,7 @@ services:
## Lbrycrd ## ## Lbrycrd ##
############# #############
lbrycrd: lbrycrd:
image: lbry/lbry-docker:lbrycrd-linux-x86_64-production image: lbry/lbrycrd:linux-x86_64-production
restart: always restart: always
ports: ports:
- "11336:9246" - "11336:9246"

View file

@ -2,7 +2,7 @@
## Compiler container ## Compiler container
The [Dockerfile-compiler-linux](Dockerfile-compiler-linux) is for building lbrynet for any architecture supported The [Dockerfile-linux-multiarch-compiler](Dockerfile-linux-multiarch-compiler) is for building lbrynet for any architecture supported
by an Ubuntu 18.04 base image. by an Ubuntu 18.04 base image.
### Register qemu to run docker images built for platforms other than your host ### Register qemu to run docker images built for platforms other than your host
@ -14,17 +14,17 @@ docker run --rm --privileged multiarch/qemu-user-static:register
### Build for the default x86_64 platform: ### Build for the default x86_64 platform:
``` ```
docker build -t lbrynet -f Dockerfile-compiler-linux . docker build -t lbrynet -f Dockerfile-linux-multiarch-compiler .
``` ```
### Build for an ARM 32-bit platform: ### Build for an ARM 32-bit platform:
``` ```
docker build -t lbrynet-armhf -f Dockerfile-compiler-linux --build-arg BASE_IMAGE=multiarch/ubuntu-core:armhf-bionic . docker build -t lbrynet-armhf -f Dockerfile-linux-multiarch-compiler --build-arg BASE_IMAGE=multiarch/ubuntu-core:armhf-bionic .
``` ```
### Build for an ARM 64-bit platform: ### Build for an ARM 64-bit platform:
``` ```
docker build -t lbrynet-arm64 -f Dockerfile-compiler-linux --build-arg BASE_IMAGE=multiarch/ubuntu-core:arm64-bionic . docker build -t lbrynet-arm64 -f Dockerfile-linux-multiarch-compiler --build-arg BASE_IMAGE=multiarch/ubuntu-core:arm64-bionic .
``` ```

View file

@ -9,19 +9,18 @@ services:
## Lbrynet ## ## Lbrynet ##
############# #############
lbrynet: lbrynet:
build: . image: lbry/lbrynet:Linux-x86_64-production
restart: always restart: always
## TODO: Does this container ever need a static network address or is it better to be dynamic?
networks: networks:
# lbrynet: # lbrynet:
# ipv4_address: 10.6.1.4 # ipv4_address: 10.6.1.4
- lbry-network - lbry-network
labels:
- "traefik.expose=false"
## TODO: Verify the need for the following port binds on host networking.
ports: ports:
- 4444:4444 - 4444:4444
- 50001:50001 - 50001:50001
env_file:
- ../environment/lbrynet.env
## host volumes for persistent data such as wallet private keys. ## host volumes for persistent data such as wallet private keys.
# This container will not run without a persistent wallet as it may contain a LBC balance.
volumes: volumes:
- ./data:/lbrynet - ../persist/lbrynet/data:/lbrynet

View file

@ -1 +0,0 @@
# TODO: Prepare a container for deployment on armhf architecture.

View file

@ -1,11 +1,12 @@
#!/bin/bash #!/bin/bash
## TODO: Make a bit more aware of the run mode of this appliance in case there is ever a test mode enabled in the start.sh
mountpoint=/home/lbrynet mountpoint=/home/lbrynet
if ! grep -qs ".* $mountpoint " /proc/mounts; then if ! grep -qs ".* $mountpoint " /proc/mounts; then
echo "$mountpoint not mounted, refusing to run." echo "$mountpoint not mounted, refusing to run."
## TODO: We should have documentation that this error references directly with a URL as to why it won't run without a volume.
exit 1 exit 1
else else
`$@` `$@`
fi fi

0
persist/.gitkeep Normal file
View file