release: add new release script and documentation

In this commit, we add the new release script that will be used to build all release binaries going forward. We also remove the existing Conformal key as it's no longer in use, updating the README to reflect the new release build/verification process.
2019-10-10 16:17:57 -07:00 · 2019-10-10 16:17:57 -07:00 · 2083acdd24
commit 2083acdd24
parent e47518c978
4 changed files with 186 additions and 93 deletions
--- a/README.md
+++ b/README.md
@ -35,7 +35,7 @@ which are both under active development.

 ## Requirements

-[Go](http://golang.org) 1.11 or newer.
+[Go](http://golang.org) 1.12 or newer.

 ## Installation

@ -118,25 +118,12 @@ is used for this project.

 The documentation is a work-in-progress.  It is located in the [docs](https://github.com/btcsuite/btcd/tree/master/docs) folder.

-## GPG Verification Key
+## Release Verification

-All official release tags are signed by Conformal so users can ensure the code
-has not been tampered with and is coming from the btcsuite developers.  To
-verify the signature perform the following:
-
- Download the Conformal public key:
-  https://raw.githubusercontent.com/btcsuite/btcd/master/release/GIT-GPG-KEY-conformal.txt
-
- Import the public key into your GPG keyring:
-  ```bash
-  gpg --import GIT-GPG-KEY-conformal.txt
-  ```
-
- Verify the release tag with the following command where `TAG_NAME` is a
-  placeholder for the specific tag:
-  ```bash
-  git tag -v TAG_NAME
-  ```
+Please see our [documentation on the current build/verification
+process](https://github.com/btcsuite/btcd/tree/master/release) for all our
+releases for information on how to verify the integrity of published releases
+using our reproducible build system.

 ## License

--- a/release/GIT-GPG-KEY-conformal.txt
+++ b/release/GIT-GPG-KEY-conformal.txt
@ -1,74 +0,0 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
-Version: GnuPG v1
-
-mQINBFGJW70BEAC6cmsUVSeaaOTUfiWl8ngiI65ryOYZUCBwXGftTh4KvIuYguU1
-y9aws3ppH80D9+EzlpZbx7lNqGG85LiBd27yqgDbayYStz0e/R3vsYOMSt63rfxe
-GsOc3yFxmPcYjjyJQDIbhGf0T04cf98+Mtdr6zz88MP0eHABQGmwcc7C/en3MC/B
-Wwu/uKZOmv7I6fgGKOJFjXPqHNggnah+XWEBZEg1eCkMktmZrswGpJP4wjOCxatj
-Dg30jt0gvfmFdB9bjJdBoikRKwUUPFMYhMjo2vheSwbobwjeOjzgLx9y1Xl1x7J4
-ZgBfm+MoShNyEN66eSTX8TLmcsD62RzA+UDpGF7TvyOrTZpnhSYM2VbwOpl0yxdv
-WN3cot4qnnYVRN1FCz5pVdwpBWXhflGKCVLYyRnMCFLFiehyL8P5iMIuipu3SGlm
-ECCLWNsoPISjG09eWj7XlD2T/xEMRcQ8G2sMTKjnafzmuGcbABKDemREFknbYCYB
-nAhuCJKd4hlet8Qt+bR2GJWRlW2xKRO9eAGAwd1027W8EKr2tOg6bW6EADRkcbjs
-NIXfxIYlDsP17YV0gtuYCGalaDixyHGE/i9b1j457Tkhw82sJrsIzv0GxyPI56iH
-r2M72C+jEPS+jSqZvqyiwgFz/2xLvz80qf++10lXJJ5M0zA9oxfHRcxLvQARAQAB
-tC9Db25mb3JtYWwgU3lzdGVtcywgTExDLiA8cmVsZWFzZUBjb25mb3JtYWwuY29t
-PokCOAQTAQIAIgUCUYlbvQIbAwYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQ
-3+Onzh1FhSS++Q/9F+L6VfES9zd2IM6RUhQ/c0VtnB5kIpZNr067BR03gBrUNJAJ
-fqMXzTD2yJ2jWNdJOkSGi/vlG0Q/arB58GC+sOEU9dba2aFej0Io7mQiHsElu+Xp
-PvswckQBnkju6eowH7KxrbI9xGyqpa4b7IqhUJdDx8EUA2PHEJsVBHdSRwweZMpj
-BNnqOnD+zr1agBMYPV/wKCfy+ohqjkmLyIba+tR1cG8hFEyKhXQzcFb4chBPmMcM
-ryXQlR9d6dVzP9ZTZnb4kqckUm3LGWgAX/NW8UWxTYoqfw+YTxoZHm6NvCQCNYxz
-bqGry8Fo9guPuO6vLxNZ7J4wTJWKaBewQIIaSMgKpoL5tF2apKkSB8CaSGhZiUyc
-Q9+Vc68TcJZQMEYF856OiK/PqrhFyfdXgFkLnScMlWjSixcBDkqHwCGj4fr3yX6H
-mEiHDK/DUcOlb5HUXwvEnJXCu5j0UIuMxUD6mb3MnfGUT/ttBHPM9fZ1OOmZHiGu
-v58vIYuLcrTZ/6n7ObEWMzznJ4PxdyA3DGlmIfBnoXprrr8HDDCkI/SDAojIr9zu
-+mJ9fHppaW8+iV8rHq5Rn1TyZgJ+pS/GOAd7gwJLAl3gifjTpbq1jyQATYqoCpp2
-I3Z51+eDhld+l9QVR2BchhTMJPkK1LpkIK3Pr0MEQAVf7AQwxrQavd9mB+WJAhwE
-EAECAAYFAlHKEO0ACgkQs1UFMZPOvcmtZxAAoZmrxxkpugd72TZZfFbNRYGFfvMs
-P3s4DJYOTtL12asXqfxwd5lAxYWJhHkqkc7Y8n7+byC+gBQNVEBWnEf1yv35JjYs
-NCrDquRg2lK8S5meQGxLTEH0BoXmzQqe4PDLA1kUkY9otcHWbBq9oxORwZKcHBEw
-mW7zfAMOP9dxoDJ809KJ4WGwy9djDJsywFNYOlTQkMd2JbzfHhj2bZ2w9Zx43mE7
-rj/QLcJGgpi57FiXJNZMVXuGWVO5GPWJMPXzH2d8cH9IUmTpGBTrhvMq5sotEepd
-mXNmMUa2DNnhiDiF9GH2d66NX1cyHLG58/zI52n1Zxs7QJk9W2uYZV0rAuClxyRe
-njxVK7G+ybvpA8IWdaDaif1XzUJAj8SniaydYwO4vzA9ht2efn6vzfF2F1n567/m
-a2AI3k22dkT91fQo0L90r1UFaeu9adZ6rruMNVxMwG99TrRxbSdJqxttL5XHeIYj
-f4W/5EjABB/gthb1GBDXaX+1jxijSuVOGjfVYPFoX+Z5NYPMZnk0N4k6VfR6Kg1U
-qRRwxZlu1+2hTFK6A1HUOyFbubV9fYtaPFUdjgIXKa3lXtgR47zVRfymDlFKzLp2
-8MhnF3YfvlTdE5MkDr6e4TGCbfmdXSSs+IMxiH+GEBu+wr1Ip7+MaL8ce/98KcZQ
-NtWScrZIA8qhTjyJAhwEEAECAAYFAlHOV8QACgkQA1L7cyui5DNitxAAjctA9/x2
-zGJ3spSyw9GEao5D/zU55eDt8mmrLLrLs9zcqf61apNH75V7DJKxvc+V4yiwI09O
-uzHrsau33VgekHj9uVd+iuHgxheudwPanKDRBo/DXO0q0aN3KFLXptwz0ixiZ6RL
-Gf4cfhtQtPhtdjK8aW6mAG+PLvOxNTSWz5pYx1k29xehaLU0zPF6YaAH5HtNgFFs
-VQK5h7WO2hhs1QevuitK5RBKEjUAhHtl8iVwR81RCvzl9z5pDydkz8pTtpXzIYgf
-ap6ZkiIr10DnXVjx2S5WlQq5mYGbyuUnqMwmetzQsVQipF1RX9zYKN18noFNigYy
-1NbVf3+h49uQ2dIenT9xILewAiimEUzwDmgS7mUcBLwWnmjz9ZFJTWKpKWe5xzoR
-XkZPaWD7J93RvQ4qsuCDHZv99H4ykfKosZ2P9CQn7HOL52DEMhwc9pZza9irz5Hv
-fXGWsm5bn8NbJUN73HATpGTn/QClTBa28VZozcACuWqQre662P1/zR8BsDoeUydr
-1rUaKyi6ynGhjMfcGxHw1GGThxo/bd05+EzouP8zX/+2sJn5pSeDP/Ovigfh8LZR
-QEZx79/7p9kiDEymcRv8uLcJsC/iSI58S0o/m1g26ZQOiFH/C47/USEZ6bKQNgvD
-MAimTmvfuMR7hkQ/dA/EW7AOzcBgCPTmOfa5Ag0EUYlbvQEQAKcWaFG7y6UjX93J
-b8jxmrruMzj14qPw88QGRAmtFJzbeICiYG1gmgRq0dyAdfLst2vFjpZryKXhxxr2
-pM5IxCuxC+qaBe4oMAv/C/8B3ZANaUR3V1C0xNHunN9VWxf1XV682HXPnHUClkmG
-+HjSW7PYsnCV7N1DrIDNSD5tp+Xai3cRzpvPA6QWL/amKAIqWgBlvfAft7yXPaKo
-X8Z8WgXuz2deu9JhwSg9w8SNXyf4ZfcXhvN+HcdA9SaGnirmjxBdp/73/05qc8MI
-5KOfWPA9e/hza2/HGsdnyt2rXrDgkTmkZM4bc845dWNK6cnwcrXD6ibH6f4eOxup
-5gvpUIDKJTaeQY1qLZPGbPatpdl8EbYqu+Hc65M8N13OPgoKMcv1R9NbfMursQKt
-6cS7liOG1Xjc12Chx8btpOhNZUkOnEnFNAxpaJbJSHL3O3KuCffsKrxnlQUQq8ZC
-Tjlh/O1JmJR1tvz5nJA+zr02RyaxBYCi5QTvvPyGDS2Gn4JgiJXsXAMXHaDat42h
-qdcjENwIw2Q55kgDrLIQDMKrUwdDz5p74gyoRKaSUnh+kpyGNZqbktIpIhJN9LpR
-10QvDou4hwcnqM3BxrPmgT1jjNGoLtZriqtQ0yGuNUeicbIpgvtF/a93gWzF+kF7
-IRjLEsCDJHf064VFPHZUZ4UjXdWpABEBAAGJAh8EGAECAAkFAlGJW70CGwwACgkQ
-3+Onzh1FhSROzw//VI/a5ACU4zgmJ+GFaQsq87HCmTOWD0Q8mf4GOzwBsH60klgB
-kFwoGjfJK7dZiQFwdTts9C6Uiu88TSs11Ald1Ut0SmzaOcIEYj9IF7Suy8CGkd1f
-SdpG0bqwAddWoncTfajwUrKcWlyJIUsoEv2/kow+IcMZ609pY2oxVLSv/5wUoISs
-i2aCU+FDlAYVdsU6jMeRnMLbxlZ6NzKBROkjI61hIdpgZFRpaHk4GXsnyKybfBur
-pXxrzsy+9AK+EddXuqSClwKwBE1YIeqOVldygafcwwaD0WSyV9HUHZUaNUU3jhlo
-KuKwzRocKkuHTKBAn9hjCvpaReIc0fL9gP9SSJPt45m2pnTom1baCqadgAOr8D2n
-H2zNUBTMhF6L/w1ubMHMXaApbc6Pt0eCSoOx3UALf2DRHKXQgVrSzw1fqnkYLWxH
-eUuBBHIJqVjVAG3j/0AcvaFztSamMFceUtYFTnpo76wqh6z/RoBR6wC0CdjEvAes
-IoQscRs7ya73cuckD7Jo3G7OrnA4/tNrG87GehwfdSOhfpXy4qH21ovho+I2y4FH
-k3XVKrFgnT/g3Q1sU7GR+V/gO14nR7gbpo3LSodN0FVUTsp7kc9FZJvoUzIufwp5
-Te0Hxo1+FIse88qDcwdj5VxIDF5rIWHqdbRmoLaT0FFfckCo99R6a0n0Q9w=
-=iic8
-----END PGP PUBLIC KEY BLOCK-----
--- a/release/README.md
+++ b/release/README.md
@ -0,0 +1,71 @@
+# `btcd`'s Reproducible Build System
+
+This package contains the build script that the `btcd` project uses in order to
+build binaries for each new release. As of `go1.13`, with some new build flags,
+binaries are now reproducible, allowing developers to build the binary on
+distinct machines, and end up with a byte-for-byte identical binary. However,
+this wasn't _fully_ solved in `go1.13`, as the build system still includes the
+directory the binary is built into the binary itself. As a result, our scripts
+utilize a work around needed until `go1.13.2`.  
+
+## Building a New Release
+
+### macOS/Linux/Windows (WSL)
+
+No prior set up is needed on Linux or macOS is required in order to build the
+release binaries. However, on Windows, the only way to build the release
+binaries at the moment is by using the Windows Subsystem Linux. One can build
+the release binaries following these steps:
+
+1. `git clone https://github.com/btcsuite/btcd.git
+2. `cd btcd`
+3. `./build/release/release.sh <TAG> # <TAG> is the name of the next
+   release/tag`
+
+This will then create a directory of the form `btcd-<TAG>` containing archives
+of the release binaries for each supported operating system and architecture,
+and a manifest file containing the hash of each archive.
+
+## Verifying a Release
+
+With `go1.13`, it's now possible for third parties to verify release binaries.
+Before this version of `go`, one had to trust the release manager(s) to build the
+proper binary. With this new system, third parties can now _independently_ run
+the release process, and verify that all the hashes of the release binaries
+match exactly that of the release binaries produced by said third parties.
+
+To verify a release, one must obtain the following tools (many of these come
+installed by default in most Unix systems): `gpg`/`gpg2`, `shashum`, and
+`tar`/`unzip`.
+
+Once done, verifiers can proceed with the following steps:
+
+1. Acquire the archive containing the release binaries for one's specific
+   operating system and architecture, and the manifest file along with its
+   signature.
+2. Verify the signature of the manifest file with `gpg --verify
+   manifest-<TAG>.txt.sig`. This will require obtaining the PGP keys which
+   signed the manifest file, which are included in the release notes.
+3. Recompute the `SHA256` hash of the archive with `shasum -a 256 <filename>`,
+   locate the corresponding one in the manifest file, and ensure they match
+   __exactly__.
+
+At this point, verifiers can use the release binaries acquired if they trust
+the integrity of the release manager(s). Otherwise, one can proceed with the
+guide to verify the release binaries were built properly by obtaining `shasum`
+and `go` (matching the same version used in the release):
+
+4. Extract the release binaries contained within the archive, compute their
+   hashes as done above, and note them down.
+5. Ensure `go` is installed, matching the same version as noted in the release
+   notes. 
+6. Obtain a copy of `btcd`'s source code with `git clone
+   https://github.com/btcsuite/btcd` and checkout the source code of the
+   release with `git checkout <TAG>`.
+7. Proceed to verify the tag with `git verify-tag <TAG>` and compile the
+   binaries from source for the intended operating system and architecture with
+   `BTCDBUILDSYS=OS-ARCH ./build/release/release.sh <TAG>`.
+8. Extract the archive found in the `btcd-<TAG>` directory created by the
+   release script and recompute the `SHA256` hash of the release binaries (btcd
+   and btcctl) with `shasum -a 256 <filename>`. These should match __exactly__
+   as the ones noted above.
--- a/release/release.sh
+++ b/release/release.sh
@ -0,0 +1,109 @@
+#!/bin/bash
+
+# Simple bash script to build basic btcd tools for all the platforms we support
+# with the golang cross-compiler.
+#
+# Copyright (c) 2016 Company 0, LLC.
+# Use of this source code is governed by the ISC
+# license.
+
+set -e
+
+# If no tag specified, use date + version otherwise use tag.
+if [[ $1x = x ]]; then
+    DATE=`date +%Y%m%d`
+    VERSION="01"
+    TAG=$DATE-$VERSION
+else
+    TAG=$1
+fi
+
+go mod vendor
+tar -cvzf vendor.tar.gz vendor
+
+PACKAGE=btcd
+MAINDIR=$PACKAGE-$TAG
+mkdir -p $MAINDIR
+
+cp vendor.tar.gz $MAINDIR/
+rm vendor.tar.gz
+rm -r vendor
+
+PACKAGESRC="$MAINDIR/$PACKAGE-source-$TAG.tar"
+git archive -o $PACKAGESRC HEAD
+gzip -f $PACKAGESRC > "$PACKAGESRC.gz"
+
+cd $MAINDIR
+
+# If BTCDBUILDSYS is set the default list is ignored. Useful to release
+# for a subset of systems/architectures.
+SYS=${BTCDBUILDSYS:-"
+        darwin-386
+        darwin-amd64
+        dragonfly-amd64
+        freebsd-386
+        freebsd-amd64
+        freebsd-arm
+        illumos-amd64
+        linux-386
+        linux-amd64
+        linux-armv6
+        linux-armv7
+        linux-arm64
+        linux-ppc64
+        linux-ppc64le
+        linux-mips
+        linux-mipsle
+        linux-mips64
+        linux-mips64le
+        linux-s390x
+        netbsd-386
+        netbsd-amd64
+        netbsd-arm
+        netbsd-arm64
+        openbsd-386
+        openbsd-amd64
+        openbsd-arm
+        openbsd-arm64
+        solaris-amd64
+        windows-386
+        windows-amd64
+        windows-arm
+"}
+
+# Use the first element of $GOPATH in the case where GOPATH is a list
+# (something that is totally allowed).
+PKG="github.com/btcsuite/btcd"
+COMMIT=$(git describe --abbrev=40 --dirty)
+
+for i in $SYS; do
+    OS=$(echo $i | cut -f1 -d-)
+    ARCH=$(echo $i | cut -f2 -d-)
+    ARM=
+
+    if [[ $ARCH = "armv6" ]]; then
+      ARCH=arm
+      ARM=6
+    elif [[ $ARCH = "armv7" ]]; then
+      ARCH=arm
+      ARM=7
+    fi
+
+    mkdir $PACKAGE-$i-$TAG
+    cd $PACKAGE-$i-$TAG
+
+    echo "Building:" $OS $ARCH $ARM
+    env CGO_ENABLED=0 GOOS=$OS GOARCH=$ARCH GOARM=$ARM go build -v -trimpath -ldflags="-s -w -buildid=" github.com/btcsuite/btcd
+    env CGO_ENABLED=0 GOOS=$OS GOARCH=$ARCH GOARM=$ARM go build -v -trimpath -ldflags="-s -w -buildid=" github.com/btcsuite/btcd/cmd/btcctl
+    cd ..
+
+    if [[ $OS = "windows" ]]; then
+	zip -r $PACKAGE-$i-$TAG.zip $PACKAGE-$i-$TAG
+    else
+	tar -cvzf $PACKAGE-$i-$TAG.tar.gz $PACKAGE-$i-$TAG
+    fi
+
+    rm -r $PACKAGE-$i-$TAG
+done
+
+shasum -a 256 * > manifest-$TAG.txt