Merge #15693: travis: Switch to ubuntu keyserver to avoid timeouts

fa2056af1c travis: Properly cache and error on timeout (MarcoFalke)
fa36a333ee travis: Switch to ubuntu keyserver to avoid timeouts (MarcoFalke)

Pull request description:

  The other keyserver is consistently timing out on travis: https://travis-ci.org/bitcoin/bitcoin/jobs/512689710#L405

  Attempt to fix it by using a different server.

  Also:
  * fixes #15372
  * fixes #15738

ACKs for commit fa2056:
  ryanofsky:
    utACK fa2056af1c. All good changes (changing keyserver, getting rid of keyserver while loop, clarifying travis error, moving travis documentation to code comment).

Tree-SHA512: ac8436616ecfee0ed579114e19f03c53ceb688fbcd95a60cffe8f15b4e569772a6ba673f353bbd789e79fe27fc5626c77fab4086768844dd51e0c6c108b52fb2

.travis.yml

@@ -1,3 +1,30 @@
+# The test build matrix (stage: test) is constructed to test a wide range of
+# configurations, rather than a single pass/fail. This helps to catch build
+# failures and logic errors that present on platforms other than the ones the
+# author has tested.
+#
+# Some builders use the dependency-generator in `./depends`, rather than using
+# apt-get to install build dependencies. This guarantees that the tester is
+# using the same versions as Gitian, so the build results are nearly identical
+# to what would be found in a final release.
+#
+# In order to avoid rebuilding all dependencies for each build, the binaries
+# are cached and re-used when possible. Changes in the dependency-generator
+# will trigger cache-invalidation and rebuilds as necessary.
+#
+# These caches can be manually removed if necessary. This is one of the very
+# few manual operations that is possible with Travis, and it can be done by a
+# Bitcoin Core GitHub member via the Travis web interface [0].
+#
+# Travis CI uploads the cache after the script phase of the build [1].
+# However, the build is terminated without saving the chache if it takes over
+# 50 minutes [2]. Thus, if we spent too much time in early build stages, fail
+# with an error and save the cache.
+#
+# [0] https://travis-ci.org/bitcoin/bitcoin/caches
+# [1] https://docs.travis-ci.com/user/caching/#build-phases
+# [2] https://docs.travis-ci.com/user/customizing-the-build#build-timeouts
+
 dist: xenial
 os: linux
 language: minimal
@@ -26,6 +53,7 @@ env:
     - SDK_URL=https://bitcoincore.org/depends-sources/sdks
     - WINEDEBUG=fixme-all
     - DOCKER_PACKAGES="build-essential libtool autotools-dev automake pkg-config bsdmainutils curl git ca-certificates ccache"
+    - CACHE_ERR_MSG="Error! Initial build successful, but not enough time remains to run later build stages and tests. Please manually re-run this job by using the travis restart button or asking a bitcoin maintainer to restart. The next run should not time out because the build cache has been saved."
 before_install:
   - set -o errexit; source .travis/test_03_before_install.sh
 install:
@@ -33,8 +61,11 @@ install:
 before_script:
   - set -o errexit; source .travis/test_05_before_script.sh
 script:
-  - if [ $SECONDS -gt 1200 ]; then set +o errexit; echo "Travis early exit to cache current state"; false; else set -o errexit; source .travis/test_06_script_a.sh; fi
-  - if [ $SECONDS -gt 1800 ]; then set +o errexit; echo "Travis early exit to cache current state"; false; else set -o errexit; source .travis/test_06_script_b.sh; fi
+  - export CONTINUE=1
+  - if [ $SECONDS -gt 1200 ]; then export CONTINUE=0; fi  # Likely the depends build took very long
+  - if [ $CONTINUE = "1" ]; then set -o errexit; source .travis/test_06_script_a.sh; else set +o errexit; echo "$CACHE_ERR_MSG"; false; fi
+  - if [ $SECONDS -gt 1800 ]; then export CONTINUE=0; fi  # Likely the build took very long
+  - if [ $CONTINUE = "1" ]; then set -o errexit; source .travis/test_06_script_b.sh; else set +o errexit; echo "$CACHE_ERR_MSG"; false; fi
 after_script:
   - echo $TRAVIS_COMMIT_RANGE
   - echo $TRAVIS_COMMIT_LOG

.travis/lint_06_script.sh

@@ -20,6 +20,6 @@ test/lint/lint-all.sh
 
 if [ "$TRAVIS_REPO_SLUG" = "bitcoin/bitcoin" -a "$TRAVIS_EVENT_TYPE" = "cron" ]; then
     git log --merges --before="2 days ago" -1 --format='%H' > ./contrib/verify-commits/trusted-sha512-root-commit
-    while read -r LINE; do travis_retry gpg --keyserver hkp://subset.pool.sks-keyservers.net --recv-keys $LINE; done < contrib/verify-commits/trusted-keys &&
+    travis_retry gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys $(<contrib/verify-commits/trusted-keys) &&
     ./contrib/verify-commits/verify-commits.py --clean-merge=2;
 fi

doc/README.md

@@ -57,7 +57,6 @@ The Bitcoin repo's [root README](/README.md) contains relevant information on th
 - [Source Code Documentation (External Link)](https://dev.visucore.com/bitcoin/doxygen/)
 - [Translation Process](translation_process.md)
 - [Translation Strings Policy](translation_strings_policy.md)
-- [Travis CI](travis-ci.md)
 - [JSON-RPC Interface](JSON-RPC-interface.md)
 - [Unauthenticated REST Interface](REST-interface.md)
 - [Shared Libraries](shared-libraries.md)

doc/travis-ci.md

@@ -1,42 +0,0 @@
-Travis CI
-=========
-
-Support for using travis-ci has been added in order to automate pull-testing.
-See [travis-ci.org](https://travis-ci.org/) for more info
-
-This procedure is different than the pull-tester that came before it in a few
-ways.
-
-There is nothing to administer. This is a major feature as it means
-that builds have no local state. Because there is no ability to login to the
-builders to install packages (tools, dependencies, etc), the entire build
-procedure must instead be controlled by a declarative script `.travis.yml`.
-This script declares each build configuration, creates virtual machines as
-necessary, builds, then discards the virtual machines.
-
-A build matrix is constructed to test a wide range of configurations, rather
-than a single pass/fail. This helps to catch build failures and logic errors
-that present on platforms other than the ones the author has tested. This
-matrix is defined in the build script and can be changed at any time.
-
-All builders use the dependency-generator in the [depends dir](/depends), rather than
-using apt-get to install build dependencies. This guarantees that the tester
-is using the same versions as Gitian, so the build results are nearly identical
-to what would be found in a final release. However, this also means that builds
-will fail if new dependencies are introduced without being added to the
-dependency generator.
-
-In order to avoid rebuilding all dependencies for each build, the binaries are
-cached and re-used when possible. Changes in the dependency-generator will
-trigger cache-invalidation and rebuilds as necessary.
-
-These caches can be manually removed if necessary. This is one of the very few
-manual operations that is possible with Travis, and it can be done by the
-Bitcoin Core committer via the Travis web interface.
-
-In some cases, secure strings may be needed for hiding sensitive info such as
-private keys or URLs. The travis client may be used to create these strings:
-http://docs.travis-ci.com/user/encryption-keys/
-
-For the details of the build descriptor, see the official docs:
-http://docs.travis-ci.com/user/build-configuration/