LBRYCommunity/lbrycrd
1
0
Fork 0
Code Issues 29 Pull requests 1 Projects Releases 32 Packages Wiki Activity

Improve build-osx formatting

Browse source
This commit is contained in:
Giulio Lombardo 2019-06-14 13:50:12 +02:00
parent c5ffe8d515
commit dbd137a4ea
No known key found for this signature in database
GPG key ID: FE9875184D7D79F7
1 changed files with 128 additions and 129 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

223
doc/build-osx.md
View file

@ -1,33 +1,37 @@
macOS Build Instructions and Notes # macOS Build Instructions and Notes
====================================
The commands in this guide should be executed in a Terminal application.
The built-in one is located in `/Applications/Utilities/Terminal.app`.
Preparation The commands in this guide should be executed in a Terminal application.
----------- The built-in one is located in
```
/Applications/Utilities/Terminal.app
```
## Preparation
Install the macOS command line tools: Install the macOS command line tools:
`xcode-select --install` ```shell
xcode-select --install
```
When the popup appears, click `Install`. When the popup appears, click `Install`.
Then install [Homebrew](https://brew.sh). Then install [Homebrew](https://brew.sh).
Dependencies ## Dependencies
---------------------- ```shell
brew install automake berkeley-db4 libtool boost miniupnpc openssl pkg-config protobuf python qt libevent qrencode brew install automake berkeley-db4 libtool boost miniupnpc openssl pkg-config protobuf python qt libevent qrencode
```
See [dependencies.md](dependencies.md) for a complete overview. See [dependencies.md](dependencies.md) for a complete overview.
If you want to build the disk image with `make deploy` (.dmg / optional), you need RSVG: If you want to build the disk image with `make deploy` (.dmg / optional), you need RSVG:
```shell
brew install librsvg brew install librsvg
```
Berkeley DB ## Berkeley DB
-----------
It is recommended to use Berkeley DB 4.8. If you have to build it yourself, It is recommended to use Berkeley DB 4.8. If you have to build it yourself,
you can use [the installation script included in contrib/](/contrib/install_db4.sh) you can use [this](/contrib/install_db4.sh) script to install it
like so: like so:
```shell ```shell
@ -38,172 +42,167 @@ from the root of the repository.
**Note**: You only need Berkeley DB if the wallet is enabled (see [*Disable-wallet mode*](/doc/build-osx.md#disable-wallet-mode)). **Note**: You only need Berkeley DB if the wallet is enabled (see [*Disable-wallet mode*](/doc/build-osx.md#disable-wallet-mode)).
Build Bitcoin Core ## Build Bitcoin Core
------------------------
1. Clone the Bitcoin Core source code: 1. Clone the Bitcoin Core source code:
```shell
git clone https://github.com/bitcoin/bitcoin git clone https://github.com/bitcoin/bitcoin
cd bitcoin cd bitcoin
```
2. Build Bitcoin Core: 2. Build Bitcoin Core:
Configure and build the headless Bitcoin Core binaries as well as the GUI (if Qt is found). Configure and build the headless Bitcoin Core binaries as well as the GUI (if Qt is found).
You can disable the GUI build by passing `--without-gui` to configure. You can disable the GUI build by passing `--without-gui` to configure.
```shell
./autogen.sh ./autogen.sh
./configure ./configure
make make
```
3. It is recommended to build and run the unit tests: 3. It is recommended to build and run the unit tests:
```shell
make check make check
```
4. You can also create a .dmg that contains the .app bundle (optional): 4. You can also create a `.dmg` that contains the `.app` bundle (optional):
```shell
make deploy make deploy
```
Disable-wallet mode ## `disable-wallet` mode
-------------------- When the intention is to run only a P2P node without a wallet, Bitcoin Core may be
When the intention is to run only a P2P node without a wallet, Bitcoin Core may be compiled in compiled in `disable-wallet` mode with:
disable-wallet mode with: ```shell
./configure --disable-wallet ./configure --disable-wallet
```
In this case there is no dependency on Berkeley DB 4.8. In this case there is no dependency on Berkeley DB 4.8.
Mining is also possible in disable-wallet mode using the `getblocktemplate` RPC call. Mining is also possible in disable-wallet mode using the `getblocktemplate` RPC call.
Running ## Running
-------
Bitcoin Core is now available at `./src/bitcoind` Bitcoin Core is now available at `./src/bitcoind`
Before running, you may create an empty configuration file: Before running, you may create an empty configuration file:
```shell
mkdir -p "/Users/${USER}/Library/Application Support/Bitcoin" mkdir -p "/Users/${USER}/Library/Application Support/Bitcoin"
touch "/Users/${USER}/Library/Application Support/Bitcoin/bitcoin.conf" touch "/Users/${USER}/Library/Application Support/Bitcoin/bitcoin.conf"
chmod 600 "/Users/${USER}/Library/Application Support/Bitcoin/bitcoin.conf" chmod 600 "/Users/${USER}/Library/Application Support/Bitcoin/bitcoin.conf"
```
The first time you run bitcoind, it will start downloading the blockchain. This process could take many hours, or even days on slower than average systems. The first time you run bitcoind, it will start downloading the blockchain. This process could
take many hours, or even days on slower than average systems.
You can monitor the download process by looking at the debug.log file: You can monitor the download process by looking at the debug.log file:
```shell
tail -f $HOME/Library/Application\ Support/Bitcoin/debug.log tail -f $HOME/Library/Application\ Support/Bitcoin/debug.log
```
Other commands: ## Other commands:
------- ```shell
./src/bitcoind -daemon # Starts the bitcoin daemon. ./src/bitcoind -daemon # Starts the bitcoin daemon.
./src/bitcoin-cli --help # Outputs a list of command-line options. ./src/bitcoin-cli --help # Outputs a list of command-line options.
./src/bitcoin-cli help # Outputs a list of RPC commands when the daemon is running. ./src/bitcoin-cli help # Outputs a list of RPC commands when the daemon is running.
```
Notes ## Notes
----- * Tested on OS X 10.10 Yosemite through macOS 10.14 Mojave on 64-bit Intel
processors only.
* Building with downloaded Qt binaries is not officially supported. See the notes in [#7714](https://github.com/bitcoin/issues/7714)
* Tested on OS X 10.10 Yosemite through macOS 10.13 High Sierra on 64-bit Intel processors only. ## Deterministic macOS DMG Notes
Working macOS DMGs are created in Linux by combining a recent `clang`, the Apple
`binutils` (`ld`, `ar`, etc) and DMG authoring tools.
* Building with downloaded Qt binaries is not officially supported. See the notes in [#7714](https://github.com/bitcoin/bitcoin/issues/7714) Apple uses `clang` extensively for development and has upstreamed the necessary
functionality so that a vanilla clang can take advantage. It supports the use of `-F`,
`-target`, `-mmacosx-version-min`, and `--sysroot`, which are all necessary when
building for macOS.
Deterministic macOS DMG Notes Apple's version of `binutils` (called `cctools`) contains lots of functionality missing in the
----------------------------- FSF's `binutils`. In addition to extra linker options for frameworks and sysroots, several
other tools are needed as well such as `install_name_tool`, `lipo`, and `nmedit`. These
do not build under Linux, so they have been patched to do so. The work here was used as
a starting point: [mingwandroid/toolchain4](https://github.com/mingwandroid/toolchain4).
Working macOS DMGs are created in Linux by combining a recent clang, In order to build a working toolchain, the following source packages are needed from
the Apple binutils (ld, ar, etc) and DMG authoring tools. Apple: `cctools`, `dyld`, and `ld64`.
Apple uses clang extensively for development and has upstreamed the necessary These tools inject timestamps by default, which produce non-deterministic binaries. The
functionality so that a vanilla clang can take advantage. It supports the use `ZERO_AR_DATE` environment variable is used to disable that.
of -F, -target, -mmacosx-version-min, and --sysroot, which are all necessary
when building for macOS.
Apple's version of binutils (called cctools) contains lots of functionality This version of `cctools` has been patched to use the current version of `clang`'s headers
missing in the FSF's binutils. In addition to extra linker options for and its `libLTO.so` rather than those from `llvmgcc`, as it was originally done in `toolchain4`.
frameworks and sysroots, several other tools are needed as well such as
install_name_tool, lipo, and nmedit. These do not build under linux, so they
have been patched to do so. The work here was used as a starting point:
[mingwandroid/toolchain4](https://github.com/mingwandroid/toolchain4).
In order to build a working toolchain, the following source packages are needed To complicate things further, all builds must target an Apple SDK. These SDKs are free to
from Apple: cctools, dyld, and ld64. download, but not redistributable. To obtain it, register for an Apple Developer Account,
then download the [Xcode 7.3.1 dmg](https://developer.apple.com/devcenter/download.action?path=/Developer_Tools/Xcode_7.3.1/Xcode_7.3.1.dmg).
These tools inject timestamps by default, which produce non-deterministic This file is several gigabytes in size, but only a single directory inside is needed:
binaries. The ZERO_AR_DATE environment variable is used to disable that.
This version of cctools has been patched to use the current version of clang's
headers and its libLTO.so rather than those from llvmgcc, as it was
originally done in toolchain4.
To complicate things further, all builds must target an Apple SDK. These SDKs
are free to download, but not redistributable.
To obtain it, register for a developer account, then download the [Xcode 7.3.1 dmg](https://developer.apple.com/devcenter/download.action?path=/Developer_Tools/Xcode_7.3.1/Xcode_7.3.1.dmg).
This file is several gigabytes in size, but only a single directory inside is
needed:
``` ```
Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk
``` ```
Unfortunately, the usual linux tools (7zip, hpmount, loopback mount) are incapable of opening this file. Unfortunately, the usual Linux tools (7zip, hpmount, loopback mount) are incapable of
To create a tarball suitable for Gitian input, there are two options: opening this file. To create a tarball suitable for Gitian input, there are two options:
Using macOS, you can mount the dmg, and then create it with: Using macOS, you can mount the DMG, and then create it with:
``` ```shell
$ hdiutil attach Xcode_7.3.1.dmg hdiutil attach Xcode_7.3.1.dmg
$ tar -C /Volumes/Xcode/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ -czf MacOSX10.11.sdk.tar.gz MacOSX10.11.sdk tar -C /Volumes/Xcode/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ -czf MacOSX10.11.sdk.tar.gz MacOSX10.11.sdk
``` ```
Alternatively, you can use 7zip and SleuthKit to extract the files one by one. Alternatively, you can use 7zip and SleuthKit to extract the files one by one. The script
The script contrib/macdeploy/extract-osx-sdk.sh automates this. First ensure [`extract-osx-sdk.sh`](./../contrib/macdeploy/extract-osx-sdk.sh) automates this. First
the dmg file is in the current directory, and then run the script. You may wish ensure the DMG file is in the current directory, and then run the script. You may wish to
to delete the intermediate 5.hfs file and MacOSX10.11.sdk (the directory) when delete the `intermediate 5.hfs` file and `MacOSX10.11.sdk` (the directory) when you've
you've confirmed the extraction succeeded. confirmed the extraction succeeded.
```bash ```shell
apt-get install p7zip-full sleuthkit apt-get install p7zip-full sleuthkit
contrib/macdeploy/extract-osx-sdk.sh contrib/macdeploy/extract-osx-sdk.sh
rm -rf 5.hfs MacOSX10.11.sdk rm -rf 5.hfs MacOSX10.11.sdk
``` ```
The Gitian descriptors build 2 sets of files: Linux tools, then Apple binaries The Gitian descriptors build 2 sets of files: Linux tools, then Apple binaries which are
which are created using these tools. The build process has been designed to created using these tools. The build process has been designed to avoid including the
avoid including the SDK's files in Gitian's outputs. All interim tarballs are SDK's files in Gitian's outputs. All interim tarballs are fully deterministic and may be freely
fully deterministic and may be freely redistributed. redistributed.
genisoimage is used to create the initial DMG. It is not deterministic as-is, `genisoimage` is used to create the initial DMG. It is not deterministic as-is, so it has been
so it has been patched. A system genisoimage will work fine, but it will not patched. A system `genisoimage` will work fine, but it will not be deterministic because
be deterministic because the file-order will change between invocations. the file-order will change between invocations. The patch can be seen here: [theuni/osx-cross-depends](https://raw.githubusercontent.com/theuni/osx-cross-depends/master/patches/cdrtools/genisoimage.diff).
The patch can be seen here: [theuni/osx-cross-depends](https://raw.githubusercontent.com/theuni/osx-cross-depends/master/patches/cdrtools/genisoimage.diff). No effort was made to fix this cleanly, so it likely leaks memory badly. But it's only used for
No effort was made to fix this cleanly, so it likely leaks memory badly. But a single invocation, so that's no real concern.
it's only used for a single invocation, so that's no real concern.
genisoimage cannot compress DMGs, so afterwards, the 'dmg' tool from the `genisoimage` cannot compress DMGs, so afterwards, the DMG tool from the
libdmg-hfsplus project is used to compress it. There are several bugs in this `libdmg-hfsplus` project is used to compress it. There are several bugs in this tool and its
tool and its maintainer has seemingly abandoned the project. It has been forked maintainer has seemingly abandoned the project. It has been forked and is available
and is available (with fixes) here: [theuni/libdmg-hfsplus](https://github.com/theuni/libdmg-hfsplus). (with fixes) here: [theuni/libdmg-hfsplus](https://github.com/theuni/libdmg-hfsplus).
The 'dmg' tool has the ability to create DMGs from scratch as well, but this The DMG tool has the ability to create DMGs from scratch as well, but this functionality is
functionality is broken. Only the compression feature is currently used. broken. Only the compression feature is currently used. Ideally, the creation could be fixed
Ideally, the creation could be fixed and genisoimage would no longer be necessary. and `genisoimage` would no longer be necessary.
Background images and other features can be added to DMG files by inserting a Background images and other features can be added to DMG files by inserting a
.DS_Store before creation. This is generated by the script `.DS_Store` before creation. This is generated by the script
contrib/macdeploy/custom_dsstore.py. `contrib/macdeploy/custom_dsstore.py`.
As of OS X 10.9 Mavericks, using an Apple-blessed key to sign binaries is a As of OS X 10.9 Mavericks, using an Apple-blessed key to sign binaries is a requirement in
requirement in order to satisfy the new Gatekeeper requirements. Because this order to satisfy the new Gatekeeper requirements. Because this private key cannot be
private key cannot be shared, we'll have to be a bit creative in order for the shared, we'll have to be a bit creative in order for the build process to remain somewhat
build process to remain somewhat deterministic. Here's how it works: deterministic. Here's how it works:
- Builders use Gitian to create an unsigned release. This outputs an unsigned - Builders use Gitian to create an unsigned release. This outputs an unsigned DMG which
dmg which users may choose to bless and run. It also outputs an unsigned app users may choose to bless and run. It also outputs an unsigned app structure in the form
structure in the form of a tarball, which also contains all of the tools of a tarball, which also contains all of the tools that have been previously (deterministically)
that have been previously (deterministically) built in order to create a built in order to create a final DMG.
final dmg. - The Apple keyholder uses this unsigned app to create a detached signature, using the
- The Apple keyholder uses this unsigned app to create a detached signature, script that is also included there. Detached signatures are available from this [repository](https://github.com/bitcoin-core/bitcoin-detached-sigs).
using the script that is also included there. Detached signatures are available from this [repository](https://github.com/bitcoin-core/bitcoin-detached-sigs). - Builders feed the unsigned app + detached signature back into Gitian. It uses the
- Builders feed the unsigned app + detached signature back into Gitian. It pre-built tools to recombine the pieces into a deterministic DMG.
uses the pre-built tools to recombine the pieces into a deterministic dmg.