Skip to content

Commit

Permalink
Add deps.sh and redo install guide
Browse files Browse the repository at this point in the history
Change-Id: I1d85acb3f4dc85dc88846fd331e07d15a712b35f
  • Loading branch information
riptl authored and ripatel-fd committed Apr 10, 2023
1 parent 382e723 commit b87a298
Show file tree
Hide file tree
Showing 4 changed files with 444 additions and 51 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ OPTIONS-*
# Nix
/result

# deps.sh
opt/

# Editor junk files
# Please just use a global gitignore conf, I'm tired of this
.idea
Expand Down
100 changes: 49 additions & 51 deletions INSTALL.md
Original file line number Diff line number Diff line change
@@ -1,75 +1,73 @@
## Building with Nix
# Building Radiance

The recommended and most stable way to build Radiance is with [Nix](https://nixos.org/).
This requires an existing Nix installation on your machine.
Radiance depends on many external dependencies including optional C libraries.
This makes the build procedure slightly more complicated than a pure Go project.

nix-build
./result/bin/radiance --help
For installing C deps, the following options are available, each with their own tradeoffs:
- [Setup Script](#building-with-depssh) **(recommended)**
- Stable, one-time ~3 minute installation
- Supported on Debian, Ubuntu 20.04+, Fedora, RHEL 8+, Alpine 3, and macOS
- [Nixpkgs](#building-with-nix)
- Somewhat stable, one-time 60 second installation
- Supported on any platform with a pre-existing installation of Nix (Linux, macOS)
- [Without Cgo](#without-cgo)
- Plain old pure Go build
- Missing a lot of functionality (no Solana Labs compatibility layer)
- Manual Installation: If you know what you're doing, feel free to install C deps manually.
- Good luck. Plan in 30 minutes of debugging time

However, note that the resulting binary will only run under the same Nix environment.
Radiance requires further requires Go 1.19. Using Go 1.18 or Go 1.20 will not work.

## Building with Go
Here's a trick to download another Go version in case you have the wrong one.
(See [Managing Go versions](https://golang.org/doc/manage-install))

Radiance commands can be built with standard Go 1.19 tooling.
go get golang.org/dl/go1.19.8
go1.19.8 download
alias go=go1.19.8

Once your Go toolchain and build dependencies are installed, you can build Radiance as usual:

go mod download
go run ./cmd/radiance

### Without Cgo
## Building with deps.sh

The full set of functionality requires C dependencies via Cgo.
To create a pure-Go build use the `lite` tag.
`deps.sh` fetches deps using Git, compiles them, and installs them into the `opt` dir of your Radiance checkout.

go build -tags=lite ./cmd/radiance
To complete the one-time installation of deps, run

./deps.sh

### With Cgo dependencies
You may get prompted to install basic system packages like `pkg-config` or `cmake` if they are missing.

Radiance tools that require direct access to the blockstore (such as `blockstore` and `car`)
require a working C toolchain and extra compiler arguments to link against rocksdb.
To add the installed deps to your shell env (and Go build tools), each time you open a new shell, run:

You'll need a working C compiler toolchain as well as prerequisites listed
by [grocksdb](https://github.com/linxGnu/grocksdb#prerequisite) and
[RocksDB itself](https://github.com/linxGnu/grocksdb#prerequisite).
source activate-opt

**RHEL/Centos/Fedora**
To clean up, simply run `./deps.sh nuke` or `rm -rf opt`

dnf -y install "@Development Tools" cmake zlib zlib-devel bzip2 bzip2-devel lz4-devel libzstd-devel
### How deps.sh works

**Debian**
Managing C deps is only as complicated as you want it to be.

# With package manager RocksDB
apt install -y librocksdb-dev
`deps.sh` is a ~500 line long handwritten shell script. It just calls `git clone`, `make`, and `make install` or their
respective equivalents for each dependency. And yet it is more reliable than any tool that tries to be smart.

# With RocksDB from source
apt install -y build-essential cmake zlib1g-dev libbz2-dev liblz4-dev libzstd-dev
It will not pollute your system -- The C deps are not installed globally.

**Building RocksDB**
## Building with Nix

To build RocksDB from source, run the following commands:
Radiance provides a [Nix](https://nixos.org/) package.

git clone https://github.com/facebook/rocksdb --branch v7.10.2 --depth 1
cd rocksdb
mkdir -p build && cd build
cmake .. \
-DCMAKE_BUILD_TYPE=Release \
-DROCKSDB_BUILD_SHARED=OFF \
-DWITH_GFLAGS=OFF \
-DWITH_BZ2=ON \
-DWITH_SNAPPY=OFF \
-DWITH_ZLIB=ON \
-DWITH_ZSTD=ON \
-DWITH_ALL_TESTS=OFF \
-DWITH_BENCHMARK_TOOLS=OFF \
-DWITH_CORE_TOOLS=OFF \
-DWITH_RUNTIME_DEBUG=OFF \
-DWITH_TESTS=OFF \
-DWITH_TOOLS=OFF \
-DWITH_TRACE_TOOLS=OFF
make -j
cd ../..
nix-build
./result/bin/radiance --help

Finally, rebuild RocksDB with the appropriate Cgo flags.
Note that the resulting binary is not freestanding.
It is linked against libraries provided by Nix and will not run on non-Nix hosts.

export CGO_CFLAGS="-I$(pwd)/rocksdb/include"
export CGO_LDFLAGS="-L$(pwd)/rocksdb/build"
### Without Cgo

go run ./cmd/radiance
The full set of functionality requires C dependencies via Cgo.
To create a pure-Go build use the `lite` tag.

go build -tags=lite ./cmd/radiance
5 changes: 5 additions & 0 deletions activate-opt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/bash

# Set pkg-config path to source libraries from ./opt
export PKG_CONFIG_LIBDIR=""
export PKG_CONFIG_PATH="$(pwd)/opt/lib/pkgconfig:$(pwd)/opt/lib64/pkgconfig"
Loading

0 comments on commit b87a298

Please sign in to comment.