Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use proper p2pd binary on macOS #586

Merged
merged 5 commits into from
Aug 27, 2023
Merged

Use proper p2pd binary on macOS #586

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b395a8b
Download platform-specific binaries
borzunov Aug 23, 2023
d7cd30f
Update README.md
borzunov Aug 24, 2023
530e992
Update README.md
borzunov Aug 25, 2023
752cdda
Update README.md
borzunov Aug 25, 2023
57ee262
Merge branch 'master' into macos-binary
borzunov Aug 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,8 @@ of [Go toolchain](https://golang.org/doc/install) (1.15 or 1.16 are supported).

- __Linux__ is the default OS for which hivemind is developed and tested. We recommend Ubuntu 18.04+ (64-bit), but
other 64-bit distros should work as well. Legacy 32-bit is not recommended.
- __macOS 10.x__ can run hivemind using [Docker](https://docs.docker.com/desktop/mac/install/).
- __macOS__ is partially supported.
If you have issues, you can run hivemind using [Docker](https://docs.docker.com/desktop/mac/install/) instead.
We recommend using [our Docker image](https://hub.docker.com/r/learningathome/hivemind).
- __Windows 10+ (experimental)__ can run hivemind
using [WSL](https://docs.microsoft.com/ru-ru/windows/wsl/install-win10). You can configure WSL to use GPU by
Expand Down
46 changes: 24 additions & 22 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
import glob
import hashlib
import os
import platform
import re
import subprocess
import tarfile
Expand All @@ -13,14 +14,15 @@
from setuptools.command.build_py import build_py
from setuptools.command.develop import develop

P2PD_VERSION = "v0.3.16"
P2PD_VERSION = "v0.3.17"

P2PD_SOURCE_URL = f"https://github.com/learning-at-home/go-libp2p-daemon/archive/refs/tags/{P2PD_VERSION}.tar.gz"
P2PD_BINARY_URL = f"https://github.com/learning-at-home/go-libp2p-daemon/releases/download/{P2PD_VERSION}/"

# The value is sha256 of the binary from the release page
EXECUTABLES = {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made EXECUTABLES when we also wanted to ship p2p-keygen binary. We now moved away from it, since p2p-keygen functionality can be implemented in 3 lines of Python.

"p2pd": "057ec61edbe926cf049e9532d43ea9540da55db7b2d8c816d2bbdddce23f3cdf",
P2P_BINARY_HASH = {
"linux": "b0dd69e80f03a6fe5546f7079242b0228e93cd88d6e58442a227ed9521a95328",
"darwin": "f5cf7a86335e0264a65a6cf0fbd1033409e6f9bee65f9c4ee6c330b3cb53c3b5",
}

here = os.path.abspath(os.path.dirname(__file__))
Expand Down Expand Up @@ -71,31 +73,31 @@ def build_p2p_daemon():
with tarfile.open(dest, "r:gz") as tar:
tar.extractall(tempdir)

for executable in EXECUTABLES:
result = subprocess.run(
["go", "build", "-o", os.path.join(here, "hivemind", "hivemind_cli", executable)],
cwd=os.path.join(tempdir, f"go-libp2p-daemon-{P2PD_VERSION.lstrip('v')}", executable),
)
if result.returncode != 0:
raise RuntimeError(f"Failed to build {executable}: exited with status code: {result.returncode}")
result = subprocess.run(
["go", "build", "-o", os.path.join(here, "hivemind", "hivemind_cli", "p2pd")],
cwd=os.path.join(tempdir, f"go-libp2p-daemon-{P2PD_VERSION.lstrip('v')}", "p2pd"),
)
if result.returncode != 0:
raise RuntimeError(f"Failed to build p2pd: exited with status code: {result.returncode}")


def download_p2p_daemon():
for executable, expected_hash in EXECUTABLES.items():
binary_path = os.path.join(here, "hivemind", "hivemind_cli", executable)
binary_path = os.path.join(here, "hivemind", "hivemind_cli", "p2pd")
os_name = platform.system().lower()
expected_hash = P2P_BINARY_HASH[os_name]

if sha256(binary_path) != expected_hash:
binary_url = os.path.join(P2PD_BINARY_URL, executable)
print(f"Downloading {binary_url}")
if sha256(binary_path) != expected_hash:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to clarify: do we verify it this way to automatically download the correct binary for a non-Linux system? Would it be more clear if we simply checked that the binary exists?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sometimes devs replace the binary for testing purposes (e.g., to test an update in libp2p-daemon). As far as I understand, this code allows to replace the binary to the standard one when you reinstall hivemind (since the expected behavior in this case is to get a clean installation).

binary_url = os.path.join(P2PD_BINARY_URL, f"p2pd-{os_name}")
print(f"Downloading {binary_url}")

urllib.request.urlretrieve(binary_url, binary_path)
os.chmod(binary_path, 0o777)
urllib.request.urlretrieve(binary_url, binary_path)
os.chmod(binary_path, 0o777)

actual_hash = sha256(binary_path)
if actual_hash != expected_hash:
raise RuntimeError(
f"The sha256 checksum for {executable} does not match (expected: {expected_hash}, actual: {actual_hash})"
)
actual_hash = sha256(binary_path)
if actual_hash != expected_hash:
raise RuntimeError(
f"The sha256 checksum for p2pd does not match (expected: {expected_hash}, actual: {actual_hash})"
)


class BuildPy(build_py):
Expand Down
Loading