Invalid precompiled binaries for aarch64-linux-gnu

[hansihe]

Looking at the precompiled binaries in exgboost-nif-2.17-aarch64-linux-gnu-0.5.0.tar.gz, they appear to be for x86-64:

➜  exgboost-nif-2.17-aarch64-linux-gnu-0.5.0 file libexgboost.so
libexgboost.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=73b007f8874b86f0c888a33071ec9494e4d0ab5d, not stripped

➜  exgboost-nif-2.17-aarch64-linux-gnu-0.5.0 file lib/libxgboost.so
lib/libxgboost.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=29bbac9879aabb9cf5e18a700ddeb245612e95e3, not stripped

Just to confirm my file prints architectures correctly, I also tried the precompiled binaries for ex_tokenizers. These are correctly labelled as aarch64 binaries.

➜  Downloads file libex_tokenizers-v0.3.2-nif-2.16-aarch64-unknown-linux-gnu.so
libex_tokenizers-v0.3.2-nif-2.16-aarch64-unknown-linux-gnu.so: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, BuildID[sha1]=99368f1b15dcd5d9e451775c7d5c91784556bd95, with debug_info, not stripped

This issue causes the NIF to fail to load with a no such file or directory on a aarch64 linux machine when downloading precompiled binaries.

[acalejos]

Hey, thanks for raising this issue. I'll check the GitHub action, I might not actually have an aarch64 build target on there and it looks like the mix precompile library generates them anyways.

[acalejos]

So it seems that I either need to add a cross-compilation job or wait for the public release of the GitHub hosted ARM Linux runners (https://github.blog/changelog/2023-10-30-accelerate-your-ci-cd-with-arm-based-hosted-runners-in-github-actions/).

This release says that the runners should be publicly available by Jan 2024, so I plan to wait a bit longer to see if they release these. If that takes too long I'll take another look at adding a cross-compiled option.

I can revise the GH Actions to at least not upload an aarch linux build file since that is quite confusing and obviously erroneous.

EDIT:

Seeing as cc_precompiler seems to support cross-compilation in a straight-forward manner (https://hexdocs.pm/elixir_make/precompilation_guide.html#optional-customise-precompilation-targets) I might give this a go sometime sooner. It might not end up taking a whole lot

[amarchen]

Wow, is it possibly the same that I see in dockerbuilds which started to fail lately

#21 20.28 Downloading precompiled NIF to /root/.cache/exgboost-nif-2.17-aarch64-linux-gnu-0.5.0.tar.gz
#21 22.13 Compiling 14 files (.ex)
#21 22.21
#21 22.21 12:20:15.716 [warning] The on_load function for module Elixir.EXGBoost.NIF returned:
#21 22.21 {:error,
#21 22.21  {:load_failed,
#21 22.21   ~c"Failed to load NIF library: '/app/_build/prod/lib/exgboost/priv/libexgboost.so: cannot open shared object file: No such file or directory'"}}

Somehow locally in a docker build 2.17 is being fetched and I have this error while on CI also in docker build it's 2.16 and it works fine.
Now I just need to figure out why on earth docker builds locally and on CI are different and where exactly this 2.16 or 2.17 is coming from :)

[amarchen]

After some more investigation I figured that on CI docker was building x86 while on apple silicon is was trying arm. So indeed looks like this issue with the wrong binary for arm.