Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix detection of math library in numpy build #3520

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

Flamefire
Copy link
Contributor

@Flamefire Flamefire commented Nov 28, 2024

(created using eb --new-pr)

NumPy detects the required math by trying to link a minimal code containing a call to log(0.).
The first try is without any libraries, which works with gcc -fno-math-errno (our optimization default)
because the call gets removed due to not having any effect. So it concludes that -lm is not required.
This then fails to detect availability of functions such as acosh which do not get removed in the same
way and so less exact replacements are used instead which e.g. fail the tests on PPC.

$MATHLIB=m makes it try -lm first and is supported until the Meson backend is used in 1.26+.

The test code used is:

double log(double);

int main(){
  log(0.);
}

The issue can be reproduced by putting it in a file test.c and running:

$ gcc test.c
# OK
$ gcc -fno-math-errno test.c
/software/.../bin/ld: /tmp/ccGF6GFx.o: in function `main':
test.c:(.text+0x20): undefined reference to `log'
collect2: error: ld returned 1 exit status

On PPC I see test failures in e.g. SciPy-bundle-2023.02-gfbf-2022b.eb & SciPy-bundle-2023.07-gfbf-2023a.eb
FAILED core/tests/test_umath.py::TestComplexFunctions::test_loss_of_precision[complex256] - AssertionError: (120, 1.78095631306534887655740488957533e-10, 5.2863423150802063925011506713427e-21, 'arcsinh')

However the issue is not limited to PPC although the fallback functions seem to be "good enough" on x86.
A minimal example I made: https://godbolt.org/z/zGYcW45qE and one that it fails to detect casinh without -lm

Co-authored-by: Jan André Reuter <[email protected]>
@Flamefire
Copy link
Contributor Author

Test report by @Flamefire

Overview of tested easyconfigs (in order)

  • SUCCESS SciPy-bundle-2022.05-foss-2022a.eb
  • SUCCESS SciPy-bundle-2023.02-gfbf-2022b.eb
  • SUCCESS SciPy-bundle-2023.07-gfbf-2023a.eb
  • SUCCESS SciPy-bundle-2023.11-gfbf-2023.09.eb
  • SUCCESS SciPy-bundle-2023.11-gfbf-2023b.eb
  • SUCCESS Ninja-1.12.1-GCCcore-13.3.0.eb
  • SUCCESS Meson-1.4.0-GCCcore-13.3.0.eb
  • SUCCESS Cython-3.0.10-GCCcore-13.3.0.eb
  • SUCCESS hatchling-1.24.2-GCCcore-13.3.0.eb
  • SUCCESS expat-2.6.2-GCCcore-13.3.0.eb
  • SUCCESS Autoconf-2.72-GCCcore-13.3.0.eb
  • SUCCESS scikit-build-0.17.6-GCCcore-13.3.0.eb
  • SUCCESS Automake-1.16.5-GCCcore-13.3.0.eb
  • SUCCESS Eigen-3.4.0-GCCcore-13.3.0.eb
  • SUCCESS flit-3.9.0-GCCcore-13.3.0.eb
  • SUCCESS libtool-2.4.7-GCCcore-13.3.0.eb
  • SUCCESS Catch2-2.13.10-GCCcore-13.3.0.eb
  • SUCCESS libxml2-2.12.7-GCCcore-13.3.0.eb
  • SUCCESS Autotools-20231222-GCCcore-13.3.0.eb
  • SUCCESS setuptools-rust-1.9.0-GCCcore-13.3.0.eb
  • SUCCESS libiconv-1.17-GCCcore-13.3.0.eb
  • SUCCESS patchelf-0.18.0-GCCcore-13.3.0.eb
  • SUCCESS gettext-0.22.5-GCCcore-13.3.0.eb
  • SUCCESS Rust-1.78.0-GCCcore-13.3.0.eb
  • SUCCESS git-2.45.1-GCCcore-13.3.0.eb
  • SUCCESS maturin-1.6.0-GCCcore-13.3.0.eb
  • SUCCESS ICU-75.1-GCCcore-13.3.0.eb
  • SUCCESS gzip-1.13-GCCcore-13.3.0.eb
  • SUCCESS virtualenv-20.26.2-GCCcore-13.3.0.eb
  • SUCCESS lz4-1.9.4-GCCcore-13.3.0.eb
  • SUCCESS cffi-1.16.0-GCCcore-13.3.0.eb
  • SUCCESS zstd-1.5.6-GCCcore-13.3.0.eb
  • SUCCESS cryptography-42.0.8-GCCcore-13.3.0.eb
  • SUCCESS poetry-1.8.3-GCCcore-13.3.0.eb
  • FAIL (build issue) Boost-1.85.0-GCC-13.3.0.eb (partial log available at https://gist.github.com/Flamefire/eeb1e998dc88f7cc2ed001efcf476dd9)
  • SUCCESS Python-bundle-PyPI-2024.06-GCCcore-13.3.0.eb
  • SUCCESS meson-python-0.16.0-GCCcore-13.3.0.eb
  • FAIL (build issue) pybind11-2.12.0-GCC-13.3.0.eb (partial log available at https://gist.github.com/Flamefire/6f88b0d8d076c88a7ac52f70e48b0183)
  • SUCCESS hypothesis-6.103.1-GCCcore-13.3.0.eb
  • FAIL (build issue) SciPy-bundle-2024.05-gfbf-2024a.eb (partial log available at https://gist.github.com/Flamefire/6702fb4cd9de1a2644c53ffd22cd1f85)

Build succeeded for 37 out of 40 (6 easyconfigs in total)
ml24 - Linux AlmaLinux 8.7 (Stone Smilodon), POWER, 8335-GTX (power9le), 6 x NVIDIA Tesla V100-SXM2-32GB, 530.30.02, Python 3.8.13
See https://gist.github.com/Flamefire/d8a5f3d67e75cf17c7741a9df366cf97 for a full test report.

@Flamefire
Copy link
Contributor Author

Flamefire commented Nov 29, 2024

Test report by @Flamefire

Overview of tested easyconfigs (in order)

  • SUCCESS SciPy-bundle-2022.05-foss-2022a.eb
  • SUCCESS SciPy-bundle-2023.02-gfbf-2022b.eb
  • SUCCESS SciPy-bundle-2023.07-gfbf-2023a.eb
  • SUCCESS SciPy-bundle-2023.11-gfbf-2023.09.eb
  • SUCCESS SciPy-bundle-2023.11-gfbf-2023b.eb
  • SUCCESS make-4.4.1-GCCcore-13.3.0.eb
  • SUCCESS OpenBLAS-0.3.27-GCC-13.3.0.eb
  • SUCCESS FlexiBLAS-3.4.4-GCC-13.3.0.eb
  • SUCCESS gfbf-2024a.eb
  • SUCCESS SciPy-bundle-2024.05-gfbf-2024a.eb

Build succeeded for 9 out of 10 (6 easyconfigs in total)
n1166 - Linux RHEL 8.9 (Ootpa), x86_64, Intel(R) Xeon(R) Platinum 8470 (icelake), Python 3.8.17
See https://gist.github.com/Flamefire/5c77df272384501a0632a9d7d846fa7d for a full test report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants