Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MAINT] Trying to re-enable testing for non-x86 archs #3149

Merged
merged 17 commits into from
Mar 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
version: ~> 1.0
language: generic
group: travis_latest

# Only build for develop
branches:
only:
- develop

matrix:
fast_finish: true
include:
- python: 3.8
name: "Power PC"
os: linux
arch: ppc64le
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can probably add s390x here for the sake of #1389, but is IBM Z really that prevalent? I guess the z15 was released in 2019, but I can't think of a single cluster that uses it? (definitely nothing in the top500 list)

if: type = cron
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll be honest, I'm not 100% sure this will work. I have enabled the cron job on develop to be daily for now (eventually we can change it to weekly). So maybe it'll get triggered tomorrow, but it's not clear to me that the cron job will run if this doesn't get merged first.

before_install:
- wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-ppc64le.sh -O miniconda.sh
- mkdir $HOME/.conda
- bash miniconda.sh -b -p $HOME/miniconda
- $HOME/miniconda/bin/conda init bash
# for some reason the image does not like sourcing .bashrc
- . "/home/travis/miniconda/etc/profile.d/conda.sh"
- conda activate base
- conda update --yes conda
- conda install --yes pip pytest==6.1.2 mmtf-python biopython networkx cython matplotlib-base "scipy>=1.6" griddataformats hypothesis gsd codecov -c conda-forge -c biobuilds
- pip install pytest-xdist
- conda info
install:
- (cd package/ && python setup.py develop) && (cd testsuite/ && python setup.py install)
script:
- pytest testsuite/MDAnalysisTests --disable-pytest-warnings -n 2
after_sucess:
- echo "Override this stage for now"

- os: linux
language: python
arch: arm64-graviton2
python: 3.8
name: "ARM64"
dist: focal
virt: vm
group: edge
if: type = cron
before_install:
- python -m pip install cython numpy scipy
- python -m pip install --no-build-isolation hypothesis matplotlib pytest pytest-cov pytest-xdist tqdm
install:
- cd package
- python setup.py install
- cd ../testsuite
- python setup.py install
- cd ..
script:
- cd testsuite
- python -m pytest ./MDAnalysisTests --disable-pytest-warnings -n 8 -rsx --cov=MDAnalysis
after_success:
- echo "Override this stage for ARM64"

4 changes: 4 additions & 0 deletions package/CHANGELOG
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@ Fixes
* Fix syntax warning over comparison of literals using is (Issue #3066)

Enhancements
* Adds preliminary support for the ppc64le platform with minimal
dependencies (Issue #3127, PR #3149)
* Caches can now undergo central validation at the Universe level, opening
the door to more useful caching. Already applied to fragment caching
(Issue #2376, PR #3135)
Expand Down Expand Up @@ -131,6 +133,8 @@ Enhancements
explicit in the topology (Issue #2468, PR #2775)

Changes
* Introduces encore specific C compiler arguments to allow for lowering of
optimisations on non-x86 platforms (Issue #1389, PR #3149)
* Continuous integration uses mamba rather than conda to install the
dependencies (PR #2983)
* removes deprecated `as_Universe` function from MDAnalysis.core.universe,
Expand Down
26 changes: 14 additions & 12 deletions package/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -261,15 +261,8 @@ def extensions(config):
use_cython = config.get('use_cython', default=not is_release)
use_openmp = config.get('use_openmp', default=True)

if platform.machine() == 'aarch64':
# reduce optimization level for ARM64 machines
# because of issues with test failures sourcing to:
# MDAnalysis/analysis/encore/clustering/src/ap.c
extra_compile_args = ['-std=c99', '-ffast-math', '-O1', '-funroll-loops',
'-fsigned-zeros']
else:
extra_compile_args = ['-std=c99', '-ffast-math', '-O3', '-funroll-loops',
'-fsigned-zeros'] # see #2722
extra_compile_args = ['-std=c99', '-ffast-math', '-O3', '-funroll-loops',
'-fsigned-zeros'] # see #2722
define_macros = []
if config.get('debug_cflags', default=False):
extra_compile_args.extend(['-Wall', '-pedantic'])
Expand All @@ -281,6 +274,15 @@ def extensions(config):
if arch:
extra_compile_args.append('-march={}'.format(arch))

# encore is sensitive to floating point accuracy, especially on non-x86
# to avoid reducing optimisations on everything, we make a set of compile
# args specific to encore see #2997 for an example of this.
encore_compile_args = [a for a in extra_compile_args if 'O3' not in a]
if platform.machine() == 'aarch64' or platform.machine() == 'ppc64le':
encore_compile_args.append('-O1')
else:
encore_compile_args.append('-O3')

cpp_extra_compile_args = [a for a in extra_compile_args if 'std' not in a]
cpp_extra_compile_args.append('-std=c++11')
cpp_extra_link_args=[]
Expand Down Expand Up @@ -401,21 +403,21 @@ def extensions(config):
sources=['MDAnalysis/analysis/encore/cutils' + source_suffix],
include_dirs=include_dirs,
define_macros=define_macros,
extra_compile_args=extra_compile_args)
extra_compile_args=encore_compile_args)
ap_clustering = MDAExtension('MDAnalysis.analysis.encore.clustering.affinityprop',
sources=['MDAnalysis/analysis/encore/clustering/affinityprop' + source_suffix,
'MDAnalysis/analysis/encore/clustering/src/ap.c'],
include_dirs=include_dirs+['MDAnalysis/analysis/encore/clustering/include'],
libraries=mathlib,
define_macros=define_macros,
extra_compile_args=extra_compile_args)
extra_compile_args=encore_compile_args)
spe_dimred = MDAExtension('MDAnalysis.analysis.encore.dimensionality_reduction.stochasticproxembed',
sources=['MDAnalysis/analysis/encore/dimensionality_reduction/stochasticproxembed' + source_suffix,
'MDAnalysis/analysis/encore/dimensionality_reduction/src/spe.c'],
include_dirs=include_dirs+['MDAnalysis/analysis/encore/dimensionality_reduction/include'],
libraries=mathlib,
define_macros=define_macros,
extra_compile_args=extra_compile_args)
extra_compile_args=encore_compile_args)
nsgrid = MDAExtension('MDAnalysis.lib.nsgrid',
['MDAnalysis/lib/nsgrid' + cpp_source_suffix],
include_dirs=include_dirs,
Expand Down