Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support for launching an MAPDL instance in an SLURM HPC cluster #3497

Merged
merged 145 commits into from
Oct 29, 2024

Conversation

germa89
Copy link
Collaborator

@germa89 germa89 commented Oct 18, 2024

Description

In this PR, add support to launch MAPDL instances in an HPC cluster. At the moment only SLURM scheduler is supported.
We only run Ansys in the job, so the cluster does not need to support Python at all.
Because of how PyMAPDL-MAPDL interaction has been designed, you can use PyMAPDL methods to upload/download files from the MAPDL instance. You could also execute OS commands using mapdl.sys, but this NOT recommended.

Usage

We allow the user to specify the argument launch_on_hpc when using launch_mapdl:

from ansys.mapdl.core import launch_mapdl

mapdl = launch_mapdl(launch_on_hpc=True)

Under the hood, we "wrap" (literally) the command used to launch MAPDL on a sbatch command:

sbatch --wrap='%PYMAPDL_MAPDL_LAUNCH_CMD%'

where %PYMAPDL_MAPDL_LAUNCH_CMD% is something like /ansys_inc/v242/ansys/bin/ansys242 -j myjob -np 4 -grpc.

Additionally we provide the option for specify scheduler arguments using the argument scheduler_args:

scheduler_args = "-N 10" # it could also be a dict: {"-N": 10}
mapdl = launch_mapdl(launch_on_hpc=True, scheduler_args=scheduler_args)

In the above case, it is interesting to set the number of CPUs to the real value of allocated CPUs:

# the double minus is added for long commands (len(cmd)> 1).
scheduler_args = {"nodes":  10, "ntasks-per-node": 2} 

mapdl = launch_mapdl(launch_on_hpc=True, scheduler_args=scheduler_args, nproc=20)

The values of each argument are wrapped in single quotes (') to avoid parsing issues.

Issue linked

Related to #3467

Checklist

germa89 and others added 30 commits October 4, 2024 11:40
Squashed commit of the following:

commit c1d1a3e
Author: German <[email protected]>
Date:   Mon Oct 7 15:33:19 2024 +0200

    ci: retrigger CICD

commit b7b5c30
Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Date:   Mon Oct 7 13:31:55 2024 +0000

    ci: auto fixes from pre-commit.com hooks.

    for more information, see https://pre-commit.ci

commit 32a1c02
Author: Revathy Venugopal <[email protected]>
Date:   Mon Oct 7 15:31:24 2024 +0200

    fix: add suggestions

    Co-authored-by: German <[email protected]>

commit 575a219
Merge: f2afe13 be1be2e
Author: Revathyvenugopal162 <[email protected]>
Date:   Mon Oct 7 15:09:01 2024 +0200

    Merge branch 'fix/add-build-cheatsheet-as-env-varaible' of https://github.com/ansys/pymapdl into fix/add-build-cheatsheet-as-env-varaible

commit f2afe13
Author: Revathyvenugopal162 <[email protected]>
Date:   Mon Oct 7 15:08:58 2024 +0200

    fix: precommit

commit be1be2e
Author: pyansys-ci-bot <[email protected]>
Date:   Mon Oct 7 13:07:35 2024 +0000

    chore: adding changelog file 3468.fixed.md

commit f052a4d
Author: Revathyvenugopal162 <[email protected]>
Date:   Mon Oct 7 15:05:56 2024 +0200

    fix: add build cheatsheet as env variable within doc-build
@germa89 germa89 marked this pull request as ready for review October 25, 2024 17:28
germa89 and others added 2 commits October 25, 2024 20:49
Simplifying `launch_mapdl_on_cluster`.
* docs: expanding a bit the `PyMAPDL on HPC clusters` section

* docs: adding info about launching MAPDL in HPC.

* chore: adding changelog file 3513.documentation.md [dependabot-skip]

* fix: vale issues

* docs: changing the name to `scheduler_options`.
Add warning about adding nproc.

* fix: vale issues

* docs: apply suggestions from Kathy code review

Co-authored-by: Kathy Pippert <[email protected]>

* docs: adding CPUs.

---------

Co-authored-by: pyansys-ci-bot <[email protected]>
Co-authored-by: Kathy Pippert <[email protected]>
@wiz-inc-572fc38784
Copy link
Contributor

wiz-inc-572fc38784 bot commented Oct 25, 2024

Wiz Scan Summary

Scan Module Critical High Medium Low Info Total
IaC Misconfigurations 0 0 0 0 0 0
Vulnerabilities 0 0 0 0 0 0
Sensitive Data 0 0 0 0 1 1
Secrets 0 0 0 0 0 0
Total 0 0 0 0 1 1

View scan details in Wiz

To detect these findings earlier in the dev lifecycle, try using Wiz Code VS Code Extension.

@germa89 germa89 requested a review from koubaa October 28, 2024 16:56
@germa89
Copy link
Collaborator Author

germa89 commented Oct 29, 2024

Ignoring bandit failed job because of PyCQA/bandit#1041

See the code:

https://github.com/ansys/pymapdl/blob/3b799a256dd03a8f17e0d520da4857b14a38f9bc/src/ansys/mapdl/core/launcher.py#L2833-L2842

See my log: https://github.com/ansys/pymapdl/actions/runs/11570941634/job/32207760895?pr=3497#step:2:2452

Pinging @ansys/pyansys-core for visibility.

GitHub
Pythonic interface to MAPDL. Contribute to ansys/pymapdl development by creating an account on GitHub.
GitHub
Pythonic interface to MAPDL. Contribute to ansys/pymapdl development by creating an account on GitHub.

@germa89 germa89 merged commit 0640963 into main Oct 29, 2024
60 of 62 checks passed
@germa89 germa89 deleted the feat/adding-sbatch-support branch October 29, 2024 11:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI/CD Related with CICD, Github Actions, etc dependencies DO NOT MERGE Not ready to be merged yet documentation Documentation related (improving, adding, etc) maintenance General maintenance of the repo (libraries, cicd, etc) new feature Request or proposal for a new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants