Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add yaml format check in pre-commit #13

Merged
merged 1 commit into from
Oct 19, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/PULL_REQUEST_TEMPLATE/new_configuration.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: New configuration
name: New configuration
about: Add a new computer/code configuration
title: 'New configuration for'
labels: new configuration
Expand All @@ -21,7 +21,7 @@ Please provide below the output of `verdi computer test <computername>` for each
### New codes added

Checks:
- The codes are maintained
- The codes are maintained
- [ ] by the admins of the HPC center
- [ ] by me
- [ ] I have verified that the paths to the executables are correct
Expand Down
11 changes: 11 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,18 @@ on:


jobs:
pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.8
uses: actions/setup-python@v2
with:
python-version: 3.8
- uses: pre-commit/[email protected]

deploy-gh-pages:
needs: pre-commit
runs-on: ubuntu-latest

steps:
Expand Down
15 changes: 15 additions & 0 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: continuous-integration

on: [push, pull_request]

jobs:

pre-commit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.8
uses: actions/setup-python@v2
with:
python-version: 3.8
- uses: pre-commit/[email protected]
8 changes: 4 additions & 4 deletions .make_ghpages/generate_json.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
folder_path = Path(__file__).parent.absolute()
library_path = folder_path.parent

# Extract all the data.
# Extract all the data.

YAML_SUFFIX = '.yaml'

Expand All @@ -25,17 +25,17 @@ def parse_yaml_files_in_folder(folder_path, exclude=None):
file_list = [ fname for fname in os.listdir(folder_path) if fname not in exclude ]
else:
file_list = os.listdir(folder_path)

for fname in file_list:
with open(folder_path/fname) as yaml_file:
if fname.endswith(YAML_SUFFIX):
fname = fname[:-len(YAML_SUFFIX)]
else:
raise ValueError(f"The file {fname} has unsupported extension. Please use '{YAML_SUFFIX}'")

result[fname] = yaml.load(yaml_file, Loader=yaml.FullLoader)

return result
return result


# Loop over the available domains and extract the computes available under them.
Expand Down
18 changes: 18 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.0.1 # Use the ref you want to point at
hooks:
- id: trailing-whitespace
- id: check-yaml

- repo: https://github.com/adrienverge/yamllint.git
rev: v1.17.0
hooks:
- id: yamllint
args: [-c=.yamllint, -s]
# only check yaml files for codes and computers setting
exclude: >
(?x)^(
.github/.* |
ltalirz marked this conversation as resolved.
Show resolved Hide resolved
.pre-commit-config.yaml |
)$
5 changes: 5 additions & 0 deletions .yamllint
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
extends: default

rules:
line-length: disable
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This repository collects configurations of simulation codes on public compute re

In the following we'll take the example of [Piz Daint](https://www.cscs.ch/computers/piz-daint/), a HPC system at the Swiss National Supercomputing Centre.

### `verdi computer setup`
### `verdi computer setup`

1. Navigate to the [`daint.cscs.ch`](./daint.cscs.ch) folder in the GitHub web interface
2. Select the partition you would like to run on, for example [`hybrid`](./daint.cscs.ch/hybrid)
Expand All @@ -23,7 +23,7 @@ You can overwrite any of the parameters provided in the yaml file by appending t

Note: Alternatively, you can first create a local clone of the `aiida-code-registry` and pass the the local file path of the yaml file to the `--config` option.

### `verdi computer configure`
### `verdi computer configure`

Some computers require specific configuration options (e.g. to jump over a login node) and provide a dedicated `computer-configure.yaml` file.

Expand All @@ -38,7 +38,7 @@ At this point, you should be able to successfully run:
verdi computer test daint-hybrid
```

### `verdi code setup`
### `verdi code setup`

The [`daint.cscs.ch`](./daint.cscs.ch/) folder contains a [`codes`](./daint.cscs.ch/codes) subfolder with configuration files for individual codes.

Expand Down
1 change: 1 addition & 0 deletions daint.cscs.ch/codes/cp2k-6.1-hybrid.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "cp2k-6.1"
description: "CP2K with CUDA on daint, compiled by CSCS"
input_plugin: "cp2k"
Expand Down
1 change: 1 addition & 0 deletions daint.cscs.ch/codes/cp2k-6.1-multicore.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "cp2k-6.1"
description: "CP2K 6.1 compiled by CSCS"
input_plugin: "cp2k"
Expand Down
1 change: 1 addition & 0 deletions daint.cscs.ch/codes/cp2k-7.1-hybrid.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "cp2k-7.1"
description: "CP2K compiled for daint-gpu"
input_plugin: "cp2k"
Expand Down
1 change: 1 addition & 0 deletions daint.cscs.ch/codes/cp2k-7.1-multicore.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "cp2k-7.1"
description: "CP2K 7.1 compiled by CSCS"
input_plugin: "cp2k"
Expand Down
1 change: 1 addition & 0 deletions daint.cscs.ch/codes/cp2k-8.1-hybrid.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "cp2k-8.1"
description: "CP2K compiled for daint-gpu"
input_plugin: "cp2k"
Expand Down
1 change: 1 addition & 0 deletions daint.cscs.ch/codes/cp2k-8.1-multicore.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "cp2k-8.1"
description: "CP2K 8.1 compiled by CSCS"
input_plugin: "cp2k"
Expand Down
1 change: 1 addition & 0 deletions daint.cscs.ch/codes/pp-6.5-multicore.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "pp-6.5"
description: "PP compiled for daint-mc"
input_plugin: "quantumespresso.pp"
Expand Down
1 change: 1 addition & 0 deletions daint.cscs.ch/codes/projwfc-6.5-multicore.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "projwfc-6.5"
description: "PROJWFC compiled for daint-mc"
input_plugin: "quantumespresso.projwfc"
Expand Down
1 change: 1 addition & 0 deletions daint.cscs.ch/codes/pw-6.5-multicore.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "pw-6.5"
description: "PW compiled for daint-mc"
input_plugin: "quantumespresso.pw"
Expand Down
1 change: 1 addition & 0 deletions daint.cscs.ch/hybrid-s1005/computer-configure.yaml
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
---
safe_interval: 60
proxy_command: "ssh -q -Y [email protected] netcat ela.cscs.ch 22"
1 change: 1 addition & 0 deletions daint.cscs.ch/hybrid-s1005/computer-setup.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "daint-hybrid-s1005"
hostname: "daint.cscs.ch"
description: "Piz Daint supercomputer at CSCS Lugano, Switzerland, using the hybrid partition. Project s1005"
Expand Down
11 changes: 11 additions & 0 deletions daint.cscs.ch/hybrid-s1005/cp2k-7.1.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
label: "cp2k-7.1"
description: "CP2K compiled for daint-gpu"
input_plugin: "cp2k"
on_computer: true
remote_abs_path: "/apps/daint/UES/jenkins/7.0.UP02/gpu/easybuild/software/CP2K/7.1-CrayGNU-20.08-cuda/bin/cp2k.psmp"
computer: "daint-hybrid-s1005"
prepend_text: |
module load daint-gpu
module load CP2K/7.1-CrayGNU-20.08-cuda
append_text: " "
11 changes: 11 additions & 0 deletions daint.cscs.ch/hybrid-s1005/cp2k-8.1.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
label: "cp2k-8.1"
description: "CP2K compiled for daint-gpu"
input_plugin: "cp2k"
on_computer: true
remote_abs_path: "/apps/daint/UES/jenkins/7.0.UP02/gpu/easybuild/software/CP2K/8.1-CrayGNU-20.08-cuda/bin/cp2k.psmp"
computer: "daint-hybrid-s1005"
prepend_text: |
module load daint-gpu
module load CP2K/8.1-CrayGNU-20.08-cuda
append_text: " "
11 changes: 11 additions & 0 deletions daint.cscs.ch/hybrid-s1005/pp-6.5.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
label: "pp-6.5"
description: "PP compiled for daint-mc"
input_plugin: "quantumespresso.pp"
on_computer: true
remote_abs_path: "/apps/daint/UES/jenkins/7.0.UP02/mc/easybuild/software/QuantumESPRESSO/6.5-CrayIntel-20.08/bin/pp.x"
computer: "daint-hybrid-s1005"
prepend_text: |
module load daint-mc
module load QuantumESPRESSO/6.5-CrayIntel-20.08
append_text: " "
11 changes: 11 additions & 0 deletions daint.cscs.ch/hybrid-s1005/projwfc-6.5.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
label: "projwfc-6.5"
description: "PROJWFC compiled for daint-mc"
input_plugin: "quantumespresso.projwfc"
on_computer: true
remote_abs_path: "/apps/daint/UES/jenkins/7.0.UP02/mc/easybuild/software/QuantumESPRESSO/6.5-CrayIntel-20.08/bin/projwfc.x"
computer: "daint-hybrid-s1005"
prepend_text: |
module load daint-mc
module load QuantumESPRESSO/6.5-CrayIntel-20.08
append_text: " "
11 changes: 11 additions & 0 deletions daint.cscs.ch/hybrid-s1005/pw-6.5.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
label: "pw-6.5"
description: "PW compiled for daint-mc"
input_plugin: "quantumespresso.pw"
on_computer: true
remote_abs_path: "/apps/daint/UES/jenkins/7.0.UP02/mc/easybuild/software/QuantumESPRESSO/6.5-CrayIntel-20.08/bin/pw.x"
computer: "daint-hybrid-s1005"
prepend_text: |
module load daint-mc
module load QuantumESPRESSO/6.5-CrayIntel-20.08
append_text: " "
1 change: 1 addition & 0 deletions daint.cscs.ch/hybrid-s904/computer-configure.yaml
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
---
safe_interval: 60
proxy_command: "ssh -q -Y [email protected] netcat ela.cscs.ch 22"
1 change: 1 addition & 0 deletions daint.cscs.ch/hybrid-s904/computer-setup.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "daint-hybrid-s904"
hostname: "daint.cscs.ch"
description: "Piz Daint supercomputer at CSCS Lugano, Switzerland, using the hybrid partition. Project s904"
Expand Down
1 change: 1 addition & 0 deletions daint.cscs.ch/hybrid/computer-configure.yaml
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
---
safe_interval: 60
proxy_command: "ssh -q -Y [email protected] netcat ela.cscs.ch 22"
1 change: 1 addition & 0 deletions daint.cscs.ch/hybrid/computer-setup.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "daint-hybrid"
hostname: "daint.cscs.ch"
description: "Piz Daint supercomputer at CSCS Lugano, Switzerland, using the hybrid partition. HyperThreading is off"
Expand Down
1 change: 1 addition & 0 deletions daint.cscs.ch/multicore/computer-configure.yaml
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
---
safe_interval: 60
proxy_command: "ssh -q -Y [email protected] netcat ela.cscs.ch 22"
1 change: 1 addition & 0 deletions daint.cscs.ch/multicore/computer-setup.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "daint-mc"
hostname: "daint.cscs.ch"
description: Piz Daint supercomputer at CSCS Lugano, Switzerland, multicore partition.
Expand Down
1 change: 1 addition & 0 deletions eiger.cscs.ch/codes/pw-6.7-multicore.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "pw-6.7"
description: "QuantumESPRESSO 6.7.0 compiled by CSCS"
input_plugin: "quantumespresso.pw"
Expand Down
1 change: 1 addition & 0 deletions eiger.cscs.ch/multicore-mr0-debug/computer-configure.yaml
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
---
safe_interval: 60
proxy_command: "ssh -q -Y {username}@ela.cscs.ch netcat ela.cscs.ch 22"
1 change: 1 addition & 0 deletions eiger.cscs.ch/multicore-mr0-debug/computer-setup.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "eiger-mc-mr0-debug"
hostname: "eiger.cscs.ch"
description: Eiger is the production partition on Alps, the HPE Cray EX Supercomputer.
Expand Down
11 changes: 11 additions & 0 deletions eiger.cscs.ch/multicore-mr0-debug/pw-6.7.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
label: "pw-6.7"
description: "QuantumESPRESSO 6.7.0 compiled by CSCS"
input_plugin: "quantumespresso.pw"
on_computer: true
remote_abs_path: "/apps/eiger/UES/jenkins/1.4.0/software/QuantumESPRESSO/6.7.0-cpeIntel-21.06/bin/pw.x"
computer: "eiger-mc-mr0-debug"
prepend_text: |
module load cpeIntel/21.06
module load QuantumESPRESSO/6.7.0
append_text: " "
1 change: 1 addition & 0 deletions eiger.cscs.ch/multicore-mr0/computer-configure.yaml
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
---
safe_interval: 60
proxy_command: "ssh -q -Y {username}@ela.cscs.ch netcat ela.cscs.ch 22"
1 change: 1 addition & 0 deletions eiger.cscs.ch/multicore-mr0/computer-setup.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "eiger-mc-mr0"
hostname: "eiger.cscs.ch"
description: Eiger is the production partition on Alps, the HPE Cray EX Supercomputer.
Expand Down
1 change: 1 addition & 0 deletions fidis.epfl.ch/codes/cp2k-5.1.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "cp2k-5.1"
description: "CP2K 5.1 compiled by SCITAS"
input_plugin: "cp2k"
Expand Down
1 change: 1 addition & 0 deletions fidis.epfl.ch/codes/ddec-09_26_2017.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "ddec-09_26_2017"
description: "chargemol_09_26_2017 precompiled linux/parallel binary from https://sourceforge.net/projects/ddec/files/"
input_plugin: "ddec"
Expand Down
1 change: 1 addition & 0 deletions fidis.epfl.ch/codes/raspa-4467e14.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "raspa-4467e14"
description: "RASPA2 cloned from https://github.com/iRASPA/RASPA2.git"
input_plugin: "raspa"
Expand Down
1 change: 1 addition & 0 deletions fidis.epfl.ch/codes/zeopp-46ce745.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "zeopp-46ce745"
description: "Zeo++ cloned from https://github.com/mharanczyk/zeoplusplus"
input_plugin: "zeopp.network"
Expand Down
1 change: 1 addition & 0 deletions fidis.epfl.ch/fidis-ch-315/computer-configure.yaml
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
---
proxy_command: "ssh -q -Y [email protected] netcat fidis.epfl.ch 22"
2 changes: 1 addition & 1 deletion fidis.epfl.ch/fidis-ch-315/computer-setup.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@

---
label: "fidis-ch-315"
hostname: "fidis.epfl.ch"
description: fidis EPFL computer
Expand Down
14 changes: 14 additions & 0 deletions fidis.epfl.ch/fidis-ch-315/cp2k-5.1.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
label: "cp2k-5.1"
description: "CP2K 5.1 compiled by SCITAS"
input_plugin: "cp2k"
on_computer: true
remote_abs_path: "/ssoft/spack/cornalin/v2/opt/spack/linux-rhel7-x86_E5v4_Mellanox/intel-17.0.2/cp2k-5.1-gke4ujsx5qhx6zt7ncfo22hajqhp6n6b/bin/cp2k.popt"
computer: "fidis-ch-315"
prepend_text: |
source /ssoft/spack/bin/slmodules.sh -r deprecated
module load intel
module load intel-mpi intel-mkl
module load cp2k/5.1-mpi
export CP2K_DATA_DIR=/work/lsmo/aiida-lsmo-codes/data/cp2k/data
append_text: " "
9 changes: 9 additions & 0 deletions fidis.epfl.ch/fidis-ch-315/ddec-09_26_2017.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
label: "ddec-09_26_2017"
description: "chargemol_09_26_2017 precompiled linux/parallel binary from https://sourceforge.net/projects/ddec/files/"
input_plugin: "ddec"
on_computer: true
remote_abs_path: "/work/lsmo/aiida-lsmo-codes/bin/chargemol_09_26_2017_ubu18"
computer: "fidis-ch-315"
prepend_text: " "
append_text: " "
1 change: 1 addition & 0 deletions fidis.epfl.ch/fidis-serial/computer-configure.yaml
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
---
proxy_command: "ssh -q -Y [email protected] netcat fidis.epfl.ch 22"
1 change: 1 addition & 0 deletions fidis.epfl.ch/fidis-serial/computer-setup.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
---
label: "fidis-serial"
hostname: "fidis.epfl.ch"
description: Fidis EPFL computer using serial partition
Expand Down
12 changes: 12 additions & 0 deletions fidis.epfl.ch/fidis-serial/raspa-4467e14.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
label: "raspa-4467e14"
description: "RASPA2 cloned from https://github.com/iRASPA/RASPA2.git"
input_plugin: "raspa"
on_computer: true
remote_abs_path: "/work/lsmo/aiida-lsmo-codes/bin/simulate_4467e14_fidis"
computer: "fidis-serial"
prepend_text: |
export RASPA_DIR=/work/lsmo/aiida-lsmo-codes/data/raspa
export DYLD_LIBRARY_PATH=/work/lsmo/aiida-lsmo-codes/lib/raspa_4467e14_fidis
export LD_LIBRARY_PATH=/work/lsmo/aiida-lsmo-codes/lib/raspa_4467e14_fidis
append_text: " "
9 changes: 9 additions & 0 deletions fidis.epfl.ch/fidis-serial/zeopp-46ce745.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
label: "zeopp-46ce745"
description: "Zeo++ cloned from https://github.com/mharanczyk/zeoplusplus"
input_plugin: "zeopp.network"
on_computer: true
remote_abs_path: "/work/lsmo/aiida-lsmo-codes/bin/network_46ce745_ubu18"
computer: "fidis-serial"
prepend_text: " "
append_text: " "
1 change: 1 addition & 0 deletions fidis.epfl.ch/fidis/computer-configure.yaml
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
---
proxy_command: "ssh -q -Y [email protected] netcat fidis.epfl.ch 22"
Loading