Skip to content

Commit

Permalink
ci(parallel): add parallel windows distribution (#1682)
Browse files Browse the repository at this point in the history
Add a parallel Windows distribution to the release workflow, making it available both on the nightly build and in official releases. The archive is named win64par.zip and contains all non-system dependencies. This avoids the need for software installations or other actions requiring elevation for the end user.

Bundled dependencies include:

- MSVC runtime
  * ucrtbase.dll
  * msvcrt.dll
  * msvcp_win.dll
  * MSVCP140.dll
  * VCRUNTIME140.dll
  * VCRUNTIME140_1.dll
- oneAPI (Fortran compiler, MKL, MPI)
  * libifcoremd.dll
  * libmmd.dll
  * mkl_mc3.2.dll
  * mkl_core.2.dll
  * mkl_avx512.2.dll
  * mkl_avx2.2.dll
  * mkl_def.2.dll
  * mkl_sequential.2.dll
  * libfabric.dll
  * impi.dll
  * mpiexec.exe
  * hydra_bstrap_proxy.exe
  * hydra_pmi_proxy.exe
  * hydra_service.exe
- PETSc
  * libpetsc.dll

Miscellaneous:

- factor out action to build mf6 parallel on windows
- add developmode toggle to release_dispatch.yml
  • Loading branch information
wpbonelli authored Mar 22, 2024
1 parent a015e68 commit bb3763c
Show file tree
Hide file tree
Showing 5 changed files with 209 additions and 161 deletions.
129 changes: 129 additions & 0 deletions .github/actions/build-par-win/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
name: Build parallel MF6 (Windows)
description: Build parallel MODFLOW 6 on Windows
runs:
using: "composite"
steps:

- name: Convert line endings
shell: cmd
run: |
unix2dos -n "modflow6\.github\common\configure_petsc.sh" "%TEMP%\configure_petsc.sh"
unix2dos -n "modflow6\.github\common\compile_petsc.sh" "%TEMP%\compile_petsc.sh"
unix2dos -n "modflow6\.github\common\compile_modflow6.bat" "%TEMP%\compile_modflow6.bat"
unix2dos -n "modflow6\.github\common\test_modflow6.bat" "%TEMP%\test_modflow6.bat"
- name: Hide Strawberry programs
shell: bash
run: |
mkdir "$RUNNER_TEMP/strawberry"
mv /c/Strawberry/c/bin/gmake "$RUNNER_TEMP/strawberry/gmake"
mv /c/Strawberry/perl/bin/pkg-config "$RUNNER_TEMP/strawberry/pkg-config"
mv /c/Strawberry/perl/bin/pkg-config.bat "$RUNNER_TEMP/strawberry/pkg-config.bat"
- name: Get date
id: get-date
shell: bash
run: echo "date=$(/bin/date -u "+%Y%m%d")" >> "$GITHUB_OUTPUT"

- name: Set oneAPI install dir
shell: bash
run: echo "ONEAPI_ROOT=C:\Program Files (x86)\Intel\oneAPI" >> "$GITHUB_ENV"

- name: Restore oneAPI cache
id: oneapi-cache
uses: actions/cache/restore@v3
with:
path: ${{ env.ONEAPI_ROOT }}
key: oneapi-${{ runner.os }}-${{ steps.get-date.outputs.date }}

- name: Install oneAPI BaseKit
shell: bash
if: steps.oneapi-cache.outputs.cache-hit != 'true'
run: |
url="https://registrationcenter-download.intel.com/akdlm/IRC_NAS/5cb30fb9-21e9-47e8-82da-a91e00191670/w_BaseKit_p_2024.0.1.45_offline.exe"
cmp="intel.oneapi.win.mkl.devel"
"modflow6/.github/common/install_intel_windows.bat" $url $cmp
rm -rf $TEMP/webimage.exe
rm -rf $TEMP/webimage_extracted
- name: Install oneAPI HPCKit
shell: bash
if: steps.oneapi-cache.outputs.cache-hit != 'true'
run: |
url="https://registrationcenter-download.intel.com/akdlm/IRC_NAS/7a6db8a1-a8b9-4043-8e8e-ca54b56c34e4/w_HPCKit_p_2024.0.1.35_offline.exe"
cmp="intel.oneapi.win.cpp-dpcpp-common:intel.oneapi.win.ifort-compiler:intel.oneapi.win.mpi.devel"
"modflow6/.github/common/install_intel_windows.bat" $url $cmp
rm -rf $TEMP/webimage.exe
rm -rf $TEMP/webimage_extracted
- name: Save oneAPI cache
if: steps.oneapi-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v3
with:
path: ${{ env.ONEAPI_ROOT }}
key: oneapi-${{ runner.os }}-${{ steps.get-date.outputs.date }}

- name: Restore PETSc cache
id: petsc-cache
uses: actions/cache/restore@v3
with:
path: petsc
key: petsc-${{ runner.os }}-${{ steps.get-date.outputs.date }}

- name: Download PETSc
if: steps.petsc-cache.outputs.cache-hit != 'true'
shell: bash
run: |
curl https://web.cels.anl.gov/projects/petsc/download/release-snapshots/petsc-3.20.5.tar.gz -O -J
mkdir petsc
tar -xzf petsc-3.20.5.tar.gz -C petsc --strip-components=1
- name: Setup Cygwin
if: steps.petsc-cache.outputs.cache-hit != 'true'
uses: egor-tensin/setup-cygwin@v4
with:
packages: python3 make gcc-core gcc-g++ pkg-config

- name: Hide Cygwin linker
if: steps.petsc-cache.outputs.cache-hit != 'true'
shell: C:\tools\cygwin\bin\bash.exe --login --norc -eo pipefail -o igncr '{0}'
run: mv /usr/bin/link.exe /usr/bin/link-cygwin.exe

- name: Configure PETSc
if: steps.petsc-cache.outputs.cache-hit != 'true'
shell: cmd
run: |
"%ONEAPI_ROOT%\setvars.bat" intel64 vs2022 && "C:\tools\cygwin\bin\bash.exe" --login --norc -eo pipefail -o igncr "%TEMP%\configure_petsc.sh"
- name: Build PETSc
if: steps.petsc-cache.outputs.cache-hit != 'true'
shell: cmd
run: |
"%ONEAPI_ROOT%\setvars.bat" intel64 vs2022 && "C:\tools\cygwin\bin\bash.exe" --login --norc -eo pipefail -o igncr "%TEMP%\compile_petsc.sh"
- name: Save PETSc cache
if: steps.petsc-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v3
with:
path: petsc
key: petsc-${{ runner.os }}-${{ steps.get-date.outputs.date }}

- name: Setup PETSC environment
shell: cmd
run: |
set PETSC_DIR=%GITHUB_WORKSPACE%\petsc
set PETSC_ARCH=arch-mswin-c-opt
echo PETSC_DIR=%PETSC_DIR%>>%GITHUB_ENV%
echo PETSC_ARCH=%PETSC_ARCH%>>%GITHUB_ENV%
echo %PETSC_DIR%\%PETSC_ARCH%\lib>>%GITHUB_PATH%
- name: Build modflow6
shell: cmd
run: |
"%ONEAPI_ROOT%\setvars.bat" intel64 vs2022 && "%TEMP%\compile_modflow6.bat"
- name: Show Meson logs
if: failure()
shell: bash
working-directory: modflow6
run: cat builddir/meson-logs/meson-log.txt
125 changes: 2 additions & 123 deletions .github/actions/test-par-win/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,129 +4,8 @@ runs:
using: "composite"
steps:

- name: Convert line endings
shell: cmd
run: |
unix2dos -n "modflow6\.github\common\configure_petsc.sh" "%TEMP%\configure_petsc.sh"
unix2dos -n "modflow6\.github\common\compile_petsc.sh" "%TEMP%\compile_petsc.sh"
unix2dos -n "modflow6\.github\common\compile_modflow6.bat" "%TEMP%\compile_modflow6.bat"
unix2dos -n "modflow6\.github\common\test_modflow6.bat" "%TEMP%\test_modflow6.bat"
- name: Hide Strawberry programs
shell: bash
run: |
mkdir "$RUNNER_TEMP/strawberry"
mv /c/Strawberry/c/bin/gmake "$RUNNER_TEMP/strawberry/gmake"
mv /c/Strawberry/perl/bin/pkg-config "$RUNNER_TEMP/strawberry/pkg-config"
mv /c/Strawberry/perl/bin/pkg-config.bat "$RUNNER_TEMP/strawberry/pkg-config.bat"
- name: Get date
id: get-date
shell: bash
run: echo "date=$(/bin/date -u "+%Y%m%d")" >> "$GITHUB_OUTPUT"

- name: Set oneAPI install dir
shell: bash
run: echo "ONEAPI_ROOT=C:\Program Files (x86)\Intel\oneAPI" >> "$GITHUB_ENV"

- name: Restore oneAPI cache
id: oneapi-cache
uses: actions/cache/restore@v3
with:
path: ${{ env.ONEAPI_ROOT }}
key: oneapi-${{ runner.os }}-${{ steps.get-date.outputs.date }}

- name: Install oneAPI BaseKit
shell: bash
if: steps.oneapi-cache.outputs.cache-hit != 'true'
run: |
url="https://registrationcenter-download.intel.com/akdlm/IRC_NAS/5cb30fb9-21e9-47e8-82da-a91e00191670/w_BaseKit_p_2024.0.1.45_offline.exe"
cmp="intel.oneapi.win.mkl.devel"
"modflow6/.github/common/install_intel_windows.bat" $url $cmp
rm -rf $TEMP/webimage.exe
rm -rf $TEMP/webimage_extracted
- name: Install oneAPI HPCKit
shell: bash
if: steps.oneapi-cache.outputs.cache-hit != 'true'
run: |
url="https://registrationcenter-download.intel.com/akdlm/IRC_NAS/7a6db8a1-a8b9-4043-8e8e-ca54b56c34e4/w_HPCKit_p_2024.0.1.35_offline.exe"
cmp="intel.oneapi.win.cpp-dpcpp-common:intel.oneapi.win.ifort-compiler:intel.oneapi.win.mpi.devel"
"modflow6/.github/common/install_intel_windows.bat" $url $cmp
rm -rf $TEMP/webimage.exe
rm -rf $TEMP/webimage_extracted
- name: Save oneAPI cache
if: steps.oneapi-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v3
with:
path: ${{ env.ONEAPI_ROOT }}
key: oneapi-${{ runner.os }}-${{ steps.get-date.outputs.date }}

- name: Restore PETSc cache
id: petsc-cache
uses: actions/cache/restore@v3
with:
path: petsc
key: petsc-${{ runner.os }}-${{ steps.get-date.outputs.date }}

- name: Download PETSc
if: steps.petsc-cache.outputs.cache-hit != 'true'
shell: bash
run: |
curl https://web.cels.anl.gov/projects/petsc/download/release-snapshots/petsc-3.20.5.tar.gz -O -J
mkdir petsc
tar -xzf petsc-3.20.5.tar.gz -C petsc --strip-components=1
- name: Setup Cygwin
if: steps.petsc-cache.outputs.cache-hit != 'true'
uses: egor-tensin/setup-cygwin@v4
with:
packages: python3 make gcc-core gcc-g++ pkg-config

- name: Hide Cygwin linker
if: steps.petsc-cache.outputs.cache-hit != 'true'
shell: C:\tools\cygwin\bin\bash.exe --login --norc -eo pipefail -o igncr '{0}'
run: mv /usr/bin/link.exe /usr/bin/link-cygwin.exe

- name: Configure PETSc
if: steps.petsc-cache.outputs.cache-hit != 'true'
shell: cmd
run: |
"%ONEAPI_ROOT%\setvars.bat" intel64 vs2022 && "C:\tools\cygwin\bin\bash.exe" --login --norc -eo pipefail -o igncr "%TEMP%\configure_petsc.sh"
- name: Build PETSc
if: steps.petsc-cache.outputs.cache-hit != 'true'
shell: cmd
run: |
"%ONEAPI_ROOT%\setvars.bat" intel64 vs2022 && "C:\tools\cygwin\bin\bash.exe" --login --norc -eo pipefail -o igncr "%TEMP%\compile_petsc.sh"
- name: Save PETSc cache
if: steps.petsc-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v3
with:
path: petsc
key: petsc-${{ runner.os }}-${{ steps.get-date.outputs.date }}

- name: Setup PETSC environment
shell: cmd
run: |
set PETSC_DIR=%GITHUB_WORKSPACE%\petsc
set PETSC_ARCH=arch-mswin-c-opt
echo PETSC_DIR=%PETSC_DIR%>>%GITHUB_ENV%
echo PETSC_ARCH=%PETSC_ARCH%>>%GITHUB_ENV%
echo %PETSC_DIR%\%PETSC_ARCH%\lib>>%GITHUB_PATH%
- name: Build modflow6
shell: cmd
run: |
"%ONEAPI_ROOT%\setvars.bat" intel64 vs2022 && "%TEMP%\compile_modflow6.bat"
- name: Show Meson logs
if: failure()
shell: bash
working-directory: modflow6
run: cat builddir/meson-logs/meson-log.txt
- name: Build MF6 parallel
uses: ./modflow6/.github/actions/build-par-win

- name: Update flopy
working-directory: modflow6/autotest
Expand Down
Loading

0 comments on commit bb3763c

Please sign in to comment.