Skip to content

Update to 1.7.0 alpha.2 #881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Apr 9, 2025
Merged

Update to 1.7.0 alpha.2 #881

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
0d87018
Update to v1.7.0-alpha.2
xylar Jan 9, 2025
b4a6a05
Udpate mpas_tools to v1.0.0
xylar Jan 9, 2025
6b56718
Update tool names in landice framework and tests
xylar Jan 9, 2025
822ec2a
Update tool names in the docs
xylar Jan 9, 2025
1619bb4
Remove jigsawpy as a required dependency
xylar Apr 6, 2025
e51fcd1
Create scrip with xarray in ismip6 forcing
xylar Apr 6, 2025
58939f9
Update syntax for pyremap 2.0.0
xylar Apr 6, 2025
6be8452
Pass if removing jigsaw/jigsaspy fails
xylar Apr 6, 2025
5786a7b
Fix GIS mesh test
xylar Apr 6, 2025
044ace2
Update to pyremap >=2.0.0,<3.0.0
xylar Apr 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/gis_coarse_meshgen.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ on:
- compass/landice/tests/greenland/mesh.py

jobs:
setup_and_run_job:
name: setup_and_run_job
gis-coarse-meshgen:
name: GIS Coarse Mesh Generation
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
Expand All @@ -23,10 +23,10 @@ jobs:
pushd compass

# create the compass environment
./conda/configure_compass_env.py --conda $root/mambaForge --mpi mpich
./conda/configure_compass_env.py --conda $root/mambaForge --env_name compass_test --mpi mpich

# enable the compass environment
source load_dev_compass_1.7.0-alpha.1_mpich.sh
source load_compass_test_mpich.sh

popd

Expand Down
24 changes: 12 additions & 12 deletions compass/landice/mesh.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -688,22 +688,22 @@ def build_mali_mesh(self, cell_width, x1, y1, geom_points,
logger.info('writing grid_converted.nc')
write_netcdf(dsMesh, 'grid_converted.nc')
levels = section.get('levels')
args = ['create_landice_grid_from_generic_MPAS_grid.py',
args = ['create_landice_grid_from_generic_mpas_grid',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xylar , to clarify, we could replace all of these changes with function calls, right? I'm assuming you retained the check_call calls to minimize how invasive this PR is. @trhille and I are on board with that, but just wanted to make sure we understood what's going on.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have to remind myself of the details. You should not try to call the entry-point functions because the command-line arguments are still expected for those and they won't work as expected. But in many cases there are other functions with "proper" argument you could call instead and that might be preferable.

In any case, yes, I tried to change things minimally.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at this further:
https://github.com/MPAS-Dev/MPAS-Tools/blob/701c8d97dfa733ec7128f32aff5ddd4f48a14e2c/conda_package/mpas_tools/landice/create.py#L10
The function create_from_generic_mpas_grid() as written is not really callable because it assumes that OptionParser.parse_args() will return the expected command-line arguments (which compass obviously wouldn't do). Elsewhere in mpas_tools, we have a function with proper arguments that can be called directly and then we have entry point function that do arg parsing and then call the other function. It would be great to move in that direction but I didn't try to do that for you all because it was one step too far for me.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xylar , got it - that makes sense. I'm happy with changing things in this way for now (here and the companion MPAS-Tools PR).

'-i', 'grid_converted.nc',
'-o', 'grid_preCull.nc',
'-l', levels, '-v', 'glimmer']

check_call(args, logger=logger)

args = ['interpolate_to_mpasli_grid.py', '-s',
args = ['interpolate_to_mpasli_grid', '-s',
gridded_dataset, '-d',
'grid_preCull.nc', '-m', 'b', '-t']

check_call(args, logger=logger)

cullDistance = section.get('cull_distance')
if float(cullDistance) > 0.:
args = ['define_cullMask.py', '-f',
args = ['define_landice_cull_mask', '-f',
'grid_preCull.nc', '-m',
'distance', '-d', cullDistance]

Expand All @@ -715,7 +715,7 @@ def build_mali_mesh(self, cell_width, x1, y1, geom_points,
if geojson_file is not None:
# This step is only necessary because the GeoJSON region
# is defined by lat-lon.
args = ['set_lat_lon_fields_in_planar_grid.py', '-f',
args = ['set_lat_lon_fields_in_planar_grid', '-f',
'grid_preCull.nc', '-p', projection]

check_call(args, logger=logger)
Expand All @@ -742,7 +742,7 @@ def build_mali_mesh(self, cell_width, x1, y1, geom_points,
write_netcdf(dsMesh, 'culled.nc')

logger.info('Marking horns for culling')
args = ['mark_horns_for_culling.py', '-f', 'culled.nc']
args = ['mark_horns_for_culling', '-f', 'culled.nc']

check_call(args, logger=logger)

Expand All @@ -753,24 +753,24 @@ def build_mali_mesh(self, cell_width, x1, y1, geom_points,
dsMesh = sort_mesh(dsMesh)
write_netcdf(dsMesh, 'dehorned.nc')

args = ['create_landice_grid_from_generic_MPAS_grid.py', '-i',
args = ['create_landice_grid_from_generic_mpas_grid', '-i',
'dehorned.nc', '-o',
mesh_name, '-l', levels, '-v', 'glimmer',
'--beta', '--thermal', '--obs', '--diri']

check_call(args, logger=logger)

args = ['interpolate_to_mpasli_grid.py', '-s',
args = ['interpolate_to_mpasli_grid', '-s',
gridded_dataset, '-d', mesh_name, '-m', 'b']

check_call(args, logger=logger)

logger.info('Marking domain boundaries dirichlet')
args = ['mark_domain_boundaries_dirichlet.py',
args = ['mark_domain_boundaries_dirichlet',
'-f', mesh_name]
check_call(args, logger=logger)

args = ['set_lat_lon_fields_in_planar_grid.py', '-f',
args = ['set_lat_lon_fields_in_planar_grid', '-f',
mesh_name, '-p', projection]
check_call(args, logger=logger)

Expand Down Expand Up @@ -1064,7 +1064,7 @@ def __guess_scrip_name(filename):

logger.info('creating scrip file for source dataset')
# Note: writing scrip file to workdir
args = ['create_SCRIP_file_from_planar_rectangular_grid.py',
args = ['create_scrip_file_from_planar_rectangular_grid',
'-i', source_file,
'-s', source_scrip,
'-p', proj,
Expand All @@ -1083,8 +1083,8 @@ def __guess_scrip_name(filename):
check_call(args, logger=logger)

# Perform actual interpolation using the weights
logger.info('calling interpolate_to_mpasli_grid.py')
args = ['interpolate_to_mpasli_grid.py',
logger.info('calling interpolate_to_mpasli_grid')
args = ['interpolate_to_mpasli_grid',
'-s', source_file,
'-d', dest_file,
'-m', 'e',
Expand Down
4 changes: 2 additions & 2 deletions compass/landice/tests/antarctica/mesh.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,8 +121,8 @@ def run(self):
# Also, it should be assessed if bilinear or
# barycentric used here is preferred for this application.
# Current thinking is they are both equally appropriate.
logger.info('calling interpolate_to_mpasli_grid.py')
args = ['interpolate_to_mpasli_grid.py', '-s',
logger.info('calling interpolate_to_mpasli_grid')
args = ['interpolate_to_mpasli_grid', '-s',
preprocessed_gridded_dataset,
'-d',
self.mesh_filename,
Expand Down
2 changes: 1 addition & 1 deletion compass/landice/tests/antarctica/mesh_gen/mesh_gen.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ bedmachine_filename = BedMachineAntarctica_2020-07-15_v02_edits_floodFill_extrap
measures_filename = antarctica_ice_velocity_450m_v2_edits_extrap.nc

# projection of the source datasets, according to the dictionary keys
# create_SCRIP_file_from_planar_rectangular_grid.py from MPAS_Tools
# create_scrip_file_from_planar_rectangular_grid from MPAS_Tools
src_proj = ais-bedmap2

# number of processors to use for ESMF_RegridWeightGen
Expand Down
9 changes: 4 additions & 5 deletions compass/landice/tests/circular_shelf/setup_mesh.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
import numpy
from netCDF4 import Dataset as NetCDFFile

from mpas_tools.planar_hex import make_planar_hex_mesh
from mpas_tools.io import write_netcdf
from mpas_tools.mesh.conversion import convert, cull
from mpas_tools.logging import check_call
from mpas_tools.mesh.conversion import convert, cull
from mpas_tools.planar_hex import make_planar_hex_mesh
from netCDF4 import Dataset as NetCDFFile

from compass.model import make_graph_file
from compass.step import Step
Expand Down Expand Up @@ -60,7 +59,7 @@ def run(self):
write_netcdf(dsMesh, 'mpas_grid.nc')

levels = section.get('levels')
args = ['create_landice_grid_from_generic_MPAS_grid.py',
args = ['create_landice_grid_from_generic_mpas_grid',
'-i', 'mpas_grid.nc',
'-o', 'landice_grid.nc',
'-l', levels,
Expand Down
11 changes: 5 additions & 6 deletions compass/landice/tests/dome/setup_mesh.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
import numpy
from netCDF4 import Dataset as NetCDFFile

from mpas_tools.planar_hex import make_planar_hex_mesh
from mpas_tools.io import write_netcdf
from mpas_tools.mesh.conversion import convert, cull
from mpas_tools.logging import check_call
from mpas_tools.mesh.conversion import convert, cull
from mpas_tools.planar_hex import make_planar_hex_mesh
from netCDF4 import Dataset as NetCDFFile

from compass.model import make_graph_file
from compass.step import Step
Expand Down Expand Up @@ -71,7 +70,7 @@ def run(self):
write_netcdf(dsMesh, 'mpas_grid.nc')

levels = section.get('levels')
args = ['create_landice_grid_from_generic_MPAS_grid.py',
args = ['create_landice_grid_from_generic_mpas_grid',
'-i', 'mpas_grid.nc',
'-o', 'landice_grid.nc',
'-l', levels]
Expand Down Expand Up @@ -158,7 +157,7 @@ def _setup_dome_initial_conditions(config, logger, filename):
thickness_field[r < r0] = h0 * (1.0 - (r[r < r0] / r0) ** 2) ** 0.5
elif dome_type == 'halfar':
thickness_field[r < r0] = h0 * (
1.0 - (r[r < r0] / r0) ** (4.0 / 3.0)) ** (3.0 / 7.0)
1.0 - (r[r < r0] / r0) ** (4.0 / 3.0)) ** (3.0 / 7.0)
else:
raise ValueError('Unexpected dome_type: {}'.format(dome_type))
thickness[0, :] = thickness_field
Expand Down
9 changes: 4 additions & 5 deletions compass/landice/tests/eismint2/setup_mesh.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
import xarray

from mpas_tools.planar_hex import make_planar_hex_mesh
from mpas_tools.io import write_netcdf
from mpas_tools.mesh.conversion import convert, cull
from mpas_tools.logging import check_call
from mpas_tools.mesh.conversion import convert, cull
from mpas_tools.planar_hex import make_planar_hex_mesh

from compass.model import make_graph_file
from compass.step import Step
Expand Down Expand Up @@ -48,7 +47,7 @@ def run(self):
dsMesh.close()

radius = section.get('radius')
args = ['define_cullMask.py',
args = ['define_landice_cull_mask',
'-f', 'mpas_grid.nc',
'-m', 'radius',
'-d', radius]
Expand All @@ -61,7 +60,7 @@ def run(self):
write_netcdf(dsMesh, 'mpas_grid2.nc')

levels = section.get('levels')
args = ['create_landice_grid_from_generic_MPAS_grid.py',
args = ['create_landice_grid_from_generic_mpas_grid',
'-i', 'mpas_grid2.nc',
'-o', 'landice_grid.nc',
'-l', levels,
Expand Down
9 changes: 4 additions & 5 deletions compass/landice/tests/enthalpy_benchmark/setup_mesh.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
from netCDF4 import Dataset as NetCDFFile

from mpas_tools.planar_hex import make_planar_hex_mesh
from mpas_tools.io import write_netcdf
from mpas_tools.mesh.conversion import convert, cull
from mpas_tools.logging import check_call
from mpas_tools.mesh.conversion import convert, cull
from mpas_tools.planar_hex import make_planar_hex_mesh
from netCDF4 import Dataset as NetCDFFile

from compass.model import make_graph_file
from compass.step import Step
Expand Down Expand Up @@ -49,7 +48,7 @@ def run(self):
dsMesh = convert(dsMesh, logger=logger)
write_netcdf(dsMesh, 'mpas_grid.nc')

args = ['create_landice_grid_from_generic_MPAS_grid.py',
args = ['create_landice_grid_from_generic_mpas_grid',
'-i', 'mpas_grid.nc',
'-o', 'landice_grid.nc',
'-l', levels,
Expand Down
2 changes: 1 addition & 1 deletion compass/landice/tests/greenland/mesh_gen/mesh_gen.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ bedmachine_filename = BedMachineGreenland-v5_edits_floodFill_extrap.nc
measures_filename = greenland_vel_mosaic500_extrap.nc

# projection of the source datasets, according to the dictionary keys
# create_SCRIP_file_from_planar_rectangular_grid.py from MPAS_Tools
# create_scrip_file_from_planar_rectangular_grid from MPAS_Tools
src_proj = gis-gimp

# number of processors to use for ESMF_RegridWeightGen
Expand Down
9 changes: 4 additions & 5 deletions compass/landice/tests/hydro_radial/setup_mesh.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
import numpy as np
from netCDF4 import Dataset as NetCDFFile

from mpas_tools.planar_hex import make_planar_hex_mesh
from mpas_tools.io import write_netcdf
from mpas_tools.mesh.conversion import convert, cull
from mpas_tools.logging import check_call
from mpas_tools.mesh.conversion import convert, cull
from mpas_tools.planar_hex import make_planar_hex_mesh
from netCDF4 import Dataset as NetCDFFile

from compass.model import make_graph_file
from compass.step import Step
Expand Down Expand Up @@ -72,7 +71,7 @@ def run(self):
write_netcdf(dsMesh, 'mpas_grid.nc')

levels = section.get('levels')
args = ['create_landice_grid_from_generic_MPAS_grid.py',
args = ['create_landice_grid_from_generic_mpas_grid',
'-i', 'mpas_grid.nc',
'-o', 'landice_grid.nc',
'-l', levels,
Expand Down
54 changes: 30 additions & 24 deletions compass/landice/tests/ismip6_forcing/create_mapfile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,12 @@
import os
import shutil

import netCDF4
import numpy as np
import xarray as xr
from mpas_tools.logging import check_call
from mpas_tools.scrip.from_mpas import scrip_from_mpas
from pyremap.descriptor.utility import (
create_scrip,
interp_extrap_corners_2d,
unwrap_corners,
)
from pyremap.descriptor.utility import interp_extrap_corners_2d, unwrap_corners
from pyremap.utility import write_netcdf


def build_mapping_file(config, cores, logger, ismip6_grid_file, mapping_file,
Expand Down Expand Up @@ -73,7 +70,7 @@ def build_mapping_file(config, cores, logger, ismip6_grid_file, mapping_file,
if scrip_from_latlon:
create_scrip_from_latlon(ismip6_grid_file, source_grid_scripfile)
else:
args = ["create_SCRIP_file_from_planar_rectangular_grid.py",
args = ["create_scrip_file_from_planar_rectangular_grid",
"--input", ismip6_grid_file,
"--scrip", source_grid_scripfile,
"--proj", ismip6_projection,
Expand All @@ -89,7 +86,7 @@ def build_mapping_file(config, cores, logger, ismip6_grid_file, mapping_file,
mali_mesh_copy = f"{mali_mesh_file}_copy"
shutil.copy(mali_mesh_file, f"{mali_mesh_file}_copy")

args = ["set_lat_lon_fields_in_planar_grid.py",
args = ["set_lat_lon_fields_in_planar_grid",
"--file", mali_mesh_copy,
"--proj", ismip6_projection]

Expand All @@ -106,9 +103,9 @@ def build_mapping_file(config, cores, logger, ismip6_grid_file, mapping_file,
"--method. Available options are 'bilinear',"
"'neareststod', 'conserve'.")

parallel_executable = config.get('parallel', 'parallel_executable')
parallel_executable = config.get("parallel", "parallel_executable")
# split the parallel executable into constituents in case it includes flags
args = parallel_executable.split(' ')
args = parallel_executable.split(" ")
args.extend(["-n", f"{cores}",
"ESMF_RegridWeightGen",
"-s", source_grid_scripfile,
Expand All @@ -133,7 +130,7 @@ def create_scrip_from_latlon(source_grid_file, source_grid_scripfile):
dataset.

This function is needed, c.f. the scrip utility in the MPAS-Tools repo
(i.e. `create_SCRIP_file_from_planar_rectangular_grid.py`), when a dataset
(i.e. `create_scrip_file_from_planar_rectangular_grid`), when a dataset
does not have `x`/`y` coordinates to generate the scrip file from. This is
the case for the atmospheric forcing datasets from ISMIP6
and for RACMO products.
Expand All @@ -148,7 +145,7 @@ def create_scrip_from_latlon(source_grid_file, source_grid_scripfile):
"""

ds = xr.open_dataset(source_grid_file)
out_file = netCDF4.Dataset(source_grid_scripfile, 'w')
ds_out = xr.Dataset()

# RACMO datasets, which use a rotated-pole grid, do not contain `x`/`y`
# dimensions, instead use `rlat`/`rlon` dimensions to find `nx`/`ny`
Expand All @@ -161,15 +158,13 @@ def create_scrip_from_latlon(source_grid_file, source_grid_scripfile):

grid_size = nx * ny

# generate common variables used in scrip files
create_scrip(out_file, grid_size, grid_corners=4, grid_rank=2,
units="degrees", meshName=source_grid_file)

# place the information from our source dataset into the scrip dataset
out_file.variables["grid_center_lat"][:] = ds.lat.values.flat
out_file.variables["grid_center_lon"][:] = ds.lon.values.flat
out_file.variables["grid_dims"][:] = [nx, ny]
out_file.variables["grid_imask"][:] = 1
ds_out["grid_center_lat"] = (("grid_size",), ds.lat.values.flat)
ds_out["grid_center_lon"] = (("grid_size",), ds.lon.values.flat)
ds_out["grid_dims"] = (("grid_rank",), [nx, ny])
ds_out["grid_imask"] = xr.DataArray(
np.ones(ds.sizes["grid_size"], dtype="int32"), dims=("grid_size",)
)

# determine the corners of gricells
if "lat_bnds" in ds and "lon_bnds" in ds:
Expand All @@ -193,7 +188,18 @@ def create_scrip_from_latlon(source_grid_file, source_grid_scripfile):
grid_corner_lat = lat_corner.reshape((grid_size, 4))
grid_corner_lon = lon_corner.reshape((grid_size, 4))

out_file.variables["grid_corner_lat"][:] = grid_corner_lat
out_file.variables["grid_corner_lon"][:] = grid_corner_lon

out_file.close()
ds_out["grid_corner_lat"] = (
("grid_size", "grid_corners"), grid_corner_lat
)
ds_out["grid_corner_lon"] = (
("grid_size", "grid_corners"), grid_corner_lon
)

ds.grid_center_lat.attrs["units"] = "degrees"
ds.grid_center_lon.attrs["units"] = "degrees"
ds.grid_corner_lat.attrs["units"] = "degrees"
ds.grid_corner_lon.attrs["units"] = "degrees"
ds.grid_imask.attrs["units"] = "unitless"

ds.attrs["mesh_name"] = source_grid_file
write_netcdf(ds, source_grid_scripfile, format="NETCDF3_64BIT")
Loading
Loading