Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[py2fgen]: support for Fortran granules #402

Merged
merged 64 commits into from
Apr 12, 2024
Merged
Show file tree
Hide file tree
Changes from 62 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
4ecc792
Support assumed size arrays in Fortran interface
samkellerhals Feb 29, 2024
6abe69b
Fix tests
samkellerhals Mar 1, 2024
a677bb9
Merge branch 'main' of github.com:C2SM/icon4py into run-granule-with-…
samkellerhals Mar 1, 2024
06209d5
Add diffusion test case with serialized data
samkellerhals Mar 1, 2024
a2be419
Add run_diffusion_model driver
samkellerhals Mar 4, 2024
d13629a
Add edge and cell geometry args to dummy diffusion driver
samkellerhals Mar 5, 2024
355a402
Merge branch 'main' of github.com:C2SM/icon4py into run-granule-with-…
samkellerhals Mar 5, 2024
7be6eeb
Python dummy driver for diffusion wrapper
samkellerhals Mar 5, 2024
bef7877
Support exposing multiple functions
samkellerhals Mar 5, 2024
e75ded8
test compilation with multiple embedded functions
samkellerhals Mar 6, 2024
7b59d7c
Cleanup generated subroutine names
samkellerhals Mar 6, 2024
8ed37ea
Compile and run diffusion granule from Fortran driver
samkellerhals Mar 6, 2024
3a8b58a
Run diffusion twice in unit test, and remove debug flag
samkellerhals Mar 6, 2024
c96a341
Expand codegen tests
samkellerhals Mar 7, 2024
5b37966
Add CFFI integration test
samkellerhals Mar 7, 2024
6c3b690
cleanup fstring
samkellerhals Mar 7, 2024
1fb335f
Return exit code from embedded Python function
samkellerhals Mar 8, 2024
4ee9c1d
Remove call.py
samkellerhals Mar 8, 2024
a13343d
Reproduce boolean memory repr error
samkellerhals Mar 8, 2024
3ab48b3
convert boolean integer arrays to numpy bool types at runtime
samkellerhals Mar 11, 2024
36a013f
add missing type hint
samkellerhals Mar 11, 2024
9955a07
Use chia rui's diffusion
samkellerhals Mar 12, 2024
782db41
Update diffusion wrapper
samkellerhals Mar 14, 2024
2a72fbf
Use log file for debug and exception information
samkellerhals Mar 14, 2024
1f1a5ba
Run granule on gpu (#413)
samkellerhals Mar 15, 2024
2a82047
Experiments with Fortran to Python GPU interoperability (#415)
samkellerhals Mar 21, 2024
e0a0a8f
pull from main, fix conflicts
samkellerhals Mar 21, 2024
8649317
Fix py2f tests
samkellerhals Mar 22, 2024
e139664
Add parsing tests
samkellerhals Mar 26, 2024
ca7faec
Install all namespace packages in CI
samkellerhals Mar 26, 2024
478640a
Move config to icon4py.model.common
samkellerhals Mar 28, 2024
0ec7f3d
remove full trace
samkellerhals Mar 28, 2024
e1bceba
remove locals from pytest
samkellerhals Mar 28, 2024
ed4072a
Merge branch 'main' of github.com:C2SM/icon4py into run-granule-with-…
samkellerhals Mar 28, 2024
8fee9ca
Minor cleanup
samkellerhals Mar 28, 2024
bd77f3a
fix python wrapper test
samkellerhals Mar 28, 2024
c0016b9
Fix KHalfDims in diffusion granule (#427)
abishekg7 Apr 2, 2024
1421ae0
Cleanup ICON_GRID_LOC config and asnumpy casts
samkellerhals Apr 3, 2024
67e5406
Remove most custom offset providers
samkellerhals Apr 3, 2024
5303dbe
make cupy optional in icon4pytools
samkellerhals Apr 3, 2024
61536fc
fix tests
samkellerhals Apr 3, 2024
660d9c0
Use as_field in test_diffusion_wrapper
samkellerhals Apr 4, 2024
2d343fb
Change CffiPlugin attr
samkellerhals Apr 4, 2024
c11b705
cleanup
samkellerhals Apr 4, 2024
9adf853
Pull main and resolve conflicts
samkellerhals Apr 5, 2024
9b16b99
Skip diffusion test
samkellerhals Apr 5, 2024
d3ef336
Conditional definition of unpacking functions
samkellerhals Apr 5, 2024
b381b5e
Add linit to diffusion_run
samkellerhals Apr 8, 2024
e13c577
Update from main
samkellerhals Apr 8, 2024
7820dfe
update diffusion config
samkellerhals Apr 9, 2024
a676495
Remove logging file
samkellerhals Apr 9, 2024
de2f37c
fix python wrapper test
samkellerhals Apr 9, 2024
f93a7bb
use xp.asarray for physical heights
samkellerhals Apr 9, 2024
5f36d38
Fix backend selection
samkellerhals Apr 10, 2024
2dcef8a
Merge branch 'main' of github.com:C2SM/icon4py into run-granule-with-…
samkellerhals Apr 10, 2024
21b6126
Log device usage in wrapper
samkellerhals Apr 10, 2024
c1ebc0e
Disable diffusion_wrapper test as grid is required
samkellerhals Apr 10, 2024
6406ef3
PR review cleanup
samkellerhals Apr 11, 2024
1ad3efd
disable test
samkellerhals Apr 11, 2024
db571b1
cleanup imports
samkellerhals Apr 11, 2024
b8cca84
Default logging level to ERROR in release mode
samkellerhals Apr 11, 2024
aa1efd6
fix test
samkellerhals Apr 11, 2024
138b9d8
Update diffusion wrapper
samkellerhals Apr 12, 2024
46bc7bd
Use int32 in vertical.py
samkellerhals Apr 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/icon4py-qa.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,7 @@ jobs:
**/base-requirements-dev.txt
**/requirements.txt
**/requirements-dev.txt
- name: Install icon4py-model packages
working-directory: model
- name: Install all icon4py namespace packages
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install -r ./requirements-dev.txt
Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# nvidia
*.ncu-rep


### Custom ####
_build
_local
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@
from enum import Enum
from typing import Final, Optional

import numpy as np
from gt4py.next import as_field
from gt4py.next.common import Dimension
from gt4py.next.ffront.fbuiltins import Field, int32
Expand Down Expand Up @@ -73,8 +72,7 @@
mo_intp_rbf_rbf_vec_interpol_vertex,
)
from icon4py.model.common.states.prognostic_state import PrognosticState
from icon4py.model.common.model_backend import backend

from icon4py.model.common.settings import xp

"""
Diffusion module ported from ICON mo_nh_diffusion.f90.
Expand Down Expand Up @@ -257,9 +255,6 @@ def _validate(self):
self.apply_to_temperature = False
self.apply_to_horizontal_wind = False
self.apply_to_vertical_wind = False
halungge marked this conversation as resolved.
Show resolved Hide resolved
else:
self.apply_to_temperature = True
self.apply_to_horizontal_wind = True

if self.shear_type not in (
TurbulenceShearForcingType.VERTICAL_OF_HORIZONTAL_WIND,
Expand Down Expand Up @@ -474,7 +469,7 @@ def _allocate(*dims: Dimension):

def _index_field(dim: Dimension, size=None):
size = size if size else self.grid.size[dim]
return as_field((dim,), np.arange(size, dtype=int32))
return as_field((dim,), xp.arange(size, dtype=int32))

self.diff_multfac_vn = _allocate(KDim)

Expand All @@ -492,7 +487,7 @@ def _index_field(dim: Dimension, size=None):
self.horizontal_cell_index = _index_field(CellDim)
self.horizontal_edge_index = _index_field(EdgeDim)
self.w_tmp = as_field(
(CellDim, KDim), np.zeros((self.grid.num_cells, self.grid.num_levels + 1), dtype=float)
(CellDim, KDim), xp.zeros((self.grid.num_cells, self.grid.num_levels + 1), dtype=float)
)

def initial_run(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,11 +88,11 @@ class DiffusionInterpolationState:

@functools.cached_property
def geofac_n2s_c(self) -> Field[[CellDim], float]:
return as_field((CellDim,), data=self.geofac_n2s.asnumpy()[:, 0])
return as_field((CellDim,), data=self.geofac_n2s.ndarray[:, 0])

samkellerhals marked this conversation as resolved.
Show resolved Hide resolved
@functools.cached_property
def geofac_n2s_nbh(self) -> Field[[CECDim], float]:
geofac_nbh_ar = self.geofac_n2s.asnumpy()[:, 1:]
geofac_nbh_ar = self.geofac_n2s.ndarray[:, 1:]
old_shape = geofac_nbh_ar.shape
return as_field(
(CECDim,),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,20 @@
# SPDX-License-Identifier: GPL-3.0-or-later
from typing import Tuple

import numpy as np
from gt4py.next import as_field
from gt4py.next.common import Dimension, Field, GridType
from gt4py.next.ffront.decorator import field_operator, program
from gt4py.next.ffront.fbuiltins import broadcast, int32, minimum

from icon4py.model.common.dimension import CellDim, EdgeDim, KDim, VertexDim
from icon4py.model.common.math.smagorinsky import _en_smag_fac_for_zero_nshift
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend, xp


# TODO(Magdalena): fix duplication: duplicated from test testutils/utils.py
def zero_field(grid, *dims: Dimension, dtype=float):
shapex = tuple(map(lambda x: grid.size[x], dims))
return as_field(dims, np.zeros(shapex, dtype=dtype))
return as_field(dims, xp.zeros(shapex, dtype=dtype))


@field_operator
Expand Down Expand Up @@ -186,8 +185,8 @@ def init_nabla2_factor_in_upper_damping_zone(
physcial_heights: vector of physical heights [m] of the height levels
"""
# TODO(Magdalena): fix with as_offset in gt4py
heights = physical_heights.asnumpy()
buffer = np.zeros(k_size)
heights = physical_heights.ndarray
buffer = xp.zeros(k_size)
buffer[1 : nrdmax + 1] = (
1.0
/ 12.0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
_update_theta_and_exner,
)
from icon4py.model.common.dimension import CECDim, CEDim, CellDim, EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
)
from icon4py.model.atmosphere.diffusion.stencils.calculate_nabla4 import _calculate_nabla4
from icon4py.model.common.dimension import ECVDim, EdgeDim, KDim, VertexDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
_calculate_nabla2_for_w,
)
from icon4py.model.common.dimension import C2E2CODim, CellDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32

from icon4py.model.common.dimension import EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, broadcast, int32, maximum

from icon4py.model.common.dimension import EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
from gt4py.next.ffront.fbuiltins import Field, int32

from icon4py.model.common.dimension import EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32, neighbor_sum

from icon4py.model.common.dimension import C2E2CO, C2E2CODim, CellDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, broadcast, int32

from icon4py.model.common.dimension import CellDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
_temporary_fields_for_turbulence_diagnostics,
)
from icon4py.model.common.dimension import CEDim, CellDim, EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype

from icon4py.model.common.dimension import CellDim, KDim, Koff
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
_temporary_field_for_grid_point_cold_pools_enhancement,
)
from icon4py.model.common.dimension import CellDim, EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32, neighbor_sum

from icon4py.model.common.dimension import C2E2CO, C2E2CODim, CellDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32, maximum, minimum, sqrt

from icon4py.model.common.dimension import E2C2V, E2ECV, ECVDim, EdgeDim, KDim, VertexDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
_calculate_nabla2_of_theta,
)
from icon4py.model.common.dimension import CEDim, CellDim, EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32, neighbor_sum

from icon4py.model.common.dimension import C2E2CO, C2E2CODim, CellDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32

from icon4py.model.common.dimension import E2C, CellDim, EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32, neighbor_sum

from icon4py.model.common.dimension import C2CE, C2E, C2EDim, CEDim, CellDim, EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32

from icon4py.model.common.dimension import E2C2V, E2ECV, ECVDim, EdgeDim, KDim, VertexDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, int32, max_over, maximum

from icon4py.model.common.dimension import E2C, CellDim, E2CDim, EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32, neighbor_sum, where

from icon4py.model.common.dimension import C2E2C, C2E2CDim, CellDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32, neighbor_sum

from icon4py.model.common.dimension import C2CE, C2E, C2EDim, CEDim, CellDim, EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32, where

from icon4py.model.common.dimension import C2CEC, C2E2C, CECDim, CellDim, KDim, Koff
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32

from icon4py.model.common.dimension import CellDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
from icon4py.model.atmosphere.diffusion.diffusion_utils import scale_k
from icon4py.model.common.grid.horizontal import CellParams, EdgeParams
from icon4py.model.common.grid.vertical import VerticalModelParams
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.test_utils.datatest_utils import GLOBAL_EXPERIMENT, REGIONAL_EXPERIMENT
from icon4py.model.common.test_utils.helpers import dallclose
from icon4py.model.common.test_utils.reference_funcs import enhanced_smagorinski_factor_numpy
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
)
from icon4py.model.common.dimension import KDim, VertexDim
from icon4py.model.common.grid.simple import SimpleGrid
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.test_utils.helpers import random_field, zero_field

from .utils import construct_config, diff_multfac_vn_numpy, smag_limit_numpy
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, int32

from icon4py.model.common.dimension import EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32

from icon4py.model.common.dimension import CellDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32

from icon4py.model.common.dimension import EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
)

from icon4py.model.common.dimension import C2E2CO, C2E2CODim, CellDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
Koff,
VertexDim,
)
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from gt4py.next.ffront.fbuiltins import Field, astype, int32, neighbor_sum

from icon4py.model.common.dimension import C2CE, C2E, C2EDim, CEDim, CellDim, EdgeDim, KDim
from icon4py.model.common.model_backend import backend
from icon4py.model.common.settings import backend
from icon4py.model.common.type_alias import vpfloat, wpfloat


Expand Down
Loading
Loading