Dycore in mixed-precision #301

ninaburg · 2023-10-24T08:12:09Z

This PR enables to use the gt4py ICON dycore in mixed-precision. Parts of the variables are mixed e.g. single or double precision (typed in ICON as vpfloat = variable precision) which depends on whether the --enable-mixed-precision compilation option has been set and some are always in double precision (wpfloat = working precision). This mode has been transfered to the gt4py dycore which is currently all in double-precision.

The key changes include:

The addition of the --enable-mixed-precision option in the command-line interface (CLI) for compiling stencils, enabling mixed-precision computation.
Definition of type aliases wpfloat and vpfloat in common/type_alias.py to be imported in stencils, providing flexibility in specifying precision.
Adaptation of data types and explicit casting using astype in dycore stencils to allow mixed-precision computations.
Integration of pytest tests using the --enable-mixed-precision flag to ensure comprehensive testing in mixed-precision mode.
The introduction of a dtype argument in the random_field testing helpers function to control data types.
Modification of unit tests to accommodate and validate mixed-precision mode, ensuring robust testing for this new feature.

To test the stencils in mixed precision:

add the --enable-mixed-precision option to the pytest command : pytest -v --enable-mixed-precision

…ixed-precision

samkellerhals · 2023-10-25T13:22:55Z

cscs-ci run

samkellerhals

I did not look in detail at ICON and double-checked if the precision level of those variables correspond to the ones here, but other than that it looks good to me. I do think it would be useful to add a unit test to test this new option in icon4pygen though.

tools/src/icon4pytools/icon4pygen/cli.py

huppd

This has been a huge effort and is quite an achievement. Congrats. The code is very solid.
I only have some minor suggestions.

One recurring thing is that I would cast all double literals, which is right now not a problem. But might cause a problem once we want to run in all single precession.
I have two suggestion that I would check before merging if they have an impact on the validation.

I focused in the review on the field_operators specially on their casting of literals and intermediate results. I didnot look on the interfaces. The unittests I reviewed only very briefly.

...sion/src/icon4py/model/atmosphere/diffusion/stencils/calculate_diagnostics_for_turbulence.py

...icon4py/model/atmosphere/diffusion/stencils/calculate_nabla2_and_smag_coefficients_for_vn.py

model/atmosphere/dycore/src/icon4py/model/atmosphere/dycore/mo_solve_nonhydro_stencil_08.py

model/atmosphere/dycore/src/icon4py/model/atmosphere/dycore/mo_solve_nonhydro_stencil_13.py

...core/src/icon4py/model/atmosphere/dycore/mo_solve_nonhydro_stencil_16_fused_btraj_traj_o1.py

model/atmosphere/dycore/src/icon4py/model/atmosphere/dycore/mo_velocity_advection_stencil_20.py

model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_18.py

model/atmosphere/dycore/src/icon4py/model/atmosphere/dycore/mo_solve_nonhydro_stencil_39.py

Conflicts: model/atmosphere/diffusion/src/icon4py/model/atmosphere/diffusion/stencils/calculate_diagnostics_for_turbulence.py model/atmosphere/diffusion/src/icon4py/model/atmosphere/diffusion/stencils/calculate_nabla2_and_smag_coefficients_for_vn.py model/atmosphere/diffusion/src/icon4py/model/atmosphere/diffusion/stencils/calculate_nabla4.py model/atmosphere/diffusion/tests/stencil_tests/test_apply_nabla2_and_nabla4_global_to_vn.py model/atmosphere/diffusion/tests/stencil_tests/test_apply_nabla2_and_nabla4_to_vn.py model/atmosphere/diffusion/tests/stencil_tests/test_apply_nabla2_to_vn_in_lateral_boundary.py model/atmosphere/diffusion/tests/stencil_tests/test_apply_nabla2_to_w.py model/atmosphere/diffusion/tests/stencil_tests/test_apply_nabla2_to_w_in_upper_damping_layer.py model/atmosphere/diffusion/tests/stencil_tests/test_calculate_diagnostics_for_turbulence.py model/atmosphere/diffusion/tests/stencil_tests/test_calculate_horizontal_gradients_for_turbulence.py model/atmosphere/diffusion/tests/stencil_tests/test_calculate_nabla2_and_smag_coefficients_for_vn.py model/atmosphere/diffusion/tests/stencil_tests/test_calculate_nabla2_for_w.py model/atmosphere/diffusion/tests/stencil_tests/test_calculate_nabla2_for_z.py model/atmosphere/diffusion/tests/stencil_tests/test_calculate_nabla2_of_theta.py model/atmosphere/diffusion/tests/stencil_tests/test_calculate_nabla4.py model/atmosphere/diffusion/tests/stencil_tests/test_enhance_diffusion_coefficient_for_grid_point_cold_pools.py model/atmosphere/diffusion/tests/stencil_tests/test_temporary_field_for_grid_point_cold_pools_enhancement.py model/atmosphere/diffusion/tests/stencil_tests/test_temporary_fields_for_turbulence_diagnostics.py model/atmosphere/diffusion/tests/stencil_tests/test_truly_horizontal_diffusion_nabla_of_theta_over_steep_points.py model/atmosphere/diffusion/tests/stencil_tests/test_update_theta_and_exner.py model/atmosphere/dycore/src/icon4py/model/atmosphere/dycore/mo_solve_nonhydro_stencil_21.py model/atmosphere/dycore/src/icon4py/model/atmosphere/dycore/mo_solve_nonhydro_stencil_36.py model/atmosphere/dycore/src/icon4py/model/atmosphere/dycore/mo_solve_nonhydro_stencil_66.py model/atmosphere/dycore/src/icon4py/model/atmosphere/dycore/mo_velocity_advection_stencil_14.py model/atmosphere/dycore/src/icon4py/model/atmosphere/dycore/mo_velocity_advection_stencil_18.py model/atmosphere/dycore/src/icon4py/model/atmosphere/dycore/mo_velocity_advection_stencil_20.py model/atmosphere/dycore/tests/stencil_tests/test_compute_airmass.py model/atmosphere/dycore/tests/stencil_tests/test_fused_solve_nonhydro_stencil_39_40.py model/atmosphere/dycore/tests/stencil_tests/test_mo_icon_interpolation_scalar_cells2verts_scalar_ri_dsl.py model/atmosphere/dycore/tests/stencil_tests/test_mo_math_divrot_rot_vertex_ri_dsl.py model/atmosphere/dycore/tests/stencil_tests/test_mo_math_gradients_grad_green_gauss_cell_dsl.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_4th_order_divdamp.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_01.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_02.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_03.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_04.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_05.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_06.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_07.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_08.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_09.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_10.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_11_lower.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_11_upper.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_12.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_13.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_14.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_15.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_16_fused_btraj_traj_o1.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_17.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_18.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_19.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_20.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_21.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_22.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_23.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_24.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_25.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_26.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_27.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_28.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_29.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_30.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_31.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_32.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_33.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_34.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_35.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_36.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_37.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_38.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_39.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_40.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_41.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_42.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_43.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_44.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_45.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_46.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_47.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_48.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_49.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_50.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_52.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_53.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_54.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_55.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_56_63.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_57.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_58.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_59.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_60.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_61.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_62.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_64.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_65.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_66.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_67.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_68.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_01.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_02.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_03.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_04.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_05.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_06.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_07.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_08.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_09.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_10.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_11.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_12.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_13.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_14.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_15.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_16.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_17.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_18.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_19.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_20.py model/common/src/icon4py/model/common/test_utils/helpers.py model/common/tests/stencil_tests/test_mo_intp_rbf_rbf_vec_interpol_vertex.py

… mixed_precision_dycore Conflicts: model/atmosphere/diffusion/tests/stencil_tests/test_apply_nabla2_and_nabla4_global_to_vn.py model/atmosphere/diffusion/tests/stencil_tests/test_calculate_horizontal_gradients_for_turbulence.py model/atmosphere/diffusion/tests/stencil_tests/test_temporary_fields_for_turbulence_diagnostics.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_01.py

ninaburg · 2023-11-13T15:57:08Z

cscs-ci run

…ycore Conflicts: model/atmosphere/diffusion/tests/stencil_tests/test_calculate_nabla2_and_smag_coefficients_for_vn.py model/atmosphere/diffusion/tests/stencil_tests/test_calculate_nabla4.py model/atmosphere/dycore/tests/stencil_tests/test_mo_solve_nonhydro_stencil_16_fused_btraj_traj_o1.py model/atmosphere/dycore/tests/stencil_tests/test_mo_velocity_advection_stencil_19.py

ninaburg · 2023-11-17T09:25:14Z

cscs-ci run

ninaburg · 2023-11-17T10:47:50Z

cscs-ci run

ninaburg · 2023-11-17T10:52:04Z

cscs-ci run

ninaburg · 2023-11-17T11:50:15Z

cscs-ci run

ninaburg · 2023-11-17T11:50:29Z

launch jenkins spack

…ycore

ninaburg · 2023-11-20T08:43:59Z

launch jenkins spack

ninaburg · 2023-11-20T09:07:12Z

cscs-ci run

ninaburg · 2023-11-20T12:11:47Z

launch jenkins spack

ninaburg · 2023-11-20T13:48:21Z

cscs-ci run

ninaburg · 2023-11-20T13:48:31Z

launch jenkins spack

…ycore Conflicts: model/common/src/icon4py/model/common/test_utils/helpers.py

github-actions · 2023-11-24T12:46:37Z

Mandatory Tests

Please make sure you run these tests via comment before you merge!

cscs-ci run
launch jenkins spack

Optional Tests

In case your change might affect downstream icon-exclaim, please consider running

launch jenkins icon

For more detailed information please look at CI in the EXCLAIM universe.

ninaburg · 2023-11-24T12:57:31Z

cscs-ci run

ninaburg · 2023-11-24T12:57:42Z

launch jenkins spack

Nina Burgdorfer added 6 commits October 24, 2023 09:05

Add '--enable-mixed-precision' option in CLI to compile stencils in m…

a4aaff1

…ixed-precision

Add Type Alias (wpfloat, vpfloat) definition to be imported in stencils

6696bc7

Adapt types and casting in dycore stencils for mixed-precision

a7d925e

Add pytest '--enable-mixed-precision' to test in mixed-mode

20613d9

Add dtype argument to random_field

8ab5341

Adapt unit tests to test in mixed-precision mode

2678197

ninaburg force-pushed the mixed_precision_dycore branch from f3171c2 to f6bfb27 Compare October 24, 2023 08:45

Run pre-commit

331b7d7

ninaburg force-pushed the mixed_precision_dycore branch from f6bfb27 to 331b7d7 Compare October 24, 2023 13:29

samkellerhals approved these changes Oct 25, 2023

View reviewed changes

tools/src/icon4pytools/icon4pygen/cli.py Show resolved Hide resolved

samkellerhals self-requested a review October 25, 2023 13:34

samkellerhals approved these changes Oct 25, 2023

View reviewed changes

huppd reviewed Oct 27, 2023

View reviewed changes

Nina Burgdorfer added 4 commits October 30, 2023 12:01

Test --enable-mixed-precision option in icon4pygen

40dfd7b

Merge remote-tracking branch 'origin/main' into mixed_precision_dycore

0cec8af

Merge remote-tracking branch 'origin/main' into mixed_precision_dycore

6765232

Address requested changes

7ac0f83

huppd approved these changes Nov 6, 2023

View reviewed changes

Nina Burgdorfer added 4 commits November 13, 2023 16:04

pre-commit and fixes

270a445

ninaburg marked this pull request as ready for review November 17, 2023 09:23

Nina Burgdorfer added 2 commits November 17, 2023 14:20

Merge branch 'main' of github.com:C2SM/icon4py into mixed_precision_d…

144d1ae

…ycore

Fix order of tests in test_codegen

3196ed4

Fix test_mixed_precision with reload fct

2809583

Merge branch 'main' of github.com:C2SM/icon4py into mixed_precision_d…

364c2a6

…ycore Conflicts: model/common/src/icon4py/model/common/test_utils/helpers.py

ninaburg merged commit 2d2460a into main Nov 27, 2023
4 of 5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dycore in mixed-precision #301

Dycore in mixed-precision #301

ninaburg commented Oct 24, 2023 •

edited

Loading

samkellerhals commented Oct 25, 2023

samkellerhals left a comment

huppd left a comment •

edited

Loading

ninaburg commented Nov 13, 2023

ninaburg commented Nov 17, 2023

ninaburg commented Nov 17, 2023

ninaburg commented Nov 17, 2023

ninaburg commented Nov 17, 2023

ninaburg commented Nov 17, 2023

ninaburg commented Nov 20, 2023

ninaburg commented Nov 20, 2023

ninaburg commented Nov 20, 2023

ninaburg commented Nov 20, 2023

ninaburg commented Nov 20, 2023

github-actions bot commented Nov 24, 2023

ninaburg commented Nov 24, 2023

ninaburg commented Nov 24, 2023

Dycore in mixed-precision #301

Dycore in mixed-precision #301

Conversation

ninaburg commented Oct 24, 2023 • edited Loading

samkellerhals commented Oct 25, 2023

samkellerhals left a comment

Choose a reason for hiding this comment

huppd left a comment • edited Loading

Choose a reason for hiding this comment

ninaburg commented Nov 13, 2023

ninaburg commented Nov 17, 2023

ninaburg commented Nov 17, 2023

ninaburg commented Nov 17, 2023

ninaburg commented Nov 17, 2023

ninaburg commented Nov 17, 2023

ninaburg commented Nov 20, 2023

ninaburg commented Nov 20, 2023

ninaburg commented Nov 20, 2023

ninaburg commented Nov 20, 2023

ninaburg commented Nov 20, 2023

github-actions bot commented Nov 24, 2023

ninaburg commented Nov 24, 2023

ninaburg commented Nov 24, 2023

ninaburg commented Oct 24, 2023 •

edited

Loading

huppd left a comment •

edited

Loading