Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timeloop #308

Merged
merged 30 commits into from
Nov 30, 2023
Merged

Timeloop #308

merged 30 commits into from
Nov 30, 2023

Conversation

OngChia
Copy link
Contributor

@OngChia OngChia commented Nov 7, 2023

Constructed a simple dycore driver that performs time integration (a timeloop) with diffusion and solve_nonhydro components in reference to mo_nh_stepping.f90. Tracer advection and IO are not included. This will make the path for Jablonowski-Williamson test in the next phase by adding analytic initial conditions and a simple IO infrastructure.

@OngChia
Copy link
Contributor Author

OngChia commented Nov 8, 2023

cscs-ci run

@OngChia
Copy link
Contributor Author

OngChia commented Nov 8, 2023

launch jenkins spack

@OngChia OngChia requested a review from halungge November 8, 2023 11:07
@OngChia
Copy link
Contributor Author

OngChia commented Nov 9, 2023

cscs-ci run

@OngChia
Copy link
Contributor Author

OngChia commented Nov 9, 2023

launch jenkins spack

Copy link
Contributor

@halungge halungge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good. Some small changes, and a lot of stuff, that we hope to fix with the cleanup project in this cycle (like SolveNonHyrdo interface...)

we should also have a discussion on how to really setup the time loop to make it run as linear as possible, but we can refactor it to that in a second round.

ndyn_substeps: int = (
5 # ndyn_substeps is not a constant in ICON, it may be modified in restart runs
)
apply_horizontal_diff_at_large_dt: bool = True # lhdiff_rcf
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that you get rid of these ununderstandable abreviations, the diff stands for diffusion, so why not also write this in full? You could also add a docstring or comment docstring for the old icon name:

Suggested change
apply_horizontal_diff_at_large_dt: bool = True # lhdiff_rcf
apply_horizontal_diffusion_at_large_dt: bool = True #: lhdiff_rcf
Suggested change
apply_horizontal_diff_at_large_dt: bool = True # lhdiff_rcf
apply_horizontal_diffusion_at_large_dt: bool = True
"""lhdiff_rcf in ICON."""

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I have replaced them with full name.

5 # ndyn_substeps is not a constant in ICON, it may be modified in restart runs
)
apply_horizontal_diff_at_large_dt: bool = True # lhdiff_rcf
apply_extra_diffusion_for_largeCFL: bool = True # lextra_diffu
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here for the comment

Suggested change
apply_extra_diffusion_for_largeCFL: bool = True # lextra_diffu
apply_extra_diffusion_for_largeCFL: bool = True #: lextra_diffu

@@ -568,17 +568,17 @@ def run_predictor_step(
else:
lvn_only = False

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still believe this if-condition is very obfuscated. (I know you did not change it...) but I always wondered what is the value of the lvn_only if l_init==False and l_recompute == False? I think it is not defined, and that is probably just ok because for some reason it never happens.

Furthermore isn't this the same:

Suggested change
if l_init:
lvn_only = False
else if l_recompute and self.config.itime_scheme == 4:
lvn_only = True

Copy link
Contributor Author

@OngChia OngChia Nov 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my understanding, the outer if (l_init or l_recompute)) determines whether velocity_advection.run_predictor_step will be run. And lvn_only is only needed in velocity_advection.run_predictor_step. So, when l_init==False and l_recompute==False, the value of lvn_only does not matter. I remember Anurag said that we will possibly switch to itime_scheme=6 in the future and we can completely remove this weird lvn_only. I suggest we just leave it as the current state as a temporary solution.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, still don't like it. The condition is just very hard to read.

sp = timeloop_nonhydro_savepoint_init
nonhydro_params = NonHydrostaticParams(nonhydro_config)
sp_v = timeloop_velocity_savepoint_init
grid = SimpleGrid()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can already delete this (SimpleGrid). All the other stuff, we will slowly improve over this cycle with the cleanup project, I hope

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay. I will now change it to ICON grid and we can remove them in the future after cleaning up.

@@ -78,116 +85,241 @@ def _copy_diagnostic_and_prognostics(
_identity_c_k(theta_v_new, out=theta_v)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are not calling that _copy_diagnostic_and_prognostic any more and indeed there should be no need anymore since you call the time integration. Can you delete it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.


self._l_init: bool = apply_initial_stabilization

self._now: int = 0 # TODO: move to PrognosticState
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I think it would be nice to keep the internals of the time - swapping somehow in the prognostic state, as we discussed. Latest, when not all fields inside the state change at the same time from now -> next, we should reconsider it.

model/driver/src/icon4py/model/driver/dycore_driver.py Outdated Show resolved Hide resolved
@OngChia
Copy link
Contributor Author

OngChia commented Nov 17, 2023

cscs-ci run

…ion was wrongly set to true after the review
@OngChia
Copy link
Contributor Author

OngChia commented Nov 17, 2023

cscs-ci run

@OngChia
Copy link
Contributor Author

OngChia commented Nov 17, 2023

launch jenkins spack

@OngChia
Copy link
Contributor Author

OngChia commented Nov 20, 2023

cscs-ci run

@OngChia
Copy link
Contributor Author

OngChia commented Nov 20, 2023

launch jenkins spack

Copy link
Contributor

@halungge halungge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now really only some small changes left...

@@ -568,17 +568,17 @@ def run_predictor_step(
else:
lvn_only = False

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, still don't like it. The condition is just very hard to read.

@@ -381,11 +385,13 @@ def test_nonhydro_predictor_step(
)

# stencil 35
# TODO: first few levels are not computed in the test, please add a bound
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am always a bit confused by comments like that? does it run or not? Why not directly add the bound

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed that. I have added the bound accordingly. I also added a bound for CellDim w_concorr_c (stencil 39, 40) and z_flxdiv_theta (stencil 41) as I founded that those stencils are executed with these bounds. Please also have a look.

else:
(model_run_config, diffusion_config, dycore_config) = _default_config(n_time_steps)
log.info("Warning: Experiment name is not specified, default configuration is used.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use a real warning instead?

Suggested change
log.info("Warning: Experiment name is not specified, default configuration is used.")
log.warning("Experiment name is not specified, default configuration is used.")

from icon4py.model.driver.icon_configuration import IconRunConfig


@pytest.fixture
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add #TODO here that should reuse the code from the diffusion tests? I have restructured this in order to allow configuration for the Aquaplanet experiment, maybe that will make it easier once we bring this together.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added a #TODO statement to reuse those pytest fixtures in the diffusion test.

self._substep_timestep: float = float(self.run_config.dtime / self._n_substeps_var)

# check validity of configurations
self._validate()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you rename this function to _validate_config then you don't need the extra comment.

for t in range(self.config.n_time_steps):
log.info(f"run timestep : {t}")
timer = Timer(self._full_name(self._integrate_one_time_step))
for i_time_step in range(self._n_time_steps):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not just call it time_step this is some very akward old Fortran habit to call stuff ixxx and lxx to indicate the type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion. I have removed the i in i_time_step. I also changed the idyn_timestep to dyn_substep in substepping.


# TODO (Chia Rui): compute airmass for prognostic_state here

l_recompute = True
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here for the variables: why not just call them recompute, clean_mflx, dyn_timestep etc. If you want to indicate that it is a flag then I would rather choose do_recompute because

if do_recompute: 
   foo()

reads better than

if l_recompute:
   foo()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion. I have systematically changed them to do_recompute, do_clean_mflx, and do_prep_adv which clearly shows their purposes.

@halungge
Copy link
Contributor

Can you update the readme?

Copy link
Contributor

@halungge halungge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You will have to update with main and replace the np.asarray(field) in your tests with field.asnumpy().

…step in solve_nonhydro test_run_solve_nonhydro_single_step and test_run_solve_nonhydro_multi_step, update README.md in driver.
Copy link

Mandatory Tests

Please make sure you run these tests via comment before you merge!

  • cscs-ci run
  • launch jenkins spack

Optional Tests

In case your change might affect downstream icon-exclaim, please consider running

  • launch jenkins icon

For more detailed information please look at CI in the EXCLAIM universe.

@OngChia
Copy link
Contributor Author

OngChia commented Nov 29, 2023

I have updated the README file and my test for the dycore driver with regards to the recent update of GT4Py and main branch. I also added a second time step test for the test_run_non_hydro_single_step and test_run_non_hydro_multi_step which I think it is necessary because, for instance, vn_only is different and some stencils are not called when it is true.

@OngChia
Copy link
Contributor Author

OngChia commented Nov 29, 2023

cscs-ci run

@OngChia
Copy link
Contributor Author

OngChia commented Nov 29, 2023

launch jenkins spack

@OngChia
Copy link
Contributor Author

OngChia commented Nov 29, 2023

cscs-ci run

@OngChia
Copy link
Contributor Author

OngChia commented Nov 30, 2023

launch jenkins spack

@OngChia OngChia merged commit c1a5ad7 into main Nov 30, 2023
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants