Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grid manager with backend #609

Merged
merged 8 commits into from
Nov 28, 2024
Merged

Grid manager with backend #609

merged 8 commits into from
Nov 28, 2024

Conversation

halungge
Copy link
Contributor

@halungge halungge commented Nov 27, 2024

use GT4Py backend in GridManager:

  • fields (coordinates geometry) are allocated using the backend
  • the existing on_gpu flag for the IconGrid construction is determined from the backend, for backwards compatibilty
  • GridManager internally explicitly uses numpy for reading

further changes and restrictions

  • test_io.py: uses a backend fixed to field view embedded as a sample CPU backend.
  • the IconGrid construction does not fully support the CUPY vs NUMPY import. It remains on CPU everywhere.

- allocate geometry and coordinates on specified backend
- determine on_gpu for icon_grid from backend
@halungge
Copy link
Contributor Author

cscs-ci run default

@halungge
Copy link
Contributor Author

cscs-ci run default

@halungge
Copy link
Contributor Author

cscs-ci run default

@halungge
Copy link
Contributor Author

cscs-ci run default

@halungge halungge marked this pull request as ready for review November 28, 2024 12:04
Comment on lines +25 to +28
try:
import cupy as xp
except ImportError:
import numpy as xp
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why we need this if we're using numpy afterwards

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the xp only for the typing. (There is this NDArray TypeAlias) Inside the grid construction I kept this flag on_gpu as it was before for compatibility, that seems to work, but I have not yet bent my head around how it works. Rather not touched the grid construction. It think the solution there will eventually be that we read (on CPU) and then copy to device probably right before we return the grid as for the other fields.

For this we will need to do the import based on the backend I think.

model/common/src/icon4py/model/common/grid/grid_manager.py Outdated Show resolved Hide resolved
model/common/tests/grid_tests/utils.py Show resolved Hide resolved
model/common/tests/io_tests/test_io.py Outdated Show resolved Hide resolved
Copy link

Mandatory Tests

Please make sure you run these tests via comment before you merge!

  • cscs-ci run default
  • launch jenkins spack

Optional Tests

To run benchmarks you can use:

  • cscs-ci run benchmark

To run tests and benchmarks with the DaCe backend you can use:

  • cscs-ci run dace

In case your change might affect downstream icon-exclaim, please consider running

  • launch jenkins icon

For more detailed information please look at CI in the EXCLAIM universe.

@halungge
Copy link
Contributor Author

cscs-ci run default

@halungge halungge merged commit 77cc179 into main Nov 28, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants