Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Netcdf output #744

Draft
wants to merge 37 commits into
base: master
Choose a base branch
from
Draft

Netcdf output #744

wants to merge 37 commits into from

Conversation

donaldwj
Copy link
Contributor

This pull request adds classes for support of netcdf output and initial tests of the writer code.

Additions

Netcdf writer class
Netcdf writer tests

Removals

None

Changes

None

Testing

  1. Git Hub tests
  2. New unit tests added

Screenshots

Notes

Todos

Add model framework integration.

Checklist

  • PR has an informative and human-readable title
  • Changes are limited to a single goal (no scope creep)
  • Code can be automatically merged (no conflicts)
  • Code follows project standards (link if applicable)
  • Passes all existing automated tests
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future todos are captured in comments
  • Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
  • Reviewers requested with the Reviewers tool ➡️

Testing checklist (automated report can be put here)

Target Environment support

  • Linux

@donaldwj donaldwj requested a review from hellkite500 February 22, 2024 20:31
@robertbartel
Copy link
Contributor

robertbartel commented Feb 23, 2024

Closes #714.

@robertbartel robertbartel linked an issue Feb 23, 2024 that may be closed by this pull request
include/core/Layer.hpp Outdated Show resolved Hide resolved
@@ -5,6 +5,7 @@
- run_bmi_unit_test.py: This is a file that runs each BMI unit test to make sure that the BMI is complete and functioning as expected.
- config.yml: This is a configuration file that the BMI reads to set inital_time (initial value of current_model_time) and time_step_seconds (time_step_size).
- environment.yml: Environment file with the required Python libraries needed to run the model with BMI. Create the environment with this command: `conda env create -f environment.yml`, then activate it with `conda activate bmi_test`
- requirments.txt: This file is used to install needed packages into system or venv directories using the command 'pip install -r requirements.txt'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- requirments.txt: This file is used to install needed packages into system or venv directories using the command 'pip install -r requirements.txt'
- requirements.txt: This file is used to install needed packages into system or venv directories using the command 'pip install -r requirements.txt'

Comment on lines 93 to 98
/**
* @brief Get a list of output variables names for this layer
*
* @return vector of output variable names
*/

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you __ a function here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that was a function that was removed as redundant.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The actual information comes from the BMI realizations of the contained models, it would be nice for the layer to have a summary but currently it is not needed to get things to work. This could be used to limit what variables are output.

@PhilMiller
Copy link
Contributor

It looks like there's a signature mismatch between Layer::update_models(unsigned long, shared_ptr<...>) and the derived class overrides thereof. In all cases, I'm not a fan of the "update" member function also doing any output at all. I think it would be preferable to have a separate member entirely for getting the outputs, either one variable at a time, or in a big lump.

There's also some efficiency tradeoffs on offer between the simplicity of making the Layer output be just "copy everything necessary into an mdframe instance, and then pass that to a file writer, and being able to eventually use Bmi::GetValuePtr() where available to pass each model's buffer straight down to the code doing IO.

@donaldwj
Copy link
Contributor Author

The information in any one bmi model can not be written to netcdf efficiently. The code for doing this could be move to a different method but it needs to happen. We need to write to each netcdf variable once a time step, not 20,000 times in the case of CONUS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Netcdf Output
3 participants