Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Nexus Writer for t-route formatted QLAT outputs #612

Closed
wants to merge 9 commits into from

Conversation

program--
Copy link
Contributor

@program-- program-- commented Aug 15, 2023

This PR implements the ngen::utils::nexus_writer base class with ngen::utils::nexus_csv_writer and ngen::utils::nexus_netcdf_writer derived classes. This implements part 1 of #605.

See NOAA-OWP/t-route#636 for the output schema. Output file names are formatted as: {%Y%m%d%H%M}NEXOUT.{nc|csv}, based on t-route's parsing logic.

Usage

The following modifications should be made before using this:

  • Before running ngen, export the NGEN_QLAT_FORMAT environment variable to be either netcdf or csv.
  • in your routing config file, ensure forcing_parameters.qlat_file_pattern_filter is set to *NEXOUT.csv or *NEXOUT.nc, depending on the format you output.
  • Optional, but recommended: set output_root in your realization config so that the outputs are aggregated under a single directory.
  • Ensure your forcing_parameters.qlat_input_folder points to the same location as your realization config's output_root, if set.

Additions

  • Adds ngen::utils::nexus_writer
  • Adds ngen::utils::nexus_csv_writer
  • Adds ngen::utils::nexus_netcdf_writer
  • Adds environment variable checking for user to choose QLAT output. For example, setting NGEN_QLAT_FORMAT=netcdf will output NetCDF files, or otherwise CSV files.

    This is a temporary workaround until a decision is made on whether this should be set in the realization configuration files.

Changes

  • Modifies CSV outputs to conform to NetCDF format.
  • Modifies src/NGen.cpp to include the logic for using ngen::utils::nexus_writer.
  • Modifies src/NGen.cpp to prevent destruction of nexus_collection, since the flowpath/segment IDs are needed for method calls to nexus_writer.

Notes

  • If NetCDF is not available at compile-time, but NGEN_QLAT_FORMAT is set to "NetCDF" at run-time, then a std::runtime_error will be thrown when any of nexus_netcdf_writer's methods are called.

Testing

  • Adds unit tests for ensuring output formats are correct by parsing temporary outputs for both CSV and NetCDF.

Todos

  • Unit Testing for Nexus Writers
  • Corresponding t-route PR (waiting on inland hydraulics)
  • MPI handling (?)

Checklist

  • PR has an informative and human-readable title
  • Changes are limited to a single goal (no scope creep)
  • Code can be automatically merged (no conflicts)
  • Code follows project standards (link if applicable)
  • Passes all existing automated tests
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future todos are captured in comments
  • Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
  • Reviewers requested with the Reviewers tool ➡️

Testing checklist (automated report can be put here)

Target Environment support

  • Linux


//Still hacking nexus output for the moment
/**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah.. can we just delete this block now?

@program-- program-- mentioned this pull request Jan 24, 2024
@program-- program-- linked an issue Feb 9, 2024 that may be closed by this pull request
@program--
Copy link
Contributor Author

Probably replaced by #744

@program--
Copy link
Contributor Author

@hellkite500 @donaldwj does it seem reasonable to close this?

@program-- program-- closed this Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Get ngen to output QLATs in a format t-route expects
2 participants