Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi level hydrofabric #615

Merged
merged 55 commits into from
Sep 28, 2023
Merged

Multi level hydrofabric #615

merged 55 commits into from
Sep 28, 2023

Conversation

donaldwj
Copy link
Contributor

@donaldwj donaldwj commented Aug 18, 2023

[Short description explaining the high-level reason for the pull request]

Additions

  • Support for multiple processing levels
  • Support for separate models runtimes for each level

Removals

  • Old time loop has been replaced

Changes

  • Added Layer Objects to Ngen driver
  • Layer objects now manage model realizations

Testing

Manual Test of 5 catchment data set

Screenshots

Notes

Todos

Checklist

  • PR has an informative and human-readable title
  • Changes are limited to a single goal (no scope creep)
  • Code can be automatically merged (no conflicts)
  • Code follows project standards (link if applicable)
  • Passes all existing automated tests
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future todos are captured in comments
  • Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
  • Reviewers requested with the Reviewers tool ➡️

Testing checklist (automated report can be put here)

Target Environment support

  • Linux

//m/timestep
//since we are operating on a 1 hour (3600s) dt, we need to scale the output appropriately
//so no response is m^2/hr...m^2/hr * 1hr/3600s = m^3/hr
response /= 3600.0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, there was no great place to have a timestep variable before... but now we have a Simulation_Time object with an output_interval... even if we assume that the output is m/h (not m/timestep!), shouldn't this response be divided by the number of timesteps per hour if the timestep is not = 1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was fixed in my branch

@mattw-nws mattw-nws mentioned this pull request Aug 21, 2023
11 tasks
auto r = features.catchment_at(id);
//TODO redesign to avoid this cast
auto r_c = std::dynamic_pointer_cast<realization::Catchment_Formulation>(r);
double response = r_c->get_response(output_time_index, description.time_step);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will description.time_step and simulation_time.get_output_interval_seconds() ever differ? Are they redundant or is there a current/potential future semantic difference?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually since it looks like description has a time_step_units, it's likely that description.time_step should not be used here and simulation_time.get_output_interval_seconds() should be used instead, or that description.time_step should be put through a unit conversion first (probably the former).

@mattw-nws mattw-nws force-pushed the multi_level_hydrofabric branch from 8aa1277 to 7a0d40a Compare August 23, 2023 18:33
include/core/Layer.hpp Outdated Show resolved Hide resolved
@mattw-nws mattw-nws force-pushed the multi_level_hydrofabric branch 2 times, most recently from 34e95c0 to fc67105 Compare August 23, 2023 21:13
This was referenced Aug 24, 2023
.gitignore Outdated
@@ -97,7 +97,6 @@ CMakeCache.txt
/cmake-build-debug
/cmake_build
/build
/cmake_build*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR should leave .gitignore alone - it's out of scope.

include/core/Layer.hpp Outdated Show resolved Hide resolved
include/core/Layer.hpp Outdated Show resolved Hide resolved
@mattw-nws mattw-nws changed the title Multi level hydrofabric batch Multi level hydrofabric Aug 25, 2023
// layers with small time steps to be updated more than once
// If a layer with a large time step is after a layer with a small time step the
// layer with the large time step will wait for multiple timesteps from the preceeding
// layer.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we're not (currently) doing temporal interpolation, should we be asserting that the various time step increments are all (close enough to) integer ratios, so that we're never left with a layer/model at an unaligned time?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting... maybe, but not necessarily...uninterpolated sampling is still a method of sampling. If the value is a rate (which we highly encourage and I think the formulations team is now favoring and moving their code to as a rule) then this doensn't matter much ...for most common cases. Rapid fluctuations or large timestep differences could lead to significant differences in results, but I don't know if we'd want to fail if they don't line up in many simpler "okay" cases. A warning might be worth considering.

@mattw-nws
Copy link
Contributor

@donaldwj Do you have any uncommitted/unpushed updates to this PR? I'm going to rebase on master again to make sure things work and clean up a few of the minor issues.

@mattw-nws mattw-nws force-pushed the multi_level_hydrofabric branch from 110de1d to 68a53bf Compare September 19, 2023 16:05
@mattw-nws mattw-nws force-pushed the multi_level_hydrofabric branch from 68a53bf to f4df6d2 Compare September 19, 2023 16:21

// get the catchment level from the hydro fabric
const auto& cat_json_node = fabric->get_feature(feat_id);
long lv = cat_json_node->has_key("level") ? cat_json_node->get_property("level").as_natural_number() : 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going through and unifying "level" to "layer" for consistency... but this appears it would actually have a functional impact, because the provided /data/catchment_data_multilayer.geojson file actually does have "layer" keys instead of "level"... of course the test code was previously looking for a different example file which was not committed, so I don't know what was in that one. @donaldwj can you look at this please? I will test some with this but am not as confident I'll figure out the impact.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After changing this, it also passes the included test with the example file that has "layer"... for better and worse.


// get the catchment level from the hydro fabric
const auto& cat_json_node = linked_hydro_fabric->get_feature(feat_id);
long lv = cat_json_node->has_key("level") ? cat_json_node->get_property("level").as_natural_number() : 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto for comment in non-MPI version of this file, @donaldwj

{
if ( feature->has_property("level") )
{
const auto& prop = feature->get_property("level");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This again doesn't match the data/catchment_data_multilayer.geojson file @donaldwj

Copy link
Member

@hellkite500 hellkite500 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to bite the bullet and merge this, we can come back and address various aspects as needed.

@hellkite500 hellkite500 merged commit a472cab into master Sep 28, 2023
@hellkite500 hellkite500 deleted the multi_level_hydrofabric branch September 28, 2023 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants