Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate allocation input #972

Merged
merged 7 commits into from
Feb 1, 2024
Merged

Validate allocation input #972

merged 7 commits into from
Feb 1, 2024

Conversation

Jingru923
Copy link
Contributor

@Jingru923 Jingru923 commented Jan 19, 2024

Fixes #710

In the validation.jl I created three functions to address validation for the positive allocation id, non-negative use demand, and for complete subnetwork. Unit tests are made for the three functions. The second task in issue #710 was skipped because it depends on main_network_allocation_main branch which is not yet finished.

@Jingru923 Jingru923 linked an issue Jan 19, 2024 that may be closed by this pull request
4 tasks
Copy link
Collaborator

@SouthEndMusic SouthEndMusic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some suggestions. I was also thinking about how to organize the test, because triggering different validation errors with the same model might be complicated due to the structure of the initialization code. Here are my thoughts per task of the issue:

  • For this one it is probably best to create an invalid test model in Ribasim Python as you already started making.
  • For this one I was a bit naughty, as I wrote the mentioned source validation function without testing it. Maybe you can skip this one for now, as I changed this validation function in the branch I am working on.
  • For this one it might be educational to create an instance of a User object within the test, just as is done for instance here for Basin. Also, you said you created an if statement in the inner constructor, but function User is not an inner constructor. Doing validation in an inner constructor looks something like this:
struct Circle
   r::Float64
   function Circle(r::Float64)
      if r < 0
         error("Circle radius must be non-negative, got $r.")
      end
      return new(r)
   end
end

We the same in our own code for instance here.

  • The last one is probably also best done with a model made in Ribasim python. Creating a subnetwork is quite easy: you only have to add a column to the Node table called allocation_network_id, see also here in the docs.

Also a small nitpick:

  • In the description of the PR you put the number of this PR instead of the number of the issue (710).

@@ -695,6 +708,10 @@ function User(db::DB, config::Config)::User
errors = true
end

if !all(all.(>(0), eachcol(interpolations)))
Copy link
Collaborator

@SouthEndMusic SouthEndMusic Jan 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might work, but its quite hard to read. I propose you write a nested loop:

for (user_node_id, demand_user) in zip(node_ids, interpolations)
   for (itp, priority) in zip(demand_user, priorities)
     ...
   end
end

This way you have the right information available to write a helpful error message, and you can use the same structure as above to check all data before crashing the program if a problem was found.

@@ -695,6 +708,10 @@ function User(db::DB, config::Config)::User
errors = true
end

if !all(all.(>(0), eachcol(interpolations)))
error("Demand should be a non-negative number")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick:

Suggested change
error("Demand should be a non-negative number")
error("Demand should be a non-negative float")

@SouthEndMusic
Copy link
Collaborator

I also saw that this docstring is not correct, so you can fix my mistake (:

# The highest priority number given, which corresponds to the least important demands

It should be something like # All provided priorities.

On a related note, we use a VSCode extension called Git Blame, which tells you at the bottom of your screen who wrote a line of code you don't like if you put your cursor on it 😄

Copy link

Quality Gate Failed Quality Gate failed

Failed conditions

66.3% Duplication on New Code (required ≤ 3%)

See analysis details on SonarCloud

core/src/create.jl Outdated Show resolved Hide resolved
core/src/utils.jl Outdated Show resolved Hide resolved
@@ -0,0 +1,46 @@
@testitem "Non-positive allocation network ID" begin end

@testitem "Incomplete subnetwork" begin
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding this test: It is good practice to create your own test model to do this, but I thought of an easier way: load the parameters of an existing model, say

toml_path = normpath(@__DIR__, "../../generated_testmodels/allocation_example/ribasim.toml")
p = Ribasim.Model(toml_path).integrator.p

remove an edge to make it invalid and run your validation test on that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This morning I tried to load an existing model and change one of the allocation id to negative number. It gave error saying the immutable struct of type cannot be changed. I remembered you mentioned this yesterday, it probably can't be changed or maybe I didn't do it the correct way. Anyhow, in my new code I created a graph that has the required feature and used it as the test model.

@Jingru923 Jingru923 marked this pull request as ready for review January 31, 2024 14:47
@Hofer-Julian
Copy link
Contributor

Looks to me like merging main into your branch went wrong.
This needs to be fixed up before we can review this :)

@Jingru923 Jingru923 force-pushed the validate-allocation-input branch 2 times, most recently from 38a42ed to e9e099b Compare February 1, 2024 13:01
added validation for positive allocation id and for non-negative user demand

Fix the bug in program, the last second validation function is working

Removed commented code

Validation of the second last input is now done inside inner constructor. Code of validation is now a function in validation.jl

made an draft test case, which is unfinished

fixed the bug in validation.jl that 'ScalarInterpolation' can't be recognized

add function to check the connection of the subnetworks

the code for checking subnetwork connection is now bug-free

made unit test for negative demand

Created "create_test.jl" and wrote part of the incomplete subnetwork test

created unit test for non-positive allocation id validating function and test allocation id of 0 and -1

removed unnecessary Ribasim Python model

finished the unit test for the subnetwork connection validation function

Resolved Bart's comment on PR
@Jingru923 Jingru923 force-pushed the validate-allocation-input branch from e9e099b to 764f23d Compare February 1, 2024 13:02
core/src/create.jl Outdated Show resolved Hide resolved
Copy link
Collaborator

@SouthEndMusic SouthEndMusic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I added a few small comments.

core/src/create.jl Outdated Show resolved Hide resolved
core/src/validation.jl Outdated Show resolved Hide resolved
core/src/validation.jl Outdated Show resolved Hide resolved
core/src/validation.jl Outdated Show resolved Hide resolved
core/test/create_test.jl Outdated Show resolved Hide resolved
core/test/create_test.jl Outdated Show resolved Hide resolved
core/test/create_test.jl Outdated Show resolved Hide resolved
@Jingru923 Jingru923 merged commit 70b2348 into main Feb 1, 2024
18 of 19 checks passed
@Jingru923 Jingru923 deleted the validate-allocation-input branch February 1, 2024 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Validation of input for allocation
3 participants