Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to support read/write of additional INP/RPT sections #219

Merged
merged 12 commits into from
Apr 21, 2024

Conversation

kaklise
Copy link
Contributor

@kaklise kaklise commented Apr 11, 2024

The following PR adds support for the PATTERNS and CONTROLS sections of the INP file along with minor updates to support INFLOWS, XSECTIONS, and CURVES. The Pumping Summary was also added to the list of supported RPT sections.

I included a test that create a swmmio model, reassign each inp section to force the use of "replace_inp_section", save an INP file, and then uses that file to run SWMM. I ran this with a few swmmio models, INP files that come with SWMM, and some utility models I'm working with.

Thanks again for creating this great package! I'm happy to iterate on changes.

PATTERNS
In core.py, I added a patterns property and setter, this reads the PATTERN section as a dataframe (using dataframe_from_inp) and then converts it to have 1 row per entry. Columns are named Factor*. The pattern is written to an INP file as a single row per entry. I did not test this with models that have a combination of patterns types (hourly, daily, etc.). Example patterns dataframe below.

        Type  Factor1  ...  Factor23  Factor24
Name                   ...                    
DWF   HOURLY   0.0121  ...   0.02789   0.02254

CONTROLS
In core.py, I added a controls property and setter, this reads the CONTROLS section as a dataframe (using dataframe_from_inp) and then converts it to have 1 row per entry. The controI is written to an INP file as multiple lines separated using keywords IF, THEN, PRIORITY, AND, OR, ELSE (in utils.py). Example controls dataframe below.

                                                                   Control
Name                                                                       
RULE PUMP1A  IF NODE SU1 DEPTH >= 4 THEN PUMP PUMP1 status = ON PRIORITY 1 
RULE PUMP1B  IF NODE SU1 DEPTH < 1 THEN PUMP PUMP1 status = OFF PRIORITY 1 

INFLOWS
In core.py, I replace "_!!!!_" with “” instead of NaN to keep the placeholder in the INP file

XSECTIONS
In dataframes.py, I find the max number of tokens (column names) and name extras as “col”+i. This helps read in xsections that have a mixed number of entries.

CURVES
In dataframes.py, I set index to just Name instead of (Name, Type)

RPT file sections
In section_headers.yml, I added "Pumping Summary" and column names
In dataframe_from_rpt, I added end_string “Analysis begun on” to read the final section


Closes #217
Makes progress on #57

@bemcdonnell
Copy link
Member

@kaklise I added you to the @pyswmm/swmmio_external team which, I believe, will grant you the ability to run the github checks without one of us approving!

@aerispaha
Copy link
Member

aerispaha commented Apr 13, 2024

@kaklise thank you for these great contributions! I'm in the process of reviewing this PR and will the rest of my comments prepared asap.

First, I want to do a bit of house keeping and connect this PR to existing issues. As as I understand it so far, this PR will

Please let me know if there are other issues that this PR may address that I missed.

Next, I think your idea to handle the PATTERNS and CONTROLS section by reformatting them to one row per entry is very clever. I would, however, like to have these new features tested. To that end, can you please provide an example INP file with patterns and controls sections that we can use for testing? With that INP file, I suggest that we create a new example model in swmmio.examples and leverage this in doc tests, like we do here:

swmmio/swmmio/core.py

Lines 816 to 829 in 24673b5

@property
def losses(self):
"""
get/set losses section of model
:return: dataframe of evaporation section in inp file
>>> from swmmio.examples import spruce
>>> spruce.inp.losses #doctest: +NORMALIZE_WHITESPACE
Inlet Outlet Average Flap Gate SeepageRate
Link
C1:C2 0 0 0 YES 0
C2.1 0 0 0 YES 0
"""

This will provide some basic documentation like this, in addition to testing the source code.

Thanks again for contributing - I can't wait to release these new features 😄

@kaklise
Copy link
Contributor Author

kaklise commented Apr 19, 2024

@aerispaha I added the Pump_Control_Model.inp from SWMM to swmmio.examples. The model includes patterns and controls. I also updated the documentation to illustrate the format. Let me know if additional updates are needed!

Copy link
Member

@aerispaha aerispaha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaklise I love these contributions - thank you! I've got a couple additional comments, but nothing that will prevent me from approving this PR.

Also, I've tested inp.patterns on a INP file that has multiple pattern types and it works fine. For patterns with less than 24 "Factors", the unused Factors are returned as NaNs in the dataframe; I think this behavior is acceptable.

I'll merge these changes, add you to the AUTHORS, then work on releasing ASAP 😄


>>> from swmmio.examples import pump_control
>>> # NOTE, only the first 5 columns are shown in the following example
>>> pump_control.inp.patterns.iloc[:,0:5] #doctest: +NORMALIZE_WHITESPACE
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love this - thanks for following our documentation pattern.

control_entry = {}
controls = self._controls_df['[CONTROLS]']
# make sure the first entry starts with RULE
assert controls[0][0:5] == "RULE "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would best to raise an exception here if this assertion fails. But I'm fine with capturing that in a future PR.

@@ -258,3 +262,71 @@ def test_polygons(test_model_02):
assert poly1.equals(test_model_02.inp.polygons)

# print()

def test_inp_sections():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like this test. My only suggestion is that we clean up the artifacts that are created after this test is run. That said, I'm fine with leaving this as is and refactoring our unit tests in a future PR.

@aerispaha aerispaha merged commit 9c885cb into pyswmm:master Apr 21, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Updates to support read/write of additional INP sections
3 participants