-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updates to support read/write of additional INP/RPT sections #219
Conversation
@kaklise I added you to the @pyswmm/swmmio_external team which, I believe, will grant you the ability to run the github checks without one of us approving! |
@kaklise thank you for these great contributions! I'm in the process of reviewing this PR and will the rest of my comments prepared asap. First, I want to do a bit of house keeping and connect this PR to existing issues. As as I understand it so far, this PR will
Please let me know if there are other issues that this PR may address that I missed. Next, I think your idea to handle the PATTERNS and CONTROLS section by reformatting them to one row per entry is very clever. I would, however, like to have these new features tested. To that end, can you please provide an example INP file with patterns and controls sections that we can use for testing? With that INP file, I suggest that we create a new example model in swmmio.examples and leverage this in doc tests, like we do here: Lines 816 to 829 in 24673b5
This will provide some basic documentation like this, in addition to testing the source code. Thanks again for contributing - I can't wait to release these new features 😄 |
@aerispaha I added the Pump_Control_Model.inp from SWMM to swmmio.examples. The model includes patterns and controls. I also updated the documentation to illustrate the format. Let me know if additional updates are needed! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kaklise I love these contributions - thank you! I've got a couple additional comments, but nothing that will prevent me from approving this PR.
Also, I've tested inp.patterns
on a INP file that has multiple pattern types and it works fine. For patterns with less than 24 "Factors", the unused Factors are returned as NaNs in the dataframe; I think this behavior is acceptable.
I'll merge these changes, add you to the AUTHORS, then work on releasing ASAP 😄
|
||
>>> from swmmio.examples import pump_control | ||
>>> # NOTE, only the first 5 columns are shown in the following example | ||
>>> pump_control.inp.patterns.iloc[:,0:5] #doctest: +NORMALIZE_WHITESPACE |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Love this - thanks for following our documentation pattern.
control_entry = {} | ||
controls = self._controls_df['[CONTROLS]'] | ||
# make sure the first entry starts with RULE | ||
assert controls[0][0:5] == "RULE " |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would best to raise an exception here if this assertion fails. But I'm fine with capturing that in a future PR.
@@ -258,3 +262,71 @@ def test_polygons(test_model_02): | |||
assert poly1.equals(test_model_02.inp.polygons) | |||
|
|||
# print() | |||
|
|||
def test_inp_sections(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really like this test. My only suggestion is that we clean up the artifacts that are created after this test is run. That said, I'm fine with leaving this as is and refactoring our unit tests in a future PR.
The following PR adds support for the PATTERNS and CONTROLS sections of the INP file along with minor updates to support INFLOWS, XSECTIONS, and CURVES. The Pumping Summary was also added to the list of supported RPT sections.
I included a test that create a swmmio model, reassign each inp section to force the use of "replace_inp_section", save an INP file, and then uses that file to run SWMM. I ran this with a few swmmio models, INP files that come with SWMM, and some utility models I'm working with.
Thanks again for creating this great package! I'm happy to iterate on changes.
PATTERNS
In core.py, I added a patterns property and setter, this reads the PATTERN section as a dataframe (using
dataframe_from_inp
) and then converts it to have 1 row per entry. Columns are named Factor*. The pattern is written to an INP file as a single row per entry. I did not test this with models that have a combination of patterns types (hourly, daily, etc.). Example patterns dataframe below.CONTROLS
In core.py, I added a controls property and setter, this reads the CONTROLS section as a dataframe (using
dataframe_from_inp
) and then converts it to have 1 row per entry. The controI is written to an INP file as multiple lines separated using keywords IF, THEN, PRIORITY, AND, OR, ELSE (in utils.py). Example controls dataframe below.INFLOWS
In core.py, I replace "_!!!!_" with “” instead of NaN to keep the placeholder in the INP file
XSECTIONS
In dataframes.py, I find the max number of tokens (column names) and name extras as “col”+i. This helps read in xsections that have a mixed number of entries.
CURVES
In dataframes.py, I set index to just Name instead of (Name, Type)
RPT file sections
In section_headers.yml, I added "Pumping Summary" and column names
In dataframe_from_rpt, I added end_string “Analysis begun on” to read the final section
Closes #217
Makes progress on #57