Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RTX Code Release (PI-4): Enhancements to replace and route to include hydrofabric v2.2 (Building off PI-3 additions) #1013

Open
wants to merge 32 commits into
base: ti
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
9965f1f
Github code migration (#1)
taddyb Aug 22, 2024
1cd8fe7
Bug Fix: Message queue erroring out and processes not working asynchr…
taddyb Aug 22, 2024
27ed9ee
merging changes (#3)
david-w-martin Aug 22, 2024
37b0f66
Create .gitkeep
taddyb Aug 23, 2024
f52b80c
Added T-Route and formatted NetCDF metadata (#4)
taddyb Aug 23, 2024
874c0b3
Plotting and post-processing function fixes + tests (#5)
david-w-martin Aug 23, 2024
eff5774
Upgrades to RnR: deploying initial states and exception handing in th…
taddyb Aug 24, 2024
2358767
Bug fix: Enhancing plotting and RnR visualizations (#7)
taddyb Aug 25, 2024
34c62ba
Create .gitkeep (#8)
taddyb Aug 25, 2024
3d3db38
deletes
taddyb Aug 25, 2024
40a2924
removing test_data files as they are causing problems (#9)
taddyb Aug 25, 2024
b88549e
merge conflicts
taddyb Aug 25, 2024
9c24c91
Assimilation (#10)
taddyb Aug 26, 2024
e12f77e
Frontend plot/R&R data view (#11)
david-w-martin Aug 27, 2024
ed6cdcf
Changes 20240827 (#12)
david-w-martin Aug 28, 2024
694103a
Code cleanup and Notebooks (#13)
taddyb Aug 28, 2024
8809d32
Update compose.yaml (#14)
taddyb Aug 28, 2024
b68c005
fixed internal 500 error (#15)
taddyb Aug 28, 2024
090d31a
updating plots and logic gates for demoing resutls
taddyb Aug 29, 2024
539ac39
added more print statements and a fix to a copy path (#17)
taddyb Aug 29, 2024
9d1e757
Frontend comment/documentation cleanup (#16)
david-w-martin Aug 29, 2024
a2478b7
Docs bindas (#19)
taddyb Aug 29, 2024
c419858
Initial MVP IaC for RnR (#18)
kevinspurrier Aug 30, 2024
28638c5
Merge branch 'NOAA-OWP:ti' into development
taddyb Sep 3, 2024
acdab76
Documentation (#20)
taddyb Sep 3, 2024
4806f2a
Update README.md
taddyb Sep 3, 2024
b60f51e
added versioning to use v2.2 HY within RnR and fixed logging problem
taddyb Nov 28, 2024
047a1d8
changed container pointer
taddyb Dec 2, 2024
3dd4830
IaC: Minor changes to support Hydro versioning
kevinspurrier Dec 11, 2024
cfd5a9f
Merge pull request #21 from kevinspurrier/hydro_versioning
taddyb Dec 11, 2024
177b171
fixed merge conflict
taddyb Dec 17, 2024
c51fdb9
added LICENSE to reflect contracted work and ODbl
taddyb Dec 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Github code migration (#1)
* migrating files into repo

* adding gitignore

* changed identifier to lid to be less ambigious

* Delete Source/RnR/tests/test_data/2930769 directory

* editing binaries

* Delete Source/RnR/tests/test_data/2930769 directory

* adding back non-malformed data
  • Loading branch information
taddyb authored Aug 22, 2024
commit 9965f1f88309382c23e3f2c4f7fa6005cf94e9aa
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,4 @@
*.tgz binary
*.tar.gz binary
*.gz binary
*.gpkg binary
57 changes: 57 additions & 0 deletions .github/workflows/pytest_rnr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: PyTest Suite

on: [pull_request]

jobs:
# lint_and_format:
# runs-on: ubuntu-latest

# steps:
# - uses: actions/checkout@v4
# - uses: actions/setup-python@v5
# with:
# python-version: "3.11"

# - name: Install pre-commit
# run: |
# python -m pip install pre-commit

# - name: Run pre-commit hooks on all files
# run: |
# pre-commit run --all-files

build:
runs-on: ubuntu-latest

strategy:
matrix:
python-version: ["3.9", "3.10", "3.11"]

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}

- name: Set UV_HTTP_TIMEOUT
run: echo "UV_HTTP_TIMEOUT=600" >> $GITHUB_ENV

- name: Install python dependencies and packages
run: |
python -m pip install --upgrade pip
pip install uv
uv venv
source .venv/bin/activate
uv pip install -r Source/RnR/requirements.txt

- name: Echo package versions
run: |
source .venv/bin/activate
uv pip freeze

- name: Run unit tests
run: |
source .venv/bin/activate
cd Source/RnR
pytest -s
20 changes: 19 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -50,4 +50,22 @@ terraform.rc
**/buildplan
**/destroyplan

Dev/env.yaml
Dev/env.yaml

###################
# Replace and Route
###################
Source/RnR/.venv
Source/RnR/src/rnr/app/core/config.ini
Source/RnR/mock_db/rnr_schema_dump_20240612.dump
Source/RnR/data

*.ruff_cache
*.pytest_cache

*.pyc
*.egg-info/
__pycache__/
*.py[cod]
.env
.python-version
72 changes: 72 additions & 0 deletions Source/RnR/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
All notable changes to this project will be documented in this file.
We follow the [Semantic Versioning 2.0.0](http://semver.org/) format.


## 0.0.3 2024-08-20

### Added

- migrated to github
- added references to t-route and hfsubset



## 0.0.2 2024-08-12

### Added

- `/static` libraries for JS and CSS
- `/data` directiories now are linked to the

### Deprecated

- Nothing.

### Removed

- Reformatted the working dir to format application under one dir

### Fixed

- Nothing.


## 0.0.2 2024-08-2

### Added

- the `consumer/` folder to handle all data services related to the consumer (similar to the tools/ dir from PI-2)

### Deprecated

- Nothing.

### Removed

- Reformatted the working dir to format application under one dir

### Fixed

- Nothing.


## 0.0.1 2024-07-17

### Added

- Added microservices to read a DB of RFC information
- Commented/documented microservices that read the NWPS API
- Commented/documented microservices that read the DB
- Added relevant tests and project dependencies

### Deprecated

- Nothing.

### Removed

- Nothing.

### Fixed

- Nothing.
15 changes: 15 additions & 0 deletions Source/RnR/Dockerfile.app
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
FROM python:3.11-slim

WORKDIR /app

COPY src/ /app/src

COPY requirements.txt ./
COPY pyproject.toml ./
COPY README.md ./

RUN pip install uv==0.2.5
RUN uv venv
RUN uv pip install --no-cache-dir -r requirements.txt

ENV PYTHONPATH=/app:$PYTHONPATH
14 changes: 14 additions & 0 deletions Source/RnR/Dockerfile.docs
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
FROM python:3.11-slim

WORKDIR /app

COPY src/ /app/src
COPY requirements.txt ./
COPY pyproject.toml ./
COPY README.md ./
COPY docs/ ./docs
COPY mkdocs.yml ./

RUN pip install uv==0.2.5
RUN uv venv
RUN uv pip install --no-cache-dir -r requirements.txt
3 changes: 3 additions & 0 deletions Source/RnR/INSTALL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Installation instructions

Detailed instructions on how to install, configure, and get the project running.
7 changes: 7 additions & 0 deletions Source/RnR/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
“Software code created by U.S. Government employees is not subject to copyright
in the United States (17 U.S.C. §105). The United States/Department of Commerce
reserve all rights to seek and obtain copyright protection in countries other
than the United States for Software authored in its entirety by the Department
of Commerce. To this end, the Department of Commerce hereby grants to Recipient
a royalty-free, nonexclusive license to use, copy, and create derivative works
of the Software outside of the United States.”
98 changes: 98 additions & 0 deletions Source/RnR/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
# Replace and Route

A near real-time system that leverages the NWM data assimilation and channel routing capabilities to extend official flood forecasts issued by RFCs to all river reaches downstream of RFC forecast points by routing the forecasts through the river network to the next downstream forecast point.




## Installation

1.

> Keep the README fresh! It's the first thing people see and will make the initial impression.

## Credits and References

Credit to the developers and maintainers of https://github.com/NOAA-OWP/owp-open-source-project-template/tree/master for providing a starting template

# Project Title

**Description**: Put a meaningful, short, plain-language description of what
this project is trying to accomplish and why it matters.
Describe the problem(s) this project solves.
Describe how this software can improve the lives of its audience.

Other things to include:

- **Technology stack**: Indicate the technological nature of the software, including primary programming language(s) and whether the software is intended as standalone or as a module in a framework or other ecosystem.
- **Status**: Alpha, Beta, 1.1, etc. It's OK to write a sentence, too. The goal is to let interested people know where this project is at. This is also a good place to link to the [CHANGELOG](CHANGELOG.md).
- **Links to production or demo instances**
- Describe what sets this apart from related-projects. Linking to another doc or page is OK if this can't be expressed in a sentence or two.


**Screenshot**: If the software has visual components, place a screenshot after the description; e.g.,

![](https://raw.githubusercontent.com/NOAA-OWP/owp-open-source-project-template/master/doc/Screenshot.png)


## Dependencies

1. RabbitMQ (Version 3.13) https://www.rabbitmq.com/docs/download
2. Rye (Version 0.36)
3. A Postgres DB of RFC locations
4. MKDocs (https://squidfunk.github.io/mkdocs-material/publishing-your-site/#with-github-actions-material-for-mkdocs)

## Installation

Detailed instructions on how to install, configure, and get the project running.
This should be frequently tested to ensure reliability. Alternatively, link to
a separate [INSTALL](INSTALL.md) document.

## Configuration

If the software is configurable, describe it in detail, either here or in other documentation to which you link.

## Usage

Show users how to use the software.
Be specific.
Use appropriate formatting when showing code snippets.

## How to test the software

Run the following command from the project's home dir
```sh
rye test
```

## Known issues

Still a POC. Will be finished 9/1/24

## Getting help

If you have questions, concerns, bug reports, etc, please file an issue in this repository's Issue Tracker.

## Getting involved

This section should detail why people should get involved and describe key areas you are
currently focusing on; e.g., trying to get feedback on features, fixing certain bugs, building
important pieces, etc.

General instructions on _how_ to contribute should be stated with a link to [CONTRIBUTING](CONTRIBUTING.md).


----

## Open source licensing info
1. [TERMS](TERMS.md)
2. [LICENSE](LICENSE)


----

## Credits and references

1. Projects that inspired you
2. Related projects
3. Books, papers, talks, or other sources that have meaningful impact or influence on this project
Loading