Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fire blending to GOCART #2883

Open
wants to merge 91 commits into
base: develop
Choose a base branch
from

Conversation

bbakernoaa
Copy link
Contributor

Description

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? YES/NO
  • Does this change require a documentation update? YES/NO
  • Does this change require an update to any of the following submodules? YES/NO (If YES, please add a link to any PRs that are pending.)
    • EMC verif-global
    • GDAS
    • GFS-utils
    • GSI
    • GSI-monitor
    • GSI-utils
    • UFS-utils
    • UFS-weather-model
    • wxflow

How has this been tested?

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • I have made corresponding changes to the documentation if necessary

@bbakernoaa bbakernoaa marked this pull request as ready for review September 13, 2024 13:05
@bbakernoaa bbakernoaa force-pushed the feature/fire_blending branch from ee128da to fd0e7a5 Compare September 13, 2024 13:48
@bbakernoaa bbakernoaa marked this pull request as draft September 13, 2024 14:03
@bbakernoaa bbakernoaa force-pushed the feature/fire_blending branch from fd0e7a5 to c8e6344 Compare September 13, 2024 15:09
@WalterKolczynski-NOAA WalterKolczynski-NOAA changed the title [GOCART] Add Fire Blending Add fire blending to GOCART Sep 17, 2024
@aerorahul aerorahul added CI-Gaea-Ready **CM use only** PR is ready for CI testing on Gaea and removed CI-Gaea-Ready **CM use only** PR is ready for CI testing on Gaea labels Sep 19, 2024
AntonMFernando-NOAA and others added 6 commits September 19, 2024 15:21
As forecast ensemble jobs are added to the global workflow, this PR
ensures the output is being cleaned up properly once it is no longer
needed.
  Resolves NOAA-EMC#833
This PR updates the parm/config/gfs/config.resources and env/WCOSS2.env
files for the BUFR sounding job "postsnd." It includes adjustments to
resource settings such as tasks per node and memory allocations for
various GFS resolutions, including C768, C1152, and others.

Here are the proposed changes:

C768: 7 nodes, 21 tasks per node
C1152: 16 nodes, 9 tasks per node
NCO has requested that each COM variable specify whether it is an input
or an output. This completes that process for the global-workflow
Unified Post Processor (UPP) task.
Refs: NOAA-EMC#2451
This PR updates the `develop` branch to use the newer operational
`obsproc/v1.2.0` and `prepobs/v1.1.0`. The obsproc/prepobs installs in
glopara space on supported platforms use tags cut from the `dev/gfsv17`
branches in the respective repos. The installation of `prepobs/v1.1.0`
on WCOSS2 is called "gfsv17_v1.1.0" to help avoid GFSv16 users using it
instead of the operational module.

Also, the `HOMEobsproc` path is updated to set an empty default for
`obsproc_run_ver`. This both removes the need to set a default (and
constantly update it, which is duplication) and avoid the unset variable
error when the fcst jobs use their own load module script that does not
know `obsproc_run_ver`:
```
export HOMEobsproc="${BASE_GIT:-}/obsproc/v${obsproc_run_ver:-}"
```

This PR also reverts the prepobs and fit2obs installs on MSU back to the
glopara space from the temporary `/work/noaa/global/kfriedma/glopara`
space installs.

Lastly, this PR also includes updates to complete issue NOAA-EMC#2844 (merge
`build.spack.ver` and `run.spack.ver`).

Resolves NOAA-EMC#2291
Resolves NOAA-EMC#2840
Resolves NOAA-EMC#2844
Copy link
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bbakernoaa
Please let me know if any of my comments need clarifications

parm/config/gefs/config.prep_emissions Show resolved Hide resolved
parm/prep/aero_emissions.yaml Outdated Show resolved Hide resolved
parm/prep/aero_emissions.yaml Outdated Show resolved Hide resolved
parm/prep/aero_emissions.yaml Outdated Show resolved Hide resolved
parm/prep/aero_emissions.yaml Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
@zmoon zmoon force-pushed the feature/fire_blending branch from 7d3f264 to 5ce4047 Compare November 21, 2024 17:26
@zmoon zmoon force-pushed the feature/fire_blending branch from 8c07b21 to 0670aae Compare November 21, 2024 17:33
Copy link
Contributor

@WalterKolczynski-NOAA WalterKolczynski-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's set up a meeting early next week to discuss so we can get this finished. Just find a time on my calendar. @zmoon @bbakernoaa

ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
zmoon and others added 6 commits November 25, 2024 15:34
Co-authored-by: Walter Kolczynski - NOAA <[email protected]>
and `logger.exception()` messages in `except` don't need to include the
error message, since the output automatically includes the exception
type, message, and traceback
Co-authored-by: Walter Kolczynski - NOAA <[email protected]>
Copy link
Contributor

@WalterKolczynski-NOAA WalterKolczynski-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just pretty small stuff, but there is also a merge conflict to resolve with develop.

ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
ush/python/pygfs/task/aero_emissions.py Outdated Show resolved Hide resolved
Co-authored-by: Walter Kolczynski - NOAA <[email protected]>
@zmoon zmoon force-pushed the feature/fire_blending branch from 62d1702 to d047882 Compare December 3, 2024 17:37
Copy link
Contributor

@WalterKolczynski-NOAA WalterKolczynski-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still awaiting the new ExtData and any changes associated with that.

@WalterKolczynski-NOAA WalterKolczynski-NOAA added the blocked Issue is currently being blocked by another issue label Dec 3, 2024
)

# TODO: if AERO_EMIS_FIRE is not 'blended' we don't want to do anything!
Copy link

@zmoon zmoon Dec 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@WalterKolczynski-NOAA could you share some insights on how we should accomplish this? Is there a way to get the value of AERO_EMIS_FIRE (from parm/config/{gefs,gfs)/config.aero) within this Task? Or should the Task just be skipped somewhere further up the workflow scripting chain?

cc: @bbakernoaa

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the job isn't needed at all, then it should be skipped at the workflow level (in gfs_tasks.py/gefs_tasks.py).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, it should be removed in workflow/applications/gfs_cycled.py etc, which may also require changed to gfs_tasks.py, etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked Issue is currently being blocked by another issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants