Adds memory cleanup functions for some Omega classes

[philipwjones]

For Omega classes that store all instances, this adds some erase/clear functions to clean up memory before the yakl finalize call. Updates developer guide to note the requirement for this cleanup.

This fixes a bug in the initial Decomp implementation that was preventing the successful use of device arrays so the Decomp device arrays are now enabled with this PR.

All tests pass on Chrysalis/Intel. Will test on others and update.

Checklist

• Documentation:
  • Design document has been generated and added to the docs
  • User's Guide has been updated
  • Developer's Guide has been updated
  • Documentation has been built locally and changes look as expected
• Testing
  • A comment in the PR documents testing used to verify the changes including any tests that are added/modified/impacted.
  • CTest unit tests for new features have been added per the approved design.
  • Unit tests have passed. Please provide a relevant CDash build entry for verification.

[grnydawn]

@philipwjones , this looks good to me. I have successfully tested this PR on Frontier and Chrysalis.

One modification that I made to run ctest is to add DECOMP_TEST at the bottom of test/CMakeLists.txt as shown below

set_tests_properties(
DATA_TYPES_TEST
MACHINE_ENV_TEST
BROADCAST_TEST
LOGGING_TEST
DECOMP_TEST
YAKL_TEST
PROPERTIES FAIL_REGULAR_EXPRESSION "FAIL"
)

[philipwjones]

@grnydawn thanks for catching that - I had fixed some of those in other PRs but looks like I missed this one, just pushed the change

[brian-oneill]

Built and tested on Frontier and Chrysalis, no issues. Will keep these requirements in mind going forward.

[xylar]

@brian-oneill, it looks like you did squash-merges when you merged this and #43. Unless I'm mistaken, I think we instead want merge commits and to preserve the commit history. @philipwjones, correct me if that's not correct.

[philipwjones]

@xylar I have been doing squash merges of some of these, especially if many of the commits are minor tweaks like formatting in response to linting. So it's been a little bit of a judgement call in keeping any significant history while not introducing too much clutter. I don't know that we ever established a formal rule on this so let me know if you prefer otherwise and we can all decide.

[xylar]

Oh, okay, that's fine. Just wanted to check since that hasn't been the E3SM practice.

[rljacob]

Feature branches should always be added to master with a merge commit. Within a feature branch, you can squash commits to clean up its history.

[xylar]

@rljacob, does the develop branch here count as a very complex "feature" branch? Or as the equivalent of master?

[rljacob]

"develop" would be like a really long-lived feature branch.

[xylar]

Thanks @philipwjones and @rljacob for the clarification. The squashing approach seems fine to me in that case. I can see that the benefits for future debugging probably outweigh the potential costs -- it's a huge pain to wade through a bunch of needless fixup commits.