Does NetCDF in Trilinos now require HDF5?

[ikalash]

A bunch of our Albany nightlies began failing last night, b/c it appears NetCDF requires HDF5 now in Trilinos. Could someone please confirm that that's the case?

CMake Error at cmake/tribits/core/package_arch/TribitsAdjustPackageEnables.cmake:952 (message):
   ***
   *** ERROR: Setting TPL_ENABLE_Netcdf=OFF which was 'ON' because Netcdf has a required library dependence on disabled package HDF5!
   ***

https://sems-cdash-son.sandia.gov/cdash/build/43480/configure

Tagging @lxmota who would be interested also in this issue.

[spdomin]

I am not sure if this is related, however our Nalu builds all started to fail due to the use of this apparently now illegal config command:

-DTrilinos_ASSERT_MISSING_PACKAGES=OFF \

Perhaps if this check is no longer occurring, some sort of build process would allow a build with the missing package and then fail at run time.

I believe an app code would want to be notified of a missing package, however, perhaps that is now default.

I need to check my records on Trilinos breaks during the Holiday break:)

[ikalash]

@spdomin : the nightly builds for Albany that did not fail with the NetCDF/HDF5 error failed with the error you are referring to:

https://sems-cdash-son.sandia.gov/cdash/build/43456/configure

Applying the fix suggested in the error message resolved the problem.

FWIW, I'm not sure pushing big changes to Trilinos that will break everything the day before the shutdown is the best idea in general...

[cgcgcg]

@bartlettroscoe Could the recent TriBITS changes have caused this?

[ikalash]

As expected, it appears turning on HDF5 in the build fixes the problem. I'm OK with this, but would be curious to know why HDF5 is now required for NetCDF in Trilinos. That was not the case before.

[spdomin]

On the Nalu side, we already require HDF5, NetCDF, and Parallel-NetCDF. In general, I believe that from an application perspective, we need to understand all new dependencies that are created on our behalf. HDF5 and NetCDF are not the easiest TPLs to build. As such, I would make sure that these are really required for an Application.

From what I have seen over the years with STK facilitating Boost deprecation, and KLU allowing for SuperLU deprecation, we are generally improving the TPL dependencies. However, I am sure that there is a Trilinos process to be followed when adding a new dependency, right? (I know that adding a new package to Trilinos-proper has a process.)

My change to the Trilinos config for Nalu resulted in a successful build.

Looking forward to more discussion in the New Year.

[dridzal]

We are seeing some unexplained memory use with HDF5 (even when it is not explicitly called in our code). Until this is resolved, we need to be able to build with NetCDF and without HDF5. Generally speaking, I would prefer it if HDF5 wasn't required for NetCDF.

[dridzal]

I should add that we need to start paying attention to the memory footprint of Trilinos. In several heavier builds, where we are not super-careful about which TPLs and packages are turned on, each MPI rank seems to consume on the order of 1GB of RAM. For explicit methods, where we want to use all available MPI ranks on a node, this lowers the total usable memory on many systems by 25%.

[ikalash]

I agree with your statements @dridzal . It also seems to me with the HDF5 enabled, the builds are taking longer, though that will not be 100% clear until the nightlies run again tonight.

[bartlettroscoe]

@ikalash, this is interesting. This is likely due to the big TriBITS refactoring that I just merged in PR #11380. This is a change in backward compatibility that I was not expecting.

I will add a test case for this use case to TriBITS and update the behavior.

[bartlettroscoe]

So this is news to me that you can configure NetCDF without HDF5 so this should not be a required TPL dependency.

I guess a question I need to ask is:

• Should there be TPLs that have required dependencies? For example, does it ever make sense to enable LAPACK without also needing BLAS?

[ikalash]

@bartlettroscoe : just so you know, we had other things in Albany get broken we think due to tribits - see for example this issue: sandialabs/Albany#879 . If you read to the bottom, it seems that tribits might be clashing with the Albany cmake config. I think we might have something in LCM that is similar - I am testing now.

[ikalash]

I think you absolutely can enable and use NetCDF in Trilinos w/o HDF5 - we do this regularly. My understanding was HDF5 was really only needed for very very large files, but perhaps that's not quite accurate.

[bartlettroscoe]

FYI: It seems this use case was a known break in backward compatibility (see TriBITSPub/TriBITS#557). (I am not sure why I was convinced at the time why that was a good idea. (But I don't have anyone to bounce ideas off of while I do this work towards TriBITSPub/TriBITS#63 so we end with up delayed feedback like this issue.)

Longer term, I think we are going to need to extend support in TriBITS for external package/TPL dependencies to be either optional or required on a case-by-case basis (e.g. LAPACK => BLAS required, NetCDF => HDF5 optional). What is implemented in TriBITS right now for TPL dependencies is not consistent with either optional or required dependencies (and is a mix of both).

The right solution for now, and the solution that maintains perfect backward compatibility is to make all TPL dependencies optional.

[bartlettroscoe]

Note that the disable of NetCDF in this case with this version of TriBITS can be avoided by just not enabling HDF5 (instead of disabling it) as described in:

• https://github.com/bartlettroscoe/Trilinos/blob/bfb7dc8f7ae500054be09074047a0a7486a31655/cmake/tribits/CHANGELOG.md#2022-10-20

But this is still a change in behavior and hence the problem reported here.

[gsjaardema]

We are seeing some unexplained memory use with HDF5 (even when it is not explicitly called in our code). Until this is resolved, we need to be able to build with NetCDF and without HDF5. Generally speaking, I would prefer it if HDF5 wasn't required for NetCDF.

There should be no memory use due to hdf5 unless it is being used. As best I can tell and from the tests I ran and showed you earlier, the only time HDF5 resulted in a memory bump (due to a buffer allocation) was when I was explicitly using an exodus file that use the netcdf-4 format which is hdf5-bnsed. Are you seeing memory increases just due to having hdf5 enabled in netcdf (other than the size of the library itself(

[dridzal]

Yes, massif reports about 20% of memory used in HDF5. The test example I have does not perform any output. Is HDF5 also used for input? I haven't been able to use stk without some memory being allocated by HDF5. I was hoping that only NetCDF would be used, which is why I tried compiling without HDF5.

[gsjaardema]

If your input file is in netcdf4 format it will use hdf5 on input.

…

On Sun, Dec 25, 2022, 04:21 Denis Ridzal ***@***.***> wrote: Yes, massif reports about 20% of memory used in HDF5. The test example I have does not perform any output. Is HDF5 also used for input? I haven't been able to use stk without some memory being allocated by HDF5. I was hoping that only NetCDF would be used, which is why I tried compiling without HDF5. — Reply to this email directly, view it on GitHub <#11426 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG3CDUFFLSJLKWC5MMQRVTWO6VTVANCNFSM6AAAAAATH4QQKM> . You are receiving this because you commented.Message ID: ***@***.***>

[bartlettroscoe]

FYI: This is resolved with TriBITSPub/TriBITS#558 and snapshotted into Trilinos in #11458

[bartlettroscoe]

Can we go ahead and close this? This should have been resolved by PR:

#11458

merged back on 1/19/2023.

[github-actions]

This issue has had no activity for 365 days and is marked for closure. It will be closed after an additional 30 days of inactivity.
If you would like to keep this issue open please add a comment and/or remove the MARKED_FOR_CLOSURE label.
If this issue should be kept open even with no activity beyond the time limits you can add the label DO_NOT_AUTOCLOSE.
If it is ok for this issue to be closed, feel free to go ahead and close it. Please do not add any comments or change any labels or otherwise touch this issue unless your intention is to reset the inactivity counter for an additional year.

[github-actions]

This issue was closed due to inactivity for 395 days.