User control of MPI threading support level

[theurich]

Currently when ESMF_Initialize() is called without MPI already being initialized, it will try to initialize MPI with MPI_THREAD_MULTIPLE (unless ESMF is built without Pthreads support, in which case it goes for MPI_THREAD_SINGLE).

Regardless of how and where MPI was initialized, ESMF checks for the actual provided threading support of the MPI layer, and makes internal threading decisions accordingly.

Some of the literature I have read seems to indicate that higher threading support levels in MPI do come with additional memory usage, and/or have performance impact (e.g. on tag matching and things like that). I think it would be nice for ESMF_Initialize() to offer the user a way to specify the requested MPI threading support level. This would then be routed through ESMX in the usual manner. It would provide an easy way to test the performance impact of different threading support levels in actual applications.

[anntsay]

Group meeting:
Gerhard: currently, we uses the highest MPI thread level, and may put pressure on memory.. so to balance that, maybe we could set this as an option in order to have better user control of performance. If this is to be modified, the code should just confined to the initialization phase, so it is pretty low risk.

could be also be a good troubleshoot tool so we could experiment whether MPI number of thread is the issue.