Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementation of Menter's 1-eq gamma transition model #1287

Merged
merged 35 commits into from
Sep 25, 2024

Conversation

BumseokLee
Copy link
Contributor

@BumseokLee BumseokLee commented Aug 21, 2024

Hi,

I would like to begin the draft of pull request for the transition model implementation in Nalu-Wind.

This is a draft because there are still some remaining work. However, the basic function of the model is ready and I've completed validation and verification of the model for 2-D airfoils and 3-D cases. The list of the validation (and verification) is as below:

2-D cases

  • ERCOFTAC T3A & T3B flat plate with five different mesh resolution: AIAA Transition Modeling Workshop cases
  • NASA NLF(1)-0416 and NREL S809 with six different mesh resolutions: AIAA Transition Modeling Workshop cases
  • Eppler 387 and NREL S822 at Re=100K, 200K, and 500K: Low Reynolds number cases
  • TU Delft DU00-W-212 at Re=3M to 15M: High Reynolds number case

3-D cases

  • DLR 6:1 Prolate Spheroid
  • NREL Phase VI rotor

All of the validations were performed without any hard-coding. The results were compared against the experimental data as well as available other numerical results, such as NASA's OVERFLOW or FUN3D predictions.

This work did not begin from scratch but from previous work by Robert Wilson. I’ve attached some related history below.

#957
#962

He initiated the implementation, but the work wasn’t merged into the main branch due to some issues (pull/957). Additionally, some of his implementation was also reverted (pull/962) due to conflicts with other parts.

I began my work from the state of pull/962. I fixed some bugs, removed hard coding, and completely re-implemented the computation of the pressure gradient term, which is a key aspect of the current transition model and computed from several different gradients.

Currently, the transition model is only coupled to the k-omega SST model, not with hybrid RANS/LES. This is because there might be interference between the transition model and hybrid RANS/LES. On one hand, the transition model changes the characteristics of the baseline turbulence model and boundary layer. On the other hand, hybrid RANS/LES models (DES/DDES/IDDES) were developed based on fully turbulent flow assumptions with the baseline turbulence model. Therefore, there could be some undesirable interference between them. Once the current work is completed, I will investigate this.

I would like to discuss the implementation of the wall boundary condition of the model, which is the most uncertain part for me. The wall boundary condition of the transition model is zero normal flux of the intermittency (gamma). I had a meeting with Phil and asked about this but couldn't fully understand that. I also tried to check the symmetric boundary conditions in Nalu-Wind which should also have zero normal flux across the symmetric plane, it didn't work well. For this reason, I just reused what was implemented before.
This needs to be definitely double-checked, and any help or suggestions would be extremely appreciated!

We've decided to implement the overset capability of the transition model in ExaWind stack.
For a typical 3-D wind turbine case, overset exchange of intermittency between different meshes is not important, as the overset boundaries, from the wall to fringe layers, are much larger than the boundary layer thickness. However, in certain cases where there are very small gaps between solid bodies or overlapping surfaces, the oveset exchange between these Nalu-Wind meshes could become important.

Currently, TIOGA does not distinguish overset exchange between Nalu-Wind / Nalu-Wind and Nalu-Wind / AMR-Wind. Due to this, we decided to implement the overset capability of the transition model in both Nalu-Wind and AMR-wind. The capability is already implemented in Nalu-Wind. For AMR-Wind, we will implement a constant, uniform intermittency field with a value of 1 for transition simulations using the hybrid solvers, which is the freestream conditions of the transition model. Note that AMR-Wind domain is still outside of the boundary layer of the Nalu-Wind meshes and thus it doesn't need to solve the transport equation of the transition model. Therefore, for AMR-Wind, the freestream condition of the transition model will be applied in the computational domain. This implementation will be included in a separate PR of AMR-Wind.( Exawind/amr-wind#1256 )

In summary, a list of the remaining work is as below to complete the implementation of the transition model in Nalu-Wind

  • Unit test
  • Regression Test
  • Confirm the wall boundary condition of the model
  • Implementation of the two user options for the freestream turbulence intensity

@gantech gantech requested a review from psakievich August 22, 2024 17:46
@gantech gantech requested review from ndevelder and marchdf and removed request for psakievich September 3, 2024 19:47
Copy link
Contributor

@marchdf marchdf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just did a pass for esthetics really. We are chatting later this week about other things.

include/ShearStressTransportEquationSystem.h Outdated Show resolved Hide resolved
src/SpecificDissipationRateEquationSystem.C Outdated Show resolved Hide resolved
src/SpecificDissipationRateEquationSystem.C Outdated Show resolved Hide resolved
src/TurbKineticEnergyEquationSystem.C Outdated Show resolved Hide resolved
src/node_kernels/BLTGammaM2015NodeKernel.C Show resolved Hide resolved
src/node_kernels/BLTGammaM2015NodeKernel.C Outdated Show resolved Hide resolved
src/node_kernels/BLTGammaM2015NodeKernel.C Outdated Show resolved Hide resolved
src/node_kernels/SDRSSTBLTM2015NodeKernel.C Outdated Show resolved Hide resolved
src/node_kernels/SDRSSTBLTM2015NodeKernel.C Outdated Show resolved Hide resolved
@marchdf
Copy link
Contributor

marchdf commented Sep 19, 2024

Ok we are getting close! Things left to do:

  • make the CI pass
  • address the last comments I left
  • documentation, mostly in theory/turbulenceModeling
  • run the new regression test and unit tests on GPU

@marchdf
Copy link
Contributor

marchdf commented Sep 25, 2024

Helping out to make sure the tests are working on GPU

Unit tests are fine:

❯ ./unittestX --gtest_filter="SSTKernelHex8Mesh*"
   Nalu-Wind Version: v2.0.0-89-g8e945c58
   Nalu-Wind GIT Commit SHA: 8e945c58274bddca6c7fff63bbf25c16518d0d7e
   Trilinos Version: 15.1.1                                                                                                                                                                                                                                                        [25/1905]

Note: Google Test filter = SSTKernelHex8Mesh*
[==========] Running 27 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 27 tests from SSTKernelHex8Mesh
[ RUN      ] SSTKernelHex8Mesh.StreletsUpwindComputation
[       OK ] SSTKernelHex8Mesh.StreletsUpwindComputation (39 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_scalar_edge_open_solver_alg
[       OK ] SSTKernelHex8Mesh.NGP_scalar_edge_open_solver_alg (40 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_scalar_open_edge
[       OK ] SSTKernelHex8Mesh.NGP_scalar_open_edge (25 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_tke_sst_node
[       OK ] SSTKernelHex8Mesh.NGP_tke_sst_node (15 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_tke_sst_sust_node
[       OK ] SSTKernelHex8Mesh.NGP_tke_sst_sust_node (9 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_tke_sstlr_node
[       OK ] SSTKernelHex8Mesh.NGP_tke_sstlr_node (12 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_tke_sst_des_node
[       OK ] SSTKernelHex8Mesh.NGP_tke_sst_des_node (12 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_tke_sst_iddes_node
[       OK ] SSTKernelHex8Mesh.NGP_tke_sst_iddes_node (12 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_tke_sst_des_sust_node
[       OK ] SSTKernelHex8Mesh.NGP_tke_sst_des_sust_node (9 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_tke_sst_trans_node
[       OK ] SSTKernelHex8Mesh.NGP_tke_sst_trans_node (12 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_sdr_sst_node
[       OK ] SSTKernelHex8Mesh.NGP_sdr_sst_node (12 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_sdr_sst_sust_node
[       OK ] SSTKernelHex8Mesh.NGP_sdr_sst_sust_node (10 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_sdr_sstlr_node
[       OK ] SSTKernelHex8Mesh.NGP_sdr_sstlr_node (12 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_sdr_sst_des_node
[       OK ] SSTKernelHex8Mesh.NGP_sdr_sst_des_node (12 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_sdr_sst_des_sust_node
[       OK ] SSTKernelHex8Mesh.NGP_sdr_sst_des_sust_node (10 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_sdr_sst_trans_node
[       OK ] SSTKernelHex8Mesh.NGP_sdr_sst_trans_node (12 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_blt_gamma_node
[       OK ] SSTKernelHex8Mesh.NGP_blt_gamma_node (13 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_nodal_grad_edge
[       OK ] SSTKernelHex8Mesh.NGP_nodal_grad_edge (11 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_nodal_grad_elem
[       OK ] SSTKernelHex8Mesh.NGP_nodal_grad_elem (10 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_nodal_grad_elem_shifted
[       OK ] SSTKernelHex8Mesh.NGP_nodal_grad_elem_shifted (9 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_nodal_grad_bndry
[       OK ] SSTKernelHex8Mesh.NGP_nodal_grad_bndry (23 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_nodal_grad_bndry_shifted
[       OK ] SSTKernelHex8Mesh.NGP_nodal_grad_bndry_shifted (19 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_turb_visc_sst_alg
[       OK ] SSTKernelHex8Mesh.NGP_turb_visc_sst_alg (12 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_turb_visc_sstlr_alg
[       OK ] SSTKernelHex8Mesh.NGP_turb_visc_sstlr_alg (9 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_sdr_wall_lowRE
[       OK ] SSTKernelHex8Mesh.NGP_sdr_wall_lowRE (26 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_sdr_wall_func
[       OK ] SSTKernelHex8Mesh.NGP_sdr_wall_func (24 ms)
[ RUN      ] SSTKernelHex8Mesh.NGP_SST_Max_Length_Scale
[       OK ] SSTKernelHex8Mesh.NGP_SST_Max_Length_Scale (11 ms)
[----------] 27 tests from SSTKernelHex8Mesh (422 ms total)

[----------] Global test environment tear-down
[==========] 27 tests from 1 test suite ran. (422 ms total)
[  PASSED  ] 27 tests.

@marchdf
Copy link
Contributor

marchdf commented Sep 25, 2024

And here's the log file for the reg test. It fails with:

Cuda Runtime Configuration:
macro  KOKKOS_ENABLE_CUDA      : defined
macro  CUDA_VERSION          = 12050 = version 12.5
Kokkos::Cuda[ 0 ] NVIDIA A100-PCIE-40GB capability 8.0, Total Global Memory: 39.5 G, Shared Memory per Block: 48 K : Selected
Kokkos::Cuda[ 1 ] NVIDIA A100-PCIE-40GB capability 8.0, Total Global Memory: 39.5 G, Shared Memory per Block: 48 K
(CudaInternal::singleton().cuda_device_synchronize_wrapper()) error( cudaErrorIllegalAddress): an illegal memory access was encountered /mnt/vdb/home/mhenryde/exawind/exawind-manager/stage/spack-stage-trilinos-15.1.1-izwc2enrza5c5uyoizre5fgn3ghedsxx/spack-src/packages/kokkos/core/src/Cuda/Kokkos_Cuda_Instance.cpp:153

out.log

@marchdf
Copy link
Contributor

marchdf commented Sep 25, 2024

Ok figured out the problem and pushed the fix. Here's the completed log file:
out.log

@BumseokLee BumseokLee marked this pull request as ready for review September 25, 2024 19:51
@BumseokLee
Copy link
Contributor Author

Made the CI pass & added the documentation about the transition model.

@BumseokLee BumseokLee requested a review from marchdf September 25, 2024 21:19
@BumseokLee BumseokLee requested a review from marchdf September 25, 2024 21:50
@BumseokLee BumseokLee removed the request for review from ndevelder September 25, 2024 21:53
@marchdf marchdf merged commit 6155b17 into Exawind:master Sep 25, 2024
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants