Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] acoustic - elastic coupling (SEM, 2ⁿᵈ order wave equation) #2557

Closed
wants to merge 120 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
120 commits
Select commit Hold shift + click to select a range
df3a323
Add variable density inside second order acoustic wave solver
acitrain Jun 27, 2023
93f84a9
uncrustify + modify unit test to take in account the density
acitrain Jun 27, 2023
1d1a4ec
start AcousticElastic SEM coupled solver
tbeltzun Jun 28, 2023
16a0506
allow empty source / receiver
tbeltzun Jun 28, 2023
87e4291
add pseudo 2D .vtu
tbeltzun Jun 28, 2023
bad930d
add association: `fieldsToImport` and `fieldNamesInGEOSX`
tbeltzun Jul 4, 2023
a7e6064
rename conflicting common ns fields variables
tbeltzun Jul 4, 2023
8516311
fix xml
tbeltzun Jul 4, 2023
c031ce5
add `NodeManager` workaround (div by 0)
tbeltzun Jul 5, 2023
fb5fba0
add targetNodeSets that will be handeled by the current solver
Jul 6, 2023
dda3912
add `m_` prefix to `solverTargetNodesSet` (protected member)
tbeltzun Jul 6, 2023
d56d9f5
compute `interfaceNodesSet`
tbeltzun Jul 7, 2023
4fd1159
debug higher order
tbeltzun Jul 7, 2023
fa555d1
fix the number of the nodes on interfaces for high-order meshes
Jul 9, 2023
d9409cc
clarify loop on quadrature nodes
tbeltzun Jul 10, 2023
87a0560
remove debugging code
tbeltzun Jul 10, 2023
8c6915e
generate data for validation against article (160x120, SEM(5))
tbeltzun Jul 10, 2023
bfd4258
fix mesh
tbeltzun Jul 10, 2023
f8e6ec0
support higher order ricker wavelets - simplify pulse computation
tbeltzun Jul 10, 2023
49b8340
update integratedTests submodule
tbeltzun Jul 10, 2023
7db2d45
update submodules
tbeltzun Jul 10, 2023
d28e52b
try to debug perf issue
tbeltzun Jul 11, 2023
6f7a7f7
add 20k elems Q1 test
tbeltzun Jul 11, 2023
073e916
remove deb file
tbeltzun Jul 12, 2023
930fb8d
update submodules
tbeltzun Jul 15, 2023
7f6aa13
Merge branch 'develop' into feature/tbeltzun/acous-elas-sem
tbeltzun Jul 15, 2023
64e5afa
remove unrelated change
tbeltzun Jul 16, 2023
8ab54bd
update docs
tbeltzun Jul 16, 2023
2f835e3
fix CUDA crash
tbeltzun Jul 16, 2023
e8039ac
Merge branch 'develop' into feature/AddVariableDensityInAcoustic
acitrain Jul 17, 2023
014b500
checkpoint
tbeltzun Jul 17, 2023
066efa9
Merge branch 'develop' into feature/AddVariableDensityInAcoustic
acitrain Jul 18, 2023
40fe2da
fix typo in stiffnessVector description
tbeltzun Jul 18, 2023
a788b80
Merge branch 'develop' into feature/tbeltzun/acous-elas-sem
tbeltzun Jul 18, 2023
0ab0e7a
handle `solverStep` in `AcousticElasticWaveEquationSEM`
tbeltzun Jul 19, 2023
afa2aa0
split synchronizations and post-process from PDE update
tbeltzun Jul 19, 2023
38b01e4
rework structure
tbeltzun Jul 20, 2023
2fc3bcb
Merge branch 'develop' into feature/AddVariableDensityInAcoustic
acitrain Jul 20, 2023
03b6fbe
hoist out `forDiscretizationOnMeshTargets` loops - fix discretization…
tbeltzun Jul 20, 2023
1c27cfe
implement coupling skeleton
tbeltzun Jul 20, 2023
1cc65fc
Merge branch 'develop' into feature/AddVariableDensityInAcoustic
acitrain Jul 21, 2023
2bf1b4b
Fix bug after failed merge
acitrain Jul 21, 2023
9af05df
Merge branch 'develop' into feature/AddVariableDensityInAcoustic
acitrain Jul 21, 2023
c18546f
checkpoint sync
tbeltzun Jul 24, 2023
991de3c
Merge branch 'develop' into feature/tbeltzun/acous-elas-sem
tbeltzun Jul 24, 2023
0c72ecb
fix docstring typos
tbeltzun Jul 24, 2023
e58a994
Merge branch 'develop' into feature/AddVariableDensityInAcoustic
acitrain Jul 25, 2023
f8b0ec0
uncrustify
acitrain Jul 25, 2023
13b5586
update input files for integrated tests
acitrain Jul 25, 2023
a60d6e2
Merge branch 'develop' into feature/tbeltzun/acous-elas-sem
tbeltzun Jul 25, 2023
78bfb0d
Merge branch 'develop' into feature/AddVariableDensityInAcoustic
acitrain Jul 26, 2023
4a74bd0
add `AcoustiElastic` kernels
tbeltzun Jul 26, 2023
4159d95
update
tbeltzun Jul 26, 2023
821a55e
Merge branch 'develop' into feature/tbeltzun/acous-elas-sem
tbeltzun Jul 26, 2023
4058f14
uncrustify
acitrain Jul 26, 2023
f208844
Merge branch 'develop' into feature/AddVariableDensityInAcoustic
acitrain Jul 26, 2023
a532f47
implement `CouplingKernel` based on damping kernel
tbeltzun Jul 26, 2023
c8e135c
add debug code
tbeltzun Jul 26, 2023
134e6bf
checkpoint correct interface faces - coupling similar to `Damping` ke…
tbeltzun Jul 27, 2023
37a25c1
Merge branch 'develop' into feature/AddVariableDensityInAcoustic
acitrain Jul 28, 2023
e7411f9
checkpoint
tbeltzun Jul 28, 2023
31e2273
fix `prepareNextTimestep` in acous / elas solvers
tbeltzun Jul 28, 2023
b3878b1
checkpoint density
tbeltzun Jul 28, 2023
05b8477
Merge remote-tracking branch 'origin/feature/AddVariableDensityInAcou…
tbeltzun Jul 28, 2023
e4970b0
acous - elas running - coupling fails with NaNs
tbeltzun Jul 28, 2023
ca42e28
fix wrong elastic density - fixed interface density
tbeltzun Aug 9, 2023
03a1122
Merge branch 'develop' into feature/tbeltzun/acous-elas-sem
tbeltzun Aug 21, 2023
e906d8e
Merge branch 'develop' into feature/tbeltzun/acous-elas-sem
tbeltzun Aug 22, 2023
d0f2f02
fix `mediumDensity` rename in inputFiles
tbeltzun Aug 22, 2023
35d94b1
restore comment
tbeltzun Aug 22, 2023
6d6bf1e
Merge branch 'develop' into feature/tbeltzun/acous-elas-sem
tbeltzun Aug 23, 2023
9abb16e
comment dead code
tbeltzun Aug 23, 2023
e5b5019
restrict solver to `Regions`
tbeltzun Aug 25, 2023
7a5cbc8
Merge branch 'develop' into feature/tbeltzun/acous-elas-sem
tbeltzun Aug 25, 2023
81eec7b
checkpoint revert
tbeltzun Aug 25, 2023
94995d6
restrict again solvers to subRegions
tbeltzun Aug 25, 2023
0bef8cb
update internal mesher acouselas case
tbeltzun Aug 25, 2023
ba44121
remove fluid density scaling - cleanup
tbeltzun Aug 25, 2023
6fccbd2
simplify targetRegions
tbeltzun Aug 26, 2023
fb70cb9
Merge branch 'develop' into feature/tbeltzun/acous-elas-sem
tbeltzun Aug 26, 2023
0de4af5
plausible bidirectional coupling
tbeltzun Aug 29, 2023
48d735b
3D case
tbeltzun Aug 30, 2023
bf2ed8c
handle normals sign
tbeltzun Aug 30, 2023
767c75f
remove test case based on external mesh
tbeltzun Aug 30, 2023
271dd77
rm dead code - cleanup - format
tbeltzun Sep 1, 2023
60e6e94
Merge branch 'develop' into feature/tbeltzun/acous-elas-sem
tbeltzun Sep 4, 2023
20c225a
vtk hack for Q2 visualization
tbeltzun Sep 4, 2023
a0a6165
remove debugging prints
tbeltzun Sep 4, 2023
0af060b
MPI::sum reduction for interfacenodes
tbeltzun Sep 4, 2023
2345361
simplify sources - remove helpers
tbeltzun Sep 5, 2023
5fc777d
flip sign - simplify coupling
tbeltzun Sep 5, 2023
eef2207
SIGFPE workaround
tbeltzun Sep 5, 2023
96c918e
faster `SortedArray` insertion
tbeltzun Sep 5, 2023
beb664d
workaround `float-equal` warning
tbeltzun Sep 5, 2023
b866bda
add `.hdf5` outputs
tbeltzun Sep 5, 2023
b055006
checkpoint crash elastic damping kernel `seq_exec`
tbeltzun Sep 8, 2023
fa2e000
rework damping kernels
tbeltzun Sep 10, 2023
e89e6db
fix `hdf5` output
tbeltzun Sep 13, 2023
3f019a2
add solver to trace filename for multiple domains
tbeltzun Sep 15, 2023
d7f5d95
update for p3 run
tbeltzun Sep 15, 2023
f11c95a
fix elastic receivers
tbeltzun Sep 15, 2023
8040e61
fix seismo trace write
tbeltzun Sep 18, 2023
2a84fe9
rework seismo traces
tbeltzun Sep 18, 2023
9ff759f
fix RAJA `forAll` assignement - use `std::ios_base::openmode`
tbeltzun Sep 18, 2023
67e9648
add `timeSourceDelay`
tbeltzun Sep 20, 2023
6f4cb92
cleanup
tbeltzun Sep 20, 2023
5e529fd
fix seismoTraces initialization
tbeltzun Sep 21, 2023
50b28d2
simplifications - consistency
tbeltzun Sep 21, 2023
98b31c3
update
tbeltzun Sep 21, 2023
5da09b5
rework traces
tbeltzun Sep 22, 2023
f9612d9
filter mesh levels
tbeltzun Sep 22, 2023
e268d1a
more robust `writeSeismoTrace` in cleanup
tbeltzun Sep 22, 2023
43e64fd
simplifications
tbeltzun Sep 22, 2023
707d46c
add recv pos MPI workaround
tbeltzun Sep 23, 2023
18345a5
rm `eps` workaround and add more robust receivers check
tbeltzun Sep 24, 2023
200adf4
consistency - cleanup
tbeltzun Sep 24, 2023
12a028d
add decomposition fix to 1st order equations
tbeltzun Sep 24, 2023
0477125
start `highOrder` VTK flag implementation
tbeltzun Sep 24, 2023
d743133
update acouselas3D case for transverse reflections
tbeltzun Sep 27, 2023
a2ad1fe
simplifications
tbeltzun Sep 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions inputFiles/wavePropagation/acous3D_Q3_small_base.xml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,14 @@
scale="1500"
setNames="{ all }"/>

<FieldSpecification
name="cellDensity"
initialCondition="1"
objectPath="mesh/FE2/ElementRegions/Region/cb"
fieldName="mediumDensityA"
scale="1"
setNames="{ all }"/>

<!-- 3) The free surface condition the domain -->
<FieldSpecification
name="zposFreeSurface"
Expand Down
8 changes: 8 additions & 0 deletions inputFiles/wavePropagation/acous3D_Q5_small_base.xml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,14 @@
scale="1500"
setNames="{ all }"/>

<FieldSpecification
name="cellDensity"
initialCondition="1"
objectPath="mesh/FE2/ElementRegions/Region/cb"
fieldName="mediumDensityA"
scale="1500"
setNames="{ all }"/>

<!-- 3) The free surface condition the domain -->
<FieldSpecification
name="zposFreeSurface"
Expand Down
27 changes: 22 additions & 5 deletions inputFiles/wavePropagation/acous3D_pml_smoke.xml
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@
name="solverApplications"
forceDt="0.0025"
target="/Solvers/acousticSolver"/>

<!-- two events to output pressure in an hdf5 file -->
<PeriodicEvent
name="timeHistoryCollection"
Expand All @@ -99,7 +99,7 @@
name="restarts"
timeFrequency="0.05"
target="/Outputs/restartOutput"/>

</Events>

<NumericalMethods>
Expand Down Expand Up @@ -168,7 +168,7 @@
right,
front,
back}"/>

<!-- The PML field specification assigned to the corresponding region -->
<PML
name="pml"
Expand All @@ -185,6 +185,15 @@
scale="2000"
setNames="{ all }"/>

<FieldSpecification
name="cellDensity1"
initialCondition="1"
objectPath="ElementRegions/interiorDomain"
fieldName="mediumDensityA"
scale="1"
setNames="{ all }"/>


<FieldSpecification
name="cellVelocity2"
initialCondition="1"
Expand All @@ -193,8 +202,16 @@
scale="2000"
setNames="{ all }"/>

<FieldSpecification
name="cellDensity2"
initialCondition="1"
objectPath="ElementRegions/pmlDomain"
fieldName="mediumDensityA"
scale="1"
setNames="{ all }"/>

</FieldSpecifications>

<!-- collect the pressure values at the nodes -->
<Tasks>
<PackCollection
Expand All @@ -214,7 +231,7 @@

<Restart
name="restartOutput"/>

</Outputs>

</Problem>
8 changes: 8 additions & 0 deletions inputFiles/wavePropagation/acous3D_small_base.xml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,14 @@
scale="1500"
setNames="{ all }"/>

<FieldSpecification
name="cellDensity"
initialCondition="1"
objectPath="ElementRegions/Region/cb"
fieldName="mediumDensityA"
scale="1"
setNames="{ all }"/>

<!-- 3) The free surface condition the domain -->
<FieldSpecification
name="zposFreeSurface"
Expand Down
230 changes: 230 additions & 0 deletions inputFiles/wavePropagation/acouselas3D.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
<?xml version="1.0" ?>

<Problem>
<Solvers>
<!-- define the solver, source coordinates, time source frequency, receiver coordinates -->
<AcousticSEM
name="acousticSolver"
discretization="FE1"
targetRegions="{ Fluid }"
sourceCoordinates="{ { 1575, 2400, 2900 } }"
timeSourceFrequency="10"
timeSourceDelay="0.115"
rickerOrder="3"
outputSeismoTrace="1"
dtSeismoTrace="1e-3"
receiverCoordinates="{ { 3752.2936, 2400, 2933.3333 } }"/>
<ElasticSEM
name="elasticSolver"
discretization="FE1"
targetRegions="{ Solid }"
sourceCoordinates="{ }"
outputSeismoTrace="1"
dtSeismoTrace="1e-3"
receiverCoordinates="{ { 3752.2936, 2400, 1866.6667 } }"/>
<!-- NOTES:
- coupled solver must be declared at the end (for interface computations)
- discretization is also needed for the correct mesh to be passed through from the coupled solver to the elastic / acoustic solvers
-->
<AcousticElasticSEM
name="acousticelasticSolver"
discretization="FE1"
targetRegions="{ Solid, Fluid }"
acousticSolverName="acousticSolver"
elasticSolverName="elasticSolver"/>
</Solvers>

<!--
hexahedral mesh generated internally by GEOSX
6400x1600x4800m domain
# R1-Q5 (pseudo-2D)
sourceCoordinates="{ { 1500, 20, 2900 } }"
yCoords="{ 0, 40 }"
nx="{ 160 }"
ny="{ 1 }"
nz="{ 60, 60 }"
# R2-Q2 (pseudo-2D)
sourceCoordinates="{ { 1500, 10, 2900 } }"
yCoords="{ 0, 20 }"
nx="{ 320 }"
ny="{ 1 }"
nz="{ 120, 120 }"
# R5-Q1 (pseudo-2D)
sourceCoordinates="{ { 1500, 4, 2900 } }"
yCoords="{ 0, 8 }"
nx="{ 800 }"
ny="{ 1 }"
nz="{ 300, 300 }"
NOTES:
nx="{ 320 }"
ny="{ 240 }"
nz="{ 120, 120 }
=> local threshold for local builds 3D Q1 Δt=1e-4 (18e6 elems, 10Gb per 1 GPU)
=> Q4 180 procs / GPU 16h
=> Q1 180 procs / GPU 1e40min
nx="{ 160 }"
ny="{ 120 }"
nz="{ 60, 60 }
=> local threshold for local builds 3D Q2 Δt=1e-4 (2e6 elems, 4Gb per 1 GPU)
nx="{ 80 }"
ny="{ 60 }"
nz="{ 30, 30 }
=> coarse 10 min frontal run 3D Q5 Δt=1e-3 (288e3 elems, 8Gb per 6 GPU)
-->
<Mesh>
<InternalMesh
name="mesh"
elementTypes="{ C3D8 }"
xCoords="{ 0, 6400 }"
yCoords="{ 0, 4800.0001 }"
zCoords="{ 0, 2400, 4800 }"
nx="{ 320 }"
ny="{ 240 }"
nz="{ 120, 120 }"
cellBlockNames="{ solid, fluid }"/>
</Mesh>

<NumericalMethods>
<FiniteElements>
<FiniteElementSpace
name="FE1"
order="3"
formulation="SEM"/>
</FiniteElements>
</NumericalMethods>

<Events
maxTime="3">
<PeriodicEvent
name="outputs"
timeFrequency="5e-2"
targetExactTimestep="0">
<PeriodicEvent name="outputs_vtk" target="/Outputs/vtkOutput"/>
<!--
<PeriodicEvent name="tsk_ux_np1" target="/Tasks/displacementxCollection"/>
<PeriodicEvent name="out_ux_np1" target="/Outputs/timeHistoryOutputx"/>
<PeriodicEvent name="tsk_uy_np1" target="/Tasks/displacementyCollection"/>
<PeriodicEvent name="out_uy_np1" target="/Outputs/timeHistoryOutputy"/>
<PeriodicEvent name="tsk_uz_np1" target="/Tasks/displacementzCollection"/>
<PeriodicEvent name="out_uz_np1" target="/Outputs/timeHistoryOutputz"/>
<PeriodicEvent name="tsk_p_np1" target="/Tasks/pressureCollection"/>
<PeriodicEvent name="out_p_np1" target="/Outputs/timeHistoryOutput"/>
-->
</PeriodicEvent>
<!-- control the timestepping here with forceDt -->
<PeriodicEvent
name="solidfluidSolverApplications"
forceDt="1e-4"
target="/Solvers/acousticelasticSolver"/>
<!-- FIXME: forceDt hack to satisfy pre-timeloop computation -->
<SoloEvent
name="solidSolverApplications"
forceDt="1e-4"
endTime="0"
target="/Solvers/elasticSolver"/>
<SoloEvent
name="fluidSolverApplications"
forceDt="1e-4"
endTime="0"
target="/Solvers/acousticSolver"/>
</Events>

<ElementRegions>
<CellElementRegion
name="Fluid"
cellBlocks="{ fluid }"
materialList="{ nullModel }"/>
<CellElementRegion
name="Solid"
cellBlocks="{ solid }"
materialList="{ nullModel }"/>
</ElementRegions>

<Constitutive>
<NullModel
name="nullModel"/>
</Constitutive>

<FieldSpecifications>
<!-- initial condition for ρ, 𝓋ₚ and 𝓋ₛ -->
<FieldSpecification
name="acousticVelocity"
initialCondition="1"
objectPath="mesh/FE1/ElementRegions/Fluid/fluid"
fieldName="mediumVelocity"
scale="1500.0"
setNames="{ all }"/>
<FieldSpecification
name="acousticDensity"
initialCondition="1"
objectPath="mesh/FE1/ElementRegions/Fluid/fluid"
fieldName="mediumDensityA"
scale="1020.0"
setNames="{ all }"/>
<FieldSpecification
name="elasticVelocityVp"
initialCondition="1"
objectPath="mesh/FE1/ElementRegions/Solid/solid"
fieldName="mediumVelocityVp"
scale="3400.0"
setNames="{ all }"/>
<FieldSpecification
name="elasticVelocityVs"
initialCondition="1"
objectPath="mesh/FE1/ElementRegions/Solid/solid"
fieldName="mediumVelocityVs"
scale="1963.0"
setNames="{ all }"/>
<FieldSpecification
name="elasticDensity"
initialCondition="1"
objectPath="mesh/FE1/ElementRegions/Solid/solid"
fieldName="mediumDensityE"
scale="2500.0"
setNames="{ all }"/>
</FieldSpecifications>

<Tasks>
<PackCollection
name="pressureCollection"
objectPath="mesh/FE1/nodeManager"
fieldName="pressure_np1"/>
<PackCollection
name="displacementxCollection"
objectPath="mesh/FE1/nodeManager"
fieldName="displacementx_np1"/>
<PackCollection
disableCoordCollection="1"
name="displacementyCollection"
objectPath="mesh/FE1/nodeManager"
fieldName="displacementy_np1"/>
<PackCollection
disableCoordCollection="1"
name="displacementzCollection"
objectPath="mesh/FE1/nodeManager"
fieldName="displacementz_np1"/>
</Tasks>

<Outputs>
<VTK
name="vtkOutput"
levelNames="{ FE1 }"
plotLevel="0"/>
<TimeHistory
name="timeHistoryOutput"
sources="{ /Tasks/pressureCollection }"
filename="pressure_history"/>
<TimeHistory
name="timeHistoryOutputx"
sources="{ /Tasks/displacementxCollection }"
filename="displacementx_history"/>
<TimeHistory
name="timeHistoryOutputy"
sources="{ /Tasks/displacementyCollection }"
filename="displacementy_history"/>
<TimeHistory
name="timeHistoryOutputz"
sources="{ /Tasks/displacementzCollection }"
filename="displacementz_history"/>
</Outputs>
</Problem>
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@
name="cellDensity"
initialCondition="1"
objectPath="ElementRegions/Region/cb"
fieldName="mediumDensity"
fieldName="mediumDensityE"
scale="1"
setNames="{ all }"/>

Expand Down
2 changes: 1 addition & 1 deletion inputFiles/wavePropagation/elas3D_DAS_smoke.xml
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@
name="cellDensity"
initialCondition="1"
objectPath="ElementRegions/Region/cb"
fieldName="mediumDensity"
fieldName="mediumDensityE"
scale="2000"
setNames="{ all }"/>

Expand Down
2 changes: 1 addition & 1 deletion inputFiles/wavePropagation/elas3D_Q3_small_base.xml
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@
name="cellDensity"
initialCondition="1"
objectPath="mesh/FE1/ElementRegions/Region/cb"
fieldName="mediumDensity"
fieldName="mediumDensityE"
scale="1"
setNames="{ all }"/>

Expand Down
2 changes: 1 addition & 1 deletion inputFiles/wavePropagation/elas3D_Q5_small_base.xml
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@
name="cellDensity"
initialCondition="1"
objectPath="mesh/FE1/ElementRegions/Region/cb"
fieldName="mediumDensity"
fieldName="mediumDensityA"
scale="1"
setNames="{ all }"/>

Expand Down
2 changes: 1 addition & 1 deletion inputFiles/wavePropagation/elas3D_small_base.xml
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@
name="cellDensity"
initialCondition="1"
objectPath="ElementRegions/Region/cb"
fieldName="mediumDensity"
fieldName="mediumDensityE"
scale="1"
setNames="{ all }"/>

Expand Down
Loading