How to improve convergence of the adjoint linear equation

[xiaopingqiu]

Hello Dr He,
I have some trouble on convergence of the adjoint linear equation, especially when using pressure boundary condition (i.e., for pressure: totalPressure on inlet , fixedValue on outlet; for velocity: pressureInletOutletVelocity on inlet, inletOutlet on outlet) , it is almost impossible to get a convergent solution of the adjoint linear equation. I have tried your suggestions in the documentation, i.e., increase pcFillLevel to 2 or using 'nd' as jacMatReOrdering method, but both are not helping.
So what are the factors that prevent adjoint linear equation from convergence? If my case fail due to divergence of the adjoint linear equation, is there a way to find out the key issue and improve it by tuning some parameters, may be grid quality or normalization of state variables? Thank you.

[friedenhe]

Can you elaborate on which solver you are using, what type of geometry you try to simulate, and what is the flow condition (subsonic or transonic)?

[friedenhe]

You have to use DARhoSimpleFoam if the flow is subsonic. Also, I suggest you try the velocity-inlet + static pressure-outlet boundary conditions (like the NACA0012 case). The total pressure inlet + static pressure outlet boundary condition does not work very well in our adjoint. We had a numerical difficulty and did not have time to fix it. If you really need to prescribe the pressure at the inlet, try the static inlet pressure + static outlet pressure. Refer to the Rotor37 tutorial. Also, keep the mean y+ ~ 30 and use wall functions. This will also help to converge the adjoint.

[xiaopingqiu]

OK, I will try it, thanks.
By the way, when using velocity inlet + static pressure outlet boundary, the adjoint linear equation may also diverge. Sometimes I got converged solution when I use coarse mesh as a trial, but when I refine my grids to a reasonable level, the adjoint linear equation diverge. So if adjoint linear equation diverge, do you have any suggestion on how to figure out the causation? Thanks.

[uttamcadambi07]

@friedenhe Dr. He, I am revisiting this thread because of a point you made here that I need clarity on - you said keep the mean y+ to 30 and use wall functions to help the convergence of the adjoint. Why is that? Why is it that the adjoint struggles to converge if I fully resolve the viscous sublayer? I am struggling with this at the moment for a Reynolds number of 75000. Please do elaborate!

[friedenhe]

It is related to the aspect ratio of the mesh. y+=1 mesh usually has a very large aspect ratio. It makes the adjoint preconditioner have a larger conditioning number, so the adjoint becomes harder to converge in general. We managed to run optimization with many y+=1 cases, the adjoint just converged slower; it just needed more iterations.

[uttamcadambi07]

That is very interesting. I ask this because we managed to converge the adjoint in very few (about 200 iterations) each time for a NACA0018 airfoil with max aspect ratio of 513 by having an average yplus of 0.1. I suppose in the region close to the wall, the aspect ratio of the cells in this case might have been sufficiently low to promote good adjoint solver convergence. But we are currently facing issues wit ha naca0012 airfoil at Reynolds number of 75000. I have added this in a separate thread.

[friedenhe]

How many cells did you have for the refined mesh? How many orders of magnitude did the flow residuals converge for refined mesh? Did you have a large flow separation? If the flow did converge well, but the adjoint still not converging, try to: 1. Change adjStateOrdering to cell and change jacMatReOrdering to natural. 2. Change the div(phi, U) scheme to upwind. 3. Increase the y+ to 50 to 100. 4. Always use pcFillLevel = 2.

[xiaopingqiu]

Thank you for your kind reply.
I tried to change some parameters as you suggested this weekend, but still couldn't get converged solution of adjoint linear equation.
The cell number of the case is 65532, this is not refined mesh, since when using DARhoSimpleFoam, coarse mesh is also difficult to converge. Important boundary conditions are list below:
U:

FoamFile
{
    version     2.1;
    format      ascii;
    class       volVectorField;
    object      U;
}
dimensions      [0      1       -1      0       0       0       0];
internalField   uniform (-8.4   0       0);
boundaryField
{
    ".*"
    {
        type    fixedValue;
        value   uniform (0      0       0);
    }
    motor_10mm_stable_intake_inlet
    {
        type fixedValue;
        value $internalField;
    }
    motor_10mm_stable_outlet
    {
        type    zeroGradient;
    }
    processor
    {
        type    processor;
    }
}

p:

FoamFile
{
    version     2.1;
    format      ascii;
    class       volScalarField;
    object      p;
}
dimensions      [1 -1 -2 0 0 0 0];
internalField   uniform 100000;
boundaryField
{
    ".*"
    {
        type    zeroGradient;
    }
    motor_10mm_stable_intake_inlet
    {
        type zeroGradient;
    }
    motor_10mm_stable_outlet
    {
        type    fixedValue;
        value   uniform 100000;
    }
    processor
    {
        type    processor;
    }
}

daOptions:

# Set the parameters for optimization
daOptions = {
    "solverName": "DARhoSimpleFoam",
    "designSurfaces": ["motor_10mm_stable_intake_pipe"],
    "primalMinResTol": 1e-8,
    "primalVarBounds": {
        "UUpperBound": 500.0,
        "ULowerBound": -500.0,
        "pUpperBound": 1000000.0,
        "pLowerBound": 20000.0,
        "hUpperBound": 500000.0,
        "hLowerBound": 50000.0,
        "rhoUpperBound": 10.0,
        "rhoLowerBound": 0.2,
    },
    "objFunc": {
        "tumble":
        {
            "part1":
            {
                "type": "tumbleRatio",
                "source": "zoneToCell",
                "zone": "cylinderZone",
                "center": [0.0, -0.0655, 0.0],
                "direction": "z",
                "omegac": 1,
                "scale": 1.0,
                "addToAdjoint": True,
            }
        },
    },
    "normalizeStates": {"U": 10.0, "p": 100000.0, "nuTilda": 1e-3, "phi": 1.0, "T": 300.0},
    "adjPartDerivFDStep": {"State": 1e-6, "FFD": 1e-3},
    "adjEqnOption": {
            "globalPCIters": 0,
            "asmOverlap": 1,
            "localPCIters": 1,
            "jacMatReOrdering": "rcm",
            "pcFillLevel": 2,
            "gmresMaxIters": 1000,
            "gmresRestart": 1000,
            "gmresRelTol": 1.0e-6,
            "gmresAbsTol": 1.0e-14,
            "gmresTolDiff": 1.0e2,
            },

    "adjStateOrdering": "cell",
    "transonicPCOption": 1,
    "adjPCLag": 5,#1,
    # Design variable setup

    "designVar": {
        "shapez": {"designVarType": "FFD"},
        "shapey": {"designVarType": "FFD"},
        "shapezouter1": {"designVarType": "FFD"},
        "shapezouter2": {"designVarType": "FFD"},
        "shapeyinner": {"designVarType": "FFD"},
        "shapezinner": {"designVarType": "FFD"},

    },
    #"decomposeParDict": {"preservePatches": ["per1", "per2"]},
    "checkMeshThreshold": {"maxAspectRatio": 1000.0, "maxNonOrth": 75.0, "maxSkewness": 20.0},
}

The residual reached below 1e-8 after the first primal solving,

Time = 7000

U Initial residual: (1.6161394e-09 5.0221876e-09 8.7440836e-09)
U   Final residual: (4.9372304e-11 1.4882732e-10 2.6347086e-10)
he Initial residual: 1.9783492e-08
he   Final residual: 5.8801545e-10
p Initial residual: 1.7894625e-07
p   Final residual: 2.0600593e-09
Time step continuity errors : sum local = 8.2558706e-08
                                 global = -1.6886935e-08
                             cumulative = 0.021668091
nuTilda Initial residual: 4.6218051e-08
          Final residual: 2.1572322e-09
Bounding nuTilda>1e-16
yPlus min: 0 max: 755.58729 mean: 30.649003
tumble-part1-tumbleRatio: 47.164502
ExecutionTime = 1186.08 s  ClockTime = 1187 s

Time = 7006
Minimal residual 9.9187741e-09 satisfied the prescribed tolerance 1e-08

Printing Primal Residual Statistics.
U Residual Norm2: (25.145432 51.966491 61.976007)
U Residual Mean: (0.0019291683 0.0025917082 0.0034602627)
U Residual Max: (12.180764 32.729548 35.071967)
p Residual Norm2: 5.1158801
p Residual Mean: 0.00029077804
p Residual Max: 3.2428395
T Residual Norm2: 252.38373
T Residual Mean: 0.075183102
T Residual Max: 185.17866
nuTilda Residual Norm2: 0.82373678
nuTilda Residual Mean: 2.9436344e-05
nuTilda Residual Max: 0.55890404
phi Residual Norm2: 0.001641679
phi Residual Mean: 7.1033022e-08
phi Residual Max: 0.00080667175
End

but the adjoint linear equation didn't converge,

Solving Linear Euqation... 1791 s
Main iteration 0 KSP Residual norm 1.432066133357e+01 1950 s
Main iteration 3 KSP Residual norm 1.432066133357e+01 1950 s 
Total iterations 3
Solving Linear Equation... Completed! 1950 s
Residual tolerance not satisfied, solution failed!

A detailed log file is attached below
log_adj4.txt

The geometry is a intake pipe of engine, as shown in the figure below,

not the exact geometry, but very similar.

Flow separation occurs around the valve, as shown below

This case converges well when using DASimpleFoam.

[friedenhe]

It seems that you implemented a new objective function, which may be the cause of the issue. To quickly test it, you can use the drag as the objective function and see if it converges. If the drag-adjoint converges, I suggest you set writeJacobians to True in runScript.py, run the adjoint, then you will see the dF/dW vector saved to disk as dFdW_ tumble.dat. Check the values in this file and see if they are normal. If they are all zeros (or very small values), something may be wrong in the new objective function implementation. BTW: no need to set transonicPCOption = 1 for subsonic cases.

[xiaopingqiu]

Yes, I implemented a new objective function, tumbleRatio. I will try as you suggested and update my progress here soon.
Thanks.

[uttamcadambi07]

Hi @xiaopingqiu did you manage to resolve this issue?