RAFT DOE to create Surrogate Model (updated)

[yonghoonlee]

Purpose

This draft PR is to implement streamlined process of creating surrogate model out of design of experiment runs within runWEIS code. The reason for training surrogate model is to use in design coupling analysis, and ultimately in the Formulation Tool for WEIS (FTW) being developed by UIUC and U of Memphis.

The whole process is completely parallelized using MPI, so running DOE, processing recorder files, and training surrogate models can be done in any MPI-based parallel environment, including HPC.

PR#300 has an unexpected abrupt merge, so this new PR is created not to include unnecessary complications.

Type of change

What types of change is it?
Select the appropriate type(s) that describe this PR

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (non-backwards-compatible fix or feature)
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no API changes)
• Documentation update
• Maintenance update
• Other (please describe)

Testing

In example 15 directory, run

python weis_driver_level1_doe.py

or if MPI is available, run

mpirun -np [number_of_cores] python weis_driver_level1_doe.py

or if under HPC environment, run slurm script.

Checklist

Put an x in the boxes that apply.

• I have run existing tests which pass locally with my changes
• I have added new tests or examples that prove my fix is effective or that my feature works
• I have added necessary documentation

[yonghoonlee]

I made an unexpected abrupt merge on the previous doe_parallel branch, so I created another branch that only contains the commits before that accidental merge activity, and named doe_surrogate which reflects the contained work better. The code included in this PR remains the same as the previous one.

[yonghoonlee]

@elefer3 Your code seems to have started from release version of WEIS at some point. Would you please start from my doe_surrogate as the starting point of your branch? Thank you.
@saeidb71 You can also start your contribution from this branch: doe_surrogate. Thank you.

[yonghoonlee]

I think it is ready for review.

With the latest updates, the surrogate model is created using sparse Gaussian process (SGP) included in the SMT toolbox, and giving both very stable and accurate prediction, as R^2 values mostly fit within 0.99 or higher, if the parameter is well correlated by its own nature. Other types of Gaussian process methods created stability issues when the sample number increases more than a few hundreds. So, there is no meaning to explore other methodologies.

[dzalkind]

Hi @yonghoonlee, thanks for putting this PR together! It looks like nice work.

Some notes and questions:

• I added a few minor comments about code style and readability.
• Seeing the process demonstrated on a simple case would be good.
• Is the method general or only for the RAFT example.
• Which parts of the method must be in runWEIS? Which can be postprocessing scripts on a DOE and/or before an optimization?

[yonghoonlee]

@dzalkind Thank you for your review.

• Code style and readability comments - where can I check them? I will address your comments once I find them.
• A simple case to create the surrogate model based on RAFT DOE is included in weis_driver_level1_doe.py in Example 15 directory. Associated yaml inputs are in the same file names. Subsequent processes (Surrogate-based optimization, Design coupling analysis) will be implemented in separate pull requests later.
• For now, RAFT only. However, the code considers potential future expansions, so expanding to OpenFAST is not a big problem. Once full design coupling analysis workflow is implemented, I am planning to expand this to OpenFAST as well.
• DOE driver is in runWEIS code, so surrogate model training following DOE driver is also implemented in runWEIS code. Surrogate based optimization and Design coupling analysis can be separate scripts, as they will be optional functionalities.

[dzalkind]

Hi @yonghoonlee, the comments should show up on this page (above): #303

[yonghoonlee]

Hi @yonghoonlee, the comments should show up on this page (above): #303

@dzalkind I cannot find your code comments. Would you please let me know how I can access them? I do not see anything commented on my code from this page (or other places).

[dzalkind]

Can these sets of duplicated code be made into a function?

[yonghoonlee]

Not all variables can be processed in a consistent manner. So, making a function that works with every variables might not be easy.

[yonghoonlee]

@dzalkind This is the only remaining item that needs your attention. Otherwise, all other review comments are addressed.

[yonghoonlee]

A small bug fix in 0fa00a0 for MPI with 1 core. This is particularly necessary when single core is used with slurm.

[yonghoonlee]

Rebased the newest updates from develop branch onto this PR. The PR now needs to be tested again, so please do not merge onto the develop branch.