Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MPI logic is not working with RAFT (level 1) optimization #324

Open
yonghoonlee opened this issue Nov 26, 2024 · 10 comments
Open

MPI logic is not working with RAFT (level 1) optimization #324

yonghoonlee opened this issue Nov 26, 2024 · 10 comments

Comments

@yonghoonlee
Copy link
Contributor

Description

The new MPI implementation utilizing the OpenMDAO's MPI object seems to work fine with OpenFAST examples, but it is not working with level 1 RAFT examples.

Steps to reproduce issue

  1. Install the current develop branch and move to examples/15_RAFT_Studies directory
  2. Run mpirun -np 4 python weis_driver_raft_opt.py

Current behavior

Using weis.aeroelasticse in rosco.toolbox...
Traceback (most recent call last):
  File "/Users/username/work/WEIS2/examples/15_RAFT_Studies/weis_driver_raft_opt.py", line 16, in <module>
Using weis.aeroelasticse in rosco.toolbox...
Traceback (most recent call last):
  File "/Users/username/work/WEIS2/examples/15_RAFT_Studies/weis_driver_raft_opt.py", line 16, in <module>
Using weis.aeroelasticse in rosco.toolbox...
Traceback (most recent call last):
  File "/Users/username/work/WEIS2/examples/15_RAFT_Studies/weis_driver_raft_opt.py", line 16, in <module>
Using weis.aeroelasticse in rosco.toolbox...
    wt_opt, modeling_options, opt_options = run_weis(fname_wt_input, fname_modeling_options, fname_analysis_options)
Traceback (most recent call last):
  File "/Users/username/work/WEIS2/examples/15_RAFT_Studies/weis_driver_raft_opt.py", line 16, in <module>
    wt_opt, modeling_options, opt_options = run_weis(fname_wt_input, fname_modeling_options, fname_analysis_options)
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/work/WEIS2/weis/glue_code/runWEIS.py", line 170, in run_weis
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/work/WEIS2/weis/glue_code/runWEIS.py", line 170, in run_weis
    wt_opt, modeling_options, opt_options = run_weis(fname_wt_input, fname_modeling_options, fname_analysis_options)
    wt_opt, modeling_options, opt_options = run_weis(fname_wt_input, fname_modeling_options, fname_analysis_options)
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/work/WEIS2/weis/glue_code/runWEIS.py", line 170, in run_weis
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/username/work/WEIS2/weis/glue_code/runWEIS.py", line 170, in run_weis
    wt_opt.setup(derivatives=False)
    wt_opt.setup(derivatives=False)
    wt_opt.setup(derivatives=False)
    wt_opt.setup(derivatives=False)
  File "/Users/username/miniforge3/envs/weis-env/lib/python3.12/site-packages/openmdao/core/problem.py", line 1055, in setup
  File "/Users/username/miniforge3/envs/weis-env/lib/python3.12/site-packages/openmdao/core/problem.py", line 1055, in setup
  File "/Users/username/miniforge3/envs/weis-env/lib/python3.12/site-packages/openmdao/core/problem.py", line 1055, in setup
  File "/Users/username/miniforge3/envs/weis-env/lib/python3.12/site-packages/openmdao/core/problem.py", line 1055, in setup
    model._setup(model_comm, self._metadata)
  File "/Users/username/miniforge3/envs/weis-env/lib/python3.12/site-packages/openmdao/core/group.py", line 748, in _setup
    model._setup(model_comm, self._metadata)
  File "/Users/username/miniforge3/envs/weis-env/lib/python3.12/site-packages/openmdao/core/group.py", line 748, in _setup
    model._setup(model_comm, self._metadata)
  File "/Users/username/miniforge3/envs/weis-env/lib/python3.12/site-packages/openmdao/core/group.py", line 748, in _setup
    model._setup(model_comm, self._metadata)
  File "/Users/username/miniforge3/envs/weis-env/lib/python3.12/site-packages/openmdao/core/group.py", line 748, in _setup
    self._setup_procs(self.pathname, comm, self._problem_meta)
    self._setup_procs(self.pathname, comm, self._problem_meta)
  File "/Users/username/miniforge3/envs/weis-env/lib/python3.12/site-packages/openmdao/core/group.py", line 580, in _setup_procs
    self._setup_procs(self.pathname, comm, self._problem_meta)
    self._setup_procs(self.pathname, comm, self._problem_meta)
  File "/Users/username/miniforge3/envs/weis-env/lib/python3.12/site-packages/openmdao/core/group.py", line 580, in _setup_procs
  File "/Users/username/miniforge3/envs/weis-env/lib/python3.12/site-packages/openmdao/core/group.py", line 580, in _setup_procs
  File "/Users/username/miniforge3/envs/weis-env/lib/python3.12/site-packages/openmdao/core/group.py", line 580, in _setup_procs
    raise RuntimeError(msg)
RuntimeError: <model> <class Group>: num_par_fd = 4 but FD is not active.
    raise RuntimeError(msg)
    raise RuntimeError(msg)
RuntimeError: <model> <class Group>: num_par_fd = 4 but FD is not active.
    raise RuntimeError(msg)
RuntimeError: <model> <class Group>: num_par_fd = 4 but FD is not active.
RuntimeError: <model> <class Group>: num_par_fd = 4 but FD is not active.

Expected behavior

Runs well with utilizing all cores in finite differencing

Code versions

# packages in environment at /Users/username/miniforge3/envs/weis-env:
#
# Name                    Version                   Build  Channel
alabaster                 1.0.0              pyhd8ed1ab_0    conda-forge
ampl-mp                   3.1.0             hbec66e7_1006    conda-forge
anyio                     4.6.2.post1              pypi_0    pypi
appnope                   0.1.4                    pypi_0    pypi
argon2-cffi               23.1.0                   pypi_0    pypi
argon2-cffi-bindings      21.2.0                   pypi_0    pypi
arrow                     1.3.0                    pypi_0    pypi
asttokens                 2.4.1                    pypi_0    pypi
async-lru                 2.0.4                    pypi_0    pypi
attrs                     24.2.0             pyh71513ae_0    conda-forge
babel                     2.16.0             pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.12.3                   pypi_0    pypi
bleach                    6.2.0                    pypi_0    pypi
blinker                   1.9.0              pyhff2d567_0    conda-forge
brotli                    1.1.0                hd74edd7_2    conda-forge
brotli-bin                1.1.0                hd74edd7_2    conda-forge
brotli-python             1.1.0           py312hde4cb15_2    conda-forge
bzip2                     1.0.8                h99b78c6_7    conda-forge
c-ares                    1.34.3               h5505292_1    conda-forge
ca-certificates           2024.8.30            hf0a4a13_0    conda-forge
certifi                   2024.8.30          pyhd8ed1ab_0    conda-forge
cffi                      1.17.1          py312h0fad829_0    conda-forge
charset-normalizer        3.4.0              pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.2.2                    pypi_0    pypi
contourpy                 1.3.1           py312hb23fbb9_0    conda-forge
control                   0.10.1             pyhd8ed1ab_0    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
dash                      2.18.1             pyhd8ed1ab_0    conda-forge
dash-bootstrap-components 1.6.0                    pypi_0    pypi
dash-core-components      2.0.0                    pypi_0    pypi
dash-html-components      2.0.0                    pypi_0    pypi
dash-mantine-components   0.15.0                   pypi_0    pypi
dash-table                5.0.0                    pypi_0    pypi
dearpygui                 2.0.0                    pypi_0    pypi
debugpy                   1.8.9                    pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
defusedxml                0.7.1                    pypi_0    pypi
dill                      0.3.9              pyhd8ed1ab_0    conda-forge
docopt                    0.6.2                    pypi_0    pypi
docutils                  0.21.2             pyhd8ed1ab_0    conda-forge
et_xmlfile                2.0.0              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.2.2              pyhd8ed1ab_0    conda-forge
executing                 2.1.0                    pypi_0    pypi
fastjsonschema            2.20.0                   pypi_0    pypi
fatpack                   0.7.7              pyhd8ed1ab_0    conda-forge
fftw                      3.3.10          mpi_mpich_hd148bab_10    conda-forge
flask                     3.0.3                    pypi_0    pypi
fonttools                 4.55.0          py312h998013c_0    conda-forge
fqdn                      1.5.1                    pypi_0    pypi
freetype                  2.12.1               hadb7bae_2    conda-forge
gmp                       6.3.0                h7bae524_2    conda-forge
h11                       0.14.0                   pypi_0    pypi
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
hdf5                      1.14.4          mpi_mpich_h13a04de_3    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
httpcore                  1.0.7                    pypi_0    pypi
httpx                     0.27.2                   pypi_0    pypi
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
hypre                     2.32.0          mpi_mpich_h189fe77_1    conda-forge
idna                      3.10               pyhd8ed1ab_0    conda-forge
imagesize                 1.4.1              pyhd8ed1ab_0    conda-forge
importlib-metadata        8.5.0              pyha770c72_0    conda-forge
importlib_resources       6.4.5              pyhd8ed1ab_0    conda-forge
iniconfig                 2.0.0              pyhd8ed1ab_0    conda-forge
ipopt                     3.14.16             h920a7db_10    conda-forge
ipykernel                 6.29.5                   pypi_0    pypi
ipython                   8.29.0                   pypi_0    pypi
ipywidgets                8.1.5                    pypi_0    pypi
isoduration               20.11.0                  pypi_0    pypi
itsdangerous              2.2.0              pyhd8ed1ab_0    conda-forge
jedi                      0.19.2                   pypi_0    pypi
jenn                      1.0.8              pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
joblib                    1.4.2              pyhd8ed1ab_0    conda-forge
json5                     0.9.28                   pypi_0    pypi
jsonmerge                 1.9.2              pyhd8ed1ab_0    conda-forge
jsonpointer               3.0.0           py312h81bd7bf_1    conda-forge
jsonschema                4.23.0             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2024.10.1          pyhd8ed1ab_0    conda-forge
jupyter                   1.1.1                    pypi_0    pypi
jupyter-client            8.6.3                    pypi_0    pypi
jupyter-console           6.6.3                    pypi_0    pypi
jupyter-core              5.7.2                    pypi_0    pypi
jupyter-events            0.10.0                   pypi_0    pypi
jupyter-lsp               2.2.5                    pypi_0    pypi
jupyter-server            2.14.2                   pypi_0    pypi
jupyter-server-terminals  0.5.3                    pypi_0    pypi
jupyterlab                4.2.6                    pypi_0    pypi
jupyterlab-pygments       0.3.0                    pypi_0    pypi
jupyterlab-server         2.27.3                   pypi_0    pypi
jupyterlab-widgets        3.0.13                   pypi_0    pypi
kiwisolver                1.4.7           py312h6142ec9_0    conda-forge
krb5                      1.21.3               h237132a_0    conda-forge
lcms2                     2.16                 ha0e7c42_0    conda-forge
lerc                      4.0.0                h9a09cb3_0    conda-forge
libaec                    1.1.3                hebf3989_0    conda-forge
libblas                   3.9.0           25_osxarm64_openblas    conda-forge
libbrotlicommon           1.1.0                hd74edd7_2    conda-forge
libbrotlidec              1.1.0                hd74edd7_2    conda-forge
libbrotlienc              1.1.0                hd74edd7_2    conda-forge
libcblas                  3.9.0           25_osxarm64_openblas    conda-forge
libcurl                   8.10.1               h13a7ad3_0    conda-forge
libcxx                    19.1.4               ha82da77_0    conda-forge
libdeflate                1.22                 hd74edd7_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h93a5062_2    conda-forge
libexpat                  2.6.4                h286801f_0    conda-forge
libfabric                 1.22.0               h5505292_2    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0           13_2_0_hd922786_3    conda-forge
libgfortran5              13.2.0               hf226fd6_3    conda-forge
libhwloc                  2.11.2          default_hbce5d74_1001    conda-forge
libiconv                  1.17                 h0d3ecfb_2    conda-forge
libjpeg-turbo             3.0.0                hb547adb_1    conda-forge
liblapack                 3.9.0           25_osxarm64_openblas    conda-forge
libnghttp2                1.64.0               h6d7220d_0    conda-forge
libopenblas               0.3.28          openmp_hf332438_1    conda-forge
libosqp                   0.6.3                h5833ebf_1    conda-forge
libpng                    1.6.44               hc14010f_0    conda-forge
libptscotch               7.0.5                h1b0e71d_3    conda-forge
libqdldl                  0.1.7                hb7217d7_0    conda-forge
libscotch                 7.0.5                hd19769c_3    conda-forge
libsodium                 1.0.20               h99b78c6_0    conda-forge
libsqlite                 3.47.0               hbaaea75_1    conda-forge
libssh2                   1.11.1               h9cc3647_0    conda-forge
libtiff                   4.7.0                hfce79cd_1    conda-forge
libwebp-base              1.4.0                h93a5062_0    conda-forge
libxcb                    1.17.0               hdb1d25a_0    conda-forge
libxml2                   2.13.5               h376fa9f_0    conda-forge
libzlib                   1.3.1                h8359307_2    conda-forge
llvm-openmp               19.1.4               hdb05f8b_0    conda-forge
markupsafe                3.0.2           py312ha0ccf2a_0    conda-forge
marmot-agents             0.2.5                    pypi_0    pypi
mat4py                    0.6.0              pyhd8ed1ab_0    conda-forge
matplotlib                3.9.2           py312h1f38498_2    conda-forge
matplotlib-base           3.9.2           py312h9bd0bc6_2    conda-forge
matplotlib-inline         0.1.7                    pypi_0    pypi
mdolab-baseclasses        1.8.0              pyhd8ed1ab_0    conda-forge
meson                     1.6.0                    pypi_0    pypi
metis                     5.1.0             h15f6cfe_1007    conda-forge
mistune                   3.0.2                    pypi_0    pypi
moorpy                    1.1.0              pyhd8ed1ab_0    conda-forge
mpfr                      4.2.1                hb693164_3    conda-forge
mpi                       1.0.1                     mpich    conda-forge
mpi4py                    4.0.1           py312h5450814_0    conda-forge
mpich                     4.2.3              h6b0954e_102    conda-forge
mumps-include             5.7.3                hce30654_5    conda-forge
mumps-mpi                 5.7.3                h83d4000_5    conda-forge
mumps-seq                 5.7.3                he17653c_5    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
nbclient                  0.10.0                   pypi_0    pypi
nbconvert                 7.16.4                   pypi_0    pypi
nbformat                  5.10.4                   pypi_0    pypi
ncurses                   6.5                  h7bae524_1    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
networkx                  3.4.2              pyh267e887_2    conda-forge
ninja                     1.11.1.2                 pypi_0    pypi
nlopt                     2.9.0           py312hace9f44_0    conda-forge
notebook                  7.2.2                    pypi_0    pypi
notebook-shim             0.2.4                    pypi_0    pypi
numpy                     1.26.4          py312h8442bc7_0    conda-forge
numpydoc                  1.8.0              pyhd8ed1ab_0    conda-forge
openfast                  3.5.4                h2c1d878_1    conda-forge
openjpeg                  2.5.2                h9f1df11_0    conda-forge
openmdao                  3.35.0             pyhd8ed1ab_0    conda-forge
openpyxl                  3.1.5           py312hf6e0af7_1    conda-forge
openraft                  1.3.1              pyhd8ed1ab_0    conda-forge
openssl                   3.4.0                h39f12f2_0    conda-forge
orjson                    3.10.12         py312hcd83bfe_0    conda-forge
osqp                      0.6.7.post3     py312h72a45a2_1    conda-forge
overrides                 7.7.0                    pypi_0    pypi
packaging                 24.2               pyhff2d567_1    conda-forge
pandas                    2.2.2           py312h8ae5369_1    conda-forge
pandocfilters             1.5.1                    pypi_0    pypi
parmetis                  4.0.3             ha4b917a_1007    conda-forge
parso                     0.8.4                    pypi_0    pypi
patsy                     1.0.1              pyhff2d567_0    conda-forge
pcrunch                   1.1.0              pyhd8ed1ab_0    conda-forge
petsc                     3.22.1          real_hda3b05f_101    conda-forge
petsc4py                  3.22.1          py312h948044d_0    conda-forge
pexpect                   4.9.0                    pypi_0    pypi
pillow                    11.0.0          py312haf37ca6_0    conda-forge
pip                       24.3.1             pyh8b19718_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.3.6                    pypi_0    pypi
plotly                    5.24.1             pyhd8ed1ab_0    conda-forge
pluggy                    1.5.0              pyhd8ed1ab_0    conda-forge
prometheus-client         0.21.0                   pypi_0    pypi
prompt-toolkit            3.0.48                   pypi_0    pypi
psutil                    6.1.0                    pypi_0    pypi
pthread-stubs             0.4               hd74edd7_1002    conda-forge
ptyprocess                0.7.0                    pypi_0    pypi
pure-eval                 0.2.3                    pypi_0    pypi
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pydoe3                    1.0.4              pyhd8ed1ab_0    conda-forge
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pyhams                    1.3.0           py312hf9c0c93_2    conda-forge
pyoptsparse               2.11.3          py312hee86bde_0    conda-forge
pyparsing                 3.2.0              pyhd8ed1ab_1    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
pytest                    8.3.3              pyhd8ed1ab_0    conda-forge
python                    3.12.7          h739c21a_0_cpython    conda-forge
python-benedict           0.33.1                   pypi_0    pypi
python-dateutil           2.9.0.post0        pyhff2d567_0    conda-forge
python-fsutil             0.14.1             pyhd8ed1ab_0    conda-forge
python-json-logger        2.0.7                    pypi_0    pypi
python-slugify            8.0.4              pyhd8ed1ab_0    conda-forge
python-tzdata             2024.2             pyhd8ed1ab_0    conda-forge
python_abi                3.12                    5_cp312    conda-forge
pytz                      2024.2             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.2           py312h024a12e_1    conda-forge
pyzmq                     26.2.0          py312hf8a1cbd_3    conda-forge
qdldl-python              0.1.7.post4     py312hcd31e36_1    conda-forge
qhull                     2020.2               h420ef59_5    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
referencing               0.35.1             pyhd8ed1ab_0    conda-forge
requests                  2.32.3             pyhd8ed1ab_0    conda-forge
retrying                  1.3.3              pyhd8ed1ab_3    conda-forge
rfc3339-validator         0.1.4                    pypi_0    pypi
rfc3986-validator         0.1.1                    pypi_0    pypi
rosco                     2.9.4           py312h8ae986c_2    conda-forge
rpds-py                   0.21.0          py312hcd83bfe_0    conda-forge
ruamel.yaml               0.18.6          py312h0bf5046_1    conda-forge
ruamel.yaml.clib          0.2.8           py312h0bf5046_1    conda-forge
scalapack                 2.2.0                h71a4f75_4    conda-forge
scikit-learn              1.5.2           py312h387f99c_1    conda-forge
scipy                     1.14.1          py312h20deb59_1    conda-forge
send2trash                1.8.3                    pypi_0    pypi
setuptools                75.6.0             pyhff2d567_1    conda-forge
simpy                     4.1.1              pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
slycot                    0.6.0           py312h42037e1_2    conda-forge
smt                       2.7.0           py312h02baea5_0    conda-forge
sniffio                   1.3.1                    pypi_0    pypi
snowballstemmer           2.2.0              pyhd8ed1ab_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.6                      pypi_0    pypi
sphinx                    8.1.3              pyhd8ed1ab_0    conda-forge
sphinxcontrib-applehelp   2.0.0              pyhd8ed1ab_0    conda-forge
sphinxcontrib-devhelp     2.0.0              pyhd8ed1ab_0    conda-forge
sphinxcontrib-htmlhelp    2.1.0              pyhd8ed1ab_0    conda-forge
sphinxcontrib-jsmath      1.0.1              pyhd8ed1ab_0    conda-forge
sphinxcontrib-qthelp      2.0.0              pyhd8ed1ab_0    conda-forge
sphinxcontrib-serializinghtml 1.1.10             pyhd8ed1ab_0    conda-forge
sqlitedict                2.1.0              pyhd8ed1ab_0    conda-forge
stack-data                0.6.3                    pypi_0    pypi
statsmodels               0.14.4          py312h755e627_0    conda-forge
suitesparse               7.8.3                h12cb078_1    conda-forge
superlu                   5.2.2                hc615359_0    conda-forge
superlu_dist              9.1.0                h89afcdd_0    conda-forge
tabulate                  0.9.0              pyhd8ed1ab_1    conda-forge
tbb                       2022.0.0             h0cbf7ec_0    conda-forge
tenacity                  9.0.0              pyhd8ed1ab_0    conda-forge
terminado                 0.18.1                   pypi_0    pypi
text-unidecode            1.3                pyhd8ed1ab_1    conda-forge
threadpoolctl             3.5.0              pyhc1e730c_0    conda-forge
tinycss2                  1.4.0                    pypi_0    pypi
tk                        8.6.13               h5083fa2_1    conda-forge
tomli                     2.1.0              pyhff2d567_0    conda-forge
tornado                   6.4.2           py312hea69d52_0    conda-forge
traitlets                 5.14.3                   pypi_0    pypi
treon                     0.1.4                    pypi_0    pypi
types-python-dateutil     2.9.0.20241003           pypi_0    pypi
typing-extensions         4.12.2               hd8ed1ab_0    conda-forge
typing_extensions         4.12.2             pyha770c72_0    conda-forge
tzdata                    2024b                hc8b5060_0    conda-forge
unicodedata2              15.1.0          py312h0bf5046_1    conda-forge
unixodbc                  2.3.12               h0e2417a_0    conda-forge
uri-template              1.3.0                    pypi_0    pypi
urllib3                   2.2.3              pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.13                   pypi_0    pypi
webcolors                 24.11.1                  pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
websocket-client          1.8.0                    pypi_0    pypi
weis                      1.4                      pypi_0    pypi
werkzeug                  3.0.6                    pypi_0    pypi
wheel                     0.45.1             pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.13                   pypi_0    pypi
wisdem                    3.18.0          nompi_hedc07dd_101    conda-forge
xorg-libxau               1.0.11               hd74edd7_1    conda-forge
xorg-libxdmcp             1.1.5                hd74edd7_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
yaml                      0.2.5                h3422bc3_2    conda-forge
zeromq                    4.3.5                hc1bb282_7    conda-forge
zipp                      3.21.0             pyhd8ed1ab_0    conda-forge
zstandard                 0.23.0          py312h15fbf35_1    conda-forge
zstd                      1.5.6                hb46c0d2_0    conda-forge
@yonghoonlee
Copy link
Contributor Author

@dzalkind This is the issue that I discussed with you today. I will follow up with this issue once I figure out more details. Thank you.

@dzalkind
Copy link
Collaborator

I think part of the issue here is that the opt_flag is not True here:

if opt_options['opt_flag']:

If the optimization flag in the analysis options is set to True, this example works.

However, we should be able to detect the flags in the design variables. Looking into that next.

@dzalkind
Copy link
Collaborator

I think, based on the schema, this condition should always be true: https://github.com/WISDEM/WISDEM/blob/992076a14bf2f18defc1dd5be17e72c89e83431f/wisdem/glue_code/gc_LoadInputs.py#L650

@gbarter or @ptrbortolotti, has anything changed in this area or should we do that recursive search regardless?

@gbarter
Copy link
Member

gbarter commented Nov 27, 2024

I think, based on the schema, this condition should always be true: https://github.com/WISDEM/WISDEM/blob/992076a14bf2f18defc1dd5be17e72c89e83431f/wisdem/glue_code/gc_LoadInputs.py#L650

@gbarter or @ptrbortolotti, has anything changed in this area or should we do that recursive search regardless?

I think that line predates the schema. I agree that with the schema and enforcement of defaults, the check is unnecessary.

@ptrbortolotti
Copy link
Collaborator

I also don't like it. I am not sure how that check made its way into the code, but essentially if any DV is set to be optimized, the opt solver flag is set to True regardless of the user input. do we want that? I like to have that master switch turning on and off optimizations in my analysis options yamls, especially because it helps switch between optimization and DOE. all in all, I'm in favor of changing those four line to
self.analysis_options["opt_flag"] = self.analysis_options["driver"]["optimization"]["flag"], remove the if and more importantly the else and the recursive_flag

@dzalkind
Copy link
Collaborator

I agree that it's confusing when flags are changed by the software and not the user.

I believe the code is never going to the recursive_flag() function, so it is functioning as you prefer, and no change is required, unless we want to be 100% sure with the small change you suggest.

@yonghoonlee
Copy link
Contributor Author

yonghoonlee commented Nov 30, 2024

@dzalkind I didn't pay attention to the schema before, but even if DV is flagged, it would be ideal to let users easily switch to do the optimization or not. Thank you for figuring this out. At this moment, I can just be consistent in setting the opt_flag along with what I want to solve.

@yonghoonlee
Copy link
Contributor Author

@dzalkind Another problem I wanted to bring up is that it behaves differently when MPI is used or not. I think the code should behave consistently whether or not the MPI is used.

@dzalkind
Copy link
Collaborator

dzalkind commented Dec 2, 2024

I agree with that. We can address that in our upcoming MPI work.

What exactly is happening that's different? Is it trying to optimize just because we're using MPI?

@AthulKrishnaSundarrajan
Copy link
Contributor

A different but related issue is when we need to run OpenFAST/DFSM simulations in parallel as part of the DOE studies. I'm adding a workaround I found for this issue, per @dzalkind's directions.

Running DOE studies with OpenFAST simulations in parallel hangs when generating the turbsim files. The workaround that runs just the openfast simulations in parallel for every DOE iteration is as follows:

  1. Generate the turbsim files in serial, and copy and paste the 'wind' directory to the 'rank_0' directory in OF_run_dir.
  2. Comment out this specific line in openmdao_openfast.py:
    self.FAST_namingOut = self.FAST_InputFile+'_%000d'%int(rank)
  3. In runWEIS.py : set n_FD and n_DV to be 1, update the code to be False in lines line 89 line 139 when running level3 simulations
  4. Change line 241 from (not opt_options['driver']['design_of_experiments']['flag']) to (opt_options['driver']['design_of_experiments']['flag']) so that the other subprocessors stay up.

It would be nice if this issue can also be fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants