Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update decoupled MOBO tutorial to use input transforms & Log NEHVI #2488

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

saitcakmak
Copy link
Contributor

Previously, this tutorial used normalize & unnormalize helpers to manually transform the inputs before feeding them to the model & acquisition functions. This requires transforming the data in many places and can be confusing to the users (see #2478). My original goal was to replace this with the Normalize input transform.

  • Replaced normalize & unnormalize with Normalize input transform.
  • Noticed that the tutorial was using non-log NEHVI, so replaced that with LogNEHVI.
  • Running the tutorial was producing extremely noisy plots that didn't show any improvement from any of the methods. This was due to pymoo imports silently failing (pymoo has gone through a BC breaking refactor) and the HV being computed using the brute force approach. The brute force approach seemed too inaccurate, so I removed it. The tutorial now requires pymoo (works with 0.6.1.3, should we fix the version?).

With all these changes, the plots are still sensitive to the optimization budget used in pymoo to compute the HV. I cannot produce plots that differentiate the methods as clearly as the originals.

The internal pymoo version is 0.6.0.1. We also need to upgrade that before this can land.

saitcakmak and others added 6 commits August 23, 2024 09:33
Summary:
Moves the base classes into their own file to avoid circular imports. Previosly, we couldn't import the base class and the log acqfs in the same file without introducing a circular import.

Also fixed typing of `eta`, as it cannot be `None`.

Differential Revision: D61721280
Summary:
Pull Request resolved: #2486

My first idea was to change this to use Log-EHVI, but this helper is used in the tutorial to compute the hypervolume, so we can't change the return value. Since this is not directly constructed by the user, raising a warning is not productive. This diff suppresses the warning.

Differential Revision: D61722790
@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Aug 26, 2024
Copy link

codecov bot commented Aug 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.98%. Comparing base (831072c) to head (2da31e8).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2488   +/-   ##
=======================================
  Coverage   99.98%   99.98%           
=======================================
  Files         193      193           
  Lines       16941    16941           
=======================================
  Hits        16939    16939           
  Misses          2        2           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed Do not delete this pull request or issue due to inactivity.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants