Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create a benchmark based on PDEBench #11

Open
Deathn0t opened this issue Mar 29, 2023 · 4 comments
Open

Create a benchmark based on PDEBench #11

Deathn0t opened this issue Mar 29, 2023 · 4 comments
Assignees
@iamyixuan

Comments

@Deathn0t
Copy link
Member

Let's define a benchmark suite based on the PDEBench: https://github.com/pdebench/PDEBench
@Deathn0t
Copy link
Member Author

Deathn0t commented Jun 7, 2023 •
edited
Loading

@iamyixuan advanced on this here: https://github.com/deephyper/benchmark/tree/main/lib/PINNBench/Diffusion-reaction

Next, we need to

  • record learning curves metadata
  • add skip connection capabilities
  • add dataset selection capability DEEPHYPER_BENCHMARK_DATASET=...

@Deathn0t
Copy link
Member Author

Deathn0t commented Jul 7, 2023 •
edited
Loading

  • document the possible values for DEEPHYPER_BENCHMARK_DATASET, what happens if another dataset is used? it looks like the data generation is happening in the installation.
  • for the dataset document the storage size of the dataset
  • fvcore.nn is not part of the installation procedure (not inside the requirements.txt) is it normal?
  • the flops could be an additional objective in MOO
  • would be great to have more parameters to tune such as:
    • regularization hyperparameters
      • dropout_rate
      • early_stopping_patience
      • batch_norm (batch normalization)
      • weight_decay
    • skip_co for skip connections (can be just a boolean with "regular skip co")
    • stochastic weight averaging
    • optimizer

@iamyixuan
Copy link
Contributor

How to check the checkboxes above? Tried many times and only got to check two.

@Deathn0t
Copy link
Member Author

  • make weights as hyperparameters

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@iamyixuan iamyixuan

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Deathn0t @iamyixuan