Skip to content

Commit

Permalink
Add READM
Browse files Browse the repository at this point in the history
  • Loading branch information
nabenabe0928 committed Dec 3, 2024
1 parent 2c49a24 commit 129ec25
Showing 1 changed file with 22 additions and 44 deletions.
66 changes: 22 additions & 44 deletions package/samplers/ctpe/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ The default parameter set of this sampler is the recommended setup from the pape

- cTPESampler

Although most arguments in [`TPESampler`](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html) are also available for `cTPESampler`, the original paper uses the default arguments of this sampler.
However, `constraints_func` must be provided to use this sampler.
For the reproducibility purpose, please set a fixed `seed`.

## Installation

The version constraint of this package is Optuna v4.0.0 or later.
Expand All @@ -33,62 +37,36 @@ $ pip install -r https://hub.optuna.org/samplers/ctpe/requirements.txt

## Example

TODO: Change here.

This sampler supports the arguments discussed in [the original paper](https://arxiv.org/abs/2304.11127) and can be used in this way.
This sampler is the official implementation of [the c-TPE paper](https://arxiv.org/abs/2211.14411).

```python
import numpy as np
from __future__ import annotations

import numpy as np
import optuna

import optunahub


def objective(trial):
x = trial.suggest_float("x", -5, 5)
y = trial.suggest_int("y", -5, 5)
z = trial.suggest_categorical("z", ["a", "aa", "aaa"])
return len(z) * (x**2 + y**2)


module = optunahub.load_module(package="samplers/tpe_tutorial")
optuna.logging.set_verbosity(optuna.logging.CRITICAL)
arg_choices = {
"consider_prior": [True, False],
"consider_magic_clip": [True, False],
"multivariate": [True, False],
"b_magic_exponent": [0.5, 1.0, 2.0, np.inf],
"min_bandwidth_factor": [0.01, 0.1],
"gamma_strategy": ["linear", "sqrt"],
"weight_strategy": ["uniform", "old-decay", "old-drop", "EI"],
"bandwidth_strategy": ["optuna", "hyperopt", "scott"],
"categorical_prior_weight": [0.1, None],
}
for arg_name, choices in arg_choices.items():
results = []
for choice in choices:
print(arg_name, choice)
sampler = module.CustomizableTPESampler(seed=0, **{arg_name: choice})
study = optuna.create_study(sampler=sampler)
study.optimize(objective, n_trials=100 if arg_name != "b_magic_exponent" else 200)
results.append(study.trials[-1].value)
def objective(trial: optuna.Trial) -> float:
x = trial.suggest_float("x", 0.0, 2 * np.pi)
y = trial.suggest_float("y", 0.0, 2 * np.pi)
return float(np.sin(x) + y)

print(f"Did every setup yield different results for {arg_name}?: {len(set(results)) == len(results)}")

```
def constraints(trial: optuna.trial.FrozenTrial) -> tuple[float]:
x = trial.params["x"]
y = trial.params["y"]
c = float(np.sin(x) * np.sin(y) + 0.95)
trial.set_user_attr("c", c)
return (c, )

In the paper, the following arguments, which do not exist in Optuna, were researched:

- `gamma_strategy`: The splitting algorithm in Table 3. The choices are `linear` and `sqrt`.
- `gamma_beta`: The beta parameter for the splitting algorithm in Table 3. This value must be positive.
- `weight_strategy`: The weighting algorithm in Table 3. The choices are `uniform`, `old-decay`, `old-drop`, and `EI`.
- `categorical_prior_weight`: The categorical bandwidth in Table 3. If `None`, the Optuna default algorithm will be used.
- `bandwidth_strategy`: The bandwidth selection heuristic in Table 6. The choices are `optuna`, `hyperopt`, and `scott`.
- `min_bandwidth_factor`: The minimum bandwidth factor in Table 6. This value must be positive.
- `b_magic_exponent`: The exponent alpha in Table 6. Optuna takes 1.0 by default.
sampler = optunahub.load_module(package="samplers/ctpe").cTPESampler(constraints_func=constraints)
study = optuna.create_study(sampler=sampler)
study.optimize(objective, n_trials=30)
print(study.best_trials)

For more details, please check [the paper](https://arxiv.org/abs/2304.11127).
```

### Bibtex

Expand Down

0 comments on commit 129ec25

Please sign in to comment.