Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review the tutorial #21

Merged
merged 5 commits into from
May 20, 2024
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 35 additions & 34 deletions recipes/001_first.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,13 @@
"""
.. _first:

How to implement and register your algorithm in OptunaHub
=========================================================
How to Implement and Register Your Algorithm with OptunaHub
===========================================================

OptunaHub is a registry of third-party Optuna packages.
It provides a platform for users to share their own optimization algorithms and to use others' algorithms.
This recipe shows how to implement and register your own sampling algorithm in OptunaHub.
OptunaHub is an Optuna package registry, which is a user platform to share their optimization algorithms.
This recipe shows how to implement and register your own sampling algorithm with OptunaHub.

How to implement your own algorithm
How to Implement Your Own Algorithm
-----------------------------------

Usually, Optuna provides `BaseSampler` class to implement your own sampler.
Expand All @@ -24,7 +23,7 @@
"""

###################################################################################################
# First of all, import `optuna`, `optunahub`, and other necessary modules.
# First of all, import `optuna`, `optunahub`, and other required modules.
from __future__ import annotations

import os
Expand All @@ -38,9 +37,11 @@

###################################################################################################
# Next, define your own sampler class by inheriting `SimpleSampler` class.
# In this example, we implement a sampler that always returns a random value.
# The `SimpleSampler` class can be loaded using `optunahub.load_module` function.
# The `force_load` argument is set to `True` to force loading the sampler without caching and consent to use stats.
# In this example, we implement a sampler that returns a random value.
# `SimpleSampler` class can be loaded using `optunahub.load_module` function.
# To download the registered sampler, we need to set `force_reload` to `False`.
nabenabe0928 marked this conversation as resolved.
Show resolved Hide resolved
# `force_reload=True` argument forces downloading the sampler from the registry.
# If we set `force_reload` to `False`, we use the cached data in our local storage if available.

SimpleSampler = optunahub.load_module(
"samplers/simple",
Expand All @@ -49,24 +50,24 @@


class MySampler(SimpleSampler): # type: ignore
# The `search_space` argument is necessary for the concrete implementation of the `SimpleSampler` class.
# `search_space` argument is necessary for the concrete implementation of `SimpleSampler` class.
def __init__(self, search_space: dict[str, optuna.distributions.BaseDistribution]) -> None:
super().__init__(search_space)
self._rng = np.random.RandomState()

# You need to implement the `sample_relative` method.
# You need to implement `sample_relative` method.
# This method returns a dictionary of hyperparameters.
# The keys of the dictionary are the names of the hyperparameters, which must be the same as the keys of the `search_space` argument.
# The values of the dictionary are the values of the hyperparameters.
# In this example, the `sample_relative` method returns a dictionary of hyperparameters with random values.
# In this example, `sample_relative` method returns a dictionary of randomly sampled hyperparameters.
def sample_relative(
self,
study: optuna.study.Study,
trial: optuna.trial.FrozenTrial,
search_space: dict[str, optuna.distributions.BaseDistribution],
) -> dict[str, Any]:
# The `search_space` argument is exactly same as the `search_space` argument of the `__init__` method.
# It is automatically handled by Optuna and `SimpleSampler`.
# `search_space` argument must be identical to `search_space` argument input to `__init__` method.
# This method is automatically invoked by Optuna and `SimpleSampler`.

params = {}
for n, d in search_space.items():
Expand All @@ -92,26 +93,26 @@ def objective(trial: optuna.trial.Trial) -> float:

###################################################################################################
# This sampler can be used in the same way as other Optuna samplers.
# In the following example, we create a study and optimize it using the `MySampler` class.
# In the following example, we create a study and optimize it using `MySampler` class.
sampler = MySampler({"x": optuna.distributions.FloatDistribution(-10, 10)})
study = optuna.create_study(sampler=sampler)
study.optimize(objective, n_trials=100)

###################################################################################################
# You can get the best parameter as follows.
# The best parameter can be fetched as follows.

best_params = study.best_params
found_x = best_params["x"]
print("Found x: {}, (x - 2)^2: {}".format(found_x, (found_x - 2) ** 2))
print(f"Found x: {found_x}, (x - 2)^2: {(found_x - 2) ** 2}")

###################################################################################################
# We can see that the ``x`` value found by Optuna is close to the optimal value of ``2``.
# We can see that ``x`` value found by Optuna is close to the optimal value ``2``.

###################################################################################################
# How to register your implemented algorithm in OptunaHub
# -------------------------------------------------------
# How to Register Your Implemented Algorithm with OptunaHub
# ---------------------------------------------------------
#
# After implementing your own algorithm, you can register it in OptunaHub.
# After implementing your own algorithm, you can register it with OptunaHub.
# You need to create a pull request to the `optunahub-registry <https://github.com/optuna/optunahub-registry>`_ repository.
#
# The following is an example of the directory structure of the pull request.
Expand All @@ -128,20 +129,20 @@ def objective(trial: optuna.trial.Trial) -> float:
# | ├── requirements.txt
# | └── YOUR_ALGORITHM_NAME.py
#
# If you implement not visualizarion feature but sampler, you should put your implementation in the `samplers` directory.
# An implemented sampler should be put in the `samplers` directory.
# In the `samplers` directory, you should create a directory named after your algorithm.
# In the directory, you should put the following files:
# The created directory should include the following files:
#
# - `README.md`: A description of your algorithm. This file is used to create an `web page of OptunaHub <TBD>`_. Let me explain the format of the `README.md` file later.
# - `__init__.py`: An initialization file. This file must import your impelemented sampler from `YOUR_ALGORITHM_NAME.py`.
# - `__init__.py`: An initialization file. This file must import your implemented sampler from `YOUR_ALGORITHM_NAME.py`.
# - `LICENSE`: A license file. This file must contain the license of your algorithm. It should be the MIT license in the alpha version.
# - `images`: This is optional. A directory that contains images. You can put images that are used in the `web page of OptunaHub <TBD>`_. Note that the image file named `thumbnail.png` is used as a thumbnail in the web page. If you want to put other images like `screenshot.png` and use them in the `README.md` file, please use the relative path like `images/screenshot.png`.
# - `images`: This is optional. A directory that contains images. The images in this directory will be used the `web page of OptunaHub <TBD>`_. `thumbnail.png` will be used as a thumbnail in the web page. Note that `README.md` can also refer to image files, e.g. `images/screenshot.png`, in this directory.
# - `requirements.txt`: This is optional. A file that contains the additional dependencies of your algorithm. If there are no additional dependencies, you do not need to create this file.
# - `YOUR_ALGORITHM_NAME.py`: Your implemented sampler.
#
# The `README.md` file must contain the following sections:
# `README.md` must contain the following sections:
#
# - An header section written in the following format:
# - A header section written in the following format:
#
# .. code-block:: markdown
#
Expand All @@ -154,22 +155,22 @@ def objective(trial: optuna.trial.Trial) -> float:
# license: 'MIT License'
# ---
#
# - `author`: The author of the package. It can be your name or the name of your organization.
# - `title`: The title of the package. It should not be a class/function name but a human-readable name. For example, `Demo Sampler` is a good title, but `DemoSampler` is not.
# - `author`: The author of the package. It can be your name or your organization name.
# - `title`: The package title. It should not be a class/function name but a human-readable name. For example, `Demo Sampler` is a good title, but `DemoSampler` is not.
# - `description`: A brief description of the package. It should be a one-sentence summary of the package.
# - `tags`: Tags of the package. It should be a list of strings. The tags must include `sampler` or `visualization` depending on the type of the package. You can add other tags as needed. For example, "['sampler', 'LLM]".
# - `tags`: The package tags. It should be a list of strings. The tags must include `sampler` or `visualization` depending on the type of the package. You can add other tags as needed. For example, "['sampler', 'LLM']".
# - `optuna_versions`: A list of Optuna versions that the package supports. It should be a list of strings. For example, "['3.5.0', '3.6.1']".
# - `license`: The license of the package. It should be a string. For example, `'MIT License'`. The license must be `MIT` in the alpha version.
#
# - A `Class or Function Names` section that describes the classes or functions provided by the package. If you provide multiple classes or functions, you should list them in this section. Note that the section must be a markdown list. If you provide only one class or function, you can just write the name of the class or function. Note that the documentation of the classes or functions must be written in the docstrings of the classes or functions. If you want to refer to the documentation, please leavea a link to the source code, or write them in the following `Others` section. For example:
# - `Class or Function Names` section that describes the classes or functions provided by the package. If you provide multiple classes or functions, you should list them in this section. Note that the section must be a markdown list. If you provide only one class or function, you can simply write the class or function name. Note that the documentation of the classes or functions must be written in their docstrings. If you want to refer to the documentation, please leave the source code link, or write them in the following `Others` section. For example:
#
# .. code-block:: markdown
#
# - `DemoSampler1`
# - `DemoSampler2`
# - `demo_function1`
#
# - An `Installation` section that describes how to install the additional dependencies if there are. For example:
# - An `Installation` section that describes how to install the additional dependencies if required. For example:
#
# .. code-block:: markdown
#
Expand All @@ -185,7 +186,7 @@ def objective(trial: optuna.trial.Trial) -> float:
# ```
# See `example.py <path/to/example.py>` for more details.
#
# - An `Others` section that describes other information about the package, like the reference to the original paper or the link to the source code. For example:
# - An `Others` (COMMENT: miscellaneous is better) section that describes supplementary information about the package such as the paper reference or the original source code link. For example:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comment. I will apply this change after this PR.

nabenabe0928 marked this conversation as resolved.
Show resolved Hide resolved
#
# .. code-block:: markdown
#
Expand Down