Add the implementation of Differential Evolution

[JLX0]

Contributor Agreements

Please read the contributor agreements and if you agree, please click the checkbox below.

• I agree to the contributor agreements.

Motivation

This PR introduces the DESampler, which implements a hybrid Differential Evolution (DE) and Random Sampling algorithm for hyperparameter optimization.

Differential Evolution is a well-established optimization algorithm known for its robustness and performance across numerical optimization tasks. By integrating Random Sampling for categorical parameters, this implementation bridges the gap between numerical and categorical hyperparameter spaces, making it suitable for real-world machine learning and AI applications where mixed-type parameters are common.

The implementation is particularly motivated by the need to handle dynamic search spaces efficiently, enabling flexible adaptation to changing parameter dimensions during optimization. This makes the DESampler well-suited for tasks involving complex search spaces, such as neural architecture search or expensive optimization tasks with evolving requirements.

For simplicity, this implementation provides a default setup of Differential Evolution, with automatic population size determination and seamless integration with Optuna’s framework for tracking trials and results. The sampler ensures computational efficiency while maintaining high diversity in the population, enabling faster convergence to optimal solutions.

Description of the changes

• Added DESampler in de.py
• Added comprehensive examples and benchmarking for using DESampler in example.py

TODO List towards PR Merge

• Copy ./template/ to create your package
• Replace <COPYRIGHT HOLDER> in LICENSE of your package with your name
• Fill out README.md in your package
• Add import statements of your function or class names to be used in __init__.py
• (Optional) Add from __future__ import annotations at the head of any Python files that include typing to support older Python versions
• Apply the formatter based on the tips in README.md
• Check whether your module works as intended based on the tips in README.md

[JLX0]

for Checks/checks:

package/samplers/differential_evolution/example.py:182:9: F841 Local variable z is assigned to but never used
package/samplers/differential_evolution/example.py:199:9: F841 Local variable z is assigned to but never used
package/samplers/differential_evolution/example.py:220:9: F841 Local variable z is assigned to but never used

The variable z is just a dummy variable used to test whether the sampler works for dynamic search space.

[y0z]

LGTM!