hyperparameter tuning

[qqyqqyqqy]

Although Optuna is used for hyperparameter tuning in the project, the tuning results may vary across different environments. How can the reproducibility of hyperparameter tuning be ensured?

[tingtingwuwu]

To ensure consistency in hyperparameter tuning results, the following measures can be implemented:

Set Random Seeds: Establish fixed random seeds in all relevant code components (e.g., torch.manual_seed() and numpy.random.seed()) to guarantee the reproducibility of the tuning process.

Log All Experimental Configurations: Automatically save configuration files and parameter combinations in a log file during each hyperparameter tuning run, ensuring that the results can be fully replicated.

Use a Fixed Validation Set: Ensure that the validation set remains unchanged during hyperparameter tuning to avoid discrepancies in tuning outcomes caused by variations in data distribution.

Dockerized Environment: Recommend that users run tuning scripts within a consistent Docker environment to maintain uniformity in dependencies and environment configurations.