What hyperparameters do you use when tuning?

[saad-palapa]

What is a good turning space?

These are the default hyperparameters from the repo:

default_space = {
    # 'optimizer': tune.choice(['SGD', 'Adam', 'AdamW', 'RMSProp']),
    'lr0': tune.uniform(1e-5, 1e-1),
    'lrf': tune.uniform(0.01, 1.0),  # final OneCycleLR learning rate (lr0 * lrf)
    'momentum': tune.uniform(0.6, 0.98),  # SGD momentum/Adam beta1
    'weight_decay': tune.uniform(0.0, 0.001),  # optimizer weight decay 5e-4
    'warmup_epochs': tune.uniform(0.0, 5.0),  # warmup epochs (fractions ok)
    'warmup_momentum': tune.uniform(0.0, 0.95),  # warmup initial momentum
    'box': tune.uniform(0.02, 0.2),  # box loss gain
    'cls': tune.uniform(0.2, 4.0),  # cls loss gain (scale with pixels)
    'hsv_h': tune.uniform(0.0, 0.1),  # image HSV-Hue augmentation (fraction)
    'hsv_s': tune.uniform(0.0, 0.9),  # image HSV-Saturation augmentation (fraction)
    'hsv_v': tune.uniform(0.0, 0.9),  # image HSV-Value augmentation (fraction)
    'degrees': tune.uniform(0.0, 45.0),  # image rotation (+/- deg)
    'translate': tune.uniform(0.0, 0.9),  # image translation (+/- fraction)
    'scale': tune.uniform(0.0, 0.9),  # image scale (+/- gain)
    'shear': tune.uniform(0.0, 10.0),  # image shear (+/- deg)
    'perspective': tune.uniform(0.0, 0.001),  # image perspective (+/- fraction), range 0-0.001
    'flipud': tune.uniform(0.0, 1.0),  # image flip up-down (probability)
    'fliplr': tune.uniform(0.0, 1.0),  # image flip left-right (probability)
    'mosaic': tune.uniform(0.0, 1.0),  # image mixup (probability)
    'mixup': tune.uniform(0.0, 1.0),  # image mixup (probability)
    'copy_paste': tune.uniform(0.0, 1.0)}  # segment copy-paste (probability)

task_metric_map = {
    'detect': 'metrics/mAP50-95(B)',
    'segment': 'metrics/mAP50-95(M)',
    'classify': 'top1_acc',
    'pose': None}

[glenn-jocher]

@saad-palapa when tuning hyperparameters for YOLOv8, the default search space you've mentioned is a good starting point. It covers a broad range of values for each hyperparameter, allowing for a comprehensive search to find the best combination for your specific dataset and task.

The choice of hyperparameters and their ranges should be guided by the specific characteristics of your dataset, the complexity of the model, and the computational resources available to you. For instance, learning rate (lr0) and its final decay factor (lrf) are critical for convergence, while momentum and weight decay are important for the optimization process.

Augmentation parameters like degrees, translate, scale, shear, hsv_h, hsv_s, hsv_v, flipud, fliplr, mosaic, mixup, and copy_paste can significantly affect the model's ability to generalize to new data, so they should be tuned according to the variability you expect in your application.

The task_metric_map helps in selecting the appropriate metric to focus on during the tuning process, depending on whether you're working on detection, segmentation, classification, or pose estimation.

Remember to monitor the validation metrics closely to ensure that the model is not overfitting and is improving in terms of generalization to unseen data. Happy tuning! 🚀