Help regarding a custom loss for yolov8

[Shaurya-Rathore]

I am trying to run a custom loss function. Im getting the head output shape as 1,10,8400(6 classes + x y w h). Im unsure what the shapes of the targets and the predictions should be and what steps I am missing between 1,10,8400 to the required prediciton shape. Also in what manner should I be extracting the data from the yaml as in with what split.

[UltralyticsAssistant]

👋 Hello @Shaurya-Rathore, thank you for reaching out to Ultralytics 🚀!

For customizing a loss function with YOLOv8, you're diving into an advanced topic, and we appreciate your interest and initiative! For your specific questions about shape transformations and target preparations, I recommend checking out our detailed Docs for more insights into model architecture and data handling.

If this is a 🐛 Bug Report, please provide a Minimum Reproducible Example (MRE) to help us understand and potentially resolve any issues.

If this is a custom training ❓ question, providing additional context such as the dataset, example images, and any relevant logs can be helpful. Ensure you're considering our Tips for Best Training Results, which might offer some guidance on resolving common pitfalls.

Meanwhile, it might be helpful to connect with our vibrant Ultralytics community! For real-time assistance, you can join our Discord 🎧. Alternatively, for more in-depth conversations, visit our Discourse or check out threads on our Subreddit.

Upgrade

Make sure your ultralytics package is up-to-date, along with all dependencies, in a Python>=3.8 environment with PyTorch>=1.8:

pip install -U ultralytics

Environments

Explore running YOLOv8 in various verified environments, which come with preinstalled dependencies like CUDA, Python, and PyTorch:

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. Refer to the GCP Quickstart Guide
• Amazon Deep Learning AMI. Check out the AWS Quickstart Guide
• Docker Image. Follow the Docker Quickstart Guide

Your query will be addressed by an Ultralytics engineer soon to provide more specific guidance. Thanks again for your patience and interest in leveraging Ultralytics tools! 😊

[glenn-jocher]

@Shaurya-Rathore for custom loss functions in YOLOv8, ensure your predictions and targets match in shape. Predictions should be reshaped to match your target format, typically [batch, num_anchors, num_classes + 4]. Extract data from the YAML using the data argument in your training script. For more guidance, refer to the YOLOv8 documentation.