YOLOv8: model misses 5-10% of similar objects in clustered formations

[augusts-bit]

We have trained a YOLOv8 nano model on 5000+ images with each 1 to 50+ birds from a top down view (drone footage). Since the birds are seemingly 'easy' objects to detect (they are all white with black on top), we really doubt the issue here is the (size of the) training data. Moreover, we have the checked the dataset multiple times for noise and errors, which of course will remain here and there but we think it is negligble. We have had a similar problem using RCNN models.

The birds breed in clustered formations. When we apply the model on new drone footage (inference), there is often 5-10% of the birds that is missed. And we fail to really understand why. The missed birds are logically (almost exactly) similar to birds that are detected and are seemingly randomly missed. Below a few close ups of predictions to illustrate this, with yellow outline to birds that are missed.

For further reference, below is an image of the training:

Since drone mosaics are large, we tile them to use during training and prediction, with each tile a size of about (mostly) 5 to (some times) 10 meters. During training, we resize the images to slightly lower than the average pixel size. During prediction, we use the exact size of the slices. To increase the training data, we have added some overlap here and there between the tiles used for training. During prediction, we standard use overlap (varying 10 to 50%) but we still encounter the same problem. We have tested varying tile sizes and confidence thresholds as low as 10%. The drone imagery used for training has been taken at varying locations (bushy, sandy, etc.) and the tested imagery above is actually from a location that is included. Despite all this, we fail to see the exact root cause.

We are therefore interested in whether people here have some experience in this issue or have suggestions. Why are some birds missed, even though they are nearly 100% identical to birds that are detected? I can clarify further things if needed.

Thanks in advance.

[glenn-jocher]

@augusts-bit it sounds like a challenging issue. You might want to experiment with adjusting the non-maximum suppression (NMS) threshold or using a different model size to see if it improves detection in clustered formations. Additionally, consider augmenting your dataset with more varied examples of clustered birds. For more insights, you can refer to our common issues guide: https://docs.ultralytics.com/guides/yolo-common-issues/.