Unable to properly handle the onnx model output in c++

[AndyFitia]

Greeetings everyone,

I really need help...
When I am inferencing the model in pyhton with Ultralytics, the model works fine with great predictions.
But on my c++, I can't figure out how to properly handle the output.
I find myself drawing many squares on the image which do not make any sense at all

.
I also notice something odd: since the output is [1, 16, 8400], I loop until 16 * 8400 with i += 16.
But when i = 33600, the number start to get very low 0.xxxx.

Here is how my code looks like :

const float* outputData = outputTensors[0].GetTensorData<float>();
// Get the shape: [1, 16, 8400]
std::vector<int64_t> outputShape = outputTensors[0].GetTensorTypeAndShapeInfo().GetShape();
size_t elementSize = outputShape[1];
size_t numDetections = outputShape[2];  // 8400 predictions

start = std::chrono::high_resolution_clock::now();
float x, y, w, h;
for (int i = 0; i < numDetections / 4 * elementSize; i+=elementSize) {
    //std::cout << "OutputData[" << i << "] : " << outputData[i] << std::endl;
    x = outputData[i + 0] / 640 * (float)frame.cols;
    y = outputData[i + 1] / 640 * (float)frame.rows;
    w = (outputData[i + 2] - x) / 640 * (float)frame.cols;
    h = (outputData[i + 3] - y) / 640 * (float)frame.rows;
    std::cout << "Output[" << i << "] : " << outputData[i] << std::endl;
    std::cout << "Output[" << i+1 << "] : " << outputData[i+1] << std::endl;
    std::cout << "Output[" << i+2 << "] : " << outputData[i+2] << std::endl;
    std::cout << "Output[" << i+3 << "] : " << outputData[i+3] << std::endl;
    cv::rectangle(frame, cv::Rect2f(x, y, w, h), cv::Scalar(255, 255, 0), 2, 8, 0);
}
end = std::chrono::high_resolution_clock::now();
elapsed = end - start;
std::cout << "Loop duration: " << elapsed.count() << "ms" << std::endl;

cv::imshow("Result", frame);
cv::waitKey(0);

Thanks

[UltralyticsAssistant]

👋 Hello @AndyFitia, thank you for reaching out to Ultralytics 🚀! It sounds like you're encountering an issue with handling the ONNX model output in C++.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it. This will assist our engineers in understanding the exact conditions under which your issue arises.

For handling model outputs effectively, please ensure you are processing the output tensor correctly. It seems there might be a misunderstanding with the bounding box coordinates or the loop logic in your C++ code.

An Ultralytics engineer will assist you soon, but meanwhile, you might consider checking the expected output format and range for your specific model. 🌟

Feel free to engage with the community or explore more about best practices for model inference across different platforms if you're looking into deeper customization or troubleshooting.

We appreciate your patience and enthusiasm! 🙌

[glenn-jocher]

@AndyFitia it looks like you're encountering issues with interpreting the ONNX model output in C++. The output shape [1, 16, 8400] suggests that each detection has 16 attributes, likely including bounding box coordinates and class scores. Ensure you're correctly interpreting these attributes, especially the bounding box coordinates. You might need to adjust your loop to handle these attributes properly. For more detailed guidance, consider checking the ONNX model's documentation or the Ultralytics export guide.

[AndyFitia]

Ok, thank you very much for your answer.
I'll dig into some documentations.

[glenn-jocher]

@AndyFitia you're welcome! If you have more questions, feel free to ask.

[AndyFitia]

This was the doc that helped me. I use Yolo11 but it appears to share the same output as Yolov8
https://medium.com/@reachraktim/how-to-get-bounding-boxes-and-class-probabilities-from-yolov5-and-yolov8-raw-output-4c4d3dd93fae
Thank you.

[glenn-jocher]

It looks like you're on the right track. Ensure you're interpreting the model's output correctly, focusing on the bounding box coordinates and confidence scores. For further guidance, check out the Ultralytics documentation or the YOLO community resources.

[AndyFitia]

😂 I still can't figure out something, in Python with Utralytics and torch with CUDA my model predicts almost well but why in my C++ with OnnxRuntime DirectML, it seems a bit weird. Or is it still related with the way how I handle the output ?
Or is it related to the fact that is it ran on ONNX ?

Python

from ultralytics import YOLO
model = YOLO("Models/best_9_10_2024.pt")
results = model.predict("Test Data/Screenshot (442).png", save=True, imgsz=640)

C++, OnnxRuntime

the class 11 which is for Zebular seems odd, and some predictions are just wrong.

float x_scale_factor = float(frame.cols / 640.0);
float y_scale_factor = float(frame.rows / 640.0);
for (int i = 0; i < numPredictions; i++) {
    // numPredictions = 8400
    double x = *(outputData + i); // data->pointer to raw output
    double y = *(outputData + (numPredictions * 1) + i);
    double w = *(outputData + (numPredictions * 2) + i);
    double h = *(outputData + (numPredictions * 3) + i);

    double left = (x - (0.5 * w)) * x_scale_factor; // convert predictions to original
    double top = (y - (0.5 * y)) * y_scale_factor;
    double width = w * x_scale_factor;
    double height = h * y_scale_factor;


    std::vector<float> cls_confidences;
    for (int j = 0; j < numClasses; j++) {
        cls_confidences.push_back(*(outputData + (numPredictions * (4 + j)) + i));
    }

    unsigned int maxClsIdx = std::max_element(cls_confidences.begin(), cls_confidences.end()) - cls_confidences.begin();
    float maxClsConfidence = cls_confidences[maxClsIdx];
    //std::cout << "maxClsIdx: " << maxClsIdx << std::endl;

    if (maxClsConfidence < 0.3) {
        continue;
    }
    boxes.push_back(cv::Rect2d(left, top, width, height));
    classIDs.push_back(maxClsIdx);
    confidences.push_back(maxClsConfidence);
}
std::vector<int> indices;
float nmsThreshold = 0.4;  // NMS threshold (adjust as needed)

// Perform Non-Maximum Suppression
cv::dnn::NMSBoxes(boxes, confidences, 0.3, nmsThreshold, indices);
for (int idx : indices) {
    cv::Rect2f box = boxes[idx];
    int clsID = classIDs[idx];
    float confidence = confidences[idx];

    // Draw the rectangle
    cv::rectangle(frame, box, cv::Scalar(255, 255, 0), 2);

    // Create the label text
    //std::string label = "Class: " + std::to_string(clsID) + " Conf: " + std::to_string(confidence);
    std::string label = std::format("Class: {}  {:.2f}", clsID, confidence);

    // Put the label above the rectangle
    int baseline = 0;
    cv::Size labelSize = cv::getTextSize(label, cv::FONT_HERSHEY_SIMPLEX, 0.5, 1, &baseline);
    cv::rectangle(frame,
        cv::Rect2d(cv::Point(box.x, box.y - labelSize.height - 5),
            cv::Size(labelSize.width, labelSize.height + 5)),
        cv::Scalar(255, 255, 0), cv::FILLED);
    cv::putText(frame, label, cv::Point(box.x, box.y - 5),
        cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0), 1);
}
cv::imshow("Result", frame);
cv::waitKey(0);

[glenn-jocher]

@AndyFitia it seems like the issue might be related to how the output is being handled in C++. Ensure that the output tensor is being interpreted correctly, especially the bounding box coordinates and class scores. Double-check the scaling factors and the logic for extracting and applying the bounding box coordinates. For further guidance, you might find useful information in the YOLO Common Issues guide: https://docs.ultralytics.com/guides/yolo-common-issues/.

[AndyFitia]

Thank you so much, I found the issue. I was dumb.
I went from :
double top = (y - (0.5 * y)) * y_scale_factor;
to
double top = (y - (0.5 * h)) * y_scale_factor;
I used y instead of h.

Thank you so much for your assistance.

[glenn-jocher]

@AndyFitia glad you found the solution! If you have any more questions, feel free to ask.