guides/sahi-tiled-inference/

[giscus[bot]]

guides/sahi-tiled-inference/

A comprehensive guide on how to use YOLOv8 with SAHI for standard and sliced inference in object detection tasks.

https://docs.ultralytics.com/guides/sahi-tiled-inference/

[baozh166]

I am using SAHI's Batch Prediction to predict large images (150000*150000). I need to set the --agnostic-nms=True in the Yolov8 to avoid multiple classes being detected for single object. But the SAHI's batch predict function does not accept --agnostic-nms argument for Yolov8. Is there a way to deal with this issue?

[glenn-jocher]

Hey there! 👋

For integrating --agnostic-nms=True with SAHI's batch prediction, you can directly set the agnostic_nms parameter when creating the AutoDetectionModel instance. Here's a quick example:

from sahi import AutoDetectionModel

# Create a YOLOv8 model with agnostic NMS enabled
detection_model = AutoDetectionModel.from_pretrained(
    model_type='yolov8',
    model_path='path/to/yolov8n.pt',
    confidence_threshold=0.3,
    device="cpu",  # or 'cuda:0'
    agnostic_nms=True,
)

Now, when you use detection_model for batch prediction with SAHI, it will apply agnostic NMS during inference. Hope this helps! 😊

Let me know if you have any more questions.

[Sachi9soni]

Hey I was trying SAHI with triton server inference

So I was unable to do it
My methodology is

detection_model = AutoDetectionModel.from_pretrained(
model_type='yolov8',
model_path='http://localhost:8000/yolo'
confidence_threshold=0.3,
device="cpu", # or 'cuda:0')

[baozh166]

I figured out a way to address this issue.

1. Make sure "agnostic-nms=true" in your customer-data-trained Yolov8's args.yaml configuration file.
2. Assign the path of this args.yaml to the model_config_path parameter of SAHI's predict().
3. Set "postprocess_class_agnostic=True" in SAHI's predict().

An example of parameters in SAHI's predict() looks like:

model_path="2-3images_640x640.v1i.yolov8/results_v8x/200_epochs-/weights/best.pt"
model_config_path="2-3images_640x640.v1i.yolov8/results_v8x/200_epochs-/args_for_sahi.yaml" # agnostic_nms=True in the .yaml file
images_dir = "/fullsize_images/in_test/"

predict(
model_type="yolov8",
model_path=model_path,
model_config_path=model_config_path,
model_device="cuda:0", # "cpu" or 'cuda:0'
model_confidence_threshold=0.6,
postprocess_class_agnostic=True,
source=images_dir,
slice_height=640,
slice_width=640,
overlap_height_ratio=0.2,
overlap_width_ratio=0.2,
visual_bbox_thickness=1,
visual_text_size=0.5,
visual_text_thickness=1,
export_pickle=True
)

[glenn-jocher]

@baozh166 hey there! Great job figuring out a solution for the agnostic NMS issue with SAHI and YOLOv8. 👍 Your approach to modify the args.yaml and use the postprocess_class_agnostic parameter in SAHI's predict() function is spot on.

For anyone else encountering a similar issue, here's a quick recap of the solution:

from sahi.predict import predict

# Set your model and config paths
model_path = "path/to/your/model/best.pt"
model_config_path = "path/to/your/config/args_for_sahi.yaml"  # agnostic_nms=True in this file

# Set your images directory
images_dir = "/path/to/your/images/"

# Run SAHI's predict function with the necessary parameters
predict(
    model_type="yolov8",
    model_path=model_path,
    model_config_path=model_config_path,
    model_device="cuda:0",  # or "cpu"
    model_confidence_threshold=0.6,
    postprocess_class_agnostic=True,
    source=images_dir,
    slice_height=640,
    slice_width=640,
    overlap_height_ratio=0.2,
    overlap_width_ratio=0.2,
    # ... other parameters as needed
)

Remember to ensure that agnostic_nms is set to true in your YOLOv8 args.yaml file. This will help prevent multiple classes from being detected for a single object. Happy detecting! 😊

If you need further assistance, feel free to reach out!

[Bharathteja07]

How to use SAHI predict in case of incoming images on-the-fly, rather than processing a batch of images from a folder, and Infered images needs to be visualized in real-time.

[glenn-jocher]

Hey there! 😊

For handling high-resolution images like 5k*4k, SAHI's slicing can indeed help manage the computational load, but as you've noticed, inference time can still be a challenge. If you're looking to use .engine files with SAHI for potentially better performance, here's a workaround since direct support might not be available in SAHI's repo:

1. Export YOLOv8 to ONNX: First, export your YOLOv8 model to ONNX format. This step is straightforward and well-supported.

   yolo export model=yolov8n.pt format=onnx

2. Convert ONNX to TensorRT Engine: Next, use tools like trtexec (comes with TensorRT installation) to convert the ONNX model to a TensorRT engine. This step is where you might see significant inference speed improvements.

   trtexec --onnx=yolov8n.onnx --saveEngine=yolov8n.engine

3. Integrate TensorRT Engine with SAHI: While SAHI might not directly support .engine files, you can load and run inference with the TensorRT engine in your script and then use SAHI for post-processing the results. This involves a bit of custom coding to bridge the gap between raw TensorRT inference outputs and SAHI's expected input format for result stitching.

Here's a pseudo-code snippet to give you an idea:

import tensorrt as trt
from sahi.postprocess.combine import NmsPostprocess

# Load your TensorRT engine
# [Your code to load TensorRT engine and perform inference]

# Assuming 'detections' is the output from your TensorRT inference
# Convert 'detections' to SAHI's format
object_prediction_list = convert_to_sahi_format(detections)

# Use SAHI's NmsPostprocess for combining and post-processing
nms_postprocess = NmsPostprocess(object_prediction_list=object_prediction_list)
combined_results = nms_postprocess.run()

# Now, you can visualize or further process 'combined_results'

This approach requires a bit of manual effort to convert and handle the inference results, but it can be a powerful way to leverage the speed of TensorRT with the flexibility and utility of SAHI.

Hope this helps you proceed with your project! If you have more questions or need further assistance, feel free to ask. Happy coding! 🚀

[ram-12358]

TypeError: model='/content/best (10).engine' should be a *.pt PyTorch model to run this method, but is a different format.

from sahi.predict import get_sliced_prediction
from sahi import AutoDetectionModel

model_path = r"/content/best (10).engine"

Load the YOLOv8 model

detection_model = AutoDetectionModel.from_pretrained(
model_type="yolov8",
model_path=model_path,
image_size= 640,
confidence_threshold=0.3,
device='cuda:0', # or 'cuda:0'
)

[glenn-jocher]

@ram-12358 sAHI currently supports PyTorch models for inference. To use a TensorRT engine file, you'll need to handle the inference separately using TensorRT APIs and then pass the results to SAHI for post-processing. Unfortunately, direct support for .engine files isn't available in SAHI. For more details on integrating TensorRT, you might need to explore custom implementations.

[lajuardhie]

so SAHI can only supports Pytorch? it doesn't support Tensorflow yet? cause I have a project and the output model is tflite file.

[glenn-jocher]

SAHI currently supports PyTorch models, and direct support for TensorFlow or TFLite is not available. For TFLite models, consider using TensorFlow Lite's own inference tools.

[Zeyad-Saadawy]

I would like to use yolov8 and sahi for video detection and every time i use cv2 for video processing, the output video does not return the same detection as each frame individually if taken as image. If you can please provide a code to do it or help with the logic

[pderrenger]

@tayesh21 it sounds like you're encountering an issue where cv2-processed video frames do not yield the same detection results as individual images. This discrepancy can occur due to differences in image processing or slicing techniques when handling video streams as opposed to static images.

To ensure consistency in detection across both video frames and individual images, I recommend incorporating SAHI's sliced inference approach for video frame processing. This method involves breaking down each video frame into smaller slices, similar to how you might handle large or high-resolution single images, ensuring that detection accuracy remains high even in constrained computational environments.

Here's a quick code snippet to get you started with SAHI and YOLOv8 for video detection, focusing on video frame slicing and processing:

from sahi.predict import get_sliced_prediction
from ultralytics import YOLO

# Load your YOLOv8 model
model = YOLO("path/to/your/yolov8_model.pt")

# Define your video capture
cap = cv2.VideoCapture("path/to/your/video.mp4")

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break
    
    # Perform sliced inference on each frame
    result = get_sliced_prediction(
        image = frame,
        detection_model = model,
        slice_height = 256,
        slice_width = 256,
        overlap_height_ratio = 0.2,
        overlap_width_ratio = 0.2
    )
    
    # Here you can add code to visualize/save the detection results for each frame
    # ...

cap.release()

This example leverages SAHI's get_sliced_prediction for each frame read from the video. Adjust the slice_height, slice_width, overlap_height_ratio, and overlap_width_ratio according to your specific requirements to optimize detection performance.

Make sure to review Ultralytics Docs: Using YOLOv8 with SAHI for Sliced Inference for more details and advanced usage. Hope this helps! Let us know if you have any more questions. 🙂

[joshkulick]

In AutoDetectionModel.from_pretrained argument config should I define image_size as 640 for the best possible YOLO prediction or will YOLO preprocess the image automatically to account for this?

[glenn-jocher]

@joshkulick hey there! 👋 Great question! When using AutoDetectionModel.from_pretrained with YOLOv8, it's generally a good idea to specify the image_size that matches your model's training configuration for optimal performance. However, YOLOv8 is pretty robust and does handle image resizing internally if the input image dimensions differ from the model's expected image_size.

For the best possible YOLO predictions, using the default size (640 for many models) is a solid starting point. If you're working with images that significantly vary in size from what your model was trained on, or if you're aiming for higher accuracy in specific scenarios (like detecting small objects), adjusting the image_size might be beneficial. Just keep in mind that larger image sizes can increase processing time.

Here's a quick example on setting it:

from sahi import AutoDetectionModel

# Load the model, specify the image size if different from the default
detection_model = AutoDetectionModel.from_pretrained(
    model_type='yolov8',
    model_path='path/to/yolov8-model.pt',
    image_size=640,  # Adjust according to your needs
)

Hope this helps! If you have more questions, feel free to ask. Happy detecting! 🚀

[hardworki]

Welcome my friends, i have problem and ican not sloved it, my problem is that object names are large in images, and the borders of the box that defines the object are also large. How can I change them? Or make modifications to them? The other issue is that I only want vehicles objects, meaning cars, buses, and motorcycles. The other issue is that I previously applied Yolo in vscode. When I applied Yolo in vccode, it showed me statistical results with the number of objects. How can this option exist? With many thanks.

[pderrenger]

Hey there! 😊 Thanks for reaching out with your questions, let's tackle them one by one.

1. For large object names and box borders:
   You can adjust the size of the object names and the borders of the detected box by modifying the line thickness and font size in the code where the detections are being visualized. Here's a quick example for PyTorch:

# Assuming 'results' is the output of your model's prediction
results.render()  # Renders predictions on images
for img in results.imgs:
    img = img[:, :, ::-1]  # Convert BGR to RGB
    # Display the image, adjust window and font sizes as needed
    cv2.imshow('image', img)  
    cv2.waitKey(0)
# Note: You might need to dig into the `results.show()` or `results.save()` methods to adjust font and line thickness directly in the source code.

2. To filter only vehicle detections (cars, buses, motorcycles):
   You could filter the results based on the class names or IDs specified in your model's *.yaml file. Here's how you could do it:

car_indices = [results.names.index(n) for n in ['car', 'bus', 'motorcycle']]  # Get class indices for vehicles
vehicle_preds = [pred for pred in results.pred if pred[5] in car_indices]  # Filter predictions

3. For statistical results similar to vscode output:
   If you want to get statistical results (like count of each detected object type), you can achieve this by iterating through your detection results:

# Count each detected class
from collections import Counter
pred_names = [results.names[int(x[5])] for x in results.xyxy[0]]
count = Counter(pred_names)
print(count)

This gives you a dictionary with the object names as keys and their counts as values.

I hope this helps you out! If you have any more questions, feel free to ask. Good luck with your project! 🚀

[hardworki]

I am very grateful for your answer, but I hope that you can explain more to me based on the things that I have available. First, I use the small sized YOLO standard, and the following is the implementation code from sahi.utils.yolov8 import download_yolov8s_model
from sahi import AutoDetectionModel
from sahi.utils.cv import read_image
from sahi.utils.file import download_from_url
from sahi.predict import get_prediction, get_sliced_prediction, predict
from pathlib import Path
from IPython. display import Image

Download YOLOv8 model

yolov8_model_path = "models/yolov8s.pt"
download_yolov8s_model(yolov8_model_path)

Download test images

download_from_url('https://raw.githubusercontent.com/obss/sahi/main/demo/demo_data/small-vehicles1.jpeg', 'demo_data/small-vehicles1.jpeg')
download_from_url('https://raw.githubusercontent.com/obss/sahi/main/demo/demo_data/terrain2.png', 'demo_data/terrain2.png')
detection_model = AutoDetectionModel.from_pretrained(
model_type='yolov8',
model_path=yolov8_model_path,
confidence_threshold=0.3,
device="cpu", # or 'cuda:0'
)

With an image path

result = get_prediction("demo_data/small-vehicles1.jpeg", detection_model)

With a numpy image

result = get_prediction(read_image("demo_data/small-vehicles1.jpeg"), detection_model)

result.export_visuals(export_dir="demo/")
Image("demo/visual.png")

result = get_sliced_prediction(
"demo_data/small-vehicles1.jpeg",
detection_model,
slice_height=256,
slice_width=256,
overlap_height_ratio=0.2,
overlap_width_ratio=0.2
)
result.export_visuals(export_dir="demo_data/")
Image("demo_data/prediction_visual1.png")
Regarding the first matter, the truth is that I do not know where to put this code and how to control the measurements. I wish there were clear answers.
Also, regarding the second problem, when I execute the code, I do not have a yaml file available. Only the weights file for the model yolov8s.pt is available.
As for the third problem, I also faced a problem. I hope you can explain it based on the code sent. I also inform you that I use vscod.

[pderrenger]

Hey! Thanks for reaching out with your questions. I'm here to help! 😊

1. Placing the Code & Adjusting Box/Text Sizes: The code you've shown sets up detection pretty nicely. If you're looking to adjust the box sizes or text labels in your detections, you'd typically modify the visualization parameters in the export_visuals function call or directly in the SAHI source if needed. SAHI might not expose direct settings for text/box size, so you might consider custom drawing functions after detections for more control.

2. Filtering for Vehicles: With your current setup, there's no direct way to filter detections by class (like vehicles only) unless you modify the prediction processing code. You'd iterate through result.object_prediction_list, keep only detections of your interest (cars, buses, motorcycles), and perhaps use a custom visualization method to draw the filtered results.

3. No YAML, Only Weights: It's perfectly fine to operate with just the .pt model file for inference tasks. The SAHI and YOLO integration primarily requires the model weight file for running predictions.

4. Statistical Results in VSCode: Getting statistics like object counts involves iterating through result.object_prediction_list as mentioned. You can tally object types with standard Python collections (like Counter) for a summary. This part of processing would be manual based on the detected objects.

For VSCode-specific output or enhanced control over your detections in a development environment like VSCode, consider implementing logging or print statements that summarize result.object_prediction_list content post-prediction.

If you've got more questions or need further clarifications, feel free to ask. Happy coding!

[dimaspratama08]

I am using SAHI to predict on all present images in a folder using the predict() function, how do i get the PredictionResults object for each image? i've tried with: result = predict() and then prediction_list = results.object_prediction_list but it returns an error

[dimaspratama08]

Perform batch inference

results = predict(
detection_model = detection_model,
source=source,
slice_height=4096,
slice_width=4096,
overlap_height_ratio=0.2,
overlap_width_ratio=0.2,
export_pickle = True,
visual_bbox_thickness = 10,
visual_hide_labels = True,
visual_hide_conf = True,
visual_export_format = 'jpg'
)

for result in results:
object_prediction_list = result.object_prediction_list

it still returned the same error
TypeError Traceback (most recent call last)
Cell In[42], line 1
----> 1 for result in results:
2 object_prediction_list = result.object_prediction_list

TypeError: 'NoneType' object is not iterable

[pderrenger]

Hey there! 👋 It seems like you're encountering a TypeError when trying to iterate over the results of a batch prediction with SAHI, indicating that results might be None. This usually happens when the predict() function doesn't execute as expected. It'd be helpful to double-check a few things:

1. Ensure the source is correctly pointing to a directory with images.
2. Verify that your model_type, model_path, and other parameters in predict() match your setup and the models you intend to use.
3. Make sure that SAHI and YOLOv8 installations are up to date.

Here's a small tweak for debugging purposes:

results = predict(
    model_type="yolov8",
    model_path="path/to/yolov8n.pt",
    source="path/to/images/dir",
    # your other parameters
)

# Check if 'results' is None
if results is None:
    print("No results returned. Check your source and model path.")
else:
    for result in results:
        object_prediction_list = result.object_prediction_list
        # Your code to handle the predictions

This will help you catch any immediate issues with the prediction call.

If everything seems correct but you're still facing issues, kindly share more details about the error or the setup. We're here to help! 😊

[dimaspratama08]

The source, model_type, and and path is correctly defined and it worked too, but for some reason the results is None. I don't think the parameter is the problem because when i pass the same parameter to get_sliced_prediction() function i can access the object_prediction_list

[pderrenger]

Hey there! 😊 It seems like you're running into an interesting issue where predict() returns None while get_sliced_prediction() works just fine. This behavior suggests the function might exit before compiling the results, possibly due to how input images are being handled or an internal check within predict().

Since get_sliced_prediction() is working, you're right; your parameters likely aren't the problem. It might be helpful to ensure that predict() is being used correctly in the context of batch prediction, as it's designed for processing multiple images from a directory or a list. Here's a quick snippet on how to use predict() for a directory, assuming source points to your folder with images:

from sahi.predict import predict

results = predict(
    model_type="yolov8",
    model_path="path/to/your/model.pt",
    source="path/to/your/images",
    # Other parameters as needed
)

Ensure that source is correctly pointing to the folder containing your images. If you're still encountering issues, it might be beneficial to delve deeper into SAHI's documentation or even reach out on their GitHub issues page for more specific support. There's a chance you might be hitting an edge case they can help with! 🛠️

Hope this helps a bit! Let us know if there are any improvements or further questions.

[codingxperience]

Greetings @pderrenger and @dimaspratama08 ,

I’m encountering a similar issue with extracting prediction results in batch mode using SAHI’s predict() function. I’ve referred to the SAHI documentation but haven’t found specific guidance on handling batch results. In my case, while the get_predictions attribute works well for single images, I’m getting a None result for batch predictions, despite verifying my model path and source settings.

If anyone has successfully worked around this, I’d appreciate any insights or advice on the correct setup. Additionally, after running SAHI with a YOLO model on a full-image test set, I wonder: does SAHI automatically stitch sliced bounding boxes together in the final output, or are the results based on individual slices?

[robob80]

I have a dataset of high-resolution images; I have trained my yolo8 model on the lower resolution of size 640, so if I use my existing model with SAHI, it will still receive results based on my previous training. Is there any guidance on retraining my model so I will obtain better results?

[pderrenger]

@robob80 absolutely! Utilizing SAHI for slicing high-resolution images allows for potentially better performance with YOLOv8 models trained on lower resolutions. However, to maximize results from high-resolution images, consider retraining your model directly on higher resolution inputs when feasible. This approach could improve the model's capability to discern finer details that were not present in lower resolution training data.

For retraining, you might incrementally increase the imgsz parameter in your training command to gradually adapt your model to higher resolutions. For example:

yolo detect train data=your_dataset.yaml model=yolov8n.pt imgsz=1024 epochs=100

Ensure your dataset contains high-resolution images or adjust imgsz to the maximum resolution present in your dataset. This method helps in fine-tuning the model to better learn and detect objects from high-res images. 🚀

For more insights on integrating YOLOv8 with SAHI, our guide on sliced inference provides a comprehensive overview. Remember, the goal is to enhance detection performance while balancing computational efficiency. Best of luck with your model enhancement!

[rashboldb]

How about using SAHI to make slices for training as well? Let's say we slice our dataset, let's say we've also kept care about good labels. Then we can images with same resolution for actual inference. Two questions I got:

1. Is it a good idea at all, i.e. might it help to achieve better performance during SAHI tiled inference?
2. How would I adjust the validation to make sure the resulted metrics are correct?

[pderrenger]

Hi @rashboldb,

Great questions! Let's dive into them:

1. Using SAHI for Training:
   Yes, slicing your dataset for training can indeed be beneficial, especially if your images are high-resolution. This approach can help the model learn more effectively from detailed regions of the images, potentially improving performance during SAHI tiled inference. By training on slices, the model becomes adept at detecting objects in smaller sections, which aligns well with how SAHI processes images during inference.

2. Adjusting Validation for Correct Metrics:
   To ensure your validation metrics are accurate when using sliced inference, you should validate your model using the same slicing parameters as during inference. This means applying the same slice dimensions and overlap ratios during validation. Here's how you can do it:

   from sahi.predict import get_sliced_prediction
   
   # Perform sliced validation
   result = get_sliced_prediction(
       "path/to/validation/image.jpg",
       detection_model,
       slice_height=256,
       slice_width=256,
       overlap_height_ratio=0.2,
       overlap_width_ratio=0.2,
   )
   
   # Export and visualize results
   result.export_visuals(export_dir="path/to/export_dir/")

   By validating with the same slicing parameters, you ensure that the metrics reflect the model's performance under the same conditions it will encounter during inference.

For more detailed guidance on using YOLOv8 with SAHI, including installation and usage examples, please refer to our SAHI Tiled Inference Guide.

Feel free to reach out if you have any more questions. Happy coding! 🚀

[illink7]

How I can combine SAHI with segmentation model for aerial images, could you provide me some details please

[pderrenger]

Hey there! It looks like you're trying to call .eval() on a dictionary object, which isn't possible since .eval() is a method typically used on PyTorch models to set them to evaluation mode. If you're trying to set your model to evaluation mode, make sure you're applying .eval() to your model instance, not a dictionary.

Here's a quick example to ensure you're using it correctly:

import torch

# Assuming 'model' is your loaded model instance
model = torch.load('path_to_your_model.pt')
model.eval()  # Set the model to evaluation mode

Make sure that model is indeed a PyTorch model object. If you're loading the model and it's returning a dictionary (perhaps with model state and other metadata), you'll need to properly extract the model from that dictionary. Typically, this looks something like:

model = torch.load('path_to_your_model.pt')
actual_model = model['model']  # Adjust the key according to how your model is saved
actual_model.eval()

Hope this helps! Let me know if there's anything else you need! 😊

[illink7]

thank you for your reply but unfortunately I didn't get the result I wanted or I don't understand your idea. initially I wanted to use the YOLOv8-seg model to segment aerial images. it worked for me, but along with the result there were labels and bounding boxes that hindered me. maybe there is a way to export the segmented image (data) separately or any other ideas? I am very grateful for your attention and tips, thank you so much

[pderrenger]

Hello! Thank you for your feedback and for sharing your experience with the YOLOv8-seg model. It sounds like you're looking to extract just the segmentation masks without the bounding boxes and labels. Here's a straightforward way to achieve this:

If you're using a custom script or the SAHI library for segmentation, you'll typically handle the output directly from the model's forward pass. This output often includes both the segmentation masks and the detection outputs (like bounding boxes and labels). To get just the segmentation masks, you can modify your script to save only the mask data.

Here's a basic example of how you might adjust your code to save only the segmentation masks:

import torch
from PIL import Image
import numpy as np

# Assuming 'model' is your segmentation model and 'img' is your input image
model.eval()
with torch.no_grad():
    prediction = model(img)  # Get the raw output from the model

# Assuming the segmentation mask is in 'prediction[0]' and it's a tensor
mask = prediction[0].cpu().numpy()  # Convert to numpy array if not already
mask_image = Image.fromarray((mask * 255).astype(np.uint8))  # Convert to an image
mask_image.save('path_to_save_your_mask.png')  # Save the segmentation mask

This snippet is quite generic and might need adjustments based on the exact output format of your model. If your model outputs are in a different format or include additional post-processing steps, you'll need to adapt the code accordingly.

If you need more specific guidance, please let me know more about your model's output format or any libraries you're using to handle the model outputs! 😊

[illink7]

Thank you so much

[pderrenger]

Hello! Thanks for reaching out. 🌟

If you're looking to use YOLOv8-seg for segmentation and want to export the segmented images without bounding boxes and labels, you can directly access the segmentation masks from the model's output. Here’s a quick way to do it:

import torch
from PIL import Image
import numpy as np

# Load your model and set to evaluation mode
model = torch.load(MODEL_PATH)
model.eval()

# Prepare your image
image = Image.open(IMAGE_PATH)
image = image.convert('RGB')  # Ensure image is RGB
transform = transforms.Compose([
    transforms.ToTensor(),
])
input_tensor = transform(image).unsqueeze(0)  # Add batch dimension

# Send to device
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
input_tensor = input_tensor.to(device)
model = model.to(device)

# Get prediction
with torch.no_grad():
    output = model(input_tensor)

# Assuming the output is a tensor with segmentation masks
masks = output['masks'].cpu().numpy()  # Adjust based on your model output
mask_image = Image.fromarray((masks[0] * 255).astype(np.uint8))  # Convert first mask to image
mask_image.save('segmentation_output.png')  # Save the mask image

This code snippet assumes that your model outputs a dictionary with a key 'masks' containing the segmentation masks. Adjust the key based on your actual model output. This will save the segmentation masks directly, bypassing any bounding boxes or labels.

Hope this helps! Let me know if there's anything else you need. 😊

[mxxxbbbb]

When I use Sahi, the inference speed is normal in a single main thread, but when I load Sahi into one of multiple threads, it causes the inference speed to be much slower. What is the reason for this

[glenn-jocher]

Hello! It sounds like you're experiencing a slowdown when running SAHI in a multi-threaded environment. This can often happen due to the increased overhead of managing multiple threads, especially if the threads are competing for limited computational resources like CPU or GPU.

Here are a couple of suggestions to potentially improve the performance:

1. Ensure Adequate Resources: Make sure that your system has enough resources (CPU/GPU) to handle multiple threads efficiently. If each thread is performing intensive computations, you might need more powerful hardware or fewer concurrent threads.

2. Optimize Thread Management: Try to manage the number of threads based on the workload and the hardware capabilities. Sometimes, reducing the number of threads can actually improve overall performance.

3. Profile Your Application: Use profiling tools to understand where the bottlenecks are. This can provide insights into whether the issue is due to thread management, I/O operations, or something else.

If you continue to experience issues, consider providing more details about your setup, such as the hardware specifications and the specific code implementation, so we can help you better. 🚀

[mxxxbbbb]

I have loaded many deep learning models in one thread across multiple threads, among which sahi is one. When testing with a single main thread, the speed is not a problem, but with multiple threads, the speed is very slow

[mxxxbbbb]

def pre_sahi(self,img_all):

    resize_img = self.resize_long_image(img_all,640)
    t1 = time.time()
    slice_image_result = slice_image(
        image=resize_img,
        slice_height=640,
        slice_width=640,
        overlap_height_ratio=0.2,
        overlap_width_ratio=0.2,
        auto_slice_resolution=True,
    )

    result = get_sliced_prediction(
        resize_img,
        slice_image_result,
        self.model_sahi,
    )
    box = []
    t2 = time.time()
    print("time:",t2-t1)

This is my call to this class's function in a certain thread, and the model is actually loaded by the constructor

[glenn-jocher]

Hello! It sounds like you're encountering a common issue when using multiple threads for inference, especially when each thread is loading a deep learning model like SAHI. This slowdown can occur due to several reasons:

1. Resource Contention: Each thread may be competing for limited computational resources (CPU/GPU). This is particularly significant if each model in the threads is substantial in size and computational demand.

2. Global Interpreter Lock (GIL) in Python: If you're using Python, the GIL can become a bottleneck in multi-threaded applications, as it allows only one thread to execute Python bytecode at a time. This can severely limit performance when using CPU-bound tasks.

3. I/O Bottlenecks: If your application involves reading from or writing to disk, such as loading models or accessing data, I/O operations can become a bottleneck when scaled across multiple threads.

To mitigate these issues, consider the following:

• Use multiprocessing instead of multithreading: This can help bypass the GIL and better utilize multiple cores.
• Optimize resource allocation: Ensure that each thread or process has enough resources to perform its tasks efficiently.
• Load models once: If possible, load your models once and share them across threads, rather than loading them separately in each thread.

For more detailed guidance, you might find our documentation on using YOLOv8 with SAHI helpful: Using YOLOv8 with SAHI for Sliced Inference. This guide provides insights into efficiently managing resources and optimizing inference tasks.

[Rushita69]

How can I predict only the specific classes I need and not all the classes used during training in Sahi inference?

[glenn-jocher]

@Rushita69 to predict only specific classes during SAHI inference with YOLOv8, you can filter the predictions based on class IDs after the detection process. Here's a simple way to achieve this using Python:

from sahi.predict import get_sliced_prediction

# Instantiate your model as usual
detection_model = AutoDetectionModel.from_pretrained(
    model_type="yolov8",
    model_path="path/to/yolov8_model.pt",
    confidence_threshold=0.3,
    device="cpu",  # or 'cuda:0'
)

# Perform sliced prediction
result = get_sliced_prediction(
    image_path="path/to/image.jpeg",
    detection_model=detection_model,
    slice_height=256,
    slice_width=256,
    overlap_height_ratio=0.2,
    overlap_width_ratio=0.2,
)

# Filter predictions to include only specific classes, e.g., class IDs 1 and 3
filtered_predictions = [pred for pred in result.object_prediction_list if pred.category_id in [1, 3]]

# Now you can use filtered_predictions as needed

This code snippet will help you filter out detections and keep only those that belong to the classes you're interested in. For more detailed guidance on using YOLOv8 with SAHI, you can refer to our comprehensive guide here.

[ChongCheeKaiClarence]

Hi, I need help inresolving this.

ModuleNotFoundError: No module named 'sahi.predict'; 'sahi' is not a package

This is my package environment

asttokens           2.0.5
backcall            0.2.0
Brotli              1.0.9
certifi             2024.2.2
cffi                1.16.0
charset-normalizer  2.0.4
click               8.1.7
colorama            0.4.6
contourpy           1.1.1
cycler              0.12.1
decorator           5.1.1
executing           0.8.3
filelock            3.13.1
fire                0.6.0
fonttools           4.51.0
fsspec              2024.3.1
gmpy2               2.1.2
idna                3.7
importlib_resources 6.4.0
intel-openmp        2021.4.0
ipython             8.12.2
jedi                0.18.1
Jinja2              3.1.3
kiwisolver          1.4.5
MarkupSafe          2.1.3
matplotlib          3.7.5
matplotlib-inline   0.1.6
mkl                 2021.4.0
mkl-fft             1.3.8
mkl-random          1.2.4
mkl-service         2.4.0
mpmath              1.3.0
networkx            3.1
numpy               1.24.3
opencv-python       4.8.1.78
packaging           24.0
pandas              2.0.3
parso               0.8.3
pickleshare         0.7.5
pillow              10.3.0
pip                 24.0
prompt-toolkit      3.0.43
psutil              5.9.8
pure-eval           0.2.2
py-cpuinfo          9.0.0
pybboxes            0.1.6
pycparser           2.21
Pygments            2.15.1
pyparsing           3.1.2
PySocks             1.7.1
python-dateutil     2.9.0.post0
pytz                2024.1
PyYAML              6.0.1
requests            2.31.0
sahi                0.11.16
scipy               1.10.1
seaborn             0.13.2
setuptools          69.5.1
shapely             2.0.1
six                 1.16.0
stack-data          0.2.0
sympy               1.12
tbb                 2021.12.0
termcolor           2.1.0
terminaltables      3.1.10
thop                0.1.1.post2209072238
torch               2.2.0
torchvision         0.18.0
tqdm                4.66.4
traitlets           5.7.1
typing_extensions   4.11.0
tzdata              2024.1
ultralytics         8.2.20
urllib3             2.2.1
wcwidth             0.2.5
wheel               0.43.0
win-inet-pton       1.1.0
zipp                3.18.2

[pderrenger]

Hi there! It looks like Python isn't recognizing sahi as an installed package, despite it appearing in your environment list. This could be due to a few reasons:

1. Environment Issue: Make sure you're activating the correct Python environment where sahi is installed if you're using virtual environments.

2. Path Issue: Python might not be looking in the correct directory for installed packages. You can verify where Python is searching for packages with:

   import sys
   print(sys.path)

3. Reinstallation: Sometimes, a simple reinstallation can fix the issue. Try:

   pip install --upgrade --force-reinstall sahi

If these steps don't resolve the issue, consider checking if there are any naming conflicts or if the installation was somehow corrupted. Let me know how it goes! 🛠️

[noval779]

I use YOLOv8 + SAHI for tree detection, I need to calculate the diameter of the tree canopy in cm/m, I read in several articles to use pixels from the bounding box and convert them into cm/m, how do I implement it in SAHI?

[pderrenger]

@noval779 hi there!

Thanks for sharing the details. It looks like you're on the right track! The BoundingBox object in SAHI indeed doesn't have direct width and height attributes. Instead, you can access the bounding box coordinates and calculate the dimensions.

Here's how you can do it:

for object_prediction in result.object_prediction_list:
    bbox = object_prediction.bbox
    x_min, y_min, x_max, y_max = bbox.minx, bbox.miny, bbox.maxx, bbox.maxy
    width_pixels = x_max - x_min
    height_pixels = y_max - y_min
    # Assuming you have a known scale, e.g., 1 pixel = 0.5 cm
    width_cm = width_pixels * 0.5
    height_cm = height_pixels * 0.5
    print(f"Width: {width_cm} cm, Height: {height_cm} cm")

This should help you convert the bounding box dimensions from pixels to cm/m. If you have any further questions, feel free to ask! 😊

[noval779]

from sahi.predict import get_prediction, get_sliced_prediction, predict

results = predict(
model_type="yolov8",
model_path="/content/best.pt",
model_device="gpu",
model_confidence_threshold=0.4,
source= img_dir,
slice_height=2500,
slice_width=2500,
overlap_height_ratio=0.25,
overlap_width_ratio=0.25,
visual_bbox_thickness=6,
visual_text_size=4,
visual_text_thickness=4
)

how do I make the result show in the label section of each object

[pderrenger]

Hello!

To display labels for each detected object in your results, you can modify the visualization parameters when exporting the visuals. SAHI allows you to customize how the results are displayed, including the option to show labels on the bounding boxes.

Here’s how you can adjust your code to include labels in the visualization:

# Perform the prediction
results = predict(
    model_type="yolov8",
    model_path="/content/best.pt",
    model_device="gpu",
    model_confidence_threshold=0.4,
    source=img_dir,
    slice_height=2500,
    slice_width=2500,
    overlap_height_ratio=0.25,
    overlap_width_ratio=0.25,
    visual_bbox_thickness=6,
    visual_text_size=4,
    visual_text_thickness=4
)

# Export and visualize the predicted bounding boxes and masks with labels
for result in results:
    result.export_visuals(export_dir="output/", draw_labels=True)

Make sure to set draw_labels=True in the export_visuals() function to ensure that labels are drawn on the visualization output. Adjust the export_dir to the directory where you want to save the visual outputs.

If you have any more questions or need further assistance, feel free to ask. Happy detecting! 😊

[noval779]

can SAHI be used with Segment Anything Model (SAM) for segmentation

[pderrenger]

@noval779 sAHI can indeed be used with the Segment Anything Model (SAM) for segmentation tasks. SAHI's slicing and stitching capabilities are designed to work with various models, including SAM. This allows you to handle large images efficiently by breaking them into smaller slices, running segmentation on each slice, and then merging the results. For detailed instructions on using SAHI with different models, please refer to the Ultralytics Docs on SAHI.

[illink7]

Hi folks. I still work with SAHI and YOLOv8-seg
When I tried to do segmentation with small aerial image without SAHI it works well and segment my objects (forest and buildings)
But when I try to do this with SAHI it doesn't provide results with mask.
I've installed all nececcary libraries, and they have the same environment but still no results
Could you check my code and maybe you find the best solution?

from sahi.predict import get_sliced_prediction
from sahi import AutoDetectionModel
from PIL import Image
import os
from ultralytics import YOLO
import torch
import numpy as np
from PIL import Image

Set the path to the image

IMAGE_PATH = r"PATH"

Setting the path to the model

MODEL_PATH = r"PATH"

Upload an image

image = Image.open(IMAGE_PATH)

Initialize the model

detection_model = AutoDetectionModel.from_pretrained(
model_type='yolov8',
model_path=MODEL_PATH,
confidence_threshold=0.25,
device="cpu" # if torch.cuda.is_available() else "cpu"
)

Performing predictions with cutting the image into 512x512 parts

result = get_sliced_prediction(
image,
detection_model,
slice_height=512,
slice_width=512,
overlap_height_ratio=0.2,
overlap_width_ratio=0.2
)

Saving the results

output_dir = 'output_vs'
os.makedirs(output_dir, exist_ok=True)
result.export_visuals(export_dir=output_dir, file_name='segmented_image.png')

Initialize the dictionary for merged masks by class

combined_masks_by_class = {}

Export masks and combine them by class

for prediction in result.object_prediction_list:
mask = prediction.mask
category_name = prediction.category.name
if mask is not None:
mask_array = mask.bool_mask
# Scale the mask to the size of the original image
mask_image = Image.fromarray((mask_array.astype(np.uint8) * 255))
mask_resized = mask_image.resize((image.width, image.height), resample=Image.NEAREST)
mask_resized_np = np.array(mask_resized)

    # Add a mask to the appropriate class in the dictionary
    if category_name not in combined_masks_by_class:
        combined_masks_by_class[category_name] = np.zeros((image.height, image.width), dtype=np.uint8)
    combined_masks_by_class[category_name] = np.maximum(combined_masks_by_class[category_name], mask_resized_np)

Store merged masks by class

for category_name, combined_mask in combined_masks_by_class.items():
combined_mask_image = Image.fromarray(combined_mask)
combined_mask_image.save(os.path.join(output_dir, f'combined_mask_{category_name}.png'))

print(f"Results saved in {output_dir}")

After that code I receive aerial image with boxes and label but without masks (
How I can export masks divided by classes which exist in my model?

[pderrenger]

Hi @illink7,

Thank you for providing a detailed description and the minimum reproducible example (MRE). Let's work together to resolve the issue with the segmentation masks not appearing in your results.

Steps to Troubleshoot and Resolve

1. Verify Library Versions: Ensure you are using the latest versions of torch, ultralytics, and sahi. You can update them using:

   pip install -U torch ultralytics sahi

2. Environment Consistency: Since the code works on your laptop but not on other machines, ensure that all environments are consistent. Verify that the same versions of Python and all dependencies are installed.

3. Debugging the Mask Generation:

   • Ensure that the mask attribute in prediction.mask is not None.
   • Verify that the mask.bool_mask is correctly processed and resized.

Here’s a slightly modified version of your code with additional checks:

from sahi.predict import get_sliced_prediction
from sahi import AutoDetectionModel
from PIL import Image
import os
import numpy as np
import torch

# Set the path to the image and model
IMAGE_PATH = r"D:\Illia\YOLO_test\weights_v2_multiModel\img4test\img_139_shift.png"
MODEL_PATH = r"D:\Illia\YOLO_test\weights_v2_multiModel\best.pt"

# Upload an image
image = Image.open(IMAGE_PATH)

# Initialize the model
detection_model = AutoDetectionModel.from_pretrained(
    model_type='yolov8',
    model_path=MODEL_PATH,
    confidence_threshold=0.25,
    device="cuda" if torch.cuda.is_available() else "cpu"
)

# Performing predictions with slicing the image into 512x512 parts
result = get_sliced_prediction(
    image,
    detection_model,
    slice_height=512,
    slice_width=512,
    overlap_height_ratio=0.2,
    overlap_width_ratio=0.2
)

# Check if masks are present in the predictions
if not result.object_prediction_list:
    print("No predictions found. Please check your model and input image.")
else:
    print(f"Found {len(result.object_prediction_list)} predictions.")

# Saving the results
output_dir = 'output_vs2'
os.makedirs(output_dir, exist_ok=True)
result.export_visuals(export_dir=output_dir, file_name='segmented_image.png')

# Initialize the dictionary for merged masks by class
combined_masks_by_class = {}

# Export masks and combine them by class
for prediction in result.object_prediction_list:
    mask = prediction.mask
    category_name = prediction.category.name
    if mask is not None:
        mask_array = mask.bool_mask
        # Scale the mask to the size of the original image
        mask_image = Image.fromarray((mask_array.astype(np.uint8) * 255))
        mask_resized = mask_image.resize((image.width, image.height), resample=Image.NEAREST)
        mask_resized_np = np.array(mask_resized)

        # Add a mask to the appropriate class in the dictionary
        if category_name not in combined_masks_by_class:
            combined_masks_by_class[category_name] = np.zeros((image.height, image.width), dtype=np.uint8)
        combined_masks_by_class[category_name] = np.maximum(combined_masks_by_class[category_name], mask_resized_np)

# Store merged masks by class
for category_name, combined_mask in combined_masks_by_class.items():
    combined_mask_image = Image.fromarray(combined_mask)
    combined_mask_image.save(os.path.join(output_dir, f'combined_mask_{category_name}.png'))

print(f"Results saved in {output_dir}")

Additional Recommendations:

• Check the Model Output: Ensure that your model is indeed capable of generating masks. You can test this by running standard inference without slicing to verify mask generation.
• Review SAHI Documentation: For more detailed guidance, refer to the SAHI Tiled Inference Guide.

If the issue persists, please provide any error messages or additional details that could help us diagnose the problem further.

Feel free to reach out if you have any further questions or need additional assistance. We're here to help! 😊

[illink7]

Thank you for help, but again there is no masks
Could you provide me maybe another solution
And try as I mentioned above with open model and image
IMAGE_PATH = r"D:\Illia\dog.jpg"
here is the link on image - https://www.princeton.edu/sites/default/files/styles/1x_full_2x_half_crop/public/images/2022/02/KOA_Nassau_2697x1517.jpg?itok=Bg2K7j7J
MODEL_PATH = "yolov8n-seg.pt"

[pderrenger]

Hi @illink7,

Thank you for your patience and for providing the detailed information. Let's work through this together to resolve the issue with the segmentation masks not appearing in your results.

Steps to Troubleshoot and Resolve

1. Verify Library Versions: Ensure you are using the latest versions of torch, ultralytics, and sahi. You can update them using:

   pip install -U torch ultralytics sahi

2. Environment Consistency: Since the code works on your laptop but not on other machines, ensure that all environments are consistent. Verify that the same versions of Python and all dependencies are installed.

3. Debugging the Mask Generation:

   • Ensure that the mask attribute in prediction.mask is not None.
   • Verify that the mask.bool_mask is correctly processed and resized.

Here’s a slightly modified version of your code with additional checks:

from sahi.predict import get_sliced_prediction
from sahi import AutoDetectionModel
from PIL import Image
import os
import numpy as np
import torch

# Set the path to the image and model
IMAGE_PATH = r"D:\Illia\dog.jpg"
MODEL_PATH = "yolov8n-seg.pt"

# Upload an image
image = Image.open(IMAGE_PATH)

# Initialize the model
detection_model = AutoDetectionModel.from_pretrained(
    model_type='yolov8',
    model_path=MODEL_PATH,
    confidence_threshold=0.25,
    device="cuda" if torch.cuda.is_available() else "cpu"
)

# Performing predictions with slicing the image into 1024x1024 parts
result = get_sliced_prediction(
    image,
    detection_model,
    slice_height=1024,
    slice_width=1024,
    overlap_height_ratio=0.2,
    overlap_width_ratio=0.2
)

# Check if masks are present in the predictions
if not result.object_prediction_list:
    print("No predictions found. Please check your model and input image.")
else:
    print(f"Found {len(result.object_prediction_list)} predictions.")

# Saving the results
output_dir = 'output_vs2'
os.makedirs(output_dir, exist_ok=True)
result.export_visuals(export_dir=output_dir, file_name='segmented_image.png')

# Initialize the dictionary for merged masks by class
combined_masks_by_class = {}

# Export masks and combine them by class
for prediction in result.object_prediction_list:
    mask = prediction.mask
    category_name = prediction.category.name
    if mask is not None:
        mask_array = mask.bool_mask
        # Scale the mask to the size of the original image
        mask_image = Image.fromarray((mask_array.astype(np.uint8) * 255))
        mask_resized = mask_image.resize((image.width, image.height), resample=Image.NEAREST)
        mask_resized_np = np.array(mask_resized)

        # Add a mask to the appropriate class in the dictionary
        if category_name not in combined_masks_by_class:
            combined_masks_by_class[category_name] = np.zeros((image.height, image.width), dtype=np.uint8)
        combined_masks_by_class[category_name] = np.maximum(combined_masks_by_class[category_name], mask_resized_np)

# Store merged masks by class
for category_name, combined_mask in combined_masks_by_class.items():
    combined_mask_image = Image.fromarray(combined_mask)
    combined_mask_image.save(os.path.join(output_dir, f'combined_mask_{category_name}.png'))

print(f"Results saved in {output_dir}")

Additional Recommendations:

• Check the Model Output: Ensure that your model is indeed capable of generating masks. You can test this by running standard inference without slicing to verify mask generation.
• Review SAHI Documentation: For more detailed guidance, refer to the SAHI Tiled Inference Guide.

If the issue persists, please provide any error messages or additional details that could help us diagnose the problem further.

Feel free to reach out if you have any further questions or need additional assistance. We're here to help! 😊

[pinnintipraneethkumar]

Hi, @illink7 can you please check the sahi version, i have the same problem i fixed this by updating the sahi library.

[Rohan-Python]

@pinnintipraneethkumar Do you know how to disable bounding boxes on segmented images?

[echo385]

I find that when I use different overlap_height_ratio and overlap_width_ratio in sahi to predict my images , I will get different detection results, I want to know the reason for it

[glenn-jocher]

@echo385 hi there!

Great question! The difference in detection results when using various overlap_height_ratio and overlap_width_ratio values in SAHI is due to how the slicing and stitching process works. Here’s a brief explanation:

How Slicing and Overlap Affect Detection

1. Slicing: When an image is divided into smaller slices, each slice is processed independently. The size of these slices and the amount of overlap between them can influence the detection results.
2. Overlap: Overlapping slices ensure that objects near the edges of slices are not missed. However, different overlap ratios can lead to variations in how objects are detected and merged.

Why Different Overlap Ratios Yield Different Results

• Detection Consistency: Higher overlap ratios can lead to more consistent detections, as objects are likely to appear in multiple slices. This can help in better merging but might also introduce more duplicate detections that need to be handled.
• Edge Effects: Lower overlap ratios might miss objects that are split between slices, leading to less accurate detections.

Example Code for Sliced Inference

Here’s an example of how you can perform sliced inference with different overlap ratios:

from sahi.predict import get_sliced_prediction
from sahi import AutoDetectionModel

# Load the YOLOv8 model
detection_model = AutoDetectionModel.from_pretrained(
    model_type="yolov8",
    model_path="models/yolov8s.pt",
    confidence_threshold=0.3,
    device="cpu",  # or 'cuda:0'
)

# Perform sliced inference with different overlap ratios
result_1 = get_sliced_prediction(
    "demo_data/small-vehicles1.jpeg",
    detection_model,
    slice_height=256,
    slice_width=256,
    overlap_height_ratio=0.2,
    overlap_width_ratio=0.2,
)

result_2 = get_sliced_prediction(
    "demo_data/small-vehicles1.jpeg",
    detection_model,
    slice_height=256,
    slice_width=256,
    overlap_height_ratio=0.5,
    overlap_width_ratio=0.5,
)

# Visualize results
result_1.export_visuals(export_dir="demo_data/")
result_2.export_visuals(export_dir="demo_data/")

Further Reading

For more detailed information on how to use SAHI with YOLOv8, including installation and additional examples, please refer to our SAHI Tiled Inference Guide.

Feel free to experiment with different overlap ratios to find the optimal settings for your specific use case. If you have any more questions or need further assistance, don't hesitate to ask! 😊

[natashahrycan]

Hi! I have been using SAHI with YOLOv8 for segmentation in my images, and it works great. Unfortunately, I am having problems to disable the labels and bounding boxes, as well as exporting data from the masks (I need the area and average color of the mask to classify the objecttts later with K-means). Regarding the labels and bounding boxes, I have tried introducing the arguments show_labels=False and visual_hide_labels = True in the coresponding functions, but no success and I cannot find the documentation for arguments of functions for SAHI. For the area of the mask (in pixels), I have only been able to find the height and width of the box, but that is not accurate enough for me, how can I find the area and average color of a mask?

[glenn-jocher]

Hi! To get started with SAHI and YOLOv8 for segmentation, you'll need to install the ultralytics and sahi libraries. Then, download a YOLOv8 model and test images. You can perform standard or sliced inference by specifying the slice dimensions and overlap ratios. For more detailed steps, check out the Ultralytics documentation on SAHI. If you need further assistance, feel free to ask!

[Rohan-Python]

@glenn-jocher Sir I tried using "visual_hide_labels=True,
visual_hide_boxes=True" but it is giving a error
My concern is same, I just want to remove the bounding boxes from the inferenced image because nothing is readable

I just want to remove bounding box and lables

[glenn-jocher]

It looks like you're encountering issues with disabling labels and bounding boxes using SAHI with YOLOv8. The correct arguments should be visual_hide_labels=True and visual_hide_boxes=True. If these are not working, please ensure you are using the latest versions of both ultralytics and sahi packages. For calculating the area and average color of masks, you can access the mask data directly from the object_prediction_list in the result object. If you need further assistance, please refer to the SAHI documentation for more details.

[Rohan-Python]

I am using ultralytics-8.2.64 and sahi-0.11.18 versions also to be sure I used "pip install -U sahi ultralytics" to update to latest versions
I used "visual_hide_labels=True" and "visual_hide_boxes=True" as suggested by you but I am getting a unexpected keyword argument 'visual_hide_labels'. Attached a screen snip for reference. Please help me out through this.

[glenn-jocher]

Thank you for updating to the latest versions. The error indicates that visual_hide_labels and visual_hide_boxes are not recognized arguments. These arguments might not be supported in the current version of SAHI. To remove labels and bounding boxes, you may need to manually modify the visualization code or check the SAHI documentation for alternative methods. For calculating mask areas and average colors, you can access the mask data directly from the object_prediction_list in the result object. For further details, please refer to the SAHI documentation.

[sergiev]

Batch Prediction sounds misleading, since no real batch inference are supported. Or am I missing something?

[PiyushBagde]

i am getting this kind of error? what does this mean

Performing prediction on 30 slices.

ImportError Traceback (most recent call last)
in <cell line: 3>()
1 from sahi.predict import get_sliced_prediction
2
----> 3 result = get_sliced_prediction(
4 "/content/test1/Screenshot 2024-07-16 000529.jpg",
5 detection_model,

7 frames
/usr/local/lib/python3.10/dist-packages/ultralytics/utils/callbacks/hub.py in
4 from time import time
5
----> 6 from ultralytics.hub import HUB_WEB_ROOT, PREFIX, HUBTrainingSession, events
7 from ultralytics.utils import LOGGER, RANK, SETTINGS
8

ImportError: cannot import name 'HUBTrainingSession' from 'ultralytics.hub' (/usr/local/lib/python3.10/dist-packages/ultralytics/hub/init.py)

[pderrenger]

@PiyushBagde it looks like you're encountering an ImportError related to HUBTrainingSession. This issue might be due to an outdated version of the Ultralytics package. Please ensure you have the latest versions of both ultralytics and sahi installed by running pip install -U ultralytics sahi. If the problem persists, try reinstalling the packages. For more detailed guidance on using SAHI with YOLOv8, you can refer to our documentation. Let me know if you need further assistance!

[PiyushBagde]

ImportError: cannot import name 'HUBTrainingSession' from 'ultralytics.hub' (/usr/local/lib/python3.10/dist-packages/ultralytics/hub/init.py)
i am getting this error now.

[pderrenger]

@PiyushBagde it looks like you're encountering an import error with HUBTrainingSession. This issue might be due to an outdated package version. Please ensure you're using the latest versions of the Ultralytics and SAHI libraries by running pip install -U ultralytics sahi. If the problem persists, try reinstalling the packages. For more details on using YOLOv8 with SAHI, you can refer to the comprehensive guide at https://docs.ultralytics.com/guides/sahi-tiled-inference/.

[Rohan-Python]

How to disable bounding boxes and labels on images on which segmentation has been run by yolov8+SAHI

[glenn-jocher]

@Rohan-Python to disable bounding boxes and labels on images where segmentation has been run using YOLOv8 and SAHI, you can modify the visualization settings in the export_visuals method. By default, this method includes bounding boxes and labels. You can customize it to exclude these elements. For more detailed guidance, please refer to the SAHI Tiled Inference Guide.

[Rohan-Python]

@glenn-jocher I read the article on Ultralytics about export_visuals but I am not getting how to modify the settings in the export_visuals.
Please guide me.

[glenn-jocher]

To disable bounding boxes and labels in the export_visuals method, you need to adjust the visualization settings. Unfortunately, the current export_visuals method does not provide direct parameters to disable these elements. You might need to modify the source code or use a custom visualization function to achieve this. For more detailed guidance, please refer to the SAHI Tiled Inference Guide.

[SudhanshuBlaze]

Hi, I am using yolov9e for traffic counting, this model has been trained on the visdrone dataset. However, I was wondering if its performance can be improved by using SAHI, I read your messages on this thread but couldn't figure how to integrate SAHI with yolo for object counting in a video.

import cv2
from ultralytics import YOLO, solutions

# Load the YOLO model
model = YOLO("yolov9e.pt")

# Open the video file
cap = cv2.VideoCapture("resources/DJI_0505.MP4")
assert cap.isOpened(), "Error reading video file" 
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

classes_to_count = list(range(0, 23))  # classes to count 

# Define the lines or regions points
line_points_1 = [(300, 600), (700, 550)]  # first line points
line_points_2 = [(800, 550), (1700, 350)]  # second line points
line_points_3 = [(174, 1150), (174, 1800)]  # second line points
line_points_4 = [(640, 1674), (1006, 1578)]  # second line points


# Video writer
video_writer = cv2.VideoWriter("out_object_counting_output.mp4", cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h))

# Init Object Counters
counter_1 = solutions.ObjectCounter(
    view_img=True,
    reg_pts=line_points_1,
    classes_names=model.names,
    draw_tracks=True,
    line_thickness=2,
    text_offset=(0, 0),
    count_reg_color=(229, 255, 204),  # Very light green
    count_bg_color=(229, 255, 204)  # Very light green
)

counter_2 = solutions.ObjectCounter(
    view_img=True,
    reg_pts=line_points_2,
    classes_names=model.names,
    draw_tracks=True,
    line_thickness=2,
    text_offset=(-250, 0),
    count_reg_color=(153, 204, 255),  # Light blue
    count_bg_color=(153, 204, 255)  # Light blue
)

counter_3 = solutions.ObjectCounter(
    view_img=True,
    reg_pts=line_points_3,
    classes_names=model.names,
    draw_tracks=True,
    line_thickness=2,
    text_offset=(-500, 0),
    count_reg_color=(255, 204, 204),  # Light red
    count_bg_color=(255, 204, 204)  # Light red
)

counter_4 = solutions.ObjectCounter(
    view_img=True,
    reg_pts=line_points_4,
    classes_names=model.names,
    draw_tracks=True,
    line_thickness=2,
    text_offset=(-750, 0),
    count_reg_color=(210, 180, 140),  # Light brown
    count_bg_color=(210, 180, 140)  # Light brown
)


# Init separate trackers for each counter
tracker = model  # Tracker for the first counter

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break

    # Perform tracking for the first counter, options: tracker="bytetrack.yaml"
    tracks = tracker.track(im0, persist=True, show=False, classes=classes_to_count)
    
    im0 = counter_1.start_counting(im0, tracks)

    # Perform tracking for the second counter
    im0 = counter_2.start_counting(im0, tracks)
    
    im0 = counter_3.start_counting(im0, tracks)
    
    im0 = counter_4.start_counting(im0, tracks)
    
    # Write the frame to the output video
    video_writer.write(im0)

# Release resources
cap.release()
video_writer.release()
cv2.destroyAllWindows()

[pderrenger]

@SudhanshuBlaze to integrate SAHI with YOLOv9e for improved performance in your traffic counting task, you can use SAHI's sliced inference capabilities. This approach will help manage large video frames by breaking them into smaller slices, optimizing memory usage, and maintaining detection accuracy.

First, ensure you have the latest versions of SAHI and Ultralytics installed:

pip install -U ultralytics sahi

Next, you can follow the detailed guide on integrating SAHI with YOLO models for sliced inference at SAHI Tiled Inference Guide. This guide provides comprehensive steps and examples to help you set up and run sliced inference effectively.

Feel free to reach out if you have any more questions!

[SudhanshuBlaze]

Hi Paula, thanks for your prompt response. Yes, I have already checked out the guide which you shared it with me, but in that guide there's no example of how to use SAHI with a video. It only has examples with sample images.

[Rohan-Python]

@SudhanshuBlaze https://ultralytics.medium.com/using-ultralytics-yolov8-with-sahi-on-videos-3d524087dd33
this might help

[wainuo]

I am performing instance segmentation, however the output from SAHI is only an image. How can I operate to obtain the polygonal contour information of the segmented areas?How do I extract contour information from the instance segmentation mask and convert it into polygon coordinate text?

[pderrenger]

@wainuo hi, to handle the segmentation output from result.object_prediction_list, you should first ensure that the segmentation data is in the correct format. The segmentation result is typically a list of coordinates representing the polygon. You can convert these coordinates into a numpy array for further processing. If you encounter issues reshaping the segmentation result, please verify that the data structure matches the expected format. For more detailed guidance, you can refer to the SAHI documentation. If you need further assistance, feel free to ask!

[wainuo]

When I run such code
'''
from sahi.predict import get_sliced_prediction
from sahi import AutoDetectionModel
from PIL import Image
import os
import numpy as np
import torch

Set the path to the image and model

IMAGE_PATH = r"E:\11reshape_photo\45.jpg"
MODEL_PATH = "D:\ultralytics-main\runs\segment\train81\weights\best.pt"

Upload an image

image = Image.open(IMAGE_PATH)

Initialize the model

detection_model = AutoDetectionModel.from_pretrained(
model_type='yolov8',
model_path=MODEL_PATH,
confidence_threshold=0.25,
device="cuda" if torch.cuda.is_available() else "cpu",
)

Performing predictions with slicing the image into 1024x1024 parts

result = get_sliced_prediction(
image,
detection_model,
slice_height=320,
slice_width=320,
overlap_height_ratio=0.1,
overlap_width_ratio=0.1,

)

Check if masks are present in the predictions

if not result.object_prediction_list:
print("No predictions found. Please check your model and input image.")
else:
print(f"Found {len(result.object_prediction_list)} predictions.")

i=0
for prediction in result.object_prediction_list:
contour=prediction.mask.segmentation
i+=1
print(f'{i}',contour)
'''
The result shows the number of segment objects is 1538, however ,the segmentation of one object sometimes is not include only one polygon.As the follow output:
'''
1519 [[1116, 870, 1117, 869, 1118, 870, 1117, 871], [1112, 866, 1113, 865, 1114, 866, 1113, 867], [1107, 834, 1106, 835, 1108, 837, 1109, 837, 1109, 836, 1110, 835, 1109, 834], [1003, 808, 1003, 841, 1002, 842, 1002, 873, 1005, 873, 1007, 871, 1008, 871, 1009, 870, 1009, 869, 1010, 868, 1010, 867, 1011, 866, 1011, 864, 1012, 863, 1012, 856, 1013, 855, 1013, 854, 1012, 853, 1011, 853, 1009, 851, 1009, 849, 1007, 847, 1007, 846, 1006, 845, 1006, 844, 1004, 842, 1004, 834, 1003, 833, 1003, 828, 1004, 827, 1004, 813, 1003, 812], [1075, 788, 1076, 787, 1078, 789, 1078, 791, 1077, 792, 1076, 791, 1076, 790, 1075, 789],...
'''
Is that mean one object include more than one polygon,or I can integrate these polygons into one polygon.

[pderrenger]

It seems that your segmentation output includes multiple polygons for a single object. This is normal as complex objects can be represented by multiple disjoint polygons. To handle this, you can integrate these polygons into one by merging them or processing them individually based on your requirements. If you need to convert these polygons into a specific format, you can use the to_imantics_annotations() method provided by SAHI. For more details, please refer to the SAHI documentation.

[wainuo]

Thank you so much,this is helpful!

[pderrenger]

You're welcome! If you need more detailed information on using YOLOv8 with SAHI for sliced inference, you can check out our SAHI documentation. It covers everything from installation to handling prediction results. If you have any more questions, feel free to ask!

[dearMOMO]

Slicing reasoning with YOLOv8 and SAHI How to do target tracking.please 🙏

[glenn-jocher]

@dearMOMO to perform target tracking using YOLOv8 and SAHI, you can follow the steps outlined in the Ultralytics documentation on sliced inference. SAHI allows you to partition large images into smaller slices, run object detection on each slice, and then stitch the results back together. This approach is particularly useful for high-resolution images and limited computational resources. For detailed instructions and code examples, please refer to the guide at https://docs.ultralytics.com/guides/sahi-tiled-inference/. This resource will provide you with all the necessary steps to implement SAHI with YOLOv8 for efficient and accurate object detection and tracking.

[felipetobars]

I would like to know how to adjust the IoU threshold used by the model from the SAHI commands.

[glenn-jocher]

@felipetobars to adjust the IoU threshold in SAHI commands, you can use the iou_threshold parameter when calling the get_sliced_prediction function. This allows you to set the desired IoU threshold for your inference tasks. For more detailed instructions, please refer to the SAHI documentation.

[cmilanes93]

Hi, this code make any sense?
import os
from sahi.slicing import slice_image
from sahi.annotation import BoundingBox
from sahi.utils.file import save_json

Function to load YOLO labels

def load_yolo_labels(label_path):
bounding_boxes = []
with open(label_path, 'r') as file:
for line in file:
parts = line.strip().split()
category_id = int(parts[0])
x_center, y_center, width, height = map(float, parts[1:])
bounding_boxes.append(BoundingBox.from_yolo_format(x_center, y_center, width, height, category_id))
return bounding_boxes

Directories for images and labels

image_dir = "path/to/your/images"
label_dir = "path/to/your/labels"
output_image_dir = "path/to/output/images"
output_label_dir = "path/to/output/labels"

Create output directories if they don't exist

os.makedirs(output_image_dir, exist_ok=True)
os.makedirs(output_label_dir, exist_ok=True)

Process each image and its corresponding label

for image_filename in os.listdir(image_dir):
if image_filename.endswith(".jpg"):
image_path = os.path.join(image_dir, image_filename)
label_path = os.path.join(label_dir, image_filename.replace(".jpg", ".txt"))

    # Load YOLO annotations
    bounding_boxes = load_yolo_labels(label_path)

    # Slice image and adjust annotations
    sliced_image_result = slice_image(
        image_path=image_path,
        output_dir=output_image_dir,
        slice_height=512,
        slice_width=512,
        overlap_height_ratio=0.2,
        overlap_width_ratio=0.2,
        annotations=bounding_boxes
    )

    # Save adjusted YOLO labels for each slice
    for slice_result in sliced_image_result:
        output_label_path = os.path.join(output_label_dir, os.path.basename(slice_result.image_path).replace('.jpg', '.txt'))
        with open(output_label_path, 'w') as file:
            for annotation in slice_result.annotations:
                yolo_format = annotation.to_yolo_format()
                file.write(f"{yolo_format['category_id']} {yolo_format['x_center']} {yolo_format['y_center']} {yolo_format['width']} {yolo_format['height']}\n")

The idea is to use sahi on labeled images to slice them, and then train the model, the idea is to adapt the code to a segmentation format of labeling

[glenn-jocher]

@cmilanes93 your code for slicing images and adjusting YOLO labels with SAHI looks well-structured. Ensure the BoundingBox and slice_image functions are correctly handling your specific segmentation needs. For more details on integrating SAHI with YOLOv8, you can refer to the SAHI documentation.

[cmilanes93]

Hi glenn, thanks for your answer, im getting this error:
bounding_box = BoundingBox.from_yolo_format(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: type object 'BoundingBox' has no attribute 'from_yolo_format'

I have tried to fixe it, but i keep finding new ones, any suggestion?

[glenn-jocher]

@cmilanes93 it seems like the BoundingBox class doesn't have a from_yolo_format method. Please check the SAHI documentation for the correct method to convert YOLO annotations. If you need further assistance, feel free to ask!

[mohammed01997]

Hello... If I wanted to calculate the system's performance, how can I do it exactly? I want to know if I put Yolo8 + Sahi in the server, what will be the image execution rate? Are there specific examples or a specific method?

[glenn-jocher]

@mohammed01997 to calculate the system's performance with YOLOv8 and SAHI, you can measure the image execution rate by running inference tests on your server. Use the get_sliced_prediction function to process images and monitor the time taken for each slice. This will give you an estimate of the execution rate. For more details, refer to the SAHI documentation.

[jorgemasgomez]

Hello! I want to use SAHI for segmentation prediction with a custom fine-tuned YOLO model, but I would like to export the segmentation contours in YOLO format as the output. Is there any way to do this? Thank you!

[jorgemasgomez]

I've seen that the function to_imantics_annotations() can be used to convert to COCO format, is that right? And from there, could I convert it to YOLO segmentation format, right?

[glenn-jocher]

@jorgemasgomez yes, you can use to_imantics_annotations() to convert predictions to COCO format. From there, you can transform the COCO annotations into YOLO segmentation format. For detailed steps, refer to the SAHI documentation.

[jorgemasgomez]

Thanks for the reply!
I'm fine-tuning YOLO for segmentation with a pre-trained model. Is there a way to integrate SAHI into the training/fine-tuning process? Right now, the only idea I have is to slice my images, segment the slices with CVAT, create the model, and then use that model to predict in SAHI. But I'm not sure if there is a way to segment directly on the original image, train/fine-tune the model with SAHI (so that SAHI does the slicing, etc.), and then use that model to predict in SAHI. I hope I've explained myself well. Thanks!

[glenn-jocher]

SAHI is primarily used for inference, not training. You can slice images manually for training, then use SAHI for inference with your fine-tuned model. For more on SAHI's capabilities, visit the documentation.

[Dasadaly]

Hi, can you please advise me how I can output the coordinates of segmentation objects? In YOLO I do something like this

    for result in results:
        coords_lst = [mask_xy.tolist() for mask_xy in result.masks.xy]

How do I get something like this out of sahi?

[glenn-jocher]

To extract segmentation coordinates using SAHI, you can access the object_prediction_list from the PredictionResult object. This list contains the details of each detected object, including their coordinates. For more information, please refer to the SAHI documentation.

[Dasadaly]

I tried extracting from object_prediction_list but it doesn't work, could you please elaborate on how to do it. Thanks for your support

[glenn-jocher]

To extract segmentation coordinates in SAHI, access the object_prediction_list from the PredictionResult and iterate over it to retrieve the coordinates. If you need further assistance, please refer to the SAHI documentation for detailed guidance.

[Dasadaly]

I was able to get the mask using result.mask.segmentation, but I understood that it outputs the whole mask by area, how can I get the mask of just the contour boundary?

[glenn-jocher]

To extract the contour boundary of masks using SAHI, you can iterate over the object_prediction_list and access the contour attribute for each object. This will give you the boundary coordinates. For more detailed guidance, please refer to the SAHI documentation.

[jorgemasgomez]

Hello! Is there any way to implement the retina_mask argument from the YOLO predict function in SAHI?

[glenn-jocher]

@jorgemasgomez hello! The retina_mask argument isn't directly supported in SAHI. For detailed guidance on integrating YOLO with SAHI, please refer to the SAHI Tiled Inference Guide.

[herrig]

How to save the txt file with the detections when performed using batch prediction? For example, save the txt file for each image to create an auto annotated dataset.

[glenn-jocher]

@herrig to save detection results as text files during batch prediction with SAHI, use the export_visuals method to specify the directory for saving outputs, which includes text files for each image. For further details, refer to the SAHI documentation.

[noval779]

I am using SAHI and YOLO11l-seg to segmentation large .tif image (100000 x 100000), but i getting error ValueError: Invalid segmentation mask., i have tried reinstalling but noting happens.

[noval779]

I have checked the model and image, and there is no issue, I tried to create the environment again with conda, but I have the same error. This is the code I used

from sahi import AutoDetectionModel
from sahi.predict import get_sliced_prediction

model_path = "..."
image_path = "..."

detection_model = AutoDetectionModel.from_pretrained(
    model_type='yolov8',
    model_path=model_path,
    confidence_threshold=0.7,
    device="cuda:0",
)

result = get_sliced_prediction(
    image_path,
    detection_model,
    slice_height=1500,
    slice_width=1500,
    overlap_height_ratio=0.1,
    overlap_width_ratio=0.1,
)

[noval779]

These are all the errors I get.

ValueError                                Traceback (most recent call last)
Cell In[20], line 15
      5 image_path = "..."
      7 detection_model = AutoDetectionModel.from_pretrained(
      8     model_type='yolov8',
      9     model_path=model_path,
     10     confidence_threshold=0.7,
     11     device="cuda:0",
     12 )
---> 15 result = get_sliced_prediction(
     16     image_path,
     17     detection_model,
     18     slice_height=1500,
     19     slice_width=1500,
     20     overlap_height_ratio=0.1,
     21     overlap_width_ratio=0.1,
     22 )

File c:\Users\Admin\anaconda3\envs\ultralytics\Lib\site-packages\sahi\predict.py:283, in get_sliced_prediction(image, detection_model, slice_height, slice_width, overlap_height_ratio, overlap_width_ratio, perform_standard_pred, postprocess_type, postprocess_match_metric, postprocess_match_threshold, postprocess_class_agnostic, verbose, merge_buffer_length, auto_slice_resolution, slice_export_prefix, slice_dir)
    281 # merge matching predictions
    282 if len(object_prediction_list) > 1:
--> 283     object_prediction_list = postprocess(object_prediction_list)
    285 time_end = time.time() - time_start
    286 durations_in_seconds["prediction"] = time_end

File c:\Users\Admin\anaconda3\envs\ultralytics\Lib\site-packages\sahi\postprocess\combine.py:555, in GreedyNMMPostprocess.__call__(self, object_predictions)
    548     for merge_ind in merge_ind_list:
    549         if has_match(
    550             object_prediction_list[keep_ind].tolist(),
    551             object_prediction_list[merge_ind].tolist(),
    552             self.match_metric,
    553             self.match_threshold,
    554         ):
--> 555             object_prediction_list[keep_ind] = merge_object_prediction_pair(
    556                 object_prediction_list[keep_ind].tolist(), object_prediction_list[merge_ind].tolist()
    557             )
    558     selected_object_predictions.append(object_prediction_list[keep_ind].tolist())
    560 return selected_object_predictions

File c:\Users\Admin\anaconda3\envs\ultralytics\Lib\site-packages\sahi\postprocess\utils.py:220, in merge_object_prediction_pair(pred1, pred2)
    218     segmentation = None
    219     full_shape = None
--> 220 return ObjectPrediction(
    221     bbox=merged_bbox.to_xyxy(),
    222     score=merged_score,
    223     category_id=merged_category.id,
    224     category_name=merged_category.name,
    225     segmentation=segmentation,
    226     shift_amount=shift_amount,
    227     full_shape=full_shape,
    228 )

File c:\Users\Admin\anaconda3\envs\ultralytics\Lib\site-packages\sahi\prediction.py:79, in ObjectPrediction.__init__(self, bbox, category_id, category_name, segmentation, score, shift_amount, full_shape)
     53 """
     54 Creates ObjectPrediction from bbox, score, category_id, category_name, segmentation.
     55 
   (...)
     76         the form of [height, width]
     77 """
     78 self.score = PredictionScore(score)
---> 79 super().__init__(
     80     bbox=bbox,
     81     category_id=category_id,
     82     segmentation=segmentation,
     83     category_name=category_name,
     84     shift_amount=shift_amount,
     85     full_shape=full_shape,
     86 )

File c:\Users\Admin\anaconda3\envs\ultralytics\Lib\site-packages\sahi\annotation.py:526, in ObjectAnnotation.__init__(self, bbox, segmentation, category_id, category_name, shift_amount, full_shape)
    524         bbox = bbox_from_segmentation
    525     else:
--> 526         raise ValueError("Invalid segmentation mask.")
    527 else:
    528     self.mask = None

ValueError: Invalid segmentation mask.

[glenn-jocher]

The error ValueError: Invalid segmentation mask occurs when SAHI encounters segmentation masks that are improperly formatted or invalid. To mitigate this, ensure your .tif file is compatible with SAHI and verify that the model you are using supports segmentation fully for your specific task. Additionally, you might try reducing the slice dimensions or adjusting overlap ratios. If the issue persists, you can review the SAHI slicing documentation at SAHI Tiled Inference for further assistance.

[noval779]

I try to downgrade the version SAHI from 0.11.19 to 0.11.18 and its work, maybe you can check again the new version (0.11.19).

[glenn-jocher]

Thank you for sharing your findings. It seems the issue is related to a regression in SAHI version 0.11.19. I recommend opening an issue directly on the SAHI GitHub repository to inform the maintainers and track any related updates: https://github.com/obss/sahi. We appreciate your feedback and collaboration!