Skip to content

Add multi-points input, foreground/background points input and box input to EfficientSAM model #291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 20 commits into
base: main
Choose a base branch
from

Conversation

Zhang-Yang-Sustech
Copy link
Contributor

@Zhang-Yang-Sustech Zhang-Yang-Sustech commented Apr 23, 2025

The EfficientSAM model in the model zoo can only support single point input to infer before. This time I change the model input dimension to 6, and add a dynamic supplement method to support multi-points input. Users can input 6 points to select segment object at most. Additionally, I select from the output 3 masks to avoid to pick the mask with background points. I also add foreground/background points and box input in the demo.

  1. new model(image_segmentation_efficientsam_ti_2024may.onnx) with new input dimension:
    'batched_images': (1, 3, 1024, 1024), previous is (1, 3, 640, 640);
    'batched_point_coords':(1, 1, 6, 2), previous is (1, 1, 1, 2), can only support single point input, now can support 6 points;
    'batched_point_labels':(1, 1, 6), previous is (1, 1, 1), only one label, now each point one label.
    with new output dimension:
    'output_masks': (1, 1, 3, 1024, 1024), previous is (1, 1, 1, 640, 640), can only output single mask, now output three mask, we can select the mask we want to enhance performance;
    'iou_predictions': (1, 1, 3), new output which is used to select the masks by scores.
  2. new model file(efficientSAM.py), with dynamical supplement points method to support multi-points input in preprocess, with masks selection method in postprocess.
  3. new demo file(demo.py), with foreground/background points input and box input.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant