Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plots: Add the bounding boxes informations in the dvc plots diff command so that VScode can display them. #10312

Closed
wants to merge 5 commits into from

Conversation

AlexandreKempf
Copy link
Contributor

@AlexandreKempf AlexandreKempf commented Feb 20, 2024

Works with this DVClive PR and this VSCode PR.

DVClive is responsible for saving JSON annotation files. The annotations files have the same path as the image they match, but ends with .json.

Example:

plots/images/dogs.png
plots/images/dogs.json

Or for step images:

plots/images/cats/0.png
plots/images/cats/0.json
plots/images/cats/1.png
plots/images/cats/1.json

The JSON files have this format:

{
    "annotations": {
        "person": [
            {
                "box": {
                    "top": 1,
                    "left": 10,
                    "bottom": 30,
                    "right": 40
                },
                "score": 0.8
            }
        ],
        "dog": [
            {
                "box": {
                    "top": 200,
                    "left": 50,
                    "bottom": 240,
                    "right": 160
                },
                "score": 0.3
            },
            {
                "box": {
                    "top": 57,
                    "left": 34,
                    "bottom": 246,
                    "right": 144
                },
                "score": 0.2
            }
        ]
    }
}

After the dvc plots diff --split --json the expected output is to have the annotations alongside the image

{
  "data": {
    "plots/bounding_boxes.png": [
      {
        "type": "image",
        "revisions": [
          "workspace"
        ],
        "url": "//bounding_boxes.png",
        "annotations": {
          ... content from the json file "annotations" field should be here
        }
      }
    ]
  }
}

⚠️ Mising unit tests, I might need help on this. Tests in this repo are complicated to navigate on a first PR.

Fixes #10198

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

@dberenbaum dberenbaum requested review from mattseddon and a team February 20, 2024 18:25
@dberenbaum
Copy link
Collaborator

@AlexandreKempf I guess the tests are still WIP?

Comment on lines 475 to 510
if ImageRenderer.matches(path) and annotation_file in target["data"]:
annotations = {"annotations": annotation_file}
# empty dict all share the same reference, so override them
if target["data"][path]:
target["data"][path].update({"annotations": annotation_file})
else:
target["data"][path] = annotations
path_to_remove.append(annotation_file)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain a bit what's happening here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know if it is clear now ;)

@mattseddon
Copy link
Member

Needs tests

@skshetry
Copy link
Member

skshetry commented Mar 1, 2024

Hey @mattseddon. Could you please take a look at this PR (and maybe takeover)?

@mattseddon mattseddon self-assigned this Mar 4, 2024
Copy link

codecov bot commented Mar 5, 2024

Codecov Report

Attention: Patch coverage is 56.89655% with 25 lines in your changes are missing coverage. Please review.

Project coverage is 90.58%. Comparing base (b0384f8) to head (7b5a55a).

Files Patch % Lines
dvc/render/converter/image.py 59.45% 13 Missing and 2 partials ⚠️
dvc/repo/plots/__init__.py 47.36% 7 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #10312      +/-   ##
==========================================
- Coverage   90.63%   90.58%   -0.06%     
==========================================
  Files         500      500              
  Lines       38626    38675      +49     
  Branches     5584     5594      +10     
==========================================
+ Hits        35010    35034      +24     
- Misses       2971     2991      +20     
- Partials      645      650       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mattseddon
Copy link
Member

will get back to this later

@mattseddon mattseddon closed this Mar 12, 2024
@mattseddon mattseddon deleted the bbox-supported-in-plots branch March 12, 2024 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

plots: interactive plots with toggling bounding box
4 participants