Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support of overwriting signals if harsh backlight is detected #5852

Conversation

ktro2828
Copy link
Contributor

@ktro2828 ktro2828 commented Dec 12, 2023

Description

In the case of backlight, current traffic light classifier outputs incorrect color signals as video below.

original.mp4

In this PR, I added support of overwriting signals with unknown if the backlight is detected.
Check each RoI image whether backlight is and overwrite the signal with unknown corresponding to the RoI image.

According to my analysis, in the case of harsh backlight, the intensity values become higher. (RED: mean, CYAN: stddev)

backlight_analysis

Related links

TIER IV INTERNAL LINK

Tests performed

The result of backlight_threshold=0.85(default) in [0.0, 1.0].

output.mp4

Notes for reviewers

Interface changes

Effects on system behavior

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

  • The PR follows the pull request guidelines.
  • The PR has been properly tested.
  • The PR has been reviewed by the code owners.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

  • There are no open discussions or they are tracked via tickets.
  • The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

@github-actions github-actions bot added the component:perception Advanced sensor data processing and environment understanding. (auto-assigned) label Dec 12, 2023
@ktro2828 ktro2828 force-pushed the feat/traffic_light_classifier/backlit branch 6 times, most recently from 1ce08c0 to f9a976e Compare December 12, 2023 17:43
@ktro2828 ktro2828 marked this pull request as ready for review December 13, 2023 01:32
@ktro2828 ktro2828 requested a review from tzhong518 December 13, 2023 01:32
@yukkysaito
Copy link
Contributor

yukkysaito commented Dec 13, 2023

What camera did you use?

@ktro2828
Copy link
Contributor Author

ktro2828 commented Dec 13, 2023

@yukkysaito Leopard Imaging LI-IMX490 was used in this rosbag

@ktro2828 ktro2828 force-pushed the feat/traffic_light_classifier/backlit branch from f9a976e to a7f9cf2 Compare December 13, 2023 03:51
@yukkysaito
Copy link
Contributor

yukkysaito commented Dec 13, 2023

@ktro2828

Understood. While this feature seems promising, I'll note an alternative approach here:

  • Implementing HDR for camera support.
  • Enhancing training accuracy by avoiding detection if not achievable with the fine detector.
  • Training the classifier to recognize "unknown" classes.

Since the parameters here may change depending on the camera, the thresholds should be very generous.

@ktro2828
Copy link
Contributor Author

ktro2828 commented Dec 13, 2023

@ktro2828

Understood. While this feature seems promising, I'll note an alternative approach here:

  • Implementing HDR for camera support.
  • Enhancing training accuracy by avoiding detection if not achievable with the fine detector.
  • Training the classifier to recognize "unknown" classes.

@yukkysaito Yes, the points you mentioned are reasonable, and there is a movement aiming to improve with the way like these in our team.

Since the parameters here may change depending on the camera, the thresholds should be very generous.

I agree with you. In terms of this PR, I think current default threshold is higher enough as far as I experimented.

@ktro2828 ktro2828 self-assigned this Dec 14, 2023
@ktro2828 ktro2828 requested a review from YoshiRi as a code owner December 19, 2023 18:06
@github-actions github-actions bot added type:documentation Creating or refining documentation. (auto-assigned) component:launch Launch files, scripts and initialization tools. (auto-assigned) labels Dec 19, 2023
@miursh miursh added the run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Dec 20, 2023
Copy link

codecov bot commented Dec 20, 2023

Codecov Report

Attention: 18 lines in your changes are missing coverage. Please review.

Comparison is base (2edd55e) 15.29% compared to head (e6cb61b) 15.26%.
Report is 9 commits behind head on main.

Files Patch % Lines
...erception/traffic_light_classifier/src/nodelet.cpp 0.00% 18 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5852      +/-   ##
==========================================
- Coverage   15.29%   15.26%   -0.03%     
==========================================
  Files        1750     1753       +3     
  Lines      120566   120741     +175     
  Branches    36783    36783              
==========================================
  Hits        18435    18435              
- Misses      81465    81640     +175     
  Partials    20666    20666              
Flag Coverage Δ *Carryforward flag
differential 0.00% <0.00%> (?)
total 15.29% <ø> (+<0.01%) ⬆️ Carriedforward from 2edd55e

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ktro2828 ktro2828 force-pushed the feat/traffic_light_classifier/backlit branch from 51639ec to e6cb61b Compare December 26, 2023 05:04
Copy link
Contributor

@miursh miursh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ktro2828 ktro2828 merged commit 743eb0e into autowarefoundation:main Dec 28, 2023
21 of 24 checks passed
@ktro2828 ktro2828 deleted the feat/traffic_light_classifier/backlit branch April 2, 2024 18:04
karishma1911 pushed a commit to Interplai/autoware.universe that referenced this pull request May 26, 2024
…ed (autowarefoundation#5852)

* feat: add support of overwriting signals if backlit is detected

Signed-off-by: ktro2828 <[email protected]>

* feat: remove default parameter in nodelet and update lauch for composable node

Signed-off-by: ktro2828 <[email protected]>

* docs: update README

Signed-off-by: ktro2828 <[email protected]>

* docs: update README

Signed-off-by: ktro2828 <[email protected]>

* feat: update confidence to 0.0 corresponding signals overwritten by unkonwn

Signed-off-by: ktro2828 <[email protected]>

---------

Signed-off-by: ktro2828 <[email protected]>
karishma1911 pushed a commit to Interplai/autoware.universe that referenced this pull request May 28, 2024
…ed (autowarefoundation#5852)

* feat: add support of overwriting signals if backlit is detected

Signed-off-by: ktro2828 <[email protected]>

* feat: remove default parameter in nodelet and update lauch for composable node

Signed-off-by: ktro2828 <[email protected]>

* docs: update README

Signed-off-by: ktro2828 <[email protected]>

* docs: update README

Signed-off-by: ktro2828 <[email protected]>

* feat: update confidence to 0.0 corresponding signals overwritten by unkonwn

Signed-off-by: ktro2828 <[email protected]>

---------

Signed-off-by: ktro2828 <[email protected]>
karishma1911 pushed a commit to Interplai/autoware.universe that referenced this pull request May 28, 2024
…ed (autowarefoundation#5852)

* feat: add support of overwriting signals if backlit is detected

Signed-off-by: ktro2828 <[email protected]>

* feat: remove default parameter in nodelet and update lauch for composable node

Signed-off-by: ktro2828 <[email protected]>

* docs: update README

Signed-off-by: ktro2828 <[email protected]>

* docs: update README

Signed-off-by: ktro2828 <[email protected]>

* feat: update confidence to 0.0 corresponding signals overwritten by unkonwn

Signed-off-by: ktro2828 <[email protected]>

---------

Signed-off-by: ktro2828 <[email protected]>
karishma1911 pushed a commit to Interplai/autoware.universe that referenced this pull request Jun 3, 2024
…ed (autowarefoundation#5852)

* feat: add support of overwriting signals if backlit is detected

Signed-off-by: ktro2828 <[email protected]>

* feat: remove default parameter in nodelet and update lauch for composable node

Signed-off-by: ktro2828 <[email protected]>

* docs: update README

Signed-off-by: ktro2828 <[email protected]>

* docs: update README

Signed-off-by: ktro2828 <[email protected]>

* feat: update confidence to 0.0 corresponding signals overwritten by unkonwn

Signed-off-by: ktro2828 <[email protected]>

---------

Signed-off-by: ktro2828 <[email protected]>
t4-x2 pushed a commit to tier4/autoware.universe that referenced this pull request Jun 10, 2024
…ed (autowarefoundation#5852)

* feat: add support of overwriting signals if backlit is detected

Signed-off-by: ktro2828 <[email protected]>

* feat: remove default parameter in nodelet and update lauch for composable node

Signed-off-by: ktro2828 <[email protected]>

* docs: update README

Signed-off-by: ktro2828 <[email protected]>

* docs: update README

Signed-off-by: ktro2828 <[email protected]>

* feat: update confidence to 0.0 corresponding signals overwritten by unkonwn

Signed-off-by: ktro2828 <[email protected]>

---------

Signed-off-by: ktro2828 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:launch Launch files, scripts and initialization tools. (auto-assigned) component:perception Advanced sensor data processing and environment understanding. (auto-assigned) run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) type:documentation Creating or refining documentation. (auto-assigned)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants