feat: unrecify image projection based fusion #7030
Conversation
Signed-off-by: Yukihiro Saito <[email protected]>
…ojection-based-fusion
Signed-off-by: Yukihito Saito <[email protected]>
…ction-based-fusion' into feature/unrecify-image-projection-based-fusion
yukkysaito
requested review from
badai-nguyen,
kminoda,
ktro2828,
miursh,
tzhong518 and
YoshiRi
as code owners
github-actions
bot
added
the
component:perception
|
|
||
| cv::Point2d raw_image_point = rectified_image_point; | ||
|
|
||
| try { |
There was a problem hiding this comment.
What is this try sentence for?
There was a problem hiding this comment.
The code for determining whether to unrectify in the library of image_geometry seems to change occasionally. If it can't be done, it is thrown, so I want to be able to catch it.
There was a problem hiding this comment.
Hmm, understood. It is a difficult decision, but IMHO we should avoid catching this error and just logging warning without killing the node, since it will be difficult for developers to be aware of a breaking change or a bug in image_geometry library, increasing a risk of a potential degradation of Autoware. What do you think?
There was a problem hiding this comment.
@kminoda So, what exactly do you want to do?
There was a problem hiding this comment.
Please see here #7030 (comment)
There was a problem hiding this comment.
@kminoda This time, when d is 0 in the image_geometry class, it is determined that the image is not calibrated, and thus there is no need to unrectify, resulting in an exception being thrown. This determination of being uncalibrated seems to change frequently in the code. Even if an exception is thrown, there is no need to terminate the process since it only means that unrectifying is not possible. Wouldn't it be better to skip this with a try-catch block?
There was a problem hiding this comment.
I will close this PR as there seems to have been a misunderstanding on my part.
| try { | ||
| const bool need_unrectify = std::any_of( | ||
| pinhole_camera_model.cameraInfo().d.begin(), pinhole_camera_model.cameraInfo().d.end(), | ||
| [](double distortion) { return distortion != 0.0; }); | ||
|
|
||
| if (need_unrectify) { | ||
| raw_image_point = pinhole_camera_model.unrectifyPoint(rectified_image_point); | ||
| } | ||
| } catch (cv::Exception & e) { | ||
| RCLCPP_WARN_STREAM(rclcpp::get_logger("image_projection_based_fusion"), e.what()); | ||
| } |
There was a problem hiding this comment.
| try { | |
| const bool need_unrectify = std::any_of( | |
| pinhole_camera_model.cameraInfo().d.begin(), pinhole_camera_model.cameraInfo().d.end(), | |
| [](double distortion) { return distortion != 0.0; }); | |
| if (need_unrectify) { | |
| raw_image_point = pinhole_camera_model.unrectifyPoint(rectified_image_point); | |
| } | |
| } catch (cv::Exception & e) { | |
| RCLCPP_WARN_STREAM(rclcpp::get_logger("image_projection_based_fusion"), e.what()); | |
| } | |
| const bool need_unrectify = std::any_of( | |
| pinhole_camera_model.cameraInfo().d.begin(), pinhole_camera_model.cameraInfo().d.end(), | |
| [](double distortion) { return distortion != 0.0; }); | |
| if (need_unrectify) { | |
| raw_image_point = pinhole_camera_model.unrectifyPoint(rectified_image_point); | |
| } |
Regarding https://github.com/autowarefoundation/autoware.universe/pull/7030/files#r1604255216, here's my proposal.
Description
Fixed this PR
In image_projection_based_fusion, only the internal parameters of the camera were considered, and no distortion correction was applied. As a result, when projecting onto images with significant distortion, misalignments occurred. This Pull Request has been modified to account for distortion.
Related links
Tests performed
In roi_cluster_fusion, operation has been confirmed on the actual machine.
before
after
Notes for reviewers
Interface changes
Effects on system behavior
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.
In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.
Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.
After all checkboxes are checked, anyone who has write access can merge the PR.