Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(image_projection_based_fusion): add cache for camera projection #9635

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

a-maumau
Copy link
Contributor

@a-maumau a-maumau commented Dec 12, 2024

Description

This PR introduces a caching feature for calculating raw image projections (unrectifyPoint).
This PR must merged with autowarefoundation/autoware_launch#1275

It introduces the CameraProjection class to handle the related processes. The cache is computed based on specific grid centers, and the projected points are represented using these grid centers. These grid centers are determined by approximation_grid_width_size and approximation_grid_height_size.

For example, when approximation_grid_*_size is set to 1.0 pixel, the grid centers will be created as shown in the following image:
image_projection_based_fusion_PR_image

Note that edge pixels ((0, *), (w, *), (*, 0), (*, h)) will be rounded to fall outside the image bounds.

When using caching, memory consumption is calculated as:
ceil(image_width / grid_size_w) * ceil(image_height / grid_size_h) * (4 * 2) * roi_numbers [bytes]

Here, 4 * 2 represents the bytes for two float values.
For example, for an image with dimensions 1440 × 1080, a grid size of 1 pixel, and caching enabled for 6 cameras, the memory consumption is:
(1440 / 1.0) * (1080 / 1.0) * 4 * 2 * 6 / (1024 * 1024) ≈ 71.2 MiB (theoretical).

other changes

  • This PR fixes a bug where the camera parameters were used before being properly initialized in pointpainting_fusion.
  • take a list input instead of single bool for point_project_to_unrectified_image param

[add speed comparison figure]

Related links

Parent Issue:

  • Link

How was this PR tested?

Notes for reviewers

None.

Interface changes

None.

Effects on system behavior

None.

@github-actions github-actions bot added type:documentation Creating or refining documentation. (auto-assigned) component:perception Advanced sensor data processing and environment understanding. (auto-assigned) tag:require-cuda-build-and-test labels Dec 12, 2024
Copy link

github-actions bot commented Dec 12, 2024

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

@a-maumau a-maumau force-pushed the mau/perf/image_projection_based_fusion/add_projection_cache branch 2 times, most recently from 3a0b75c to be70bd7 Compare December 12, 2024 09:24
@technolojin technolojin self-assigned this Dec 13, 2024
@a-maumau a-maumau force-pushed the mau/perf/image_projection_based_fusion/add_projection_cache branch from d518133 to bf52055 Compare December 16, 2024 07:08
@github-actions github-actions bot added the component:planning Route planning, decision-making, and navigation. (auto-assigned) label Dec 16, 2024
@a-maumau a-maumau force-pushed the mau/perf/image_projection_based_fusion/add_projection_cache branch from bf52055 to 6e00c9c Compare December 16, 2024 07:51
@github-actions github-actions bot removed the component:planning Route planning, decision-making, and navigation. (auto-assigned) label Dec 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:perception Advanced sensor data processing and environment understanding. (auto-assigned) tag:require-cuda-build-and-test type:documentation Creating or refining documentation. (auto-assigned)
Projects
Status: To Triage
Development

Successfully merging this pull request may close these issues.

2 participants