Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add overview diagram of the perception and rearrange README file #501

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Add overview diagram of the perception and rearrange README file #501

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
18318b4
Add overview diagram of the perception and rearrange README file
Nov 13, 2024
b506f0e
Add camera to overview diagram and adjust the vision node summary sen…
Dec 9, 2024
0286e81
Merge branch 'main' into 500-general-add-an-overview-of-the-perceptio…
asamluka Dec 15, 2024
6e2ac95
Merge branch 'main' into 500-general-add-an-overview-of-the-perceptio…
asamluka Dec 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added doc/assets/perception/perception_overview.png
View file
Open in desktop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not obvious where Center/image, Back/image... come from. Maybe you can add the camera-module to the graph

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you suggest a box similar to the one used for localization? I'm not sure if the graph would then still be consistent with the other topics from the other sensors. Do you think the topics from all sensors should get an own color?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was more like referring to the source of the topics. Like for the 'Center/dist_array' and co, you can trace back, that the values somehow come from 'LIDAR' (over the 'lidar_distance' node), which is a CARLA module. Because now it seems like, the 'Camera/image' and co just come out of thin air. To solve I would just add "Camera" and four arrows to 'Center/image' and co.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
48 changes: 29 additions & 19 deletions doc/perception/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,26 +1,36 @@
# Documentation of perception component

This folder contains further documentation of the perception components.

1. [Vision Node](./vision_node.md)
- The Visison Node provides an adaptive interface that is able to perform object-detection and/or image-segmentation on multiple cameras at the same time.
2. [Position Heading Filter Debug Node](./position_heading_filter_debug_node.md)
3. [Kalman Filter](./kalman_filter.md)
4. [Position Heading Publisher Node](./position_heading_publisher_node.md)
5. [Distance to Objects](./distance_to_objects.md)
6. [Traffic Light Detection](./traffic_light_detection.md)
7. [Coordinate Transformation (helper functions)](./coordinate_transformation.md)
8. [Dataset Generator](./dataset_generator.md)
9. [Dataset Structure](./dataset_structure.md)
10. [Lidar Distance Utility](./lidar_distance_utility.md)
1. not used since paf22
11. [Efficient PS](./efficientps.md)
1. not used scince paf22 and never successfully tested

## Overview Localization
This folder contains documentation of the perception components.

In the following diagram you can see all the nodes and topics used in the perception to show how they work together.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In the following diagram you can see all the nodes and topics used in the perception to show how they work together.
In the following diagram you can see all the nodes and topics used in the perception, to show how they work together.


![Perception Overview](../../doc/assets/perception/perception_overview.png)

## Object Detection / Distance and Segmentation

- [Vision Node](./vision_node.md)
- The Vision Node provides an adaptive interface that is able to perform object detection and/or image segmentation on multiple cameras at the same time.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does "and/or" mean in this case? Are there restrictions to the simultaneous execution?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just copied this summary sentence from the "old" README file. The other nodes don't get a summary so I wasn't sure if I should leave it in but as the vision_node is quite important I thought a short summary might be nice. I think some models that can be selected in the node can only do object detection or image segmentation while others can do both, right? I think that's why "and/or" was used.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As of now, the vision node is capable of providing object detection via the NN with YOLO/Ultralytics and image segmentation for the traffic lights. As far as I know. So I would write 'and'.
But at the same time, I would clarify, that at the current state it is not possible to use multiple cameras (correct me if I'm wrong)

- [Distance to Objects](./distance_to_objects.md)
- [Traffic Light Detection](./traffic_light_detection.md)
- [Dataset Generator](./dataset_generator.md)
- [Dataset Structure](./dataset_structure.md)

## Localization

An overview over the different nodes working together to localize the vehicle is provided in the [localization](./localization.md) file.

- [Kalman Filter](./kalman_filter.md)
- [Position Heading Publisher Node](./position_heading_publisher_node.md)
- [Position Heading Filter Debug Node](./position_heading_filter_debug_node.md)
- [Coordinate Transformation](./coordinate_transformation.md) (helper functions)

## Unused files

- [Lidar Distance Utility](./lidar_distance_utility.md)
- Not used since paf22
- [Efficient PS](./efficientps.md)
- Not used scince paf22 and never successfully tested

## Experiments

- The overview of performance evaluations is located in the [experiments](./experiments/README.md) folder.
The overview of performance evaluations is located in the [experiments](./experiments/README.md) folder.
Loading