Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

368 visionnode and segmentation #404

Merged
merged 13 commits into from
Nov 4, 2024
Merged

368 visionnode and segmentation #404

Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
4a1d131
Add files via upload
RoyaLxPole Oct 30, 2024
ddfa0d0
Create segmentation.md
RoyaLxPole Oct 30, 2024
8dca793
Add files via upload
RoyaLxPole Oct 30, 2024
22bc531
Update segmentation.md
RoyaLxPole Oct 30, 2024
4ba4706
Update segmentation.md
RoyaLxPole Oct 30, 2024
8d08b45
Delete doc/assets/research_assets/Carla_Segmentation_Sensor.png
RoyaLxPole Oct 31, 2024
3ad9f53
Update segmentation.md
RoyaLxPole Oct 31, 2024
22c8434
Update segmentation.md
RoyaLxPole Oct 31, 2024
3b70699
Delete doc/assets/perception/Carla_Segmentation_Sensor.png
RoyaLxPole Oct 31, 2024
dc1deff
Merge branch 'main' into 368-visionnode-and-segmentation
RoyaLxPole Oct 31, 2024
8bfb710
Update segmentation.md to pass linter
RoyaLxPole Oct 31, 2024
5cbcd68
Update segmentation.md linter update
RoyaLxPole Oct 31, 2024
62afa47
Merge branch 'main' into 368-visionnode-and-segmentation
RoyaLxPole Nov 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions doc/research/paf24/segmentation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@

# Segmentation

**Summary:** This page contains the research into the segmentation component of Autoware.

- [Segmentation](#Segmentation)
- [Already implemented solutions](#already-implemented-solutions)
- [Implemented but dropped](#implemented-but-dropped)
- [Carla Sensors](#carla-sensors)
- [Follow-up Question](#follow-up-question)

## Already implemented solutions

<https://github.com/una-auxme/paf/blob/c3011ee70039e199e106c54aa162a8f52be241a6/code/perception/launch/perception.launch?plain=1#L59-L61>

probably trained with the generated dataset:
<https://github.com/una-auxme/paf/blob/8e8f9a1a03ae09d5ac763c1a11b398fc1ce144b0/code/perception/src/dataset_generator.py#L109-L110>

## Implemented but dropped

<https://github.com/una-auxme/paf/blob/8c968fb5c6c44c15b2733c5a181c496eb9b244be/doc/perception/efficientps.md#efficientps>

## Carla Sensors

![Alt text](https://carla.readthedocs.io/en/0.9.14/img/ref_sensors_semantic.jpg)
![Alt text](https://carla.readthedocs.io/en/0.9.14/img/tuto_sem.jpg)

for more context:
the pedestrian walks will be labled as roadlines

example:

```Python

# --------------
# Add a new semantic segmentation camera to my ego
# --------------
sem_cam = None
sem_bp = world.get_blueprint_library().find('sensor.camera.semantic_segmentation')
sem_bp.set_attribute("image_size_x",str(1920))
sem_bp.set_attribute("image_size_y",str(1080))
sem_bp.set_attribute("fov",str(105))
sem_location = carla.Location(2,0,1)
sem_rotation = carla.Rotation(0,180,0)
sem_transform = carla.Transform(sem_location,sem_rotation)
sem_cam = world.spawn_actor(sem_bp,sem_transform,attach_to=ego_vehicle, attachment_type=carla.AttachmentType.Rigid)
# This time, a color converter is applied to the image, to get the semantic segmentation view
sem_cam.listen(lambda image: image.save_to_disk('tutorial/new_sem_output/%.6d.jpg' % image.frame,carla.ColorConverter.CityScapesPalette))

```

For more information:
<https://carla.readthedocs.io/en/0.9.14/ref_sensors/#semantic-segmentation-camera:~:text=the%20object%20it.-,Semantic%20segmentation%20camera,-Blueprint%3A%20sensor>

<https://carla.readthedocs.io/en/0.9.14/tuto_G_retrieve_data/#semantic-segmentation-camera:~:text=on%20the%20right.-,Semantic%20segmentation%20camera,-The%20semantic%20segmentation>

## Follow up Question

Why the last group used bounding boxes and not the segmentation model is it to slow or maybe not reliable?
Comment on lines +57 to +59
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Improve grammar and clarity in the follow-up question.

The question needs better formatting and grammar.

Apply this diff:

-## Follow up Question
+## Follow-up Question
 
-Why the last group used bounding boxes and not the segmentation model is it to slow or maybe not reliable?
+Why did the previous group use bounding boxes instead of the segmentation model? Was it too slow or perhaps not reliable enough?
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
## Follow up Question
Why the last group used bounding boxes and not the segmentation model is it to slow or maybe not reliable?
## Follow-up Question
Why did the previous group use bounding boxes instead of the segmentation model? Was it too slow or perhaps not reliable enough?
🧰 Tools
🪛 LanguageTool

[grammar] ~57-~57: ‘Follow up’ seems to be a compound adjective before a noun. Use a hyphen: “Follow-up”.
Context: ...era,-The%20semantic%20segmentation> ## Follow up Question Why the last group used bound...

(CA_FOLLOW_UP)

[uncategorized] ~59-~59: Possible missing comma found.
Context: ...bounding boxes and not the segmentation model is it to slow or maybe not reliable?

(AI_HYDRA_LEO_MISSING_COMMA)

[uncategorized] ~59-~59: “to” seems less likely than “too” (as well, very, ‘too much’).
Context: ...es and not the segmentation model is it to slow or maybe not reliable?

(AI_HYDRA_LEO_CP_TO_TOO)

Loading