Skip to content

Commit

Permalink
Update 07_vision_node.md
Browse files Browse the repository at this point in the history
  • Loading branch information
okrusch authored Nov 27, 2023
1 parent eacb5d0 commit f183cc7
Showing 1 changed file with 6 additions and 9 deletions.
15 changes: 6 additions & 9 deletions doc/06_perception/07_vision_node.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,17 @@ The following code shows how the Vision-Node is specified in perception.launch
<node pkg="perception" type="vision_node.py" name="VisionNode" output="screen">
<param name="role_name" value="$(arg role_name)" />
<param name="side" value="Center" />
<!--
Object-Detection:
- fasterrcnn_resnet50_fpn_v2
<!--
Object-Detection:
- fasterrcnn_resnet50_fpn_v2
- fasterrcnn_mobilenet_v3_large_320_fpn
Image-Segmentation:
- deeplabv3_resnet101
- deeplabv3_resnet101
-->
<param name="model" value="deeplabv3_resnet101" />
</node>
`


Depending on preferences and targets a different model can be used by replacing the value of the model parameter
by one of the lines from the comment above.

Expand All @@ -32,7 +31,6 @@ The Vision-Node will automatically switch between object-detection, imagesegment
For now the Vision-Node only supports pyTorch models. Within the next sprint it should be able to
accept other frameworks aswell. It should also be possible to run object-detection and image-segmentation at the same time.


## How it works

### Initialization
Expand Down Expand Up @@ -61,7 +59,6 @@ This function is automatically triggered by the Camera-Subscriber of the Vision-
5. Convert CV2-Image to ImageMsg
6. Publish ImageMsg over ImagePublisher


## Visualization

The Vision-Node implements an ImagePublisher under the topic: "/paf//Center/segmented_image"
Expand All @@ -72,10 +69,10 @@ The Configuartion File of RViz has been changed accordingly to display the publi

### Time

First experiments showed that the handle_camera_image function is way to slow to be used reliably. It takes around 1.5 seconds to handle one image.
First experiments showed that the handle_camera_image function is way to slow to be used reliably. It takes around 1.5 seconds to handle one image.

Right now the Vision-Node is not using cuda due to cuda-memory-issues that couldn't be fixed right away.

The performance is expected to rise quite a bit when using cuda.

Also their is lots more room for testing different models inside the Vision-Node to evualte their accuracy and time-performance.
Also their is lots more room for testing different models inside the Vision-Node to evualte their accuracy and time-performance.

0 comments on commit f183cc7

Please sign in to comment.