feat(design/autoware-architecture/sensing): add radar sensing document (

autowarefoundation#518) * add radar sensing design document Signed-off-by: scepter914 <[email protected]> * style(pre-commit): autofix * update document Signed-off-by: scepter914 <[email protected]> * fix typo Signed-off-by: scepter914 <[email protected]> * fix from supported-function to reference-implementation Signed-off-by: scepter914 <[email protected]> * fix from supported-function to reference-implementation Signed-off-by: scepter914 <[email protected]> * Update docs/design/autoware-architecture/sensing/data-types/radar-data/reference-implementations/device-driver.md Co-authored-by: Shintaro Tomie <[email protected]> Signed-off-by: scepter914 <[email protected]> * Update docs/design/autoware-architecture/sensing/data-types/radar-data/radar-objects-data.md Co-authored-by: Shintaro Tomie <[email protected]> Signed-off-by: scepter914 <[email protected]> * Update docs/design/autoware-architecture/sensing/data-types/radar-data/radar-objects-data.md Co-authored-by: Shintaro Tomie <[email protected]> Signed-off-by: scepter914 <[email protected]> * Update docs/design/autoware-architecture/sensing/data-types/radar-data/radar-objects-data.md Co-authored-by: Shintaro Tomie <[email protected]> Signed-off-by: scepter914 <[email protected]> * Update docs/design/autoware-architecture/sensing/data-types/radar-data/radar-objects-data.md Co-authored-by: Shintaro Tomie <[email protected]> Signed-off-by: scepter914 <[email protected]> * Update docs/design/autoware-architecture/sensing/data-types/radar-data/reference-implementations/data-message.md Co-authored-by: Shintaro Tomie <[email protected]> Signed-off-by: scepter914 <[email protected]> * Update docs/design/autoware-architecture/sensing/data-types/radar-data/reference-implementations/data-message.md Co-authored-by: Shintaro Tomie <[email protected]> Signed-off-by: scepter914 <[email protected]> * Update docs/design/autoware-architecture/sensing/data-types/radar-data/radar-pointcloud-data.md Co-authored-by: Shintaro Tomie <[email protected]> Signed-off-by: scepter914 <[email protected]> * Update docs/design/autoware-architecture/sensing/data-types/radar-data/radar-pointcloud-data.md Co-authored-by: Shintaro Tomie <[email protected]> Signed-off-by: scepter914 <[email protected]> * Update docs/design/autoware-architecture/sensing/data-types/radar-data/radar-objects-data.md Co-authored-by: Shintaro Tomie <[email protected]> Signed-off-by: scepter914 <[email protected]> * fix documents Signed-off-by: scepter914 <[email protected]> --------- Signed-off-by: scepter914 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Shintaro Tomie <[email protected]>
tier4 · Feb 21, 2024 · 95cf4b2 · 95cf4b2
1 parent efec0c2
commit 95cf4b2
Show file tree

Hide file tree

Showing 11 changed files with 1,650 additions and 7 deletions.
diff --git a/docs/design/autoware-architecture/sensing/data-types/radar-data.md b/docs/design/autoware-architecture/sensing/data-types/radar-data.md
diff --git a/...chitecture/sensing/data-types/radar-data/image/radar-objects-sensing.drawio.svg b/...chitecture/sensing/data-types/radar-data/image/radar-objects-sensing.drawio.svg
diff --git a/...tecture/sensing/data-types/radar-data/image/radar-pointcloud-sensing.drawio.svg b/...tecture/sensing/data-types/radar-data/image/radar-pointcloud-sensing.drawio.svg
diff --git a/...esign/autoware-architecture/sensing/data-types/radar-data/radar-objects-data.md b/...esign/autoware-architecture/sensing/data-types/radar-data/radar-objects-data.md
@@ -0,0 +1,88 @@
+# Radar objects data pre-processing design
+
+## Overview
+
+### Pipeline
+
+This diagram describes the pre-process pipeline for radar objects.
+
+![radar-objects-sensing](image/radar-objects-sensing.drawio.svg)
+
+### Interface
+
+- Input
+  - Radar data from device
+  - Twist information of ego vehicle motion
+- Output
+  - Merged radar objects information
+
+### Note
+
+- The radar pre-process package filters noise through the `ros-perception/radar_msgs/msg/RadarTrack.msg` message type with sensor coordinate.
+- It is recommended to change the coordinate system from each sensor to base_link with message converter.
+- If there are multiple radar objects, the object merger package concatenates these objects.
+
+## Input
+
+Autoware uses radar objects data type as [radar_msgs/msg/RadarTracks.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarTracks.msg).
+In detail, please see [Data message for radars](reference-implementations/data-message.md).
+
+## Reference implementations
+
+### Device driver for radars
+
+Autoware supports `ros-perception/radar_msgs/msg/RadarScan.msg` and `autoware_auto_perception_msgs/msg/TrackedObjects.msg` for Radar drivers.
+
+In detail, please see [Device driver for radars](reference-implementations/device-driver.md).
+
+### Noise filter
+
+- [radar_tracks_noise_filter](https://github.com/autowarefoundation/autoware.universe/tree/main/sensing/radar_tracks_noise_filter)
+
+Radar can detect x-axis velocity as doppler velocity, but cannot detect y-axis velocity. Some radar can estimate y-axis velocity inside the device, but it sometimes lack precision. This package treats these objects as noise by y-axis threshold filter.
+
+### Message converter
+
+- [radar_tracks_msgs_converter](https://github.com/autowarefoundation/autoware.universe/tree/main/perception/radar_tracks_msgs_converter)
+
+This package converts from `radar_msgs/msg/RadarTracks` into `autoware_auto_perception_msgs/msg/DetectedObject` with ego vehicle motion compensation and coordinate transform.
+
+### Object merger
+
+- [object_merger](https://github.com/autowarefoundation/autoware.universe/tree/main/perception/object_merger)
+
+This package can merge 2 topics of `autoware_auto_perception_msgs/msg/DetectedObject`.
+
+- [simple_object_merger](https://github.com/autowarefoundation/autoware.universe/tree/main/perception/simple_object_merger)
+
+This package can merge simply multiple topics of `autoware_auto_perception_msgs/msg/DetectedObject`.
+Different from [object_merger](https://github.com/autowarefoundation/autoware.universe/tree/main/perception/object_merger), this package doesn't use association algorithm and can merge with low calculation cost.
+
+- [topic_tools](https://github.com/ros-tooling/topic_tools)
+
+If a vehicle has 1 radar, the topic relay tool in `topic_tools` can be used.
+Example is as below.
+
+```xml
+<launch>
+  <group>
+    <push-ros-namespace namespace="radar"/>
+    <group>
+      <push-ros-namespace namespace="front_center"/>
+      <include file="$(find-pkg-share example_launch)/launch/ars408.launch.xml">
+        <arg name="interface" value="can0" />
+      </include>
+    </group>
+    <node pkg="topic_tools" exec="relay" name="radar_relay" output="log" args="front_center/detected_objects detected_objects"/>
+  </group>
+</launch>
+```
+
+## Appendix
+
+### Discussion
+
+Radar architecture design is discussed as below.
+
+- [Discussion 2531](https://github.com/orgs/autowarefoundation/discussions/2531)
+- [Discussion 2532](https://github.com/orgs/autowarefoundation/discussions/2532).
diff --git a/...gn/autoware-architecture/sensing/data-types/radar-data/radar-pointcloud-data.md b/...gn/autoware-architecture/sensing/data-types/radar-data/radar-pointcloud-data.md
@@ -0,0 +1,76 @@
+# Radar pointcloud data pre-processing design
+
+## Overview
+
+### Pipeline
+
+This diagram describes the pre-process pipeline for radar pointcloud.
+
+![radar-pointcloud-sensing](image/radar-pointcloud-sensing.drawio.svg)
+
+### Interface
+
+- Input
+  - Radar data from device
+  - Twist information of ego vehicle motion
+- Output
+  - Dynamic radar pointcloud (`ros-perception/radar_msgs/msg/RadarScan.msg`)
+  - Noise filtered radar pointcloud (`sensor_msgs/msg/Pointcloud2.msg`)
+
+### Note
+
+- In the sensing layer, the radar pre-process packages filter noise through the `ros-perception/radar_msgs/msg/RadarScan.msg` message type with sensor coordinate.
+- For use of radar pointcloud data by LiDAR packages, we would like to propose a converter for creating `sensor_msgs/msg/Pointcloud2.msg` from `ros-perception/radar_msgs/msg/RadarScan.msg`.
+
+## Reference implementations
+
+### Data message for radars
+
+Autoware uses radar objects data type as [radar_msgs/msg/RadarScan.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarScan.msg).
+In detail, please see [Data message for radars](reference-implementations/data-message.md).
+
+### Device driver for radars
+
+Autoware support `ros-perception/radar_msgs/msg/RadarScan.msg` and `autoware_auto_perception_msgs/msg/TrackedObjects.msg` for Radar drivers.
+
+In detail, please see [Device driver for radars](reference-implementations/device-driver.md).
+
+### Basic noise filter
+
+- [radar_threshold_filter](https://github.com/autowarefoundation/autoware.universe/tree/main/sensing/radar_threshold_filter)
+
+This package removes pointcloud noise with low amplitude, edge angle, too near pointcloud by threshold.
+The noise depends on the radar devices and installation location.
+
+### Filter to static/dynamic pointcloud
+
+- [radar_static_pointcloud_filter](https://github.com/autowarefoundation/autoware.universe/tree/main/sensing/radar_static_pointcloud_filter)
+
+This package extracts static/dynamic radar pointcloud by using doppler velocity and ego motion.
+The static radar pointcloud can be used for localization like NDT scan matching, and the dynamic radar pointcloud can be used for dynamic object detection.
+
+### Message converter from RadarScan to Pointcloud2
+
+- [radar_scan_to_pointcloud2](https://github.com/autowarefoundation/autoware.universe/tree/main/sensing/radar_scan_to_pointcloud2)
+
+For convenient use of radar pointcloud within existing LiDAR packages, we suggest a `radar_scan_to_pointcloud2_convertor` package for conversion from `ros-perception/radar_msgs/msg/RadarScan.msg` to `sensor_msgs/msg/Pointcloud2.msg`.
+
+|            |           LiDAR package           |                 Radar package                 |
+| :--------: | :-------------------------------: | :-------------------------------------------: |
+|  message   | `sensor_msgs/msg/Pointcloud2.msg` | `ros-perception/radar_msgs/msg/RadarScan.msg` |
+| coordinate |             (x, y, z)             |                   (r, θ, φ)                   |
+|   value    |             intensity             |          amplitude, doppler velocity          |
+
+For considered use cases,
+
+- Use [pointcloud_preprocessor](https://github.com/autowarefoundation/autoware.universe/tree/main/sensing/pointcloud_preprocessor) for radar scan.
+- Apply obstacle segmentation like [ground segmentation](https://github.com/autowarefoundation/autoware.universe/tree/main/perception/ground_segmentation) to radar points for LiDAR-less (camera + radar) systems.
+
+## Appendix
+
+### Discussion
+
+Radar architecture design is discussed as below.
+
+- [Discussion 2531](https://github.com/orgs/autowarefoundation/discussions/2531)
+- [Discussion 2532](https://github.com/orgs/autowarefoundation/discussions/2532).
diff --git a/...tecture/sensing/data-types/radar-data/reference-implementations/data-message.md b/...tecture/sensing/data-types/radar-data/reference-implementations/data-message.md
@@ -0,0 +1,87 @@
+# Data message for radars
+
+## Summary
+
+To sum up, Autoware uses radar data type as below.
+
+- [radar_msgs/msg/RadarScan.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarScan.msg) for radar pointcloud
+- [radar_msgs/msg/RadarTracks.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarTracks.msg) for radar objects.
+
+## Radar data message for pointcloud
+
+### Message definition
+
+- [ros2/msg/RadarScan.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarScan.msg)
+
+```sh
+std_msgs/Header header
+radar_msgs/RadarReturn[] returns
+```
+
+- [ros2/msg/RadarReturn.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarReturn.msg)
+
+```sh
+# All variables below are relative to the radar's frame of reference.
+# This message is not meant to be used alone but as part of a stamped or array message.
+
+float32 range                            # Distance (m) from the sensor to the detected return.
+float32 azimuth                          # Angle (in radians) in the azimuth plane between the sensor and the detected return.
+                                         #    Positive angles are anticlockwise from the sensor and negative angles clockwise from the sensor as per REP-0103.
+float32 elevation                        # Angle (in radians) in the elevation plane between the sensor and the detected return.
+                                         #    Negative angles are below the sensor. For 2D radar, this will be 0.
+float32 doppler_velocity                 # The doppler speeds (m/s) of the return.
+float32 amplitude                        # The amplitude of the of the return (dB)
+```
+
+## Radar data message for tracked objects
+
+### Message definition
+
+- [radar_msgs/msg/RadarTrack.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarTrack.msg)
+
+```sh
+# Object classifications (Additional vendor-specific classifications are permitted starting from 32000 eg. Car)
+uint16 NO_CLASSIFICATION=0
+uint16 STATIC=1
+uint16 DYNAMIC=2
+
+unique_identifier_msgs/UUID uuid            # A unique ID of the object generated by the radar.
+                                            # Note: The z component of these fields is ignored for 2D tracking.
+geometry_msgs/Point position                # x, y, z coordinates of the centroid of the object being tracked.
+geometry_msgs/Vector3 velocity              # The velocity of the object in each spatial dimension.
+geometry_msgs/Vector3 acceleration          # The acceleration of the object in each spatial dimension.
+geometry_msgs/Vector3 size                  # The object size as represented by the radar sensor eg. length, width, height OR the diameter of an ellipsoid in the x, y, z, dimensions
+                                            # and is from the sensor frame's view.
+uint16 classification                       # An optional classification of the object (see above)
+float32[6] position_covariance              # Upper-triangle covariance about the x, y, z axes
+float32[6] velocity_covariance              # Upper-triangle covariance about the x, y, z axes
+float32[6] acceleration_covariance          # Upper-triangle covariance about the x, y, z axes
+float32[6] size_covariance                  # Upper-triangle covariance about the x, y, z axes
+```
+
+### Message usage for RadarTracks
+
+- Object classifications
+
+In object classifications of [radar_msgs/msg/RadarTrack.msg](https://github.com/ros-perception/radar_msgs/blob/ros2/msg/RadarTrack.msg), additional classifications label can be used by the number starting from 32000.
+
+To express for [Autoware label definition](https://gitlab.com/autowarefoundation/autoware.auto/autoware_auto_msgs/-/blob/master/autoware_auto_perception_msgs/msg/ObjectClassification.idl), Autoware defines object classifications for `RadarTracks.msg` as below.
+
+```sh
+uint16 UNKNOWN = 32000;
+uint16 CAR = 32001;
+uint16 TRUCK = 32002;
+uint16 BUS = 32003;
+uint16 TRAILER = 32004;
+uint16 MOTORCYCLE = 32005;
+uint16 BICYCLE = 32006;
+uint16 PEDESTRIAN = 32007;
+```
+
+For detail implementation, please see [radar_tracks_msgs_converter](https://github.com/autowarefoundation/autoware.universe/tree/main/perception/radar_tracks_msgs_converter).
+
+## Note
+
+### Survey for radar message
+
+Depending on the sensor manufacturer and its purpose, each sensor might exchange raw, post-processed data. This section introduces a survey about the previously developed messaging systems in the open-source community. Although there are many kinds of outputs, radar mainly adopt two types as outputs, pointcloud and objects. Related discussion for message definition in ros-perception are [PR #1](https://github.com/ros-perception/radar_msgs/pull/1), [PR #2](https://github.com/ros-perception/radar_msgs/pull/2), and [PR #3](https://github.com/ros-perception/radar_msgs/pull/3). Existing open source softwares for radar are summarized in these PR.
diff --git a/...ecture/sensing/data-types/radar-data/reference-implementations/device-driver.md b/...ecture/sensing/data-types/radar-data/reference-implementations/device-driver.md
@@ -0,0 +1,33 @@
+# Device driver for radars
+
+## Interface for radar devices
+
+The radar driver converts the communication data from radars to ROS 2 topics.
+The following communication types are considered:
+
+- CAN
+- CAN-FD
+- Ethernet
+
+![draw.io figure](image/radar-communication.drawio.svg)
+
+## Software Interface
+
+Autoware support `ros-perception/radar_msgs/msg/RadarScan.msg` and `autoware_auto_perception_msgs/msg/TrackedObjects.msg` for Radar drivers.
+
+![draw.io figure](image/radar-driver.drawio.svg)
+
+ROS driver receive the data from radar devices.
+
+- Scan (pointcloud)
+- Tracked objects
+- Diagnostics results
+
+The ROS drivers receives the following data from the radar device.
+
+- Time synchronization
+- Ego vehicle status, which often need for tracked objects calculation
+
+## Radar driver example
+
+- [ARS408 driver](https://github.com/tier4/ars408_driver)