feat(carla_autoware): add interface to easily use CARLA with Autoware

[mraditya01]

Description

The interface between CARLA and Autoware aims to bridge both systems.
Required this launch PR: autowarefoundation/autoware_launch#924

Related links

• Main Repositories with instruction (https://github.com/mraditya01/autoware.universe/tree/updated_version/simulator/carla_autoware)
• CARLA Lanelet2 Maps (https://bitbucket.org/carla-simulator/autoware-contents/src/master/maps/)
• Update to e2e launch files (https://github.com/mraditya01/autoware_launch/tree/updated_version)
• Issues related to CARLA Autoware Bridge

• Autoware sometimes drives over the velocity limit. #6487
• controller_node_exe makes an error and the vehicle stops forever #5701

Tests performed

Tested on Town01 map from Link.

• Environment Setup:
  CARLA 0.9.15
  Autoware.universe Main Branch
  ROS2 Humble
  Python 3.10
  Ubuntu 22.04

• Testing:

1. Tested the interface on the Town01 map provided by the CARLA simulator.
2. Verified the accuracy of vehicle localization within the simulated environment.
3. Checked that the vehicle's estimated position matches the ground truth position provided by CARLA.
4. Verified that the localization module in Autoware correctly interprets data received from CARLA.
5. Tested the perception module's ability to detect obstacle within the scene.
6. Tested vehicle control under various scenarios, including lane following, obstacle stop, and intersection.

• Issues Related

1. Tested velocity limit issue:

Confirmed the functionality of limiting vehicle velocity to 30 km/h by providing input via Autoware.
Verified that the vehicle's velocity did not exceed the specified limit during the test.

2. Tested vehicle stopped issue on particular path:

Replicated the reported issue by recreating the same path on the same map used in the initial observation.
Monitored the vehicle's behavior and trajectory during the test.
Confirmed that no issues occurred, indicating that the reported problem was not reproducible or no longer exists after further testing and investigation.

Video Working Example:
Video

Notes for reviewers

Interface changes

Effects on system behavior

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

• I've confirmed the contribution guidelines.
• The PR follows the pull request guidelines.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

• The PR follows the pull request guidelines.
• The PR has been properly tested.
• The PR has been reviewed by the code owners.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

• There are no open discussions or they are tracked via tickets.
• The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

[maxime-clem]

I am still testing this PR but here are some comments.
Mostly I think the instructions in the README need to be improved.

[maxime-clem]

@Kim-mins can I ask you to test this PR ? I think you are using CARLA a lot so I am very interested in your feedback.

[Kim-mins]

@maxime-clem Absolutely!

I read the instruction of the bridge, and I got some questions.

Q1. Spawning the ego vehicle

I could build and run the latest Autoware with the modified autoware_launch and autoware universe, but I cannot know how to spawn the ego vehicle with the bridge. (seems it works differently what I've used).

First I ran Carla 0.9.15, and it was loaded with Town10. I also ran the latest Autoware with Town01(not 10), but nothing happens.
There seem some logics for changing world and spawning the ego vehicle, but I cannot how to use it.

Q2. Building the bridge

I also tried to build the bridge with the following commands,

cd colcon_ws
colcon build --symlink-install

but the log is as below, which seems the build was unsuccessful:

Starting >>> carla_autoware
--- stderr: carla_autoware                   
/usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
package init file 'src/carla_autoware/__init__.py' not found (or not a regular file)
---
Finished <<< carla_autoware [1.48s]

Actually, I could not know where to run the command. I didn't know where the colcon_ws is.

Thank you for all your hard work! I've been really looking forward to it.

[Kim-mins]

It is not relevant to the testing, but I have one more question regarding the frequency.
According to the implementation, it seems the maximum frequencies are annotated here.
I think, when I get to use this bridge, I'll run the co-simulation synchronously with at most 20 fps(tick for every 0.05 second), following the example above.
Wouldn't it result in performance issue?

[Kim-mins]

And this could be minor, README.md's content Supported Environment might be changed to the Environment Setup of this PR (I found it also supports the latest version after building and running 2024.04 🥲)

[mraditya01]

@Kim-mins Thank you very much for the Feedback! We have updated the README file to provide more detailed explanation regarding the code and parameter used.
I will also answer some of the questions,

Q1. Spawning the Ego vehicle

Ego vehicle should have been spawned automatically at random location by default as you run the bridge. You can change the location to any specific coordinates (x, y, z) and orientation (roll, pitch, yaw) by setting the spawn_point parameter in the carla_autoware.launch.xml file.
Same as the world map for CARLA, it should load automatically based on carla_map parameter on carla_autoware.launch.xml, with the default value set to Town01. If the map is not loaded on CARLA, there may be some error during the bridge initialization, such as timeout.

Q2. Building the bridge

I apologize for the confusion. I neglected to update the instructions for building the bridge individually, which still referenced the old cd colcon_ws command, which no longer exists. The README has now been updated with the correct instructions.

Q3. Frequency Annotation

By default, our bridge operates in synchronous mode with a 0.05-second timestep. The performance is not negatively impacted because the annotated frequency only affects the publishing rate of ROS messages, not the sensor tick rate within CARLA itself. In fact, reducing the publishing frequency can improve simulation time performance, as fewer computationally intensive functions during CARLA -> Autoware data conversion, such as create_cloud, are called.

Additional Question

And this could be minor, README.md's content Supported Environment might be changed to the Environment Setup of this PR (I found it also supports the latest version after building and running 2024.04 🥲)

Thank you for testing the latest version! I'll update the supported versions once the 2024.05 version is released so we have more stable version. Small updates can occasionally disrupt the bridge. (Happens before when Autoware required the traffic_signal message as input for initialization and the bridge suddenly not working 😓).

[Kim-mins]

Thank you for the response and modifications @mraditya01!

Actually, I'm not familiar to ros2. Could you please provide me more details?

Here are the commands I executed:

# Build bridge
colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=Release
# Run Carla
./CarlaUE4.sh -prefernvidia -quality-level=Low -RenderOffScreen
# Run Autoware
ros2 launch autoware_launch e2e_simulator.launch.xml map_path:=$HOME/autoware_map/Town01 vehicle_model:=sample_vehicle sensor_model:=awsim_sensor_kit simulator_type:=carla carla_map:=Town01

and I got RViz with no pose initialization, and Carla with Town10.

At this stage, I guess I should run carla_autoware.py, but I could not run it.
I ran the command python3 -m src.carla_autoware.carla_autoware at the path autoware.universe/simulator/carla_autoware, but I got the error below:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/workspace/src/universe/autoware.universe/simulator/carla_autoware/src/carla_autoware/carla_autoware.py", line 21, in <module>
    from .carla_ros import carla_interface
  File "/workspace/src/universe/autoware.universe/simulator/carla_autoware/src/carla_autoware/carla_ros.py", line 20, in <module>
    from autoware_auto_control_msgs.msg import AckermannControlCommand
ModuleNotFoundError: No module named 'autoware_auto_control_msgs'

It seems I should install dependencies, but I have no idea. Could you please help me?

Thank you!

[maxime-clem]

@Kim-mins

• Did you also checkout the autoware_launch PR ?
• Did you setup the "Python Package for CARLA 0.9.15 Ros2 Humble communication" ?

If yes to the above, then can you share your logfile ? I want to make sure there is no issue with the carla_autoware node.
(If you want we can discuss your issues on Discord. It will be easier to help you that way)

[Kim-mins]

Thank you for the response @maxime-clem.
I think I can communicate on discord. May I message to you?

Here are the answers to your questions:

• Did you also checkout the autoware_launch PR ?
  Yes I changed the autoware.repos in autoware repository as below:

  ...
    universe/autoware.universe:
      type: git
      url: https://github.com/mraditya01/autoware.universe.git
      version: 1f726f1c8bd64b014f8e2c2bb9f74928843918d4
  ...
    launcher/autoware_launch:
      type: git
      url: https://github.com/mraditya01/autoware_launch.git
      version: def3a97910ef57f689a5f564426e7fe860b23efd
  ...
  

  and I followed the source installation procedure.

• Did you setup the "Python Package for CARLA 0.9.15 Ros2 Humble communication" ?
  Yes I installed by .whl file here.

For the log file, I checked that Autoware runs well, but I cannot run the bridge now. To run the bridge, in my understanding, I should run carla_autoware.py in autoware/src/universe/autoware.universe/simulator/carla_autoware/src/carla_autoware and I executed the command python3 -m src.carla_autoware.carla_autoware, but I got the error below:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/workspace/src/universe/autoware.universe/simulator/carla_autoware/src/carla_autoware/carla_autoware.py", line 20, in <module>
    from .carla_ros import carla_interface
  File "/workspace/src/universe/autoware.universe/simulator/carla_autoware/src/carla_autoware/carla_ros.py", line 20, in <module>
    from autoware_auto_control_msgs.msg import AckermannControlCommand
ModuleNotFoundError: No module named 'autoware_auto_control_msgs'

Thank you!

[Kim-mins]

@maxime-clem

I did not noticed your mention on Discord. I think we can continue the discussion there.

Thanks!

[github-actions]

Documentation URL: https://autowarefoundation.github.io/autoware.universe/pr-6859/
Modified URLs:

• https://autowarefoundation.github.io/autoware.universe/pr-6859/simulator/autoware_carla_interface/

[github-actions]

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

• You've checked our contribution guidelines.
• Your PR follows our pull request guidelines.
• All required CI checks pass before marking the PR ready for review.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.

Project coverage is 0.00%. Comparing base (3d849e9) to head (11d352c).
Report is 797 commits behind head on main.

Files	Patch %	Lines
simulator/autoware_carla_interface/setup.py	0.00%	6 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #6859       +/-   ##
==========================================
- Coverage   15.09%   0.00%   -15.10%     
==========================================
  Files        1967       1     -1966     
  Lines      135941       6   -135935     
  Branches    42122       0    -42122     
==========================================
- Hits        20520       0    -20520     
+ Misses      92700       6    -92694     
+ Partials    22721       0    -22721     

Flag	Coverage Δ
differential	0.00% <0.00%> (?)
total	?

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.