Unable to set the initial pose of the vehicle

[Hjqhhhhhc]

The issue I encountered during the quick start demo is that the initial pose doesn't automatically match in the Autoware and AWSim joint simulation. Even manually setting the initial pose doesn't work. What could be the problem?

[mozhoku]

• If it's your first time launching Autoware with new build these packages needs some time to build in the first launch of the autoware: https://github.com/autowarefoundation/autoware/tree/main/ansible/roles/artifacts It might be related to that.

• There is a message type migration in the Autoware at the moment Migrate to autoware_msgs and release #88 If message types changed on the Autoware side, then it won't be able to communicate with the simulator until next AWSIM release.

• If you've used TierIV AWSIM before, AWSIM Labs has different vehicle sensor kit configuration when launching. Please make sure you follow this guide for launching AWSIM Labs.

Other than that, I've no idea about what might be causing this issue. Please make sure that you've followed the steps in the documentation. If you've encountered any errors or missing steps while following the Quick Start Demo, please let us know. 👍

[Hjqhhhhhc]

This is my first time using Autoware, and the build process using Ansible went smoothly. However, during the colcon build compilation of Autoware, I encountered some warnings (I'm not sure if they affect the build). I haven't used Tier IV's AWSIM before. The second solution mentions migrating autoware_msgs messages, which I don't quite understand. Could you please explain it in more detail? Thank you.

[KTKTKT0401]

I've also encountered the same issue(unable to set initial pose).
I tried both AWSIM and AWSIM Lab but both have the same problem.
I was able to use planning simulation and rosbag simulation in the autoware environment I've built(main version, git cloned June24th 2024).
To solve this issue, do I have to wait for the AWSIM new releases?

P.S. This is my first time installing Autoware and AWSIM.

[KTKTKT0401]

I figured out that my Autoware's msg type of '/vehicle/status/velocity_status' has changed from ''autoware_auto_vehicle_msgs/msg/VelocityReport" to "autoware_vehicle_msgs/msg/VeocityReport".
This causes the '/vehicle/status/velocity_status' connection error between AWSIM and Autoware. I guess this is the reason why localization failures.

Now I'm considering to make changes below.

• Change the ros2 topic publisher regarding velocity_status via AWSIM.

[mozhoku]

@KTKTKT0401
We are currently updating message types in the AWSIM Labs: #91

Hopefully we will release the updated version soon. Sorry for the inconvenience with the current release 🙏

[KTKTKT0401]

Thank you for your urgent reply!
I'm looking forward for the latest release of AWSIM.

[mozhoku]

@KTKTKT0401 @Hjqhhhhhc Hi!
We've released new version here: https://github.com/autowarefoundation/AWSIM-Labs/releases/tag/v1.1.0

Please let us know if your issues still persists with the new release.

[KTKTKT0401]

@mozhoku
Hi!
I've checked the new version(v1.1.0) of awsim lab and the digital twin simulation worked in my environment!
Thank you for fixing the bug!

[Hjqhhhhhc]

@mozhoku 嗨！ 我检查了 awsim lab 的新版本（v1.1.0），数字孪生模拟在我的环境中运行正常！ 感谢您修复了错误！

@KTKTKT0401 @Hjqhhhhhc嗨！ 我们在这里发布了新版本：https://github.com/autowarefoundation/AWSIM-Labs/releases/tag/v1.1.0

如果您在新版本中仍然存在问题，请告知我们。

thank you. After the update, the problem was resolved.

[Hjqhhhhhc]

我发现我的 Autoware 的 '/vehicle/status/velocity_status' 消息类型已从 ''autoware_auto_vehicle_msgs/msg/VelocityReport' 更改为 "autoware_vehicle_msgs/msg/VeocityReport"。 这导致 AWSIM 和 Autoware 之间的 '/vehicle/status/velocity_status' 连接错误。我猜这就是本地化失败的原因。

现在我正在考虑做出以下更改。

• 通过AWSIM更改有关velocity_status 的ros2主题发布者。

Hello, I would like to ask how you discovered changes in Autoware message types. This is very important to me, thank you!

[Hjqhhhhhc]

我发现我的 Autoware 的 '/vehicle/status/velocity_status' 消息类型已从 ''autoware_auto_vehicle_msgs/msg/VelocityReport' 更改为 "autoware_vehicle_msgs/msg/VeocityReport"。 这导致 AWSIM 和 Autoware 之间的 '/vehicle/status/velocity_status' 连接错误。我猜这就是本地化失败的原因。

现在我正在考虑做出以下更改。

• 通过AWSIM更改有关velocity_status 的ros2主题发布者。

After updating AWSIM, the initial pose of the car displays correctly, but it cannot autonomously navigate after setting a goal. There is an error in the terminal: [service_log_checker_node-4] [ERROR] [1720076453.268919025] [system.service_log_checker]: /api/operation_mode/change_to_autonomous: status code 1 'The mode change is blocked by the system.' (/default_ad_api/node/operation_mode). Additionally, in RViz, there is an error: Multiple Markers in the same MarkerArray message had the same ns and id: (traffic_light,70127). Have you encountered this?

[mozhoku]

我发现我的 Autoware 的 '/vehicle/status/velocity_status' 消息类型已从 ''autoware_auto_vehicle_msgs/msg/VelocityReport' 更改为 "autoware_vehicle_msgs/msg/VeocityReport"。 这导致 AWSIM 和 Autoware 之间的 '/vehicle/status/velocity_status' 连接错误。我猜这就是本地化失败的原因。
现在我正在考虑做出以下更改。

• 通过AWSIM更改有关velocity_status 的ros2主题发布者。

Hello, I would like to ask how you discovered changes in Autoware message types. This is very important to me, thank you!

You can see the changes here:

• https://github.com/orgs/autowarefoundation/discussions/4803#discussioncomment-9645234

我发现我的 Autoware 的 '/vehicle/status/velocity_status' 消息类型已从 ''autoware_auto_vehicle_msgs/msg/VelocityReport' 更改为 "autoware_vehicle_msgs/msg/VeocityReport"。 这导致 AWSIM 和 Autoware 之间的 '/vehicle/status/velocity_status' 连接错误。我猜这就是本地化失败的原因。
现在我正在考虑做出以下更改。

• 通过AWSIM更改有关velocity_status 的ros2主题发布者。

After updating AWSIM, the initial pose of the car displays correctly, but it cannot autonomously navigate after setting a goal. There is an error in the terminal: [service_log_checker_node-4] [ERROR] [1720076453.268919025] [system.service_log_checker]: /api/operation_mode/change_to_autonomous: status code 1 'The mode change is blocked by the system.' (/default_ad_api/node/operation_mode). Additionally, in RViz, there is an error: Multiple Markers in the same MarkerArray message had the same ns and id: (traffic_light,70127). Have you encountered this?

@Hjqhhhhhc
I'm not familiar with that issue.
If you are using a different environment for the simulator the "same traffic_light id" could be caused by your lanelet2 file annotation but I'm not really sure 😓

[KTKTKT0401]

I haven't tried AWSIM, but in my AWSIM_lab environment, I succeeded my planning and control until the vehicle reaches the goal pose.

By the way, how I figured the msg type is below.(I leave this answer, for the aim of leaving my personal memo of how I found the issue as well as answering the question above.)
1)I launched both AWSIM_lab and Autoware Universe, as guided in the AWSIM_lab document.

2)I used rqt and listed up the topic used in the '/localization/pose_estimator/ndt_scan_matcher'. I looked for this ros2 node, because the localization error was occuring.
In this step, I figured this node subscribes the '/localization/pose_twist_fusion_filter/biased_pose_with_covariance'.

3)I looked up for a source of the ros2 topic '/localization/pose_twist_fusion_filter/biased_pose_with_covariance' using rqt, and I figured the two topics below are the two ROS2 topics, published from AWSIM_lab.
1.'/sensing/gnss/pose_with_covariance'
2.'/vehicle/status/velocity_status'

4)I used the command ros2 topic info and figured out when I tried to ros2 topic info the '/vehicle/status/velocity_status' , the error occured.
The error said there are two types of topic msg being published/subscribed as the same topic name.

5)I launched Autoware Planning simulation and checked the msg type of '/vehicle/status/velocity_status' , the msg type was ''autoware_vehicle_msgs/msg/VelocityReport".

6)In the unity project, I found the file "VehicleReportRos2Publisher.cs".
Inside that .cs file, the '/vehicle/status/velocity_status' was defined as "autoware_auto_vehicle_msgs/msg/VeocityReport".

[Hjqhhhhhc]

我还没有尝试过 AWSIM，但在我的 AWSIM_lab 环境中，我成功进行了规划和控制，直到车辆达到目标姿势。

顺便说一下，我如何确定 msg 类型如下。（我留下这个答案，是为了留下我个人的备忘录，说明我如何发现这个问题以及回答上面的问题。） 1）我启动了 AWSIM_lab 和 Autoware Universe，按照 AWSIM_lab 文档中的指导。

2)我使用 rviz 并列出了 '/localization/pose_estimator/ndt_scan_matcher' 中使用的主题。我寻找这个 ros2 节点，因为发生了定位错误。 在此步骤中，我发现该节点订阅了 '/localization/pose_twist_fusion_filter/biased_pose_with_covariance'。

3)我使用 rviz 查找了 ros2 主题 '/localization/pose_twist_fusion_filter/biased_pose_with_covariance' 的来源，我发现下面的两个主题是从 AWSIM_lab 发布的两个 ROS2 主题。1 .'/sensing/gnss/pose_with_covariance' 2.'/vehicle/status/velocity_status'

4)我使用了命令 ros2 topic info，并发现当我尝试使用 ros2 topic info '/vehicle/status/velocity_status' 时，发生了错误。 错误表明有两种类型的主题消息以相同的主题名称发布/订阅。

5)我启动了 Autoware Planning 模拟并检查了 '/vehicle/status/velocity_status' 的消息类型，消息类型为 ''autoware_auto_vehicle_msgs/msg/VelocityReport"。

6)在 Unity 项目中，我找到了文件“VehicleReportRos2Publisher.cs”。 在该 .cs 文件中，'/vehicle/status/velocity_status' 被定义为“autoware_vehicle_msgs/msg/VeocityReport”。

I am also using the Awsim_labs and following the steps in the documentation，But it didn't succeed.https://autowarefoundation.github.io/AWSIM-Labs/main/GettingStarted/QuickStartDemo/

[xmfcx]

@Hjqhhhhhc Hi, I would like to help you and make sure AWSIM Labs and latest version of Autoware works well with your setup.

First, we need to make sure everything is set up correctly on the Autoware side.

For this I need basic system information from you.

System info

General

Run:

sudo apt update && sudo apt install -y screenfetch
screenfetch -n

And paste the output here?

CUDA version

Run nvcc -V and paste the results here.

TensorRT and cuDNN version

Run both

dpkg -l | grep -i tensor
dpkg -l | grep -i cudnn

and paste the results here.

Loopback multicast state

Run ip link show lo and report the status here.

Multicast state

If you got:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

then, your multicast is not enabled.

Run sudo ip link set lo multicast on to enable it. (this should be run every time your computer restarts)

You can follow this tutorial to add this as a service so it will run on start automatically.

Correct output:

mfc@mfc-leo:~$ ip link show lo
1: lo: <LOOPBACK,MULTICAST,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

RMW checkup

Run:

echo $RMW_IMPLEMENTATION

Correct output: rmw_cyclonedds_cpp

Cyclonedds checkup

Run: cat $CYCLONEDDS_URI

Correct output:

<?xml version="1.0" encoding="UTF-8" ?>
<CycloneDDS xmlns="https://cdds.io/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://cdds.io/config https://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/master/etc/cyclonedds.xsd">
    <Domain Id="any">
        <General>
            <Interfaces>
                <NetworkInterface name="lo" priority="default" multicast="default" />
            </Interfaces>
            <AllowMulticast>default</AllowMulticast>
            <MaxMessageSize>65500B</MaxMessageSize>
        </General>
        <Internal>
            <SocketReceiveBufferSize min="10MB"/>
            <Watermarks>
                <WhcHigh>500kB</WhcHigh>
            </Watermarks>
        </Internal>
    </Domain>
</CycloneDDS>

If not, let's fix it following these:

• https://autowarefoundation.github.io/autoware-documentation/main/support/troubleshooting/performance-troubleshooting/#solution_3
• https://autowarefoundation.github.io/autoware-documentation/main/support/troubleshooting/performance-troubleshooting/#solution_4

Rebuild Autoware

First, navigate to the autoware workspace.

Then, call rm -rf src build install log to remove these folders. (Normally we wouldn't remove src folder but I don't know how old your repo is, it's best to do it)

# Fetch all updates
git fetch

# Discard all changes to tracked files
git checkout .

# Remove all untracked files 
git clean -fd

# Checkout to latest main
git reset --hard origin/main

mkdir src
vcs import src < autoware.repos

Make sure system is up to date:

sudo apt update
sudo apt dist-upgrade

You should have an Nvidia GPU on your machine to run AWSIM Labs demo since it requires lidar and camera neural networks.

Make sure you have the correct CUDA version (12.3).

Source humble:

source /opt/ros/humble/setup.bash

Install the ROS dependencies:

rosdep update
rosdep install -y --from-paths src --ignore-src --rosdistro humble

Build Autoware

colcon build --symlink-install  --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=1

Make sure you have the latest model artifacts

Remove the ~/autoware_data folder.

Follow these to download them:

• https://github.com/autowarefoundation/autoware/blob/main/ansible/README.md#ansible-collection---autowaredev_env
• https://github.com/autowarefoundation/autoware/tree/main/ansible/roles/artifacts#install-ansible-collections

Run AWSIM Labs and Autoware again

Follow: https://autowarefoundation.github.io/AWSIM-Labs/main/GettingStarted/QuickStartDemo/

[sasa42]

Hi. I have the similar problems with the inital pose.

The debug info you can see on the left.

I also followed the Quickstart and your rebuild tips.

I also tried to set the 2D Pose Est manually and hit Initialize with GNSS.

The EgoCar in awsim labs is jumping to the pos i set.

[xmfcx]

@sasa42 I'm very sorry for the inconvenience 🙇.

I think your set up is now ready, all the versions check out!

And you can place the car through rviz, this means communication is also working well.

But your problem is related to this issue:

• AWSIM Labs not working after latest Autoware update #114
• feat: migrating pointcloud types autoware.universe#6996

This feature was merged to Autoware just yesterday so right now AWSIM Labs or anything else, doesn't work with the latest Autoware.

Until the issue is resolved, I've created this autoware/awsim-labs-stable branch where I've fixed the repo versions to 3 days ago (2024-07-07).

To test the AWSIM Labs with Autoware for now, here are the steps:

# cd into autoware folder

git fetch --all

git checkout origin/awsim-labs-stable
git switch awsim-labs-stable

rm -rf src build install log
mkdir src
vcs import src < autoware.repos

source /opt/ros/humble/setup.bash

rosdep update
rosdep install -y --from-paths src --ignore-src --rosdistro humble

colcon build --symlink-install  --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=1

Now the AWSIM Labs should work with this version of the Autoware.

We will also fix the issue as soon as possible and update you.

[xmfcx]

Also don't reopen the sensors on 1.1.1

• Toggling lidar sensors causes fps drop #104 we had this issue on it (it's resolved).

Tomorrow we'll do a 1.1.2 release to cover that bug, it's fixed on the main branch of AWSIM Labs.

[sasa42]

Thanks for the detailed help.
Was quite a journey to get it to this point.

1. install WSL2 with Ubuntu

2. Install WSLg

3. set up Hardware-accelerated vulkan in WSL2 ·
   sudo add-apt-repository ppa:kisak/kisak-mesa
   sudo apt update
   sudo apt upgrade

4. AWSIM Labs Quickstart setup

would be the fast track for a WSL2 setup.

Is there some benchmark data like fps from awsim labs unity scene and rviz available for this awsim labs quickstart drive?

[xmfcx]

Hmm, I don't have experience with WSL2 but it may cause performance drop.

Normally with your machine, everything should be very fast on a native Ubuntu installation. (I generally prefer dual boot)

But I'm curious about the AWSIM Labs and Autoware performance on WSL2 too, excited to see your results!

Edit:

I see you have very little VRAM allocated, normally it should use at least 2->4GB VRAM for AWSIM Labs.

• fix: memory usage #86 This was also merged after 1.1.1 was released. But you can see the performance for it. (it's a desktop rtx4080)

[xmfcx]

Is there some benchmark data like fps from awsim labs unity scene and rviz available for this awsim labs quickstart drive?

We can share it tomorrow. But for your machine, on a native ubuntu installation, you should get 50+FPS on AWSIM Labs and no lag on the Autoware side (both running on the same machine).

I don't know about WSL2 though.

[sasa42]

I did a rebuild with...

with

cd into autoware folder

git fetch --all

git checkout origin/awsim-labs-stable
git switch awsim-labs-stable

rm -rf src build install log
mkdir src
vcs import src < autoware.repos

source /opt/ros/humble/setup.bash

rosdep update
rosdep install -y --from-paths src --ignore-src --rosdistro humble

colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=1

but there is no initial pose given.

[xmfcx]

For some reason you cannot see the camera image as well and also your simulation is still very slow. (because of WSL2)

Could you share the autoware logs in a https://gist.github.com/ ?

Also could you turn off both AWSIM and Autoware and just open AWSIM Labs?

Then check for output of these commands:

mfc@mfc-leo:~$ ros2 topic list
/awsim/ground_truth/localization/kinematic_state
/awsim/ground_truth/vehicle/pose
/clock
/control/command/control_cmd
/control/command/emergency_cmd
/control/command/gear_cmd
/control/command/hazard_lights_cmd
/control/command/turn_indicators_cmd
/initialpose
/parameter_events
/perception/traffic_light_recognition/external/traffic_signals
/rosout
/sensing/camera/traffic_light/camera_info
/sensing/camera/traffic_light/image_raw
/sensing/gnss/pose
/sensing/gnss/pose_with_covariance
/sensing/imu/tamagawa/imu_raw
/sensing/lidar/left/pointcloud_raw
/sensing/lidar/left/pointcloud_raw_ex
/sensing/lidar/right/pointcloud_raw
/sensing/lidar/right/pointcloud_raw_ex
/sensing/lidar/top/pointcloud_raw
/sensing/lidar/top/pointcloud_raw_ex
/vehicle/status/control_mode
/vehicle/status/gear_status
/vehicle/status/hazard_lights_status
/vehicle/status/steering_status
/vehicle/status/turn_indicators_status
/vehicle/status/velocity_status

mfc@mfc-leo:~$ ros2 topic hz /sensing/camera/traffic_light/image_raw
average rate: 10.033
	min: 0.091s max: 0.117s std dev: 0.00849s window: 12
average rate: 10.042
	min: 0.091s max: 0.117s std dev: 0.00789s window: 23
average rate: 10.054
	min: 0.091s max: 0.117s std dev: 0.00769s window: 34
average rate: 10.019
	min: 0.091s max: 0.117s std dev: 0.00763s window: 45

mfc@mfc-leo:~$ ros2 topic hz /sensing/lidar/top/pointcloud_raw_ex
average rate: 10.054
	min: 0.093s max: 0.117s std dev: 0.00722s window: 11
average rate: 9.982
	min: 0.093s max: 0.117s std dev: 0.00746s window: 22
average rate: 10.031
	min: 0.093s max: 0.117s std dev: 0.00621s window: 33

Also if they are low, try pulling the time scale slider to 0.2 to see how they change.

[sasa42]

... some lidar are missing....
and the pointcloud_raw_ex

[sasa42]

....also after i enabled the pointcloud_raw in AWSIM Labs UI...

do i have to activate rhese 3 checkboxes?
They were disabled after startup.

[sasa42]

[xmfcx]

something is wrong with your screenshot link above.

I've just realized, your lidars are not visible at all on your setup.

Normally it should open like this, I did not click anything in UI:

Also I've learnt Graphy was showing the RAM usage and 1.1.1 is optimized for it and those numbers are normal.

But you don't have lidar simulation at all. I don't know why this happens (apart from WSL2)

[xmfcx]

Let's keep discussing this in:

• Lidar simulation is not working in WSL2 #115

[mozhoku]

Is this issue resolved @xmfcx ?

Can we close this one if you are continuing in the separate issue?