hardware_interface does not work well when Isaac Sim is running

[hijimasa]

Describe the bug
The command that is sent from controller is not sometimes reflected to hardware when Isaac Sim is running.

To Reproduce
Steps to reproduce the behavior:

1. launch Isaac Sim
2. launch ros2_control_demo_example_3 rrbot_system_multi_interface.launch.py
3. send topic by following command

   ros2 topic pub -1 /forward_velocity_controller/commands std_msgs/msg/Float64MultiArray "{layout: {dim: [], data_offset: 0}, data: [7.0, 7.0] }"
   or
   ros2 topic pub -1 /forward_velocity_controller/commands std_msgs/msg/Float64MultiArray "{layout: {dim: [], data_offset: 0}, data: [0.0, 0.0] }"
   

4. Sometimes the command that is sent from controller is not reflected to hardware

Expected behavior
All command that is sent from controller is reflected to hardware.

Screenshots
Screencast from 2024年08月24日 01時45分17秒.webm

Environment (please complete the following information):

• OS: Ubuntu 22.04
• ROS 2 Version: Humble
• ros2_control Version: 2.42.0
• Isaac Sim Version: 4.1

Additional context
The topic pub/sub works fine, so I assume it is a hardware_interface specific issue.
I have asked on Isaac Sim's forum, but I have also make this issue here, where there are contributors who are familiar with the hardware_interface implementation.

[saikishor]

Hello @hijimasa!

Can you check this issue: ros-controls/ros2_controllers#537 and see if it what you are facing?

Thank you!

[hijimasa]

Hello @saikishor !

I already read it. And use_sim_time option is removed.
I think this is another issue.

[saikishor]

Hello @saikishor !

I already read it. And use_sim_time option is removed.
I think this is another issue.

Hello @hijimasa!

The example 3 itself is working on its own. It's probably the environment you are using with Issac. Could you please recheck this?

Thank you!

[hijimasa]

Hello @saikishor !

I know the example is working its own. I don't know how hardware_interface work.
So I cannot understand where I should recheck in Isaac Sim.

Any ideas for a situation where the topic pub/sub works and hardware_interface stops working in the first place?

[hijimasa]

I changed ros2_controllers/forward_command_controller as below.
forward_command_controller.zip

This change shows the command is lost between RealtimeBuffer read/write.
Screencast from 2024年08月25日 11時51分12秒.webm

I will try to check RealtimeBuffer.

[hijimasa]

I changed ros2_controllers/forward_command_controller as below.
forward_command_controller.zip

This change shows the command is lost between topic pub/sub, not RealtimeBuffer read/write.
test.webm

I will try to check the controller QoS.

[hijimasa]

This is topic pub/sub problem not ros2_control problem.
So I close this issue.

Thank you.

[saikishor]

Hello @hijimasa!

Could you please highlight the main change that you did?. It would be easier for future references. Did you find the problem already in QoS configuration?

[hijimasa]

Hello @saikishor !
Thank you for your reply.

I changed two points in src/forward_contollers_base.cpp.

1. on_configure() : I add RCLCPP_INFO() in subscriber callback function to check subscription.
2. update() : I add RCLCPP_INFO() to check RealtimeBuffer update.

I could not find the problem in QoS configuration but I think this is topic pub/sub problem not ros2_control problem.

[saikishor]

It is indeed weird to have caused due to publisher and subscriber. If you get some lead on it. Let us know. It would help other users in future

[hijimasa]

I don't know why Fast DDS doesn't work well, but Cyclone DDS solved the problem.
Isaac Sim and Fast DDS may be incompatible.

[christophfroehlich]

One more reason to make cyclone the default RMW 🙈

[saikishor]

One more reason to make cyclone the default RMW 🙈

Agreed! DDS stuff is making the debugging much harder