Standardization of simulation interfaces

[adamdbrw]

Problem

The number of simulator choices for ROS increased over time, and these simulators offer different advantages and trade-offs. For certain applications Gazebo is a match, for others, Open 3D Engine (O3DE) does the job the best, and for others yet, Isaac Sim might be a fit. There are many more yet.

Open 3D Engine robotic simulation (the ROS 2 Gem) depends on gazebo_msgs package, which was deprecated with the Jazzy release, but also, as an exception, released to make life easier for O3DE users. The gazebo_msgs package interfaces are quite generic and thus were reused in O3DE. However, one simulator interface user experience should not depend on a release cycle of another simulator, but rather be tied to a certain standard.

There is currently no standard for simulation interfaces. Simulation interfaces include spawning and de-spawning of robots and other entities, moving them around, pausing and resetting the scene, and so on. They are also highly important for automation of simulation-based testing. These interfaces are mostly services as they offer valuable response interaction which is needed for almost all the interface features.

Proposed solution

A new simulation_interfaces package which reuses much of gazebo_msgs but also introduces new interfaces and changes some fields to fully support current Gazebo and O3DE community needs. The priority for the first release should be minimizing impact (including semantic differences) for the users, and capturing new requirements in new services rather than re-imagining the entire package.

Current state of O3DE interfaces in use

I come with the O3DE perspective and invite others to supply other perspectives. Note that current use for some of services and their fields is not adhering to their intended semantics, and includes extension Gems beyond the core ROS 2 Gem.

Services for simulation entities spawning and control (all are gazebo_msgs/srv/ services):

Service name	Service Type	Description
get_available_spawnables	GetWorldProperties	Return a list of spawnable names (e.g. turtlebot4).
get_spawn_points	GetWorldProperties	List named (pre-set) spawn points to use, e.g., “kitchen.”
get_spawn_point_info	GetModelState	Get a pre-set spawn point by name, returning its pose and description.
spawn_entity	SpawnEntity	Spawning robots and other entities by prefab name.
delete_entity	DeleteEntity	Despawn by unique name (e.g. turtlebot4_1).
get_scene_objects	GetModelList	Acquire dynamic objects which can be controlled (other than the robot).
get_object_state	GetEntityState	Acquire information about a specific simulation object.
set_object_state	SetEntityState	Instantly change the pose or Twist of an object.

Services for simulation scenes and control, especially important for automation with scenario testing (multiple runs of simulation on different scenes). Note that some services are not used semantically as intended, and some services are custom here indicating there is no great fit in current gazebo_msgs interface set:

Service name	Service Type	Description
load_level	custom	Load a named level, if another was loaded before, it is unloaded first.
get_current_level	custom	Get the current level of the simulation. Fails if no level is running.
get_available_levels	GetModelList	Get the available levels of the simulation.
reset_current_level	std_msgs/srv/Trigger	Reset the current level of the simulation.
unload_current_level	std_msgs/srv/Trigger	Unload the current level of the simulation.
set_simulation_paused	std_srvs/srv/SetBool	Pausing and un-pausing.

The benefits

Standardization would improve user experience when using their validation, testing and ML pipelines with several simulators, or when switching between one simulator and the other as they needs come to better match another platform. It would also make it easier to benchmark simulators.

Discussion

• What would be the minimal simulation_interfaces package that would have the lowest adaptation cost for Gazebo?
• Going from there, how can we best cover O3DE utility with reasonable addition of optional fields and services to such package?
• As a community member, could you provide what is different in perspective of your simulator?

Please take a look, @azeey.

[ros-discourse]

This issue has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/tools-to-use-other-simulators-as-an-alternative-to-gazebo/40556/2

[ZhenshengLee]

considering OSI in https://github.com/OpenSimulationInterface and ISO 23150 in https://www.iso.org/standard/83293.html ?

[adamdbrw]

@ZhenshengLee in my opinion, OSI it is not a good match for the task for several reasons:

• In my understanding, OSI is focused on Automotive, and specifically on ground truth data.
• For the simulation_interfaces package as intended, with predecessors currently in use by O3DE and Gazebo, the focus is on automation of validation and testing (spawning items, moving things around to cause scenarios, changing levels, restarting etc.) for all kinds of robots and environments.
• I believe that for ROS system testing, the interface package should be ROS package, primarily utilizing services. This is up for discussion, but anything else would be a hindrance for Open 3D Engine at least, which is directly integrated with ROS.

This does not mean that OSI is not useful, but rather that it solves another problem.

[hijimasa]

Is it common practice to use the services?
I usually use a launch file when I start up a robot in a simulation, and rather than calling the spawn_entity service, I start up a create node.
So the tool I'm making is implemented as a node.

• Isaac Sim
• Unity
• Unreal Engine is now developing

Should I re-implement it so that it can be started up as a service?

[peci1]

So the running node represents the spawned robot and when you kill the node, the robot is removed? Or how is it done?

It makes sense to me to have this functionality as a service because then you can easier integrate spawning multiple robots e.g. in a program loop.

[hijimasa]

I am referring to the create node in the ros_gz_sim package.
In the tool I made, the node automatically ends after the robot is generated in the simulation.

[azeey]

Thanks @adamdbrw for kicking this off. I do agree that having a standardized simulation interface would be good for the community.

Generally, I think we want to start small and try to make the interfaces as generic as possible. Here a few comments/questions toward that goal:

1. Can you clarify what's meant by a spawnable? It's not clear if spawnable is what Gazebo calls model which might be any static or dynamic object with or without kinematics, or if it's specific to dynamic objects like robots
2. I can't find the concept of spawn points in Isaac Sim or Webots, and Gazebo doesn't have them, so I think we should leave those out of the standard.
3. The concept of levels seems different between Gazebo and O3DE. In Gazebo, levels are not used by default. When enabled, levels are loaded automatically based on a robot's location. Their main use case for creating large worlds/scenes. From what I can tell, and @iche033's observation, levels might map better to worlds in Gazebo. I'm also not sure if Webots, or IsaacSim have similar concepts either...at least based on a a quick google search. So I propose leaving get_current_level, get_available_levels and unload_current_level out of the standard.
4. I agree we should have services for resetting the simulation, but we should come up with different terminology. Maybe reset_simulation or simply reset? I would also suggest that we use a different service type. It is not recommended to use messages form std_msgs (see Add deprecation messages to all messages now in example_interfaces ros2/common_interfaces#116), and I think the same logic applies to std_srvs. Instead, I suggest we create a new type called Reset or ResetSimulation. This would allow us to extend the message in the future, for example with the ability to reset to a specific time or reset just the poses of objects without resetting the time.
5. Similarly, I would suggest creating a new type for pausing/unpausing simulation for along the same lines of reasoning as above.

[adamdbrw]

@hijimasa Service carries a response, which is useful in handling of various cases (sim not running, service can be not ready, there can be an error which is then communicated to the user, etc.). As for running a node, I think this is a part of implementation choices. O3DE runs a ROS 2 simulation node(s) natively but other sims might have a different approach, and instead run a kind of wrapper / bridge / communication node.

@azeey I agree to start lean for the core interface, and would also love to make good use of optional fields and the concept of standard extensions. The standard does not have to be implemented entirely from the get-go.

1. Spawnable could be anything you can create in a simulation (object / robot / human).
2. I am not against removing spawn points them from standard, however I think it would be useful for sim to at least be able to provide a list of pre-set poses for spawning with (optional) extra information, even if we subsequently use only the pose part to spawn. It is perfectly ok if the list of pre-defined spawn points is empty for some simulators to start with.
3. Levels map to worlds, yes. I think that ability to switch worlds is an important part of the simulation interfaces, to support the goal of testing automation, to be able to switch between worlds. Would renaming level to scene or world make more sense? Or is it something that you foresee as costly to support in Gazebo?
4. Agree on terminology and use of dedicated service type.
5. Agreed.

I would love to go ahead and prepare a PR to accompany the discussion, where we can have a review process. Could you help with creating a repository?

[DLu]

I commend this effort, and would agree that an empty repository would help in furthering the discussion.

[azeey]

1. Thanks for clarifying about spawnables. I'm not opposed to having the get_available_spawnables service, but my concern is that it might be expensive for the simulator to implement. For example, the sim might have to traverse all installed ROS packages to find all spawnables available. Maybe we can take advantage of ament resource_index, but I haven't explored this much. I did notice that GetWorldProperties is deprecated and replaced with GetModelList in gazebo_ros_pkgs.

2. For spawn_entity, the description says that the robots would be spawned by prefab name. I'm guessing prefab is an O3DE term. Maybe we can say entity name? Also, I think this should optionally take a URI to an an entity describtion (in whatever format) on disk or a remote location.

3. I would agree that switching worlds is important, but I don't think we want to make it part of the standard. The typical way to change worlds in Gazebo is to load a new SDFormat file. This can be done via ROS launch file or by instantiating a Gazebo C++ class in your test executable with the new SDFormat file. Historically, this has been sufficient for Gazebo-Classic and modern Gazebo, and I think for Webots as well. At least for now, I'd rather avoid having to implement new concepts to support this standard and limit ourselves to standardizing the most common interfaces.

4. What's the usecase for get_scene_objects separating the dynamic objects from the robot? Typically Gazebo doesn't make distinctions between robots and other dynamic objects.

Regarding the repository, we'd have to first decide on the org it'll go in. Maybe @gbiggs or @clalancette can comment whether this belongs in ros or ros2.

[adamdbrw]

@azeey Good thinking! Let me comment on each point.

1. I think that get_available_spawnables is a good UX feature and would help in testing as well. It would definitely need to use a new message type, and could start as optional in standard for the first release. In general, I think we won't be able to avoid the need for simulators to implement the standard at some cost, because currently things are done differently and even in different concept spaces.
2. Indeed, prefab should be abstracted away, and instead this should be a resource name, could also support URI. Entity name, I believe, would refer to each instance of such resource / prefab / model, which could be many.
3. I see, we can keep switching worlds / levels / scenes away from the standard, at least to start with.
4. Seems reasonable, I think we also should make sure that the user knows which ways a particular object can be interacted.

Looking forward to submit a PR to the new repo!

[clalancette]

Regarding the repository, we'd have to first decide on the org it'll go in. Maybe @gbiggs or @clalancette can comment whether this belongs in ros or ros2.

It definitely doesn't belong in the ros organization; we are more-or-less keeping that around for ROS 1 purposes.

I also don't think it necessarily belongs in the ros2 organization, because it isn't really "core" to ROS 2.

What do you think about the https://github.com/ros-simulation organization? It is lightly used, but it at least has a bit of history of supporting multiple simulators (it has repositories for both Gazebo and Stage).

[azeey]

ros-simulation sounds good. I've gone ahead created https://github.com/ros-simulation/simulation_interfaces/.

[adamdbrw]

I will draft a PR to the repository early next week