Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Low real time factor and sky rendering issue #717

Open
lijianwen1997 opened this issue Aug 11, 2023 · 5 comments
Open

Low real time factor and sky rendering issue #717

lijianwen1997 opened this issue Aug 11, 2023 · 5 comments
Assignees

Comments

@lijianwen1997
Copy link

lijianwen1997 commented Aug 11, 2023

Describe the bug
I cloned the latest VRX repo and ran the simultor with default settings. However, the real time factor was very low (38%). If I increase the step size to 0.05, it can increase to 78% but I don't think it is the right way to solve the problem. Plus the sky is not rendered properly. I tried both nvidia-driver-470 and nvidia-driver-530 but it did not help. It will be great if you can let me know how to increase the real time rate and fix the rendering issue.

To Reproduce

  1. Run ros2 launch vrx_gz competition.launch.py

System Configuration:

  • OS: Ubuntu 22.04
  • ROS Version: ROS 2, Humble
  • Gazebo Version: Gazebo Sim 7.5.0
  • Graphics Card: Nvidia RTX 2080
  • GPU Driver: nvidia-driver-530 / nvidia-driver-470
  • CPU: i7-8700
  • Memory: 32 GB (only 30% is used when VRX is running)
  • VRX

Screenshots

Screenshot from 2023-08-11 17-38-46

Additional notes
We also reproduced this low RTF and redenring issue on a Laptop with i7-8565 CPU and UHD 620 GPU.

@M1chaelM
Copy link
Collaborator

M1chaelM commented Aug 14, 2023

We usually see these symptoms when the system is unable to access the GPU (so it makes sense that you would see this on the laptop). Are you using Docker or running directly on hardware?

Assuming you are running directly on hardware:

  • If you could try driver version 470 for now that would be helpful for troubleshooting.
  • What is the output of nvidia-smi?

@lijianwen1997
Copy link
Author

lijianwen1997 commented Aug 14, 2023

Thanks very much for your help!
I am running directly on hardware. I changed the driver to 470 but it did not fix the issues. The output of nvidia-smi when vrx is running is as below:
Screenshot from 2023-08-14 16-05-49

What's more, I saw ruby 3.0 has been stopped unexpectedly. Not sure if that caused the issues.
Screenshot from 2023-08-14 16-06-08

@lijianwen1997
Copy link
Author

https://gazebosim.org/docs/garden/troubleshooting#prime-select-command-line-tool

I visited gazebo troubleshoot page and configured Gazebo to always use Nvidia and it fixed the sky rendering issue! The RTF increased to 60%.

Please let me know if you have any suggestions to increase the RTF.

Screenshot from 2023-08-14 16-35-12

@caguero caguero self-assigned this Aug 22, 2023
@caguero
Copy link
Contributor

caguero commented Aug 22, 2023

@tejalbarnwal was kind enough to test a few things:

  • Confirm that commenting the cameras and laser of the WAM-V restores RTF to 100%.

https://github.com/osrf/vrx/blob/main/vrx_urdf/wamv_gazebo/urdf/wamv_gazebo.urdf.xacro#L162-L166

  • Try out a simplified version of the WAM-V ideally with just a single link (no joints) and the minimum amount of plugins. Maybe just one buoyancy plugin and one hydrodynamics plugin. See if this makes any difference.

@tejalbarnwal
Copy link
Collaborator

tejalbarnwal commented Aug 27, 2023

Hey,
I tested out disabling the 3D lidar and the camera sensors, and I found the following outcomes:

Test Condition RTF
With Lidar and Cameras 0.82
Without Lidar approximately 0.82
Without cameras and disabled Lidar visualization 0.95
Without cameras and enabled Lidar visualization fluctuating between 0.82 and 0.96(average around 0.89)
Without cameras and Lidar 0.995

It appears that the cameras may be the primary cause of the reduced RTF. I plan to assess a basic URDF with minimal joints and plugins later tonight.

@tejalbarnwal tejalbarnwal self-assigned this Aug 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants