Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement video capturing system #29

Draft
wants to merge 23 commits into
base: main
Choose a base branch
from

Conversation

lorycontixd
Copy link

No description provided.

lorycontixd and others added 21 commits October 9, 2024 11:22
- Generalised urdf_path input for robotModel class to pass either string or a pathlib object
- [To review] Added save_xml flag on robotModel.get_mujoco_model to save the modified xml file
… with associated default values taken from mujoco
Still work in progress, implemented basic structure for PR.

- ScoreCallback associates a simulation to a score which can be customly defined
- TrackerCallback helps to store variable values for either printing or eventual plotting
- Dumping urdf string to temp file if mujoco fails to load
- Added should_stop flag to mujocoSimulator class
@lorycontixd lorycontixd marked this pull request as draft December 13, 2024 14:55
@lorycontixd lorycontixd self-assigned this Dec 13, 2024
@lorycontixd
Copy link
Author

First draft of a video recording system for mujoco.

The idea is to read an image at each frame and save it in a temporary folder. At the end of the simulation, save the video using opencv.

Unfortunately for now it is quite slow.

@lorycontixd
Copy link
Author

@flferretti Do you have any suggestions on the implementation by any chance? (The changes are in the last commit only)

@flferretti
Copy link
Contributor

@flferretti Do you have any suggestions on the implementation by any chance? (The changes are in the last commit only)

I'd avoid writing each frame to a tempfile as it is extremely slow. You can just save it as a numpy array (the default output) in a class attribute. Moreover, try to use the mujoco passive viewer, so the process of headless rendering does not overlap with the interactive viewer used in MujocoSimulator:

self.viewer = mujoco_viewer.MujocoViewer(self.model, self.data)

You might also take inspiration from how we use the mujoco passive viewer in JaxSim https://github.com/ami-iit/jaxsim/blob/main/src/jaxsim/mujoco/visualizer.py

@lorycontixd
Copy link
Author

As of the last commit, the recorder kills the python process because it runs out of memory. Investigating...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants