Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Making this work again #15

Open
alexswerner opened this issue Oct 24, 2024 · 2 comments
Open

Making this work again #15

alexswerner opened this issue Oct 24, 2024 · 2 comments

Comments

@alexswerner
Copy link

Hi,

amazing work here. But the software has moved on and I would like to make it work again. So far I have:

  • Fixed the code to work with the new pyelastica API
  • Updated from stable_baselines to stable_baselines3
  • Updated from gym to gymnasium v26 (also required by stable_baselines3)
  • Note that stable_baselines3 is based on pytorch
  • Parallelized the training for e.g. PPO

Right now I can execute the training for PPO in Case1 which is at least something.
This is by directly calling:

python3 logging_bio_args.py --total_timesteps=10000000.0 --SEED=0 --timesteps_per_batch=32000 --algo=PPO

I pushed the code here: https://github.com/alexswerner/Elastica-RL-control

However, the output of the post processing script does not show a moving robot.
:-(

So there is still some work to do.

@alexswerner
Copy link
Author

The PPO test case seems to work again after some debugging. Key findings:

  • Passing an empty self.spline_points_func_array_normal_dir = [] to the constructor of MuscleTorquesWithVaryingBetaSplines does not work as intended.
  • Switching to the dampen API using AnalyticalLinearDamper needs apparently a different nu value
  • Parallelizing PPO does not yield the nice convergence without further modification of the parameters

One option question for precisely reproducing the test case is how to transfer the
nu specified in the old Cosserat rod constructor to the new dampen API.

@armantekinalp
Copy link
Contributor

Hey,

Thank you for your interest.

For the current PyElastica damping implementation, you can scale the nu as nu_new = nu_old / density / base_area

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants