From 135f4b635ee97673207f6b3b8ea5b5f8a1cee429 Mon Sep 17 00:00:00 2001 From: Charlie Li <33972674+CharlieLeee@users.noreply.github.com> Date: Sat, 8 Jan 2022 13:08:22 +0100 Subject: [PATCH] Update README.md --- README.md | 45 +++++---------------------------------------- 1 file changed, 5 insertions(+), 40 deletions(-) diff --git a/README.md b/README.md index a9c4c48..09f6fce 100644 --- a/README.md +++ b/README.md @@ -1,41 +1,6 @@ -# Quadruped-Sim -This repository contains an environment for simulating a quadruped robot. +# Final version of Reinforcement Learning -## Installation - -Recommend using a virtualenv (or conda) with python3.6 or higher. After installing virtualenv with pip, this can be done as follows: - -`virtualenv {quad_env, or choose another name venv_name} --python=python3` - -To activate the virtualenv: - -`source {PATH_TO_VENV}/bin/activate` - -Your command prompt should now look like: - -`(venv_name) user@pc:path$` - -The repository depends on recent versions of pybullet, numpy, etc., which you can install inside your virtual environment with: - -`pip install -r requirements.txt ` - - - -## Code structure - -- [env](./env) for the quadruped environment files, please see the gym simulation environment [quadruped_gym_env.py](./env/quadruped_gym_env.py), the robot specific functionalities in [quadruped.py](./env/quadruped.py), and config variables in [configs_a1.py](./env/configs_a1.py). You will need to make edits in [quadruped_gym_env.py](./env/quadruped_gym_env.py), and review [quadruped.py](./env/quadruped.py) carefully for accessing robot states and calling functions to solve inverse kinematics, return the leg Jacobian, etc. -- [a1_description](./a1_description) contains the robot mesh files and urdf. -- [utils](./utils) for some file i/o and plotting helpers. -- [hopf_network.py](./hopf_polar.py) provides a CPG class skeleton for various gaits, and maps these to be executed on an instance of the [quadruped_gym_env](./env/quadruped_gym_env.py) class. Please fill in this file carefully. -- [run_sb3.py](./run_sb3.py) and [load_sb3.py](./load_sb3.py) provide an interface to training RL algorithms based on [stable-baselines3](https://github.com/DLR-RM/stable-baselines3). You should review the documentation carefully for information on the different algorithms and training hyperparameters. - -## Code resources -- The [PyBullet Quickstart Guide](https://docs.google.com/document/d/10sXEhzFRSnvFcl3XxNGhnD4N2SedqwdAvK3dsihxVUA/edit#heading=h.2ye70wns7io3) is the current up-to-date documentation for interfacing with the simulation. -- The quadruped environment took inspiration from [Google's motion-imitation repository](https://github.com/google-research/motion_imitation) based on [this paper](https://xbpeng.github.io/projects/Robotic_Imitation/2020_Robotic_Imitation.pdf). -- Reinforcement learning algorithms from [stable-baselines3](https://github.com/DLR-RM/stable-baselines3). Also see for example [ray[rllib]](https://github.com/ray-project/ray) and [spinningup](https://github.com/openai/spinningup). - -## Conceptual resources -The CPGs are based on the following papers: -- L. Righetti and A. J. Ijspeert, "Pattern generators with sensory feedback for the control of quadruped locomotion," 2008 IEEE International Conference on Robotics and Automation, 2008, pp. 819-824, doi: 10.1109/ROBOT.2008.4543306. [link](https://ieeexplore.ieee.org/document/4543306) -- M. Ajallooeian, S. Pouya, A. Sproewitz and A. J. Ijspeert, "Central Pattern Generators augmented with virtual model control for quadruped rough terrain locomotion," 2013 IEEE International Conference on Robotics and Automation, 2013, pp. 3321-3328, doi: 10.1109/ICRA.2013.6631040. [link](https://ieeexplore.ieee.org/abstract/document/6631040) -- M. Ajallooeian, S. Gay, A. Tuleu, A. Spröwitz and A. J. Ijspeert, "Modular control of limit cycle locomotion over unperceived rough terrain," 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 3390-3397, doi: 10.1109/IROS.2013.6696839. [link](https://ieeexplore.ieee.org/abstract/document/6696839) \ No newline at end of file +1. Most of the models in the report can be found in the folder `logs/intermediate_models/`. +2. Each model comes with a training description named as `train_detail.txt`, please modify accordingly in the 'load_sb3.py` file. +3. Please refer to [our report](https://www.overleaf.com/read/zkthgpqkjygg) for more info. +4. Feel free to contact us through {chengkun.li, shuhan.li, jiangfan.li}@epfl.ch if you have any questions on this branch.