Skip to content

Latest commit

 

History

History
87 lines (65 loc) · 3 KB

README.md

File metadata and controls

87 lines (65 loc) · 3 KB
Raw

PBRL - Population-Based Reinforcement Learning

Website | Paper | Videos

About this repository

This repository contains the code to train RL agents in Population-Based manner.

Installation

  1. Create a new conda environment with:

    conda create -n pbrl python=3.8
conda activate pbrl

  2. Install IsaacGym (tested with Preview 4 Release). Follow the instructions to download the package.

    tar -xvf IsaacGym_Preview_4_Package.tar.gz
cd isaacgym/python
pip install -e .
# Test IsaacGym installation
cd examples
python joint_monkey.py

  3. Install this repo:

    git clone https://github.com/Asad-Shahid/PBRL.git
cd PBRL 
pip install -e .

Getting Started

Navigate to the isaacgymenvs directory and run:

python -m pbrl.main --task ShadowHand --num_envs 4096 --num_agents 4

Some key arguments are:

  • --task selects a task from isaacgymenvs. All tasks released in isaacgymenvs==1.5.1 are supported.
  • --num_envs selects the number of environments to run.
  • --num_agents chooses the number of agents to train in parallel for pbrl. Note: num_envs must divide num_agents.
  • --pbrl whether to use PBRL.
    Note: when True, num_agents must be a multiple of 4 (Top 25% of the agents are selected to replace/get replaced).
  • --algo which RL algorithm to use for training. Options are: ppo, sac, ddpg.
  • --pbrl_params name of .json file with hyperparameters to tune in cfg/pbrl. Only hyperparameters listed in corresponding .json files are currently supported. Initial values are sampled uniformly from the specified range. When training a single RL agent, specify inital values in the file.
  • --mut_scheme which mutation scheme to use for mutating hyperparameters.

All other arguments can be found in cfg/pbrl/_init__.py

Citing

Please cite this work as:

@article{Shahid2024pbrl,
  author = {Asad Ali Shahid and Yashraj Narang and Vincenzo Petrone and Enrico Ferrentino and Ankur Handa and Dieter Fox and Marco Pavone and Loris Roveda},
  title = {Scaling Population-Based Reinforcement Learning with GPU Accelerated Simulation},
  journal = {arXiv preprint},
  year = {2024},
  doi = {10.48550/arXiv.2404.03336},
  url = {https://doi.org/10.48550/arXiv.2404.03336},
}

If you reuse our code, you can site this repo as:

@software{Shahid2024pbrlrepo,
author = {Shahid, Asad Ali and Narang, Yashraj and Petrone, Vincenzo and Ferrentino, Enrico and Handa, Ankur and Fox, Dieter and Pavone, Marco and Roveda, Loris},
doi = {10.5281/zenodo.TBD},
month = mar,
title = {{Scaling Population-Based Reinforcement Learning with GPU Accelerated Simulation}},
url = {https://github.com/Asad-Shahid/PBRL},
version = {1.0.0},
year = {2024}
}