Skip to content

Commit

Permalink
Add pre-commit
Browse files Browse the repository at this point in the history
  • Loading branch information
Dechen committed Aug 16, 2024
1 parent 8d38757 commit 360f0d3
Show file tree
Hide file tree
Showing 171 changed files with 12,169 additions and 12,131 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ jobs:
runs-on: ubuntu-latest
name: A job to automate contrib in readme
permissions:
contents: write
pull-requests: write
contents: write
pull-requests: write
steps:
- name: Contribute List
uses: akhilmhdh/[email protected]
Expand Down
59 changes: 59 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files

- repo: https://github.com/psf/black
rev: 24.8.0
hooks:
- id: black
args: [--line-length=150]

- repo: https://github.com/pycqa/isort
rev: 5.13.2
hooks:
- id: isort
args: ["--profile", "black", "--line-length", "150"]

- repo: https://github.com/pycqa/flake8
rev: 7.1.1
hooks:
- id: flake8
args:
- --max-line-length=150
- --ignore=E203,E266,E501,W503,F401,E402,E731
- --per-file-ignores=__init__.py:F401,F403

- repo: https://github.com/adrienverge/yamllint
rev: v1.35.1
hooks:
- id: yamllint
args:
- --config-data
- |
extends: default
rules:
line-length:
max: 150
truthy: disable
document-start: disable
braces:
min-spaces-inside: 0
max-spaces-inside: -1
comments:
min-spaces-from-content: 1
- repo: https://github.com/google/yapf
rev: v0.40.2
hooks:
- id: yapf
args: ["--style", "{column_limit: 150, align_closing_bracket_with_visual_indent: true, dedent_closing_brackets: true}"]

- repo: https://github.com/executablebooks/mdformat
rev: 0.7.17
hooks:
- id: mdformat
46 changes: 21 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,16 +22,15 @@

</div>

---

______________________________________________________________________

Unleash the power of **imagination** and **generalization** of world models for self-driving cars.

> [!NOTE]
> \[!NOTE\]
>
> - **July 2024:** Created a stop-sign task and a traffic-light task!
> - **July 2024:** Uploaded all the task checkpoints to [HuggingFace](https://huggingface.co/ucd-dare/CarDreamer/tree/main)

## **Can world models imagine traffic dynamics for training autonomous driving agents? The answer is YES!**

Integrating the high-fidelity CARLA simulator with world models, we are able to train a world model that not only learns complex environment dynamics but also have an agent interact with the neural network "simulator" to learn to drive.
Expand All @@ -44,18 +43,17 @@ Dive into our demos to see the agent skillfully navigating challenges and ensuri

**Explore** world model based autonomous driving with CarDreamer, an open-source platform designed for the **development** and **evaluation** of **world model** based autonomous driving.

* 🏙️ **Built-in Urban Driving Tasks**: flexible and customizable observation modality and observability; optimized rewards
* 🔧 **Task Development Suite**: create your own urban driving tasks with ease
* 🌍 **Model Backbones**: integrated state-of-the-art world models
- 🏙️ **Built-in Urban Driving Tasks**: flexible and customizable observation modality and observability; optimized rewards
- 🔧 **Task Development Suite**: create your own urban driving tasks with ease
- 🌍 **Model Backbones**: integrated state-of-the-art world models

**Documentation:** [CarDreamer API Documents](https://car-dreamer.readthedocs.io/en/latest/).

**Looking for more techincal details? Check our report here! [Paper link](https://arxiv.org/abs/2405.09111)**


## :sun_with_face: Built-in Task Demos

> [!TIP]
> \[!TIP\]
> A world model is learnt to model traffic dynamics; then a driving agent is trained on world model's imagination! The driving agent masters diverse driving skills including lane merge, left turn, and right turn, to random roadming purely **from scratch**.
We train DreamerV3 agents on our built-in tasks with a single 4090. Depending on the observation spaces, the memory overhead ranges from 10GB-20GB alongwith 3GB reserved for CARLA.
Expand All @@ -72,30 +70,26 @@ We train DreamerV3 agents on our built-in tasks with a single 4090. Depending on
| :-----------: | :-------: |
| ![Traffic Light](https://raw.githubusercontent.com/ucd-dare/cardreamer.github.io/main/static/gifs/bev/tl_right.gif) | ![Stop Sign](https://raw.githubusercontent.com/ucd-dare/cardreamer.github.io/main/static/gifs/bev/stop%20sign.gif) |


## :blossom: The Power of Intention Sharing

> [!TIP]
> \[!TIP\]
> **Human drivers use turn signals to inform their intentions** of turning left or right. **Autonomous vehicles can do the same!**
Let's see how CarDreamer agents communicate and leverage intentions. Our experiment have demonstrated that through sharing intention, the policy learning is much easier! Specifically, a policy without knowing other agents' intentions can be conservative in our crossroad tasks; while intention sharing allows the agents to find the proper timing to cut in the traffic flow.


<!-- Table 1: Sharing waypoints vs. Without sharing waypoints -->

| Sharing waypoints vs. Without sharing waypoints | Sharing waypoints vs. Without sharing waypoints |
| :---------------------------------------------: | :---------------------------------------------: |
| **Right turn hard** | **Left turn hard** |
| ![Right turn hard](./.assets/right_turn_hard.gif) &emsp;&emsp;&emsp; ![Right turn hard no waypoint](./.assets/right_turn_raw_fail.gif) | ![Left turn hard](./.assets/left_turn_hard.gif) &emsp;&emsp;&emsp;<img src="./.assets/left turn raw.gif" style="width: 100%"> |
| ![Right turn hard](./.assets/right_turn_hard.gif)     ![Right turn hard no waypoint](./.assets/right_turn_raw_fail.gif) | ![Left turn hard](./.assets/left_turn_hard.gif)    <img src="./.assets/left turn raw.gif" style="width: 100%"> |

<!-- Table 2: Full observability vs. Partial observability -->

| Full observability vs. Partial observability |
| :------------------------------------------: |
| **Right turn hard** |
| ![Right turn hard](./.assets/right_turn_hard.gif) &emsp;&emsp;&emsp; ![Right turn hard FOV](./.assets/right_turn_hard_fov.gif) |




| ![Right turn hard](./.assets/right_turn_hard.gif)     ![Right turn hard FOV](./.assets/right_turn_hard_fov.gif) |

## 📋 Prerequisites

Expand All @@ -108,14 +102,14 @@ git clone https://github.com/ucd-dare/CarDreamer
cd CarDreamer
```

Download [CARLA release](https://github.com/carla-simulator/carla/releases) of version ``0.9.15`` as we experiemented with this version. Set the following environment variables:
Download [CARLA release](https://github.com/carla-simulator/carla/releases) of version `0.9.15` as we experiemented with this version. Set the following environment variables:

```bash
export CARLA_ROOT="</path/to/carla>"
export PYTHONPATH="${CARLA_ROOT}/PythonAPI/carla":${PYTHONPATH}
```

Install the package using flit. The ``--symlink`` flag is used to create a symlink to the package in the Python environment, so that changes to the package are immediately available without reinstallation. (``--pth-file`` also works, as an alternative to ``--symlink``.)
Install the package using flit. The `--symlink` flag is used to create a symlink to the package in the Python environment, so that changes to the package are immediately available without reinstallation. (`--pth-file` also works, as an alternative to `--symlink`.)

```bash
conda create python=3.10 --name cardreamer
Expand All @@ -132,7 +126,7 @@ The model backbones are decoupled from CarDreamer tasks or the development sutie

### :mechanical_arm: Training

Find ``README.md`` in the corresponding directory of the algorithm you want to use and follow the instructions. For example, to train DreamerV3 agents, use
Find `README.md` in the corresponding directory of the algorithm you want to use and follow the instructions. For example, to train DreamerV3 agents, use

```bash
bash train_dm3.sh 2000 0 --task carla_four_lane --dreamerv3.logdir ./logdir/carla_four_lane
Expand Down Expand Up @@ -170,7 +164,7 @@ Users can enable built-in observation handlers such as BEV, camera, LiDAR, and s
To implement new handlers for different observation sources and modalities (e.g., text, velocity, locations, or even more complex data), `CarDreamer` provides two methods:

1. Register a callback as a [SimpleHandler](https://github.com/ucd-dare/CarDreamer/blob/master/car_dreamer/toolkit/observer/handlers/simple_handler.py) to fetch data at each step.
2. For observations requiring complex workflows that cannot be conveyed by a `SimpleHandler`, create an handler maintaining the full lifecycle of that observation, similar to our built-in message, BEV, spectator handlers.
1. For observations requiring complex workflows that cannot be conveyed by a `SimpleHandler`, create an handler maintaining the full lifecycle of that observation, similar to our built-in message, BEV, spectator handlers.

For more details on defining new observation sources, see [CarDreamer Docs: Defining a new observation source](https://car-dreamer.readthedocs.io/en/latest/customization.html#defining-a-new-observation-source).

Expand All @@ -197,7 +191,7 @@ your_task_name:
observation.enabled: [camera, collision, spectator, birdeye_view]
```
#### Environment \& Observer Communications
#### Environment & Observer Communications
One might need transfer information from the environements to a handler to compute their observations. E.g., a BEV handler might need a location to render the destination spot. These environment information can be accessed either through [WorldManager](https://car-dreamer.readthedocs.io/en/latest/api/toolkit.html#car_dreamer.toolkit.WorldManager) APIs, or through environment state management.
Expand Down Expand Up @@ -249,6 +243,7 @@ To easily customize your own driving tasks, and observation spaces, etc., please
If you find this repository useful, please cite this paper:

**[Paper link](https://arxiv.org/abs/2405.09111)**

```
@article{CarDreamer2024,
title = {{CarDreamer: Open-Source Learning Platform for World Model based Autonomous Driving}},
Expand All @@ -259,9 +254,10 @@ If you find this repository useful, please cite this paper:
}
```
# Suppliment Material
## World model imagination
<p align="center">
Birdeye view training
</p>
Expand All @@ -275,12 +271,12 @@ If you find this repository useful, please cite this paper:
</p>
<img src="./.assets/right_turn_hard_pre_lidar.gif">
# 👥 Contributors
Special thanks to the community for your valuable contributions and support in making CarDreamer better for everyone! The contributor list is automatically generated based on commit history.
<!-- readme: contributors -start -->
<table>
<tbody>
<tr>
Expand Down
36 changes: 21 additions & 15 deletions car_dreamer/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,19 @@
"""

from .carla_base_env import CarlaBaseEnv
from .carla_roundabout_env import CarlaRoundaboutEnv
from .carla_right_turn_env import CarlaRightTurnEnv
from .carla_overtake_env import CarlaOvertakeEnv
from .carla_navigation_env import CarlaNavigationEnv
from .carla_left_turn_env import CarlaLeftTurnEnv
from .carla_lane_merge_env import CarlaLaneMergeEnv
from .carla_four_lane_env import CarlaFourLaneEnv
from .carla_traffic_lights_env import CarlaTrafficLightsEnv
from .carla_lane_merge_env import CarlaLaneMergeEnv
from .carla_left_turn_env import CarlaLeftTurnEnv
from .carla_navigation_env import CarlaNavigationEnv
from .carla_overtake_env import CarlaOvertakeEnv
from .carla_right_turn_env import CarlaRightTurnEnv
from .carla_roundabout_env import CarlaRoundaboutEnv
from .carla_stop_sign_env import CarlaStopSignEnv
from .carla_wpt_fixed_env import CarlaWptFixedEnv
from .carla_traffic_lights_env import CarlaTrafficLightsEnv
from .carla_wpt_env import CarlaWptEnv
__version__ = '0.2.0'
from .carla_wpt_fixed_env import CarlaWptFixedEnv

__version__ = "0.2.0"

from . import toolkit

Expand All @@ -34,13 +35,15 @@ def load_task_configs(task_name: str):
:return: the task configs
"""
import yaml
import os
dir = os.path.join(os.path.dirname(__file__), 'configs')
with open(os.path.join(dir, 'common.yaml'), 'r') as f:

import yaml

dir = os.path.join(os.path.dirname(__file__), "configs")
with open(os.path.join(dir, "common.yaml")) as f:
config = yaml.safe_load(f)
config = toolkit.Config(config)
with open(os.path.join(dir, 'tasks.yaml'), 'r') as f:
with open(os.path.join(dir, "tasks.yaml")) as f:
task_config = yaml.safe_load(f)
config = config.update(task_config[task_name])
return config
Expand All @@ -57,20 +60,23 @@ def create_task(task_name: str, argv=None):
:return: a tuple of the created environment and the configs
"""
import gym

config = load_task_configs(task_name)
config, _ = toolkit.Flags(config).parse_known(argv)
return gym.make(config.env.name, config=config.env), config


def _register_envs():
import os
from gym.envs.registration import register
from re import sub

from gym.envs.registration import register

def toClassName(s):
return sub(r"(_|-)+", " ", s).title().replace(" ", "")

for file in os.listdir(os.path.dirname(__file__)):
if file.endswith('env.py') and file != '__init__.py':
if file.endswith("env.py") and file != "__init__.py":
file_name = file[:-3]
class_name = toClassName(file_name)
exec(f"register(id='{class_name}-v0', entry_point='car_dreamer.{file_name}:{class_name}')")
Expand Down
Loading

0 comments on commit 360f0d3

Please sign in to comment.