README.md

DiffusionDrive

Truncated Diffusion Model for End-to-End Autonomous Driving

Bencheng Liao^1,2, Shaoyu Chen^2,3, Haoran Yin³, Bo Jiang², Cheng Wang^1,2, Sixu Yan², Xinbang Zhang³, Xiangyu Li³, Ying Zhang³, Qian Zhang³, Xinggang Wang^{2 📧}

¹ Institute of Artificial Intelligence, HUST, ² School of EIC, HUST, ³ Horizon Robotics

(^📧) corresponding author, [email protected]

   

News

• Dec. 16th, 2024: We release the initial version of code and weight on NAVSIM, along with documentation and training/evaluation scripts.
• Nov. 25th, 2024: We released our paper on Arxiv. Code/Models are coming soon. Please stay tuned! ☕️

Table of Contents

• Introduction
• Qualitative Results on NAVSIM Navtest Split
• Video Demo on Real-world Application
• Getting Started
• Contact
• Acknowledgement
• Citation

Introduction

Diffusion policy exhibits promising multimodal property and distributional expressivity in robotic field, while not ready for real-time end-to-end autonomous driving in more dynamic and open-world traffic scenes. To bridge this gap, we propose a novel truncated diffusion model, DiffusionDrive, for real-time end-to-end autonomous driving, which is much faster (10x reduction in diffusion denoising steps), more accurate (3.5 higher PDMS on NAVSIM), and more diverse (64% higher mode diversity score) than the vanilla diffusion policy. Without bells and whistles, DiffusionDrive achieves record-breaking 88.1 PDMS on NAVSIM benchmark with the same ResNet-34 backbone by directly learning from human demonstrations, while running at a real-time speed of 45 FPS.

Truncated Diffusion Policy. Pipeline of DiffusionDrive. DiffusionDrive is highly flexible to integrate with onboard sensor data and existing perception modules.

Qualitative Results on NAVSIM Navtest Split

Going straight with car-following and lane-changing behaviors. Going straight with diverse lane-changing behavior, which interacts with traffic light and stops at the stop line. Turning left with diverse lane-changing behavior, which interacts with surrounding agents. Turning right with car-following and overtaking behaviors.

Video Demo on Real-world Application

final_github.mp4

Getting Started

• Getting started from NAVSIM environment preparation
• Preparation of DiffusionDrive environment
• Training and Evaluation

Checkpoint

Method	Model Size	Backbone	PDMS	Weight Download
DiffusionDrive	60M	ResNet-34	88.1	Hugging Face

Contact

If you have any questions, please contact Bencheng Liao via email ([email protected]).

Acknowledgement

DiffusionDrive is greatly inspired by the following outstanding contributions to the open-source community: NAVSIM, Transfuser, Diffusion Policy, MapTR, VAD, SparseDrive.

Citation

If you find DiffusionDrive is useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.

 @article{diffusiondrive,
  title={DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving},
  author={Bencheng Liao and Shaoyu Chen and Haoran Yin and Bo Jiang and Cheng Wang and Sixu Yan and Xinbang Zang and Xiangyu Li and Ying Zhang and Qian Zhang and Xinggang Wang},
  journal={arXiv preprint arXiv:2411.15139},
  year={2024}
}