2023-12-02-chen23b.md

title	section	openreview	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulation	Poster	efaE7iJ2GJv	The ability for robots to comprehend and execute manipulation tasks based on natural language instructions is a long-term goal in robotics. The dominant approaches for language-guided manipulation use 2D image representations, which face difficulties in combining multi-view cameras and inferring precise 3D positions and relationships. To address these limitations, we propose a 3D point cloud based policy called PolarNet for language-guided manipulation. It leverages carefully designed point cloud inputs, efficient point cloud encoders, and multimodal transformers to learn 3D point cloud representations and integrate them with language instructions for action prediction. PolarNet is shown to be effective and data efficient in a variety of experiments conducted on the RLBench benchmark. It outperforms state-of-the-art 2D and 3D approaches in both single-task and multi-task learning. It also achieves promising results on a real robot.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	chen23b	0	PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulation	1761	1781	1761-1781	1761	false	Chen, Shizhe and Pinel, Ricardo Garcia and Schmid, Cordelia and Laptev, Ivan	given family Shizhe Chen given family Ricardo Garcia Pinel given family Cordelia Schmid given family Ivan Laptev	2023-12-02		Proceedings of The 7th Conference on Robot Learning	229	inproceedings	date-parts 2023 12 2	https://proceedings.mlr.press/v229/chen23b/chen23b.pdf