2024-12-23-dahan24a.md

title	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
The Multiscale Surface Vision Transformer	Surface meshes are a favoured domain for representing structural and functional information on the human cortex, but their complex topology and geometry pose significant challenges for deep learning analysis. While Transformers have excelled as domain-agnostic architectures for sequence-to-sequence learning, the quadratic cost of the self-attention operation remains an obstacle for many dense prediction tasks. Inspired by some of the latest advances in hierarchical modelling with vision transformers, we introduce the Multiscale Surface Vision Transformer (MS-SiT) as a backbone architecture for surface deep learning. The self-attention mechanism is applied within local-mesh-windows to allow for high-resolution sampling of the underlying data, while a shifted-window strategy improves the sharing of information between windows. Neighbouring patches are successively merged, allowing the MS-SiT to learn hierarchical representations suitable for any prediction task. Results demonstrate that the MS-SiT outperforms existing surface deep learning methods for neonatal phenotyping prediction tasks using the Developing Human Connectome Project (dHCP) dataset. Furthermore, building the MS-SiT backbone into a U-shaped architecture for surface segmentation demonstrates competitive results on cortical parcellation using the UK Biobank (UKB) and manually-annotated MindBoggle datasets. Code and trained models are publicly available at https://github.com/metrics-lab/surface-vision-transformers.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	dahan24a	0	The Multiscale Surface Vision Transformer	289	305	289-305	289	false	Dahan, Simon and Williams, Logan Zane John and Rueckert, Daniel and Robinson, Emma Claire	given family Simon Dahan given family Logan Zane John Williams given family Daniel Rueckert given family Emma Claire Robinson	2024-12-23		Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning	250	inproceedings	date-parts 2024 12 23	https://raw.githubusercontent.com/mlresearch/v250/main/assets/dahan24a/dahan24a.pdf