2024-10-06-goemaere24a.md

title	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Accelerating Hopfield Network Dynamics: Beyond Synchronous Updates and Forward Euler	The Hopfield network serves as a fundamental energy-based model in machine learning, capturing memory retrieval dynamics through an ordinary differential equation (ODE). The model’s output, the equilibrium point of the ODE, is traditionally computed via synchronous updates using the forward Euler method. This paper aims to overcome some of the disadvantages of this approach. We propose a conceptual shift, viewing Hopfield networks as instances of Deep Equilibrium Models (DEQs). The DEQ framework not only allows for the use of specialized solvers, but also leads to new insights on an empirical inference technique that we will refer to as ’even-odd splitting’. Our theoretical analysis of the method uncovers a parallelizable asynchronous update scheme, which should converge roughly twice as fast as the conventional synchronous updates. Empirical evaluations validate these findings, showcasing the advantages of both the DEQ framework and even-odd splitting in digitally simulating energy minimization in Hopfield networks. The code is available at https://github.com/cgoemaere/hopdeq.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	goemaere24a	0	Accelerating Hopfield Network Dynamics: Beyond Synchronous Updates and Forward Euler	1	21	1-21	1	false	Goemaere, C\'{e}dric and Deleu, Johannes and Demeester, Thomas	given family Cédric Goemaere given family Johannes Deleu given family Thomas Demeester	2024-10-06		Proceedings of the 1st ECAI Workshop on "Machine Learning Meets Differential Equations: From Theory to Applications"	255	inproceedings	date-parts 2024 10 6	https://raw.githubusercontent.com/mlresearch/v255/main/assets/goemaere24a/goemaere24a.pdf