2024-09-12-bowyer24a.md

title	abstract	openreview	software	section	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Using Autodiff to Estimate Posterior Moments, Marginals and Samples	Importance sampling is a popular technique in Bayesian inference: by reweighting samples drawn from a proposal distribution we are able to obtain samples and moment estimates from a Bayesian posterior over latent variables. Recent work, however, indicates that importance sampling scales poorly — in order to accurately approximate the true posterior, the required number of importance samples grows is exponential in the number of latent variables [Chatterjee and Diaconis, 2018]. Massively parallel importance sampling works around this issue by drawing $K$ samples for each of the $n$ latent variables and reasoning about all $K^n$ combinations of latent samples. In principle, we can reason efficiently over $K^n$ combinations of samples by exploiting conditional independencies in the generative model. However, in practice this requires complex algorithms that traverse backwards through the graphical model, and we need separate backward traversals for each computation (posterior expectations, marginals and samples). Our contribution is to exploit the source term trick from physics to entirely avoid the need to hand-write backward traversals. Instead, we demonstrate how to simply and easily compute all the required quantities — posterior expectations, marginals and samples — by differentiating through a slightly modified marginal likelihood estimator.	QUMZJgrjN0	https://github.com/sambowyer/MPIS	Papers	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	bowyer24a	0	Using Autodiff to Estimate Posterior Moments, Marginals and Samples	394	417	394-417	394	false	Bowyer, Sam and Heap, Thomas and Aitchison, Laurence	given family Sam Bowyer given family Thomas Heap given family Laurence Aitchison	2024-09-12		Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence	244	inproceedings	date-parts 2024 9 12	https://raw.githubusercontent.com/mlresearch/v244/main/assets/bowyer24a/bowyer24a.pdf