2021-07-21-jung21a.md

title	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Moment Multicalibration for Uncertainty Estimation	We show how to achieve the notion of "multicalibration" from Hebert-Johnson et al. (2018) not just for means, but also for variances and other higher moments. Informally, this means that we can find regression functions which, given a data point, can make point predictions not just for the expectation of its label, but for higher moments of its label distribution as well—and those predictions match the true distribution quantities when averaged not just over the population as a whole, but also when averaged over an enormous number of finely defined subgroups. It yields a principled way to estimate the uncertainty of predictions on many different subgroups—and to diagnose potential sources of unfairness in the predictive power of features across subgroups. As an application, we show that our moment estimates can be used to derive marginal prediction intervals that are simultaneously valid as averaged over all of the (sufficiently large) subgroups for which moment multicalibration has been obtained.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	jung21a	0	Moment Multicalibration for Uncertainty Estimation	2634	2678	2634-2678	2634	false	Jung, Christopher and Lee, Changhwa and Pai, Mallesh and Roth, Aaron and Vohra, Rakesh	given family Christopher Jung given family Changhwa Lee given family Mallesh Pai given family Aaron Roth given family Rakesh Vohra	2021-07-21		Proceedings of Thirty Fourth Conference on Learning Theory	134	inproceedings	date-parts 2021 7 21	http://proceedings.mlr.press/v134/jung21a/jung21a.pdf