2021-07-01-bhadane21a.md

title	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Principal Bit Analysis: Autoencoding with Schur-Concave Loss	We consider a linear autoencoder in which the latent variables are quantized, or corrupted by noise, and the constraint is Schur-concave in the set of latent variances. Although finding the optimal encoder/decoder pair for this setup is a nonconvex optimization problem, we show that decomposing the source into its principal components is optimal. If the constraint is strictly Schur-concave and the empirical covariance matrix has only simple eigenvalues, then any optimal encoder/decoder must decompose the source in this way. As one application, we consider a strictly Schur-concave constraint that estimates the number of bits needed to represent the latent variables under fixed-rate encoding, a setup that we call \emph{Principal Bit Analysis (PBA)}. This yields a practical, general-purpose, fixed-rate compressor that outperforms existing algorithms. As a second application, we show that a prototypical autoencoder-based variable-rate compressor is guaranteed to decompose the source into its principal components.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	bhadane21a	0	Principal Bit Analysis: Autoencoding with Schur-Concave Loss	852	862	852-862	852	false	Bhadane, Sourbh and Wagner, Aaron B and Acharya, Jayadev	given family Sourbh Bhadane given family Aaron B Wagner given family Jayadev Acharya	2021-07-01		Proceedings of the 38th International Conference on Machine Learning	139	inproceedings	date-parts 2021 7 1	http://proceedings.mlr.press/v139/bhadane21a/bhadane21a.pdf