2024-04-18-chen24a.md

title	software	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Lower-level Duality Based Reformulation and Majorization Minimization Algorithm for Hyperparameter Optimization	https://github.com/libra-licoho/LDMMA	Hyperparameter tuning is an important task of machine learning, which can be formulated as a bilevel program (BLP). However, most existing algorithms are not applicable for BLP with non-smooth lower-level problems. To address this, we propose a single-level reformulation of the BLP based on lower-level duality without involving any implicit value function. To solve the reformulation, we propose a majorization minimization algorithm that marjorizes the constraint in each iteration. Furthermore, we show that the subproblems of the proposed algorithm for several widely-used hyperparameter turning models can be reformulated into conic programs that can be efficiently solved by the off-the-shelf solvers. We theoretically prove the convergence of the proposed algorithm and demonstrate its superiority through numerical experiments.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	chen24a	0	Lower-level Duality Based Reformulation and Majorization Minimization Algorithm for Hyperparameter Optimization	784	792	784-792	784	false	Chen, He and Xu, Haochen and Jiang, Rujun and Man-Cho So, Anthony	given family He Chen given family Haochen Xu given family Rujun Jiang given family Anthony Man-Cho So	2024-04-18		Proceedings of The 27th International Conference on Artificial Intelligence and Statistics	238	inproceedings	date-parts 2024 4 18	https://proceedings.mlr.press/v238/chen24a/chen24a.pdf