2024-08-14-semola24a.md

title	abstract	openreview	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Adaptive Hyperparameter Optimization for Continual Learning Scenarios	Hyperparameter selection in continual learning scenarios is a challenging and underexplored aspect, especially in practical non-stationary environments. Traditional approaches, such as grid searches with held-out validation data from all tasks, are unrealistic for building accurate lifelong learning systems. This paper aims to explore the role of hyperparameter selection in continual learning and the necessity of continually and automatically tuning them according to the complexity of the task at hand. Hence, we propose leveraging the nature of sequence task learning to improve Hyperparameter Optimization efficiency. By using the functional analysis of variance-based techniques, we identify the most crucial hyperparameters that have an impact on performance. We demonstrate empirically that this approach, agnostic to continual scenarios and strategies, allows us to speed up hyperparameters optimization continually across tasks and exhibit robustness even in the face of varying sequential task orders. We believe that our findings can contribute to the advancement of continual learning methodologies towards more efficient, robust and adaptable models for real-world applications.	ZWrG1YlEMY	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	semola24a	0	Adaptive Hyperparameter Optimization for Continual Learning Scenarios	1	14	1-14	1	false	Semola, Rudy and Hurtado, Julio and Lomonaco, Vincenzo and Bacciu, Davide	given family Rudy Semola given family Julio Hurtado given family Vincenzo Lomonaco given family Davide Bacciu	2024-08-14		Proceedings of the 1st ContinualAI Unconference, 2023	249	inproceedings	date-parts 2024 8 14	https://raw.githubusercontent.com/mlresearch/v249/main/assets/semola24a/semola24a.pdf