2023-07-02-geng23a.md

abstract	openreview	title	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
In dynamic discrete choice models, a commonly studied problem is estimating parameters of agent reward functions (also known as ’structural’ parameters) using agent behavioral data. This task is also known as inverse reinforcement learning. Maximum likelihood estimation for such models requires dynamic programming, which is limited by the curse of dimensionality [Bellman, 1957]. In this work, we present a novel algorithm that provides a data-driven method for selecting and aggregating states, which lowers the computational and sample complexity of estimation. Our method works in two stages. First, we estimate agent Qfunctions, and leverage them alongside a clustering algorithm to select a subset of states that are most pivotal for driving changes in Q-functions. Second, with these selected "aggregated" states, we conduct maximum likelihood estimation using a popular nested fixed-point algorithm [Rust, 1987]. The proposed two-stage approach mitigates the curse of dimensionality by reducing the problem dimension. Theoretically, we derive finite-sample bounds on the associated estimation error, which also characterize the trade-off of computational complexity, estimation error, and sample complexity. We demonstrate the empirical performance of the algorithm in two classic dynamic discrete choice estimation applications.	jWQ_LchjnL	A Data-Driven State Aggregation Approach for Dynamic Discrete Choice Models	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	geng23a	0	A Data-Driven State Aggregation Approach for Dynamic Discrete Choice Models	647	657	647-657	647	false	Geng, Sinong and Nassif, Houssam and Manzanares, Carlos A.	given family Sinong Geng given family Houssam Nassif given family Carlos A. Manzanares	2023-07-02		Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence	216	inproceedings	date-parts 2023 7 2	https://proceedings.mlr.press/v216/geng23a/geng23a.pdf	label link Supplementary PDF https://proceedings.mlr.press/v216/geng23a/geng23a-supp.pdf