2024-08-09-han24a.md

title	booktitle	year	abstract	software	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Improving Assessment of Tutoring Practices using Retrieval-Augmented Generation	Proceedings of the 2024 AAAI Conference on Artificial Intelligence	2024	One-on-one tutoring is an effective instructional method for enhancing learning, yet its efficacy hinges on tutor competencies. Novice math tutors often prioritize content-specific guidance, neglecting aspects such as social-emotional learning. Social-emotional learning promotes equity and inclusion and nurtures relationships with students, which is crucial for holistic student development. Assessing the competencies of tutors accurately and efficiently can drive the development of tailored tutor training programs. However, evaluating novice tutor ability during real-time tutoring remains challenging as it typically requires experts-in-the-loop. To address this challenge, this study harnesses Generative Pre-trained Transformers (GPT), such as GPT-3.5 and GPT-4, to automatically assess tutors’ ability of using social-emotional tutoring strategies. Moreover, this study also reports on the financial dimensions and considerations of employing these models in real-time and at scale for automated assessment. Four prompting strategies were assessed: two basic Zero-shot prompt strategies, Tree of Thought prompting, and Retrieval-Augmented Generator (RAG) prompting. The results indicate that RAG prompting demonstrated the most accurate performance (assessed by the level of hallucination and correctness in the generated assessment texts) and the lowest financial costs. These findings inform the development of personalized tutor training interventions to enhance the educational effectiveness of tutored learning.	https://tutorevaluation.vercel.app	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	han24a	0	Improving Assessment of Tutoring Practices using Retrieval-Augmented Generation	66	76	66-76	66	false	Han, Zifei F. and Lin, Jionghao and Gurung, Ashish and Thomas, Danielle R and Chen, Eason and Borchers, Conrad and Gupta, Shivang and Koedinger, Kenneth R	given family Zifei F. Han given family Jionghao Lin given family Ashish Gurung given family Danielle R Thomas given family Eason Chen given family Conrad Borchers given family Shivang Gupta given family Kenneth R Koedinger	2024-08-09		Proceedings of the 2024 AAAI Conference on Artificial Intelligence	257	inproceedings	date-parts 2024 8 9	https://raw.githubusercontent.com/mlresearch/v257/main/assets/han24a/han24a.pdf