Microsoft Research introduces StudentSim to evaluate AI tutors before real-world testing

Microsoft Research describes StudentSim as a research system designed to evaluate and improve AI tutoring tools before they are tested with real learners. The system is not a product, is not deployed in classrooms, and is not intended to replace teachers.

StudentSim trains personalized AI student simulators using publicly available, de-identified learner records, with one simulator adapted to each student. This approach aims to approximate student feedback during the research and development process, addressing the slow and costly nature of human studies.

The research evaluates simulators along two education-grounded dimensions: behavioral fidelity, which measures whether a simulator matches a specific student’s recorded responses, and guidance responsiveness, which assesses whether the simulator updates its response after receiving tutor guidance.

StudentSim is evaluated using StudentSimEval, a benchmark spanning chess, second-language English writing, and mathematics. Across these domains, StudentSim outperforms leading baselines and provides a useful feedback signal for training AI tutors.

The research emphasizes that StudentSim is intended solely for research-stage evaluation and improvement, not as a replacement for classroom learning or real learner testing. All datasets used are publicly available, de-identified, and released for research use, with no personally identifying information included.