Detect, Explain, Act: How Teachers Trust and Use an Explainable Real-Time Monitoring Dashboard to Detect Student Outliers in Class

Abstract

Adaptive learning platforms can personalise learning in classrooms, yet teachers need tools to monitor student progress and rapidly identify students who need additional attention. Monitoring dashboards that detect outlier students can fulfil this need, but insufficient explanation of the detection logic may undermine trust and lead to inappropriate use. To address this challenge, we iteratively designed and implemented a real-time monitoring dashboard with model-centric and data-centric explanations, informed by teacher input. We then conducted a counterbalanced within-group experiment with follow-up interviews to study how 11 teachers used the dashboard in a real classroom and how they engaged with the explanations to calibrate their trust. We found that teachers successfully integrated the dashboard into their classes, and that their trust was shaped by many dispositional, situational, and learnt factors. Crucially, the data-centric explanations enabled teachers to validate the accuracy of outlier predictions, check alignment with their prior knowledge of students, and identify suitable interventions. Based on these findings, we present design recommendations for explainable outlier detection systems in education.