Beyond numbers: the merit of routine student evaluation for starting academic teacher development

Abstract

Despite its controversy, Student Evaluation of Teaching (SET) remains a potentially central tool to inform academic teaching skills development. This study evaluated a SET procedure that was developed by starting academic teachers seeking to inform and monitor their teaching skill development. The procedure involves administering a SET twice per year, generating scores and open-ended feedback that are discussed with supervisors to aid interpretation and use. Mixed methods were used to evaluate whether the procedure serves its purpose. Analyses of evaluations from 5050 students for 42 starting Psychology teachers provided some support that aggregating the evaluations per measurement occasion can offer teachers reliable and structurally valid feedback. While data visualisations showed changes in specific teaching skills over time for certain teachers, overall, there was little significant development in scores across teachers, with notable between-teacher differences in trajectories. Interviews with 16 teachers indicated a general perception that the SET procedure was insightful and useful for their development, but individual experiences varied. We conclude that contextualising scores with visualisations of development over time and open-ended feedback can result in a meaningful tool for teacher development. A more qualitative and individualised approach is recommended when using SET in higher education.