Signal detection theory as a tool for successful student selection

Abstract

Prediction accuracy of academic achievement for admission purposes requires adequate sensitivity and specificity of admission tools, yet the available information on the validity and predictive power of admission tools is largely based on studies using correlational and regression statistics. The goal of this study was to explore signal detection theory as a tool to extend the available information; signal detection theory allows for comparisons of selection outcomes on both group and individual levels and the development of tailor-made criteria for specific programmes and admission goals. We investigated who would or would not have been admitted applying specific criteria for each of three common admission tools, how many admitted students would fail and how many applicants who would have been successful would be rejected. Both comparisons at an individual level and the receiver operating characteristic curves at a group level revealed that scores obtained in a programme-specific matching programme and non-cognitive factors appear more valuable than regression statistics suggest when it comes to admitting applicants who will become successful students. Signal detection theory allows not only for admission-goal-specific and programme-specific fine-tuning of the content of admission tools, it also informs about the effects of criteria and thus allows the setting of criteria.