Cognitive diagnostic assessment in higher education: Using diagnostic classification models to support self-directed learning

Abstract

This dissertation is about formative assessment in higher education. The emerging use of technology in education enables administration of online formative assessments that can be used to automatically generate feedback that supports students’ learning processes. In current practice, this feedback often solely consists of information about correctness of responses and a proportion correct sore. A promising approach to obtain more detailed, diagnostic feedback is cognitive diagnostic assessment. A set of skills is defined and items are administered to measure those skills. A skill profile is estimated based on students’ responses to the items using a psychometric model, indicating for each skill whether a student masters it or not. Diagnostic classification models (DCMs) are suitable psychometric models to estimate these profiles, after which feedback can be provided to support self-directed learning. This dissertation shows how online formative assessment with DCMs can be designed and implemented for effective feedback in higher education. For this end, we studied (1) psychometric properties of DCMs, (2) the construction of assessments that provide a valid and reliable measurement of skills in statistics education, and (3) effects of feedback on self-directed learning processes. Although the mathematical foundation of DCMs is complex and designing assessments is an integrative process, the results of this dissertation show that the implementation is feasible and valuable.