In-Depth Data Exploration for Reliable Learning Curve Analysis: Insights from a Secondary School Python Course

Abstract

The increased use of digital educational technologies has led to an increased availability of educational data. The field of Educational Data Mining (EDM) uses this data to perform various analyses. To successfully apply methodologies from EDM, the data must be of good quality. Looking at the data in detail before doing any EDM analyses gives insights that contribute to a reliable interpretation of the results of EDM. While this is relevant for all EDM methodologies, this paper focuses only on using student data for drawing learning curves. In this experience report, we look at the question: Which analyses are useful to assess whether data collected in a digital learning platform can be used for learning curve analysis? Such data are suitable if they are not biased by the collection method. We use real life data from two iterations of a Dutch secondary school course, Python Programming for Beginners. We show how platform features, such as fine-grained links between hierarchical learn ing goals and activities, and diverse assessment methods (self-, automated, and teacher grading), affect data quality. Our findings highlight how systematic data exploration adds crucial context to learning curves, which can also be beneficial for course designers. Finally, we propose guidelines for embedding this step into EDM practices.