Chatting with Code: Exploring LLMs as Learning Partners in Programming Education

Abstract

With LLM-based applications now widely accessible, students increasingly leverage them to support their studies, especially in programming education. From completing specific tasks to managing their study routines, students can use LLMs to self-regulate their learning. However, while LLMs have the potential to support students and improve educational outcomes, they could hamper learning. This exploratory case study seeks to understand how students taking programming courses interact with LLM-based applications. We analyzed and clustered the content of student prompts (N = 364) and coded the prompts for self-regulated learning (SRL) strategies. We identified seven distinct clusters of prompts that were characterized by student task (e.g., debugging, seeking help) and prompt topic (e.g., mathematical models, security). Students primarily relied on LLMs for elaboration and help-seeking, while SRL strategies like effort regulation, critical thinking, and organization were used less frequently. Overreliance on LLMs for immediate support may hinder the development of deeper cognitive strategies and impede learning, suggesting a need for student support.