Scaling goal-setting interventions in higher education using a conversational agent: Examining the effectiveness of guidance and adaptive feedback

Abstract

Goal setting is the first and driving stage of the self-regulated learning cycle. Studies have shown that supporting goal setting is an effective means of improving academic performance among higher education students. However, doing so can be complex and resource intensive. In this study, a goal-setting conversational agent was designed and deployed to support higher education students in setting academic goals. Across 5-weeks, we tested the effects of goal-setting prompts (guided vs. unguided) and adaptive feedback (with vs. without) when delivered via a goal-setting conversational agent. We explored the effects of these supports (i.e., guidance and feedback) on students' 1) goal quality and 2) goal attainment. Findings showed that guidance and feedback combined had the largest positive effect on goal quality. They also revealed that guidance alone produced initially high-quality goals which decreased in quality overtime, whereas feedback had a delayed but cumulative effect on quality across multiple goal setting iterations. However, neither guidance nor feedback had significant effects on goal attainment, and there was no significant relationship between goal quality and attainment. This study provides insights into how a goal-setting conversational agent and adaptive feedback can be used to support the academic goal setting process for higher education students.