Improving Text Readability to Support Student Comprehension and Learning: An LLM-Powered Approach

Abstract

Reading proficiency is predictive of academic success, yet many students, especially those with diverse learning needs, struggle with complex academic texts. Existing support tools often fail to adequately address challenges related to the complexity of text vocabulary and grammar. However, large language models (LLMs) might be able to meet this need. We compared the effectiveness of two prompting strategies for simplifying academic texts (N = 2,000): one that used plain-text instructions and another that incorporated a readability metric. The Metric-Guided Prompt demonstrated a significant reduction in text complexity as measured by the Flesch-Kincaid Grade Level. Following this intrinsic evaluation, we conducted a between-subjects study with 37 students to determine whether there were differences in learner perceptions of the texts and their learning gains, based on the source of the information provided (i.e., the original and simplified texts). The results of both the intrinsic evaluation and the user study indicate that the Metric-Guided Prompt improves text readability without hindering learning. These findings underscore the potential for appropriately prompted LLMs to foster academic success for diverse learners by improving information access and supporting comprehension.