Towards Accessible Information Retrieval for Children With a Mild Intellectual Disability

Abstract

The ability to generate simple text is essential for Large Language Models (LLMs) to support individuals with mild intellectual disabilities (MID). This study compares GPT-4o and Llama3 on text simplification (TS) benchmarks, evaluating five metrics: FKGL, BLEU, METEOR, BERTScore, and SARI. We also conduct an LLM-based evaluation and compare all benchmarks with human judgments. Our findings show that GPT-4o consistently outperforms Llama3 across all benchmarks with statistical significance. However, the LLM-based evaluation slightly favors Llama3. Human judgments highlight that performance is more nuanced—while GPT-4o is preferred for structure, simplicity, and trust, Llama3 is valued for engagement and friendliness. In an experimental study on children with MID, we compare a chat system powered by GPT-4o to a control system using Google search in a self-exploration task. Results show that children with MID can understand GPT-4o well linguistically, but struggle to formulate inputs to the model. Additionally, we find that personalization and continuity play important roles in sustaining engagement. Our findings suggest that AI has the potential to support education and self-exploration for students with MID, but requires personalization for forming a bond.