Automated detection of referential features in schizophrenic speech using large language models

Abstract

Cross-linguistic studies have demonstrated that individuals with schizophrenia—particularly those exhibiting formal thought disorder (FTD)—show distinctive distributions of noun phrases (NPs) in spontaneous speech. NPs (e.g., the picture; a husband) serve to organize the referential structure of meaning. Extracting such referential NP features, however, has traditionally required manual annotations. In this study we applied state-of-the-art large language models (LLMs) to extract these features automatically, using an existing, manually annotated dataset, in which English-speaking participants described a comic strip: 30 individuals with schizophrenia (SZ) (15 with moderate or severe FTD (SZ + FTD), 15 with minimal or no FTD (SZ−FTD), 15 neurotypical controls (NC). We first show that LLM-based analyses replicate the findings based on manual annotation, particularly highlighting that definite NPs tied to prior discourse, markers of grammatical and cognitive complexity and narrative coherence, were underused in the SZ + FTD group. Secondly, we demonstrate that LLMs, especially when used with in-context (few-shot) learning, offer a promising avenue for the automatic extraction of referential features. These results show that a cross-linguistically validated and clinically important linguistic pattern of deviance is accessible to automatized assessment with NLP.