A visual perceptual perspective on gaze in social robotics

Abstract

Where and when should a robot look during interaction with humans? Intuitively, human-human interaction serves as the blueprint. In such interactions, gaze plays a central role, both in visually guiding action and in signaling communication. Yet, most social robotics research still relies on the oversimplified eye-mind assumption, equating gaze with visual attention, and often implements gaze behaviors without accounting for perceptual limits or individual variability. We identify three core challenges at the intersection of psychology, visual perception, and robotics: (1) disentangling the dual functions of gaze, (2) understanding the visual perceptual mechanisms underlying gaze perception, and (3) adapting to individual, cultural, and clinical variability in gaze behavior. By examining these challenges in both robot interpretation and production of gaze, we argue that social robotics benefits from insights into human gaze and provides a powerful testbed for theory development, enabling controlled, interactive experiments that deepen our understanding of social attention. We conclude by conceptualizing social gaze as a soft-constraints inference and production process under uncertainty, linking visual perceptual mechanisms and social meaning.