Visual design effects in a cross-national context: Are Indian and US respondents differently affected by the number of text boxes?

Abstract

Research indicates that multiple text boxes for list-style open-ended questions and probes yield a better response quality than a single box. However, these findings are based on single-country studies from Europe and the US. In the current cross-national study, we evaluate the stability of visual design effects across two countries (US and India), and three languages (American-English, Indian-English, Hindi). In our web survey, we implemented a between-subject design comparing hard nonresponse, soft nonresponse, mismatches, and the number of themes across 1 and 5 answer box designs at two specific probes. The survey was fielded in the US (N=415) and India (N=537) between December 2022 and May 2023. In India, the survey was available in English (N=327) and Hindi (N=247). We found that in India and the US, the number of substantive themes significantly increased with multiple answer boxes. For one of the probes, multiple answer boxes also significantly reduced soft nonresponse and mismatches. Only for hard nonresponse were significant differences between languages found.