On the Road to Equivalence: Expanding Bayesian Measurement Invariance Testing and Bridging Western and Non-Western Contexts in Web Probing

Abstract

This dissertation examines methodological challenges in cross-cultural survey research, with a particular focus on establishing equivalence when comparing latent constructs across Western and non-Western populations. Meaningful comparisons of latent variables such as well-being, health, or environmental attitudes require that survey items are interpreted similarly across groups and that measurement properties are equivalent. Because full measurement invariance is often difficult to achieve, the dissertation explores both quantitative and qualitative approaches to improving cross-cultural comparability. The first part of the dissertation investigates Bayesian Approximate Measurement Invariance (BAMI) as an alternative to traditional Multi-Group Confirmatory Factor Analysis when scalar invariance cannot be established. Using data on willingness to sacrifice for the environment (WTS) across 30 countries, the analyses demonstrate that BAMI can facilitate cross-national comparisons by allowing small deviations from strict equality constraints. However, the findings also show that the choice of prior specifications substantially influences the results. Different priors can produce different country rankings even when model fit is satisfactory, highlighting the importance of theoretically and empirically justified prior selection. The second part of the dissertation focuses on web probing (WP), a survey technique that combines closed-ended questions with open-ended follow-up probes to gain insight into respondents’ interpretations and reasoning. To extend the application of web probing beyond Western contexts, a survey was conducted among respondents in the United States and India, with the Indian sample available in both English and Hindi. The analyses reveal significant differences in response times and survey break-off rates between countries. However, most differences in response quality were associated with the type of probe used rather than with language or cultural background. These findings suggest that improving probe design may be more effective than developing culture-specific adaptations. The dissertation also examines the role of visual design in web probing. An experimental study compared a single large response box with five smaller response boxes for specific probes. Across all three language groups, the five-box design resulted in fewer unusable responses and a greater number of examples or themes mentioned by respondents. This finding indicates that visual presentation can substantially improve response quality and may help overcome some culturally driven differences in response behavior. Overall, the dissertation demonstrates that achieving comparability in cross-cultural survey research requires both rigorous statistical modeling and careful questionnaire design. It contributes to the literature by showing that prior specification is critical in Bayesian measurement invariance analyses, that probe type is more influential than cultural context in generating useful web probing responses, and that effective visual design significantly enhances response quality across cultures. The dissertation further identifies promising directions for future research, including integrating web probing with BAMI, developing universally effective probe designs, automating qualitative coding, and addressing the growing challenge of AI-generated survey responses