Sources of Inconsistency in the Measurement of Religious Affiliation: Evidence from a Survey Experiment and Cognitive Interviews

Abstract

Research has argued that estimates of the percentage of Americans without a religious affiliation may be influenced by a measurement artifact caused by the poor reliability of conventional survey questions. Using a question-order experiment and cognitive interviews, we assess religious affiliation measures like those commonly used on surveys. A filter question and a full question presenting a list of religious affiliations as response options are asked in a random order. Findings suggest that nominally religious respondents have higher odds of reporting an affiliation to the full question than the filter. They describe nominal or residual religious affiliations, highlighting the role of family connections and religious pasts in their affiliation reports. Conversely, highly religious and nonreligious respondents are, respectively, only slightly and not at all influenced by question characteristics. Findings suggest that using a filter question before a full question avoids this source of inconsistency and yields better data quality.