Does panel conditioning affect data quality in ego-centered social network questions?

Abstract

This article investigates the data quality of ego-centered social network modules in web surveys. It specifically examines whether these modules are subject to the effects of the repeated measurement of the same questions known as panel conditioning effects. Ego-centered social network modules are especially at risk of panel conditioning effects because many of the components in these modules are repetitive. Based on the theories of motivated underreporting and survey satisficing, we hypothesized that respondents reduce the length of the module by underreporting their network size and/or network density. To systematically test for panel conditioning effects, we experimentally varied the treatment frequency in a longitudinal study design, which included three panel waves. The results of our study showed that we generally obtained high quality data with relatively large reported network sizes and densities, low rates of item non-response, and low non-differentiation. In contrast to our expectations, the reported average network sizes were not smaller, and the network densities were not lower when respondents were asked to answer the same social network module multiple times. We found, however, patterns of individual change in network sizes that might be due to panel conditioning. Respondents with large network sizes in a panel wave reported smaller network sizes in the subsequent wave, while respondents with small network sizes reported larger network sizes in the subsequent wave. Respondents’ ability and motivation did not affect these results. Thus, we would like to encourage researchers to further explore the opportunity of implementing ego-centered social network modules in cross-sectional as well as longitudinal self-administered surveys, while being cautious that in longitudinal surveys the chance of panel conditioning effects may increase with the average network size and the response burden of the network module.