Propensity score matching and unmeasured covariate imbalance: A simulation study

Abstract

Background: Selecting covariates for adjustment or inclusion in propensity score (PS) analysis is a trade-off between reducing confounding bias and a risk of amplifying residual bias by unmeasured confounders. Objectives: To assess the covariate balancing properties of PS matching with respect to unmeasured covariates and its impact on bias. Methods: Simulation studies were conducted in binary covariates, treatment and outcome data. In different scenarios, instrumental variables (IV, i.e., variables related to treatment but not to the outcome or other covariates), risk factors (variables related only to the outcome), unmeasured covariates, and confounders with various associations among each other were considered.Treatment effects estimates (risk ratio) were derived after PS matching using Poisson models; balance for each covariate was checked before and after matching using the absolute standardized difference.The choice of covariates for the PS model was compared with respect to bias in the treatment-outcome relation and balance of (unobserved) covariates. Results: PS matching improved balance of measured covariates included in the PS model but exacerbated the imbalance of the unmeasured covariate that was unrelated to measured covariates compared to the full unmatched sample. Inclusion of instrumental variables, independent of unmeasured covariates, exacerbated the imbalance in unmeasured covariates and amplified the residual bias. However, including instrumental variables that were associated with unmeasured covariates improved the balance of unmeasured covariates and reduced bias. When the PS model included variables related to the outcome, exclusion of instrumental variables that were related to unmeasured covariates exacerbated the balance of unmeasured covariates and increased the bias. Conclusions: In choosing covariates for a PS model, the pattern of association among covariates has substantial impact on other covariates' balance and the bias of the treatment effect.Investigators should not rely only on covariate association with treatment or outcome but should take into account possible associations among covariates and explore the balance of other covariates after PS matching.