Assessment of balance in propensity score analysis in the medical literature: A systematic review

Abstract

Background: Assessing balance on co-variate distributions between treatment groups with a given propensity score (PS) is a crucial step in PS analysis. Several methodological papers comparing different balance measures have been published in the last decade. However, the current practice on measuring and reporting of balance in PS analysis is not well documented. Objectives: To investigate the current practice of PS analysis with emphasis on assessment of balance of confounders. Methods: A PubMed search was performed to identify studies using propensity score methods published December 2011-May 2012. We extracted information on the applied PS method, whether and how balance on confounders was checked, if PS matching was applied (including the algorithm), and information on the initial sample size of the number of matched pairs. Results: In total, 261 studies that employed PS methods were found. Balance of confounders between treatment groups was checked and reported in 149 (57.1%) of the articles. p-Values from hypothesis testing was the most commonly used statistical tool to report balance (110 studies, 73.8%). The standardized difference and graphical displays of balance were used in 42 (28.2%) and nine (6%) articles, respectively. The most commonly used approach to control for confounding using PS was matching on the PS (67%), followed by co-variate adjustment for the PS (22.2%), PS stratification (14.6%) and inverse probability weighting (7.3%). Balance was most often checked in articles using PS matching and inverse probability weighting: 68.6% and 73.6% respectively. Conclusions: When using PS methods, assessment of balance on confounders between treatment groups is not often conducted or reported. Appropriate methods such as standardized difference should be used to quantify and report balance.