Unrestricted weighted least squares represent medical research better than random effects in 67,308 Cochrane meta-analyses

Abstract

OBJECTIVES: To evaluate how well meta-analysis mean estimators represent reported medical research and establish which meta-analysis method is better using widely accepted model selection measures: Akaike information criterion (AIC) and Bayesian information criterion (BIC). STUDY DESIGN AND SETTING: We compiled 67,308 meta-analyses from the Cochrane Database of Systematic Reviews (CDSR) published between 1997 and 2020, collectively encompassing nearly 600,000 medical findings. We compared unrestricted weighted least squares (UWLS) vs. random effects (RE); fixed effect was also secondarily considered. RESULTS: The probability that a randomly selected systematic review from the CDSR would favor UWLS over RE is 79.4% (95% confidence interval [CI95%]: 79.1; 79.7). The odds ratio that a Cochrane systematic review would substantially favor UWLS over RE is 9.33 (CI95%: 8.94; 9.73) using the conventional criterion that a difference in AIC (or BIC) of two or larger represents a 'substantial' improvement. UWLS's advantage over RE is most prominent in the presence of low heterogeneity. However, UWLS also has a notable advantage in high heterogeneity research, across different sizes of meta-analyses and types of outcomes. CONCLUSION: UWLS frequently dominates RE in medical research, often substantially. Thus, the UWLS should be reported routinely in the meta-analysis of clinical trials.