Methodical advances in reproducibility research: A proof of concept qualitative comparative analysis of reproducing animal data in humans

Abstract

Background: While the term reproducibility crisis mainly reflects reproducibility of experiments between laboratories, reproducibility between species also remains problematic. We previously summarised the published reproducibility between animal and human studies; i.e. the translational success rates, which varied from 0% to 100%. Based on analyses of individual factors, we could not predict reproducibility. Several potential analyses can assess effect of combinations of predictors on an outcome. Regression analysis (RGA) is common, but not ideal to analyse multiple interactions and specific configurations (≈ combinations) of variables, which could be highly relevant to reproducibility. Qualitative comparative analysis (QCA) is based on set theory and Boolean algebra, and was successfully used in other fields. We reanalysed the data from our preceding review with QCA. Results: This QCA resulted in the following preliminary formula for successful translation: ∼Old*∼Intervention*∼Large*MultSpec*Quantitative Which means that within the analysed dataset, the combination of relative recency (∼ means not; >1999), analyses at event or study level (not at intervention level), n < 75, inclusion of more than one species and quantitative (instead of binary) analyses always resulted in successful translation (>85%). Other combinations of factors showed less consistent or negative results. An RGA on the same data did not identify any of the included variables as significant contributors. Conclusions: While these data were not collected with the QCA in mind, they illustrate that the approach is viable and relevant for this research field. The QCA seems a highly promising approach to furthering our knowledge on between-species reproducibility.