Possible Solution to Publication Bias Through Bayesian Statistics, Including Proper Null Hypothesis Testing

Abstract

The present paper argues that an important cause of publication bias resides in traditional frequentist statistics forcing binary decisions. An alternative approach through Bayesian statistics provides various degrees of support for any hypothesis allowing balanced decisions and proper null hypothesis testing, which may prevent publication bias. Testing a null hypothesis becomes increasingly relevant in mediated communication and virtual environments. To illustrate our arguments, we re-analyzed three data sets of previously published data --media violence effects, mediated communication, and visuospatial abilities across genders. Results are discussed in view of possible Bayesian interpretations, which are more open to a content-related argumentation of varying levels of support. Finally, we discuss potential pitfalls of a Bayesian approach such as BF-hacking (cf., “God would love a Bayes Factor of 3.01 nearly as much as a BF of 2.99”). Especially when BF values are small, replication studies and Bayesian updating are still necessary to draw conclusions.