JMIR Publications

ABSTRACT

When should a trial stop? Such a seemingly innocent question evokes concerns of type I and II errors among those who believe that certainty can be the product of uncertainty, and researchers who have been told that they need to carefully calculate sample sizes, take into consideration multiplicity, and not spend P values on interim analyses. However, the strive to dichotomize evidence into significant and non-significant has led to the basic driving force of science, namely uncertainty, to take a back seat. In this viewpoint we discuss that if testing the null hypothesis is the ultimate goal of science, then we need not worry about writing protocols, consider ethics, apply for funding, or run any experiments at all – all null hypotheses will be rejected at some point – everything has an effect. The job of science should be to unearth the uncertainties of the effects of treatments, not to test their difference from zero. We also show the fickleness of P values, how they may one day point to statistically significant results, and after a few more participants have been recruited, the once statistically significant effect suddenly disappears. We show plots which we hope intuitively highlights that all assessments of evidence will fluctuate over time. Finally, we discuss the remedy in the form of Bayesian methods, where uncertainty leads, and which allow for continuous decision making to stop or continue recruitment as new data from a trial is accumulated.