Clinical and practical importance vs statistical significance: Limitations of conventional statistical inference

Background: Decisions about support for therapies are often made using statistical inference in light of data. The dominant approach is null hypothesis significance testing (NHST). Applied correctly, NHST provides a procedure for making dichotomous decisions about zero-effect null hypotheses with known and controlled error rates. Type I and Type II error rates must be specified in advance, and the latter controlled by a priori sample size calculation. However, NHST does not provide the probability of hypotheses or the strength of support for hypotheses in light of data. The outcomes allow conclusions about the existence of non-zero effects, but provide no information about the likely size of true effects or their practical or clinical value. Content: Magnitude-based inference allows researchers to estimate the 'true' or large sample magnitude of effects with a specified likelihood, and how likely they are to exceed an effect magnitude of practical or clinical importance. This approach integrates the elements of subjective judgement that are central to clinical practice into a formal analysis of data, which facilitates more considered and enlightened interpretations of data, and avoids rejection of possibly highly beneficial therapies that are not statistically significant. Conclusions: Magnitude-based inference is gaining acceptance, but progress will be hastened if the shortcomings of NHST are understood.