Statistical power, effect size and animal welfare: recommendations for good practice

Despite the particular relevance of statistical power to animal welfare studies, we noticed an apparent lack of sufficient information reported in papers published in Animal Welfare to facilitate post hoc calculation of statistical power for use in meta-analyses. We therefore conducted a survey of all papers published in Animal Welfare in 2009 to assess compliance with relevant instructions to authors, the level of statistical detail reported and the interpretation of results regarded as statistically non-significant. In general, we found good levels of compliance with the instructions to authors except in relation to the level of detail reported for the results of each test. Although not requested in the instructions to authors, exact P-values were reported in just over half of the tests but effect size was not explicitly reported for any test, there was no reporting of a priori statistical analyses to determine sample size and there was no formal assessment of non-significant results in relation to type II errors. As a first stage to addressing this we recommend more reporting of a priori power analyses, more comprehensive reporting of the results of statistical analysis and the explicit consideration of possible statistical power issues when interpreting P-values. We also advocate the calculation of effect sizes and their confidence intervals and a greater emphasis on the interpretation of the biological significance of results rather than just their statistical significance. This will enhance the efforts that are currently being made to comply with the 3Rs, particularly the principle of reduction.