On defining P-values

The Fisherian prescription of reporting P-values as a summary of a result, as compared to the Neyman-Pearson system of acceptance or rejection of a null hypothesis, is more common in applied science. This popularity is largely due to the fact that the P-value provides a more complete, meaningful and useful evidence regarding the null hypothesis. Conventionally, P-values are defined in the context of one-sided alternatives, although there exist some ideas in the literature concerning two-sided alternatives; see e.g. [Gibbons, J.D., Pratt, J.W., 1975. P-values: Interpretation and methodology. American Statistician 24, 20-25; George, E.O., Mudholkar, G.S., 1990. P-values for two-sided tests. Biometrical journal 32, 747-751]. This note takes an axiomatic approach for defining P-values which involves at most ordering of the alternatives but is not restricted by their nature. It also involves a correspondence between a P-value and the associated level alpha test for each alpha. A P-value turns out to be valid if and only if the associated level alpha test is unbiased in the traditional sense for each alpha. Furthermore, it is shown that the resulting optimal tests agree with those given by the Neyman-Person framework when the ordering is stochastic. Thus, a theory based on optimal P-values parallels to the Neyman-Pearson theory and bridges the two approaches to testing of hypotheses. (C) 2009 Elsevier B.V. All rights reserved.