Check Your Knowledge
RA Fisher and the Pvalue
en.wikipedia.org/wiki/Ronald_Fisher
Check in
The PValue is the probability ... ?
a) that the null hypothesis is true.
b) that the alternative hypothesis is true.
c) of obtaining the observed difference in the outcome measure, or a larger one, given that no difference exists between treatments in the population.
d) that the observed difference in the outcome measure was due to random chance.
For a discussion of the above see What is a P value? BMJ, 2012.
Correct Answer is c.
Background\Overview
A central concept in Biostatistics is the pvalue. Generally attributed to RA Fisher, the pvalue is widely used to test for assocations among population parameters. To calculate a pvalue one must first state a hypothesis about the relationship between population parameters. This is called the null hypothesis. An example would be if we were interested in testing if the mean time to recovery differed between two surgical proceedures. A possible null hypothesis for this might be:
Ho: mu1  mu2 = 0
where mu1 and mu2 are the two mean times to recovery in the population.
The pvalue is the probability of getting a test statistic as large as, or larger than, the observed value when the null hypothesis is true ("c" in the above checkin). A small pvalue would indicate that if the null hypothesis were true it would be unlikely to see our test result. Thus, either we have an unusual sample or our assumption about no difference is incorrect. As such, a "small" pvalue is associated with increased doubt about the null hypothesis.
The pvalue may be used as is and reported in journals. For decisionmaking, it is often used in consort with the level of significance, which is denoted by alpha. The level of significance is a fixed value, assigned as part of the experimental design, that is a the propability of rejecting the null hypothesis given the null hypothesis is true. Note that the level of significance is fixed in the design and the pvalue is derived from the sample data. This is often set to 0.05 (5%). In this context an observed pvalue less than the level of significance (p < alpha) leads to rejecting the null hypothesis.
For more information on pvalues check out the following short articles at the British Medical Journal:
Try It Yourself
In a clinical trial researchers investigated potential differences in the proportion of individuals with a full recovery within 30 days for two different procedures. In the first group 40 out of 45 (89%) study participants had a full recovery comparted to 42 out of 50 (84%) in the group receiving the second procedure. What is the pvalue for testing the null hypothesis that the two proportions are the same in the population?
Answer: A common test statistic to use for this comparison is the chisquare test with 1 degree of freedom. For the above sample the observed value of the test statistic is 0.478.
p = Prob( Chisquare > 0.478) = 0.49
If there is no difference in the recovery rates between the two proceedures in the population then 49% of the time we would expect to see a test statistic at least as large as 0.478. Note that this is not that unusual and using a standard 0.05 level of significance we would not reject the null hypothesis and are unable to claim a statistically significant difference in recovery between the two procedures.
Using the online app GraphPad.com:

Go online to GraphPad.com

Select "Statistical distributions and interpreting P values"

Select "Calculate P value from z, t, F, r or chisquare"

Enter 0.478 in the box for Chi Square and 1 in the box for DF

Select "Compute P"
In SAS
data one; p=1probchi(0.478,1); run;
proc print; run;