Check Your Knowledge
RA Fisher and the P-value
The P-Value is the probability ... ?
a) that the null hypothesis is true.
b) that the alternative hypothesis is true.
c) of obtaining the observed difference in the outcome measure, or a larger one, given that no difference exists between treatments in the population.
d) that the observed difference in the outcome measure was due to random chance.
For a discussion of the above see What is a P value? BMJ, 2012.
Correct Answer is c.
A central concept in Biostatistics is the p-value. Generally attributed to RA Fisher, the p-value is widely used to test for assocations among population parameters. To calculate a p-value one must first state a hypothesis about the relationship between population parameters. This is called the null hypothesis. An example would be if we were interested in testing if the mean time to recovery differed between two surgical proceedures. A possible null hypothesis for this might be:
Ho: mu1 - mu2 = 0
where mu1 and mu2 are the two mean times to recovery in the population.
The p-value is the probability of getting a test statistic as large as, or larger than, the observed value when the null hypothesis is true ("c" in the above check-in). A small p-value would indicate that if the null hypothesis were true it would be unlikely to see our test result. Thus, either we have an unusual sample or our assumption about no difference is incorrect. As such, a "small" p-value is associated with increased doubt about the null hypothesis.
The p-value may be used as is and reported in journals. For decision-making, it is often used in consort with the level of significance, which is denoted by alpha. The level of significance is a fixed value, assigned as part of the experimental design, that is a the propability of rejecting the null hypothesis given the null hypothesis is true. Note that the level of significance is fixed in the design and the p-value is derived from the sample data. This is often set to 0.05 (5%). In this context an observed p-value less than the level of significance (p < alpha) leads to rejecting the null hypothesis.
For more information on p-values check out the following short articles at the British Medical Journal:
Try It Yourself
In a clinical trial researchers investigated potential differences in the proportion of individuals with a full recovery within 30 days for two different procedures. In the first group 40 out of 45 (89%) study participants had a full recovery comparted to 42 out of 50 (84%) in the group receiving the second procedure. What is the p-value for testing the null hypothesis that the two proportions are the same in the population?
Answer: A common test statistic to use for this comparison is the chi-square test with 1 degree of freedom. For the above sample the observed value of the test statistic is 0.478.
p = Prob( Chi-square > 0.478) = 0.49
If there is no difference in the recovery rates between the two proceedures in the population then 49% of the time we would expect to see a test statistic at least as large as 0.478. Note that this is not that unusual and using a standard 0.05 level of significance we would not reject the null hypothesis and are unable to claim a statistically significant difference in recovery between the two procedures.
Using the online app GraphPad.com:
Go online to GraphPad.com
Select "Statistical distributions and interpreting P values"
Select "Calculate P value from z, t, F, r or chi-square"
Enter 0.478 in the box for Chi Square and 1 in the box for DF
Select "Compute P"
data one; p=1-probchi(0.478,1); run;
proc print; run;