Which statement about the relationship between p-values and alpha levels is true?

• A. P-values alone determine effect size.
• B. Alpha level determines sample size.
• C. P-values are independent of alpha.
• D. P-values compare to alpha to decide significance.

Answer: (D)

The key idea is how p-values are used with a pre-set significance threshold to decide if results are statistically meaningful. A p-value represents the probability, under the assumption that the null hypothesis is true, of obtaining results as extreme or more extreme than what was observed. The alpha level is chosen before the study and sets the maximum acceptable risk of a false positive, commonly 0.05. The decision rule is to compare the p-value to this alpha: if the p-value is at or below alpha, the result is deemed statistically significant and you reject the null; if it is above alpha, you do not reject the null. This is why that statement is true—the p-value is directly used in relation to alpha to determine significance.

P-values do not measure how large or important an effect is, so they don’t determine effect size. That’s a separate concept that requires effect-size metrics like Cohen’s d or the odds ratio. Alpha doesn’t set the sample size on its own; while changing alpha can influence the required sample size to achieve a desired power, the actual sample size is determined by the study design and power considerations. And p-values are not independent of alpha—they are interpreted in light of the pre-specified alpha threshold.