A randomized trial of a new antihypertensive versus placebo reports a difference in the primary endpoint with p=0.03 (prespecified α=0.05). Which statement correctly interprets this p-value?

If no true difference exists, results this extreme would occur 3% of the time

The difference is statistically significant and therefore clinically important

There is a 3% probability that the null hypothesis of no difference is true

There is a 97% probability that the new drug is truly more effective than placebo

A case-control study of 600 patients with hip fracture and 1,200 matched controls reports an adjusted odds ratio of 1.40 (95% CI 1.15–1.70) for current proton pump inhibitor use. Which interpretation is most appropriate?

Current PPI use is associated with higher odds of hip fracture

PPI use causes a 40% absolute increase in hip fracture risk

The result proves that PPIs cause hip fractures in older adults

No conclusion is possible because no p-value is reported

A randomized trial of 60 patients finds no significant difference in pain scores between a new analgesic and an established one (p=0.21). The authors conclude that the two drugs are equally effective. Which critique is most appropriate?

The study may be underpowered and cannot show equivalence

A type I error is the most likely explanation of the result

A one-tailed test would make the equivalence claim valid

The p-value proves that the null hypothesis is true

A 12-week COPD trial reports that a new inhaler improved trough FEV1 by 40 mL more than placebo (95% CI 25–55 mL; p<0.001). The minimal clinically important difference for trough FEV1 is about 100 mL. Which interpretation is most appropriate?

Significant but unlikely to be clinically important

Clinically important because the p-value is below 0.001

Not statistically significant because it is below the MCID

Clinically important because the CI excludes zero

A randomized trial reports that a new heart failure drug changed the relative risk of hospitalization to 0.85 (95% CI 0.72–1.01) compared with placebo. Which interpretation is most appropriate?

No statistically significant difference when α is 0.05

A significant 15% relative reduction in hospitalization

Proof that the drug and placebo are equally effective

A significant reduction because the CI excludes zero

A randomized trial of a new antiplatelet drug showed no difference in the primary endpoint (p=0.42). Among 14 post hoc subgroup analyses, women younger than 65 showed a significant benefit (p=0.04). Which interpretation is most appropriate?

Hypothesis-generating only; likely chance from multiplicity

Evidence that the primary analysis was underpowered overall

Proof that the drug benefits women younger than 65 years

Justification to recommend the drug for all women in practice

BCPS Biostatistics: P Values and Confidence Intervals

Q: Is p = 0.05 a hard line?

It is a convention, fixed as α before the study. Results at p = 0.049 and p = 0.051 carry almost the same evidence, and the interval shows that more honestly than the label does.

Q: Can a confidence interval and a p value disagree?

For the same test and α they agree: a two-sided 95% CI that excludes the null corresponds to p below 0.05. An apparent mismatch in a paper usually means two different analyses or rounding.

Q: What do trial reports have to show?

The CONSORT 2025 statement asks reports to give, for each outcome, the estimated effect with a measure of its precision, such as a 95% confidence interval, so a bare p value falls short.

1Six terms, one sentence each

p value: The probability of results at least as extreme as those observed, calculated on the assumption that the null hypothesis holds.
Alpha (α): The false-positive rate accepted before the study, usually 0.05; a p value below it is labeled statistically significant.
95% confidence interval (CI): The range of effect sizes compatible with the data at that level of confidence; it narrows as the sample grows.
Type I error: Declaring a difference that does not exist; its probability is α.
Type II error and power: Missing a difference that does exist, with probability β; power is 1 − β, commonly planned at 80% or 90%.
Minimal clinically important difference (MCID): The smallest change patients would notice or value, set from outside the trial; the yardstick for clinical significance.

2Ratio or difference: find the null first

≠ marks rows where the two differ.

Where the line of no effect sits for each kind of measure.
Point	Ratio measures	Difference measures
Differs: Examples	Relative risk, odds ratio, hazard ratio	Absolute risk difference, mean difference
Differs: Line of no effect	1	0
Differs: Significant at α = 0.05 when the 95% CI	excludes 1	excludes 0
Differs: Illustration	Hazard ratio 0.80 (95% CI 0.66–0.97): significant	Mean difference 3 points (95% CI −1 to 7): not significant
Differs: Shape of the interval	Symmetric on a log scale, so lopsided around the estimate	Symmetric around the estimate
Differs: Typical source	Case-control studies give odds ratios; trials and cohorts give relative risks; time-to-event analyses give hazard ratios	Trials and cohorts

The illustrations are invented numbers, not results from a specific trial.

3Four places an interval can land

A difference measure with a known MCID. The same reading works for ratios, with 1 as the null.
Where the 95% CI sits	Statistically significant?	Clinically important?
Entirely beyond the MCID	Yes	Yes: even the smallest plausible effect matters
Excludes 0 but stays short of the MCID	Yes	Probably not: every plausible value falls below the yardstick
Excludes 0 and straddles the MCID	Yes	Uncertain: some plausible values matter, some do not
Includes 0	No	Unknown: a wide interval can hide an effect worth having

4Why significance is a separate question from importance

A p value shrinks as a sample grows. A very large trial can make a trivial difference highly significant, and a small one can fail to detect a large difference. Keeping the two kinds of significance apart is the central habit of biostatistics and evidence-based practice.

4.1A miss is not a match

A nonsignificant result means the study did not detect a difference. If its interval is wide enough to include both no effect and a meaningful one, the study simply could not tell. Showing two treatments are interchangeable takes a design built for it, with a margin set in advance.

4.2From significance to patients

Once an absolute difference is significant and large enough to matter, convert it into people: the reciprocal of the absolute risk reduction is the number needed to treat, which a relative measure cannot give you without the baseline risk.

5Reading the interval

Name the null value for the measure in front of you before you judge its interval.

Study cases written by our editors, not BPS items. The real BCPS exam is computer-based multiple choice, answered in order with no going back.

Case 1 of 6 · 0 answered

How case rounds work

Pick an option to see the verdict and a note on every option. Keys while the round has focus: 1–5 or A–E answer, ← → move between cases. Nothing is saved or sent anywhere.

Case 01
A randomized trial of a new antihypertensive versus placebo reports a difference in the primary endpoint with p=0.03 (prespecified α=0.05). Which statement correctly interprets this p-value?
Pick an option: the verdict and a note on every option open here.
Best answer: D.
A p-value is the probability of observing results at least this extreme if the null hypothesis were true, so p=0.03 is below α and the result is statistically significant. It is not the probability that the null hypothesis is true or that the drug works; those are the most common misreadings. Statistical significance also says nothing about whether the size of the effect is clinically meaningful.
A
Statistical significance does not establish clinical importance; that depends on the size of the effect.
B
A p-value is computed assuming the null is true, so it cannot give the probability that the null is true.
C
The p-value is not the probability that the drug works; that misreading confuses p with a posterior probability.
D
Correct. A p-value is the chance of results at least this extreme if no true difference exists, and 0.03 is below the 0.05 α.
Case 02
A case-control study of 600 patients with hip fracture and 1,200 matched controls reports an adjusted odds ratio of 1.40 (95% CI 1.15–1.70) for current proton pump inhibitor use. Which interpretation is most appropriate?
Pick an option: the verdict and a note on every option open here.
Best answer: A.
Because the 95% CI for the odds ratio excludes 1, the association is statistically significant, but a case-control design shows association, not causation. An OR of 1.40 means 40% higher odds, a relative measure, not a 40% absolute increase in risk. A confidence interval that excludes the null value of 1 already shows significance at p<0.05, so no p-value is needed.
A
Correct. An OR of 1.40 with a CI excluding 1 shows a significant association, and a case-control design cannot show causation.
B
An OR is a relative measure; 1.40 means 40% higher odds, not a 40% absolute rise in risk.
C
Case-control studies show association, not cause, and confounding may explain the link.
D
A 95% CI that excludes 1 already shows significance at p below 0.05.
Case 03
A randomized trial of 60 patients finds no significant difference in pain scores between a new analgesic and an established one (p=0.21). The authors conclude that the two drugs are equally effective. Which critique is most appropriate?
Pick an option: the verdict and a note on every option open here.
Best answer: C.
A nonsignificant superiority test means only that a difference was not detected; with 60 patients the study may lack power (type II error risk), and equivalence requires a prespecified equivalence or noninferiority design with a margin. A type I error is a false-positive finding, which cannot occur when the result is not significant. A p-value never proves the null hypothesis, and changing to a one-tailed test does not create an equivalence analysis.
A
A type I error is a false positive, which cannot explain a nonsignificant result.
B
A one-tailed superiority test is still not an equivalence design and has no equivalence margin.
C
Correct. With 60 patients the study may lack power, and equivalence needs a prespecified margin and design.
D
A p-value can never prove the null hypothesis; absence of evidence is not evidence of equivalence.
Case 04
A 12-week COPD trial reports that a new inhaler improved trough FEV1 by 40 mL more than placebo (95% CI 25–55 mL; p<0.001). The minimal clinically important difference for trough FEV1 is about 100 mL. Which interpretation is most appropriate?
Pick an option: the verdict and a note on every option open here.
Best answer: C.
The CI excludes 0, so the difference is statistically significant, but even its upper bound (55 mL) is well below the 100-mL MCID, so the benefit is unlikely to matter to patients. A very small p-value reflects precision (often a large sample), not the size of the effect. The MCID is a clinical threshold and does not determine statistical significance.
A
A tiny p-value reflects precision, often a large sample, not a large effect.
B
The MCID is a clinical threshold and does not determine statistical significance.
C
Correct. The CI excludes 0, so the effect is significant, but even its upper bound of 55 mL is well short of the 100-mL MCID.
D
Excluding zero shows significance, not that the effect matters; the whole interval sits below the MCID.
Case 05
A randomized trial reports that a new heart failure drug changed the relative risk of hospitalization to 0.85 (95% CI 0.72–1.01) compared with placebo. Which interpretation is most appropriate?
Pick an option: the verdict and a note on every option open here.
Best answer: A.
For a ratio, the null value is 1, and this 95% CI (0.72–1.01) includes 1, so the difference is not statistically significant at α=0.05. Reading the point estimate as a significant 15% reduction ignores the interval, and zero is the null value only for differences, not ratios. A nonsignificant result does not prove equivalence; the interval is still compatible with a meaningful benefit.
A
Correct. The null value for a ratio is 1, and the interval 0.72–1.01 includes it, so the result is not significant.
B
The point estimate suggests 15%, but the interval crosses 1, so the reduction is not statistically significant.
C
A nonsignificant result does not prove equivalence; the interval still allows a meaningful benefit.
D
Zero is the null value for differences, not ratios; for a relative risk the line is 1.
Case 06
A randomized trial of a new antiplatelet drug showed no difference in the primary endpoint (p=0.42). Among 14 post hoc subgroup analyses, women younger than 65 showed a significant benefit (p=0.04). Which interpretation is most appropriate?
Pick an option: the verdict and a note on every option open here.
Best answer: D.
With 14 unplanned comparisons at α=0.05, about one ‘significant’ result is expected by chance alone, so a post hoc subgroup finding in a negative trial can only generate a hypothesis for a new trial. It does not prove benefit in that subgroup, and extending it to all women goes beyond the data. A single subgroup with p=0.04 says nothing about the power of the overall comparison.
A
One subgroup at p=0.04 says nothing about the power of the overall comparison.
B
A post hoc finding among 14 tests cannot prove benefit in that subgroup.
C
Extending a chance-prone subgroup result to all women goes well beyond the data.
D
Correct: with 14 unplanned tests at α=0.05, about one significant result is expected by chance, so the finding only suggests a new trial.

6Thresholds, agreement and reporting

Is p = 0.05 a hard line?