Tutorials › Introduction to Statistics › Reporting Two-Proportion Results Without Overclaiming

Home  /  Tutorials  /  Two-Proportion Inference

Two-Proportion Inference · Tutorial 722 of 996

Reporting Two-Proportion Results Without Overclaiming

Build concise, interpretable reports of two-group proportions by stating the comparison order, uncertainty, and limits of the evidence.

Intermediate 11 min read

What You'll Learn

  • Report each group’s event count, sample size, and proportion in context.
  • State which group is subtracted from which when reporting a difference.
  • Express a difference in proportions as percentage points, not percent.
  • Report a relative risk with its group order and confidence interval.
  • Use confidence intervals and p-values to describe uncertainty without overclaiming.
  • Distinguish evidence of association from evidence that a group difference is causal.

Why the Wording of a Two-Group Result Matters

A calculation can be correct while its report is misleading. Saying only that “the groups differed” hides how the groups were compared, how large the observed difference was, and how uncertain the estimate is. Saying that one group’s outcome was “caused by” its membership may go further than the study design allows.

A useful report gives the reader a short, traceable account: name both groups and their observed proportions, state the comparison in a clearly defined order, report uncertainty, and give a conclusion that matches the evidence and study design. The examples below use hypothetical data. Their purpose is to show how to communicate results, not to describe actual studies.

Definition: For two groups labelled A and B, the ordered difference in proportions is \(\hat{p}_A-\hat{p}_B\). The order is part of the meaning: reversing the groups changes the sign. A relative risk comparing A with B is \(\hat{p}_A/\hat{p}_B\), provided the comparison is meaningful and the denominator proportion is not zero. A complete report identifies the groups, the outcome counted as an event, and the order of comparison.

A Clear Reporting Structure

Begin with the group-specific results. Give each event count and sample size, then the proportion or percentage. Next give the comparison and explicitly name its direction. For a difference, “A minus B” or “A lower than B” makes the order clear. For a ratio, say “risk in A divided by risk in B” or “A compared with B.” Report an interval or test result as appropriate, and interpret it in the original context.

A confidence interval communicates a range of values reasonably compatible with the data under the procedure’s assumptions. Its endpoints must be attached to the same estimand and group order as the point estimate. A p-value describes how unusual the observed test statistic, or one more extreme, would be under the null model; it is not the probability that the null hypothesis is true. Neither result alone determines whether a difference matters practically.

The scale also needs to be named. A difference between proportions is naturally reported in percentage points: a change from 45% to 30% is a difference of minus 15 percentage points in the stated order. A relative risk is a ratio, so a value of 1 means equal estimated risks, a value above 1 means the first risk is larger, and a value below 1 means the first risk is smaller. These quantities answer different questions and should not be described as if interchangeable.

Worked Example: Reporting a Difference With Its Interval and Test

In a hypothetical observational comparison, 30 of 100 patients in Clinic A and 45 of 100 patients in Clinic B experienced a specified event during follow-up. Report the group proportions, the difference in a stated order, and the uncertainty.

Check the setting. The outcome is binary, the two patient samples are described as independent random samples, and the success and failure counts are at least 10 in each group: A has 30 events and 70 non-events; B has 45 events and 55 non-events. These counts support the usual large-sample two-proportion procedures. Because clinic membership was observed rather than randomly assigned, a difference can be reported as an association, not as a causal effect of clinic.

The observed proportions are

$$ \hat{p}_A=\frac{30}{100}=0.300,\qquad \hat{p}_B=\frac{45}{100}=0.450. $$

Choose the order A minus B and calculate the difference:

$$ \hat{p}_A-\hat{p}_B=0.300-0.450=-0.150. $$

Multiplying by 100 expresses this as \(-15.0\) percentage points. Thus, the observed event proportion in Clinic A was 15.0 percentage points lower than in Clinic B. The negative sign is important: it follows from the explicitly chosen A-minus-B order.

For a 95% confidence interval for the difference, use the unpooled standard error:

$$ SE=\sqrt{\frac{0.300(1-0.300)}{100}+ \frac{0.450(1-0.450)}{100}} =\sqrt{0.002100+0.002475} =\sqrt{0.004575} =0.06764. $$

Using the 95% normal critical value 1.96 gives

$$ -0.150\pm1.96(0.06764) =-0.150\pm0.1326 =(-0.2826,-0.0174). $$

In percentage points, the interval is approximately \(-28.3\) to \(-1.74\). The data are compatible, under the interval procedure’s assumptions, with Clinic A’s event proportion being about 1.74 to 28.3 percentage points lower than Clinic B’s. The interval does not mean there is a 95% probability that this fixed population difference lies inside these particular endpoints.

For a two-sided test of equal proportions, the pooled estimate under the null is

$$ \hat{p}_{\mathrm{pool}}=\frac{30+45}{100+100}=0.375. $$

The pooled standard error and test statistic are

$$ SE_0=\sqrt{0.375(1-0.375)\left(\frac{1}{100}+\frac{1}{100}\right)} =\sqrt{0.0046875} =0.06847, \qquad z=\frac{0.300-0.450}{0.06847}=-2.192. $$

The two-sided p-value is approximately \(2P(Z\leq-2.192)=0.0284\). At a 0.05 significance level, this is evidence against equal population proportions. A concise report is: “The event was observed in 30/100 patients (30.0%) at Clinic A and 45/100 (45.0%) at Clinic B. The A-minus-B difference was \(-15.0\) percentage points (95% CI \(-28.3\) to \(-1.74\)); the two-sided test gave \(p=0.0284\). This observational comparison provides evidence of an association between clinic and event proportion, but does not establish that clinic caused the difference.”

Reporting a Ratio Instead of a Difference

A relative risk describes proportional rather than absolute separation. Its orientation must be stated because exchanging the groups takes the reciprocal of the ratio. For example, “A compared with B” means the estimated risk in A is divided by the estimated risk in B. A confidence interval for a relative risk is commonly formed on the log scale and then exponentiated, so the interval endpoints are positive and are interpreted as ratios.

When the confidence interval for a relative risk includes 1, it includes equal risks as a value compatible with the data at that confidence level. This does not prove that the risks are equal. Similarly, an interval that excludes 1 is evidence of a difference under the model and assumptions, not automatically evidence of causation or practical importance.

Worked Example: Reporting a Relative Risk With Its Confidence Interval

In a second hypothetical observational comparison, 40 of 200 people in Group A and 30 of 200 people in Group B experienced an event. Report the risks and a 95% confidence interval for the relative risk, with A as the first group.

Check the setting. The two samples are independent random samples, and the event and non-event counts are 40 and 160 in A and 30 and 170 in B. There are no sparse event or non-event counts for this large-sample log-ratio interval. Since group membership was not assigned randomly, results describe an association and do not by themselves establish a causal effect.

The observed risks and their ratio are

$$ \hat{p}_A=\frac{40}{200}=0.200,\qquad \hat{p}_B=\frac{30}{200}=0.150,\qquad \widehat{RR}_{A:B}=\frac{0.200}{0.150}=1.333. $$

The point estimate says that the observed event risk in A was 1.33 times the risk in B, or 33.3% higher relative to B. It does not say that the difference was 33.3 percentage points: the absolute difference is \(0.200-0.150=0.050\), or 5.00 percentage points.

For the log relative risk, the standard error is

$$ SE\{\log(\widehat{RR}_{A:B})\} =\sqrt{\frac{1}{40}-\frac{1}{200}+\frac{1}{30}-\frac{1}{200}} =\sqrt{0.04833} =0.2198. $$

The log estimate is \(\log(1.333)=0.2877\). The 95% interval on the log scale is

$$ 0.2877\pm1.96(0.2198) =0.2877\pm0.4309 =(-0.1432,0.7186). $$

Exponentiating both endpoints gives the interval for the ratio:

$$ \left(e^{-0.1432},e^{0.7186}\right)=(0.8666,2.052). $$

A suitable report is: “The event occurred in 40/200 people in A (20.0%) and 30/200 in B (15.0%). The estimated relative risk for A compared with B was 1.33 (95% CI 0.87 to 2.05). Although the point estimate is higher in A, the interval includes 1 and is compatible with equal risks as well as with lower or higher risk in A; this observational comparison does not establish a causal increase.”

When a Test Does Not Show Clear Evidence

A non-small p-value is not evidence that the groups have identical proportions. It means the data do not provide sufficiently strong evidence against the null hypothesis under the selected test and significance level. A confidence interval makes the range of effects still compatible with the data more visible. A careful conclusion therefore avoids both declaring a difference proved when evidence is weak and declaring equality proved simply because a test is not statistically significant.

Worked Example: Reporting an Uncertain Difference

In a third hypothetical comparison, 20 of 200 people in A and 30 of 200 in B experienced an event. Use A minus B, and report both a confidence interval and a two-sided test result.

Check the setting. Suppose these are independent random samples from the two populations. A has 20 events and 180 non-events, while B has 30 events and 170 non-events, so each group has at least 10 events and 10 non-events. The large-sample procedures are reasonable under these conditions. The comparison is observational, so any conclusion is about association.

The proportions and ordered difference are

$$ \hat{p}_A=\frac{20}{200}=0.100,\qquad \hat{p}_B=\frac{30}{200}=0.150,\qquad \hat{p}_A-\hat{p}_B=0.100-0.150=-0.050. $$

The observed difference is \(-5.00\) percentage points, meaning the observed proportion in A is 5.00 percentage points lower than in B. For the confidence interval, the unpooled standard error and interval are

$$ SE=\sqrt{\frac{0.100(0.900)}{200}+ \frac{0.150(0.850)}{200}} =\sqrt{0.0010875} =0.03298, $$
$$ -0.050\pm1.96(0.03298) =-0.050\pm0.06464 =(-0.1146,0.01464). $$

In percentage points, the 95% interval is approximately \(-11.5\) to \(1.46\). It includes zero, and so is compatible with no difference as well as with A having a lower or slightly higher event proportion.

For the test, the pooled estimate is \((20+30)/(200+200)=0.125\), and

$$ SE_0=\sqrt{0.125(0.875)\left(\frac{1}{200}+\frac{1}{200}\right)} =\sqrt{0.00109375} =0.03307, \qquad z=\frac{-0.050}{0.03307}=-1.512. $$

The two-sided p-value is approximately \(2P(Z\leq-1.512)=0.1305\). A suitable conclusion is: “The event proportions were 10.0% in A and 15.0% in B, giving an A-minus-B difference of \(-5.00\) percentage points (95% CI \(-11.5\) to \(1.46\)); the two-sided test gave \(p=0.131\). These data do not provide clear evidence of a difference in population proportions. They do not establish that the proportions are equal.”

A Short Justification for Stating the Order

The direction of a difference is not a stylistic detail. If the difference in the first example is defined as \(d=\hat{p}_A-\hat{p}_B=-0.150\), reversing the order gives

$$ \hat{p}_B-\hat{p}_A=-(\hat{p}_A-\hat{p}_B)=-d=0.150. $$

An interval must reverse in the same way. If the interval for \(d\) is \((L,U)\), then the interval for \(-d\) is \((-U,-L)\): negating every value changes the order of the endpoints. In the first example, \((-0.2826,-0.0174)\) for A minus B becomes \((0.0174,0.2826)\) for B minus A. Both statements describe the same comparison, but only when the order is stated and the sign is interpreted consistently.

This simple check catches a common reporting error: giving a negative difference and then describing the first group as higher. Before writing the interpretation, check that the sign, the subtraction order, and the words “higher” or “lower” agree.

Association, Causal Evidence, and Common Pitfalls

A two-proportion calculation quantifies a difference or association; it does not decide by itself whether a group caused an outcome. In an observational study, people may differ between groups in other ways that also relate to the outcome. A statement such as “Group A had a higher observed proportion than Group B” is descriptive. Calling that difference an effect caused by A requires a design and assumptions that support a causal interpretation.

Random assignment to groups can provide a basis for causal conclusions about the assigned intervention, when the study is conducted appropriately and other relevant conditions are met. Even then, be precise about what was assigned and to whom the conclusion applies. Random sampling supports generalizing to the population sampled; it is not the same as random assignment and does not, on its own, establish causation.

  • Leaving out the order. “The difference was 5 percentage points” is ambiguous. State which proportion is subtracted from which, and retain the sign or describe the direction accurately.
  • Calling percentage points a percent change. A shift from 45% to 30% is \(-15\) percentage points. A relative percent change uses a different denominator and is not the same quantity.
  • Reporting only a p-value. A p-value does not tell the reader the estimated size or range of a difference. Include the group proportions and an estimate with uncertainty when available.
  • Concluding equality from a non-significant test. A large p-value does not prove equal proportions. Describe the evidence as insufficient to show a difference and use the interval to show what values remain plausible.
  • Turning association into causation. A statistically significant difference in an observational comparison is still an association unless the design and assumptions justify a causal claim.
Key takeaway: A strong two-proportion report names the outcome and groups, gives both observed proportions, states an ordered difference or ratio, reports uncertainty on the matching scale, and draws a conclusion no stronger than the data and study design support.

Check Your Understanding

For each question, focus on the direction and strength of the claim as well as the arithmetic.

  1. Group C has an event proportion of 0.24 and Group D has an event proportion of 0.31. What is the C-minus-D difference in proportions, in percentage points, and how would you describe its direction?
  2. A report gives a relative risk of 0.72 for Group A compared with Group B. Which group is the numerator group, and how should the estimate be described?
  3. A 95% confidence interval for an ordered difference includes zero. Does that prove that the population proportions are equal? Explain.
  4. A two-sided test gives \(p=0.18\). What is a defensible conclusion, and what conclusion would overstate the evidence?
  5. Why does random sampling differ from random assignment when deciding whether a two-group result supports a causal claim?

Previous

Next

Back to Tutorials