Interpreting Confidence Intervals for Risk Differences

Why the Order of the Difference Matters

A risk is the proportion of a population or group that experiences a specified event. Comparing two risks requires subtracting one from the other, so the order of subtraction determines the direction of the comparison. A confidence interval for that difference summarizes both an estimated contrast and the uncertainty around it.

Suppose Group 1 has an event risk of 12% and Group 2 has a risk of 20%. Subtracting the first risk from the second gives \(0.20-0.12=0.08\), or an 8-percentage-point difference. Reversing the subtraction gives \(0.12-0.20=-0.08\), or a difference of minus 8 percentage points. The size is the same, but the sign—and therefore the meaning—changes. Always identify the group order before interpreting an interval.

Definition: For two population risks \(p_1\) and \(p_2\), the risk difference in the order Group 1 minus Group 2 is \(\Delta=p_1-p_2\). Its estimate is \(\hat{p}_1-\hat{p}_2\). A positive difference means the event risk is higher in Group 1; a negative difference means it is higher in Group 2. A difference of zero means the two population risks are equal.

A risk difference is an absolute difference, not a ratio. If it is reported on a percentage scale, describe it in percentage points. For instance, a change from 12% to 20% is an increase of 8 percentage points, not an increase of 8%. Keeping this distinction clear makes both the estimate and its interval easier to interpret.

What the Interval Says

A confidence interval gives a range of values for the population risk difference that are compatible with the observed data and the interval procedure. Its endpoints are expressed in the same units as the difference. Thus, an interval from 0.02 to 0.10 can also be reported as a range from 2 to 10 percentage points.

In the examples below, the intervals are approximate 95% confidence intervals calculated using the unpooled standard error for two independent sample proportions. This method is appropriate when the groups are independent, observations within each group can reasonably be treated as independent, and the success and failure counts in each group are large enough for a normal approximation. The hypothetical examples state how these conditions are met. In an actual analysis, the study design and the chosen interval method must be appropriate for the data.

(\hat{p}_1-\hat{p}_2) \ \pm\ 1.96\sqrt{ \frac{\hat{p}_1(1-\hat{p}_1)}{n_1} + \frac{\hat{p}_2(1-\hat{p}_2)}{n_2} }

Here \(n_1\) and \(n_2\) are the group sample sizes. The center is the estimated risk difference; the quantity added to and subtracted from it is the margin of error. The interval’s interpretation depends on the group order, the size of its endpoints, and whether zero is between them.

Worked Example: An Interval Entirely Above Zero

In a hypothetical observational study, researchers independently sample 120 people from each of two populations and record whether each person experiences a specified event. In Group 1, 36 people experience the event; in Group 2, 21 do. Interpret the 95% confidence interval for Group 1 minus Group 2.

Check the setting. The groups are independent samples, and the observations within each sample are assumed independent. The success and failure counts are 36 and 84 in Group 1, and 21 and 99 in Group 2, so each group has at least 10 observed successes and 10 observed failures. These facts support using the stated normal-approximation interval, assuming the samples represent the populations of interest.

Calculate the sample risks and difference. The sample proportions are \(\hat{p}_1=36/120=0.30\) and \(\hat{p}_2=21/120=0.175\). In the specified order, the estimated difference is \(0.30-0.175=0.125\), or 12.5 percentage points.

SE= \sqrt{\frac{0.30(0.70)}{120}+\frac{0.175(0.825)}{120}} = \sqrt{0.002953125} \approx 0.05434

The margin of error is \(1.96(0.05434)\approx0.1065\). Therefore, the interval is

0.125\pm0.1065 = (0.0185,\ 0.2315) \approx (0.019,\ 0.232).

The entire interval is positive. In context, the data support a Group 1 population event risk about 1.9 to 23.2 percentage points higher than the Group 2 risk, under the assumptions of this interval procedure. The estimate is 12.5 percentage points higher, but the interval shows that the size of the difference is uncertain. This interval alone does not establish that the group membership caused the difference; the study design and other evidence matter for causal conclusions.

Intervals That Include Zero

Zero has a direct meaning for a risk difference: it represents equal population risks. If zero lies strictly between the interval’s endpoints, then the interval contains values corresponding to Group 1 having a lower risk, equal risks, and Group 1 having a higher risk. If zero is exactly an endpoint, the interval contains equal risks and values on only one side. How much of each direction is included depends on the endpoints.

An interval that includes zero is not proof that the risks are equal. It indicates that the interval procedure has not narrowed the plausible values to only positive differences or only negative differences. The interval may still rule out some large differences, or it may be wide enough to include a broad range of effects. Its width is therefore important, not just the fact that zero lies within it.

Worked Example: An Interval That Includes Zero

In a hypothetical study, independent samples of 100 people are drawn from each of two populations. The event occurs for 18 people in Group 1 and 24 people in Group 2. Find and interpret the 95% confidence interval for Group 1 minus Group 2.

Check the setting. Assume the samples are independent and represent their respective populations, with independent observations within each sample. The success and failure counts are 18 and 82 in Group 1, and 24 and 76 in Group 2; all are at least 10. The normal-approximation interval is reasonable under these assumptions.

The sample risks are \(0.18\) and \(0.24\). Thus the estimated difference is \(0.18-0.24=-0.06\), or minus 6 percentage points.

SE= \sqrt{\frac{0.18(0.82)}{100}+\frac{0.24(0.76)}{100}} = \sqrt{0.0033} \approx0.05745

The margin of error is \(1.96(0.05745)\approx0.1126\), so the interval is

-0.06\pm0.1126 = (-0.1726,\ 0.0526) \approx (-0.173,\ 0.053).

In context, the interval ranges from a Group 1 risk about 17.3 percentage points lower than the Group 2 risk to a Group 1 risk about 5.3 percentage points higher. Since zero falls inside the interval, the data do not pin down the direction of the population difference at this confidence level. The interval does not show that the risks are equal, nor does it show that any difference is unimportant. It describes uncertainty about the size and direction of the difference.

Reading the Sign and Size Together

For an interval with lower endpoint \(L\) and upper endpoint \(U\), the sign tells you which direction of difference is represented, while the endpoints show the range of magnitudes. An interval entirely below zero means Group 1’s risk is lower throughout the interval. An interval entirely above zero means Group 1’s risk is higher throughout it. If zero lies strictly between \(L\) and \(U\), both directions and equality are among the values in the interval; if zero equals one endpoint, equality and only one direction are included.

This sign-based reading follows directly from the definition of the risk difference. If \(p_1-p_2>0\), then \(p_1>p_2\); if \(p_1-p_2<0\), then \(p_1<p_2\); and if \(p_1-p_2=0\), then \(p_1=p_2\). Therefore, an interval entirely on one side of zero gives a range of differences in one direction. An interval that spans zero includes values in more than one direction.

Worked Example: A Negative Difference and the Importance of Order

Consider hypothetical independent samples of 200 people per group. The event occurs for 24 people in Group 1 and 40 people in Group 2. Find the 95% confidence interval for Group 1 minus Group 2, then explain how reversing the order changes the interpretation.

Check the setting. Assume independent, representative samples and independent observations within each sample. Group 1 has 24 successes and 176 failures; Group 2 has 40 successes and 160 failures. Each count is at least 10, supporting the normal approximation.

The sample risks are \(\hat{p}_1=24/200=0.12\) and \(\hat{p}_2=40/200=0.20\). For Group 1 minus Group 2, the estimated difference is \(0.12-0.20=-0.08\), or minus 8 percentage points.

SE= \sqrt{\frac{0.12(0.88)}{200}+\frac{0.20(0.80)}{200}} = \sqrt{0.001328} \approx0.03644

The margin of error is \(1.96(0.03644)\approx0.07143\). The interval is

-0.08\pm0.07143 = (-0.15143,\ -0.00857) \approx (-0.151,\ -0.009).

Every value in this interval is negative. In context, Group 1’s population event risk is estimated to be about 0.9 to 15.1 percentage points lower than Group 2’s. If the subtraction order were reversed, the point estimate would be \(0.20-0.12=0.08\), or plus 8 percentage points, and the interval endpoints would also change signs: approximately \((0.009,\ 0.151)\). That reversed interval describes Group 2’s risk minus Group 1’s, not a different underlying comparison.

What Confidence Means—and a Common Pitfall

A 95% confidence level describes the long-run performance of the interval procedure. If the same sampling process were repeated many times and a 95% interval constructed each time, approximately 95% of those intervals would contain the fixed population risk difference, provided the model and method assumptions hold. The parameter is fixed; the interval varies from sample to sample.

Common mistake: “There is a 95% probability that the true risk difference lies in this particular interval.” In the usual frequentist interpretation, that wording assigns a probability to the fixed parameter after the interval has been calculated. The correct interpretation is that the procedure has 95% long-run coverage. For a particular computed interval, report its bounds and explain what values they represent in context.

Another mistake is to describe a confidence interval using only its sign or only its width. “The interval is positive” identifies direction but not the range of possible magnitudes. “The interval is wide” describes uncertainty but not which group tends to have higher risk. A complete interpretation states the subtraction order, the range in percentage points, and whether zero is included.

Key takeaway: Interpret the risk difference in its stated order. Positive values mean the first group has higher risk; negative values mean it has lower risk; zero means equal risks. Use both interval endpoints to describe the magnitude and uncertainty, and do not treat an interval containing zero as proof of equality.

Check Your Understanding

Answer each question using the group order and units precisely.

Group A has a risk of 9% and Group B has a risk of 14%. What is the risk difference for A minus B, in percentage points, and what does its sign mean?
A confidence interval for Group 1 minus Group 2 is \((0.03,\ 0.11)\). State its direction and range in percentage points.
An interval for Group 1 minus Group 2 is \((-0.08,\ 0.02)\). What does including zero indicate, and what does it not prove?
Why should an interval for Group 2 minus Group 1 have endpoints with the opposite signs from an interval for Group 1 minus Group 2?
Rewrite this claim correctly: “There is a 95% probability that the true difference is in the interval we calculated.”