A Complete Two-Group Inference Case Study

From Counts to a Coherent Comparison

A two-group study can produce several summaries of the same binary outcome. The risk difference describes how far apart the groups' event probabilities are; the relative risk compares those probabilities as a ratio; and the odds ratio compares their odds. A hypothesis test can assess evidence against equal risks. These measures answer related but different questions, so a sound analysis keeps their meanings and calculations distinct.

We will use one hypothetical randomized health study throughout. Participants are assigned to either a new preventive program or usual care, and the outcome is whether each participant experiences a specified adverse event during follow-up. The program group is called group 1 and usual care is group 2. The event is the “success” for purposes of counting, even though it is an undesirable outcome.

Definition: Let \(n_1\) and \(n_2\) be the numbers of participants in groups 1 and 2, and let \(x_1\) and \(x_2\) be their respective event counts. The sample proportions are \(\hat{p}_1=x_1/n_1\) and \(\hat{p}_2=x_2/n_2\), estimating the group event risks \(p_1\) and \(p_2\). With group 1 defined as the program group, the risk difference is \(p_1-p_2\), estimated by \(\hat{p}_1-\hat{p}_2\). The relative risk is \(p_1/p_2\), and the odds ratio is \([p_1/(1-p_1)]/[p_2/(1-p_2)]\).

The sign and order of a comparison matter. Here, a negative risk difference means the observed event risk is lower in the program group. A relative risk below 1 likewise indicates a lower observed risk in group 1, while an odds ratio below 1 indicates lower observed odds. Neither ratio is measured in percentage points; both are unitless.

The Hypothetical Study and Its Conditions

Suppose 120 participants are assigned to each group, all have an observed outcome, and the groups do not overlap. The study records 18 events in the program group and 30 in usual care. Thus, the non-event counts are 102 and 90. The data are summarized below.

Group	Events	No event	Total
Program	18	102	120
Usual care	30	90	120

The usual two-proportion procedures require a suitable independent-groups model: within each group, outcomes are binary observations with a common group risk, and observations are independent; the two groups are independent of one another. In this hypothetical randomized study, each participant contributes one outcome to one group, and the groups contain different participants. We also assume complete follow-up and no clustering or other dependence that would make the participant-level outcomes correlated.

The normal approximations used for the risk-difference interval and pooled test need adequate event and non-event counts. The observed counts are all comfortably above 10. For the equality test, the pooled risk is used under the null; the resulting expected event counts are also adequate, as checked below. Random assignment supports comparing the groups as an effect of assignment in this hypothetical study. It does not, by itself, establish that the participants represent every population to which someone might want to generalize.

Worked Example: Calculating the Observed Risks

First calculate the event proportion in each group and the observed risk difference. The program group has \(x_1=18\) events among \(n_1=120\) participants; usual care has \(x_2=30\) among \(n_2=120\).

Program risk:

\hat{p}_1=\frac{x_1}{n_1}=\frac{18}{120}=0.150

Usual-care risk:

\hat{p}_2=\frac{x_2}{n_2}=\frac{30}{120}=0.250

With the program risk first, the estimated risk difference is

\hat{p}_1-\hat{p}_2=0.150-0.250=-0.100

The observed event risk was 10.0 percentage points lower in the program group than in usual care. This is a difference in proportions, not a 10% relative reduction. The relative change will be summarized separately.

Risk-Difference Interval and Test

A confidence interval for \(p_1-p_2\) estimates the size and direction of the risk difference. For two independent groups, the estimated standard error uses a separate estimated risk in each group:

SE_{\mathrm{diff}}= \sqrt{\frac{\hat{p}_1(1-\hat{p}_1)}{n_1} +\frac{\hat{p}_2(1-\hat{p}_2)}{n_2}}

The reason for adding the two terms is that independent estimates have a variance for their difference equal to the sum of their variances. For a group of \(n\) independent binary outcomes with event probability \(p\), the sample proportion has variance \(p(1-p)/n\). Replacing the unknown risks by their sample estimates gives the two terms above. The usual approximate 95% interval is the estimate plus or minus 1.96 standard errors.

Worked Example: A 95% Interval for the Risk Difference

Use the two observed proportions from the study. The independent-group condition is satisfied by the study setup, and the observed event and non-event counts in both groups are adequate for this normal approximation.

The unpooled standard error is

SE_{\mathrm{diff}}= \sqrt{\frac{0.150(1-0.150)}{120} +\frac{0.250(1-0.250)}{120}} =\sqrt{0.0010625+0.0015625} =\sqrt{0.002625} =0.05123

The margin of error is \(1.96(0.05123)=0.1004\). Therefore,

(\hat{p}_1-\hat{p}_2)\pm1.96SE_{\mathrm{diff}} =-0.100\pm0.1004 =(-0.2004,\ 0.0004)

The approximate 95% confidence interval for the program-minus-usual-care risk difference runs from \(-0.2004\) to \(0.0004\), or about \(-20.04\) to \(0.04\) percentage points. In repeated studies under the model and conditions, this interval procedure would capture the true risk difference about 95% of the time. For this study, the interval is compatible with a decrease in risk as large as about 20 percentage points, and with a very small increase. Because it includes zero, it does not rule out equal risks at the corresponding two-sided 5% level.

To test equality of risks, the hypotheses are \(H_0:p_1-p_2=0\) and \(H_a:p_1-p_2\ne0\). Under \(H_0\), both groups share one risk, so the test estimates that common risk by pooling the event counts:

\hat{p}_{\mathrm{pool}}=\frac{x_1+x_2}{n_1+n_2}

The pooled standard error is used for this null hypothesis test. It is different from the unpooled standard error used for the confidence interval. The test statistic is the observed difference divided by this pooled standard error; under the null and adequate-count conditions, it is compared with the standard normal distribution.

Worked Example: Testing for a Difference in Risks

For the study, the pooled event proportion is

\hat{p}_{\mathrm{pool}}=\frac{18+30}{120+120} =\frac{48}{240}=0.200

Under \(H_0\), the expected event count in each group is \(120(0.200)=24\), and the expected non-event count in each group is \(120(0.800)=96\). All four expected counts are adequate for the normal approximation.

The pooled standard error and test statistic are

SE_0=\sqrt{0.200(0.800)\left(\frac{1}{120}+\frac{1}{120}\right)} =\sqrt{0.0026667}=0.05164

z=\frac{0.150-0.250}{0.05164}=-1.937

For a two-sided alternative, the p-value is \(2P(Z\le -1.937)\), where \(Z\) has the standard normal distribution. This gives a p-value of approximately 0.0528. At significance level \(\alpha=0.05\), the p-value is slightly larger than \(\alpha\), so we do not reject \(H_0\). The data do not provide sufficient evidence at this level that the two event risks differ. This conclusion is not proof that the risks are equal.

The interval and test agree on the 5% decision: the 95% interval includes zero, and the two-sided test does not reject equality. Their numerical procedures are not identical: the interval uses an unpooled standard error, while the test uses a pooled one under \(H_0\). Here, both give a result just on the non-significant side of the threshold.

Relative Risk and Odds Ratio Estimates

The risk difference gives an absolute comparison: how many more or fewer events per participant are observed in one group. The relative risk and odds ratio instead give multiplicative comparisons. They can be calculated from the same four counts, but their denominators differ.

Worked Example: Calculating Relative Risk and Odds Ratio

The estimated relative risk compares the program event risk with the usual-care event risk:

\widehat{RR}=\frac{\hat{p}_1}{\hat{p}_2} =\frac{0.150}{0.250}=0.600

The estimated relative risk is 0.600. In this hypothetical study, the observed event risk in the program group was 0.60 times the observed risk in usual care, or 40% lower on a relative-risk scale. This does not mean a decrease of 40 percentage points: the observed absolute difference was 10 percentage points.

To obtain the odds, divide the event risk by the non-event risk. The program group's estimated odds are \(18/102\), and usual care's are \(30/90\). Their ratio is

\widehat{OR} =\frac{18/102}{30/90} =\frac{18(90)}{102(30)} =\frac{1620}{3060} =0.5294

The estimated odds of the event in the program group were about 0.529 times the odds in usual care, or about 47.1% lower odds. This is an odds comparison, not a statement that the event risk was 47.1% lower. The numerical distinction matters because odds and risks are not interchangeable, particularly when events are not rare. Here the risks are 0.150 and 0.250, so interpreting the odds ratio as if it were a relative risk would overstate the relative reduction.

Why Keep All Three Measures?

The measures emphasize different aspects of the same comparison. In the example, the risk difference is \(-0.100\), the relative risk is \(0.600\), and the odds ratio is \(0.5294\). The first directly describes an absolute change in event probability; the second compares event probabilities proportionally; the third compares event-to-non-event odds. Giving the measure and its direction explicitly prevents an estimate from being mistaken for another.

The point estimates alone do not communicate their uncertainty. The risk-difference interval here includes zero, and the test does not reject equal risks at \(\alpha=0.05\). A relative risk or odds ratio estimate below one should not be presented as conclusive evidence of a difference simply because its point estimate is below one. The study's design, uncertainty, and the chosen measure all matter.

Common pitfall: A reader sees \(\widehat{RR}=0.600\) and says, “The program reduced the event risk by 40 percentage points.” That confuses a relative reduction with an absolute difference. The relative reduction is 40% compared with the usual-care risk; the observed absolute difference is \(0.150-0.250=-0.100\), or 10 percentage points lower. State the scale and the reference group whenever reporting an effect measure.

Define the groups and event.
Fix which group is group 1, which is group 2, and what counts as an event before computing any comparison.

Check the study structure.
Confirm binary outcomes, independent observations and groups, complete outcome information, and adequate counts for the planned approximation.

Calculate the risks and risk difference.
Report the group proportions and the group-1-minus-group-2 difference, including its sign and units.

Match each inference calculation to its purpose.
Use the unpooled standard error for the risk-difference interval and the pooled standard error for the equality test.

Give ratio estimates their correct interpretation.
Calculate relative risk from risks and the odds ratio from odds; do not describe either as a risk difference.

A complete two-group analysis is not a search for one universally best number. It is a consistent account of the event counts, the absolute difference, the evidence about equality, and any relative comparisons that address the question. Always preserve the group order and distinguish a lack of statistically significant evidence from evidence that there is no difference.

Check Your Understanding

Use the definitions and reasoning in this case study to answer the following.

If group 1 has 24 events among 160 participants and group 2 has 32 among 160, calculate both observed risks and the group-1-minus-group-2 risk difference.
Why does the pooled standard error belong to the equality test, while the risk-difference interval uses an unpooled standard error?
In the study above, the confidence interval includes zero and the p-value is 0.0528. What can and cannot be concluded at \(\alpha=0.05\)?
Using the original study counts, explain in context what the estimated relative risk of 0.600 means, and why it is not a 40-percentage-point decrease.
Which counts are used to calculate the odds ratio, and why should an odds ratio below one not automatically be described as the same relative decrease in risk?