Confounding and Simpson's Paradox

Why a Pooled Comparison Can Mislead

Suppose two groups have different outcome rates. A pooled comparison summarizes each group across everyone in it, but the groups may contain different mixes of people or circumstances. If some subgroups tend to have higher outcome rates than others, those different mixes can change the pooled rates—even when the comparison within each subgroup points in the opposite direction.

This is the setting for Simpson’s paradox: an association seen in separate subgroups reverses when the data are pooled, or an association in the pooled data reverses after the data are separated into subgroups. The reversal is a mathematical feature of the table. Confounding is an interpretation: a third variable is a plausible explanation for why the pooled comparison differs from the within-subgroup comparisons. The two ideas are related, but a reversal alone does not establish that a particular variable is a confounder.

Definition: A pooled comparison combines observations across subgroups. A stratified comparison compares groups separately within categories (strata) of a third variable. In a descriptive comparison, a variable is a candidate confounder when it is associated with group membership and with the outcome, and accounting for it changes the pooled comparison. This pattern can suggest confounding, but a table alone does not prove a causal explanation.

The two-way table within each stratum supplies a conditional outcome proportion: the number with the outcome divided by the total in that group and stratum. The pooled proportion uses the group’s total across all strata as its denominator. These are different quantities, because the strata can have different sizes in each group.

How the Weights Produce a Reversal

Let \(r_{gj}\) be the outcome proportion for group \(g\) in stratum \(j\), and let \(w_{gj}\) be the proportion of group \(g\) that belongs to stratum \(j\). The pooled outcome proportion for group \(g\) is a weighted average of its stratum-specific proportions:

r_g=\sum_j w_{gj}r_{gj}, \qquad \sum_j w_{gj}=1.

The weights are group-specific: if one group has many observations in a high-outcome stratum and the other group has few, their pooled rates give that stratum different influence. The identity follows directly from the counts. If \(n_{gj}\) is the number in group \(g\) and stratum \(j\), and \(y_{gj}\) is the number with the outcome, then \(r_{gj}=y_{gj}/n_{gj}\), \(w_{gj}=n_{gj}/n_g\), and \(n_g=\sum_j n_{gj}\). Therefore:

\sum_j w_{gj}r_{gj} =\sum_j\frac{n_{gj}}{n_g}\frac{y_{gj}}{n_{gj}} =\frac{\sum_j y_{gj}}{n_g} =r_g.

A pooled comparison therefore combines two features: the outcome rates within strata and the proportions of each group in those strata. A reversal can occur when the groups have very different weights and the strata themselves have different outcome rates.

Worked Example: A Pooled Rate Reverses Both Stratified Comparisons

Imagine a hypothetical comparison of two treatments, A and B, where the outcome is recovery. Patients are also classified as having lower or higher baseline severity. The table gives recoveries and totals; every rate is a recovery proportion.

Severity stratum	Treatment A	Treatment B
Lower	81/90 = 90.0%	19/20 = 95.0%
Higher	1/10 = 10.0%	72/180 = 40.0%
Pooled	82/100 = 82.0%	91/200 = 45.5%

Within the lower-severity stratum, B has the higher recovery proportion: \(95.0\%-90.0\%=5.0\) percentage points. Within the higher-severity stratum, B is also higher: \(40.0\%-10.0\%=30.0\) percentage points. But pooled across severity groups, A has the higher proportion: \(82.0\%-45.5\%=36.5\) percentage points. The pooled comparison reverses both stratified comparisons, so this is Simpson’s paradox.

The group compositions differ sharply. Of A’s 100 patients, 90 are in the lower-severity stratum; of B’s 200 patients, only 20 are. The lower-severity stratum has higher recovery proportions for both treatments, and it makes up 90% of A but only 10% of B. Thus the pooled rate for A receives much more weight from the stratum with generally higher recovery.

In this hypothetical setting, baseline severity is a plausible candidate confounder: it is related to treatment-group membership in the table and is related to recovery proportions. The stratified rates show why the pooled comparison is distorted by the different mix of severity groups. The table describes an association; by itself, it does not establish why patients received a treatment or what would happen under a different assignment.

Separating Composition From Within-Stratum Rates

One way to isolate the composition issue is to calculate standardized proportions: weighted averages that give each group the same chosen stratum weights. Standardization is a descriptive comparison, not a new set of observed pooled rates. It asks what the weighted rates would be under a common mix of strata.

Worked Example: Comparing the Treatments With Common Weights

Use the stratum-specific recovery proportions from the previous table, but give the lower- and higher-severity strata equal weight for both treatments. The common weights are \(0.50\) and \(0.50\).

For A, the standardized proportion is:

$$ (0.50)(0.90)+(0.50)(0.10)=0.45+0.05=0.50. $$

For B, it is:

$$ (0.50)(0.95)+(0.50)(0.40)=0.475+0.200=0.675. $$

Under this equal mix of severity strata, the standardized recovery proportions are 50.0% for A and 67.5% for B, a difference of 17.5 percentage points in favor of B. This differs from the observed pooled comparison, which favors A by 36.5 percentage points. The standardized figures make clear how much the choice of subgroup weights can matter; they should not be reported as the groups’ actual pooled recovery rates.

The common weights in this example were chosen for illustration. Other choices—such as weights based on the combined sample—can produce different standardized values. What matters for a fair descriptive comparison is applying the same weights to both groups and stating which weights were used.

Worked Example: A Pooled Difference Disappears Within Strata

A hypothetical service compares the proportion of requests completed on time by two teams. Requests are classified as routine or complex. The table gives on-time requests divided by all requests in each category.

Request type	Team A	Team B
Routine	90/100 = 90.0%	18/20 = 90.0%
Complex	30/100 = 30.0%	54/180 = 30.0%
Pooled	120/200 = 60.0%	72/200 = 36.0%

Within each request type, the teams have identical on-time proportions. Yet the pooled proportions differ by \(60.0\%-36.0\%=24.0\) percentage points. Team A handled 100 routine requests out of 200, while Team B handled only 20 routine requests out of 200. Since routine requests have a higher on-time rate for both teams, the different mix of request types creates the pooled difference.

This is a compositional distortion, but it is not a reversal of the within-stratum comparisons: there is no within-stratum difference to reverse. Request type is a plausible candidate confounder for the descriptive comparison because it is related to team membership and to the outcome rate. The distinction matters: not every case in which stratification changes a pooled difference is Simpson’s paradox.

When Is “Confounding” the Right Description?

A two-way table can show a reversal, a weakened difference, or a difference that disappears after stratification. Those are numerical patterns. Calling the third variable a confounder requires a contextual argument as well as arithmetic. Ask:

Is the third variable associated with group membership? Compare the subgroup distributions across groups. Very different subgroup shares make different weighting plausible.
Is it related to the outcome? The outcome proportions should vary across its strata, or there should be a sound contextual reason to expect them to vary.
Does stratifying change the comparison? Compare the pooled difference with the differences within the strata, keeping the direction and size of each clear.
Is the variable meaningful in context? A table’s labels and counts can reveal patterns, but subject-matter knowledge is needed to judge whether the variable offers a credible explanation.

These checks help distinguish a plausible confounding explanation from the bare fact that weighted averages can differ. They do not prove that the third variable is responsible for an association, nor do they establish a causal effect. The same numerical reversal can be described accurately from the table while its explanation remains uncertain.

Key takeaway: Simpson’s paradox names a reversal between pooled and stratified comparisons. Confounding is a proposed explanation involving a third variable related to both group membership and outcome. Different subgroup weights can generate the reversal, but the table’s arithmetic alone cannot establish the explanation.

Common Mistakes

Using the pooled rate as if it described every subgroup. A pooled proportion averages across the group’s actual subgroup mix; it need not match any subgroup proportion.
Calling every change after stratification Simpson’s paradox. A difference can shrink or disappear without reversing. Reserve “reversal” for a change in direction.
Assuming the reversal proves confounding. It establishes a numerical pattern, not why it occurred. Identify a plausible third variable and examine how it relates to group membership and the outcome.
Comparing unweighted averages without saying so. A simple average of subgroup percentages generally is not the observed pooled rate. The pooled rate weights each subgroup by its share of the group.
Turning an association into a causal conclusion. A stratified table can describe and clarify associations, but it cannot by itself show what caused the outcome.

Check Your Understanding

For each question, focus on the denominators, subgroup weights, and what the table can support.

In a pooled comparison, what determines the weight given to each stratum within a group?
Two groups have outcome proportions of 80% and 60% in one stratum, and 40% and 20% in another. If the first group’s pooled proportion is lower than the second’s, what pattern would make this a Simpson reversal?
Why can two groups have identical outcome proportions within every stratum but different pooled proportions?
In the treatment example, why is baseline severity a plausible candidate confounder? Name the two associations visible in the table.
Why does observing a Simpson reversal not, by itself, prove that the stratifying variable caused the pooled association?