Simpson's Paradox in a Worked Example

When a Comparison Reverses

A comparison can look clear within every relevant subgroup and still point in the opposite direction when all the observations are combined. This is not an arithmetic contradiction. The subgroup comparisons and the overall comparison use different collections of observations, and the groups being compared may be represented in different proportions across the subgroups.

This reversal is called Simpson’s paradox. The name describes a pattern that can arise in data; it does not mean that a table is incorrect or that a particular explanation has been established. To see the pattern, calculate the rates carefully, first within each subgroup and then from the combined totals.

For a success or outcome recorded as present or absent, the rate for a group is the number with the outcome divided by the number in that group. Always identify the denominator before comparing rates. A count of successes alone is not a rate: groups of different sizes can have different counts even when their rates are similar.

A Worked Reversal in Two Departments

Suppose, in a hypothetical example, two applicant groups—Redwood and Blue—apply to two departments. The outcome is whether an applicant is accepted. The table reports accepted and not accepted applicants, with the acceptance rate calculated within each group and department.

The table also shows a major difference in the groups’ department distributions. Redwood’s 60 applicants include 20 in Engineering and 40 in Arts. Blue’s 100 applicants include 90 in Engineering and 10 in Arts. In this example, Engineering has higher acceptance rates for both groups than Arts does. Blue has many more applicants in Engineering, while a greater share of Redwood’s applicants are in Arts. As a result, combining the counts gives greater influence to the department where acceptance is more common for each group.

This explanation describes the arithmetic pattern in this table. It does not establish why the applicant distributions differ, nor does the table alone show what would happen if individuals or applications were assigned differently. Keep the conclusion tied to the rates actually calculated.

Why the Reversal Is Possible

The short derivation below expresses the essential arithmetic without requiring any special statistical model. Suppose group \(G\) has \(a_1\) outcomes among \(n_1\) observations in subgroup 1, and \(a_2\) outcomes among \(n_2\) observations in subgroup 2. When the observations are pooled, the outcome count is \(a_1+a_2\), and the total number of observations is \(n_1+n_2\). Therefore the pooled rate is

For the Redwood example, the numerator is \(18+8=26\), and the denominator is \(20+40=60\). For Blue, they are \(72+1=73\) and \(90+10=100\). Thus the pooled comparison depends on how many observations from each subgroup enter each group’s totals, not just on the direction of the two subgroup comparisons.

Here is a direct numerical check using only the totals. The pooled difference, Redwood minus Blue, is

The negative result says Redwood’s pooled rate is about 29.7 percentage points lower. Within Engineering, the difference is \(90\%-80\%=10\) percentage points; within Arts, it is \(20\%-10\%=10\) percentage points. The subgroup differences are both positive, while the pooled difference is negative. That sign change is the reversal.

A Second Example With Different Counts

The pattern is not limited to acceptance decisions. In another hypothetical example, two groups complete a task in three settings with different levels of difficulty. The outcome is completing the task successfully.

It would be incorrect to report only “P does better” based on the subgroup rates, or only “Q does better” based on the pooled rates, without saying which comparison is being described. Each statement leaves out the level of comparison. A complete description names the outcome, the groups, and whether the rate is within a particular setting or pooled across settings.

Reading a Reversal Carefully

A useful way to read a possible Simpson’s paradox is to keep three questions separate:

A reversal should be described only when the rates support it. If one group is higher in some subgroups and lower in others, but the pooled direction differs from only some of them, that is not the defining pattern given here. Likewise, a difference in raw outcome counts is not enough: the relevant quantities are rates with the appropriate denominators.

Common Mistake

• Wrong reasoning: “Group P has the higher rate in every setting, so P must have the higher overall rate.” This assumes the subgroup comparisons automatically determine the pooled comparison.
• Correction: Calculate the pooled rate from the combined outcome counts and combined group totals. The subgroup sizes can differ between groups, so the pooled comparison can point in the opposite direction. Report the subgroup and pooled results distinctly.

Another mistake is to describe the reversal as proof that one comparison is the “real” one. The table supports multiple descriptive comparisons, each with its own denominator and scope. Deciding which comparison is relevant depends on the question being asked; the arithmetic alone does not settle that question.