When the Overall Pattern Tells a Different Story
A scatterplot can show a clear overall trend even when the pattern within each of its visible groups goes in the opposite direction. The key is that the groups may not be represented in the same proportions across the predictor range. If one group tends to have higher responses than another, a change in group composition can outweigh the within-group pattern in a pooled summary.
This phenomenon is commonly called Simpson’s paradox. The word “paradox” does not mean that arithmetic has failed. Rather, the overall and within-group summaries answer different questions: the overall summary combines groups in the proportions present in the data, while each within-group summary holds the category fixed.
The summaries must be comparable for this description to be meaningful: the response should have the same definition and units in each category, and the predictor values being compared should have a clear interpretation. A reversal in group means or proportions describes an observed pattern; by itself, it does not establish why the pattern occurred.
A Worked Reversal in Group Means
Consider a hypothetical training exercise. The predictor is the number of practice sessions completed, either one or two, and the response is a test score in points. The tasks are classified as either easier or harder. The table reports the number of people and the mean score in each combination.
| Task category | Practice sessions | Number of people | Mean score (points) |
|---|---|---|---|
| Easier | 1 | 90 | 80 |
| Easier | 2 | 10 | 82 |
| Harder | 1 | 10 | 50 |
| Harder | 2 | 90 | 52 |
Worked Example: More Practice, Lower Pooled Mean
Within the easier-task category, the mean score increases from 80 to 82 points, a rise of 2 points. Within the harder-task category, it increases from 50 to 52 points, also a rise of 2 points. What happens when the categories are pooled?
After one session: there are 90 people in the easier category and 10 in the harder category, for 100 people total. The pooled mean is the count-weighted average of the two category means:
After two sessions: there are 10 people in the easier category and 90 in the harder category. The pooled mean is
The pooled mean changes by \(55-77=-22\) points, even though the mean rises by 2 points in each category. The overall comparison combines very different category mixes: after one session, 90% of people are in the higher-scoring category; after two sessions, only 10% are. Thus, the pooled decrease is not evidence that the within-category summaries decreased. It reflects both the within-category changes and the shift in category composition.
This distinction matters when describing a scatterplot with groups. A pooled smoother or a single overall summary reflects the observations as they are mixed in the data. A faceted scatterplot or a plot that encodes the categories can help make within-category patterns visible alongside the overall pattern. Neither view is automatically the uniquely correct one: the appropriate summary depends on whether the question concerns the observed mixture or comparisons within categories.
Why a Reversal Is Possible
The arithmetic behind a pooled mean makes the role of composition explicit. Suppose the predictor has two values, called low and high, and the categories are indexed by \(g\). Let \(\bar{y}_{g,L}\) and \(\bar{y}_{g,H}\) be the category-specific response means at the low and high predictor values. Let \(w_{g,L}\) and \(w_{g,H}\) be the proportions of observations in category \(g\) at those values. Within each predictor value, the weights sum to 1.
Add and subtract \(\sum_g w_{g,H}\bar{y}_{g,L}\) when taking the difference between the pooled means. This gives an exact decomposition:
The first sum represents within-category changes, weighted by the category mix at the high predictor value. The second sum represents a composition term: it depends on how the category weights change and on the category means at the low predictor value. If within-category changes are positive but the composition term is sufficiently negative, the total pooled difference is negative. The decomposition is an algebraic identity, not a claim that one term is a causal effect.
Worked Example: Checking the Two Parts of the Difference
Use the training-exercise summaries above. At two sessions, the category weights are 0.10 for easier tasks and 0.90 for harder tasks. The within-category changes are both 2 points, so the first term in the decomposition is
For the composition term, the easier-task weight changes from 0.90 to 0.10, and the harder-task weight changes from 0.10 to 0.90. Using the low-predictor means, the term is
Adding the two parts gives \(2+(-24)=-22\) points, exactly the pooled difference \(55-77=-22\) points. The positive within-category changes are outweighed by the negative composition term.
Reversals with Response Proportions
The same logic applies when the response is binary, such as whether an application is accepted. The mean of a binary response coded 1 for “yes” and 0 for “no” is the proportion of “yes” responses. Consequently, pooled proportions are weighted averages of category-specific proportions, with weights determined by the numbers of observations in the categories.
Worked Example: Acceptance Rates by Department
Suppose, hypothetically, two departments consider applicants with lower or higher academic index values. In Department A, 72 of 90 lower-index applicants and 9 of 10 higher-index applicants are accepted. In Department B, 1 of 10 lower-index applicants and 18 of 90 higher-index applicants are accepted. Within each department, the acceptance rate is higher for the higher-index group:
For lower-index applicants pooled across departments, 73 of 100 are accepted, so the rate is \(73/100=0.73\), or 73%. For higher-index applicants, 27 of 100 are accepted, so the pooled rate is \(27/100=0.27\), or 27%. The pooled rate is lower for the higher-index group, despite the higher rate within each department.
The mix of departments changes sharply. Of the lower-index applicants, 90% are in Department A; of the higher-index applicants, only 10% are. Since the departments have very different acceptance rates, that change in composition is large enough to reverse the pooled comparison. These rates describe this hypothetical set of applications. They do not, by themselves, explain the departments’ decisions or show that an applicant’s index value caused acceptance.
Holding the Group Mix Fixed
Sometimes a question calls for a comparison using the same category weights at each predictor value. This is called standardization: choose weights that sum to 1, then apply those same weights to each set of category-specific means or proportions. It answers a different descriptive question from the observed pooled comparison. The chosen weights should be stated, because different choices can give different standardized summaries.
Worked Example: Comparing Scores with Equal Category Weights
Return to the training exercise and give the easier and harder categories equal weight, 0.50 each, at both practice levels. With one session, the standardized mean is
With two sessions, it is
Under this equal-weight comparison, the standardized mean rises by \(67-65=2\) points. This agrees with the equal 2-point rise in each category. It does not replace the observed pooled means of 77 and 55 points: those pooled means describe the actual, unequal group mixes. Standardization simply makes the chosen comparison explicit by fixing the weights.
How to Read a Possible Reversal
When a pooled trend and group-specific trends appear to disagree, check the summaries before deciding that a calculation is inconsistent. A useful sequence is to identify the response and predictor, identify the categories, compare the response within each category, and then examine how the category proportions vary over the predictor. Finally, state which comparison is being described: the observed mixture, the within-category patterns, or a comparison standardized to fixed weights.
A reversal can be easy to miss if the plot shows only an overall smoother or if the categories are not visually distinguishable. Conversely, seeing a reversal does not mean the pooled pattern should be ignored. It is a real feature of the observed mixture; it simply does not summarize the same relationship as the category-specific patterns. The question determines which view is relevant.
Check Your Understanding
Use the ideas in this tutorial to answer the following questions.
- In the training example, what are the within-category mean changes, and what is the pooled mean change?
- In the pooled-mean decomposition, what does the composition term depend on?
- Why can a binary response’s mean be interpreted as a proportion?
- In the department example, which department makes up most of the lower-index applicants, and which makes up most of the higher-index applicants?
- What question does an equal-weight standardized mean answer that the observed pooled mean does not?