Why Similar Scores Can Hide Dissimilar Donors
Predictive mean matching (PMM) chooses donors by comparing predicted scores, then imputes an observed outcome from a selected donor. The score is useful only to the extent that it organizes cases in a way that is relevant to their conditional outcome distributions. If the regression used to calculate scores misses important structure, two cases can have nearly identical scores even though their expected outcomes, or the variability of those outcomes, differ substantially.
This is a limitation of the ranking step, not a claim that the donor’s observed value should equal the missing value. A missing outcome is unknown, so its case cannot be ranked against donors by the unobserved outcome itself. The practical question is whether the score brings together cases with sufficiently comparable outcome distributions for the imputation goal.
A score need not reproduce every outcome value to be useful. Even a well-specified conditional mean model leaves individual outcomes variable. The specific concern here is that the score may fail to distinguish cases because the fitted mean relationship is wrong, or because the score represents the mean but conceals important differences in spread.
Omitted Curvature Can Collapse the Ranking
Suppose the true conditional mean changes nonlinearly with a predictor \(X\), but the score is calculated from a straight-line regression. The fitted score is then a linear projection of the observed outcomes, not necessarily a good approximation to the conditional mean at every value of \(X\). In particular, different predictor values can receive the same fitted score even when their expected outcomes differ.
For example, if the true mean is quadratic, \(m(x)=\alpha+\beta x+\gamma x^2\), a linear score \(s(x)=a+bx\) cannot generally represent the curvature term. If the fitted slope is small, or if values on opposite sides of a turning point receive similar linear predictions, the score distance can be small while the true mean difference is large. PMM then ranks on a feature of the fitted model that has discarded relevant structure.
Worked Example: A Straight-Line Score Misses a U-Shaped Mean
Consider a hypothetical setting where the true conditional mean is \(m(x)=10+2x^2\). Four observed donors have \(x=-2,-1,1,2\), and their outcomes equal their conditional means: \(18,12,12,18\). A case with \(x=0\) has a missing outcome.
First fit a linear regression \(Y=a+bx+\varepsilon\) to the four donors. The donor means are \(\bar{x}=0\) and \(\bar{y}=15\). The least-squares slope is
Thus the fitted intercept is \(\hat{a}=\bar{y}-\hat{b}\bar{x}=15\), and all four donors and the missing case receive the same linear score, 15. Their score distances are all \(|15-15|=0\), so the linear score offers no ranking among them. Yet the true conditional mean for the missing case is \(m(0)=10\), whereas the donors’ conditional means are 18, 12, 12, and 18. The linear score has hidden the curvature that distinguishes these cases.
This example isolates model misspecification: the missing case lies within the range of the donor predictors, and the ranking failure occurs because the fitted score is linear while the mean relationship is curved. Ties do not mean donors are equivalent; they mean this score cannot distinguish them.
Changing Variance Is Also Hidden by a Mean Score
A conditional mean score describes the center of an outcome distribution, not its spread. Even when the mean model is correct, cases with the same predicted mean can have different conditional variances. If those differences matter for the analysis, matching only on the mean score can pool donors whose outcome distributions are not comparable.
This point requires care. A donor with an outcome far from the predicted mean is not automatically a bad donor: a valid draw from a high-variance conditional distribution can be far from its mean. Conversely, a donor close to the mean is not automatically suitable if the missing case belongs to a much more variable group. Suitability concerns the distribution represented by the donor pool, not whether one donor happens to match an unknowable missing value.
Worked Example: Equal Means, Different Conditional Spread
Suppose, hypothetically, that \(Y\) has conditional mean \(10+2X\), while its conditional variance depends on a recorded group indicator \(Z\): \(\operatorname{Var}(Y\mid X,Z=0)=1\) and \(\operatorname{Var}(Y\mid X,Z=1)=25\). Consider a missing case with \(X=3,Z=1\). Its mean score from a model containing \(X\) but not a variance-sensitive feature is \(10+2(3)=16\).
Observed donors with \(X=3,Z=0\) also have mean score 16, as do donors with \(X=3,Z=1\). For illustration, suppose the observed outcomes in these two groups are \(15,16,17\) and \(6,20,22\), respectively. Every donor has score distance \(|16-16|=0\) from the missing case. But the \(Z=0\) outcomes cluster near 16, while the \(Z=1\) outcomes are much more dispersed, consistent with the stated variance difference.
If the donor pool mixes these groups, random donor selection can fail to represent the wider conditional spread for the missing case. The score has not made an arithmetic error: it has represented the mean and failed to encode a relevant distributional difference. The example does not imply that every high-variance donor should be chosen; it shows why equal mean scores alone cannot justify treating the groups as interchangeable.
Diagnosing a Poor Score Ranking
The ranking itself is based on predicted scores, so diagnostics should examine how those scores relate to observed outcomes and predictors among cases whose outcomes are observed. Useful checks include plotting observed \(Y\) against each important predictor and against fitted scores, looking for systematic curvature, and examining whether residual spread changes across fitted values or across recorded groups. A pattern in the conditional mean suggests a missing nonlinear term; changing spread suggests that a mean-only score may conceal heteroscedasticity.
A practical check is to temporarily mask observed outcomes, calculate scores as if those outcomes were missing, select donors using the intended PMM procedure, and compare the resulting imputations with the masked values. Repeating this over many observed cases can reveal systematic errors, such as under-representing large outcomes in a curved region or compressing variability in a high-variance group. This check is informative only to the extent that the observed cases used for masking resemble the cases with missing outcomes.
Inspecting donor distances is also useful, but it has limits. A small distance only says the fitted scores are close; it does not certify that the score is correctly specified. Repeated ties, or a range of very different observed donor outcomes among near-ties, are prompts to inspect the model and relevant predictors rather than proof of failure by themselves.
Plot observed outcomes against predictors and fitted scores. Look for curvature, thresholds, or interactions that a straight-line mean model cannot represent.
Compare residual variability across fitted values and important recorded groups. A changing spread may be hidden even when the average pattern looks adequate.
Mask observed outcomes, apply the planned donor selection, and assess errors and conditional distributions across relevant predictor regions.
Where supported by the data and subject matter, consider a more appropriate mean specification, such as a justified transformation or nonlinear term, then repeat the checks.
What to Change—and What Not to Conclude
When curvature is visible, a score model that represents it may improve donor ranking. For example, a justified quadratic term or other flexible mean specification can distinguish cases that a straight line collapses together. The revised model should be checked rather than assumed to solve the problem: it may still omit relevant predictors, misrepresent the spread, or produce unstable predictions.
When variance differences matter, adding predictors to the mean equation is not automatically a solution if those predictors affect spread but not the mean. The analyst should assess whether the donor pool represents the conditional variability relevant to the missing cases and to the intended analysis. Any modification should respect the design and substantive meaning of the predictors; creating arbitrary matching groups can discard information rather than improve comparability.
Other approaches have different trade-offs. A simpler linear score is transparent and may be adequate when the observed conditional mean is approximately linear and relevant variance differences are minor. A more flexible score can capture curvature but may be less stable when data are limited. A parametric imputation model specifies a conditional distribution more directly, but relies on its distributional assumptions. None of these choices is universally best; the key comparison is whether the method represents the features of the conditional distribution needed for the analysis.
A frequent mistake is to reason, “These two cases have almost identical predicted means, so their outcomes should be similar and either is a suitable donor.” The first part confuses a fitted center with an outcome distribution; the second ignores model misspecification and changing variance. Correct reasoning is narrower: similar scores establish proximity on the model’s chosen scale. Suitability also depends on whether that scale captures the important mean structure and whether the donor pool reflects relevant conditional spread.
Worked Example: Checking Whether a Curved Score Improves Ranking
Return to the hypothetical quadratic relationship \(m(x)=10+2x^2\), with donors at \(x=-2,-1,1,2\), outcomes \(18,12,12,18\), and a missing case at \(x=0\). The linear fit gave score 15 for every case, so all donor distances were zero. Now use the correctly specified quadratic mean score \(s(x)=10+2x^2\).
The missing case receives score \(s(0)=10\). The donor scores are 18, 12, 12, and 18, giving distances \(8,2,2,8\). The nearest donors are now those at \(x=-1\) and \(x=1\), each with observed outcome 12. In this deliberately noise-free illustration, their outcomes are 2 units above the missing case’s true mean of 10; the donors at \(x=-2\) and \(x=2\) have outcomes 8 units above it.
The revised score separates donors in a way consistent with the stipulated mean curve. This is not proof that a quadratic model is appropriate in real data: the specification must be supported by diagnostics and subject-matter reasoning, and residual variation remains relevant. It demonstrates precisely what a better score can accomplish: restore distinctions that the misspecified linear prediction erased.
Check Your Understanding
Answer each question, then compare with the concise solutions below.
- If two cases have identical fitted mean scores, what has PMM established about their outcomes?
- A plot of observed outcomes against a predictor shows a U-shaped pattern, but the imputation model uses only a straight line. What failure can follow for PMM donor selection?
- For a missing case with score 20, two donors have scores 19.8 and 20.2. Does this establish that their outcomes are suitable substitutes? Why or why not?
- What does a masked-observed-outcome check evaluate, and what limitation should be kept in mind?
Solutions. (1) Only that the cases are tied on the fitted score; it does not establish equal conditional means under the true process or equal outcome distributions. (2) Different predictor values can receive similar linear scores despite different conditional means, so the donor ranking may ignore curvature. (3) No. Their score distances are both 0.2, but the score alone does not establish that the mean model is adequate or that their conditional spreads are comparable. (4) It evaluates how the donor procedure performs when known outcomes are treated as missing; its results may not generalize if cases with observed outcomes differ from those with missing outcomes.