Why Audit Methods on the Same Data?
A completed dataset can look ready for analysis while concealing how its missing values were supplied. Comparing methods on the same records makes the consequences visible: the completed-data variance and correlation may change substantially even though the observed values have not. This audit uses a small hypothetical dataset to calculate those changes directly. It illustrates possible distortions; its numbers are not evidence that any one method will always behave in the same way.
There are six records. \(X\) is observed for all six, while \(Y\) is missing for three. The observed pairs are \((0,1)\), \((1,4)\), and \((5,10)\). For the audit, the rows with missing \(Y\) have \(X\) values 2, 3, and 4.
| Record | \(X\) | Observed \(Y\) |
|---|---|---|
| 1 | 0 | 1 |
| 2 | 1 | 4 |
| 3 | 2 | Missing |
| 4 | 3 | Missing |
| 5 | 4 | Missing |
| 6 | 5 | 10 |
The calculations below use the sample variance, with divisor \(n-1=5\) after each completion, and the ordinary sample correlation between \(X\) and \(Y\). In every completed dataset, \(X\) has mean 2.5 and corrected sum of squares \(S_{XX}=17.5\), so its sample variance is \(17.5/5=3.50\). For \(Y\), and for the cross-product used in the correlation, the audit uses
These are descriptive summaries of each completed dataset. They do not by themselves estimate how much uncertainty the missing values introduce.
Mean Imputation: A Common Reference Point
Mean imputation fills every missing \(Y\) with the mean of the observed \(Y\) values. In this dataset, that observed mean is \((1+4+10)/3=5\). The method’s simplicity is useful for showing how repeated replacement values affect summaries, but it assumes no relationship between the missing values and other information is needed to make the replacements.
Worked Example: Completing with the Observed Mean
The completed \(Y\) values are \(1,4,5,5,5,10\). Their sum is 30, and their squared-value sum is \(1+16+25+25+25+100=192\). Therefore
The cross-product sum is \(0(1)+1(4)+2(5)+3(5)+4(5)+5(10)=99\). Since \(\sum X_i=15\),
Thus, for this particular completion, the sample variance of \(Y\) is 8.40 squared units of \(Y\), and the completed-data correlation is about 0.885. Three identical replacements create a point mass at 5; they do not recreate the spread of the unobserved values.
Regression Imputation: Fitted Values and Added Residuals
Deterministic regression imputation fits a regression of \(Y\) on \(X\) using records with observed \(Y\), then assigns each missing record its fitted conditional mean. Its underlying model is that the conditional mean is adequately represented by a linear function of \(X\). For a prediction to represent a plausible missing value, the model must also be relevant to the missing records; extrapolation or a wrong functional form can produce poor replacements. The observed records here are very few, so the fitted line is an illustration rather than a trustworthy diagnostic of model adequacy.
For these three observed pairs, \(\bar{x}=2\), \(\bar{y}=5\), \(S_{XX}=14\), and \(S_{XY}=24\). The fitted slope is \(24/14=12/7\), and the intercept is \(5-(12/7)(2)=11/7\). The predictions at \(X=2,3,4\) are \(5,47/7,\) and \(59/7\), approximately \(5.00,6.71,\) and \(8.43\).
Worked Example: Deterministic Regression Completion
The completed values are \(1,4,5,47/7,59/7,10\). Their sum is \(246/7\), and their squared-value sum is \(142+5690/49\). Hence
The cross-product sum is \(4+10+141/7+236/7+50=825/7\). Thus
The correlation is very strong in the completed data. This is partly a mechanical consequence of assigning fitted values to the missing rows: those values lie exactly on the fitted line. The observed outcomes still have residual variation, but the imputed ones have none around their fitted values.
Stochastic regression imputation adds a random residual component rather than using only fitted values. This can restore some outcome spread, but one random draw does not automatically represent uncertainty about the regression coefficients or the missing values. The precise draw mechanism matters: a residual draw should reflect an appropriate predictive model and residual distribution, and the fitted model must remain reasonable for the missing records.
Worked Example: One Stochastic Regression Completion
Suppose a hypothetical stochastic procedure produces the missing values \(5.5,4.5,6\). These values are a possible draw, not uniquely determined by the observed data. The full \(Y\) sequence is \(1,4,5.5,4.5,6,10\), with sum 31 and squared-value sum 203.5. Therefore
The cross-product sum is \(0(1)+1(4)+2(5.5)+3(4.5)+4(6)+5(10)=102.5\), so
This particular stochastic completion has more \(Y\)-variance than the mean-imputed completion and less than the deterministic-regression completion. That comparison is conditional on the stated draw; another random draw could give different summaries. A single stochastic completion does not reveal the full range of imputation uncertainty.
Hot-Deck Imputation: Reusing Observed Values
Hot-deck imputation replaces each missing value with a value observed for another record, called a donor. A donor may be selected from the whole observed group or from a group made comparable using auxiliary variables. Its key requirement is that the donor values be relevant to the recipients: within the chosen donor pool or stratum, observed outcomes should plausibly represent the missing outcomes. If matching variables fail to capture important differences, donor reuse can preserve neither the correct distribution nor the relationship with \(X\).
Worked Example: One Hot-Deck Allocation
For illustration, take the observed \(Y\) values 4, 1, and 10 as donors for the missing records at \(X=2,3,4\), respectively. The completed sequence is \(1,4,4,1,10,10\). The sum is 30 and the squared-value sum is 234, giving
The cross-product sum is \(0(1)+1(4)+2(4)+3(1)+4(10)+5(10)=105\). Therefore
Here donor reuse creates repeated outcomes and a different association with \(X\) than the regression-based completions. A different donor allocation could produce different results. The calculation does not establish that this allocation is appropriate; that depends on whether these donors are suitable for these recipients.
Audit Summary, Diagnostics, and Limits
The results can be compared directly. The observed-data correlation, calculated from just the three observed pairs, is \(24/\sqrt{14(42)}\approx0.990\). It is not a full-data benchmark: it uses fewer records and may describe a selected subset. The completed-data summaries are:
| Completion method | \(s_Y^2\) | Completed-data \(r\) |
|---|---|---|
| Mean imputation | 8.40 | 0.885 |
| Deterministic regression | 10.46 | 0.992 |
| Stochastic regression, stated draw | 8.667 | 0.908 |
| Hot deck, stated donor allocation | 16.8 | 0.782 |
These differences show why an audit should record the imputed values and recompute the statistics of interest, rather than describing a method only by its name. They do not rank the methods generally. A stochastic draw or donor allocation is variable, and the methods rely on different assumptions. The earlier material on variance and correlation distortion explains why single-completed-data summaries can be misleading; the present audit makes the differences concrete.
Useful checks include comparing observed and completed distributions, plotting \(Y\) against \(X\) with imputed records marked, examining whether imputed values fall in plausible ranges, and checking whether donor pools or regression predictions are appropriate for recipients. Warning signs include a pile-up at a single mean, unusually tight alignment to the regression line, implausible extrapolations, or a few reused donor values dominating the completed distribution. Such checks can reveal problems, but no diagnostic can verify missing values that were never observed.
The main alternatives have distinct trade-offs. Mean imputation is simple but suppresses variation and ignores relationships with predictors. Deterministic regression uses predictors but removes residual spread from imputed outcomes. Stochastic regression restores some residual variation, yet one draw still leaves uncertainty incompletely represented. Hot deck preserves observed outcome values, but depends on defensible donor matching and may repeat values. All four methods yield one completed dataset here. None of its ordinary sample variances or correlations captures the uncertainty about what the missing values could have been.
Common Mistake
A common error is to conclude that the method with completed-data variance closest to an observed-data variance must be best. The observed-data variance is based on only three outcomes, and its relation to the full-data variance depends on why values are missing. A numerically close result can arise by chance, while a larger or smaller result may reflect either a real feature or a poor imputation. Compare summaries as diagnostics, not as a rule for choosing a method; assess the modeling or donor assumptions and keep the missing-data uncertainty in view.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why is the observed-data correlation not a valid known target for judging the completed-data correlations in this example?
- What feature of deterministic regression completion helps explain its correlation of about 0.992?
- For the stated stochastic completion, verify the cross-product correction used to obtain \(S_{XY}=25\).
- Which assumption is most directly challenged if the hot-deck donors are unlike the recipients after accounting for the matching variables?
Solutions. (1) It is based on only the three records with observed \(Y\); their relationship may differ from that of all six records, and the missing outcomes are unknown. (2) Each imputed value is exactly its fitted regression value, with no residual spread for those rows, which can make the completed data align strongly with the fitted trend. (3) The cross-product sum is 102.5 and the correction is \((15)(31)/6=77.5\), so \(S_{XY}=102.5-77.5=25\). (4) The donor-pool comparability assumption is in doubt: the selected donors may not represent the recipients’ missing outcomes.