DFFITS and Casewise Prediction Changes

Why Look at a Casewise Prediction Change?

A regression fit can be sensitive to particular observations. One way to describe that sensitivity is to ask what happens to the fitted value at an observation’s predictor settings when that observation is removed and the model is refitted. DFFITS expresses this casewise prediction change on a scale that accounts for residual uncertainty and leverage.

DFFITS is an influence diagnostic, not a test of whether an observation is erroneous or should be removed. Its sign gives a direction of change; its magnitude indicates how large the change is relative to its scale. The calculation is for the fitted value at the case’s predictors, not for the observed response itself.

Definition and Model Conditions

Consider an ordinary least-squares regression with \(n\) observations and \(p\) fitted coefficients, including an intercept if the model has one. Write \(y_i\) for the observed response, \(\hat{y}_i\) for the fitted value from the full data, \(e_i=y_i-\hat{y}_i\) for the residual, and \(h_{ii}\) for the observation’s diagonal hat value. Let \(\hat{y}_{(i)}\) be the fitted value at observation \(i\)’s predictor settings after observation \(i\) is omitted and the model is refitted. Let \(s_{(i)}\) be the residual standard error from that deleted fit.

The conventional DFFITS calculation uses an unweighted, full-rank OLS model. The usual model conditions are a correctly specified linear mean, independent errors with mean zero and common variance, and a design for which deleting the case leaves the model estimable. Normal errors are additionally used for exact finite-sample distributional results for studentized quantities; the DFFITS calculation itself is used descriptively.

The factor \(s_{(i)}\) measures residual uncertainty in the deleted fit. The factor \(\sqrt{h_{ii}}\) makes the scale specific to the observation’s leverage. DFFITS therefore is not simply a raw change in response units: it describes that change relative to a leverage-scaled residual standard error.

Reading the Sign and Magnitude

If DFFITS is positive, then \(\hat{y}_i>\hat{y}_{(i)}\): including the observation produces a higher fitted value at its predictor settings than omitting it does. If DFFITS is negative, the full-data fitted value is lower. Because \(s_{(i)}\sqrt{h_{ii}}\) is positive, the sign of DFFITS is the sign of \(\hat{y}_i-\hat{y}_{(i)}\).

A value near zero indicates a small casewise fitted-value change relative to its scale. A larger absolute value indicates a more substantial standardized change. This is a diagnostic description, not a p-value or a universal judgment that a case is problematic.

A Useful Formula and Its Justification

The OLS case-deletion relation gives the difference between the two fitted values in terms of the full-fit residual and leverage: $$ \hat{y}_i-\hat{y}_{(i)}=\frac{h_{ii}e_i}{1-h_{ii}}. $$ Substituting this relation into the definition gives a convenient equivalent expression. The externally studentized deleted residual is \(t_i=e_i/[s_{(i)}\sqrt{1-h_{ii}}]\), so the same DFFITS value can be written in terms of that residual.

This shows why residual size alone does not determine DFFITS: the studentized deleted residual is multiplied by a factor that grows with leverage. It also shows that, when the quantities are defined, DFFITS has the same sign as the residual \(e_i\). The displayed relation is useful for calculation and interpretation; DFFITS itself remains the standardized change in the fitted value.

Magnitude and Screening

A commonly used screening guideline flags cases for closer examination when $$ |\operatorname{DFFITS}_i|>2\sqrt{\frac{p}{n}}, $$ where \(p\) is the number of fitted coefficients and \(n\) is the full sample size. This is a rule of thumb, not a significance test: it does not give a probability that a case is influential, and it is not a rule for deleting observations. The guideline’s value depends on the model size and sample size, so the cutoff should be reported with those quantities if used.

Leverage Can Amplify a Prediction Change

Two cases with similar residual sizes can have different DFFITS values because their leverages differ. The following hypothetical diagnostic summaries illustrate the calculation. In each row, the reported full-fit residual, leverage, and deleted-fit residual standard error are used with the same deletion relation.

For case A, the change is \(0.64(0.50)/(1-0.64)=0.889\) response units, and DFFITS is \(0.889/[3.00\sqrt{0.64}]=0.370\). For case B, the change is \(0.04(3.00)/(1-0.04)=0.125\) response units, and DFFITS is \(0.125/[3.00\sqrt{0.04}]=0.208\). Case B has the larger residual, but case A has the larger DFFITS value because its greater leverage amplifies the casewise prediction change.

Conditions, Limits, and a Common Pitfall

The definition requires a meaningful full and deleted fit and a positive deleted-fit residual standard error. If \(h_{ii}=1\), the factor \(1-h_{ii}\) in the deletion relation is zero; deleting that case may make the fit or prediction unavailable. Such a case does not have an ordinary finite DFFITS value from this formula. More generally, leverage close to one can make the deletion change very large, so calculations should be checked against the design and the actual refitted model.

A common mistake is to interpret a large absolute DFFITS value as proof that the observation is a data error and then remove it. That reasoning confuses a diagnostic flag with a decision. DFFITS identifies a case whose inclusion is associated with a comparatively large change in its own fitted value. The case may be valid, and its influence may be relevant to the scientific question. Check its provenance and measurement, examine the model context, and report the sensitivity responsibly rather than deleting it solely because of a cutoff.