DFBETAS for Case Influence on Individual Coefficients

Why Measure Influence One Coefficient at a Time?

A regression case can affect a fitted model in more than one way. Its removal may change the fitted value at that case, the overall coefficient vector, or the uncertainty of the estimates. DFBETAS focuses on a particular question: how much does deleting one observation change one specified coefficient? This is useful when the meaning of a single slope matters, even if the case has only a modest effect on the model’s fitted values overall.

The same observation need not affect every coefficient equally. Its predictor values determine how it relates to the design columns, while the residual indicates how poorly it agrees with the fitted response. DFBETAS combines these features into a separate standardized change for each coefficient.

Definition: Consider an ordinary least squares regression with $n$ observations and $p$ estimated coefficients, including the intercept. Let $\hat{\beta}_j$ be the full-data estimate of coefficient $j$, and let $\hat{\beta}_{j(i)}$ be the estimate after deleting observation $i$. Define $C=(X'X)^{-1}$, where $X$ is the full design matrix, and let $s_{(i)}^2=\operatorname{SSE}_{(i)}/(n-p-1)$ be the residual mean square from the deletion fit. Then $$ \operatorname{DFBETAS}_{j(i)} = \frac{\hat{\beta}_j-\hat{\beta}_{j(i)}}{s_{(i)}\sqrt{C_{jj}}}. $$ This standard definition applies when the design matrix has full column rank and the deletion fit has positive residual degrees of freedom. The numerator uses full-data estimate minus deletion estimate.

The denominator $s_{(i)}\sqrt{C_{jj}}$ is the estimated standard error of coefficient $j$ using the deleted-model residual scale but the full design matrix’s coefficient geometry. Thus, DFBETAS expresses the coefficient change in estimated standard-error units. It is not the raw change in the coefficient, and it is not a standardized residual.

The sign is informative. A positive value means the full-data coefficient is larger than the coefficient after deletion; the observation’s inclusion has pulled that coefficient upward relative to the deletion fit. A negative value means inclusion has pulled it downward. The absolute value describes the size of the standardized change.

Deriving the Case-Deletion Formula

For computation and interpretation, it helps to connect the coefficient change to familiar regression quantities. Write $x_i$ for the column vector of predictors for observation $i$, including a 1 for the intercept, and $e_i=y_i-x_i'\hat{\beta}$ for its full-data residual. Let $h_{ii}=x_i'(X'X)^{-1}x_i$ be its leverage.

The deletion estimate satisfies the normal equations for all cases except $i$. Since removing the case subtracts $x_ix_i'$ from $X'X$ and $x_iy_i$ from $X'y$, subtracting the full and deletion normal equations gives

(X'X)(\hat{\beta}-\hat{\beta}_{(i)}) = x_i\bigl(y_i-x_i'\hat{\beta}_{(i)}\bigr).

The quantity in parentheses is the residual for case $i$ when the model is fitted without that case. Call it $r_{i(i)}=y_i-x_i'\hat{\beta}_{(i)}$. Multiplying the normal-equation difference by $C=(X'X)^{-1}$ gives $\hat{\beta}-\hat{\beta}_{(i)}=Cx_i r_{i(i)}$. The full-data residual obeys

e_i = y_i-x_i'\hat{\beta} = r_{i(i)}-x_i'Cx_i r_{i(i)} = (1-h_{ii})r_{i(i)}.

Therefore, provided $h_{ii}<1$, the case-deletion coefficient change is

\hat{\beta}-\hat{\beta}_{(i)} = \frac{Cx_i e_i}{1-h_{ii}}.

This identity explains why a large residual alone does not determine coefficient influence. The direction and size of $Cx_i$ matter too, and the factor $1/(1-h_{ii})$ increases the change as leverage approaches 1. The identity is an algebraic result for a full-rank least squares model; it does not require normally distributed errors. If deleting a case destroys full rank, or leaves no residual degrees of freedom for estimating $s_{(i)}$, the usual DFBETAS value is not defined.

Worked Example: One Case Changes the Intercept and Slope Differently

A hypothetical simple regression relates a portfolio’s annual servicing cost $y$, in thousands of dollars, to the number of active accounts $x$, measured in thousands. The five observations have $x=0,1,2,3,4$ and $y=3,1,4,7,5$. Calculate the DFBETAS values for observation 5. Use the usual intercept-and-slope model.

Method and full fit. For all five observations, $\bar{x}=2$ and $\bar{y}=4$. Also, $S_{xx}=\sum (x_i-\bar{x})^2=4+1+0+1+4=10$. The slope is $ \hat{\beta}_1=\sum (x_i-\bar{x})(y_i-\bar{y})/S_{xx}=10/10=1$, and the intercept is $\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}=4-2=2$. The fitted line is $\hat{y}=2+x$, with residuals $1,-2,0,2,-1$. In particular, $e_5=-1$.

Deletion fit. After deleting observation 5, the remaining $x$-values are $0,1,2,3$, with mean $1.5$; the remaining $y$-values have mean $3.75$. Their cross-product about the means is $7.5$, and their $S_{xx}$ is $5$, so the deletion slope is $7.5/5=1.5$. The deletion intercept is $3.75-1.5(1.5)=1.5$. Thus, $\hat{\beta}_{0(5)}=1.5$ and $\hat{\beta}_{1(5)}=1.5$.

For the deletion fit, the four fitted values are $1.5,3,4.5,6$. The residuals are $1.5,-2,-0.5,1$, so $\operatorname{SSE}_{(5)}=2.25+4+0.25+1=7.5$. There are $4-2=2$ residual degrees of freedom, giving $s_{(5)}^2=7.5/2=3.75$ and $s_{(5)}=\sqrt{3.75}$.

Coefficient standardizers. In a simple regression with an intercept, $C_{00}=1/n+\bar{x}^2/S_{xx}=1/5+4/10=0.6$, and $C_{11}=1/S_{xx}=0.1$. The intercept and slope changes, in the defined full-minus-deletion order, are $2-1.5=0.5$ and $1-1.5=-0.5$. Therefore,

\operatorname{DFBETAS}_{0(5)} =\frac{0.5}{\sqrt{3.75}\sqrt{0.6}} =\frac{0.5}{1.5} =0.3333, \qquad \operatorname{DFBETAS}_{1(5)} =\frac{-0.5}{\sqrt{3.75}\sqrt{0.1}} \approx -0.8165.

Check and interpretation. Deleting the case lowers the intercept by $0.5$ and raises the slope by $0.5$, consistent with the signs of the two DFBETAS values. The standardized slope change is larger in magnitude, even though the raw coefficient changes have equal magnitude. The calculation avoids treating the two coefficient changes as equally important without accounting for their different standardizers. Values are rounded to four decimal places.

Computing Several Coefficient Effects from One Case

For multiple regression, the case-deletion identity provides all coefficient changes at once. The vector $Cx_i e_i/(1-h_{ii})$ is full estimate minus deletion estimate. To obtain DFBETAS, divide component $j$ by $s_{(i)}\sqrt{C_{jj}}$. This makes it possible for one observation to have a strong effect on one slope and a weaker effect on another.

Worked Example: A Case Has Unequal Effects in a Multiple Regression

A hypothetical model predicts a renewal measure from two predictors, $x_1$ and $x_2$, with an intercept. There are six observations. The design columns are orthogonal, with $X'X=\operatorname{diag}(6,10,12)$. The fitted coefficients are $(5,2,-1)$, and the full-data residuals are $(1,0,-4,1,2,0)$. For observation 1, the predictor vector is $x_1=(1,-2,1)'$. Calculate its DFBETAS for all three coefficients.

Check the inputs and leverage. The residuals sum to zero and have zero cross-product with each predictor column, so they are consistent with the stated least squares fit. Their squared sum is $1+0+16+1+4+0=22$. Since $C=(X'X)^{-1}=\operatorname{diag}(1/6,1/10,1/12)$, the leverage for observation 1 is $h_{11}=1/6+(-2)^2/10+1^2/12=0.65$. Its residual is $e_1=1$.

Coefficient changes. Multiplying $C$ by the case’s predictor vector gives $Cx_1=(1/6,-1/5,1/12)'$. Since $1-h_{11}=0.35=7/20$,

\hat{\beta}-\hat{\beta}_{(1)} = \frac{Cx_1e_1}{1-h_{11}} = \left(\frac{1}{7},-\frac{4}{7},\frac{5}{21}\right)'.

Thus the deletion coefficients are $(5-1/7,\;2-(-4/7),\;-1-5/21)=(34/7,\;18/7,\;-26/21)$. The deleted residual sum of squares is $\operatorname{SSE}_{(1)}=22-e_1^2/(1-h_{11})=22-20/7=134/7$. The deletion fit has $6-1-3=2$ residual degrees of freedom, so $s_{(1)}^2=67/7$ and $s_{(1)}=\sqrt{67/7}\approx3.0938$.

Standardize each change. The three diagonal entries of $C$ are $1/6,1/10,1/12$. Consequently,

\begin{aligned} \operatorname{DFBETAS}_{0(1)} &=\frac{1/7}{\sqrt{67/7}\sqrt{1/6}} \approx 0.1131,\\ \operatorname{DFBETAS}_{1(1)} &=\frac{-4/7}{\sqrt{67/7}\sqrt{1/10}} \approx -0.5841,\\ \operatorname{DFBETAS}_{2(1)} &=\frac{5/21}{\sqrt{67/7}\sqrt{1/12}} \approx 0.2666. \end{aligned}

Check and interpretation. The signs agree with the changes: deleting the case lowers the intercept, raises the first slope, and lowers the second slope. The largest absolute standardized change is for the first slope. This example shows why a case should not be described as simply “influential” without specifying which coefficient is affected. Values are rounded to four decimal places.

Using DFBETAS as a Diagnostic

A common screening guideline flags a case when at least one coefficient-specific value has absolute magnitude greater than $2/\sqrt{n}$. This is a heuristic, not a significance test or a universal boundary. Its value depends on sample size, and it should direct attention to a case rather than determine whether that case is removed. The number of coefficients also matters: checking many coefficients increases the chance that at least one value crosses a screening threshold.

Worked Example: Applying a Screening Guideline Without Treating It as a Rule

A hypothetical three-coefficient regression uses $n=25$ records. For one record, the full estimates are $(0.80,1.20,-0.40)$, and the deletion estimates are $(0.70,1.05,-0.50)$. The deletion residual scale is $s_{(i)}=2$, and the relevant diagonal entries of $C$ are $(0.04,0.01,0.09)$. Calculate the three DFBETAS values and apply the $2/\sqrt{n}$ screening guideline.

Coefficient changes. Full minus deletion gives $0.10$, $0.15$, and $0.10$. The standardizers are, respectively, $2\sqrt{0.04}=0.40$, $2\sqrt{0.01}=0.20$, and $2\sqrt{0.09}=0.60$. The resulting values are $0.10/0.40=0.25$, $0.15/0.20=0.75$, and $0.10/0.60=0.1667$.

Screen and interpret. The guideline is $2/\sqrt{25}=0.40$. Only the second coefficient’s absolute DFBETAS, $0.75$, exceeds that guideline. It indicates a relatively large standardized change for that coefficient, not proof of a model error or a reason to discard the observation. Values are rounded to four decimal places.

Common Mistakes and Practical Interpretation

Using the wrong sign order. This definition uses $\hat{\beta}_j-\hat{\beta}_{j(i)}$. Reversing the subtraction reverses the sign, so state the convention before interpreting direction.
Reporting the raw coefficient change as DFBETAS. A change of $0.5$ has no standardized interpretation until it is divided by $s_{(i)}\sqrt{C_{jj}}$.
Using a full-data residual scale without checking the definition. The standard DFBETAS denominator uses the deletion-fit residual scale $s_{(i)}$. Substituting another scale produces a different measure.
Assuming a large residual guarantees coefficient influence. The case’s predictor configuration and leverage also affect the coefficient change. Conversely, an unusual predictor configuration can change a coefficient even when the response residual is not especially large.
Automatically deleting a flagged case. First check data accuracy, the case’s context, and whether the model is appropriate. A valid observation may identify a limitation of the model rather than an error in the data.

DFBETAS is most useful when the coefficient itself has a substantive interpretation—for example, a slope representing the association between a rating variable and a response. Compare both the signed values and their magnitudes across coefficients, then investigate cases that appear influential. The screening guideline is a prompt for that investigation, not a substitute for it. DFBETAS describes coefficient changes; it does not by itself establish that the model assumptions hold or that a coefficient is statistically significant.

Key takeaway: DFBETAS standardizes the change in one coefficient caused by deleting one case. Its sign gives the direction of the full-versus-deleted change, and its magnitude is measured in estimated standard-error units. Because the standardizer and predictor geometry differ by coefficient, the same observation can influence regression coefficients unequally.

Check Your Understanding

Use the stated full-minus-deletion convention. Treat screening guidelines as heuristics, not automatic deletion rules.

A coefficient is $1.8$ in the full fit and $2.0$ after deleting a case. For that deletion fit, $s_{(i)}=0.5$ and $\sqrt{C_{jj}}=0.25$. Calculate the DFBETAS value. (Answer: $-1.6000$)
A regression has $n=100$. Using the $2/\sqrt{n}$ guideline, which of the two absolute DFBETAS values, $0.19$ and $0.24$, exceeds the screening value? (Answer: $0.24$ only; the guideline is $0.20$)
A regression has $n=30$, $p=4$, and for one deletion fit $\operatorname{SSE}_{(i)}=52$. For a coefficient, $\hat{\beta}_j-\hat{\beta}_{j(i)}=0.30$ and $C_{jj}=0.04$. Calculate its DFBETAS value. (Answer: approximately $1.0400$)
For a case, $e_i=0.5$, $h_{ii}=0.5$, and the corresponding row of $Cx_i$ is $(0.2,-0.1,0.3)$. Find the vector $\hat{\beta}-\hat{\beta}_{(i)}$. (Answer: $(0.2,-0.1,0.3)'$)
A full fit has $\operatorname{SSE}=40$. For a deleted case, $e_i=2$ and $h_{ii}=0.5$. Assuming the deletion fit is estimable, calculate $\operatorname{SSE}_{(i)}$. (Answer: $32$)

What You'll Learn