Why Look at Influence Coefficient by Coefficient?
A case can matter to a regression fit in more than one way. Deleting it might noticeably change a predicted response, or it might change a particular slope while leaving other coefficients nearly unchanged. DFBETAS focuses on the latter question: how much does deleting one case change one specified coefficient, measured on a standardized scale?
This makes DFBETAS useful when the interpretation of a particular coefficient is central. For example, a case might have little effect on the estimated coefficient for one predictor but substantially affect the coefficient for another. The sign also matters: it indicates whether the coefficient estimate from the full data is numerically higher or lower than the estimate after deletion.
Definition and Model Conditions
Consider an ordinary least-squares regression with \(n\) cases and \(p\) fitted coefficients, including the intercept if the model has one. Write the model as \(Y=X\beta+\varepsilon\), where \(X\) is the design matrix and has full column rank. The ordinary least-squares estimate from all cases is \(\hat{\beta}\). After deleting case \(i\), let the estimate be \(\hat{\beta}_{(i)}\). The subscript \((i)\) indicates that case \(i\) has been left out.
For the conventional DFBETAS calculation, the deleted design must still have full column rank, so the deleted model's coefficients can be estimated. Its residual degrees of freedom are \(n-p-1\), which must be positive to estimate the residual variance in the deleted fit. Let \(s_{(i)}\) be the residual standard error from that fit, and let \(c_{jj}\) be the \(j\)th diagonal element of \((X^\mathsf{T}X)^{-1}\), calculated from the full design.
The regression model used for this diagnostic is an ordinary least-squares model with a full-rank design. The conventional variance interpretation of its scale uses the usual homoskedastic, uncorrelated-error model, \(\operatorname{Var}(\varepsilon\mid X)=\sigma^2 I\). Normality is not needed to calculate DFBETAS, and DFBETAS is not itself a hypothesis test. It is a diagnostic summary of sensitivity to deleting one case.
How to Read the Sign and Size
The sign follows directly from the order in the numerator: full fit minus deleted fit. A positive value means \(\hat{\beta}_j\) is greater than \(\hat{\beta}_{j(i)}\); including case \(i\) is associated with a higher estimate of that coefficient than deleting the case. A negative value means the full-fit estimate is lower. “Higher” and “lower” refer to numerical order, not to the magnitude of the coefficient: for negative coefficients, a higher value can be less negative.
The magnitude measures the coefficient change relative to the standardizing scale \(s_{(i)}\sqrt{c_{jj}}\). A DFBETAS of \(0.5\), for example, means the full-minus-deleted coefficient change is half that scale. A DFBETAS of \(-2\) means the change is negative and has magnitude twice the scale. Neither value is a probability, a p-value, nor proof that a case is erroneous.
Each coefficient has its own DFBETAS value for each case. A coefficient-by-case table therefore has coefficients as rows and cases as columns, or vice versa. Its entries answer a specific question: which cases most affect this particular coefficient, and in which direction?
Why the Standardizing Scale Is Used
The raw difference \(\hat{\beta}_j-\hat{\beta}_{j(i)}\) has the units of coefficient \(j\), so its size alone can be hard to compare across coefficients measured on different scales. The denominator expresses the change relative to a regression-based coefficient scale. To see its connection with coefficient uncertainty, under the classical homoskedastic model the variance of the full-fit coefficient estimate is
Replacing the unknown \(\sigma\) by the deleted-fit residual standard error \(s_{(i)}\) gives the scale \(s_{(i)}\sqrt{c_{jj}}\). Thus DFBETAS expresses the deletion change in units of a coefficient standard-error scale. The full-design value \(c_{jj}\) is held fixed across cases, so the standardizing scale does not change from one deleted case to another except through \(s_{(i)}\). This is a definition of a diagnostic, not a claim that DFBETAS has a standard normal distribution.
Worked Example: Calculating a DFBETAS Value
Suppose a hypothetical model predicts monthly production in units from machine age, measured in years. The full fit gives an estimated age coefficient of \(1.80\) production units per year. After deleting case 7, the estimate is \(1.56\). For this deletion, suppose \(s_{(7)}=1.60\) production units and \(\sqrt{c_{jj}}=0.20\) per year. These are assumed summary values from the two fits.
The full-minus-deleted coefficient change is \(1.80-1.56=0.24\) production units per year. The standardizing scale is \(1.60(0.20)=0.32\) production units per year. Therefore,
The positive sign means the full-fit age coefficient is higher than the deleted-fit coefficient. In this example, including case 7 is associated with an age coefficient \(0.24\) production units per year higher than the estimate after deleting it. The standardized change is \(0.750\) of the stated coefficient scale.
Worked Example: Interpreting a Negative Coefficient Correctly
Suppose a hypothetical regression estimates the association between machine age and monthly downtime, in hours. In the full fit, the machine-age coefficient is \(-0.35\) hours per month per year of age. Deleting case 19 gives an estimate of \(-0.61\). Let \(s_{(19)}=0.52\) hours and \(\sqrt{c_{jj}}=0.25\) per year.
The numerator is \(-0.35-(-0.61)=-0.35+0.61=0.26\) hours per month per year. The denominator is \(0.52(0.25)=0.130\) in the same coefficient units. Thus,
The positive value is consistent with the arithmetic: the full estimate, \(-0.35\), is higher (less negative) than the deleted-fit estimate, \(-0.61\). Including case 19 is associated with a less negative machine-age coefficient; deleting it makes the estimate more negative. It would be incorrect to say that the full estimate is lower or more negative.
Reading a Coefficient-by-Case Table
The next hypothetical example compares two coefficients across two deleted cases. The model contains machine age and distance travelled as predictors of monthly maintenance cost. For clarity, the table gives each case's full-minus-deleted change, its standardizing denominator, and the resulting DFBETAS. The denominator for each entry is \(s_{(i)}\sqrt{c_{jj}}\) for that case and coefficient.
Worked Example: Finding Direction-Specific Influence in a Table
Suppose there are \(n=25\) cases. In the full fit, the age coefficient is \(0.80\) cost units per year, and the distance coefficient is \(-1.20\) cost units per thousand kilometres. The following hypothetical deleted-fit summaries are given:
| Case and coefficient | Full estimate | Deleted estimate | Denominator | DFBETAS |
|---|---|---|---|---|
| Case A: age | 0.80 | 0.68 | 0.060 | 2.00 |
| Case A: distance | -1.20 | -1.23 | 0.150 | 0.200 |
| Case B: age | 0.80 | 0.77 | 0.060 | 0.500 |
| Case B: distance | -1.20 | -0.75 | 0.150 | -3.00 |
For case A and age, the full-minus-deleted change is \(0.80-0.68=0.12\), so \(0.12/0.060=2.00\). For case A and distance, the change is \(-1.20-(-1.23)=0.03\), giving \(0.03/0.150=0.200\). Case A is more influential for the age coefficient than for the distance coefficient in these standardized comparisons.
For case B and age, \(0.80-0.77=0.03\), and \(0.03/0.060=0.500\). For case B and distance, \(-1.20-(-0.75)=-0.45\), so \(-0.45/0.150=-3.00\). The negative sign indicates that including case B lowers the distance coefficient relative to the deleted fit. Since the coefficient is negative, the full estimate of \(-1.20\) is more negative than the deleted estimate of \(-0.75\).
A common screening guideline flags values with magnitude greater than \(2/\sqrt{n}\). Here, \(2/\sqrt{25}=2/5=0.400\). By this guideline, case A's age value, case B's age value, and case B's distance value exceed the cutoff in magnitude; case A's distance value does not. This is a screening rule, not a significance test or an automatic reason to remove a case.
Using DFBETAS Responsibly
DFBETAS helps focus attention on which coefficient is sensitive to which case. A case with a large absolute DFBETAS for one coefficient need not have a large value for every other coefficient. This coefficient-specific view can clarify a result when one slope is substantively important, or when the scientific interpretation depends on the stability of a particular estimate.
The screening guideline \(2/\sqrt{n}\) is only a rule of thumb. It does not account for every feature of a data set, and a value below the guideline does not guarantee that a case is unimportant. A flagged value should prompt examination of the observation, the fitted model, and the coefficient's role in the analysis. Whether to retain a case depends on the validity of the data and the goals of the analysis, not on the diagnostic value alone.
Coefficient values also depend on how predictors are coded and scaled. For example, changing a predictor from years to months changes the numerical value of its slope. DFBETAS uses a coefficient-specific scale, but its interpretation still belongs to the stated model and coding. If the regression analysis uses a different fitting or variance procedure, the conventional ordinary-least-squares DFBETAS definition may not capture that procedure's uncertainty.
Common Mistakes
- Reversing the subtraction. The numerator is full fit minus deleted fit. A positive DFBETAS means the full estimate is numerically higher, not lower.
- Misreading negative coefficients. For a negative coefficient, a higher estimate is less negative. Compare the numbers on the number line rather than using “higher” to mean “larger absolute value.”
- Treating one case's value as a summary for the whole model. Each DFBETAS refers to one case and one coefficient; inspect the relevant row or column.
- Treating the guideline as a verdict. Exceeding \(2/\sqrt{n}\) flags a case for review; it does not establish an error, prove that the case should be deleted, or test a null hypothesis.
Check Your Understanding
Use the definition and sign convention to answer these questions.
- A full-fit coefficient is \(4.2\), and the estimate after deleting case 3 is \(4.5\). Is the DFBETAS numerator positive or negative, and what does its sign say about including case 3?
- A full-fit coefficient is \(-0.40\), while the deleted-fit coefficient is \(-0.65\). Which estimate is higher? In which direction does deleting the case move the coefficient?
- If the coefficient change is \(0.18\) and its standardizing denominator is \(0.06\), calculate DFBETAS and interpret its magnitude in standardizing-scale units.
- Why can one case have a large absolute DFBETAS for one predictor's coefficient but a small value for another coefficient?
- For \(n=100\), calculate the guideline \(2/\sqrt{n}\). Explain why exceeding it does not, by itself, justify deleting a case.