Coefficient Influence and DFBETAs

Why Look at Coefficient Influence?

A generalized linear model (GLM) diagnostic may indicate that one observation has a large overall effect on the fitted model. A separate question is often more useful: which particular coefficient does that observation influence, and in what direction? For example, a hypothetical insurer might fit a logistic model for whether a policy has a particular claim outcome. An observation could have a modest effect on the fitted model overall yet noticeably change the estimated coefficient for one rating factor.

DFBETAs are coefficient-specific case-deletion diagnostics. For each observation and each coefficient, they compare the estimate from the full data with the estimate obtained after deleting that observation. They help locate observations that drive a modeled effect; they do not, by themselves, determine whether an observation is erroneous or should be removed.

Definition: Fit a GLM to all $n$ observations, and denote the estimated coefficient vector by $\hat{\boldsymbol{\beta}}$. Let $\hat{\boldsymbol{\beta}}_{(i)}$ be the estimate from refitting the same model after deleting observation $i$. Under the sign convention used here, the unstandardized DFBETA for coefficient $j$ and observation $i$ is $$ \operatorname{DFBETA}_{ij}=\hat{\beta}_j-\hat{\beta}_{j(i)}. $$ The standardized DFBETA is $$ \operatorname{DFBETAS}_{ij} =\frac{\hat{\beta}_j-\hat{\beta}_{j(i)}}{\operatorname{SE}_{(i)}(\hat{\beta}_{j(i)})}, $$ where $\operatorname{SE}_{(i)}(\hat{\beta}_{j(i)})$ is the standard error of coefficient $j$ from the deleted-data fit.

Thus, a positive DFBETA means the full-data estimate is larger than the estimate after deleting observation $i$. A negative DFBETA means it is smaller. The standardized value expresses the change in units of the deleted-fit standard error, making the size easier to assess relative to the uncertainty in that coefficient. It does not have the interpretation of a hypothesis-test statistic with a universal reference distribution.

Model Conditions and Standardization

The comparisons assume that the same GLM, predictors, response definition, and estimation method are used for the full fit and every deleted fit. The usual model-based standard errors also rely on the model being estimable: in particular, the coefficient estimates should be finite, the design should identify the coefficients, and the response distribution and mean-variance specification should be appropriate for the chosen GLM. If deleting an observation causes separation in a logistic model or makes a coefficient non-identifiable, the corresponding deleted estimate or DFBETAS may not exist as a finite quantity.

In a GLM, coefficient standard errors are obtained from an estimated covariance matrix based on the model information. When the dispersion parameter $\phi$ is estimated, it contributes to that covariance calculation; when it is fixed at one, as in the usual binomial and Poisson models, the standard errors use that fixed value. Consequently, the deleted-fit standard error must match the model’s dispersion convention. Some software reports a differently standardized diagnostic or reverses the subtraction order. Always check the definition used before comparing reported signs or magnitudes.

Worked Example: A Logistic Coefficient Driven by One Outcome

Suppose a hypothetical study models a binary claim outcome $Y$ for independent policies with a logistic GLM, $$ Y\mid x\sim\operatorname{Bernoulli}(p),\qquad \log\left(\frac{p}{1-p}\right)=\beta_0+\beta_1x, $$ where $x=0$ denotes one policy group and $x=1$ another. Assume the model is correctly specified and the usual model-based covariance estimate is used. The observed counts are 5 outcomes and 5 non-outcomes for $x=0$, and 6 outcomes and 5 non-outcomes for $x=1$. Calculate the DFBETA and DFBETAS for $\beta_1$ when one outcome from the $x=1$ group is deleted.

Method. For this two-group logistic model, the fitted intercept is the log odds in the $x=0$ group, and the fitted slope is the difference in log odds between the groups.

Full fit. The $x=0$ odds are $5/5=1$, so $\hat{\beta}_0=\log(1)=0$. The $x=1$ odds are $6/5$. Therefore, $$ \hat{\beta}_1=\log(6/5)-\log(1)=\log(6/5)\approx 0.1823. $$

Deleted fit. After removing one outcome from the $x=1$ group, its counts are 5 outcomes and 5 non-outcomes. Its odds are $5/5=1$, equal to the $x=0$ odds. Hence $\hat{\beta}_{1(i)}=0$, and $$ \operatorname{DFBETA}_{i1}=\log(6/5)-0=\log(6/5)\approx 0.1823. $$

Standardization. For two independent binomial groups, the estimated variance of the log odds ratio is the sum of the reciprocals of the four cell counts. In the deleted data, all four counts are 5, so $$ \operatorname{SE}_{(i)}(\hat{\beta}_{1(i)}) =\sqrt{\frac15+\frac15+\frac15+\frac15} =\sqrt{0.8} \approx 0.8944. $$ Thus, $$ \operatorname{DFBETAS}_{i1} =\frac{0.1823216}{0.8944272} \approx 0.2038. $$

The positive sign says that the full-data slope is larger than the slope without this outcome. The standardized change is about 0.2038 deleted-fit standard errors. The common mistake is to subtract separately rounded coefficients, such as 0.1823 and 0.0000, and then report a result with unjustified extra precision. The exact change is $\log(6/5)$, which rounds to 0.1823.

Why the Approximation Has This Sign

Refitting the model once for each observation can be computationally demanding. A first-order approximation explains how an individual case’s score contribution relates to the change in the coefficient vector. This approximation is useful for understanding the diagnostic and can be used as a screening calculation, but it is not generally the exact deleted-fit result.

Notation: Let $\mathbf{u}_i$ be observation $i$'s contribution to the full-model score, evaluated at $\hat{\boldsymbol{\beta}}$. Let $\mathcal{I}_{(i)}$ be the information matrix for the model without observation $i$, evaluated near the full-data estimate. The score is the derivative of the log-likelihood; the information is the negative derivative of the score.

At the full-data estimate, the total score is zero, so the deleted-data score evaluated there is the negative of the removed observation’s contribution. A first-order Taylor expansion of the deleted-data score around $\hat{\boldsymbol{\beta}}$ gives the relationship below.

Start from the full score equation.
The full score is zero, so the deleted-data score at the full estimate is $ -\mathbf{u}_i$.

Linearize the deleted-data score.
Its derivative is approximately the negative information matrix. Writing $\boldsymbol{\beta}_{(i)}$ for the deleted-data estimate gives $0\approx-\mathbf{u}_i-\mathcal{I}_{(i)}(\boldsymbol{\beta}_{(i)}-\hat{\boldsymbol{\beta}})$.

Rearrange to obtain the coefficient change.
Multiplying by the inverse information gives $\boldsymbol{\beta}_{(i)}-\hat{\boldsymbol{\beta}}\approx-\mathcal{I}_{(i)}^{-1}\mathbf{u}_i$, or $\hat{\boldsymbol{\beta}}-\boldsymbol{\beta}_{(i)}\approx\mathcal{I}_{(i)}^{-1}\mathbf{u}_i$.

The last expression approximates the vector of unstandardized DFBETAs. Its $j$th component depends not only on observation $i$'s score for coefficient $j$, but also on the inverse information matrix. The matrix accounts for relationships among coefficient estimates: a case’s contribution to one part of the score can affect multiple fitted coefficients when the information matrix is not diagonal.

Worked Example: Approximating a Two-Coefficient Change

Consider an independent Poisson GLM with log link for a hypothetical count outcome: $$ Y_i\mid\mathbf{x}_i\sim\operatorname{Poisson}(\mu_i),\qquad \log(\mu_i)=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}. $$ Assume the Poisson variance equals the mean, dispersion is fixed at one, and the fit is estimable. For one observation, suppose its score contribution at the full estimate is $\mathbf{u}_i=(0.4,\,1.5)^\mathsf{T}$ for the two coefficients of interest. The deleted-data information matrix is approximately $$ \mathcal{I}_{(i)}= \begin{pmatrix} 20&0\\ 0&10 \end{pmatrix}. $$ Use the first-order approximation to calculate the two coefficient changes and their approximate standardized values.

Method. Use $\hat{\boldsymbol{\beta}}-\boldsymbol{\beta}_{(i)}\approx\mathcal{I}_{(i)}^{-1}\mathbf{u}_i$, then divide each component by its deleted-fit standard error.

First invert the diagonal matrix: $$ \mathcal{I}_{(i)}^{-1}= \begin{pmatrix} 1/20&0\\ 0&1/10 \end{pmatrix} = \begin{pmatrix} 0.05&0\\ 0&0.10 \end{pmatrix}. $$ Then multiply by the score contribution: $$ \begin{pmatrix} 0.05&0\\ 0&0.10 \end{pmatrix} \begin{pmatrix} 0.4\\ 1.5 \end{pmatrix} = \begin{pmatrix} 0.05(0.4)\\ 0.10(1.5) \end{pmatrix} = \begin{pmatrix} 0.02\\ 0.15 \end{pmatrix}. $$ The approximate full-minus-deleted changes are therefore $0.02$ for $\beta_1$ and $0.15$ for $\beta_2$. The corresponding standard errors, using the diagonal entries of the approximate covariance matrix, are $$ \operatorname{SE}_{(i)}(\hat{\beta}_{1(i)})\approx\sqrt{0.05}=0.2236,\qquad \operatorname{SE}_{(i)}(\hat{\beta}_{2(i)})\approx\sqrt{0.10}=0.3162. $$ Thus, the approximate standardized values are $$ \frac{0.02}{0.2236}\approx 0.0894,\qquad \frac{0.15}{0.3162}\approx 0.4743. $$ Both signs are positive: the full-data estimates are approximately larger than their deleted-data counterparts. This is an approximation based on the supplied score and information, not a substitute for an exact refit. The sign trap is to use a positive derivative for the score: because the score derivative is negative information, doing so would reverse both estimated changes.

Reading DFBETAS in Context

A DFBETAS value is tied to a specific coefficient. A large absolute value for one coefficient does not imply that every coefficient is strongly affected, and a small value for one coefficient does not establish that the observation has little effect on the fit overall. For categorical predictors, the coefficient’s meaning depends on the reference category and coding. For a logit or log link, the coefficient scale is logarithmic, so its change can also be described through the corresponding odds ratio or mean ratio.

For example, if a coefficient in a log-link model changes from $-0.40$ in the full fit to $-0.10$ after deletion, the full-minus-deleted DFBETA is $-0.30$. The negative sign means the observation made the full estimate more negative. On the ratio scale, the full-fit multiplier is $e^{-0.40}\approx0.6703$, compared with $e^{-0.10}\approx0.9048$ in the deleted fit. The observation therefore contributes to a smaller fitted ratio for that contrast in the full model.

DFBETAS should be treated as a screening diagnostic, not an automatic exclusion rule. There is no universal cutoff that applies to every GLM, sample size, coefficient scale, and data structure. Large absolute values warrant inspection of the observation, its predictor pattern, and the stability of the coefficient under refitting. A valid but unusual observation may be important precisely because it contains information not shared by the rest of the data.

Worked Example: Locating the Effect in a Poisson Model

A hypothetical study fits an independent Poisson GLM with log link for counts, with dispersion fixed at one: $$ Y_i\mid\mathbf{x}_i\sim\operatorname{Poisson}(\mu_i),\qquad \log(\mu_i)=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}. $$ For one observation, the full fit gives $\hat{\beta}_1=0.30$ and $\hat{\beta}_2=-0.40$. After deleting it, the estimates are $\hat{\beta}_{1(i)}=0.28$ and $\hat{\beta}_{2(i)}=-0.10$. The deleted-fit standard errors are 0.10 and 0.15, respectively. Identify which modeled effect this observation drives more strongly on the standardized coefficient scale, and interpret the direction for $\beta_2$.

For $\beta_1$: $$ \operatorname{DFBETA}_{i1}=0.30-0.28=0.02,\qquad \operatorname{DFBETAS}_{i1}=\frac{0.02}{0.10}=0.20. $$

For $\beta_2$: $$ \operatorname{DFBETA}_{i2}=-0.40-(-0.10)=-0.30,\qquad \operatorname{DFBETAS}_{i2}=\frac{-0.30}{0.15}=-2.00. $$

The absolute standardized change is larger for $\beta_2$: $2.00$, compared with $0.20$ for $\beta_1$. The negative sign indicates that including the observation makes the estimated $\beta_2$ more negative. On the log-link ratio scale, the full-fit multiplier is $e^{-0.40}\approx0.6703$, while the deleted-fit multiplier is $e^{-0.10}\approx0.9048$. The observation therefore substantially strengthens the estimated negative effect in the full fit. The common mistake is to call $-2.00$ the larger influence merely because it is negative; compare absolute magnitudes, then use the sign to interpret direction.

Common Interpretation Traps

Reversing the subtraction order. With the convention here, subtract the deleted estimate from the full estimate. A software package using the opposite convention will show the same magnitude with the opposite sign.
Using the wrong standard error. The definition here uses the standard error from the deleted fit. A value standardized using the full-fit standard error may differ, especially when deletion changes the estimated uncertainty.
Confusing coefficient influence with overall influence. Each DFBETAS value refers to one observation-coefficient pair. It is not a single summary of the observation’s effect on every aspect of the model.
Deleting an observation solely because of a large value. First check data quality, model estimability, and the coefficient’s stability. A diagnostic identifies sensitivity; it does not establish an error.

Key takeaway: DFBETA measures the full-minus-deleted change in one GLM coefficient; DFBETAS expresses that change in deleted-fit standard-error units. Read the sign for direction and the absolute standardized value for relative size, while verifying the convention and treating the result as a diagnostic rather than a deletion rule.

Check Your Understanding

For each problem, use the full-minus-deleted sign convention and the stated deleted-fit standard error.

A binomial GLM gives a full-fit coefficient of $0.25$, a deleted-fit coefficient of $0.10$, and a deleted-fit standard error of $0.30$. Find the DFBETA and DFBETAS. (Answer: DFBETA $=0.15$; DFBETAS $=0.50$.)
In a logistic model, a predictor’s full-fit odds ratio is $1.5$, while its deleted-fit odds ratio is $1.2$. The deleted-fit standard error of the coefficient is $0.20$. Find the coefficient-scale DFBETA and DFBETAS. (Answer: DFBETA $=\log(1.5)-\log(1.2)=\log(1.25)\approx0.2231$; DFBETAS $\approx1.1157$.)
A log-link GLM coefficient is $-0.60$ in the full fit and $-0.20$ after deletion. Its deleted-fit standard error is $0.25$. Calculate both diagnostics and state which fit has the more negative coefficient. (Answer: DFBETA $=-0.40$; DFBETAS $=-1.60$; the full fit is more negative.)
For a two-coefficient GLM, suppose $\mathcal{I}_{(i)}=\begin{pmatrix}25&0\\0&4\end{pmatrix}$ and $\mathbf{u}_i=(0.5,-0.8)^\mathsf{T}$. Using the first-order approximation, find the two full-minus-deleted changes and standardized changes. (Answer: changes $(0.02,-0.20)^\mathsf{T}$; standard errors $(0.20,0.50)$; standardized changes $(0.10,-0.40)$.)
A Poisson log-link model gives $\hat{\beta}_{3}=0.35$, $\hat{\beta}_{3(i)}=0.50$, and deleted-fit standard error $0.10$. Find both diagnostics and compare the two fitted mean multipliers for a one-unit increase in the predictor. (Answer: DFBETA $=-0.15$; DFBETAS $=-1.50$; full multiplier $e^{0.35}\approx1.4191$, deleted-fit multiplier $e^{0.50}\approx1.6487$.)

What You'll Learn