Computing And Checking GLM Influence Diagnostics

Why a Reproducible Workflow Matters

Influence diagnostics are calculations tied to a particular fitted model, not labels that can be read off a residual plot. Their values depend on the converged fit, the working weights, the dispersion convention, and whether a reported change is an approximation or comes from refitting after deletion. A reproducible analysis records these choices before interpreting which cases merit investigation.

The workflow below uses the GLM hat matrix and one-step case-deletion approximations. It does not treat a large diagnostic as proof that an observation is erroneous: it identifies a case whose removal may materially change specified fitted quantities. Exact refits are a useful check because the one-step calculation is local to the original fit.

Set the Scale and Notation

Suppose independent observations $Y_i$ follow a GLM with linear predictor $\eta_i=x_i^\top\beta$, mean $\mu_i$, and variance $\operatorname{Var}(Y_i)=\phi V(\mu_i)/a_i$, where $a_i>0$ is a prior weight. The link satisfies $\eta_i=g(\mu_i)$. Assume the fitted means are in the interior of the parameter space, $g$ is differentiable there with finite nonzero derivative, and the weighted design has full column rank. Let $p$ be the number of fitted coefficients, and define the fitting weight and working residual by

q_i=\frac{a_i}{V(\hat\mu_i)\{g'(\hat\mu_i)\}^2}, \qquad e_i=g'(\hat\mu_i)(y_i-\hat\mu_i).

Write $X$ for the design matrix, $W=\operatorname{diag}(q_i)$, and $A=X^\top W X$. These quantities are evaluated at the converged full-data estimate $\hat\beta$. A fixed known dispersion is used as $\phi$; if dispersion is estimated, its estimator and the convention for using it must be stated. For binomial and Poisson models commonly fitted with fixed dispersion, $\phi=1$.

Definition: The weighted GLM hat matrix and its diagonal leverages are $$ H=W^{1/2}X(X^\top W X)^{-1}X^\top W^{1/2}, \qquad h_i=H_{ii}. $$ The Pearson residual is $r_{Pi}=(y_i-\hat\mu_i)/\sqrt{\phi V(\hat\mu_i)/a_i}$. The working residual $e_i$ and Pearson residual have the relation $e_i\sqrt{q_i/\phi}=\operatorname{sign}\{g'(\hat\mu_i)\}r_{Pi}$.

The sign in this relation matters. For an increasing link it is positive; differentiability alone does not imply that the link is increasing. The leverage satisfies $0\le h_i\le1$. It is strictly below one for case $i$ precisely when deleting that row leaves the weighted design full rank.

One-Step Diagnostics and Their Conventions

Let $\hat\beta_{(-i)}$ denote the estimate after deleting case $i$. The one-step approximation uses the working system at the full-data estimate rather than solving the nonlinear likelihood equations anew. Define $\widetilde\Delta_i$ as the approximation to $\hat\beta-\hat\beta_{(-i)}$, so the sign of a coefficient or fitted-value change is explicit.

Definition: When $1-h_i>0$, the one-step coefficient change, its linear-predictor change at case $i$, and Cook’s distance are $$ \widetilde\Delta_i=\frac{A^{-1}x_iq_i e_i}{1-h_i}, \qquad x_i^\top\widetilde\Delta_i=\frac{h_i e_i}{1-h_i}, \qquad \widetilde D_i=\frac{\widetilde\Delta_i^\top A\widetilde\Delta_i}{p\phi} =\frac{r_{Pi}^2h_i}{p(1-h_i)^2}. $$

For the component diagnostics in this tutorial, use the full-model coefficient standard errors and full-model scale. Let $A^{-1}_{jj}$ denote diagonal element $j$ of $A^{-1}$. The definitions are

\widetilde{\operatorname{DFBETAS}}_{ij} =\frac{\widetilde\Delta_{ij}}{\sqrt{\phi(A^{-1})_{jj}}}, \qquad \widetilde{\operatorname{DFFITS}}_i =\frac{x_i^\top\widetilde\Delta_i}{\sqrt{\phi h_i/q_i}} =\operatorname{sign}\{g'(\hat\mu_i)\} \frac{r_{Pi}\sqrt{h_i}}{1-h_i}.

These are specified conventions, not the only variants in use. Some software uses a deleted-model scale estimate or reports coefficient changes with the opposite sign. Comparisons are meaningful only after checking that the scale, denominator, sign, and direction of subtraction agree. If the fitted response rather than the linear predictor is of interest, the delta-method change is approximately $(x_i^\top\widetilde\Delta_i)/g'(\hat\mu_i)$, and its corresponding standard error is $\sqrt{\phi V(\hat\mu_i)/a_i}\sqrt{h_i}$. Their ratio is $r_{Pi}\sqrt{h_i}/(1-h_i)$, without the sign factor; the sign-adjusted expression above applies to the linear-predictor-scale DFFITS.

Worked Example: Calculating the Diagnostics from a Working System

Consider a converged working system with three observations, $p=2$, $\phi=1$, $q_i=1$, and design rows $x_1^\top=(1,-1)$, $x_2^\top=(1,0)$, and $x_3^\top=(1,1)$. Suppose the working residuals are $e_1=0.2$, $e_2=-0.4$, and $e_3=0.2$. The weighted score is zero because $\sum_i x_i e_i=(0,0)^\top$.

The information matrix is

A=X^\top X= \begin{pmatrix}3&0\\0&2\end{pmatrix}, \qquad A^{-1}= \begin{pmatrix}1/3&0\\0&1/2\end{pmatrix}.

Thus $h_i=x_i^\top A^{-1}x_i$, giving $h_1=1/3+1/2=5/6$, $h_2=1/3$, and $h_3=5/6$. For case 1, $1-h_1=1/6$, and

\widetilde\Delta_1 =\frac{A^{-1}x_1 e_1}{1-h_1} =6 \begin{pmatrix}1/3\\-1/2\end{pmatrix}(0.2) = \begin{pmatrix}0.4\\-0.6\end{pmatrix}.

Its linear-predictor change is $x_1^\top\widetilde\Delta_1=0.4+0.6=1.0$, also obtained as $(5/6)(0.2)/(1/6)=1.0$. Here $r_{P1}=e_1=0.2$ for an increasing-link working system with these weights and scale. Cook’s distance is $\widetilde D_1=0.2^2(5/6)/\{2(1/6)^2\}=0.6$. The coefficient standard errors are $\sqrt{1/3}$ and $\sqrt{1/2}$, so the two DFBETAS are $0.4/\sqrt{1/3}\approx0.693$ and $-0.6/\sqrt{1/2}\approx-0.849$. Finally, $\widetilde{\operatorname{DFFITS}}_1=1/\sqrt{(5/6)}\approx1.095$.

Why the One-Step Formula Holds

The full-data Fisher-scoring information is $A/\phi$. Removing case $i$ subtracts its information contribution, leaving $A_{(-i)}=A-q_i x_ix_i^\top$. At the full-data estimate, the full score is zero, so the deleted-data score there is the negative of case $i$'s contribution. A single scoring step from that estimate gives the approximate deleted estimate. The following identity gives its change and also explains the condition $1-h_i>0$.

Result: If $A$ is positive definite and $1-h_i>0$, then $$ (A-q_i x_ix_i^\top)^{-1}x_i=\frac{A^{-1}x_i}{1-h_i}. $$ Consequently, the one-step change from full estimate to approximate deleted estimate has the formula for $\widetilde\Delta_i$ above.

Proof. Put $u=A^{-1}x_i$. Since $h_i=q_i x_i^\top A^{-1}x_i$,

(A-q_i x_ix_i^\top)\frac{A^{-1}x_i}{1-h_i} =\frac{x_i-q_i x_i(x_i^\top A^{-1}x_i)}{1-h_i} =\frac{x_i(1-h_i)}{1-h_i}=x_i.

This proves the identity. At the full-data estimate, the deleted score is $-q_i x_i e_i/\phi$. A Fisher-scoring step for the deleted fit is therefore $-A_{(-i)}^{-1}q_i x_i e_i$. Subtracting this approximate deleted estimate from $\hat\beta$, then applying the identity, gives $\widetilde\Delta_i=A^{-1}x_iq_i e_i/(1-h_i)$. $\square$

The condition also has a rank interpretation. The matrix determinant lemma gives

\det(A-q_i x_ix_i^\top)=\det(A)(1-q_i x_i^\top A^{-1}x_i) =\det(A)(1-h_i).

Because $A-q_i x_ix_i^\top$ is positive semidefinite, it is positive definite exactly when its determinant is positive. Thus $h_i<1$ exactly when the deleted weighted design retains full column rank. A full-rank full-data design, even with more observations than parameters, does not guarantee this condition for every case.

Worked Example: A Decreasing Link Changes the DFFITS Sign

Suppose a grouped-binomial observation has $m=4$ trials, observed proportion $y=0.75$, and fitted mean $\hat\mu=0.5$. Use the valid decreasing link $g(\mu)=-\operatorname{logit}(\mu)$, fixed $\phi=1$, and suppose the case leverage in the fitted system is $h=0.25$. At $\mu=0.5$, $V(\mu)=0.25$ and $g'(\mu)=-1/\{\mu(1-\mu)\}=-4$. Hence $q=4/(0.25\cdot16)=1$, and $e=-4(0.75-0.5)=-1$.

The Pearson residual is $r_P=0.25/\sqrt{0.25/4}=1$. The linear-predictor change is $he/(1-h)=0.25(-1)/0.75=-1/3$, and its standard error is $\sqrt{\phi h/q}=0.5$. Thus DFFITS is $(-1/3)/0.5=-2/3$. The sign-adjusted residual expression gives the same result: $\operatorname{sign}(-4)\cdot1\cdot\sqrt{0.25}/0.75=-2/3$. Omitting the sign factor would incorrectly report $+2/3$.

Compare with Exact Case Deletion

An exact deletion analysis removes case $i$, refits the same model to the remaining observations, and records $\hat\beta-\hat\beta_{(-i)}$ and any fitted-value changes of interest. It can differ from the one-step approximation because the means, working weights, and information change as the model is refitted. The exact refit also reveals practical problems that a formula at the full fit cannot settle, such as nonconvergence or a rank-deficient deleted design.

Worked Example: One-Step Versus Exact Logistic Deletion

Take an intercept-only logistic model with three Bernoulli observations $1,1,0$. The full-data estimate is $\hat\mu=2/3$, $\hat\beta=\operatorname{logit}(2/3)=\log 2$, and $A=3(2/3)(1/3)=2/3$. Each leverage is $h=1/3$. Delete one of the observations equal to 1. At the full fit, its working residual is $e=(1-2/3)/\{(2/3)(1/3)\}=1.5$, its fitting weight is $q=(2/3)(1/3)=2/9$, and its Pearson residual is $r_P=(1/3)/\sqrt{(2/3)(1/3)}=1/\sqrt2$.

The one-step linear-predictor and coefficient change are both $he/(1-h)=(1/3)(1.5)/(2/3)=0.75$. Cook’s distance, with $p=1$ and $\phi=1$, is \[ \widetilde D=\frac{(1/2)(1/3)}{(2/3)^2}=\frac{3}{8}=0.375. \] The full-model coefficient standard error is $\sqrt{1/A}=\sqrt{3/2}$, so the one-step DFBETAS is $0.75/\sqrt{3/2}\approx0.612$; DFFITS has the same value here.

The exact refit uses the remaining observations $1,0$, whose fitted probability is $1/2$ and whose coefficient is $0$. The exact full-minus-deleted coefficient change is therefore $\log2\approx0.693$, not $0.75$. The one-step approximation is close but not identical; the exact deleted fit provides the definitive deletion comparison for this dataset.

A Practical Audit Sequence

A useful report makes the calculation reproducible and makes clear where approximations enter. Apply the following checks before ranking cases.

Verify the fitted model.
Record the response representation, link, offsets, prior weights, dispersion convention, and the optimizer’s convergence status. Check that fitted means and coefficients are finite and that the stated model has a full-rank weighted design.

Recompute the working system.
At the converged fit, form $q_i$, $e_i$, $A$, $h_i$, and $r_{Pi}$. Check $0\le h_i\le1$, and verify the score is numerically near zero. Flag $h_i=1$ separately: the one-step deletion expressions with $1-h_i$ in the denominator are undefined.

Declare diagnostic conventions.
State whether $\phi$ is fixed or estimated, which scale estimates standard errors use, whether changes are full minus deleted or deleted minus full, and whether DFFITS refers to the linear predictor or a response-scale approximation.

Compute and inspect.
Calculate Cook’s distance, the coefficient-specific DFBETAS, and DFFITS using the same fitted system. Inspect both magnitudes and signs, and interpret them with the case’s residual and design pattern rather than in isolation.

Refit selected cases exactly.
For cases flagged by the scientific question or diagnostics, delete each case and refit the same model. Record convergence, rank, coefficient changes, and fitted changes; compare exact values with one-step values on matching scales.

Common Pitfall and Interpretation

A common mistake is to apply a familiar residual formula without checking the link derivative’s sign, or to assume $h_i<1$ merely because the full design has more rows than columns. The correction is to retain $\operatorname{sign}\{g'(\hat\mu_i)\}$ in the DFFITS identity and to check the deleted design’s rank: leverage is always at most one, but it is strictly less than one for a case only when that deletion leaves full rank.

Another practical danger is treating one-step diagnostics as exact deletion results. They are local approximations from the full-data fit; exact refits can shift substantially, fail to converge, or expose nonidentifiability. Diagnostics help prioritize investigation, but neither a threshold nor a large value alone establishes that a case should be removed. Report the data, model, scale convention, and substantive consequences of any proposed exclusion.

Key takeaway: A defensible GLM influence analysis ties every diagnostic to a converged fit, a stated scale and sign convention, and the appropriate weighted design. Use one-step quantities to screen and summarize; use exact case-deletion refits to check consequential cases.

Check Your Understanding

Answer each question using the conventions defined above.

What model and scale information should be recorded before computing GLM influence diagnostics?
For what design condition is a case leverage strictly below one, and what does $h_i=1$ imply for the one-step deletion formula?
Why must the sign of $g'(\hat\mu_i)$ be included when expressing DFFITS using Pearson residuals?
In what sense is a one-step case-deletion estimate different from an exact case-deletion refit?
If an exact deleted fit does not converge, what should be reported before interpreting the corresponding one-step diagnostic?