Three Different Diagnostic Questions
A case can be unusual in its predictor values, unusual in its response given those predictors, or consequential for the fitted model. These are different properties. A large residual signals a response that is poorly predicted by the fitted model; high leverage signals a design point with unusual weight and position in the fitted design; influence concerns how much the fitted model changes when that case is removed or perturbed.
In binomial and count-response generalized linear models (GLMs), these distinctions matter because the fitting weights depend on the fitted means. A case with a large raw count need not have a large residual, and a point with unusual predictors need not have an unusual response. The useful diagnostic question is therefore not simply “Which cases have the largest residuals?” but “Which cases have response discrepancy, leverage, and a consequential effect on the fit?”
For a binomial observation with \(m_i\) trials and \(Y_i\) successes, write the fitted success probability as \(\hat p_i\). Its Pearson residual is \(r_{P,i}=(Y_i-m_i\hat p_i)/\sqrt{m_i\hat p_i(1-\hat p_i)}\). For a Poisson count \(Y_i\) with fitted mean \(\hat\mu_i\), it is \(r_{P,i}=(Y_i-\hat\mu_i)/\sqrt{\hat\mu_i}\). These scale the response discrepancy by its model-based standard deviation. Deviance residuals use a likelihood-based discrepancy instead. Neither residual type, by itself, measures how much the fitted coefficients would change.
The GLM hat matrix is the weighted projection arising in an iteratively reweighted least-squares step. Its diagonal entry \(h_i\) is the leverage for case \(i\) at the fitted weights. In familiar binomial and Poisson models, the weights depend on the fitted mean and, for grouped binomial data, on the number of trials. Thus a case can have substantial leverage because it carries much information, even if its response is close to its fitted value.
What Leverage Does—and Does Not—Tell Us
A useful basic property follows from the weighted projection form. Suppose the design matrix \(X\) has \(p\) linearly independent columns and the working-weight matrix \(W\) is diagonal with positive entries. The weighted hat matrix is \(H=W^{1/2}X(X^\mathsf{T}WX)^{-1}X^\mathsf{T}W^{1/2}\). Its diagonal elements are the working leverages.
To verify the result, set \(A=W^{1/2}X\). Then \(H=A(A^\mathsf{T}A)^{-1}A^\mathsf{T}\). This matrix is symmetric and idempotent, since \(H^\mathsf{T}=H\) and \(H^2=A(A^\mathsf{T}A)^{-1}(A^\mathsf{T}A)(A^\mathsf{T}A)^{-1}A^\mathsf{T}=H\). It is therefore an orthogonal projection. For the \(i\)th coordinate vector \(e_i\), \(h_i=e_i^\mathsf{T}He_i=\|He_i\|^2\), so \(h_i\geq0\). Also \(I-H\) is an orthogonal projection, giving \(1-h_i=e_i^\mathsf{T}(I-H)e_i=\|(I-H)e_i\|^2\geq0\). Finally, \(\sum_i h_i=\operatorname{tr}(H)=p\): the projection has rank \(p\), so its trace is \(p\). \(\square\)
In GLMs these statements apply to the working-weight matrix at a particular fit; the weights can change as the model is refitted. Leverage is consequently not a fixed property of the raw predictor values alone. It also depends on the model, the fitted means, and any trial-count or exposure weights.
Worked Example: Grouped Binomial Residuals and Leverage
Consider an intercept-only logistic model for two independent groups. Group A has \(m_A=2\) trials and \(Y_A=0\) successes. Group B has \(m_B=18\) trials and \(Y_B=10\) successes. Calculate the fitted probability, Pearson residuals, and leverages, then interpret their different rankings.
The pooled success proportion is \(\hat p=(0+10)/(2+18)=10/20=0.5\). The fitted means are \(m_A\hat p=1\) and \(m_B\hat p=9\). The Pearson residuals are \(r_{P,A}=(0-1)/\sqrt{2(0.5)(0.5)}=-1/\sqrt{0.5}\approx-1.414\) and \(r_{P,B}=(10-9)/\sqrt{18(0.5)(0.5)}=1/\sqrt{4.5}\approx0.471\). Thus Group A has the larger absolute Pearson residual.
For an intercept-only binomial fit, the working weight of group \(i\) is \(m_i\hat p(1-\hat p)\). Both groups have the same fitted probability, so their leverage shares are proportional to their trial counts: \(h_A=2/(2+18)=0.10\) and \(h_B=18/(2+18)=0.90\). Group B has far greater leverage but a smaller residual. This identifies a high-leverage point with modest response discrepancy; it does not, on its own, establish whether deleting Group B would materially change a target of interest.
Exposure and Leverage in a Poisson Rate Model
An exposure is an amount of time, population, or opportunity during which a count could occur. In a Poisson rate model with an intercept and log-exposure offset, the fitted mean is \(\hat\mu_i=t_i\hat\lambda\), where \(t_i\) is exposure and \(\hat\lambda\) is the fitted rate. The Poisson working weight is \(\hat\mu_i\). In this intercept-only setting, leverage is therefore proportional to exposure.
This result makes the role of exposure explicit: a high-exposure observation can carry a large share of the information about the common rate. The following example also shows why a larger residual need not mean a larger effect on the refitted model.
Worked Example: A Large Count Residual Versus a High-Exposure Case
Fit an intercept-only Poisson rate model to 101 independent observations. One observation has exposure \(t=100\) and count \(Y=150\). The other 100 observations each have exposure \(1\); their counts sum to \(100\), including one count of \(8\). The total count is \(150+100=250\), and total exposure is \(100+100=200\). Calculate fitted means, Pearson residuals, leverage, and the effects of refitting after removing each of the two highlighted observations.
The full-data rate estimate is \(\hat\lambda=250/200=1.25\). The high-exposure case has fitted mean \(100(1.25)=125\), so its Pearson residual is \((150-125)/\sqrt{125}=25/\sqrt{125}\approx2.236\). The ordinary-exposure case with count \(8\) has fitted mean \(1.25\), giving \((8-1.25)/\sqrt{1.25}=6.75/\sqrt{1.25}\approx6.037\). The ordinary case has the larger absolute residual.
The high-exposure case has working weight \(125\); each ordinary case has working weight \(1.25\). Hence its leverage is \(125/[125+100(1.25)]=125/250=0.5\), whereas the leverage of each ordinary case is \(1.25/250=0.005\). The case with count \(8\) has a much larger residual but very low leverage. The high-exposure observation has a smaller residual but high leverage.
Now compare actual refits for the rate coefficient \(\beta=\log\lambda\). Removing the high-exposure case leaves 100 events over 100 exposure units, so \(\hat\lambda_{-}=1\). The coefficient changes from \(\log(1.25)\) to \(\log(1)\), a change of \(0-\log(1.25)\approx-0.2231\). Removing the ordinary case with count \(8\) leaves \(250-8=242\) events over 199 exposure units. Thus \(\hat\lambda_{-}=242/199\approx1.2161\), and the coefficient change is \(\log[(242/199)/1.25]=\log(968/995)\approx-0.0275\).
Here the high-leverage observation changes the rate coefficient more, despite having the smaller residual. The comparison is specific to this model and target: a different model or target can produce a different ordering. The calculation illustrates why influence must be judged from a refitted change rather than inferred from residual size alone.
Reading Residuals, Leverage, and Refitted Changes Together
For binomial and count responses, Pearson and deviance residuals are useful screens for response discrepancy, while leverage flags cases with unusual weighted design position. A refitted comparison addresses influence directly. Examine changes in the quantities that matter for the analysis: coefficient estimates, fitted probabilities or rates in relevant covariate regions, or predictions. A small coefficient change can still matter if it alters an important prediction; a noticeable coefficient change may have little practical consequence for a particular prediction.
Residuals are themselves model-dependent. For example, a Pearson residual scales the response difference by an estimated standard deviation, while a deviance residual measures a likelihood-based discrepancy. Discrete responses also have discrete sampling behavior, so a residual’s numerical size should be interpreted with the fitted distribution and sample size in view. Neither type has a universal cutoff that proves a case is an outlier or influential.
Inspect an appropriate residual, such as a Pearson or deviance residual, in the context of the fitted binomial or count distribution.
Use leverage to identify cases that occupy unusual positions or carry substantial working weight. Remember that GLM weights depend on fitted means and may include trial counts or exposures.
Compare the full fit with a fit that excludes or perturbs the case, focusing on parameters or predictions relevant to the scientific question.
Check data quality, model specification, exposure or trial definitions, and whether the observation represents a scientifically meaningful subgroup.
Worked Example: A Diagnostic Pattern Is Not a Verdict
Suppose a grouped-binomial fit produces a case with a large absolute deviance residual and small leverage, while another case has a modest deviance residual and large leverage. State what can be concluded before refitting and what remains unresolved.
The first case has a response that is relatively discrepant from its fitted distribution, given the residual scale, but its low leverage suggests its design position is not unusual in the weighted fit. The second case has a design position or weight that gives it substantial leverage, but its modest residual indicates no comparably large response discrepancy by that measure.
Neither observation is thereby certified as influential. Influence depends on the change in the chosen target after refitting. The residual and leverage pattern determines what to investigate; it does not replace the refitted comparison. In particular, a large residual at low leverage can be scientifically important as a possible response outlier even when coefficient estimates are stable.
Common Mistakes and Practical Interpretation
- Equating a large residual with influence. A large residual indicates response discrepancy, not necessarily a large change in the fitted coefficients or predictions. Check the refitted target.
- Equating high leverage with an outlier. Leverage is a property of the weighted design. A high-leverage case can have a response close to its fitted mean.
- Ignoring trial counts or exposure. In grouped binomial and Poisson rate models, these quantities affect working weights and therefore leverage. Raw predictor distance alone does not capture the fitted geometry.
- Assuming one diagnostic ordering applies to every target. A case can substantially change a slope but barely change a prediction near the center of the data, or the reverse. Specify what “influential” means for the analysis.
- Using a cutoff as a deletion rule. A diagnostic flag prompts investigation. It does not justify removal without checking data validity, model assumptions, and scientific context.
Check Your Understanding
Try these questions without looking back.
- In an intercept-only grouped-binomial model with a common fitted probability, why is leverage proportional to the number of trials?
- A Poisson observation has a large Pearson residual but very low leverage. What does this suggest, and what does it not establish?
- In an intercept-only Poisson rate model, exposures are \(2\), \(3\), and \(5\). What are the three leverages?
- Why can a high-leverage observation have a modest residual?
- What should be compared when deciding whether a case is influential for a particular scientific question?