What Leverage Measures in a GLMM
A fitted outcome can look ordinary even when the observation that helped produce it occupies an unusual part of the predictor space. Leverage is intended to flag that design feature: it describes how much an observation can contribute to its own fitted value in a local, approximately linear representation of the model. In sparse binary or count data, it is particularly important to separate this geometric signal from an unusual observed outcome.
Unlike the ordinary linear-model hat matrix, a GLMM does not generally have one exact, fixed hat matrix. Its fitted values depend on estimated fixed effects, random effects, and variance parameters; for non-Gaussian outcomes, the likelihood is nonlinear. A useful diagnostic is therefore an approximate leverage calculated from a local weighted working model. Its meaning depends on the approximation used.
The penalty \(G^{-1}\) reflects shrinkage of the random effects; fixed effects are not penalized in this expression. For a Bernoulli observation with fitted conditional probability \(\hat p_i\), the usual local working weight is \(w_i=\hat p_i(1-\hat p_i)\). For a binomial observation with \(m_i\) trials, it is \(w_i=m_i\hat p_i(1-\hat p_i)\). Other GLMM families use weights appropriate to their working-likelihood representation. Thus sparse outcomes can have small weights, particularly when fitted probabilities are near zero or one.
Under this stated construction, \(A\) is symmetric and its eigenvalues lie between zero and one, so each diagonal value is between zero and one. The sum of the diagonal values is an approximate effective degrees of freedom for the penalized working fit, not necessarily the number of fixed-effect coefficients. Variance-component estimation, integration over random effects, and refitting the nonlinear model are not fully represented by this local matrix.
Why the Working Hat Matrix Has This Form
At a fitted GLMM, the likelihood can be locally approximated by a weighted least-squares problem. Write \(z\) for the working response and \(\theta=(\beta',b')'\). The local penalized criterion, ignoring constants, is $$ (z-C\theta)'W(z-C\theta)+b'G^{-1}b. $$ The second term penalizes random effects according to their assumed covariance. Differentiating with respect to \(\theta\), setting the derivative to zero, and solving gives $$ \hat\theta=(C'WC+P)^{-1}C'Wz. $$ The fitted working response is \(C\hat\theta\). Multiplying by \(W^{1/2}\) on both sides expresses the mapping from weighted working responses to weighted fitted responses: $$ W^{1/2}C\hat\theta = W^{1/2}C(C'WC+P)^{-1}C'W^{1/2}(W^{1/2}z). $$ The matrix multiplying \(W^{1/2}z\) is \(A\), which justifies using its diagonal as a local measure of self-contribution. This derivation applies to the working approximation; it does not turn the nonlinear GLMM fit into an exact linear model.
Worked Leverage Calculations
Worked Example: A Two-Cluster Random-Intercept Approximation
Consider hypothetical Bernoulli data with one observation in each of two clusters. At a particular fitting iteration, both fitted probabilities are \(0.50\), so both working weights are \(w=0.50(1-0.50)=0.25\). Suppose the random-intercept variance is \(\tau^2=1\). With only an intercept as a fixed effect, the two-observation working fit separates into a common direction and a contrast between clusters.
The common direction is fitted without penalty by the fixed intercept, so its hat-matrix eigenvalue is \(1\). In the contrast direction, the data contribute weight \(0.25\), while the random-intercept penalty contributes precision \(1/\tau^2=1\). The contrast eigenvalue is therefore $$ \lambda_{\text{contrast}}=\frac{w}{w+1/\tau^2} =\frac{0.25}{0.25+1}=0.20. $$
Each observation receives half of the common-direction contribution and half of the contrast-direction contribution to its diagonal. Thus $$ h_i=\frac{1+\lambda_{\text{contrast}}}{2} =\frac{1+0.20}{2}=0.60. $$ Both approximate leverages are \(0.60\). This small example shows why random-effect shrinkage matters: the cluster-specific contrast is only partly fitted, while the overall intercept direction is unpenalized. With only two observations, these values are illustrative, not a basis for a general cutoff.
Worked Example: A Sparse Binary Observation at an Extreme Predictor Value
Consider a hypothetical logistic-model working calculation with an intercept and one predictor \(x\), at predictor values \(0,1,4\). Suppose the current fitted probabilities are \(0.50,0.50,0.90\), respectively. To isolate fixed-design leverage, first calculate the weighted fixed-effect benchmark; a full GLMM calculation also includes its random-effect design and penalty.
The working weights are \(0.25,0.25,0.09\). Their sum is \(W_+=0.59\), and the weighted predictor mean is $$ \bar x_w=\frac{0.25(0)+0.25(1)+0.09(4)}{0.59} =\frac{0.61}{0.59}=1.034. $$ The weighted sum of squared deviations is $$ S_{xx,w}=0.25(0-1.034)^2+0.25(1-1.034)^2+0.09(4-1.034)^2 \approx 1.059. $$ For an intercept and one predictor, the fixed-effect weighted hat diagonal is $$ h_i^{(X)}=w_i\left(\frac{1}{W_+}+\frac{(x_i-\bar x_w)^2}{S_{xx,w}}\right). $$ At \(x=4\), this gives $$ h_3^{(X)}=0.09\left(\frac{1}{0.59}+\frac{(4-1.034)^2}{1.059}\right) \approx 0.900. $$ The other two values are approximately \(0.676\) and \(0.424\); they sum with \(0.900\) to \(2.000\), the rank of the two-column fixed-effect design in this calculation. The extreme predictor value has high fixed-design leverage despite its small working weight. These are not full GLMM leverages: random-effect structure and shrinkage can change the values.
Worked Example: High Leverage Without an Unusual Outcome
Use the last observation in the preceding hypothetical calculation, but suppose its observed binary outcome is \(y=1\). Its fitted probability is \(0.90\), and its fixed-design benchmark leverage is about \(0.900\). The Bernoulli Pearson residual is $$ r_P=\frac{y-\hat p}{\sqrt{\hat p(1-\hat p)}} =\frac{1-0.90}{\sqrt{0.90(0.10)}} =\frac{0.10}{0.30}=0.333. $$
The predictor configuration is unusual relative to the other two observations, but the observed success is close to what the fitted model predicts. The high leverage is not evidence of outcome misfit. If instead the outcome were \(y=0\), the same fitted probability would give \(r_P=(0-0.90)/0.30=-3.00\), a much more surprising outcome under the fitted probability. Even then, leverage and residual answer different questions: the first concerns design, the second concerns outcome discrepancy. Neither alone establishes that the model is wrong.
Using Leverage in Sparse-Data Diagnostics
Leverage is most informative when read alongside residual displays and the model's design structure. The earlier material on residuals against fitted values and covariates provides complementary checks: residuals describe discrepancies between observed and fitted outcomes, while leverage identifies observations whose predictor configurations can give them substantial self-contribution in the working fit.
A high value should prompt inspection, not automatic deletion. Check whether the predictor values are valid, whether the observation lies outside the range well represented by the rest of the data, and whether its cluster or covariate combination is sparsely populated. Then examine an appropriate residual for the outcome type and compare the observation's fitted value with its observed outcome. In binary data, a fitted probability near zero or one also makes the working weight small; this can complicate interpretation even when the predictor configuration is unusual.
Common warning signs that the approximation needs caution include fitted probabilities extremely close to zero or one, very few observations in a cluster, a random-effect variance near a boundary, and a design matrix with near-dependencies. In such settings, working weights and the fitted random effects may change substantially under refitting or small data perturbations. The hat value is local to the current fit; it is not a guarantee that the same case will remain high-leverage under another model or estimation method.
| Diagnostic | Main question | What it does not establish |
|---|---|---|
| Approximate leverage \(h_i\) | Is this observation's design configuration unusual or influential in the local working fit? | That its observed outcome is poorly fitted |
| Residual | Does the observed outcome differ from the fitted outcome? | That the observation has an unusual predictor configuration |
| Ordinary GLM hat value | How unusual is the fixed-effect design under a GLM weighted fit? | How random-effect shrinkage changes GLMM leverage |
| Simulation-based checks | Are chosen features of replicated data compatible with the fitted model? | A unique leverage value with the same interpretation as \(h_i\) |
The ordinary GLM weighted hat value is a useful comparison for fixed-effect geometry, but it omits the random-effect penalty and dependence structure. The conditional penalized working hat value includes those elements in a local approximation. Marginal or simulation-based diagnostics address different targets, such as predictions averaged over random effects or replicated-data behavior; they should not be treated as interchangeable definitions of leverage. No universal threshold separates “high” from “acceptable” GLMM leverage. A value is best judged relative to other observations, the design, the fitted model, and the residual evidence.
Common Pitfall and Limitations
A common mistake is to call an observation a model outlier because its hat value is large. That reasoning confuses predictor configuration with outcome fit. A second mistake is to apply a linear-model rule of thumb mechanically to a GLMM, ignoring the working weights, random-effect shrinkage, and sparse-data context. Use leverage to identify cases worth examining, then use residuals and suitable predictive checks to assess outcome fit.
Approximate leverage is also not the same as a case-deletion influence measure. It does not by itself quantify how much fixed-effect estimates or predictions would change if an observation were removed; that requires a separate influence analysis. In addition, the displayed conditional working matrix conditions on estimated quantities and local linearization. It does not fully propagate uncertainty in variance parameters or represent all marginal effects of integrating over random effects. These limitations are especially relevant for rare outcomes and small samples.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In the penalized working matrix, what role does \(G^{-1}\) play, and which coefficient block is unpenalized?
- For a Bernoulli observation fitted at \(\hat p=0.20\), calculate its working weight.
- A case has high approximate leverage and a small Pearson residual. What can and cannot be concluded?
- Why might a fixed-effect GLM hat value differ from a conditional penalized GLMM hat value?
Solutions. (1) \(G^{-1}\) penalizes the random effects according to their covariance; the fixed-effect block is unpenalized. (2) \(w=\hat p(1-\hat p)=0.20(0.80)=0.160\). (3) Its predictor configuration has substantial self-contribution in the working fit, but its observed outcome is not strongly discrepant by that residual measure; high leverage alone does not establish outcome misfit. (4) The GLM calculation omits random-effect design and shrinkage, whereas the conditional penalized GLMM calculation includes them and uses weights at the GLMM fit.