Why examine observation-level influence?
A response can be surprising under a fitted generalized linear mixed model (GLMM) without having much effect on the fitted model. Conversely, a record that looks fairly ordinary in a residual display can matter because it is unusual in its covariate pattern or because the data are sparse. Influence asks a direct sensitivity question: how much would the fitted model change if one observation were not included?
This is a diagnostic, not a rule for removing data. A large change may identify a record worth checking for data errors, unusual measurement circumstances, or a weakly supported part of the model. It does not establish that the observation is invalid, nor does it show that the model without it is preferable. The comparison is meaningful only when the two fits use the same model specification, estimation method, and parameterization.
Model and deletion comparison
For a common formulation, let \(Y_{ij}\) be a discrete response for observation \(i\) in grouping unit \(j\), let \(x_{ij}\) and \(z_{ij}\) be its fixed- and random-effect design vectors, and let \(b_j\) be a random effect. Conditional on the random effects, responses are modeled using an exponential-family distribution and a link function:
Here \(\beta\) is the fixed-effect parameter vector and \(D\) is the random-effects covariance matrix. The usual model assumptions include a correctly specified conditional response distribution and link, the stated random-effects distribution, and conditional independence of responses given the random effects and covariates. The fitting procedure also needs to produce a stable, converged estimate. Influence is assessed relative to this model; it cannot by itself verify these assumptions.
To delete observation \(r=(i,j)\), remove its response and its corresponding row from the fitting data, then fit the same model again. Other observations from the same grouping unit remain. Compare like with like: fixed effects on the same scale, variance parameters represented consistently, and predictions at the same specified covariate values. In likelihood-based GLMM fitting, keep the integration approximation and optimizer settings consistent when possible; otherwise numerical differences can be mistaken for case influence.
The marginal likelihood for a grouping unit integrates over its shared random effect. Consequently, observations in that unit do not generally contribute independent marginal-likelihood terms that can simply be subtracted one at a time. The most dependable basic definition is therefore operational: fit the full model, delete the selected observation, refit, and compare. A local approximation can still explain the comparison. Let \(U_{(-r)}(\theta)\) be the score (the gradient of the delete-\(r\) log likelihood) and \(H_{(-r)}(\theta)\) its Hessian. A Taylor expansion around the full estimate gives:
and therefore, if the delete-\(r\) Hessian is nonsingular and the change is small,
This is a first-order justification for why a record can shift estimates: its removal changes the score and curvature of the fitted likelihood. It is not a substitute for refitting when the sample is sparse, the likelihood is irregular, or the deletion causes a large change.
Quantifying changes in estimates and predictions
A raw change \(\Delta_r\) is easy to interpret in the parameter's units, but its size should be judged relative to estimation uncertainty and the purpose of the analysis. For a fixed-effect coefficient \(\beta_k\), one simple scale-free summary is the delete-one standardized change:
The denominator here is the standard error from the full-data fit. This summary states how large the change is in full-fit standard-error units; it is not a test statistic with a universal cutoff. For a vector of \(p\) fixed effects, a covariance-scaled summary is:
This combines correlated coefficient changes using the estimated covariance matrix from the full fit. It is a descriptive sensitivity measure, not automatically a chi-squared test or a significance level. Changes in variance components should also be reported in a meaningful scale. For a standard deviation, report both the change in the standard deviation and, where useful, a relative change; a change in a variance parameter is not interchangeable with a change in its standard deviation.
Influence can also be local to a prediction. Specify the target first. A marginal prediction averages over the random-effects distribution; a conditional prediction uses a particular grouping unit's random effect. If evaluating a conditional prediction for the unit containing the deleted record, re-estimate that unit's random effect from the remaining records. Comparing predictions at different covariate values or changing from conditional to marginal prediction during the comparison does not isolate the effect of deletion.
Worked Example: A Binary-Outcome Record and a Slope Estimate
Suppose a hypothetical logistic random-intercept model relates a binary event to a continuous exposure \(x\), with exposure measured in ten-unit increments. One record has \(x=3\) and \(y=1\), an event at a relatively high exposure. The record is checked and is not known to be erroneous. Assume both fits converge under the same fitting settings. The full fit gives \(\hat{\beta}_1=0.72\) with \(SE(\hat{\beta}_1)=0.24\); deleting this response gives \(\hat{\beta}_{1,(-r)}=0.65\).
The change in the exposure coefficient is:
Thus the slope decreases by 0.07 log-odds units per ten-unit exposure increment, or about 0.29 full-fit standard errors, when this record is deleted. The full-fit odds ratio for a ten-unit increase is \(\exp(0.72)\approx2.05\); after deletion it is \(\exp(0.65)\approx1.92\). These are model-based odds ratios, not risk ratios. The example indicates a modest change on this coefficient scale; whether it matters depends on the scientific use of the estimate and its uncertainty. The fitted values are hypothetical outputs, not calculations determined by this single record alone.
Worked Example: Comparing a Marginal Count Prediction
Consider a hypothetical Poisson random-intercept model for counts with a log link. At a fixed covariate pattern and exposure of one unit, the full fit gives a marginal predicted count of 1.80. After deleting one unusually large count, the model is refitted and the marginal prediction at that same covariate pattern is 1.50. Assume both fits converge and use the same model and integration settings.
The absolute change is \(1.50-1.80=-0.30\) expected counts per one unit of exposure. The relative change, measured against the full prediction, is:
So the predicted mean is about 16.7% lower after deletion. This comparison is specifically marginal: each prediction averages over the random-effects distribution. It does not mean that the deleted count itself had a 16.7% probability of being unusual, and it does not establish that the full prediction is wrong. If the scientific target were a particular existing grouping unit, a conditional prediction would instead need to use that unit's random effect re-estimated from the retained observations.
Practical workflow and interpretation
Identify whether the concern is a fixed effect, a random-effects variance parameter, or a prediction at a specified covariate pattern. State the scale and units.
Confirm convergence, save estimates and standard errors, and record the prediction target and whether it is conditional or marginal.
Remove only that observation, retaining the other records and the same model structure. Check convergence and numerical stability again.
Report signed changes, standardized coefficient changes where useful, and changes in predictions in context. Investigate large or unstable shifts rather than applying a universal cutoff.
Three issues deserve particular attention. First, a deletion fit may fail to converge or may yield a variance estimate at a boundary. In that case, a numerical change cannot be interpreted as a reliable sensitivity estimate without investigating the fitting problem. Second, sparse binary outcomes or counts with many zeros can make estimates sensitive to a few records; refitting may also expose separation-like behavior or weak identification. Third, a large coefficient change need not imply a large change in every fitted prediction. Conversely, a modest coefficient change can matter for a prediction in a covariate region where the linear predictor is especially sensitive.
Observation-level influence complements rather than replaces residual and leverage diagnostics. Residuals describe how the observed response differs from a fitted distribution under a chosen residual definition. Leverage describes aspects of how a record's design position can affect fitting. Influence assesses the resulting change in estimates or predictions after deletion. A large residual or high leverage can prompt a deletion check, but neither guarantees a large change; the refit is the direct sensitivity comparison. Standard GLM influence summaries can be useful reference points, but shared random effects and integrated likelihoods mean that mixed-model deletion effects need not follow independent-observation GLM intuition.
Use the result as a reason to examine the record and the model, not as an automatic exclusion criterion. Verify data processing and measurement, check whether the observation occupies a scientifically important covariate region, and report conclusions that are materially sensitive to defensible deletion choices. Deleting one response while retaining its grouping unit is a different diagnostic question from removing an entire grouping unit.
Worked Example: A Covariance-Scaled Change
In a hypothetical two-coefficient model, suppose the full-fit covariance matrix for two fixed effects is diagonal with standard errors 0.20 and 0.30. Deleting one record changes the coefficient estimates by \(0.10\) and \(-0.15\), respectively. Because the covariance is diagonal, the covariance-scaled measure simplifies to:
Each change is one-half of its corresponding full-fit standard error in magnitude, with opposite signs. The combined value \(D_r^2=0.50\) summarizes both changes relative to their estimated uncertainty. It does not provide a probability or a universal threshold for deciding that the record is influential. If the coefficients were correlated, the off-diagonal covariance terms would also affect the calculation, so simply summing squared standardized changes would be incorrect.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- A coefficient is 0.50 in the full fit and 0.42 after deletion; its full-fit standard error is 0.20. What is its delete-one standardized change?
- Why should a marginal prediction be compared with another marginal prediction at the same covariate pattern?
- A record has an extreme residual, but deleting it barely changes the fitted coefficients or predictions. Is it necessarily influential?
- What should be reported if the deletion fit fails to converge?
Solutions. (1) \((0.42-0.50)/0.20=-0.40\), a decrease of 0.40 full-fit standard errors. (2) Keeping the target and prediction type fixed isolates the effect of deleting the response rather than mixing deletion with a change of estimand. (3) No. Its response is discrepant under the fitted model, but the deletion comparison shows little effect on the selected estimates or predictions. (4) Report that the delete-one comparison was not reliably obtained, investigate numerical or identification problems, and do not interpret an unstable fit as a dependable influence measure.