Cluster-Level Influence Diagnostics

Why Assess Influence by Cluster?

In a generalized linear mixed model, observations within a cluster share a grouping structure, such as patients within clinics or repeated measurements within people. A cluster can affect a fit in ways that a single observation cannot: it may contain many observations, have unusual covariate values, or provide much of the information about between-cluster variation. An observation-level diagnostic can identify an unusual record, but it does not directly answer whether the fitted conclusions depend on one entire cluster.

Cluster-level influence diagnostics address that question by comparing the model fitted to all clusters with models refitted after omitting one cluster at a time. The comparison is a sensitivity analysis, not a test of whether a cluster is erroneous. A large change signals that the full-data result depends materially on that cluster and merits investigation in context.

Definition and Model Setup

Write clusters as \(j=1,\ldots,J\), and observations within cluster \(j\) as \(i=1,\ldots,n_j\). For a GLMM, a common specification is

Here \(g\) is the link, \(\beta\) is the vector of fixed effects, and \(b_j\) is a vector of cluster-specific random effects with covariance matrix \(D\). The model ordinarily assumes that clusters are independent conditional on the model parameters, and that the random effects follow the specified distribution and are independent across clusters. The conditional response distribution, link, covariate structure, and random-effects structure must be specified consistently in every refit. If fitting relies on approximate likelihood or numerical integration, the refits also need to converge reliably.

The sign of \(\Delta_{j,k}\) gives the direction of change when cluster \(j\) is omitted. Its size must be judged on a meaningful scale. A change of 0.2 may be negligible for one coefficient and consequential for another; comparing it with a standard error or translating it to an odds ratio, rate ratio, or probability can help.

What to Compare Across Refits

A useful analysis does not reduce influence to a single number. At minimum, examine fixed-effect estimates, random-effect variance or covariance estimates, and fitted predictions relevant to the scientific question. For each cluster, report the full-data estimate, the deletion estimate, their difference, and—where meaningful—a relative or standardized change.

• Fixed effects: compare coefficients and, where appropriate, their exponentiated values. Coefficient changes are on the link scale.
• Variance components: compare estimated variances and standard deviations. A relative change can be informative, but is unstable when the full-data estimate is near zero.
• Predictions: compare predictions for clearly specified covariate profiles. State whether these are conditional predictions involving random effects or predictions marginalized over the random-effects distribution.
• Uncertainty: inspect standard errors, intervals, convergence, and boundary estimates as well as point estimates. A small point-estimate change does not guarantee that uncertainty is unchanged.

For fixed-effect coefficients, a simple standardized change is \(\Delta_{j,k}/SE(\hat\beta_k)\), using the full-data standard error as a reference scale. This is a descriptive yardstick, not a universally calibrated test statistic or cutoff. A multivariate summary can use a quadratic form, for example \(D_j^2=\Delta_j^{\mathsf T}\widehat{\mathrm{Var}}(\hat\beta)^{-1}\Delta_j\), where \(\Delta_j=\hat\beta_{(-j)}-\hat\beta\). Such summaries depend on the chosen parameter set and covariance estimate; they do not remove the need to inspect individual effects and predictions.

Why Deletion Changes the Fit

A short approximation explains why clusters with unusual covariate patterns or substantial information can have a larger effect. Consider fixed-effect estimation with other parameters held fixed, and let \(\ell_j(\beta)\) be cluster \(j\)'s contribution to the log likelihood. At the full-data maximum, the total score is zero: \(\sum_j U_j(\hat\beta)=0\), where \(U_j\) is the derivative of \(\ell_j\) with respect to \(\beta\). After deleting cluster \(j\), the score at the original estimate is therefore \(-U_j(\hat\beta)\).

Let \(I_{(-j)}\) be the negative Hessian of the remaining log likelihood at \(\hat\beta\), that is, its observed information matrix. A first-order Taylor expansion of the deletion score around \(\hat\beta\) gives

Thus deletion changes can be substantial when the cluster score is large relative to the information in the remaining clusters. This is a local approximation: in a GLMM, variance components and random effects are also estimated, and refitting can change all of them. The approximation motivates the diagnostic but does not replace the actual deletion refits, especially with few clusters, a boundary variance estimate, or a large change.

Interpreting Influence When Clusters Are Few or Unequal

With few clusters, each cluster can contribute a substantial fraction of the information. Estimates of the random-effects distribution may be especially sensitive, and one deletion can move a variance estimate sharply or place it at a boundary. In this setting, the range of deletion estimates is more informative than a rigid cutoff. Report the number of clusters and show how conclusions vary across the deletion refits.

Unequal cluster sizes create a different interpretive challenge. A large cluster may influence the fit because it contributes many observations. A small cluster can also be influential if its covariate pattern is unusual or it contains outcomes that are hard for the model to accommodate. Cluster size alone does not establish influence, and a small cluster is not automatically harmless. Examine the data, covariate support, fitted patterns, and the effect of deletion together.

A cluster deletion also changes the analyzed collection of clusters. If the cluster is genuinely part of the target population, its influence is a property of the available evidence and model—not a reason to remove it from the analysis. Exclusion requires a defensible data-quality or design rationale established independently of whether exclusion improves the result.

Practical Workflow, Alternatives, and Failure Modes

Observation-level influence diagnostics focus on the effect of an individual record, whereas cluster-level deletion assesses the combined effect of all records in a group and the group’s contribution to estimating heterogeneity. Residual diagnostics address how observed outcomes compare with fitted behavior; they do not by themselves measure how much the estimates would change if a cluster were removed. These approaches complement one another. For example, a cluster may have an unusual aggregate pattern without a single extreme observation, or one record may be influential while its cluster as a whole is not.

Common failures include treating a change in a coefficient as an automatic reason to discard a cluster, using one arbitrary threshold as a universal definition of influence, and interpreting a variance percentage change when the full estimate is nearly zero. Refit failures also matter: if deleting a cluster causes nonconvergence or a boundary estimate, that is important information about fragility, but the failed fit does not provide a trustworthy numerical influence estimate. Check optimization, starting values, and model identifiability before interpreting the comparison. Finally, a large change in one parameter need not imply a large change in the scientific prediction; assess the outcome-scale quantities that matter for the intended use.