Residual Dependence Within Clusters

Why Check Dependence After Fitting Random Effects?

Repeated observations from the same cluster are often related. A GLMM can represent some of that relationship through its specified random effects. The diagnostic question here is narrower: after accounting for those random effects and the fixed effects, do the ordered residuals still show within-cluster structure? For example, residuals that stay positive for several visits and then stay negative may indicate a time pattern that the fitted model has not captured.

The distinction matters because observations can be correlated marginally even when the model is adequate. A shared random intercept, for instance, can make outcomes within a cluster resemble one another. That dependence is already part of the model. A residual-dependence check should ask whether the observed pattern is unusual under the fitted model, including its random-effects structure—not whether the outcomes are associated in the first place.

Residuals, Ordering, and the Model Assumptions

Let \(Y_{ij}\) denote the response at ordered occasion \(j\) in cluster \(i\), and let \(b_i\) denote the cluster’s random effects. Under the fitted GLMM, the observations are assumed conditionally independent given \(b_i\) and the covariates, unless the model explicitly includes an additional dependence structure. The model also specifies the conditional mean and variance of each response. A residual diagnostic looks for a violation of that conditional-independence assumption, not for dependence that the random effects were included to represent.

A common residual for a count or binary observation is the conditional Pearson residual \(r_{ij}=(y_{ij}-\hat{\mu}_{ij})/\sqrt{\widehat{V}_{ij}}\), where \(\hat{\mu}_{ij}\) and \(\widehat{V}_{ij}\) are the fitted conditional mean and variance. The exact residual choice depends on the outcome and fitting procedure. For discrete outcomes, residuals can be coarse or tied; randomized quantile residuals are one alternative. Whichever definition is used, apply it consistently to the observed data and every simulated replicate.

An ordered display typically plots residual against occasion within each cluster, with clusters separated or clearly identified. Look for runs of the same sign, alternating signs, gradual drift, or a repeating pattern. Connect residuals only within their own cluster: the last observation of one cluster is not adjacent to the first observation of another. The order must have scientific meaning. Arbitrarily sorting residuals by size manufactures a pattern rather than diagnosing one.

A useful summary for lag-one dependence is the mean adjacent residual product. Let \(\mathcal{P}\) be the set of all pairs of consecutive observations within clusters, and let \(M\) be the number of such pairs. Define

Large positive products tend to occur when neighboring residuals have the same sign; large negative products tend to occur when they have opposite signs. The statistic is not, by itself, a universal test with a fixed reference distribution. Its null behavior depends on the outcome distribution, fitted means, cluster sizes, random-effects structure, residual definition, and parameter estimation.

Why Simulation Is Needed

The rationale for \(T_{\text{adj}}\) follows from the model assumption. If residuals have conditional mean zero and are conditionally independent across consecutive observations, then, for fixed model quantities,

Thus the adjacent product has expected value zero under idealized conditions. In an actual fitted GLMM, parameters and random effects are estimated, residuals need not have exactly zero mean or unit variance, and discrete residuals may have non-normal shapes. Consequently, zero is not a reliable universal cutoff, nor does a standard normal reference automatically apply. Simulation supplies a reference distribution that reflects the fitted model and the diagnostic procedure.

Generate replicated outcomes under the specified GLMM, calculate residuals for each replicate by the same method, and recompute \(T_{\text{adj}}\). To isolate dependence beyond the modeled random effects, the simulation must preserve the fitted random-effects structure. Whether to condition on fitted cluster effects or draw new effects depends on the diagnostic question; the earlier material on conditional versus unconditional replications gives that distinction. If parameter estimation materially affects the residuals, refitting each replicate in a parametric bootstrap can reflect that step as well.

For \(B\) simulated statistics \(T^{(1)},\ldots,T^{(B)}\), compare \(T_{\text{obs}}\) with their distribution. A standardized descriptive measure is \(Z_{\text{sim}}=(T_{\text{obs}}-\bar{T}_{\text{sim}})/s_{\text{sim}}\), where \(\bar{T}_{\text{sim}}\) and \(s_{\text{sim}}\) are the simulated mean and standard deviation. For a prespecified positive-dependence check, an upper-tail Monte Carlo probability is \(p=(1+\#\{T^{(b)}\geq T_{\text{obs}}\})/(B+1)\). For a two-sided check, define the extremeness rule in advance—for example, compare absolute deviations from the simulated mean. Such simulated probabilities are diagnostic evidence under the fitted model; they are not a probability that the model is true.

Worked Checks and the Importance of Sequence

Diagnostics, Alternatives, and Limitations

Interpret the statistic alongside the display. A large positive adjacent-product statistic accompanied by runs of same-sign residuals suggests persistence; a negative value and frequent sign changes suggest alternation. Either direction may be relevant, depending on the scientific setting and the diagnostic question. A single summary can conceal local behavior: positive dependence in some clusters may cancel negative dependence in others. Inspect cluster-specific displays and consider summaries suited to the suspected pattern, such as a sign-run count or a prespecified longer lag.

Compare this check with other diagnostics rather than substituting one for another. Global dispersion or tail-count checks ask whether variability or extremes are unusual overall; they do not locate a sequence pattern. A marginal correlation of the raw outcomes mixes dependence represented by random effects with any remaining dependence. A residual autocorrelation display can make lag patterns visible, while a simulation-based statistic gives a calibrated summary of a chosen feature. None identifies the correct replacement model automatically.

Several failures can invalidate the interpretation:

• Ignoring the cluster boundaries. Pairing the final residual of one cluster with the first of another creates nonexistent adjacency. Construct pairs separately within each cluster.
• Using the wrong order. An arbitrary order, or an order that does not represent time or another meaningful sequence, cannot support a within-cluster serial interpretation.
• Comparing with zero or a textbook cutoff alone. The fitted model and estimation procedure affect the statistic’s distribution. Use simulations that reproduce the relevant fitting and residual steps.
• Simulating from an incomplete model. Omitting the fitted random effects can make ordinary modeled dependence look like a residual failure. Include the specified structure in the simulation.
• Relying on one statistic. A lag-one product is poorly suited to patterns at other lags, nonlinear cycles, or dependence limited to a few clusters. Inspect displays and choose additional summaries for specific concerns.

Simulation does not repair a misspecified mean, incorrect observation order, or inappropriate residual definition. Sparse outcomes can also yield tied or highly discrete residuals, making the simulated reference distribution coarse. Further, trying many statistics and reporting only the most extreme one exaggerates the apparent evidence; the chosen summaries and direction should be stated transparently. A diagnostic flags a mismatch with the fitted model, but does not by itself prove that a new dependence term is needed or that any proposed term is correct.