Why Residual Dependence Can Reveal a Missing Group
A fitted GLMM can represent dependence arising from the grouping factors included in its random-effects structure. But the recorded observations may also share another source of variation: the same clinician, classroom, laboratory batch, or family may contribute responses across the groups already represented in the model. If that shared membership is omitted, observations connected through it can remain dependent after fitting. A useful diagnostic question is therefore not only whether residuals are large, but whether residuals from observations that share a particular membership tend to vary together.
The diagnostic is an aid to model investigation, not an automatic test that identifies the correct structure. Residual patterns may also arise from an incorrect mean model, time dependence, or other unmodelled variation. The evidence becomes more persuasive when the pattern is repeatable, corresponds to a plausible and recorded membership relationship, and is not explained by the fitted structure or another known feature of the data.
Model and Diagnostic Target
Let \(Y_i\) be the response for observation \(i\), and let \(\eta_i\) be its linear predictor. In a GLMM, the conditional mean is related to the predictor by a link function: \(g(\mu_i)=\eta_i\), where \(\mu_i=E(Y_i\mid\text{random effects})\). As a generic example, suppose the fitted model contains random effects \(b\) for recorded groups, but omits a candidate grouping factor \(h\). Write \(h(i)\) for the candidate group containing observation \(i\).
A common working model for investigating an omitted random intercept is
where \(\eta_i^{(0)}\) contains the fixed-effect part of the predictor, \(b_i\) denotes the contribution of included random effects for observation \(i\), and \(u_h\) is the omitted group intercept. The variance \(\tau^2\) measures variability between the omitted groups on the linear-predictor scale. The usual assumptions for this illustrative model are that the \(u_h\) are independent across groups, have mean zero and variance \(\tau^2\), and are independent of the included random effects and covariates. Responses are assumed conditionally independent given all random effects and the fixed predictors. These are assumptions to examine, not facts guaranteed by the data.
In practice, the omitted \(u_h\) is not available to calculate residuals. Instead, examine residuals from the fitted model and compare their patterns across candidate memberships. For instance, a Pearson residual is \((Y_i-\hat\mu_i)/\sqrt{\widehat{\operatorname{Var}}(Y_i\mid\text{fitted effects})}\), with the variance calculated using the fitted conditional model. Residuals are diagnostic summaries; estimates of random effects already in the model do not remove dependence from a factor that the model left out.
Justifying the Dependence Signature
The omitted intercept gives observations in the same candidate group a shared shift in their predictors. To see how this can produce response-scale dependence, condition on the included random effects \(b\), and define \(\mu_i(u)=g^{-1}(\eta_i^{(0)}+b_i+u)\). For two distinct observations \(i\) and \(j\) in the same omitted group, conditional independence given \(b\) and \(u_h\) implies
If \(\tau^2\) is small enough for a first-order Taylor approximation around \(u_h=0\), then \(\mu_i(u_h)\approx \mu_i(0)+\mu_i'(0)u_h\), and likewise for \(j\). Taking the covariance over \(u_h\) gives
This is the approximate covariance contribution from the omitted intercept conditional on the included random effects. It is not, by itself, the total marginal covariance when the included effects are also integrated over. For different omitted groups, the omitted-intercept contribution is zero under the assumed independence of the \(u_h\). For a logit link, the derivative at zero is \(\mu_i(0)\{1-\mu_i(0)\}\), so this contribution is positive when both observations have interior fitted probabilities and \(\tau^2>0\). The approximation is local; for large \(\tau^2\), it need not accurately quantify the covariance.
Worked Example: A Shared Technician Across Clinics
Suppose a Gaussian-response model includes a clinic random intercept. Four observations have the following conditional residuals, in outcome units, after fitting that model. Each was processed by one of two technicians, and each technician worked in both clinics.
| Observation | Clinic | Technician | Residual |
|---|---|---|---|
| 1 | A | T1 | 2 |
| 2 | B | T1 | 1 |
| 3 | A | T2 | -2 |
| 4 | B | T2 | -1 |
A simple descriptive check is the average product of residuals for pairs sharing a technician. The T1 pair contributes \(2(1)=2\); the T2 pair contributes \((-2)(-1)=2\). Thus the average is \((2+2)/2=2\) squared outcome units. The positive products indicate that each technician's two residuals have the same sign. Because each pair is from different clinics, a clinic intercept alone cannot account for this particular pairing.
This tiny example is only a diagnostic illustration: two pairs cannot establish that technician effects exist, and the numerical products do not estimate a reliable variance component. With many technicians and observations, one would assess whether the positive association recurs across technician-sharing pairs, compare it with appropriate non-sharing pairs, and check whether technician membership plausibly affects the outcome. The fact that technicians occur across clinics is essential: if each technician appeared in only one clinic, technician and clinic membership would be difficult or impossible to distinguish from these data.
Using Membership to Organize Residual Checks
Membership information helps turn an unstructured residual display into a targeted diagnostic. For each plausible candidate factor, identify which observations share its levels, then ask whether their residuals show more association than observations that do not share those levels. In a nested or crossed design, a membership graph can make these connections visible: repeated links to the same candidate group identify the pairs for which an omitted shared effect could matter.
Record which observations share each plausible grouping factor, including factors that cross the groups already in the fitted model.
Use residuals suited to the response and model, and check whether unusually large or similarly signed residuals cluster by candidate membership.
Contrast residual association for pairs that share the candidate factor with pairs that do not, while accounting for overlap with included groups and covariates.
Examine predictor form, temporal or spatial order, data recording, and response-specific variation before interpreting the pattern as evidence for a group effect.
Worked Example: Comparing Residual Pair Products
A hypothetical fitted model includes school intercepts. For observations from different schools, suppose a recorded assessor contributes to several outcomes. Among the residuals from six pairs that share an assessor but not a school, the products are \(0.8,\ 0.4,\ 1.0,\ -0.2,\ 0.6,\ 0.4\) squared-score units. The average product is
The positive average is consistent with residuals tending to move together among assessor-sharing pairs. It is evidence to investigate, not a formal significance test: the pairs may not be independent, and the six pairs may involve only a few assessors. A useful next check is whether pairs that share neither assessor nor school show a similar pattern, and whether the association persists after checking the fixed-effects specification. If the pattern is concentrated in assessor-sharing pairs across many independent assessors, an omitted assessor grouping becomes a more plausible explanation.
Worked Example: Approximate Covariance for a Binary Outcome
Consider two hypothetical binary responses with a logit link. Conditional on the included random effects, suppose both fitted probabilities at omitted intercept \(u=0\) are \(0.50\), and suppose the candidate omitted intercept has variance \(\tau^2=0.36\) on the log-odds scale. The logit mean derivative is \(p(1-p)\), so each derivative is \(0.50(1-0.50)=0.25\). The first-order omitted-intercept covariance contribution is therefore
Thus, under the small-variance approximation and conditional on the included effects, the shared omitted intercept contributes about \(0.0225\) to the response-scale covariance for this same-group pair. The conditional Bernoulli variance at probability \(0.50\) is \(0.50(0.50)=0.25\), so dividing by \(0.25\) gives an approximate correlation contribution of \(0.0225/0.25=0.0900\) in this symmetric illustration. This is not an exact marginal correlation: the approximation ignores higher-order terms, and additional included random effects may contribute further covariance when they are integrated out.
What the Pattern Can and Cannot Establish
Residual association is most useful as a structured clue. It does not uniquely identify its cause. For example, a nonlinear predictor effect can leave residuals systematically high in one range of a covariate; a time trend can make nearby observations similar; and response variation may be larger than the fitted conditional distribution allows. In each case residuals may look dependent even though the proposed grouping factor is not the explanation. Conversely, a genuine omitted factor may be hard to detect when it has few observations per level or is strongly confounded with an included factor.
Diagnostics should match the response type. For continuous outcomes, residual-versus-predictor plots and residual pair comparisons can reveal patterns. For binary outcomes, raw residuals are bounded and their variability depends on the fitted probability, so raw magnitudes across observations are not directly comparable; standardized or simulation-based residual diagnostics can be more informative. In either setting, compare observations under similar fitted conditions when possible. A residual product summary is descriptive unless a valid reference distribution and dependence structure for that summary have been established.
If a candidate factor is added, assess whether it is supported by the design and data, whether its memberships overlap meaningfully with existing factors, and whether the refitted model resolves the targeted residual pattern. Changes in fit alone do not prove that the new structure is scientifically appropriate. A single-level factor, near-complete confounding, or sparse membership can make its variance difficult to estimate. Model comparison and uncertainty assessment should respect the variance-component boundary at zero when a formal test is used; ordinary large-sample reference calculations may not apply without qualification.
| Diagnostic evidence | Possible interpretation | Important check |
|---|---|---|
| Same-sign residuals among observations sharing a candidate group | A missing shared random effect may remain | Check whether the pattern repeats across multiple group levels and is not driven by one level |
| Residual patterns follow time or spatial order | Temporal or spatial dependence may be omitted | Compare order-based patterns with the candidate membership pattern |
| Residual spread changes with fitted mean | Mean or response-variance specification may be inadequate | Inspect response-appropriate residual plots and model assumptions |
| Candidate memberships nearly duplicate included groups | Group effects may be hard to distinguish | Check overlap and connectedness in the observed membership structure |
The main alternative is to explain the residual pattern by revising the fixed-effects mean structure or by using a different dependence model appropriate to the data, rather than adding a random intercept. These alternatives address different mechanisms: a fixed-effect revision changes the systematic mean, whereas a group effect models shared variation among observations with common membership. Their suitability depends on design knowledge and the pattern itself. Neither should be chosen solely because it reduces a residual summary.
Common Mistake
- Wrong reasoning: “Residuals from the same candidate group are positively associated, so that is the total covariance caused by the omitted factor, regardless of the included random effects.”
- Correction: A covariance expression derived from an omitted intercept must state what is being conditioned on. The first-order expression above is the omitted-intercept contribution conditional on included effects. If included random effects are also integrated over, they can add covariance; the total marginal covariance cannot be inferred from the omitted-factor term alone.
- Another pitfall: Treating one large residual or one positive pair product as proof of a missing group. Look for a repeated pattern across groups and rule out reasonable competing explanations.
Check Your Understanding
Answer each question, then compare with the solutions below.
- In a GLMM with a clinic random intercept, two observations share a nurse but come from different clinics. What residual pattern would be consistent with an omitted nurse effect?
- For a logit model with two same-group fitted probabilities of \(0.20\) and an omitted-intercept variance of \(0.25\), calculate the first-order conditional covariance contribution.
- Why is an observed positive average residual product across candidate-group pairs not automatically a significance test?
- Suppose residual association follows chronological order but not the candidate group memberships. Which explanation deserves attention before adding the candidate random intercept?
Solutions. (1) Repeated same-sign or positively associated residuals among nurse-sharing pairs, especially across different clinics, would be consistent with a nurse effect; the pattern should recur across nurses and be compared with non-sharing pairs. (2) The logit derivative is \(0.20(0.80)=0.16\) for each observation, so the contribution is \(0.16(0.16)(0.25)=0.0064\), conditional on the included effects and under the first-order approximation. (3) Pair products may be dependent, the pairs may reuse the same groups or observations, and the reference distribution has not been established; the average is descriptive without a justified inferential procedure. (4) Temporal dependence or an omitted time trend deserves attention, because chronological order aligns with the residual pattern more directly than the candidate membership does.