Why the Source of Extra Variation Matters
Binary outcomes from the same cluster often resemble one another more than outcomes from different clusters. A random effect can represent this shared propensity: clusters with a higher latent probability tend to have more successes, and clusters with a lower one tend to have fewer. As a result, cluster totals can vary more than they would under one pooled binomial model, even when the model is correct conditional on its random effects.
That marginal overdispersion is not, by itself, evidence of unexplained extra-binomial variation. The diagnostic question is whether the fitted random-effects structure accounts for the observed variation. After conditioning on the cluster effects, are the remaining binary outcomes consistent with the specified model? A discrepancy that remains in conditional residual patterns or conditional simulations points to structure the fitted model has not captured. Possible sources include a missing predictor, within-cluster dependence, or an inappropriate conditional outcome model.
Model and Variance Decomposition
Suppose cluster \(j\) contains \(m_j\) binary outcomes \(Y_{ij}\), where \(Y_{ij}=1\) denotes a success and \(Y_{ij}=0\) a failure. A random-intercept logistic model is
Here \(b_j\) is shared by observations in cluster \(j\). Conditional on \(b_j\) and the covariates, the model assumes the observations are independent Bernoulli variables with the stated probabilities. The random effects are independent across clusters and independent of the covariates under the usual model assumptions. A cluster total is \(K_j=\sum_iY_{ij}\). If the probabilities are equal within a cluster, then \(K_j\mid b_j\sim\operatorname{Binomial}(m_j,p_j)\). If they differ because of covariates, the total instead follows a sum of independent Bernoulli variables with potentially unequal probabilities.
For equal cluster size \(m\) and a common conditional probability \(p_j\) within each cluster, let \(\bar p=E(p_j)\) and \(\operatorname{Var}(p_j)=\tau_p^2\). Conditional variance and the law of total variance give
The first term is the variance of a pooled binomial total with probability \(\bar p\); the second is the additional marginal variation attributable to differing cluster probabilities. Equivalently, two distinct outcomes in the same cluster have covariance \(\tau_p^2\), because they are conditionally independent but share \(p_j\). Thus a large variance among cluster totals can be expected under a correctly specified random-effects model. The relevant check is not whether the totals vary, but whether they vary more or in a different pattern than the model predicts.
Worked Example: Overdispersion Explained by Cluster Heterogeneity
Consider four equally sized hypothetical clusters, each with \(m=10\) observations. Suppose their fitted conditional probabilities are \(0.10,\ 0.30,\ 0.50,\) and \(0.70\). Treating these four probabilities as an illustrative distribution, compute the mean and variance of a cluster probability.
The mean is \(\bar p=(0.10+0.30+0.50+0.70)/4=0.40\). The variance is
A pooled binomial model would give variance \(10(0.40)(0.60)=2.40\) for a cluster total. Allowing the cluster probabilities to vary gives
The higher marginal variance is accounted for by the differences in cluster probabilities: the conditional binomial component averages \(10\{p_j(1-p_j)\}\), or \(1.90\), and the variance of the cluster-specific means \(10p_j\) is \(5.00\). Their sum is \(6.90\). This calculation does not establish that a particular fitted GLMM is adequate; it shows why comparing cluster totals with a pooled binomial variance can wrongly label explained heterogeneity as unexplained overdispersion.
Diagnosing Variation After Conditioning
Conditional residuals compare outcomes with probabilities that include the fitted cluster effects. For a grouped cluster total with common fitted probability \(\hat p_j\), a Pearson contribution is
This is a useful discrepancy to inspect, not an automatic reference-distribution test. In sparse data, the denominator may be small, the distribution of the sum may be far from a continuous approximation, and the estimated random effects are uncertain and partly informed by the same outcomes being checked. A large contribution identifies a cluster worth investigating; its size alone does not identify the cause.
Residual displays add information that a single total cannot show. Look for residuals that remain associated within a cluster, vary systematically over time or covariate values, or occur in unusually long runs. A random intercept can represent a generally high or low propensity for a cluster; it does not automatically represent serial dependence, changing probabilities over time, or a cluster-specific covariate effect. For binary outcomes, the individual conditional variance is already \(p_{ij}(1-p_{ij})\). Therefore, a claim of extra-binomial variation is most useful when it specifies the level and pattern of the discrepancy, such as excess variability in grouped outcomes or dependence among repeated observations after conditioning.
Worked Example: A Large Conditional Pearson Discrepancy
Suppose six hypothetical clusters each contain eight observations. A fitted model gives conditional probabilities \(0.10,\ 0.15,\ 0.20,\ 0.25,\ 0.30,\) and \(0.35\); the observed success totals are \(3,\ 0,\ 4,\ 0,\ 5,\) and \(1\). For cluster 1, the fitted mean total is \(8(0.10)=0.80\), and its conditional variance is \(8(0.10)(0.90)=0.72\), so \(Q_1=(3-0.80)^2/0.72=6.72\).
Calculating the same contribution for each cluster gives
Several clusters contribute substantially. This is a reason to examine the conditional residuals and compare a suitable discrepancy with simulations from the fitted model. It is not justified to declare a formal lack-of-fit result by comparing \(21.105\) mechanically with a chi-square distribution: there are few sparse clusters, and the fitted probabilities and random effects were estimated. The pattern could reflect remaining structure, influential observations, or uncertainty in the fitted effects; simulation helps assess whether a discrepancy of this kind is surprising under the model.
Using Simulated Discrepancies to Separate the Sources
A simulated discrepancy is a statistic computed both from the observed data and from data generated under a fitted model. Choose it to target the suspected source of misfit. Cluster-total variability targets variation across clusters. A within-cluster ordering statistic, such as the number of adjacent success pairs in repeated measurements, targets residual arrangement that a random intercept alone does not model. Comparing the observed statistic with simulated values is more informative than asking whether a generic dispersion statistic exceeds a rule of thumb.
The replication scheme determines what the comparison can reveal. Simulations that keep fitted cluster effects fixed ask whether the observations are plausible given those estimated propensities. Simulations that draw new random effects also include the model’s expected cluster heterogeneity and assess the distribution of new clusters. A discrepancy in the first comparison suggests conditional structure is missing; a discrepancy only in the second may instead concern the distribution of cluster effects or uncertainty in its estimation. Estimated effects are not known constants, however, so conditional checks can be optimistic. Where practical, account for parameter estimation in the simulation procedure and interpret the results as diagnostics rather than as automatically calibrated hypothesis tests.
Worked Example: Detecting Within-Cluster Ordering Not Explained by a Random Intercept
In a hypothetical repeated-measures data set, 20 clusters each have four binary observations. Every cluster has exactly two successes. In 16 clusters the two successes are adjacent in time; in the other four they are separated. Consider \(T\), the total number of adjacent success-success pairs across all clusters.
Under a model with a cluster-specific intercept but no time dependence, all arrangements of two successes among four occasions have equal probability conditional on the cluster total. There are \(\binom{4}{2}=6\) arrangements: three have one adjacent success pair, and three have none. Thus, conditional on these totals, each cluster contributes an adjacent pair with probability \(1/2\), and \(T\) has mean \(20(1/2)=10\). The observed statistic is \(T=16\).
As an illustration, suppose 2,000 conditional simulations produce a central 95% range from 5 to 15 and 12 simulated statistics at least as large as 16. The observed value lies above that range. The simulated upper-tail fraction, with the common add-one adjustment, is \((12+1)/(2000+1)=0.00650\). This is a diagnostic signal of unusually strong ordering given the cluster totals—not automatically a calibrated p-value. A random intercept can explain clusters’ overall propensities, but it cannot explain why successes tend to occur next to one another within a cluster. A time effect or dependence structure would need substantive consideration.
Comparison, Failure Modes, and Limits
| Diagnostic comparison | What it can reveal | What it cannot establish alone |
|---|---|---|
| Pooled binomial versus observed cluster totals | Whether totals are more variable than a model ignoring clusters predicts | Whether that variation remains after random effects are included |
| Conditional residuals and fixed-effect simulations | Patterns inconsistent with outcomes at the fitted cluster propensities | Definitive evidence about the cause; fitted effects and parameters are uncertain |
| Simulations with newly drawn cluster effects | Whether the fitted model reproduces variation across clusters overall | Whether a mismatch arises from the random-effects distribution, conditional model, or estimation |
A common mistake is to compute a pooled dispersion statistic, find it large, and conclude that the conditional Bernoulli model has extra-binomial variation. The correction is to ask which replication level the statistic represents. Random intercepts can inflate marginal variation in cluster totals, so inspect conditional residuals and compare targeted statistics with simulations that reflect the question being asked.
Other failure modes include treating estimated random effects as error-free, relying on a chi-square approximation with very few clusters, and choosing a discrepancy only after seeing an extreme pattern without accounting for that selection. Sparse outcomes also limit what residual plots can show: a cluster may have all zeros or all ones simply by chance. Finally, this distinction is diagnostic, not causal. A signal of remaining structure does not by itself tell whether the remedy is a missing covariate, a random slope, a dependence model, or a different outcome distribution.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- For clusters of size \(m=5\), suppose the mean conditional probability is \(0.30\) and the variance of cluster probabilities is \(0.04\). Find the pooled-binomial variance and the marginal variance of a cluster total under the equal-probability-within-cluster model.
- A cluster has \(m=10\), fitted conditional probability \(\hat p=0.20\), and observed total \(K=4\). Calculate its Pearson contribution.
- If cluster totals are reproduced well by simulations that draw new random effects, but an ordering statistic is extreme in simulations holding fitted cluster effects fixed, what is the more direct concern?
- Why is a large pooled-binomial dispersion statistic insufficient to establish conditional extra-binomial variation?
Solutions. (1) The pooled-binomial variance is \(5(0.30)(0.70)=1.05\). The marginal variance is \(1.05+5(4)(0.04)=1.85\); the additional \(0.80\) is due to cluster-probability heterogeneity. (2) The fitted mean is \(10(0.20)=2\), and the variance is \(10(0.20)(0.80)=1.60\), so \(Q=(4-2)^2/1.60=2.50\). (3) The direct concern is within-cluster ordering or dependence not represented by the conditional model; good marginal totals do not rule it out. (4) Random effects themselves increase the marginal variance of cluster totals, so the comparison must account for them before attributing excess variation to conditional lack of fit.