Checking the Random-Effects Distribution Shape

Why Check the Shape of the Random-Effects Distribution?

A GLMM usually specifies a distribution for its latent cluster effects. For example, a random-intercept model may assume that clusters have independent effects from a normal distribution. This assumption is consequential: it helps determine how much outcome variation appears across clusters, and how the model averages over that variation when making population-level predictions.

The question here is not simply whether the estimated random-effects variance is large or small. It is whether the shape assumed for the random effects can plausibly generate the observed pattern of outcomes across clusters. A normal distribution may be a useful working model, but it is not guaranteed by the fact that the model has random effects. Model-based simulation provides a way to examine this assumption through the outcomes it implies.

Definition: A random-effects distribution-shape check compares cluster-level features of the observed outcomes with the same features computed from outcomes simulated under a fitted GLMM, with random effects drawn from the model’s assumed distribution. The check asks whether the assumed distribution can reproduce relevant patterns across clusters; it does not directly observe the latent effects.

Model and Assumptions

For clarity, consider a random-intercept model with clusters \(j=1,\ldots,J\) and observations \(i=1,\ldots,n_j\). Let \(Y_{ij}\) be the outcome, \(x_{ij}\) the covariate vector, and \(b_j\) the cluster-specific random intercept. A common model is

g\{E(Y_{ij}\mid b_j,x_{ij})\}=x_{ij}^{\mathsf T}\beta+b_j,\qquad b_j\mathrel{\overset{\mathrm{iid}}{\sim}}N(0,\tau^2).

Here \(g\) is the link function, \(\beta\) is the fixed-effects vector, and \(\tau^2\) is the random-intercept variance. Conditional on the random effects and covariates, the responses follow the chosen outcome distribution and are independent under the usual GLMM specification. The clusters are independent, and the random effects are assumed independent of the covariates. A shape check is meaningful only relative to these assumptions and the fitted fixed-effects structure.

The normality assumption applies to the latent \(b_j\), not to the observed outcomes, the fitted probabilities, or the residuals. In a binary GLMM, for instance, the observed cluster proportions are discrete and bounded, even when the latent effects are normally distributed. Thus, a bell-shaped histogram of observed proportions is neither required nor generally expected.

What to Simulate and Compare

A useful check starts with a cluster-level discrepancy chosen to be sensitive to a plausible departure from the assumed shape. Examples include the number of clusters with unusually high or low event rates, the range of cluster summaries, the largest absolute standardized cluster residual, or a measure of asymmetry across cluster summaries. The summary should reflect the suspected failure mode, rather than being chosen only because it is easy to calculate.

For a check of the population random-effects distribution, simulations should ordinarily be unconditional with respect to the random effects: draw new \(b_j\) values from the assumed distribution, then generate outcomes conditional on those draws. A replication that holds estimated cluster effects fixed can assess other aspects of the outcome model, but it does not ask whether the assumed population distribution for new cluster effects is plausible.

Fit the specified GLMM.
Record the fitted fixed effects, random-effects variance, outcome distribution, link, covariates, and cluster sizes or exposures.

Choose cluster-level summaries.
Specify one or more statistics in advance, such as a tail count or a measure of asymmetry, that would respond to the shape departure of concern.

Generate model replications.
For each replication, draw one random effect per cluster from the fitted distribution, generate outcomes under the fitted conditional model, and calculate the same summaries.

Compare observed and replicated results.
Locate the observed statistic within the simulated distribution and inspect plots of cluster summaries or residuals for structured departures.

When feasible, incorporate parameter uncertainty by simulating parameter values from an appropriate uncertainty distribution or by using a parametric bootstrap. Otherwise, a check based only on fitted parameter estimates can understate uncertainty, particularly when the number of clusters is small. The observed statistic and each replicated statistic must be calculated in the same way, including the same covariate adjustment and cluster-size structure.

Why the Outcome Distribution Does Not Reveal the Random-Effects Shape Directly

The observed cluster outcomes combine variation from the random effects with variation in the conditional outcome distribution. For any cluster summary \(S_j\), the model’s predictive distribution averages over the latent effect:

P(S_j=s\mid x)=\int P(S_j=s\mid b_j,x)\,f(b_j)\,db_j.

This is a mixture: \(f(b_j)\) is the assumed random-effects density, while \(P(S_j=s\mid b_j,x)\) describes outcome variation at a particular random-effect value. Consequently, even a substantial change in the shape of \(f\) may produce only a modest visible change in observed outcomes when clusters are small or outcomes are sparse.

The same point follows from the law of total variance. For a cluster outcome summary \(S_j\),

\operatorname{Var}(S_j\mid x) = E\{\operatorname{Var}(S_j\mid b_j,x)\mid x\} + \operatorname{Var}\{E(S_j\mid b_j,x)\mid x\}.

The first term is average within-cluster outcome variation; the second is variation in the conditional cluster means induced by the random effects. A discrepancy in cluster summaries can therefore arise from a wrong random-effects shape, but it may also reflect a misspecified conditional outcome distribution, an omitted covariate, or dependence not represented in the model. The simulation check diagnoses predictive mismatch; it does not uniquely identify its cause.

Worked Example: Tail Counts in Sparse Binary Clusters

Suppose a fitted random-intercept logistic model is used for eight hypothetical clinics, each with ten patients. The observed numbers of events are \(0,0,1,1,2,3,5,8\), for a total of \(20\) events among \(80\) patients. Consider a lower-tail statistic \(T_0\), the number of clinics with no events, and an upper-tail statistic \(T_7\), the number with at least seven events.

Observed statistics. Two clinics have zero events, so \(T_0=2\). One clinic has at least seven events (the clinic with eight), so \(T_7=1\). The overall event proportion is \(20/80=0.250\), but that overall proportion does not describe the distribution of events across clinics.

In a hypothetical diagnostic run with 1,000 unconditional replications, suppose the central 95% simulation ranges are 0–4 for \(T_0\) and 0–2 for \(T_7\). The observed values, 2 and 1, fall within both ranges. This result gives no conspicuous evidence, for these two tail-count summaries, that the fitted model’s random-intercept distribution fails to reproduce the observed cluster extremes.

The conclusion is deliberately limited. Being inside a simulation range does not establish that the normal random-effects assumption is true. It says only that these selected statistics are not unusually large or small relative to the fitted model’s replications. Other features, such as the ordering of clinic rates or a particular pattern of asymmetry, may still merit inspection.

Worked Example: A Maximum Residual Check

Consider a second hypothetical binary-outcome analysis with equal cluster sizes. After adjusting for the fitted covariates, the eight standardized cluster residual summaries are \(-1.1,-0.8,-0.4,0.0,0.2,0.4,0.7,2.6\). Define \(T_{\max}\) as the largest absolute value:

T_{\max}=\max_j |r_j| =\max(1.1,0.8,0.4,0.0,0.2,0.4,0.7,2.6)=2.6.

Suppose 1,000 unconditional simulations from the fitted GLMM produce a median \(T_{\max}\) of \(1.9\) and a 95th percentile of \(3.1\). Since \(2.6<3.1\), the observed maximum is not beyond this simulated upper reference point. It is larger than the median, so the most extreme cluster is worth inspecting, but this check alone does not establish a distributional failure.

The comparison is valid only if the residual summary is computed identically for the observed data and every replication. The simulated reference distribution should also reflect the actual cluster sizes and fitted covariates. A residual of \(2.6\) is not automatically evidence against normal random effects: it is the comparison with the model’s replicated maxima, not a universal cutoff, that matters here.

Worked Example: A Pattern Suggesting Too Many Extreme Clusters

Suppose twelve hypothetical clusters each contain twenty patients. Their observed event counts are \(0,0,1,1,2,3,16,17,18,19,19,20\). The total is \(116\) events out of \(240\), giving an overall proportion of \(116/240=0.483\). Define \(T_{\mathrm{mid}}\) as the number of clusters with between 4 and 16 events, inclusive. The observed counts in that interval are only the count of 16, so \(T_{\mathrm{mid}}=1\).

Assume, for illustration, that 1,000 unconditional replications from a fitted normal random-intercept model give a central 95% range of 2–9 for \(T_{\mathrm{mid}}\), and only 18 replications have a value as small as or smaller than the observed value. The observed statistic is below the reference range. The data contain many very low and very high cluster counts but few intermediate counts, a pattern the fitted model reproduces infrequently.

This is a warning to investigate, not proof that the random effects follow a mixture distribution. The pattern might be associated with an omitted cluster-level predictor, a misspecified conditional model, or chance in a sample of twelve clusters. The discrepancy is evidence that the fitted model struggles with this particular feature; it does not identify which assumption is responsible.

Reading Residual Patterns and Simulation Comparisons

A numerical discrepancy is more informative when paired with a plot. Plot cluster summaries or appropriately standardized residual summaries against fitted cluster means, cluster size, and relevant covariates. Look for systematic skew, separated groups, an excess of extreme clusters, or a trend with cluster size. For sparse outcomes, raw proportions have different precision when denominators differ; comparisons should therefore use model-based replicated summaries with the same denominators rather than treating all proportions as equally precise.

If the check indicates a mismatch, examine whether the pattern persists under reasonable choices of statistic and whether parameter uncertainty changes the comparison. Then assess plausible explanations. An asymmetric cluster pattern might motivate considering a skewed random-effects distribution; an apparent two-group pattern may instead call for a measured cluster-level covariate. A heavier-tailed distribution may be a reasonable alternative if extreme cluster effects are repeatedly underpredicted. Such alternatives should be compared using the same predictive targets and substantive knowledge, not selected solely because they improve one observed statistic.

Common Pitfalls and Limitations

Reading a histogram of estimated random effects as if it were the latent distribution. Estimated effects are shrunk toward the population mean, with differing precision across clusters. A small number of estimates can look irregular even when the assumed distribution is adequate. Use outcome-based replications as the primary check.
Checking only the variance. A model can have a plausible variance and still miss skewness or tail behavior. Conversely, a tail discrepancy can arise from conditional outcome variation rather than the random-effects shape. Choose statistics that target shape and interpret them alongside other model checks.
Conditioning on fitted effects when assessing their population distribution. Replications that hold cluster effects fixed do not test whether new effects drawn from the assumed distribution generate plausible between-cluster patterns.
Treating a simulation tail proportion as a formal, calibrated p-value. Predictive tail proportions are descriptive measures of mismatch, and fitting the model using the same data affects their reference distribution. A small proportion flags a feature to investigate; it is not the probability that the model is true or false.

The check has limited power when there are few clusters, small cluster sizes, or very sparse outcomes. The observation-level sampling noise can obscure the latent distribution, and different random-effects distributions may yield similar predictive outcomes over the observed covariate range. No simulation check can verify an assumption; it can show whether the assumption has consequences that are visibly inconsistent with the data.

Key takeaway: Assess random-effects shape by asking whether unconditional replications from the fitted GLMM reproduce selected cluster-level patterns. A discrepancy is a model-fit warning, not a unique diagnosis, and fitted random-effect estimates are not direct observations of the latent distribution.

Check Your Understanding

Answer each question, then compare with the solutions below.

In a random-intercept GLMM, what distributional assumption is being assessed when the check targets the shape of \(b_j\)?
Why are simulations that hold all fitted cluster effects fixed insufficient for checking the assumed population distribution of those effects?
A cluster-level tail statistic is 4 in the observed data, while the simulated 95% range is 1–6. What is a careful interpretation?
Why can a skewed histogram of estimated cluster effects fail to demonstrate that the latent random-effects distribution is skewed?

Solutions. (1) The check assesses whether the assumed distribution for the latent cluster effects—often independent \(N(0,\tau^2)\) random intercepts—can generate plausible outcome patterns across clusters. It is not a test that the observed outcomes themselves are normal. (2) Fixed-effect replications reuse the estimated cluster effects rather than drawing new effects from the assumed population distribution, so they do not assess how that distribution generates between-cluster variation. (3) The observed value is within the simulated reference range, so this statistic does not show a conspicuous discrepancy; it does not prove the model or the random-effects distribution is correct. (4) Estimated effects are uncertain and shrunk, often by different amounts across clusters, and there may be few clusters. Their histogram is therefore not a direct sample from the latent distribution and can look skewed by estimation noise or sampling variation.