The Random-Effects Distribution In Marginal Estimands

Why the Random-Effects Distribution Defines the Target

A GLMM specifies how an outcome mean depends on covariates and cluster effects. A marginal estimand adds a population question: over which distribution of clusters is that conditional mean being averaged? The answer is not supplied by the fixed-effect coefficients alone. It depends on the distribution assigned to the random effects and on whether that distribution represents the cluster population relevant to the question.

For example, a random-intercept model can describe conditional response means for clusters with particular intercept values. Its marginal mean for a specified covariate value averages those conditional means across a distribution of intercepts. If that distribution is changed, the average may change even when the conditional model and its coefficients are held fixed. The distribution is therefore both a modeling assumption and part of the definition of the population-averaged estimand.

Definition: Let $Y$ be an outcome, $x$ a fixed-effect covariate vector, $b$ a cluster random-effect vector, $z$ its design vector, and $G$ a probability distribution for $b$. Suppose the conditional mean is $E(Y\mid x,b)=h(x^\top\beta+z^\top b)$, where $h$ is the inverse link. If $G$ is the target distribution of cluster effects and is common across the covariate values considered, the marginal mean is $$ \mu_G(x)=\int h(x^\top\beta+z^\top b)\,dG(b). $$ It is the expected outcome for the specified covariates when cluster effects are averaged over $G$.

This definition concerns a mean, not the entire marginal outcome distribution. A marginal variance also involves the conditional outcome variance and variation in the conditional mean across $b$. Further, $G$ needs a population interpretation: it might represent the distribution across clusters in a defined target population, or be a parametric approximation used for that population. Those are not automatically the same thing.

Throughout this tutorial, $G$ is assumed not to vary with $x$. This makes the target distribution explicit and fixed for the comparisons below. When cluster-effect distributions depend on covariates, the averaging distribution itself needs additional specification.

How Distributional Shape Enters the Marginal Mean

If $h$ is linear in its argument, averaging the conditional mean can often be summarized by the mean of $b$. In a GLMM, however, $h$ is commonly nonlinear. The order of operations then matters: apply the inverse link to each conditional linear predictor, and average the resulting response-scale means. In general, the result depends on more than the mean and variance of $G$.

A Taylor expansion gives a local justification. Consider a scalar random intercept $b$ with $E_G(b)=0$, and write $g(b)=h(\eta+b)$, where $\eta$ is fixed. If $g$ has three derivatives and $E_G(|b|^3)$ is finite, expansion around zero gives

E_G\{g(b)\} = g(0)+\frac{g''(0)}{2}E_G(b^2) +\frac{g'''(0)}{6}E_G(b^3)+E_G\{R_4(b)\},

where $R_4(b)$ is the Taylor remainder. The first term uses the conditional mean at $b=0$; the next terms involve moments of the random-effects distribution. This expansion is a justification of why distributional shape can matter, not a general exact formula. The remainder can matter substantially, especially when effects are dispersed or the inverse link changes quickly over the range of $b$.

For a normal random intercept under a log link, an exact formula is available. If $b\sim N(0,\sigma_b^2)$ and the conditional mean is $\exp(\eta+b)$, then $E\{\exp(b)\}=\exp(\sigma_b^2/2)$, so the marginal mean is $\exp(\eta+\sigma_b^2/2)$. This result relies on both the log link and the normal distribution; it is not a generic GLMM formula.

Worked Example: A Normal Random Intercept With a Log Link

Suppose a hypothetical count outcome has conditional mean $\lambda(b)=\exp(\log(3)+b)$ for a fixed covariate pattern. Let the cluster random intercept satisfy $b\sim N(0,0.36)$. Find the marginal expected count per unit of exposure.

The assumptions for this calculation are a log-link conditional mean, a normal random intercept with mean zero and variance $0.36$, and a common $G$ across the covariate pattern being evaluated. The normal distribution has a finite exponential moment, so the marginal mean is finite.

Integrating the conditional mean over $G$ gives

\mu_G =E_G\{\exp(\log(3)+b)\} =3E_G\{\exp(b)\} =3\exp\left(\frac{0.36}{2}\right) =3e^{0.18} \approx 3.592.

The marginal expected count is about 3.592 counts per unit of exposure. It exceeds the conditional mean of 3 at $b=0$, because the exponential function is convex: positive and negative deviations around zero do not cancel when exponentiated. The answer describes clusters averaged according to the specified normal distribution; it is not the conditional mean for every cluster.

Same Moments Do Not Guarantee the Same Estimand

The next example compares two distributions with the same mean and variance. Both are plausible mathematical specifications, but they place probability mass differently. A nonlinear inverse link can distinguish them. The comparison holds the linear predictor and fixed-effect parameters constant so that the change in the marginal mean is attributable to $G$.

Worked Example: Two Logistic Random-Intercept Distributions

For a hypothetical binary outcome, let the conditional probability be $\operatorname{logit}^{-1}(1+b)$, where $\operatorname{logit}^{-1}(u)=1/(1+e^{-u})$. Compare:

$G_1$: probability $0.8$ at $b=0.5$ and probability $0.2$ at $b=-2$.
$G_2$: probability $0.5$ at $b=1$ and probability $0.5$ at $b=-1$.

For $G_1$, $E(b)=0.8(0.5)+0.2(-2)=0$ and $E(b^2)=0.8(0.25)+0.2(4)=1$, so its variance is 1. For $G_2$, symmetry gives mean 0 and $E(b^2)=1$, also giving variance 1. The distributions have the same first two moments.

Under $G_1$, the integrated probability is

\begin{aligned} \mu_{G_1} &=0.8\operatorname{logit}^{-1}(1.5) +0.2\operatorname{logit}^{-1}(-1)\\ &=0.8(0.817574)+0.2(0.268941)\\ &\approx 0.7078. \end{aligned}

Under $G_2$, it is

\begin{aligned} \mu_{G_2} &=0.5\operatorname{logit}^{-1}(2) +0.5\operatorname{logit}^{-1}(0)\\ &=0.5(0.880797)+0.5(0.500000)\\ &\approx 0.6904. \end{aligned}

The two marginal probabilities differ by about $0.0174$, or 1.74 percentage points, despite the matching means and variances of $b$. Thus, for this fixed covariate pattern, the population-averaged probability is about 70.78% under $G_1$ and 69.04% under $G_2$. The difference is not evidence that one distribution is correct; it shows that the estimand depends on the distributional choice.

Choosing and Interpreting a Distribution

The distribution should be chosen in relation to the population and estimand, not solely because it is a convenient default. A normal $G$ is a common parsimonious model: it is continuous, symmetric, and characterized by its mean and covariance. It can be appropriate when that shape is a reasonable approximation and the relevant moments exist. But a normal model does not follow merely from the fact that effects are called “random.”

Alternative specifications may be more interpretable for some targets. A discrete distribution can represent a population composed of a few cluster types, with probabilities reflecting their target-population shares. An empirical distribution can average over observed cluster effects, although fitted effects are estimated and typically shrunk rather than directly observed population values. A mixture or other flexible distribution can represent skewness or multiple subpopulations, but adds parameters and requires enough information to estimate them.

These alternatives answer different questions unless they are deliberately calibrated to the same target. For example, a distribution estimated from a convenience sample of clusters may not represent a policy-relevant target population. Reweighting or defining a target distribution can change the marginal estimand even if the conditional model is unchanged. This is closely related to choosing a covariate distribution for standardization, but here the object being averaged over is the cluster-effect distribution.

Worked Example: Comparing an Empirical Distribution With a Normal Approximation

Suppose a hypothetical target population has three cluster-intercept values with probabilities $0.25, 0.50,$ and $0.25$, at $b=-0.5, 0,$ and $1$, respectively. A count mean follows a log link and equals $\exp(\log(2)+b)$ conditionally. Find the marginal mean under this discrete target distribution and compare it with a normal distribution having the same mean and variance.

First, the discrete distribution has mean $0.25(-0.5)+0.50(0)+0.25(1)=0.125$. Its second moment is $0.25(0.25)+0.50(0)+0.25(1)=0.3125$; therefore its variance is $0.3125-(0.125)^2=0.296875$.

Direct integration under the specified discrete $G$ gives

\begin{aligned} \mu_G &=0.25\{2e^{-0.5}\}+0.50(2)+0.25\{2e^1\}\\ &=0.25(1.213061)+1+0.25(5.436564)\\ &\approx 2.6624. \end{aligned}

Now approximate $G$ by a normal distribution with the same mean, $0.125$, and variance, $0.296875$. Under that normal distribution, the log-link formula gives

\mu_{\mathrm{normal}} =2\exp\left(0.125+\frac{0.296875}{2}\right) =2e^{0.2734375} \approx 2.629.

The two marginal expected counts are approximately 2.662 and 2.629 per unit of exposure. The normal approximation preserves the first two moments but not the full distribution, so the integrated mean is not identical. The discrete answer is the appropriate one if those three values and weights define the target cluster population; the normal answer corresponds to a different distributional assumption.

Assumptions, Diagnostics, and Failure Modes

Before interpreting $\mu_G(x)$, check that the assumed distribution is appropriate to the stated target and that the integral exists. For a binary outcome, the conditional mean lies between zero and one, so integration against any proper probability distribution is finite. For unbounded outcomes with a log link, finite values of the conditional mean for each finite $b$ do not by themselves guarantee a finite marginal mean. The distribution must have the needed exponential moment. For example, the normal random-intercept calculation above is finite, whereas some heavy-tailed distributions can have an infinite exponential moment.

Distributional diagnostics are indirect. The random effects are latent, and fitted cluster effects are estimated with uncertainty and often shrinkage. Their empirical histogram should not be treated as if it were a sample of error-free $b$ values. Useful checks include comparing observed and simulated cluster-level outcome patterns, inspecting conditional and marginal residual summaries appropriate to the outcome, and assessing whether a small number of clusters dominate the estimated distribution. Most importantly for this estimand, calculate $\mu_G(x)$ under substantively plausible alternatives to the fitted $G$. A notable change indicates that conclusions depend on the distributional assumption.

These checks have limitations. Agreement between observed data and a normal-random-effects model does not prove normality, particularly with few clusters or little information per cluster. A flexible distribution can reduce shape restrictions but may be weakly identified. Sensitivity analysis is therefore not a test that selects a uniquely true $G$; it describes how much the reported target mean changes across reasonable assumptions.

Common pitfall: “The random effects have mean zero and variance one, so every distribution with those moments gives the same marginal probability.” This reasoning is generally false for a nonlinear inverse link. As the logistic example shows, two distributions with those exact moments can produce different marginal probabilities. Integrate the response-scale conditional mean under the specified $G$, and state which population that $G$ represents.

What the Distributional Choice Does—and Does Not—Change

It helps to distinguish three decisions. First, the conditional model specifies the response mean given covariates and a particular $b$. Second, the distribution $G$ specifies how cluster effects are represented in the target population and hence defines the averaging operation. Third, the data and estimation procedure determine how the model parameters and distributional features are estimated. Uncertainty in estimated parameters is distinct from sensitivity to the choice of distributional family or target population.

A normal random-effects model is a parametric approach that makes integration and estimation tractable, with shape restrictions that may be consequential. A discrete or empirical distribution can make population weights more transparent, but its values and weights must be supported by the target definition and data. A GLM does not include a cluster-effect distribution, so it does not by itself provide this same random-effects-based averaging definition. A marginal model can specify response means directly, but that is a different modeling route, not a reason to interpret a GLMM’s conditional coefficients as its marginal estimands.

The interpretation is always conditional on both the covariate pattern and $G$: “the expected response at $x$, averaged over cluster effects distributed as $G$.” If $G$ is a model-based approximation rather than a known population distribution, say so. If the target is a particular set of clusters, an average over that set may be more relevant than an unqualified normal distribution. These qualifications matter because marginal effects and contrasts inherit the population definition used for their component means.

Specify the target.
State which population of clusters the marginal mean is intended to describe.

Specify $G$.
Give the distributional family, its parameters or empirical weights, and the rationale for treating it as the target cluster-effect distribution.

Integrate on the response scale.
Evaluate $h(x^\top\beta+z^\top b)$ across $G$, and verify the integral is finite for the outcome and link.

Assess sensitivity.
Compare the estimand under plausible alternative distributions and explain whether the population-averaged conclusion changes materially.

Check Your Understanding

Use the definitions and calculations above to answer each question. Full solutions follow the questions.

Write the marginal mean for a random-intercept GLMM with conditional mean $h(x^\top\beta+b)$ and random-intercept distribution $G$.
Why can two random-intercept distributions with the same mean and variance yield different marginal probabilities in a logistic GLMM?
For $b\sim N(0,0.16)$ and conditional mean $\exp(\log(4)+b)$, calculate the marginal mean.
For the discrete distribution assigning probability $0.5$ to each of $b=-1$ and $b=1$, calculate the marginal mean when the conditional probability is $\operatorname{logit}^{-1}(b)$.

Solutions. (1) The definition is $\mu_G(x)=\int h(x^\top\beta+b)\,dG(b)$; $G$ is the assumed distribution of cluster effects in the target cluster population. (2) The logistic inverse link is nonlinear, so integrating it can depend on distributional shape and higher moments, not only the mean and variance. (3) The normal log-link formula gives $\exp\{\log(4)+0.16/2\}=4e^{0.08}\approx 4.333$ expected counts per unit. (4) The mean is $0.5\operatorname{logit}^{-1}(-1)+0.5\operatorname{logit}^{-1}(1)=0.5(0.268941)+0.5(0.731059)=0.5000$, by the symmetry of the logistic function.