Deriving Marginal Means By The Law Of Total Expectation

From Conditional Responses to Population Averages

A generalized linear mixed model describes a response conditional on random effects. That conditional description is useful for understanding a particular cluster or subject, but it does not by itself give the mean response across the random-effects population. To obtain that population-level quantity, keep the covariate pattern fixed and average over the random-effects distribution.

The same distinction matters for variability. Responses can vary because of their conditional sampling distribution, and their conditional means can vary across random effects. The marginal variance includes both sources. The law of total expectation and the law of total variance provide the exact relationships.

Definition: Let \(Y\) be a GLMM response, \(X=x\) a fixed covariate pattern, and \(B\) the random-effects vector with distribution \(F_B\). Write \(m(b,x)=E(Y\mid B=b,X=x)\) and \(v(b,x)=\operatorname{Var}(Y\mid B=b,X=x)\). The population-averaged mean at \(x\) is \(E(Y\mid X=x)\), where the expectation averages over \(B\). The corresponding marginal variance is \(\operatorname{Var}(Y\mid X=x)\), also averaging over \(B\).

In a standard GLMM, the conditional mean is specified through a link function, for example \(g\{m(b,x)\}=x^\mathsf{T}\beta+z^\mathsf{T}b\), with a conditional response distribution whose parameters determine \(v(b,x)\). The integration below uses the resulting conditional mean and variance, together with the assumed distribution of \(B\). The covariate pattern \(x\) remains fixed throughout; these quantities are not additionally averaged over a population distribution of covariates.

The Two Averaging Identities

Conditioning first on \(B\) and then averaging over its distribution gives the marginal mean:

E(Y\mid X=x) = E_B\{m(B,x)\} = \int m(b,x)\,dF_B(b).

For a continuous random-effects distribution, the integral is taken with respect to its probability density; for a discrete distribution, it is a probability-weighted sum. The expression requires the relevant expectation to exist. It is the conditional mean \(m(b,x)\) that is averaged, not a response observed at one particular value of \(b\).

The law of total variance gives the matching variance identity:

\operatorname{Var}(Y\mid X=x) = E_B\{v(B,x)\} + \operatorname{Var}_B\{m(B,x)\}.

The first term is the average conditional response variance. The second is the variance, across the random-effects population, of the conditional mean. Both terms are nonnegative, so the marginal variance cannot be smaller than the average conditional variance.

Why the Variance Decomposes This Way

A direct derivation makes clear why neither source of variation can be omitted. Let \(\mu(x)=E(Y\mid X=x)=E_B\{m(B,x)\}\). Add and subtract the conditional mean:

Y-\mu(x) = \{Y-m(B,x)\} + \{m(B,x)-\mu(x)\}.

Squaring and taking expectations produces two squared terms and a cross term. The cross term is zero because, conditional on \(B\) and \(X=x\), the first bracket has mean zero:

\begin{aligned} &E\big[\{Y-m(B,x)\}\{m(B,x)-\mu(x)\}\mid X=x\big]\\ &\quad = E_B\Big[ \{m(B,x)-\mu(x)\} E\{Y-m(B,x)\mid B,X=x\} \Big]=0. \end{aligned}

The expected square of \(Y-m(B,x)\) is \(E_B\{v(B,x)\}\), by the definition of conditional variance. The expected square of \(m(B,x)-\mu(x)\) is \(\operatorname{Var}_B\{m(B,x)\}\). Adding these terms proves the total-variance identity.

Key result: At a fixed covariate pattern, the marginal response mean is the random-effects average of the conditional mean. The marginal response variance is the average conditional variance plus the random-effects variance of the conditional mean. These identities require finite relevant moments; they do not require the conditional response to be normally distributed.

Worked Calculations

Worked Example: A Gaussian Random-Intercept Model

Suppose a hypothetical measurement \(Y\), in response units, follows \(Y\mid B=b\sim N(10+b,9)\), with a random intercept \(B\sim N(0,4)\). The conditional model has mean \(m(b)=10+b\) and variance \(v(b)=9\). The random intercept is independent of the conditional response noise, as specified by this model, and both required moments are finite.

Marginal mean. Average the conditional mean over \(B\):

$$ E(Y)=E(10+B)=10+E(B)=10+0=10. $$

Marginal variance. The average conditional variance is \(E\{v(B)\}=9\). The conditional mean \(10+B\) has variance \(\operatorname{Var}(10+B)=\operatorname{Var}(B)=4\). Therefore

\operatorname{Var}(Y)=9+4=13.

The marginal standard deviation is \(\sqrt{13}\approx3.606\) response units. The value 9 describes within-random-effect response variation; the additional 4 comes from differences in conditional means across the random-intercept population.

Worked Example: A Poisson Response With Two Possible Random-Effect Values

Consider a hypothetical count response. Let \(B\) equal \(-\log(2)\) or \(\log(2)\), each with probability \(1/2\), and specify \(Y\mid B=b\sim\operatorname{Poisson}(\lambda(b))\), where \(\lambda(b)=4e^b\). Thus the conditional mean and variance are both \(\lambda(b)\). The two possible rates are \(4(1/2)=2\) and \(4(2)=8\), so all relevant moments exist.

Marginal mean. Average the two conditional means:

E(Y)=\frac12(2)+\frac12(8)=5.

Average conditional variance. Since a Poisson variable has conditional variance equal to its conditional mean,

E\{\operatorname{Var}(Y\mid B)\} = \frac12(2)+\frac12(8)=5.

Variance of the conditional mean. The mean rate is 5, and its two equally likely values differ from 5 by \(-3\) and \(3\):

\operatorname{Var}\{E(Y\mid B)\} = \frac12(2-5)^2+\frac12(8-5)^2 = \frac12(9)+\frac12(9)=9.

Consequently, the marginal variance is \(5+9=14\) count-squared units. The conditional Poisson variance alone would give 5, which misses the extra variation in rates across the random-effects population.

Worked Example: A Binary Response With Two Conditional Probabilities

For a hypothetical binary response \(Y\), suppose a random effect has two equally likely values. Conditional on the first, \(P(Y=1\mid B)=0.2\); conditional on the second, \(P(Y=1\mid B)=0.6\). This specifies a Bernoulli response conditional on \(B\), so its conditional mean is \(p(B)\) and its conditional variance is \(p(B)\{1-p(B)\}\). Both probabilities are valid, and the moments are finite.

Marginal mean. Since \(Y\) is binary, this mean is also the population-averaged probability of a response equal to 1:

E(Y)=\frac12(0.2)+\frac12(0.6)=0.4.

Average conditional variance.

E\{\operatorname{Var}(Y\mid B)\} = \frac12\{0.2(0.8)\} + \frac12\{0.6(0.4)\} = \frac12(0.16)+\frac12(0.24)=0.20.

Variance of the conditional mean.

\operatorname{Var}\{E(Y\mid B)\} = \frac12(0.2-0.4)^2+\frac12(0.6-0.4)^2 = \frac12(0.04)+\frac12(0.04)=0.04.

Thus the marginal variance is \(0.20+0.04=0.24\). As a check, a marginal Bernoulli response with probability 0.4 has variance \(0.4(1-0.4)=0.24\), agreeing with the decomposition.

Repeated Responses and Practical Computation

When two responses share a random effect, averaging can also induce marginal dependence. For responses \(Y_1\) and \(Y_2\) at fixed covariates, the covariance identity is

\operatorname{Cov}(Y_1,Y_2\mid X) = E\{\operatorname{Cov}(Y_1,Y_2\mid B,X)\} + \operatorname{Cov}\{m_1(B),m_2(B)\}.

If the responses are conditionally independent given \(B\) and their covariates, the first term is zero. Their marginal covariance is then the covariance between their conditional means across the random-effects distribution. This is one way shared random effects create within-cluster association, even when the responses are conditionally independent.

For a simple random-effects distribution, the required expectations may be evaluated exactly, as in the examples. For a continuous distribution without a convenient closed-form integral, numerical quadrature or simulation can approximate them. In simulation, draw \(b^{(1)},\ldots,b^{(M)}\) from the specified distribution and approximate the mean by \(M^{-1}\sum_{r=1}^M m(b^{(r)},x)\). To estimate the variance, use the same draws to approximate \(E_B\{v(B,x)\}\) and \(\operatorname{Var}_B\{m(B,x)\}\). Simulation error should be checked by increasing \(M\) or using repeated simulations.

Interpretation, Alternatives, and Failure Modes

A conditional prediction at \(B=b\) describes the model response for that particular random-effect value. A population-averaged prediction describes the mean response over the random-effects distribution at the specified covariates. A prediction using an estimated random effect for an observed cluster targets a cluster-specific quantity, not automatically the population average. The earlier material on conditional and population-averaged estimands distinguishes these targets; the averaging identities here show how to calculate the latter from a conditional model.

A marginal variance is also not just a synonym for the conditional variance. The first term in the decomposition averages conditional response variances; the second accounts for heterogeneity in conditional means. In the Poisson example, reporting 5 as the marginal variance would omit the rate heterogeneity and understate the result, which is 14.

Using one random-effect value as the population average. Setting \(B=0\) gives a conditional quantity at \(b=0\), not generally an integral over \(F_B\). It agrees with the population mean only under additional conditions.
Dropping the variance of conditional means. This leaves only average within-cluster variation and omits variability across random effects.
Averaging variances at the wrong level. The marginal variance requires both \(E_B\{v(B,x)\}\) and \(\operatorname{Var}_B\{m(B,x)\}\); averaging conditional standard deviations or variances alone does not give it.
Using the wrong random-effects distribution. An integral is only as appropriate as the distribution and covariance structure assumed for \(B\). Poor fit or misspecification can make the population average misleading.
Ignoring numerical error. Inaccurate quadrature or too few simulation draws can distort either component. Check convergence and, for simulation, Monte Carlo variability.

These identities do not select a random-effects distribution, guarantee that a fitted GLMM is well specified, or account for uncertainty in estimated model parameters. They describe the population moments implied by a specified model and parameter values. In applied work, model adequacy and uncertainty in estimated quantities require additional assessment.

Check Your Understanding

Try each problem, then compare with the solutions below.

A response has \(E(Y\mid B)=6+B\), where \(E(B)=1\). What is its marginal mean at the fixed covariate pattern?
If \(E\{\operatorname{Var}(Y\mid B)\}=7\) and \(\operatorname{Var}\{E(Y\mid B)\}=2.5\), find the marginal variance and identify its two components.
A binary response has conditional probabilities 0.1 and 0.5 with equal probabilities for the two random-effect values. Find its marginal mean and variance.
Two responses are conditionally independent given a shared random effect. What term determines their marginal covariance?

Solutions. (1) \(E(Y)=E(6+B)=6+1=7\). (2) The marginal variance is \(7+2.5=9.5\); 7 is average conditional variance and 2.5 is variance of the conditional mean. (3) The mean is \(\{0.1+0.5\}/2=0.3\). The average conditional variance is \(\{0.1(0.9)+0.5(0.5)\}/2=(0.09+0.25)/2=0.17\). The conditional means have variance \(\{(0.1-0.3)^2+(0.5-0.3)^2\}/2=0.04\). Thus the marginal variance is \(0.17+0.04=0.21\), also \(0.3(0.7)=0.21\). (4) The covariance of the two conditional means across the random-effects distribution determines the marginal covariance, because the expected conditional covariance is zero under conditional independence.