Marginal Means and Conditional Means

Two Mean Questions, Two Conditioning Sets

For a clustered or repeated-response study, a mean can describe different information sets. One question asks for the expected response among units with specified observed covariates. Another asks for the expected response among units with those covariates and a specified latent subject characteristic, such as an unobserved propensity to experience an event. These are not interchangeable descriptions: the second conditions on more information, while the first averages over the latent information that remains unspecified.

Write $Y_{ij}$ for the response from subject $i$ at occasion $j$, $X_{ij}$ for its observed covariates, and $B_i$ for a latent subject-level quantity. A random-intercept model is one example: $B_i$ represents unobserved differences between subjects and is shared across their occasions. The notation does not require every clustered model to have such a latent effect.

Definition: The conditional mean given observed covariates and latent information is $m(x,b)=E[Y_{ij}\mid X_{ij}=x,B_i=b]$. The marginal mean given observed covariates is $\mu(x)=E[Y_{ij}\mid X_{ij}=x]$. When $Y_{ij}$ is integrable and a conditional distribution of $B_i$ given $X_{ij}$ is specified, the latter averages the former over that distribution: $$ \mu(x)=\int m(x,b)\,dF_{B_i\mid X_{ij}=x}(b). $$ If $B_i$ and $X_{ij}$ are independent, this is an average over the common distribution $F_{B_i}$. If they are dependent, the relevant distribution is instead the conditional distribution $F_{B_i\mid X_{ij}=x}$.

These are conditional expectations, so their values at individual covariate points are understood through suitable versions of the conditional-mean functions. In practice, the model supplies such functions. The distinction is about what is conditioned on, not whether one mean is intrinsically more correct. The appropriate mean depends on the question and the information available for the prediction or description.

Why Averaging Gives the Observed-Covariate Mean

The averaging rule follows from the tower property of conditional expectation. It does not require a particular response distribution, link function, or independence assumption. It does require that the response have a finite expectation; conditional distributions and conditional means are understood to exist.

Theorem: If $E[|Y_{ij}|]<\infty$ and the information given by $X_{ij}$ is contained in the information given jointly by $(X_{ij},B_i)$, then $$ E[Y_{ij}\mid X_{ij}] = E\!\left[E[Y_{ij}\mid X_{ij},B_i]\mid X_{ij}\right] \quad\text{almost surely}. $$ Thus the mean conditional on observed covariates is the conditional average of the mean that also conditions on the latent quantity.

Proof. Let $\mathcal G$ be the information generated by $X_{ij}$, and let $\mathcal H$ be the information generated jointly by $X_{ij}$ and $B_i$. Then $\mathcal G\subseteq\mathcal H$. Set $Z=E[Y_{ij}\mid\mathcal H]$, which is integrable because $Y_{ij}$ is integrable. For any event $A\in\mathcal G$, we also have $A\in\mathcal H$, so the defining property of conditional expectation gives $$ E[\mathbf{1}_A E[Z\mid\mathcal G]] =E[\mathbf{1}_A Z] =E[\mathbf{1}_A Y_{ij}]. $$ Therefore $E[Z\mid\mathcal G]$ satisfies the defining property of $E[Y_{ij}\mid\mathcal G]$. Uniqueness up to almost-sure equality proves the result. Expressing the conditional expectation over $B_i$ as an integral gives the stated averaging interpretation. $\square$

A key implication is that averaging is applied to the response mean, not necessarily to the latent quantity first. In general, evaluating a model at an average latent value does not perform the required averaging. Nonlinear response-mean functions make this difference especially visible.

Worked Example: A Binary Response With a Two-Point Latent Effect

Suppose $Y_{ij}$ is binary, $B_i$ is independent of $X_{ij}$, and $P(B_i=\log 3)=P(B_i=-\log 3)=1/2$. At a particular covariate value $x$, suppose the conditional response probability is $$ P(Y_{ij}=1\mid X_{ij}=x,B_i=b) =\frac{\exp(\log 2+b)}{1+\exp(\log 2+b)}. $$ Calculate both the probability conditional on $B_i$ and the probability given $X_{ij}=x$ alone.

When $B_i=\log 3$, the conditional odds are $\exp(\log 2+\log 3)=6$, so the conditional probability is $6/(1+6)=6/7$. When $B_i=-\log 3$, the odds are $\exp(\log 2-\log 3)=2/3$, so the probability is $(2/3)/(1+2/3)=2/5$.

Because the two latent values are equally likely at this $x$, the marginalized probability is $$ P(Y_{ij}=1\mid X_{ij}=x) =\frac{1}{2}\left(\frac{6}{7}+\frac{2}{5}\right) =\frac{1}{2}\left(\frac{30}{35}+\frac{14}{35}\right) =\frac{22}{35} \approx 0.629. $$

If instead we set $b=0$, the conditional probability is $2/(1+2)=2/3\approx0.667$, not $22/35$. Setting the latent effect to zero gives a value at one particular latent level; it does not average over the stated distribution.

Examples With Log- and Identity-Scale Means

The exact impact of averaging depends on the conditional mean function and the distribution of the latent quantity. The next example makes that calculation explicit for a log link. No assumption that the latent effect is normally distributed is needed for the first calculation.

Result: Suppose $E[Y_{ij}\mid X_{ij}=x,B_i=b]=\exp(\eta(x)+b)$, $B_i$ is independent of $X_{ij}$, and $E[\exp(B_i)]<\infty$. Then $$ E[Y_{ij}\mid X_{ij}=x] =\exp(\eta(x))E[\exp(B_i)]. $$ This follows by substituting the conditional mean into the averaging formula and taking the factor $\exp(\eta(x))$, which is fixed at $x$, outside the expectation over $B_i$.

Worked Example: A Log-Link Mean With a Two-Point Random Intercept

Suppose $E[Y_{ij}\mid X_{ij}=x,B_i=b]=\exp(\log 4+b)$, with $B_i=\log 2$ or $B_i=-\log 2$, each with probability $1/2$, independently of $X_{ij}$. Find the conditional mean at each latent value and the mean given $X_{ij}=x$ alone.

At $b=\log 2$, the mean is $\exp(\log 4+\log 2)=8$. At $b=-\log 2$, it is $\exp(\log 4-\log 2)=2$. Averaging gives $$ E[Y_{ij}\mid X_{ij}=x] =\frac{1}{2}(8)+\frac{1}{2}(2)=5. $$

At $b=0$, the conditional mean is $\exp(\log 4)=4$, which differs from 5. Equivalently, $E[\exp(B_i)]=(2+1/2)/2=5/4$, so the log-link averaging formula gives $4(5/4)=5$. The difference arises because the mean function exponentiates the latent effect before averaging.

A different special case illustrates when averaging does not change the form of the mean. Suppose the conditional mean is linear in a latent additive term, $E[Y_{ij}\mid X_{ij}=x,B_i=b]=\eta(x)+b$, and $E[B_i\mid X_{ij}=x]=0$. Then averaging gives $\eta(x)$. The zero-mean condition is important: if $E[B_i\mid X_{ij}=x]$ is not zero, the marginalized mean is $\eta(x)+E[B_i\mid X_{ij}=x]$. Even in the zero-mean case, the equality concerns the mean; it does not say that individual subjects have $B_i=0$.

Worked Example: Averaging Over Observed Covariates

Suppose a response has conditional means $E[Y\mid X=0]=2$ and $E[Y\mid X=1]=10$, with $P(X=0)=3/4$ and $P(X=1)=1/4$. Find the overall mean, and compare it with the conditional mean at each covariate value.

The tower property, now averaging over the observed covariate distribution, gives $$ E[Y] =E[E[Y\mid X]] =\frac{3}{4}(2)+\frac{1}{4}(10) =\frac{6}{4}+\frac{10}{4} =4. $$

The overall mean is 4, while the two covariate-conditional means are 2 and 10. If the covariate probabilities were instead $P(X=0)=1/4$ and $P(X=1)=3/4$, the overall mean would be $(1/4)(2)+(3/4)(10)=8$, even though the conditional means stayed the same. Thus a mean averaged over covariates depends on the covariate distribution being used.

This is a separate averaging step from integrating over a latent subject effect. When both are present, one can first average over $B_i$ conditional on observed covariates, then average the resulting mean over the distribution of those covariates to obtain an overall population mean.

Reading a Mean Model Carefully

When interpreting a stated mean, check the variables after the conditioning bar. For example, $E[Y_{ij}\mid X_{ij}=x,B_i=b]$ describes responses at a specified covariate value and latent level. By contrast, $E[Y_{ij}\mid X_{ij}=x]$ leaves $B_i$ unspecified and averages over its distribution among units with $X_{ij}=x$. If the model includes several observed covariates, all covariates held fixed should be named or included in $X_{ij}$.

The distribution used for averaging is part of the target. If latent effects depend on observed covariates, averaging with the unconditional distribution of $B_i$ can be incorrect for a particular $x$; the appropriate distribution is $F_{B_i\mid X_{ij}=x}$. Likewise, an overall mean requires a specified distribution of observed covariates. Without that distribution, a covariate-conditional mean does not determine one unique overall mean.

Common Mistakes

Setting the latent effect to zero and calling the result marginal. This evaluates the conditional mean at $b=0$. The marginal mean averages over the relevant distribution of $B_i$, as the binary and log-link examples demonstrate.
Averaging on the wrong scale. For a log-link mean, average $\exp(\eta(x)+B_i)$, not $\eta(x)+E[B_i]$ and then exponentiate, unless additional conditions justify that calculation.
Ignoring dependence between covariates and latent information. If $B_i$ and $X_{ij}$ are dependent, the distribution of $B_i$ used at covariate value $x$ is conditional on $X_{ij}=x$.
Calling a covariate-conditional mean an overall mean. To obtain $E[Y_{ij}]$, average $E[Y_{ij}\mid X_{ij}]$ over a specified distribution of $X_{ij}$.

Key takeaway: A conditional mean is defined by its conditioning information. Removing latent information means averaging over its conditional distribution; removing observed covariates as well means averaging over their distribution too. State both the conditioning set and the averaging distribution to make the target mean unambiguous.

Check Your Understanding

For each question, identify the conditioning information or show the required averaging step.

In a model with a latent subject effect $B_i$, explain the distinction between $E[Y_{ij}\mid X_{ij}=x,B_i=b]$ and $E[Y_{ij}\mid X_{ij}=x]$.
If a conditional mean is $\exp(\eta(x)+B_i)$, what distribution must be averaged over when $B_i$ and $X_{ij}$ are dependent?
A latent effect is equally likely to be $-1$ or $1$, and the conditional mean is $3+B_i$. Calculate the conditional mean given observed covariates alone.
Using the values $E[Y\mid X=0]=2$ and $E[Y\mid X=1]=10$, calculate $E[Y]$ if $P(X=1)=1/2$.
Why does evaluating a nonlinear conditional mean at the average latent effect not generally equal averaging the conditional mean over the latent-effect distribution?