Averaging A Random-Intercept Logistic Model

From a conditional mean to a marginal probability

A random-intercept logistic model describes the response probability for a cluster after accounting for that cluster’s latent intercept. But a population-averaged probability asks a different question: what is the response probability after averaging over the distribution of cluster intercepts? The averaging is straightforward to define, but its result is generally not another logistic curve. This distinction matters whenever predictions or effects are meant to describe a population rather than a cluster with a specified random intercept.

Let $Y_{ij}$ be a binary response for unit $i$ in cluster $j$, and let $x_{ij}$ be a covariate. Consider the random-intercept model

Y_{ij}\mid B_j,x_{ij}\sim \operatorname{Bernoulli}(p_{ij}(B_j)), \qquad \operatorname{logit}\{p_{ij}(B_j)\} =\beta_0+\beta_1x_{ij}+B_j, \qquad B_j\sim N(0,\tau^2).

Here the conditional probability is $p_{ij}(B_j)=\operatorname{expit}(\beta_0+\beta_1x_{ij}+B_j)$, where $\operatorname{expit}(u)=1/(1+\exp(-u))$. The random intercept is assumed to have mean zero and variance $\tau^2$, and to be independent of the covariate in this model. Conditional on $B_j$ and the covariates, responses have the specified Bernoulli means; the random intercept is shared by units in a cluster.

Definition: The marginal response probability at covariate value $x$ is the conditional probability averaged over the random-intercept distribution: $$ \mu(x)=P(Y_{ij}=1\mid x_{ij}=x) =E_{B_j}\!\left[\operatorname{expit}(\beta_0+\beta_1x+B_j)\right]. $$ For the normal random intercept, this expectation is the logistic-normal integral $$ \mu(x)=\int_{-\infty}^{\infty} \operatorname{expit}(\beta_0+\beta_1x+b) \frac{1}{\tau}\phi\!\left(\frac{b}{\tau}\right)\,db, $$ when $\tau>0$, with $\phi$ the standard normal density. If $\tau=0$, the distribution is concentrated at zero and $\mu(x)=\operatorname{expit}(\beta_0+\beta_1x)$.

This integral is an average of probabilities, not an average of log odds. In particular, it is usually not valid to average the random intercept first and then apply the inverse logit: doing so gives $\operatorname{expit}(\beta_0+\beta_1x+E[B_j])$, which is just the conditional probability at $B_j=0$, not the population average.

Approximating the logistic-normal integral

There is generally no elementary closed form for the expectation of an expit of a normally distributed quantity. Numerical integration is therefore a standard way to evaluate it. One simple illustration is three-point Gauss–Hermite quadrature for $B\sim N(0,\tau^2)$:

E\{f(B)\}\ \approx\ \frac{1}{6}f(-\sqrt{3}\tau) +\frac{2}{3}f(0) +\frac{1}{6}f(\sqrt{3}\tau).

This is a numerical approximation, not an exact identity for the logistic function. More nodes or adaptive numerical integration can improve accuracy. The approximation is useful pedagogically because it makes the averaging operation explicit and gives easily checked arithmetic.

Worked Example: Averaging at a Specified Linear Predictor

Suppose the conditional linear predictor excluding the random intercept is $\eta=\beta_0+\beta_1x=1$, and the random-intercept standard deviation is $\tau=1$. The conditional probability at $B=0$ is $\operatorname{expit}(1)\approx0.7311$. Approximate the marginal probability by three-point quadrature.

The three intercept values are $-\sqrt{3}\approx-1.732$, $0$, and $1.732$. The corresponding linear predictors are $-0.732$, $1$, and $2.732$, giving expit values approximately $0.3247$, $0.7311$, and $0.9389$. Therefore,

\begin{aligned} \mu(x) &\approx \frac{1}{6}(0.3247)+\frac{2}{3}(0.7311)+\frac{1}{6}(0.9389)\\ &=0.05412+0.4874+0.1565\\ &\approx 0.6980. \end{aligned}

The approximated population probability is about $0.6980$, compared with $0.7311$ for a cluster whose random intercept is zero. Neither number is an effect of $x$ by itself; each is a probability at the stated linear predictor, and the first averages across the assumed distribution of cluster intercepts.

Why the marginal mean is generally not logistic

Write $\eta=\beta_0+\beta_1x$ and $m(\eta)=E\{\operatorname{expit}(\eta+B)\}$. The conditional mean is logistic in $\eta+B$. The marginal mean is a smooth average of shifted logistic curves, and averaging changes the shape. The following argument shows that, for a nondegenerate normal random intercept, this average cannot in general be represented as a logistic curve with a linear predictor in $\eta$.

First, symmetry of the normal distribution and the identity $\operatorname{expit}(-u)=1-\operatorname{expit}(u)$ imply $m(0)=1/2$. Next, the derivative can be taken inside the expectation because the derivative of expit is bounded:

m'(0)=E\!\left[\operatorname{expit}(B)\{1-\operatorname{expit}(B)\}\right].

For every real $u$, $\operatorname{expit}(u)\{1-\operatorname{expit}(u)\}\leq 1/4$, with equality only at $u=0$. If $\tau^2>0$, then $P(B\ne0)=1$, so $m'(0)<1/4$. This already shows that the marginal curve is flatter at its center than the conditional logistic curve with the same linear predictor.

More generally, suppose the marginal mean could be written as $\operatorname{expit}(a+c\eta)$ for constants $a$ and $c$. The symmetry result $m(0)=1/2$ requires $a=0$. For a normal $B$, the tails of the marginal mean satisfy

m(\eta)\sim e^\eta E(e^B)=e^{\eta+\tau^2/2} \quad(\eta\to-\infty), \qquad 1-m(\eta)\sim e^{-\eta}E(e^{-B})=e^{-\eta+\tau^2/2} \quad(\eta\to\infty).

Both tails change at unit rate on the logit scale, so a logistic representation would require $c=1$. But $\operatorname{expit}(\eta)$ has derivative $1/4$ at zero, whereas $m'(0)<1/4$. This contradiction establishes that for $\tau^2>0$, $m(\eta)$ is not generally a logistic curve with a linear predictor. When $\tau^2=0$, there is no averaging over varying intercepts, and the ordinary logistic form is recovered.

Worked Example: Comparing Conditional and Marginal Probabilities

For a hypothetical binary-outcome model, suppose $\beta_0=-1$, $\beta_1=1$, and $\tau=1$. Compare $x=0$ with $x=1$. The conditional linear predictors are $-1$ and $0$, respectively. Use three-point quadrature to obtain marginal probabilities.

At $x=0$, the three linear predictors after adding the quadrature intercept values are $-2.732$, $-1$, and $0.732$. Their expit values are approximately $0.0611$, $0.2689$, and $0.6753$. Thus,

\mu(0)\approx\frac{1}{6}(0.0611)+\frac{2}{3}(0.2689)+\frac{1}{6}(0.6753) =0.01018+0.1793+0.1126 \approx0.3020.

At $x=1$, the linear predictors are $-1.732$, $0$, and $1.732$. The corresponding probabilities are approximately $0.1503$, $0.5000$, and $0.8497$, so

\mu(1)\approx\frac{1}{6}(0.1503)+\frac{2}{3}(0.5000)+\frac{1}{6}(0.8497) =0.02505+0.3333+0.1416 \approx0.5000.

The conditional probabilities at $B=0$ are $\operatorname{expit}(-1)\approx0.2689$ and $\operatorname{expit}(0)=0.5000$. The marginal probability difference is approximately $0.5000-0.3020=0.1980$, while the conditional probability difference at $B=0$ is approximately $0.2311$. The marginal odds ratio from these approximations is

\frac{0.5000/(1-0.5000)}{0.3020/(1-0.3020)} \approx \frac{1}{0.4327} \approx 2.311.

By contrast, the conditional odds ratio for a one-unit increase in $x$ is $\exp(\beta_1)=\exp(1)\approx2.718$. These are different summaries because one conditions on the random intercept and the other averages over it. The difference is not an arithmetic error; it is a consequence of nonlinear averaging.

What changes as the random-intercept variance changes?

The amount of averaging depends on $\tau^2$. When $\tau=0$, every cluster has the same intercept and the marginal and conditional means coincide. As $\tau$ grows, the conditional probabilities being averaged become more dispersed. The resulting marginal probability need not move in the same direction for every value of $\eta$, because the expit curve is curved differently on opposite sides of its midpoint.

Worked Example: The Same Linear Predictor With Less Heterogeneity

Keep $\eta=1$, but compare $\tau=0$, $0.5$, and $1$. With $\tau=0$, the marginal probability is exactly $\operatorname{expit}(1)\approx0.7311$. For $\tau=0.5$, the quadrature intercepts are $-0.866$, $0$, and $0.866$. The respective probabilities are $\operatorname{expit}(0.134)\approx0.5335$, $0.7311$, and $\operatorname{expit}(1.866)\approx0.8660$, giving

\mu(x)\approx\frac{1}{6}(0.5335)+\frac{2}{3}(0.7311)+\frac{1}{6}(0.8660) =0.08892+0.4874+0.1443 \approx0.7206.

For $\tau=1$, the preceding quadrature gave approximately $0.6980$. Thus these approximations are $0.7311$, $0.7206$, and $0.6980$ as $\tau$ increases from 0 to 0.5 to 1. This numerical comparison is for $\eta=1$; it should not be turned into a claim that increasing variance always lowers the marginal probability for every $\eta$.

Assumptions, diagnostics, and alternatives

The integral has a clear interpretation only under a specified model for the random intercept. The normal distribution and its variance are part of that specification; changing the distribution can change the marginal mean even if the conditional mean has the same logistic form. The model also treats the random intercept as independent of the covariate. If that assumption is inappropriate, averaging over the unconditional distribution of $B_j$ need not give the desired population mean at a given $x$.

In practice, check whether predictions are stable to numerical integration accuracy: compare quadrature with more nodes or another reliable integration method, especially when $\tau$ is large or predictions are near the probability boundaries. Also inspect whether the fitted random-intercept distribution is a plausible representation of cluster heterogeneity. A poor distributional assumption or an incorrect conditional mean can make the computed marginal probabilities misleading even when the integral is evaluated accurately.

A GLMM specifies the conditional response model and obtains population predictions by integrating over its random effects. GEE instead specifies a marginal mean directly and uses a working correlation structure for clustered responses. These approaches address different modeling targets and assumptions; neither is merely a numerical substitute for the other. The introductory material on conditional and marginal estimands distinguishes the targets, while the present derivation shows specifically how a logistic random intercept transforms the conditional mean upon averaging.

A common mistake is to set the random intercept to its mean, $E(B_j)=0$, and call $\operatorname{expit}(\beta_0+\beta_1x)$ the marginal probability. That is only the conditional probability for a cluster with $B_j=0$, or the marginal probability when $\tau=0$. For $\tau^2>0$, calculate or approximate the expectation of the entire expit function. As emphasized in the earlier material on why logistic odds ratios change after averaging, conditional and marginal odds-scale summaries need not agree.

Key takeaway: For a random-intercept logistic model, the population probability is $E\{\operatorname{expit}(\eta+B)\}$, not $\operatorname{expit}(\eta+E[B])$. With a nonzero normal random-intercept variance, the marginal mean is generally not logistic-linear in the covariate.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

Write the marginal probability for a random-intercept logistic model with linear predictor $\eta$ and $B\sim N(0,\tau^2)$.
Why is the marginal probability exactly $1/2$ at $\eta=0$ under a symmetric random-intercept distribution?
Using three-point quadrature with $\tau=1$, approximate the marginal probability at $\eta=-1$.
What happens to the marginal mean when $\tau=0$?
For $\tau^2>0$, why is the derivative of the marginal mean at $\eta=0$ less than $1/4$?

Solutions. (1) $\mu(\eta)=E\{\operatorname{expit}(\eta+B)\}=\int\operatorname{expit}(\eta+b)\phi(b/\tau)\,db/\tau$. (2) Symmetry gives $B$ and $-B$ the same distribution, and $\operatorname{expit}(B)+\operatorname{expit}(-B)=1$; therefore the expectation is $1/2$. (3) The quadrature predictors are $-2.732$, $-1$, and $0.732$, with probabilities $0.0611$, $0.2689$, and $0.6753$. The weighted average is $0.0611/6+(2/3)(0.2689)+0.6753/6\approx0.3020$. (4) There is no intercept variation to average over, so $\mu(\eta)=\operatorname{expit}(\eta)$. (5) Differentiating gives $m'(0)=E[\operatorname{expit}(B)\{1-\operatorname{expit}(B)\}]$. The quantity inside the expectation is at most $1/4$, with equality only at $B=0$; a nondegenerate normal variable is nonzero with probability one, so the expectation is strictly less than $1/4$.