Closed-Form Marginalization In A Probit Random-Intercept Model

Why Marginalization Is Especially Simple for Probit

A probit GLMM describes the event probability for an observation conditional on its cluster’s random intercept. A population-averaged probability instead averages over the distribution of random intercepts. That averaging usually requires an integral. For the probit link with a normal random intercept, the integral has a closed form because the probit probability can be represented using a normal latent error: the random intercept and that error add to another normal variable.

The result is useful for converting a cluster-specific prediction into a prediction for a randomly selected cluster. It is not the same as setting the random intercept to zero. The distinction is between a conditional probability at a specified random-effect value and a probability averaged over the model’s random-effect distribution.

Model and Assumptions

Definition: For cluster $i$ and observation $j$, let $Y_{ij}$ be a binary outcome, let $x_{ij}$ be its covariate vector, and let $\eta_{ij}=x_{ij}^{T}\beta$. A probit random-intercept GLMM specifies $$ Y_{ij}\mid b_i \sim \operatorname{Bernoulli}\!\left(\Phi(\eta_{ij}+b_i)\right), \qquad b_i\sim N(0,\tau^2), $$ where $\Phi$ is the standard normal cumulative distribution function. Conditional on the random intercepts and covariates, outcomes are assumed independent. The random intercepts are independent across clusters and independent of the covariates, and the same $b_i$ applies to every observation in cluster $i$.

The standard probit model fixes the latent error variance at one; this scale convention is part of the model. The marginal probability considered here is the probability for an observation at fixed covariates, averaged over a new draw $b_i$ from $N(0,\tau^2)$. It is therefore a population-averaged, or marginal, probability under the assumed random-effect distribution—not a probability conditional on an estimated intercept for a particular observed cluster.

The derivation requires the normal distribution for the random intercept, its independence from the latent error, and the unit variance of that error. If any of these assumptions changes, this particular formula need not hold.

Latent-Normal Derivation

Represent the binary response by a latent variable. For a fixed observation, take $\epsilon_{ij}\sim N(0,1)$, independent of $b_i$, and define $Y_{ij}=1$ when $\eta_{ij}+b_i+\epsilon_{ij}>0$, and $Y_{ij}=0$ otherwise. Conditional on $b_i$,

P(Y_{ij}=1\mid b_i) =P(\epsilon_{ij}>-\eta_{ij}-b_i) =\Phi(\eta_{ij}+b_i).

Thus this latent construction reproduces the stated conditional probit model. To average over $b_i$, note that $b_i+\epsilon_{ij}$ is a sum of independent normal variables, with mean zero and variance $\tau^2+1$. Consequently,

\begin{aligned} P(Y_{ij}=1\mid x_{ij}) &=P(\eta_{ij}+b_i+\epsilon_{ij}>0)\\ &=P\left(\frac{b_i+\epsilon_{ij}}{\sqrt{1+\tau^2}}> -\frac{\eta_{ij}}{\sqrt{1+\tau^2}}\right)\\ &=\Phi\left(\frac{\eta_{ij}}{\sqrt{1+\tau^2}}\right). \end{aligned}

The final equality uses the symmetry of the standard normal distribution: $P(W>-a)=\Phi(a)$ when $W\sim N(0,1)$. This proves the closed-form result.

Theorem: Under the probit random-intercept model above, the marginal event probability at covariates $x_{ij}$ is $$ P(Y_{ij}=1\mid x_{ij}) =\Phi\left(\frac{x_{ij}^{T}\beta}{\sqrt{1+\tau^2}}\right). $$ The random-intercept variance increases the latent variance from $1$ to $1+\tau^2$, so the marginal probit argument is the conditional linear predictor divided by $\sqrt{1+\tau^2}$.

When $\tau^2=0$, the formula reduces to $\Phi(\eta_{ij})$, as it must: there is no random-intercept variation to average over. For $\tau^2>0$, the denominator exceeds one. For any fixed finite $\eta_{ij}$, the marginal probability is therefore closer to $0.5$ than the conditional probability evaluated at $b_i=0$. This comparison is about probabilities at the same covariates; it does not imply that every individual cluster’s probability is closer to $0.5$.

Worked Example: Averaging One Probit Prediction

Suppose the conditional linear predictor is $\eta=0.80$, and the random-intercept variance is $\tau^2=0.36$. Assume the standard probit latent error has variance one, the random intercept is normal with mean zero, and it is independent of that error. Find the marginal event probability.

The variance of the sum of the random intercept and latent error is $1+0.36=1.36$. Hence,

\sqrt{1+\tau^2}=\sqrt{1.36}\approx1.1662, \qquad \frac{\eta}{\sqrt{1+\tau^2}} =\frac{0.80}{\sqrt{1.36}}\approx0.6860.

The closed-form marginal probability is therefore $\Phi(0.6860)\approx0.7536$. The probability conditional on $b=0$ is $\Phi(0.80)\approx0.7881$. Thus, under this model, averaging over the random intercept gives an event probability of about 0.7536 rather than the probability at a zero random intercept, about 0.7881.

Conditional and Marginal Interpretations

The conditional probit model uses $\eta_{ij}=x_{ij}^{T}\beta$, while its marginal probability can be written as a probit curve with linear predictor $x_{ij}^{T}\beta/\sqrt{1+\tau^2}$. If $\tau^2$ is fixed, the marginal probit-scale coefficient vector is therefore $\beta/\sqrt{1+\tau^2}$. This is a statement about the scale of the probit linear predictor; it is not a constant change in probability, because $\Phi$ is nonlinear.

The marginal probability averages across the model’s distribution of clusters. A cluster-specific prediction instead uses a specified or estimated value of $b_i$. These answer different questions. For example, a prediction for a new cluster uses the marginal formula, whereas a conditional prediction for an existing cluster uses its random-intercept information as appropriate to the analysis.

For two observations in one cluster, the shared random intercept generally induces dependence after it is averaged out when $\tau^2>0$, even though the outcomes are conditionally independent given $b_i$. When $\tau^2=0$, the intercept is constant and does not induce dependence; under the stated conditional-independence assumption, outcomes are independent given their covariates.

Worked Example: Comparing Two Covariate Values

Suppose a binary outcome follows the stated probit random-intercept model, with conditional predictor $\eta=-0.50+0.90x$ and random-intercept variance $\tau^2=0.64$. Assume the normal, independence, and unit latent-error variance conditions hold. Calculate the marginal probabilities at $x=0$ and $x=1$, then compare their difference with the conditional probabilities at $b=0$.

The common marginal scaling denominator is $\sqrt{1+0.64}=\sqrt{1.64}\approx1.2806$. At $x=0$, the predictor is $-0.50$, so

\frac{-0.50}{\sqrt{1.64}}\approx-0.3904, \qquad P(Y=1\mid x=0)=\Phi(-0.3904)\approx0.3481.

At $x=1$, the predictor is $-0.50+0.90=0.40$, giving

\frac{0.40}{\sqrt{1.64}}\approx0.3123, \qquad P(Y=1\mid x=1)=\Phi(0.3123)\approx0.6226.

The marginal probability difference is approximately $0.6226-0.3481=0.2745$, or 27.45 percentage points. At $b=0$, the conditional probabilities are $\Phi(-0.50)\approx0.3085$ and $\Phi(0.40)\approx0.6554$, a difference of about $0.3469$, or 34.69 percentage points. These are different contrasts on the probability scale. Neither difference is the conditional coefficient $0.90$, which is measured on the probit linear-predictor scale.

Worked Example: Checking the Effect of Random-Intercept Variation

At fixed covariates, suppose the conditional predictor is $\eta=1.20$, with random-intercept variance $\tau^2=1.25$. Assume the random intercept and standard normal latent error are independent. Calculate the marginal probability and compare it with the probability at $b=0$.

The total latent variance is $1+1.25=2.25$, so its square root is $1.50$. The marginal probit argument and probability are

\frac{1.20}{\sqrt{2.25}}=\frac{1.20}{1.50}=0.80, \qquad P(Y=1\mid x)=\Phi(0.80)\approx0.7881.

At $b=0$, the conditional probability is $\Phi(1.20)\approx0.8849$. The marginal value is lower and closer to $0.5$, consistent with the variance scaling. If instead $\tau^2=0$, the marginal probability would equal $\Phi(1.20)\approx0.8849$.

Assumptions, Diagnostics, and Alternatives

The formula is exact for the specified model, but exact algebra does not guarantee that the model is appropriate. Its key assumptions and possible warning signs include:

Normal random intercepts. The result uses $b_i\sim N(0,\tau^2)$. Strongly unusual cluster effects or poor predictive fit across clusters may motivate sensitivity analysis under alternative random-effect distributions.
Probit latent-error scale. The latent error variance is fixed at one. Replacing it with an arbitrary variance without changing the model’s scale convention changes the formula and may create an unidentified scale.
Independence. The derivation requires independence of $b_i$ and the latent error. The model also assumes random intercepts are independent of covariates; if that assumption is implausible, the marginal formula under this specification may not answer the intended population question.
Random intercept only. Here the same random shift applies to every observation within a cluster. Additional random terms change the latent variance calculation and are outside this result.

Useful checks include assessing whether fitted marginal probabilities are calibrated across covariate ranges, comparing observed and model-predicted event frequencies across clusters, and examining whether conclusions are sensitive to the assumed random-effect distribution and link. Such checks can reveal lack of fit; they do not prove that the latent normal assumptions are true.

The central alternative is numerical marginalization. It can be used when a model’s conditional probability does not yield a closed-form average under the random-effect distribution. For example, a logistic model with normal random effects generally requires numerical integration or simulation for its marginal probabilities. The closed-form probit result is simpler, but that convenience does not by itself establish that a probit link is preferable for a particular application.

A plug-in calculation also treats the fitted $\beta$ and $\tau^2$ as fixed inputs. It gives a fitted marginal probability, not an interval that accounts for estimation uncertainty. Any inferential interval must reflect uncertainty in the estimated parameters as well as the transformation to the marginal probability.

Common Mistake

Common pitfall: Using $\Phi(\eta)$ as the marginal probability because it is the probit probability at a zero random intercept. That is a conditional probability, not an average over random intercepts. The correction is to use $\Phi\!\left(\eta/\sqrt{1+\tau^2}\right)$ under the stated assumptions. Also use the variance $\tau^2$, not the standard deviation $\tau$, when adding the normal variances.

Check Your Understanding

Answer each question, then compare with the solutions below.

For $\eta=0.30$ and $\tau^2=0.44$, find the marginal event probability.
What does the marginal formula become when $\tau^2=0$, and why?
With $\tau^2>0$, is the marginal probit-scale coefficient equal to the conditional coefficient? State the relationship.
Suppose the random intercept is normal but dependent on the latent error. Can the derivation above be used as written? Explain.

Solutions.

1. The denominator is $\sqrt{1+0.44}=\sqrt{1.44}=1.20$. Thus the marginal argument is $0.30/1.20=0.25$, and the probability is $\Phi(0.25)\approx0.5987$.

2. If $\tau^2=0$, then $\sqrt{1+\tau^2}=1$, so the marginal probability is $\Phi(\eta)$. A zero random-intercept variance means there is no random-intercept variation to average over.

3. No. For fixed $\tau^2$, the marginal probit-scale coefficient vector is $\beta/\sqrt{1+\tau^2}$. Its scale differs from the conditional coefficient vector $\beta$, although the probability-scale effect also depends on the covariate values through $\Phi$.

4. No. The variance of a sum is the sum of the variances only when the covariance is zero. Dependence would generally make the variance of $b_i+\epsilon_{ij}$ include $2\operatorname{Cov}(b_i,\epsilon_{ij})$, and the independent-normal sum argument used to obtain the stated formula would no longer apply.