Conditional And Marginal Predictions For Binary Outcomes

Two Prediction Targets for a Binary Outcome

A fitted logistic mixed model can answer more than one prediction question. For a particular cluster, one may ask for the outcome probability at a specified random-effect value. For a population of clusters, one may instead ask for the probability after averaging over the random-effects distribution. These targets are related, but they are not interchangeable: the first conditions on a cluster effect, while the second averages predictions across possible cluster effects.

This distinction matters when reporting a probability. “The predicted probability for a cluster with random intercept zero” describes a particular conditional target. “The average predicted probability across clusters” describes a marginal target. A prediction is interpretable only when its target is stated.

Definition: For a binary response \(Y_{ij}\) in cluster \(j\), a conditional prediction is \(P(Y_{ij}=1\mid x_{ij},b_j)\), evaluated at a specified covariate value \(x_{ij}\) and random-effect value \(b_j\). A marginal prediction is \(P(Y_{ij}=1\mid x_{ij})\), averaged over the random-effects distribution. In a GLMM, this marginal probability is obtained by averaging the conditional probability, not by setting the random effect to its mean.

Model and Assumptions

Consider a random-intercept logistic GLMM. Let \(Y_{ij}\) equal 1 when the binary event occurs for observation \(i\) in cluster \(j\), and 0 otherwise. Let \(x_{ij}\) be a covariate, \(\beta_0\) and \(\beta_1\) fixed-effect coefficients, and \(b_j\) the cluster-specific random intercept. The model is

Y_{ij}\mid b_j,x_{ij}\sim \operatorname{Bernoulli}(p_{ij}(b_j)),\qquad \operatorname{logit}\{p_{ij}(b_j)\}=\beta_0+\beta_1x_{ij}+b_j, \qquad \operatorname{logit}(p)=\log\left(\frac{p}{1-p}\right).

The inverse-logit function is \(\operatorname{logit}^{-1}(u)=1/(1+\exp(-u))\). Thus, for fixed \(b_j\), the conditional event probability is

p_{ij}(b_j)=\operatorname{logit}^{-1}(\beta_0+\beta_1x_{ij}+b_j).

For the usual normal random-intercept model, the distributional assumption is \(b_j\sim N(0,\sigma_b^2)\), with random effects independent across clusters and independent of the covariates under the model. The conditional Bernoulli model specifies the response distribution given the covariates and random effect. The marginal prediction averages that conditional probability over the assumed distribution:

P(Y_{ij}=1\mid x_{ij}) = \int \operatorname{logit}^{-1}(\beta_0+\beta_1x_{ij}+b) \,f_b(b)\,db,

where \(f_b\) is the random-effect density. This integral is generally evaluated numerically. More generally, the same averaging definition applies to a specified non-normal random-effects distribution. The prediction depends on both the fixed effects and the random-effects distribution, including its variance.

Why the Targets Differ

A conditional prediction uses one value of \(b\), such as \(b=0\) or an estimated effect for a particular cluster. A marginal prediction combines the conditional probabilities over the distribution of \(b\). In general, applying the inverse-logit after replacing \(b\) by its mean does not perform that averaging:

E_b\!\left[\operatorname{logit}^{-1}(\beta_0+\beta_1x+b)\right] \ne \operatorname{logit}^{-1}(\beta_0+\beta_1x+E_b[b]).

For a mean-zero random effect, the right-hand side is the prediction at \(b=0\); it is not, in general, the average probability across clusters. The difference arises because the inverse-logit is nonlinear. The discrepancy can be small or large depending on the fixed linear predictor and the spread of the random effects.

Worked Example: Averaging Two Cluster Types

Suppose a hypothetical model has \(\beta_0=-1\), \(\beta_1=1\), and random intercepts that equal \(-1\) or \(1\), each with probability \(1/2\). This two-point distribution has mean zero. For the calculation below, assume the covariate value is shared across the two cluster types.

At \(x=0\), the linear predictor is \(-1+b\). For the cluster with \(b=-1\), the conditional probability is \(\operatorname{logit}^{-1}(-2)=1/(1+\exp(2))=0.1192\). For \(b=1\), it is \(\operatorname{logit}^{-1}(0)=0.5000\). Therefore, the marginal prediction is

P(Y=1\mid x=0) = \frac{1}{2}(0.1192)+\frac{1}{2}(0.5000) =0.3096.

By contrast, the conditional prediction at \(b=0\) is \(\operatorname{logit}^{-1}(-1)=1/(1+\exp(1))=0.2689\). Thus, the model predicts an event probability of about 31.0% averaged across the two cluster types, while the cluster with random intercept zero has a conditional probability of about 26.9%.

At \(x=1\), the linear predictor is \(b\). The two conditional probabilities are \(\operatorname{logit}^{-1}(-1)=0.2689\) and \(\operatorname{logit}^{-1}(1)=0.7311\). Their average is \((0.2689+0.7311)/2=0.5000\). Here, the prediction at \(b=0\) is also 0.5000. That equality at this covariate value is a consequence of the symmetric effects and the logistic function; it does not mean conditional and marginal predictions are generally equal.

Marginal Prediction Under Normal Random Effects

For a normal random intercept, the marginal prediction is an integral rather than a finite average. Numerical quadrature approximates the integral by evaluating the conditional probability at selected random-effect values and taking a weighted sum. The following example shows the calculation and makes clear which prediction is being approximated.

Worked Example: A Normal Random-Intercept Distribution

Suppose, hypothetically, that \(\beta_0=-1\), \(\beta_1=1\), and \(b\sim N(0,1)\). At \(x=0\), the conditional prediction for \(b=0\) is \(\operatorname{logit}^{-1}(-1)=0.2689\). To approximate the marginal prediction, use the three-point normal quadrature rule with random-effect values \(-\sqrt{3}\), \(0\), and \(\sqrt{3}\), and weights \(1/6\), \(2/3\), and \(1/6\), respectively.

At \(b=-\sqrt{3}\), the linear predictor is \(-1-\sqrt{3}\approx-2.732\), giving probability \(0.0610\). At \(b=0\), the probability is \(0.2689\). At \(b=\sqrt{3}\), the linear predictor is \(-1+\sqrt{3}\approx0.732\), giving probability \(0.6752\). The weighted calculation is

\begin{aligned} P(Y=1\mid x=0) &\approx \frac{1}{6}(0.0610)+\frac{2}{3}(0.2689)+\frac{1}{6}(0.6752)\\ &=0.0102+0.1793+0.1125\\ &=0.3020. \end{aligned}

The quadrature approximation is about 30.2%, compared with 26.9% for the conditional prediction at \(b=0\). The marginal value is an approximation to the model-based average over the normal distribution, not a prediction for a cluster whose random intercept is exactly one of the quadrature values. More quadrature points can be used for greater numerical accuracy.

Predictions for an Observed Cluster

If the purpose is to describe a particular observed cluster, a conditional prediction can use that cluster’s estimated random effect. This is distinct from the population-average target. The cluster-specific prediction depends on the estimated effect, which is itself uncertain and typically estimated with shrinkage toward the population mean.

Worked Example: Comparing Two Cluster-Specific Predictions

Suppose a hypothetical fitted model has \(\beta_0=-1\), \(\beta_1=0.8\), and estimated random intercepts \(\hat b_A=-0.6\) and \(\hat b_B=0.6\) for two clusters. At \(x=1\), the conditional linear predictor for cluster A is \(-1+0.8-0.6=-0.8\), so its predicted probability is \(\operatorname{logit}^{-1}(-0.8)=1/(1+\exp(0.8))=0.3100\).

For cluster B, the linear predictor is \(-1+0.8+0.6=0.4\), so its predicted probability is \(\operatorname{logit}^{-1}(0.4)=1/(1+\exp(-0.4))=0.5987\). The two cluster-specific predictions differ because the model assigns different random intercepts to the clusters.

If these two clusters alone define the target and are given equal weight, their average fitted probability is \((0.3100+0.5987)/2=0.4544\), or about 45.4%. This is an empirical average of two fitted cluster predictions. It is not automatically the same as the model’s marginal prediction over the full random-effects distribution: the empirical target uses these two estimated effects and the chosen equal weights, whereas the model-based target averages over the assumed distribution of effects.

Choosing and Checking the Prediction Target

The appropriate prediction depends on the scientific question. A cluster-specific prediction is useful when the target is a particular clinic, school, or individual with a specified random-effect estimate. A marginal prediction is appropriate when the target is the average event probability across clusters drawn from the modeled population. If clusters have different sizes or are to be weighted unequally, define the target weights explicitly; an unweighted average of cluster probabilities and a probability for a randomly selected individual need not coincide.

A population-averaged model, such as a GEE, models a marginal mean directly. A GLMM instead specifies a conditional mean given random effects and obtains its marginal prediction by averaging that conditional mean. Both approaches can address population-level questions, but their parameters and prediction calculations are not interchangeable. The target and model used should be reported together.

Name the target.
State whether the prediction concerns a specified cluster effect, the effect value zero, observed clusters with explicit weights, or a population of clusters.

Calculate conditional probabilities.
For each specified effect value \(b\), evaluate \(\operatorname{logit}^{-1}(\beta_0+\beta_1x+b)\).

Average only when the target requires it.
Use the stated cluster weights or integrate over the fitted random-effects distribution for a model-based marginal prediction.

Several checks can reveal a mismatched or poorly calculated prediction. Verify that each conditional probability lies between 0 and 1, that a marginal probability lies between the smallest and largest conditional probabilities being averaged, and that the weights sum to 1. Check that the covariate value and random-effect distribution match the intended target. For numerical integration, compare results using more quadrature points; an unstable result signals inadequate numerical approximation or a difficult integral.

These checks do not establish that the model is appropriate. The random-effects distribution, the conditional logit-linear mean structure, and the assumed relationship between random effects and covariates all matter. A misspecified random-effects distribution can affect the marginal prediction. Predictions for covariate patterns far from those observed also rely heavily on model extrapolation. Finally, plug-in predictions based on fitted coefficients and estimated random effects do not, by themselves, account for parameter uncertainty.

Common Mistakes and Limitations

Setting the random effect to zero and calling the result marginal. This gives a conditional prediction at \(b=0\). A marginal prediction requires averaging over the distribution of random effects.
Averaging the linear predictors instead of the probabilities. The marginal probability is an average of inverse-logit probabilities. Because the inverse-logit is nonlinear, applying it to the average linear predictor generally gives a different answer.
Treating a cluster-specific estimate as the cluster’s known truth. Estimated random effects have uncertainty and are generally shrunk toward zero. A fitted conditional prediction is not error-free.
Leaving the averaging population unspecified. State whether the average is over the model’s cluster distribution or over a defined set of observed clusters, and specify weights when relevant.

A GLMM’s marginal prediction is therefore conditional on the model assumptions and the chosen cluster population. It is not a universal probability independent of the distribution of clusters. Reporting the covariate value, the random-effects distribution or observed-cluster weights, and the prediction target makes the result interpretable and reproducible.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In a logistic random-intercept model, what does a prediction evaluated at \(b=0\) describe?
Suppose two equally weighted cluster types have conditional probabilities 0.20 and 0.60. What is their average probability?
Why is the inverse-logit of the mean linear predictor generally not the marginal probability?
For a population-level prediction from a GLMM, what quantity is averaged over the random-effects distribution?
Why might an empirical average over observed clusters differ from a model-based marginal prediction?

Solutions. (1) It is a conditional prediction for a cluster whose random intercept is specified to be zero, not generally the population-average prediction. (2) The average is \((0.20+0.60)/2=0.40\), or 40%. (3) The inverse-logit is nonlinear, so transforming an average linear predictor generally differs from averaging the transformed linear predictors. (4) Average the conditional event probabilities, \(\operatorname{logit}^{-1}(\beta_0+\beta_1x+b)\), rather than averaging \(b\) first. (5) The empirical average uses the particular observed or estimated cluster effects and their weights; the model-based prediction averages over the assumed random-effects distribution.