Logistic-Normal Attenuation Of Conditional Effects

Why Averaging Changes a Logistic Effect

A logistic random-intercept model describes covariate effects conditional on a cluster’s random intercept. A population-averaged effect instead compares outcome probabilities after averaging over the distribution of cluster intercepts. Because the inverse-logit function is nonlinear, that averaging changes the relationship between the covariate and the marginal log odds. For a nonzero random-intercept variance, the marginal log-odds slope is generally smaller in magnitude than the corresponding conditional slope.

The result concerns the scale of the effect: it does not imply that the covariate’s direction changes, nor that a conditional model is necessarily wrong. It explains why conditional and population-averaged effects answer different questions. The earlier material on marginal probabilities defines the integration that produces the marginal mean; here the focus is how its log odds vary with the linear predictor.

Model and Assumptions

Definition: For a binary response $Y_{ij}$ in cluster $i$, a logistic random-intercept model has conditional mean $$ P(Y_{ij}=1\mid x_{ij},b_i)=\operatorname{expit}(\eta_{ij}+b_i), \qquad \eta_{ij}=x_{ij}^{T}\beta, \qquad \operatorname{expit}(u)=\frac{1}{1+\exp(-u)}. $$ The random intercept satisfies $b_i\sim N(0,\sigma_b^2)$. The marginal probability at linear predictor $\eta$ is $m(\eta)=E_b\{\operatorname{expit}(\eta+b)\}$, and the marginal log odds are $g(\eta)=\operatorname{logit}\{m(\eta)\}$.

The attenuation result below assumes that the random-intercept distribution is the same at every covariate value and is independent of the covariates being compared. It also assumes that the only random effect in the linear predictor is an intercept: changing a covariate changes $\eta$, not the distribution of $b_i$. Conditional on the random effects and covariates, responses are Bernoulli with the stated logistic mean. Conditional independence among responses within a cluster is a common model assumption for their joint distribution; the marginal-mean calculation itself uses the conditional mean and random-intercept distribution.

A fixed covariate value gives one conditional log-odds change per unit change in that covariate. After integrating out $b_i$, $g(\eta)$ is generally nonlinear in $\eta$. Thus, a marginal effect is usually local to a covariate value, or is a contrast between specified covariate values, rather than one universal marginal coefficient.

Exact Attenuation of the Local Slope

Write $q=\operatorname{expit}(\eta+b)$, treating $q$ as a random variable through $b$. Differentiating the marginal mean with respect to $\eta$ gives $m'(\eta)=E\{q(1-q)\}$. Differentiation under the expectation is valid here because the derivative of the expit function is bounded. Since $g(\eta)=\operatorname{logit}\{m(\eta)\}$, the chain rule yields the following result.

Theorem: For a nondegenerate random intercept with the same distribution at every covariate value, $$ g'(\eta)=\frac{E\{q(1-q)\}}{m(\eta)\{1-m(\eta)\}} =1-\frac{\operatorname{Var}(q)}{m(\eta)\{1-m(\eta)\}}, \qquad 0<g'(\eta)<1. $$ If $\sigma_b^2=0$, then $q$ is constant and $g'(\eta)=1$.

To verify the second equality, use $E(q^2)=\operatorname{Var}(q)+m^2$, where $m=E(q)$. Then $$ E\{q(1-q)\}=m-E(q^2)=m(1-m)-\operatorname{Var}(q). $$ For finite $\eta$ and finite $b$, $0<q<1$, so $E\{q(1-q)\}>0$. Also, when $b$ is nondegenerate, $q$ varies with $b$, giving $\operatorname{Var}(q)>0$. These facts make the ratio defining $g'$ positive and strictly less than one.

If $\eta=x^T\beta$, the marginal log-odds derivative with respect to covariate $x_j$ is $\beta_j g'(\eta)$. It has the same sign as $\beta_j$, but is smaller in magnitude whenever the random intercept has positive variance. The multiplier depends on $\eta$, so the marginal log-odds relationship need not have a constant slope.

The same result applies to finite contrasts. For two linear predictors $\eta_0$ and $\eta_1$, the fundamental theorem of calculus gives $$ g(\eta_1)-g(\eta_0)=\int_{\eta_0}^{\eta_1}g'(u)\,du. $$ Because $g'(u)$ lies strictly between zero and one, the marginal log-odds contrast has the same sign as $\eta_1-\eta_0$ and a smaller absolute value. This conclusion relies on comparing covariate values under a common random-intercept distribution.

Worked Example: Exact Bound for a Finite Contrast

Suppose a conditional model has coefficient $\beta_1=0.6$ for a one-unit covariate, and the random-intercept variance is positive. Compare covariate values separated by two units, holding other covariates fixed.

The difference in conditional linear predictors is $0.6(2)=1.2$. Thus the conditional odds ratio is $\exp(1.2)\approx 3.320$. The exact attenuation result guarantees that the marginal log-odds contrast lies strictly between 0 and 1.2: $$ 0<g(\eta_0+1.2)-g(\eta_0)<1.2. $$ Consequently, the marginal odds ratio lies strictly between $1$ and $\exp(1.2)\approx3.320$. This is a bound, not a numerical estimate of the marginal odds ratio: calculating its exact value requires evaluating the logistic-normal marginal means at both covariate values.

Small-Variance Approximation

When $\sigma_b^2$ is small, a Taylor expansion gives a useful approximation to the marginal mean and its logit. Let $f(\eta)=\operatorname{expit}(\eta)$. For $b$ near zero, $$ f(\eta+b)=f(\eta)+b f'(\eta)+\frac{b^2}{2}f''(\eta)+\cdots. $$ Taking expectations uses $E(b)=0$ and $E(b^2)=\sigma_b^2$. The odd-power terms have zero expectation for a centered normal random intercept. Since the logistic function has bounded derivatives, the expansion for fixed $\eta$ has remainder of order $\sigma_b^4$ as $\sigma_b^2$ tends to zero: $$ m(\eta)=f(\eta)+\frac{\sigma_b^2}{2}f''(\eta)+O(\sigma_b^4). $$

To pass from marginal probability to marginal log odds, apply a first-order Taylor expansion of $h(p)=\operatorname{logit}(p)$ around $p=f(\eta)$. Here $h'(p)=1/\{p(1-p)\}$, and $h\{f(\eta)\}=\eta$. Also, if $p=f(\eta)$, then $f''(\eta)=p(1-p)(1-2p)$. Substitution therefore gives $$ g(\eta)=\eta+\frac{\sigma_b^2}{2}\{1-2f(\eta)\}+O(\sigma_b^4). $$ The correction is positive when $f(\eta)<1/2$, zero when $f(\eta)=1/2$, and negative when $f(\eta)>1/2$. Thus marginal log odds are pulled toward zero, to this order, at a fixed $\eta$. Differentiating the expansion gives the corresponding local-slope approximation: $$ g'(\eta)=1-\sigma_b^2 f(\eta)\{1-f(\eta)\}+O(\sigma_b^4). $$ The leading reduction is largest near $f(\eta)=1/2$, where $f(\eta)\{1-f(\eta)\}=1/4$, and smaller near probabilities zero or one. These are small-variance approximations, not exact formulas for arbitrary $\sigma_b^2$.

Worked Example: Approximating Marginal Log Odds at a High Probability

Suppose the conditional probability at a specified covariate value and $b=0$ is $0.8$, and let $\sigma_b^2=0.1$. The conditional linear predictor is $$ \eta=\operatorname{logit}(0.8)=\log\left(\frac{0.8}{0.2}\right)=\log(4)\approx1.3863. $$

The small-variance correction to the marginal log odds is $$ \frac{0.1}{2}\{1-2(0.8)\}=0.05(-0.6)=-0.0300. $$ Hence the approximate marginal log odds are $1.3863-0.0300=1.3563$, to four significant figures. The corresponding approximate marginal probability is $\operatorname{expit}(1.3563)\approx0.7952$.

As a check using the probability expansion, $f''(\eta)=0.8(0.2)(1-1.6)=-0.096$, so $m(\eta)\approx0.8+(0.1/2)(-0.096)=0.7952$. Applying the logit to this probability gives approximately 1.3566; the slight difference from 1.3563 is due to terms of order $\sigma_b^4$ omitted in the expansions. Both calculations show a small decrease from the conditional probability and log odds at $b=0$.

Worked Example: Approximate Local Slope Near Probability One-Half

Let the conditional coefficient for a covariate be $\beta_1=0.8$, and let $\sigma_b^2=0.2$. Evaluate the local slope at a covariate value where $\eta=0$, so $f(\eta)=0.5$.

The small-variance multiplier for the local slope is approximately $$ 1-\sigma_b^2 f(\eta)\{1-f(\eta)\} =1-0.2(0.5)(0.5)=0.950. $$ The marginal local log-odds slope is therefore approximately $0.8(0.950)=0.760$ per covariate unit, compared with the conditional slope $0.8$. The conditional odds ratio per unit is $\exp(0.8)\approx2.226$; the approximate marginal local odds ratio is $\exp(0.760)\approx2.138$. This calculation uses the first-order-in-variance approximation, so it should not be reported as an exact marginal odds ratio.

Worked Example: Approximating a Marginal Contrast at Two Covariate Values

Consider a model with intercept $\beta_0=-0.4$, slope $\beta_1=1.0$, and random-intercept variance $\sigma_b^2=0.16$. Compare $x=0$ with $x=1$. The conditional linear predictors are $\eta_0=-0.4$ and $\eta_1=0.6$, so the conditional log-odds contrast is $1.0$.

At $x=0$, $f(-0.4)\approx0.4013$. The correction is $$ \frac{0.16}{2}\{1-2(0.4013)\} =0.08(0.1974)\approx0.0158, $$ giving $g(\eta_0)\approx-0.4+0.0158=-0.3842$. At $x=1$, $f(0.6)\approx0.6457$, so $$ \frac{0.16}{2}\{1-2(0.6457)\} =0.08(-0.2913)\approx-0.0233, $$ giving $g(\eta_1)\approx0.6-0.0233=0.5767$. The approximate marginal log-odds contrast is $0.5767-(-0.3842)=0.9609$, less than the conditional contrast of $1.000$. Its approximate marginal odds ratio is $\exp(0.9609)\approx2.614$, compared with the conditional odds ratio $\exp(1)\approx2.718$. The exact attenuation result guarantees the direction and strict inequality; the numerical values here additionally rely on the small-variance approximation.

Interpretation, Alternatives, and Failure Modes

A conditional coefficient describes how the log odds change with a covariate while holding the random intercept fixed. A population-averaged log-odds contrast compares probabilities integrated over the random-intercept distribution. The latter is often useful when the target is an average across clusters. A marginal approach such as generalized estimating equations instead models a population-averaged mean directly; it targets a different parameter and does not obtain that parameter simply by re-labeling a conditional GLMM coefficient.

The exact attenuation proof and the small-variance approximation are useful checks on interpretation and computation. For larger variance, or when accurate marginal probabilities are needed, calculate the integral defining $m(\eta)$ numerically, then take its logit. Inspect results at several covariate values: a single marginal slope can hide the fact that $g'(\eta)$ varies with $\eta$. Compare numerical integration with the small-variance approximation only when the variance is plausibly small.

Changing the random-effect distribution across covariate groups. If the distribution of $b_i$ depends on the covariate, the same averaging distribution is not being compared at each value. The derivative argument above does not establish attenuation for that comparison.
Adding random slopes. A covariate-dependent random effect changes the distribution of the linear predictor with the covariate. The random-intercept-only result cannot be carried over without a separate derivation.
Using the approximation at large variance. A truncated expansion may be inaccurate even though the exact slope remains between zero and one. Use numerical integration when precision matters.
Calling attenuation evidence of bias or confounding. The conditional and marginal effects average over clusters differently. Their difference can arise from the nonlinear link and random-intercept variation alone.

A frequent mistake is to say that the marginal log-odds slope equals the conditional coefficient multiplied by one fixed attenuation factor. The exact expression is $\beta_j g'(\eta)$, and $g'(\eta)$ depends on the current linear predictor. The small-variance multiplier also contains $f(\eta)\{1-f(\eta)\}$, which changes across covariate values. Another mistake is to treat the second-order expansion as exact; its omitted terms matter when the random-intercept variance is not small.

Key takeaway: Under a common, independent random-intercept distribution, integrating a logistic model preserves the direction of a covariate association but reduces the magnitude of its marginal log-odds slope. For small $\sigma_b^2$, the leading correction is $\frac{\sigma_b^2}{2}\{1-2\operatorname{expit}(\eta)\}$, and the leading slope multiplier is $1-\sigma_b^2\operatorname{expit}(\eta)\{1-\operatorname{expit}(\eta)\}$.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

What assumptions allow the exact attenuation result to compare marginal slopes at different covariate values?
If $f(\eta)=0.5$ and $\sigma_b^2=0.12$, what is the first-order approximation to the multiplier on the marginal local slope?
For $\eta=\operatorname{logit}(0.25)$ and $\sigma_b^2=0.08$, calculate the small-variance correction to marginal log odds.
Why does a positive random-intercept variance not imply that the marginal log-odds slope changes sign?
Why should a numerical marginal slope not generally be presented as one constant coefficient for all covariate values?

Solutions. (1) The random-intercept distribution must be common across the covariate values being compared and independent of those covariates; the linear predictor has a random intercept, not a random slope. (2) The multiplier is $1-0.12(0.5)(0.5)=0.970$. (3) Since $f(\eta)=0.25$, the correction is $(0.08/2)\{1-2(0.25)\}=0.04(0.5)=0.0200$ log-odds units. (4) The exact derivative is $g'(\eta)=E\{q(1-q)\}/[m(1-m)]$, which is positive, so multiplying it by a conditional slope preserves that slope’s sign. (5) The multiplier $g'(\eta)$ varies with $\eta$; the integrated logit is generally nonlinear, so local slopes can differ across covariate values.