A Scalar Worked Example of FO Bias

Why Compare Exact and First-Order Moments?

A first-order approximation replaces a nonlinear individual prediction with its tangent at the typical random-effect value, zero. That can make the marginal calculation simple, but the tangent need not capture how the prediction changes over the full distribution of random effects. The difference matters: even if the random effect has mean zero, a curved prediction can have a marginal mean different from its value at zero, and its marginal variance can differ from the variance implied by the tangent.

Here the comparison is made for a single scalar random effect. This isolates the effect of nonlinearity and makes it possible to calculate both sets of moments directly. The first-order calculation follows the linearization used in the earlier material on the First-Order Approximation; the emphasis here is the numerical size and direction of its moment discrepancies.

For a differentiable prediction, the FO expansion is \(f(\eta)\approx f(0)+f'(0)\eta\). Since \(E[\eta]=0\), the resulting FO moments are

These are moments of the linearized model, not generally the exact moments of the nonlinear model. The additive residual variance appears in both calculations because the residual is independent of the random effect.

An Exponential Prediction: Exact Moments and Their Derivation

Consider the positive prediction \(f(\eta)=A\exp(\eta)\), where \(A>0\) has the units of the prediction and \(\eta\) is dimensionless. This form corresponds, for example, to a typical concentration multiplied by an individual-specific log-scale effect. Assume \(\eta\sim N(0,\omega^2)\), with \(\omega^2>0\), and independent additive residual error with variance \(\sigma^2\).

The normal moment-generating identity \(E[\exp(t\eta)]=\exp(t^2\omega^2/2)\) gives the exact moments. Applying it first with \(t=1\), and then with \(t=2\), yields the mean and the second moment of \(f(\eta)\). Subtracting the squared mean gives the variance:

The FO calculation uses \(f(0)=A\) and \(f'(0)=A\), so its moments are \(E_{\mathrm{FO}}[Y]=A\) and \(\operatorname{Var}_{\mathrm{FO}}(Y)=A^2\omega^2+\sigma^2\). For any \(\omega^2>0\), the exact mean is greater than \(A\). The exact random-effect variance is also greater than the FO random-effect variance in this example: with \(w=\omega^2\), their difference is \(A^2\{\exp(2w)-\exp(w)-w\}>0\). To see the inequality, set \(x=\exp(w)>1\); then \(\exp(2w)-\exp(w)=x(x-1)\), which exceeds \(w\) for \(w>0\), as also follows from the positive higher-order terms in its expansion about zero.

Why Curvature Determines the Mean Discrepancy

A short Taylor argument explains why a zero-mean random effect does not guarantee that the marginal mean equals \(f(0)\). If \(f\) is twice differentiable near zero, its local expansion includes a quadratic term:

Taking expectations and using \(E[\eta]=0\) and \(E[\eta^2]=\omega^2\) gives the local approximation

Thus, when curvature is positive near zero, the local second-order contribution raises the mean above the value at zero; when curvature is negative, it lowers it. This is a local justification, not an exact formula for an arbitrary \(f\): higher-order terms and behavior away from zero can matter, particularly when the random-effect variance is large.

Curvature Can Also Make FO Miss Variability

A particularly clear failure occurs when the derivative at zero is small even though \(f(\eta)\) changes appreciably over the random-effect distribution. Consider the positive prediction \(f(\eta)=A\exp(-\eta^2/2)\). Its derivative at zero is zero, so FO treats the prediction as constant and attributes no random-effect variance to it. But a nonconstant function of a random variable can have positive variance even when its slope at one point is zero.

Diagnostics, Alternatives, and Limits

In a scalar calculation, exact moments provide a direct benchmark when the prediction and random-effect distribution permit analytic integration. In more complicated models, compare the FO prediction and its slope at zero with the prediction over plausible random-effect values. A wide spread, marked curvature, or a derivative near zero despite substantial variation in predictions signals that the local linearization may be inadequate. Sensitivity of calculated moments to increasing \(\omega^2\) is also informative: nonlinear discrepancies often become more consequential as more random-effect probability lies away from the expansion point.

There are important limits to what the moment comparison establishes. For the exponential example, \(f(\eta)\) is lognormal, but adding independent normal residual error does not generally produce a normal marginal distribution. Matching a mean and variance alone does not determine the marginal likelihood. Also, exact moments for one scalar prediction do not by themselves quantify parameter-estimation bias in a fitted data set; that depends on the likelihood, design, and estimation procedure.

A common mistake is to reason, “The random effect has mean zero, so the mean prediction must equal the typical prediction.” That would follow for a linear prediction \(f(\eta)=a+b\eta\), but not for a curved \(f\). In the exponential example, \(E[\exp(\eta)]=\exp(\omega^2/2)>1\), so the exact mean exceeds the prediction at zero. In the negative-curvature example, the exact mean is below it. The correction is to average the nonlinear prediction over the random-effect distribution, analytically when possible or with an appropriate integration method, rather than treating a zero-mean input as if it guaranteed an unchanged output mean.