Why Inverse Links Do Not Commute With Averaging

Two Different Ways to Average

A GLMM connects a conditional linear predictor to a conditional response mean through an inverse link. When the predictor contains random effects, it varies across the random-effects distribution even at a fixed covariate value. The population-averaged response mean therefore averages the conditional response means. A tempting shortcut is to average the predictors first and then apply the inverse link. These operations are generally different because inverse links are often nonlinear.

The distinction matters whenever a prediction is meant to describe a population that includes variation in its random effects. A prediction for a cluster with random effect set to its mean is a plug-in prediction; it is not automatically the mean response across the random-effects distribution.

In a GLMM, for example, \(\eta=x^\mathsf{T}\beta+z^\mathsf{T}b\), where \(b\) denotes random effects. Conditional on \(b\) and \(x\), the response mean is \(h(\eta)\). The law of total expectation gives the marginal mean \(E(Y\mid x)=E_b[h(x^\mathsf{T}\beta+z^\mathsf{T}b)]\). By contrast, applying the inverse link after averaging the predictor gives \(h(E_b[\eta])\). If the random effects have mean zero, \(E_b[\eta]=x^\mathsf{T}\beta\), but that fact does not make the plug-in mean equal to the integrated mean.

Jensen’s Inequality and the Direction of the Difference

Jensen’s inequality gives a useful comparison when the inverse link has consistent curvature across the full support of \(\eta\). If \(h\) is convex on that support and the relevant expectations exist, then \(E[h(\eta)]\geq h(E[\eta])\). If \(h\) is concave there, then \(E[h(\eta)]\leq h(E[\eta])\). The inequality is strict when \(h\) is strictly convex or strictly concave over the relevant range and \(\eta\) is nonconstant, subject to the usual integrability conditions.

One way to justify the convex case is to use the supporting-line property of a differentiable convex function. For any \(m\) in its domain, \(h(u)\geq h(m)+h'(m)(u-m)\). Set \(m=E[\eta]\), substitute \(u=\eta\), and take expectations. The linear term has expectation zero, which yields the result:

Reversing the supporting-line inequality proves the concave case. If curvature changes sign over the support, this argument supplies no general direction. Moreover, nonlinearity alone does not guarantee a difference: the positive and negative contributions can balance for a particular distribution.

Curvature Can Change, and Differences Can Cancel

For the logistic inverse link \(h(\eta)=1/(1+\exp(-\eta))\), the second derivative is \(h''(\eta)=h(\eta)(1-h(\eta))(1-2h(\eta))\). It is positive for \(\eta<0\), zero at \(\eta=0\), and negative for \(\eta>0\). Thus the link is convex on the negative half-line and concave on the positive half-line. If the predictor distribution spans both regions, Jensen’s inequality alone does not determine the direction of the difference.

Interpreting and Computing the Relevant Mean

The target determines which calculation is appropriate. For a population-averaged response mean at fixed \(x\), integrate \(h(\eta)\) over the random-effects distribution. For a prediction at a specified random-effects value, evaluate \(h(\eta)\) at that value. For a plug-in prediction at the mean predictor, use \(h(E[\eta])\), while being explicit that this is not generally the integrated response mean.

When the integral is not available in closed form, it can be approximated by numerical integration or simulation. With simulated random effects \(b_1,\ldots,b_M\) drawn from the assumed random-effects distribution, an estimate of the integrated mean is \(M^{-1}\sum_{m=1}^M h(x^\mathsf{T}\beta+z^\mathsf{T}b_m)\). This averages after applying the inverse link to each simulated predictor. Averaging the simulated predictors first and applying \(h\) once instead estimates the plug-in quantity.

This comparison presumes a well-defined random-effects distribution, finite expectations for the quantities being averaged, and a correctly specified conditional mean and inverse link. If these assumptions are doubtful, a precise numerical integral does not repair the model. The integral is also tied to the target population represented by the random-effects distribution; changing that distribution can change the marginal mean.

Diagnostics, Alternatives, and Limitations

A practical check is to inspect the range of predictors that receives appreciable probability under the random-effects distribution and the curvature of \(h\) over that range. If curvature has one sign throughout, Jensen’s inequality provides a direction check. If curvature changes sign, compare the integrated mean and plug-in mean directly rather than guessing the direction from the mean predictor or from the link’s overall nonlinearity.

For a numerical calculation, check that the random-effects draws or integration points represent the intended distribution, that averaging is done on the response scale after applying \(h\), and that the estimate is stable as the number of draws or integration accuracy increases. A large discrepancy between two approaches is not necessarily a computational error: it may be the consequence of curvature and random-effects variability. Conversely, agreement in one example does not establish that the operations commute generally.

The main alternatives answer different questions. Integrated predictions target the average conditional response over random effects. Plug-in predictions target the response at an averaged predictor. An identity inverse link is an important special case: when \(h(u)=u\) and expectations exist, \(E[h(\eta)]=h(E[\eta])\) exactly. For nonlinear links, equality may still occur for a particular distribution, but it should be checked rather than assumed. Jensen’s inequality gives a bound only when its convexity or concavity condition holds across the support; it does not provide a universal approximation or direction when curvature changes.