Averaging Nonlinear Predictions Over Covariates

Why Averaging and Prediction May Not Commute

A marginal mean model specifies the mean response for a covariate profile. When its inverse link is nonlinear, calculating a prediction and averaging covariates are generally operations that do not commute. In particular, a prediction for a person with average covariates need not equal the average prediction across the people in a population. This distinction matters whenever the target is a population-average response, such as a standardized risk, rather than the response predicted at one constructed covariate profile.

The comparison is most transparent when the model is written in terms of its design vector. Let \(x_i\) be the design vector for covariate profile \(i\), including any indicator or interaction terms in the model, and let \(\beta\) be the coefficient vector. For inverse link \(h=g^{-1}\), the response-scale prediction is \(h(x_i^\mathsf{T}\beta)\). If the covariate distribution has finite support \(x_1,\ldots,x_K\) with weights \(w_i\), where \(w_i\geq 0\) and \(\sum_{i=1}^K w_i=1\), the average prediction is \(\sum_i w_i h(x_i^\mathsf{T}\beta)\). The prediction at the average design vector is \(h((\sum_i w_i x_i)^\mathsf{T}\beta)\). There is no general reason for these to be equal.

The weights may be population proportions, or empirical proportions in a sample being used as the standardization distribution. Standardization requires specifying that distribution: changing the weights can change the resulting mean even if the fitted coefficients stay fixed. Also, when a model has categorical variables or interactions, \(x_i\) means the full design vector. Averaging a product term such as \(A_i Z_i\) is not generally the same as multiplying the separate averages of \(A_i\) and \(Z_i\).

Curvature Explains the Difference

The direction of the difference depends on the curvature of the inverse link over the relevant range of linear predictors. The following identity makes that dependence precise. Let \(\eta_i=x_i^\mathsf{T}\beta\), \(m=\sum_i w_i\eta_i\), and suppose \(h\) is twice continuously differentiable on an interval containing every \(\eta_i\). Taylor's formula with integral remainder gives an exact expression for the difference between the average prediction and the prediction at the average linear predictor.

To prove the identity, apply Taylor's formula to each \(\eta_i\), expanding \(h(\eta_i)\) around \(m\):

Multiply by \(w_i\) and sum over \(i\). The constant terms sum to \(h(m)\), since the weights sum to one. The first-order terms vanish because \[ \sum_i w_i h'(m)(\eta_i-m) =h'(m)\left(\sum_i w_i\eta_i-m\sum_i w_i\right) =h'(m)(m-m)=0. \] The remaining terms are exactly the right-hand side of the stated identity. Each squared deviation and each weight is nonnegative, so the sign conclusions follow from the assumed sign of \(h''\). Under strict curvature, a nonzero deviation with positive weight gives a strict inequality. \(\square\)

Because \(\eta_i=x_i^\mathsf{T}\beta\), the average linear predictor is \(m=(\sum_i w_i x_i)^\mathsf{T}\beta\). Thus the comparison in the result is exactly between a weighted average of response-scale predictions and a prediction at the average design vector. The result also shows that if all supported profiles have the same linear predictor, the two quantities agree, even if the profiles themselves differ.

For a log-link model, \(h(\eta)=\exp(\eta)\) and \(h''(\eta)=\exp(\eta)>0\), so averaging predictions is at least as large as predicting at the average linear predictor. For a logistic model, writing \(p=h(\eta)\), differentiation gives \(h'(\eta)=p(1-p)\) and \(h''(\eta)=p(1-p)(1-2p)\). The inverse-logit is convex when \(\eta<0\), concave when \(\eta>0\), and changes curvature at zero. Therefore a logistic model has no universal direction of difference when its predictors span regions on both sides of zero.

Standardizing Means and Contrasts

Standardization is particularly useful when comparing interventions or exposure levels while holding a common target covariate distribution fixed. For a treatment level \(a\), let \(x_i(a)\) be the design vector for profile \(i\) with treatment set to \(a\). The standardized mean at that level is \[ \mu_{\mathrm{std}}(a)=\sum_i w_i h(x_i(a)^\mathsf{T}\beta). \] A standardized risk difference, for example, is \(\mu_{\mathrm{std}}(1)-\mu_{\mathrm{std}}(0)\). Each treatment level is predicted for every target profile first; the response-scale predictions are then averaged with the same weights.

Common Mistake and Practical Interpretation

A common mistake is to reason that because the mean covariate is representative of the population, its predicted response must equal the population's average predicted response. That reasoning would be valid for an affine prediction function, but not generally for a nonlinear inverse link. A second related mistake is to average covariates before constructing interaction terms: for example, replacing the average of \(AZ\) by the product of the averages of \(A\) and \(Z\). In general, \(\sum_i w_i A_iZ_i\neq(\sum_i w_iA_i)(\sum_i w_iZ_i)\).

The correction is procedural: specify the target covariate distribution, construct the design vector for each supported profile under the condition of interest, calculate each response-scale prediction, and only then average using the target weights. With finite support, this is an explicit sum. The profile weights and the model predictions have distinct roles: the weights define whose covariate distribution is being represented, while the model supplies the predicted mean at each profile.

This distinction does not concern within-cluster dependence. Dependence among repeated outcomes does not change the arithmetic of a specified marginal mean model's response-scale predictions. It can matter for estimation and uncertainty calculations, but the present target is the mean obtained by averaging those predictions over covariates. Likewise, a standardized mean is not automatically a causal effect; a causal interpretation requires additional design and identification assumptions beyond the averaging operation itself.