Gaussian Random-Effects Forecasts as a Conjugate Benchmark

Why Use a Gaussian Benchmark?

Individual forecasts can require substantial computation when the response model is nonlinear or its uncertainty must be propagated numerically. A useful reference case is a model that is linear in Gaussian random effects and has Gaussian residual errors. With its population parameters treated as known, both the posterior distribution of an individual’s random effects and the predictive distribution of a future response have closed forms. These results make it possible to check numerical forecasts in simple cases and to see precisely how observed data update individual-level predictions.

This benchmark is deliberately narrower than a general nonlinear mixed-effects analysis. The response must be linear in the random effects, and the random-effect and residual covariance matrices are treated as known. The formulas do not automatically account for uncertainty in estimated population parameters.

In this tutorial, \(\beta\), \(G\), and \(R\) are treated as fixed and known. The observations are generated under the stated model, and the future residual errors are assumed independent of the observed residual errors and of the random effects. These conditions are what permit the simple conjugate calculations below. If residual errors are correlated across observation times, their joint covariance must instead be represented and conditioned on.

Posterior Distribution of the Random Effects

Write \(r=y-X\beta\) for the observed responses after subtracting their fixed-effect contribution. The likelihood for \(b\) is Gaussian, with mean \(Zb\) for \(r\) and covariance \(R\). Multiplying this likelihood by the Gaussian prior for \(b\) gives a Gaussian posterior.

The matrix \(C\) is the posterior covariance, and \(m\) is the posterior mean. The observed residuals \(y-X\beta\) inform the random effects through \(Z^\mathsf{T}R^{-1}(y-X\beta)\); observations with lower residual variance receive more weight through \(R^{-1}\). The prior contributes the precision matrix \(G^{-1}\), so the posterior reflects both the population distribution and the individual’s data.

Why the Posterior Is Gaussian

The result follows by completing the square in the exponent of the joint prior and likelihood. Ignoring factors that do not depend on \(b\), their product is proportional to

Expanding the quadratic expression gives

Set \(C=(G^{-1}+Z^\mathsf{T}R^{-1}Z)^{-1}\) and \(m=CZ^\mathsf{T}R^{-1}r\). Then the terms depending on \(b\) can be written as

Indeed, its quadratic term is \(b^\mathsf{T}C^{-1}b\), and its linear term is \(-2b^\mathsf{T}C^{-1}m=-2b^\mathsf{T}Z^\mathsf{T}R^{-1}r\). The remaining term does not depend on \(b\) and is absorbed into the normalizing constant. This is the kernel of a multivariate normal distribution with mean \(m\) and covariance \(C\), proving the result.

From the Posterior to a Future Forecast

Let \(y_*\) be a vector of future responses for the same individual. Suppose its model is \(y_*=X_*\beta+Z_*b+\varepsilon_*\), where the future design matrices are known and \(\varepsilon_*\sim N(0,R_*)\). Assume \(\varepsilon_*\) is independent of \(b\) and of the residual errors in the observed data. Conditional on \(y\), the uncertain random effects have distribution \(N(m,C)\), while the future residuals remain independent Gaussian errors.

The predictive mean substitutes the posterior mean of the random effects into the future model. The predictive covariance has two parts: \(Z_*CZ_*^\mathsf{T}\), uncertainty about the individual random effects after seeing the data, and \(R_*\), future residual variation. If the target is instead the latent future mean \(X_*\beta+Z_*b\), rather than a future observed response, omit \(R_*\). Confusing these two targets produces forecasts with the wrong variance.

The result follows because a linear transformation of a Gaussian vector is Gaussian. Given \(y\), \(Z_*b\) has mean \(Z_*m\) and covariance \(Z_*CZ_*^\mathsf{T}\). Adding the independent Gaussian error \(\varepsilon_*\) adds its covariance \(R_*\), giving the stated mean and covariance.

Using the Benchmark and Recognizing Its Limits

A closed-form forecast is useful as a validation case: enter the same design matrices, prior covariance, residual covariance, and conditioning observations into a numerical procedure, then compare its posterior mean and covariance with the formulas above. Agreement checks the implementation for this Gaussian linear case. It does not establish that the numerical procedure is correct for every nonlinear model or every uncertainty source.

A frequent mistake is to forecast only at the posterior mean random effect and report the future residual variance as the total predictive variance. In the random-intercept example, that would give variance 1 rather than \(1+4/9=1.444\;(\mathrm{mg/L})^2\). The omitted \(4/9\) is uncertainty about the individual effect. Conversely, if the target is the latent mean and not a future measured response, adding residual variance would include variation that is not part of the target.

Other checks follow directly from the assumptions and formulas:

• Check the target and units. The forecast mean must be on the response scale, and its variance must be in squared response units.
• Check covariance contributions. The posterior covariance \(C\) and future error covariance \(R_*\) must be positive semidefinite; a materially negative predictive variance signals an implementation or input problem.
• Check the design. Random-effect loadings in \(Z_*\) determine how posterior uncertainty carries into the forecast. A missing or incorrectly scaled loading can substantially change the variance.
• Check the model fit and assumptions. Strong residual patterns, outliers, nonconstant residual variance, or responses incompatible with a Gaussian model can make the benchmark inappropriate.

The conditional mean \(m\) is a posterior point estimate of the random effects. Using only this estimate gives a plug-in forecast, while the full predictive distribution also propagates posterior random-effect uncertainty and future residual variation. Simulation or numerical integration may be needed when the response is nonlinear in random effects, the relevant distributions are not Gaussian, or the desired forecast includes uncertainty not represented by the known \(\beta\), \(G\), and \(R\). In particular, these formulas do not include uncertainty from estimating those population parameters.