Latent-Effect Uncertainty Versus Parameter Uncertainty

Two Different Sources of Uncertainty

A prediction of a group’s latent effect is not a directly observed quantity. It is inferred from the group’s outcomes, the fixed effects, and the assumed distribution of random effects. Even when the fitted model is held fixed, the group’s effect remains uncertain because the group’s data are finite. In addition, the fitted model is itself uncertain: its fixed-effect coefficients and variance components were estimated from data.

These sources should not be confused. Latent-effect uncertainty describes uncertainty about a particular group effect conditional on the observed data and specified model parameters. Parameter uncertainty describes how uncertainty in estimated fixed effects and variance components affects the predicted group effect and its uncertainty. A plug-in prediction substitutes parameter estimates as though they were known; its conditional interval therefore does not, by itself, account for all uncertainty due to estimating those parameters.

For a GLMM, a standard setup has observations conditionally independent given the random effects, with a response distribution from the exponential family and a link relating its conditional mean to fixed and random effects. Random effects are assigned a distribution, commonly \(b\sim N(0,D)\). The parameter vector \(\theta\) includes the fixed-effect vector and parameters of \(D\), as well as any dispersion parameters in the response model. The distinction in this tutorial applies whether \(b_j\mid y,\hat{\theta}\) is obtained analytically or approximated numerically.

In a Bayesian analysis, uncertainty about parameters can be represented by a posterior distribution for \(\theta\). In a frequentist analysis, parameter uncertainty is instead assessed through the sampling variability of estimates and predictions. These frameworks do not make every interval interchangeable: a conditional posterior interval and a frequentist prediction interval have different probability interpretations. Both make it important to state what was held fixed.

What a Gaussian Plug-In Prediction Conditions On

A random-intercept model makes the distinction explicit. Suppose a hypothetical continuous outcome \(Y_{ij}\) for observation \(i\) in group \(j\) follows

Assume the errors are independent of one another and of the random effects, and that \(\beta,\tau^2,\sigma^2\) are known for the moment. For a group with \(n_j\) observations, the group mean satisfies \(\bar{Y}_j\mid b_j\sim N(\beta+b_j,\sigma^2/n_j)\). Combining this likelihood for \(b_j\) with its normal prior gives a normal conditional distribution.

The conditional mean is the familiar shrunken group deviation. The conditional variance measures remaining uncertainty about \(b_j\) given this group’s observed mean and the stated parameter values. When parameters are estimated, the plug-in version substitutes \(\hat{\beta},\hat{\tau}^2,\hat{\sigma}^2\). It is still a conditional calculation: it does not include the sampling uncertainty of those estimates.

Parameter Uncertainty Changes the Prediction

When the parameters are estimated, the predicted conditional mean itself depends on the estimates. In the Gaussian example, write \(g(\theta)=\lambda_j(\bar{Y}_j-\beta)\). A first-order delta-method approximation describes variation in this plug-in mean as the estimates vary:

This is an approximation to parameter-related variability in the predictor, not the conditional variance of \(b_j\mid y,\hat{\theta}\). The latter answers a different question. If multiple parameters are estimated, their covariance terms matter; treating parameter estimates as independent when they are not can misstate the parameter contribution.

Combining the Sources: The Law of Total Variance

A parameter-integrated distribution accounts for the fact that the conditional distribution of \(b_j\) changes with \(\theta\). The law of total variance gives an exact decomposition in a Bayesian formulation, where both \(b_j\) and \(\theta\) have distributions conditional on the observed data:

The first term averages conditional latent-effect variance over parameter uncertainty. The second term is variation in the conditional mean as parameters vary. A plug-in conditional variance at \(\hat{\theta}\) generally equals neither the first term exactly nor the total variance. It also leaves out the second term. To justify the identity, write \(m(\theta)=E(b_j\mid y,\theta)\), then express \(b_j-E(b_j\mid y)\) as \([b_j-m(\theta)]+[m(\theta)-E(b_j\mid y)]\). The cross-product has expectation zero conditional on \(\theta,y\), since \(E[b_j-m(\theta)\mid y,\theta]=0\); taking variances yields the two terms above.

Interpretation, Diagnostics, and Alternatives

The conditional distribution \(b_j\mid y,\hat{\theta}\) is useful when the aim is to describe uncertainty about a group effect under a fitted model treated as fixed. It is not automatically an interval with nominal coverage for the latent effect across repeated samples in which the model parameters are re-estimated. Nor is it an interval for a future response: response prediction also involves outcome-level variation and, depending on the target, other sources of variation.

Several checks help reveal when a plug-in result deserves caution:

• Inspect parameter precision. Large standard errors, strong covariances, or flat likelihood profiles for fixed effects or variance components can make the predicted effect sensitive to parameter estimation.
• Check boundary estimates. If a random-effect variance is near zero, small changes in that estimate can substantially change the shrinkage calculation or make ordinary local approximations unreliable.
• Assess group information and distribution shape. Sparse groups can have broad conditional uncertainty. In binary GLMMs, conditional distributions may be skewed, so a symmetric normal interval can be a poor summary even if its center is reasonable.
• Compare conditional and parameter-integrated summaries when available. A substantial difference indicates that treating the fitted parameters as fixed is consequential for the target group prediction.

A plug-in conditional interval is simpler and can be informative when parameter estimates are precise and the conditional distribution is adequately represented. A parameter-integrated Bayesian analysis averages over parameter uncertainty directly. Frequentist approaches can instead assess the sampling uncertainty of the empirical Bayes predictor using its estimated covariance structure or resampling methods. These alternatives answer related but not identical inferential questions, so the target and interval interpretation should be stated explicitly.

A common mistake is to report the conditional standard deviation of \(b_j\mid y,\hat{\theta}\) as though it were the total uncertainty in the estimated prediction. The correction is to name what is conditioned on, then assess parameter uncertainty separately or integrate over it. Simply adding two standard errors is not generally justified: the relevant variance decomposition includes covariance and, in an integrated analysis, averages conditional variances over parameter values.