Predictions When the Estimated Random-Effect Variance Is Near Zero

Why a Near-Zero Variance Changes Predictions

A group’s empirical Bayes prediction combines its observed data with the fitted distribution of group effects. When the fitted random-effect variance is small, that distribution assigns little spread to group effects. The resulting predictions are pulled toward zero: the model estimates that groups differ little from the population-level reference, even if a particular group’s observed outcome differs from that reference.

This is a consequence of the fitted model, not by itself evidence of an optimizer failure. A variance estimate at or near zero can be a valid boundary estimate; it can also occur in a fit whose numerical behavior deserves investigation. The prediction consequence and the convergence question must be assessed separately.

Here, “toward zero” refers to the random effect on the model’s linear-predictor scale. It does not mean that the predicted outcome is zero, nor that the group’s observed outcome is replaced by zero. In a logistic model, for example, \(b_j=0\) means that the group has no estimated random-intercept departure from the fixed-effects linear predictor; the predicted probability can still be far from zero.

The Gaussian Case: An Exact Shrinkage Calculation

Start with a Gaussian random-intercept model, conditional on fitted parameters:

Assume the group effects are independent across groups, the errors are independent across observations and groups, and the errors are independent of the group effects. Suppose \(\beta_0\), \(\tau^2\), and \(\sigma^2\) are treated as known for this calculation. The group mean \(\bar y_j\), based on \(n_j\) observations, has conditional variance \(\sigma^2/n_j\) around \(\beta_0+b_j\). The empirical Bayes prediction is the conditional mean of \(b_j\) given the group data. Under this Gaussian model, it is also the conditional mode.

The shrinkage formula follows by combining the normal likelihood for \(\bar y_j-\beta_0\) with the normal distribution for \(b_j\). Up to a constant independent of \(b_j\), the log of their product is:

This is a quadratic function of \(b_j\). Differentiating with respect to \(b_j\) and setting the derivative to zero gives:

In an empirical Bayes calculation, the fitted parameters are plugged into this expression. The weight \(w_j=\hat{\tau}^2/(\hat{\tau}^2+\hat{\sigma}^2/n_j)\) lies between zero and one when the variances are positive. If \(\hat{\tau}^2\) decreases toward zero while \(\hat{\sigma}^2/n_j\) remains positive, then \(w_j\) decreases toward zero and the prediction approaches zero. The group’s raw departure \(\bar y_j-\hat{\beta}_0\) is multiplied by a smaller weight.

What the Limiting Argument Means in a GLMM

For a non-Gaussian GLMM, a group’s likelihood contribution need not be quadratic in \(b_j\), so the Gaussian shrinkage formula is not generally exact. But the same limiting behavior has a direct justification for the conditional mode. With fitted fixed effects held fixed, the conditional mode maximizes the group log-likelihood plus the log density of its random effect. Ignoring constants, the objective is:

where \(\ell_j(b_j)\) is the group’s log-likelihood contribution as a function of its random intercept. The second term penalizes departures from zero. As \(\hat{\tau}^2\) becomes small, that penalty becomes increasingly strong for any fixed nonzero \(b_j\). If the group likelihood is smooth and its score at zero is finite, the conditional mode is therefore driven toward zero.

A local quadratic approximation makes the mechanism more explicit. If the group log-likelihood near its maximum is approximated by a quadratic with an unpenalized estimate \(z_j\) and variance \(v_j\), the approximate conditional mode is:

This is an approximation, not a universal exact formula for a binary or count GLMM. It is useful for understanding the direction of shrinkage: when \(\hat{\tau}^2\) is small relative to the group likelihood’s local uncertainty \(v_j\), the prediction is small relative to \(z_j\). Conditional modes and conditional means can differ in non-Gaussian models; the limiting effect of a variance collapsing to zero is the point here, not an assumption that every GLMM prediction follows the Gaussian formula exactly.

A Boundary Estimate Is Not a Convergence Diagnosis

Random-effect variances are constrained to be nonnegative. Consequently, a likelihood-based fit can attain its maximum at the boundary \(\hat{\tau}^2=0\). If the fitted model has exactly this value for a random intercept, its implied random intercepts are all zero, and the associated empirical Bayes predictions are zero. This says what the fitted model implies at the boundary; it does not, on its own, say whether the fitting algorithm successfully found the intended optimum.

Convergence concerns the numerical procedure used to optimize the fitting criterion. A near-zero estimate can appear in a fit that has converged successfully, or in a fit with numerical difficulties. Conversely, an optimizer’s “converged” message does not show that the model is correctly specified or that the estimate is well determined. Evaluate the estimate and the numerical diagnostics as distinct questions.

Diagnostics, Alternatives, and Limitations

When a fitted variance is near zero, inspect the fitting information rather than using the size of the group predictions as a convergence test. Useful checks include whether the optimizer reports successful termination, whether the gradient at the reported solution is small, whether the fitted criterion and parameters are stable across reasonable starting values, and whether the optimization behaves sensibly near the nonnegative boundary. A likelihood profile for the variance can also help show whether the criterion favors a boundary value or is nearly flat over a range of small values. Such a profile concerns uncertainty about the variance estimate; it is not a numerical convergence certificate by itself.

Several different summaries answer different questions:

• Raw group departures describe observed differences from a reference, but do not account for the fitted random-effects distribution and the group’s data precision.
• Empirical Bayes predictions use both the group data and the fitted variance. They are the appropriate plug-in predictions of latent group effects under the fitted model, but can be close to zero when that variance is small.
• A model with no random intercept sets the group-effect variance to zero rather than estimating positive group-to-group variation. Comparing this with a model that estimates the variance changes the model being fitted; it does not resolve optimizer diagnostics or variance uncertainty automatically.

The plug-in nature of an empirical Bayes prediction matters particularly near the boundary. It uses \(\hat{\tau}^2\) as if that fitted value were fixed. If the variance is weakly determined, predictions calculated at that single value can understate how much the predictions would change across plausible variance values. A small fitted prediction should therefore not be treated as proof that the true group effect is exactly zero.

There are also limits to what a near-zero variance says. It concerns variation represented by this particular random effect under the chosen model and data; it does not establish that groups are identical in every respect. The fixed-effects structure, outcome distribution, link, and sampling design still matter. Nor does the shrinkage result justify replacing a convergence check with a visual inspection of predictions. Prediction behavior describes the consequence of the fitted parameters; convergence diagnostics assess whether the fitting computation found a stable solution.