Bayesian Evidence About Whether a Variance Component Is Zero

Two Different Questions About a Variance Component

A variance component near zero suggests little variation among groups on the scale being modeled. But “near zero” and “equal to zero” are not interchangeable claims. The first concerns a range of small positive values; the second is a comparison between distinct statistical models. Bayesian analysis can address either question, but the answer depends on how the prior is specified.

Let \(b_j\) denote the random effect for group \(j\), and suppose the model assumes \(b_j\mid\tau^2\sim N(0,\tau^2)\). Here \(\tau^2\) is the variance component and \(\tau\) is its standard deviation. At \(\tau^2=0\), every \(b_j\) is zero with probability one. The groups then have no variation attributable to this random effect, under the model. A positive but small \(\tau^2\) instead allows nonzero group effects that are tightly concentrated around zero.

The threshold \(\epsilon\) is not determined by the data alone. It should reflect what would count as negligible variation in the application, and it must be expressed on a stated scale. A threshold on the standard-deviation scale is not numerically the same as one on the variance scale: \(\tau<0.1\) is equivalent to \(\tau^2<0.01\), not \(\tau^2<0.1\).

Continuous Priors: Evidence for Small, Not Exactly Zero

Suppose the prior for \(\tau\) is continuous on the nonnegative real numbers, such as a half-normal prior. It gives probability zero to any single value, including zero. After observing data, the posterior is proportional to the likelihood times this prior. Provided the likelihood is finite and the posterior is proper, the posterior also assigns probability zero to the single point \(\tau=0\). Thus \(P(\tau=0\mid y)=0\), even if the posterior density is largest at zero or most posterior draws are very small.

This is not evidence that the variance is meaningfully positive. It is a consequence of the prior’s support structure: under a continuous prior, exact zero was not assigned positive probability before observing data. The relevant summaries are instead probabilities of intervals, such as \(P(\tau<\epsilon\mid y)\), and posterior intervals. The earlier material on posterior intervals for variance components near zero concerns such summaries; an interval reaching the boundary does not itself give a posterior probability for the exact boundary point.

A near-zero probability can be useful when a scientific decision has a meaningful tolerance. For example, investigators might regard a between-group standard deviation below 0.2 outcome units as practically negligible. The posterior probability \(P(\tau<0.2\mid y)\) then quantifies support for that defined range. It does not establish a point null, and changing the threshold changes the question.

Point-Mass Models and Exact-Zero Evidence

To assign positive posterior probability to exact zero, define separate models. Let \(M_0\) specify \(\tau^2=0\), and let \(M_1\) specify \(\tau^2>0\) with a proper prior on positive values. “Proper” means that the prior integrates to one. The prior must also assign positive probability to each model. Under \(M_1\), the data likelihood is averaged over the alternative prior to obtain its marginal likelihood.

For a composite alternative with nuisance parameters \(\theta\), the marginal likelihood is

The posterior model probability follows from Bayes’ rule. In odds form, the derivation is especially direct:

Thus data update prior model odds by multiplying them by the Bayes factor. The posterior probability of exact zero is \(P(M_0\mid y)\) when \(M_0\) is the model that fixes the variance component at zero. This quantity depends on both the observed data and the prior odds assigned to the models.

A Short Marginal-Likelihood Derivation

A simple normal example shows where a Bayes factor comes from. Suppose \(y_j\) is an estimate of group \(j\)’s deviation, with known measurement standard error \(s\). Under a normal random-effects model, integrating out the group effect gives \(y_j\mid\tau^2\sim N(0,s^2+\tau^2)\). Under \(M_0\), the variance is \(s^2\); under a fixed positive-variance alternative, it is \(s^2+\tau^2\).

For \(m\) independent group estimates, the joint density at variance \(v\) is the product of the \(m\) normal densities. Dividing the density under \(M_1\) by that under \(M_0\) cancels the common constants and yields

The determinant-like variance factor penalizes the more diffuse model, while the exponential term reflects how well that model accommodates the observed squared deviations. The ratio is a likelihood ratio for these marginal models. If the alternative has a prior distribution over \(\tau^2\) rather than a fixed value, its marginal likelihood must be integrated over that prior; one cannot simply substitute a posterior estimate of \(\tau^2\).

Assumptions, Diagnostics, and Alternatives

Evidence from a Bayes factor is meaningful only relative to well-defined models and their priors. Before interpreting it, check that the models describe the same observed data and differ in the intended way; that the zero model really sets the component to zero; and that the alternative assigns probability to scientifically plausible positive values. The likelihood, including any shared nuisance-parameter treatment, must be specified consistently. Both model priors must be proper for ordinary marginal likelihoods, and prior model probabilities must be stated to interpret posterior model probabilities.

For an alternative with a continuous prior, the Bayes factor depends on the distribution of prior mass over positive variance values. A very diffuse prior may predict data poorly across much of its range, reducing its marginal likelihood even when a narrow range fits well. Consequently, the Bayes factor is not a prior-free measure of evidence, and it is not the likelihood ratio evaluated at the best-fitting positive variance. Sensitivity to reasonable alternative priors is a diagnostic for how strongly the conclusion depends on prior specification.

These approaches answer different questions:

A common mistake is to reason, “The posterior interval for the variance includes zero, so the posterior probability that the variance equals zero is positive.” An interval that touches the boundary can contain positive values arbitrarily close to zero while assigning probability zero to the boundary point itself. Under a continuous prior, exact zero remains a probability-zero event. To obtain a posterior probability for exact zero, specify a model with a point mass there; to assess practical negligibility, define and report a meaningful near-zero threshold.

The point-mass approach also has limits. The posterior probability of \(M_0\) is conditional on the candidate models; if all candidates are poor descriptions, a large relative probability does not make \(M_0\) adequate. A Bayes factor comparing one chosen positive variance with zero is not the same as a Bayes factor integrating over a broad positive-variance prior. In a GLMM, the result is also conditional on the assumed response distribution, link, and random-effects structure. Model comparison does not replace checking whether those assumptions are reasonable.