Posterior Uncertainty for Gaussian Group Effects

Why uncertainty in a group effect matters

A predicted group effect is not known exactly, even when a model produces a single estimate for it. The amount of information about that effect depends partly on how many observations the group contributes and how variable observations are around the group effect. For a Gaussian random-intercept model, these ingredients lead to a direct expression for the conditional variance of a group effect.

The result is useful because it quantifies the uncertainty remaining within the fitted model after observing a particular group’s data, when the model parameters are treated as known. It also makes the role of group size precise: more observations increase information about the group effect and reduce its conditional variance, but the reduction becomes progressively smaller.

Model and assumptions

For group $j$, suppose the observed responses follow the Gaussian random-intercept model

Y_{ij}=\mathbf{x}_{ij}^{\mathsf T}\boldsymbol{\beta}+b_j+\varepsilon_{ij}, \qquad i=1,\ldots,n_j.

Here $Y_{ij}$ is the response for observation $i$ in group $j$, $\mathbf{x}_{ij}^{\mathsf T}\boldsymbol{\beta}$ is its fixed-effects contribution, and $b_j$ is the group-specific random intercept. The assumptions needed for the calculation are:

$b_j\sim N(0,\tau^2)$, where $\tau^2>0$ is the random-intercept variance.
$\varepsilon_{ij}\sim N(0,\sigma^2)$, where $\sigma^2>0$ is the residual variance.
The residuals are independent across observations and independent of $b_j$.
The parameters $\boldsymbol{\beta}$, $\tau^2$, and $\sigma^2$ are treated as fixed and known for this conditional calculation.

The fixed-effects contribution may differ among observations; its value is subtracted from each response before considering information about $b_j$. Define the residual after subtracting that contribution as $r_{ij}=Y_{ij}-\mathbf{x}_{ij}^{\mathsf T}\boldsymbol{\beta}$. Conditional on $b_j$, this residual equals $b_j+\varepsilon_{ij}$.

Definition: Under the stated Gaussian random-intercept assumptions, the conditional distribution of $b_j$ given the observations and fixed model parameters is normal, with conditional variance $$ \operatorname{Var}(b_j\mid \mathbf{Y}_j,\boldsymbol{\beta},\tau^2,\sigma^2) = \left(\frac{1}{\tau^2}+\frac{n_j}{\sigma^2}\right)^{-1} = \frac{\tau^2\sigma^2}{\sigma^2+n_j\tau^2}. $$ This is the model-based conditional variance of the latent group effect, not the residual variance $\sigma^2$.

Deriving the conditional variance

The prior density for $b_j$ is proportional to $\exp[-b_j^2/(2\tau^2)]$. Given $b_j$, the residuals $r_{ij}$ are independent normal observations with mean $b_j$ and variance $\sigma^2$. Therefore, the likelihood as a function of $b_j$ is proportional to $\exp[-\sum_i(r_{ij}-b_j)^2/(2\sigma^2)]$. Multiplying prior and likelihood gives the conditional density up to a factor that does not depend on $b_j$:

p(b_j\mid\mathbf{Y}_j,\boldsymbol{\beta},\tau^2,\sigma^2) \ \propto\ \exp\left\{-\frac{1}{2}\left[ \frac{b_j^2}{\tau^2} + \frac{1}{\sigma^2}\sum_{i=1}^{n_j}(r_{ij}-b_j)^2 \right]\right\}.

Expand the sum of squares. The terms involving $b_j$ are

\frac{b_j^2}{\tau^2} + \frac{1}{\sigma^2}\sum_{i=1}^{n_j}(r_{ij}-b_j)^2 = \left(\frac{1}{\tau^2}+\frac{n_j}{\sigma^2}\right)b_j^2 - 2\left(\frac{\sum_i r_{ij}}{\sigma^2}\right)b_j + \frac{\sum_i r_{ij}^2}{\sigma^2}.

Write $A=1/\tau^2+n_j/\sigma^2$ and $B=\sum_i r_{ij}/\sigma^2$. Completing the square gives $Ab_j^2-2Bb_j=A(b_j-B/A)^2-B^2/A$. The final term does not depend on $b_j$, so it changes only the normalizing constant. The conditional density is consequently normal with precision $A$, and hence variance $1/A$:

\operatorname{Var}(b_j\mid\mathbf{Y}_j,\boldsymbol{\beta},\tau^2,\sigma^2) = \frac{1}{1/\tau^2+n_j/\sigma^2} = \frac{\tau^2\sigma^2}{\sigma^2+n_j\tau^2}.

The same completion also gives the conditional mean, which helps show how the data enter the result:

\operatorname{E}(b_j\mid\mathbf{Y}_j,\boldsymbol{\beta},\tau^2,\sigma^2) = \frac{n_j\tau^2}{\sigma^2+n_j\tau^2}\,\bar r_j, \qquad \bar r_j=\frac{1}{n_j}\sum_{i=1}^{n_j}r_{ij}.

The observed residual values determine this conditional mean. In contrast, under these assumptions and with parameters fixed, the conditional variance depends on the group data through their count $n_j$, not through the residual values themselves.

How group size changes uncertainty

The variance can also be read as the inverse of a sum of precisions. The prior contributes precision $1/\tau^2$; each independent observation contributes $1/\sigma^2$. Thus a group with $n_j$ observations contributes total data precision $n_j/\sigma^2$. The variance decreases as this precision increases.

For fixed positive $\tau^2$ and $\sigma^2$, regarding group size as a continuous quantity for the purpose of examining the formula,

V(n_j)=\frac{\tau^2\sigma^2}{\sigma^2+n_j\tau^2}, \qquad \frac{dV}{dn_j} = -\frac{\tau^4\sigma^2}{(\sigma^2+n_j\tau^2)^2}<0.

So adding observations reduces conditional variance. But the absolute reduction from adding one observation gets smaller as $n_j$ grows: the magnitude of the derivative decreases with $n_j$. At $n_j=0$, the formula returns the prior variance $\tau^2$; as $n_j$ grows without bound, the conditional variance approaches zero. For finite $n_j$, it remains positive.

Larger residual variance $\sigma^2$ means each observation is less precise about the group intercept, so the conditional variance is larger. Larger random-intercept variance $\tau^2$ means the prior distribution is more diffuse, which also increases the conditional variance for a fixed, finite group size. These effects can be checked directly in the formula or by noting the prior and data precisions.

Worked Example: Conditional Variance for One Group

Suppose a hypothetical group has five observations from a Gaussian random-intercept model with known $\tau^2=4$ response-units squared and $\sigma^2=9$ response-units squared. The fixed-effects contribution is 10 response units for each observation, and the observed responses are 11, 12, 12, 13, and 12 response units. Calculate the conditional variance and standard deviation of the group effect.

The residuals after subtracting the fixed contribution are 1, 2, 2, 3, and 2 response units. Their mean is $\bar r_j=2$. The conditional variance uses $n_j=5$, $\tau^2=4$, and $\sigma^2=9$:

\operatorname{Var}(b_j\mid\mathbf{Y}_j,\boldsymbol{\beta},\tau^2,\sigma^2) = \frac{(4)(9)}{9+(5)(4)} = \frac{36}{29} \approx 1.241\ \text{response-units squared}.

The conditional standard deviation is the square root of the conditional variance:

\sqrt{\frac{36}{29}}\approx 1.114\ \text{response units}.

For context, the conditional mean is $(20/29)(2)\approx1.379$ response units. This observed mean residual affects the location of the conditional distribution, not its variance. The standard deviation describes the model-based spread of the conditional distribution of $b_j$, not the spread of individual responses around the fixed-effects contribution.

Worked Example: Comparing Groups of Different Sizes

Consider three hypothetical groups in the same model, with known $\tau^2=1.5$ and $\sigma^2=6$, both in response-units squared. Suppose the group sizes are 2, 8, and 18. Calculate and compare their conditional variances and standard deviations.

For $n_j=2$,

V(2)=\frac{(1.5)(6)}{6+(2)(1.5)} =\frac{9}{9}=1.000, \qquad \sqrt{V(2)}=1.000\ \text{response unit}.

For $n_j=8$,

V(8)=\frac{9}{6+12} =\frac{9}{18}=0.5000, \qquad \sqrt{V(8)}\approx0.7071\ \text{response unit}.

For $n_j=18$,

V(18)=\frac{9}{6+27} =\frac{9}{33}\approx0.2727, \qquad \sqrt{V(18)}\approx0.5222\ \text{response unit}.

The conditional variance falls as group size rises. It falls from 1.000 to 0.5000 when size rises from 2 to 8, and then to about 0.2727 when size rises from 8 to 18. The second increase in group size is larger, yet its reduction in variance is smaller. Group sizes alone determine this comparison under the assumed common parameters; the groups’ observed residual means would change their conditional means.

Worked Example: Finding a Group Size for a Precision Target

Suppose a hypothetical application has known $\tau^2=2.25$ and $\sigma^2=4$, in response-units squared. A group has 15 observed responses, and the goal is a conditional variance no greater than $0.25$ response-units squared. Determine the smallest integer group size that meets the target, and check whether this group meets it.

For a general group size $n_j$, substitute the parameters into the conditional variance and solve the inequality:

\frac{(2.25)(4)}{4+(2.25)n_j}\leq0.25 \quad\Longleftrightarrow\quad 9\leq1+0.5625n_j \quad\Longleftrightarrow\quad n_j\geq\frac{8}{0.5625}\approx14.22.

Because group size is an integer, the smallest size meeting the target is 15. At $n_j=14$, the variance is $9/(4+31.5)=9/35.5\approx0.2535$, which is above the target. At the observed size of 15,

V(15)=\frac{9}{4+(2.25)(15)} =\frac{9}{37.75} \approx0.2384\ \text{response-units squared}.

Thus this group meets the stated conditional-variance target. Its conditional standard deviation is $\sqrt{0.2384}\approx0.4883$ response units. This calculation establishes a model-based precision threshold; it does not establish that the model assumptions are adequate for the data.

Interpreting the result and checking its assumptions

The formula is exact for the stated Gaussian model when the parameters are fixed. It is not a universal rule for every mixed model. Its simple dependence on $n_j$ uses independent residuals with a common variance and a random intercept that enters each observation with coefficient one. If residuals are correlated, observations do not each supply the same independent increment of precision. If residual variances differ, each observation contributes a different amount of information. In either case, the derivation must use the appropriate likelihood rather than simply inserting the raw group size into this formula.

Before interpreting the result, check whether the Gaussian assumptions are plausible for the response and residuals, whether there are substantial outliers, and whether observations within groups are conditionally independent given the random intercept and fixed effects. Also check that the group count includes only observations with usable responses. A misspecified residual variance or dependence structure can make the calculated conditional variance misleadingly small or large.

The conditional variance is distinct from uncertainty about $\boldsymbol{\beta}$, $\tau^2$, and $\sigma^2$. Here those quantities are held fixed. In applications they are estimated, so the displayed variance alone does not account for uncertainty in those estimates. Likewise, a conditional variance is not the variance of a new response: a new response also contains residual variation.

The raw mean residual $\bar r_j$ is an alternative summary of the group’s data, but it is not itself a measure of uncertainty about the latent group effect. Its sampling variance as an estimate of a fixed group mean would be $\sigma^2/n_j$ under a different framing. The conditional distribution instead combines the likelihood information with the random-effect distribution, giving variance $(1/\tau^2+n_j/\sigma^2)^{-1}$. The distinction matters especially for small groups, where the prior information is a substantial part of the total precision.

Common mistake: Using $\sigma^2/n_j$ as the conditional variance of $b_j$ treats the group effect as an unrestricted fixed mean and ignores its random-effect distribution. For the Gaussian random-intercept model with known parameters, the correct conditional variance is $(1/\tau^2+n_j/\sigma^2)^{-1}$. It is smaller than either the prior variance or the sampling variance of the raw mean residual because it combines their precision contributions.

A useful diagnostic is to compare the calculated uncertainty across groups while holding the fitted variance components fixed. If reported conditional variances do not decrease as group size increases under this model, check the calculation, the group counts, and whether parameters or model structures actually differ across groups. More generally, a small numerical variance should not be interpreted as proof that a group effect is accurately captured if the Gaussian or independence assumptions are doubtful.

Check Your Understanding

Answer each question using the known-parameter Gaussian random-intercept model defined above. Solutions follow each question.

A group has $n_j=4$, $\tau^2=2$, and $\sigma^2=8$. What is the conditional variance of its random intercept?
Solution: $V=(2)(8)/[8+(4)(2)]=16/16=1.000$ response-units squared.
If $n_j$ increases while $\tau^2$ and $\sigma^2$ remain fixed, does the conditional variance increase or decrease? Give the limiting value as $n_j$ grows without bound.
Solution: It decreases because $dV/dn_j=-\tau^4\sigma^2/(\sigma^2+n_j\tau^2)^2<0$. Its limit is zero.
Two groups have equal sizes and share the same known variance parameters, but their mean residuals differ. Under the stated model, which group has the larger conditional variance?
Solution: Neither has a larger conditional variance; they are equal. The mean residual changes the conditional mean, while $n_j$, $\tau^2$, and $\sigma^2$ determine the conditional variance.
For $n_j=3$, $\tau^2=1$, and $\sigma^2=3$, calculate the conditional standard deviation.
Solution: The variance is $(1)(3)/[3+(3)(1)]=0.5000$ response-units squared. The standard deviation is $\sqrt{0.5000}\approx0.7071$ response units.
Why is $\sigma^2/n_j$ not the conditional variance of the random intercept in this model?
Solution: It is the variance of the raw mean residual under repeated sampling conditional on a fixed group mean. It omits the random-intercept distribution. The conditional variance of $b_j$ combines prior precision $1/\tau^2$ and data precision $n_j/\sigma^2$, yielding $(1/\tau^2+n_j/\sigma^2)^{-1}$.