A Hand Calculation for Several Gaussian Groups

Why calculate predictions group by group?

A fitted Gaussian random-intercept model can produce a prediction for each group, but the arithmetic behind those predictions is worth understanding. A group with a large observed mean deviation may have a modest prediction if it contains little information; a smaller deviation from a large group may be estimated more precisely. Hand calculation makes both patterns visible by separating the observed deviation, the amount of shrinkage, and the conditional uncertainty.

The calculations here use one common fixed intercept, one random intercept per group, and no group-specific covariates. The goal is to compute predictions and conditional standard deviations for several groups with different sample sizes and observed mean deviations—not to treat a group’s raw mean deviation as if it were its random-effect prediction.

Model, assumptions, and calculation formulas

Definition: In a Gaussian random-intercept model, the observation $Y_{ij}$ for unit $i$ in group $j$ is $$ Y_{ij}=\beta_0+b_j+\varepsilon_{ij}, $$ where $b_j\sim N(0,\tau^2)$ and $\varepsilon_{ij}\sim N(0,\sigma^2)$. The random intercepts are independent across groups; the residuals are independent across observations and independent of the random intercepts. Here $n_j$ is the number of observations in group $j$, and $\bar{Y}_j$ is that group’s sample mean.

For the hand calculations, the fixed intercept and variance components are treated as known. In an empirical Bayes calculation, they are instead replaced by fitted values, such as $\hat{\beta}_0$, $\hat{\tau}^2$, and $\hat{\sigma}^2$. The resulting prediction and conditional standard deviation are plug-in quantities. The formulas assume a common residual variance and a common random-intercept variance across groups, with conditionally independent Gaussian residuals.

Define the group’s observed mean deviation from the fixed intercept as $d_j=\bar{Y}_j-\beta_0$. The empirical Bayes prediction is a shrunken version of this deviation. Its shrinkage factor depends on $n_j$, $\tau^2$, and $\sigma^2$. The conditional standard deviation describes uncertainty about $b_j$ given that group’s observed data and the model parameters; it does not include uncertainty in the estimated parameters.

\hat{b}_j^{\,EB} = \lambda_j(\bar{Y}_j-\hat{\beta}_0), \qquad \lambda_j= \frac{n_j\hat{\tau}^2}{\hat{\sigma}^2+n_j\hat{\tau}^2}, \qquad \widehat{\operatorname{SD}}(b_j\mid \mathbf{Y},\hat{\beta}_0,\hat{\tau}^2,\hat{\sigma}^2) = \sqrt{\frac{\hat{\tau}^2\hat{\sigma}^2}{\hat{\sigma}^2+n_j\hat{\tau}^2}}.

The prediction is on the response’s measurement scale: it estimates how far group $j$’s intercept is above or below the population intercept. The conditional standard deviation is on the same scale. It is not the standard deviation of individual responses, nor is it a confidence interval for a fixed population parameter.

Why the formulas have this form

For a derivation, condition on the group’s sample mean. Given $b_j$, the mean satisfies $\bar{Y}_j\mid b_j\sim N(\beta_0+b_j,\sigma^2/n_j)$. Thus $d_j=\bar{Y}_j-\beta_0$ has conditional distribution $N(b_j,\sigma^2/n_j)$, while the prior distribution for $b_j$ is $N(0,\tau^2)$. Multiplying these two normal densities gives the conditional distribution of the random intercept.

Write the prior and likelihood contributions.
Ignoring factors that do not involve $b_j$, their product is proportional to $\exp\{-\tfrac12[b_j^2/\tau^2+n_j(d_j-b_j)^2/\sigma^2]\}$.

Collect the terms involving the random intercept.
The coefficient of $b_j^2$ is $1/\tau^2+n_j/\sigma^2$, and the linear term is $n_jd_jb_j/\sigma^2$. Completing the square gives a normal conditional distribution.

Read off its mean and variance.
The conditional mean is $\frac{n_j/\sigma^2}{1/\tau^2+n_j/\sigma^2}d_j=\frac{n_j\tau^2}{\sigma^2+n_j\tau^2}d_j$; the conditional variance is $(1/\tau^2+n_j/\sigma^2)^{-1}=\frac{\tau^2\sigma^2}{\sigma^2+n_j\tau^2}$.

This derivation also explains the role of sample size. As $n_j$ increases, the likelihood contributes more precision about $b_j$, so $\lambda_j$ increases toward 1 and the conditional variance decreases. When $n_j$ is small, the prior centered at zero has greater influence.

Worked Example: Calculating Predictions for Two Groups

Suppose a hypothetical fitted model has fixed intercept $\hat{\beta}_0=50$ points, random-intercept variance $\hat{\tau}^2=16$ points squared, and residual variance $\hat{\sigma}^2=36$ points squared. Group A has $n_A=2$ and mean 56 points; Group B has $n_B=8$ and mean 53 points. Calculate both predictions and conditional standard deviations.

For Group A, the observed deviation is $d_A=56-50=6$ points. Its shrinkage factor is $2(16)/(36+2(16))=32/68=0.4706$. Therefore, $$ \hat{b}_A^{\,EB}=0.4706(6)=2.824\text{ points}. $$ Its conditional variance is $16(36)/(36+32)=576/68=8.471$ points squared, so its conditional standard deviation is $\sqrt{8.471}=2.910$ points.

For Group B, $d_B=53-50=3$ points. Its shrinkage factor is $8(16)/(36+8(16))=128/164=0.7805$. Thus, $$ \hat{b}_B^{\,EB}=0.7805(3)=2.341\text{ points}. $$ Its conditional variance is $576/164=3.512$ points squared, and its conditional standard deviation is $\sqrt{3.512}=1.874$ points.

Group A has the larger observed deviation, but its prediction is only slightly larger than Group B’s because its sample size is smaller. Group B’s conditional standard deviation is lower because its group mean provides more information about its intercept.

Repeating the arithmetic across several groups

In a multi-group calculation, first compute the shrinkage factor for each distinct group size. Then apply it to that group’s deviation from the fitted intercept. The conditional standard deviation depends on the sample size and the two variance components, not on whether the observed deviation is positive or negative.

Worked Example: A Four-Group Hand Calculation

Consider a hypothetical fitted model with $\hat{\beta}_0=100$ units, $\hat{\tau}^2=25$ units squared, and $\hat{\sigma}^2=100$ units squared. The four groups have the sample sizes and means shown below. Calculate each prediction and conditional standard deviation.

Group	$n_j$	$\bar{Y}_j$ (units)	$d_j=\bar{Y}_j-100$ (units)
A	1	108	8
B	5	104	4
C	20	102	2
D	20	97	-3

For Group A, $\lambda_A=1(25)/(100+25)=0.2000$, so $\hat{b}_A^{\,EB}=0.2000(8)=1.600$ units. Its conditional variance is $25(100)/125=20.00$ units squared, giving a conditional standard deviation of $\sqrt{20}=4.472$ units.

For Group B, $\lambda_B=5(25)/(100+125)=125/225=0.5556$, so $\hat{b}_B^{\,EB}=0.5556(4)=2.222$ units. Its conditional variance is $2500/225=11.11$ units squared, giving a standard deviation of $3.333$ units.

For Groups C and D, $\lambda=20(25)/(100+500)=500/600=0.8333$, and the conditional variance is $2500/600=4.167$ units squared, giving a standard deviation of $2.041$ units for each. Group C’s prediction is $0.8333(2)=1.667$ units; Group D’s prediction is $0.8333(-3)=-2.500$ units.

The two groups with $n_j=20$ have the same conditional standard deviation because their sample sizes and model variance components match. Their predictions differ because their observed deviations differ. A negative prediction, as for Group D, means the estimated group intercept is below the population intercept; it does not mean the group’s outcome itself must be negative.

Using conditional standard deviations as a precision check

A conditional standard deviation can be used to quantify the remaining uncertainty about a group intercept under the fitted model. It should not be read as parameter uncertainty or as the standard error of a population-level coefficient. For instance, one can calculate how large a group must be for its conditional standard deviation to fall below a specified threshold.

Worked Example: Finding the Group Size Needed for a Precision Target

Suppose $\hat{\tau}^2=16$ and $\hat{\sigma}^2=36$, both in points squared. What is the smallest group size for which the conditional standard deviation of its random intercept is less than 2 points?

The conditional variance is $576/(36+16n_j)$. Requiring the conditional standard deviation to be less than 2 is equivalent to requiring the variance to be less than 4: $$ \frac{576}{36+16n_j}<4 \quad\Longleftrightarrow\quad 576<144+64n_j \quad\Longleftrightarrow\quad n_j>6.75. $$

Because $n_j$ must be a whole number, the smallest qualifying size is $n_j=7$. At $n_j=7$, the conditional variance is $576/(36+112)=576/148=3.892$ points squared, and the conditional standard deviation is $\sqrt{3.892}=1.973$ points. At $n_j=6$, it is $\sqrt{576/132}=2.089$ points, which does not meet the target.

Checks, alternatives, and limitations

Before interpreting a hand calculation, check that the inputs correspond to the model. The deviation must be the group mean minus the fitted fixed intercept, not the raw group mean itself. The sample size must count observations contributing to that group mean. Variances must be on the squared response scale, and the same fitted variance components must be used consistently across the groups in the calculation.

The Gaussian assumptions matter. Strong skewness, influential observations, dependence among residuals, or group-specific residual variances can make the simple group-mean calculation inappropriate. A random-intercept variance estimated near zero also makes predictions close to zero, since the fitted model then allows little between-group variation. If a group has no observations, its data provide no group-specific deviation; under the stated model its conditional mean is zero and its conditional variance is $\tau^2$, rather than the formula’s result for a positive sample size.

For this Gaussian random-intercept setting, the empirical Bayes prediction is also the best linear unbiased prediction when the variance components are known. The raw deviation $d_j$ is a useful descriptive quantity, but using it without shrinkage ignores the prior distribution of group effects. A fully Bayesian analysis can also account for uncertainty in variance components and fixed effects by integrating over their posterior distributions. The plug-in conditional standard deviation here does not do that; bootstrap or other parameter-uncertainty procedures address a different source of uncertainty.

A practical diagnostic is to compare the inputs and outputs across groups. With common variance components, larger $n_j$ should mean a larger shrinkage factor and a smaller conditional standard deviation. Predictions should have the same sign as the observed deviations, unless the deviation is exactly zero. If these checks fail, recheck the arithmetic and the model inputs. Passing them does not establish that the Gaussian model fits; residual patterns, unusual groups, and the plausibility of common variances still require attention.

Key takeaway: For each group, subtract the fitted intercept from its sample mean, multiply by the group-specific shrinkage factor, and calculate conditional uncertainty from the same group size and fitted variance components. The prediction depends on both the observed deviation and its information content; the conditional standard deviation reflects latent-effect uncertainty conditional on the fitted model.

Check Your Understanding

Use the formulas above. The solutions are included so you can check both the arithmetic and the interpretation.

With $\hat{\tau}^2=9$, $\hat{\sigma}^2=16$, $n_j=4$, and observed deviation $d_j=3$ units, calculate the prediction and conditional standard deviation.
For fixed positive variance components, what happens to the shrinkage factor and conditional standard deviation as $n_j$ increases?
Two groups have the same $n_j$ and share fitted variance components, but one deviation is positive and one is negative. Which quantities must be equal, and how do their predictions relate?
Why is a plug-in conditional standard deviation not a measure of uncertainty in $\hat{\tau}^2$?

Solutions. 1. The shrinkage factor is $4(9)/(16+36)=36/52=0.6923$, so the prediction is $0.6923(3)=2.077$ units. The conditional variance is $9(16)/52=144/52=2.769$ units squared, giving a standard deviation of $1.664$ units. 2. The shrinkage factor increases toward 1, while the conditional standard deviation decreases toward 0. 3. Their shrinkage factors and conditional standard deviations are equal; their predictions have opposite signs when the deviations have equal magnitudes, and in general each prediction has the sign of its own deviation. 4. The plug-in calculation treats the fitted variance components as fixed inputs. It describes conditional uncertainty about a latent group effect given those inputs, not uncertainty about the variance-component estimates themselves.