Deriving the Shrinkage Weight for a Group Effect

Why a Gaussian Group Effect Is Shrunk

A group’s observed average can differ from the overall intercept because of a genuine group effect, sampling noise within the group, or both. A Gaussian random-intercept model represents these sources separately: group effects vary around zero, while observations within a group vary around that group’s level. The resulting prediction balances the group’s raw deviation against the model’s prior mean of zero.

The balance is not an arbitrary adjustment. It follows from the relative uncertainty in the group’s observed mean and in the distribution of group effects. This tutorial derives that balance algebraically, then uses it to calculate predictions and clarify the conditions under which the familiar shrinkage-weight formula is valid.

Model and assumptions: For group $j$, suppose $$ Y_{ij}=\beta_0+b_j+\varepsilon_{ij},\qquad i=1,\ldots,n_j, $$ where $b_j\sim N(0,\tau^2)$, and the within-group errors are independent with $\varepsilon_{ij}\sim N(0,\sigma^2)$. Assume the errors are independent of $b_j$, and groups are independent. For the derivation, treat $\beta_0$, $\tau^2$, and $\sigma^2$ as known. Here $n_j$ is the number of observations in group $j$, $\bar{Y}_j$ is its sample mean, and $d_j=\bar{Y}_j-\beta_0$ is its raw deviation from the overall intercept.

The model says that before observing group $j$, its effect is centered at zero with variance $\tau^2$. Conditional on its random intercept, the group mean has expectation $\beta_0+b_j$ and variance $\sigma^2/n_j$. Thus, the data provide a noisy measurement $d_j$ of $b_j$, with measurement-error variance $\sigma^2/n_j$. The posterior mean combines that measurement with the prior distribution of the effect.

Deriving the Weight

For positive $\tau^2$ and $\sigma^2$, the group-specific likelihood for $b_j$, apart from factors not involving $b_j$, is proportional to $ \exp\{-n_j(\bar{Y}_j-\beta_0-b_j)^2/(2\sigma^2)\} $. The prior density is proportional to $ \exp\{-b_j^2/(2\tau^2)\} $. Multiplying these densities gives the posterior density up to a normalizing constant. Completing the square in $b_j$ identifies its mean.

\begin{aligned} p(b_j\mid \mathbf{Y}_j) &\propto \exp\left\{-\frac{1}{2} \left[ \frac{b_j^2}{\tau^2} +\frac{n_j(d_j-b_j)^2}{\sigma^2} \right]\right\}\\ &\propto \exp\left\{-\frac{1}{2} \left[ \left(\frac{1}{\tau^2}+\frac{n_j}{\sigma^2}\right)b_j^2 -2\frac{n_jd_j}{\sigma^2}b_j \right]\right\}. \end{aligned}

A normal density with precision $A$ and mean $m$ has a quadratic exponent $A(b_j-m)^2$, whose linear coefficient is $-2Am$. Here the precision is $A=1/\tau^2+n_j/\sigma^2$, and matching the linear coefficient gives the posterior mean. The same square completion gives the posterior variance.

\begin{aligned} E(b_j\mid \mathbf{Y}_j) &= \frac{n_jd_j/\sigma^2}{1/\tau^2+n_j/\sigma^2}\\ &= \frac{n_j\tau^2}{\sigma^2+n_j\tau^2}d_j = \frac{\tau^2}{\tau^2+\sigma^2/n_j}d_j,\\ \operatorname{Var}(b_j\mid \mathbf{Y}_j) &= \left(\frac{1}{\tau^2}+\frac{n_j}{\sigma^2}\right)^{-1}. \end{aligned}

Definition: When $\tau^2+\sigma^2/n_j>0$, the Gaussian random-intercept shrinkage weight is $$ w_j=\frac{\tau^2}{\tau^2+\sigma^2/n_j}. $$ The posterior mean, also called the Gaussian empirical Bayes prediction when parameters are estimated and plugged in, is $E(b_j\mid\mathbf{Y}_j)=w_jd_j$ under the known-parameter model. The weight multiplies the raw group deviation; it is not the weight on zero. Equivalently, the posterior mean is $w_jd_j+(1-w_j)0$.

This expression also follows from the standard normal-normal updating rule: a noisy measurement is weighted according to its precision relative to the prior precision. In the derivation above, that rule is not assumed; it is obtained directly by completing the square. Since both variances are nonnegative, if their sum in the denominator is positive then $0\leq w_j\leq1$. For strictly positive variances, $0<w_j<1$, so the posterior mean lies between zero and the raw deviation, with its sign unchanged unless the deviation is zero.

Worked Calculations

Worked Example: Predicting a Group Effect From Four Observations

Hypothetical data: In one group, the four responses are 12, 15, 13, and 16. Suppose the model intercept is $\beta_0=10$, the within-group variance is $\sigma^2=16$, and the random-intercept variance is $\tau^2=9$. These are model inputs for this calculation, not estimates from a real study.

First calculate the observed mean and raw deviation: $$ \bar{Y}_j=\frac{12+15+13+16}{4}=\frac{56}{4}=14,\qquad d_j=14-10=4. $$ The variance of the group mean conditional on its intercept is $\sigma^2/n_j=16/4=4$. Therefore, $$ w_j=\frac{9}{9+4}=\frac{9}{13}=0.6923. $$

The predicted random intercept is $$ E(b_j\mid\mathbf{Y}_j)=0.6923(4)=2.769. $$ Thus, the estimated group-specific mean response is $\beta_0+E(b_j\mid\mathbf{Y}_j)=10+2.769=12.769$. The group’s observed mean was 14, but the model prediction is closer to the overall intercept of 10 because the observed mean is not treated as a perfectly precise measurement of the group effect.

Worked Example: A Noisy Mean Is Pulled Toward Zero

Hypothetical data: A group has $n_j=3$ observations and a raw deviation $d_j=-2$ response units from the overall intercept $\beta_0=8$. Suppose $\sigma^2=9$ response-units-squared and $\tau^2=1$ response-units-squared.

The group mean’s conditional variance is $9/3=3$, so $$ w_j=\frac{1}{1+3}=\frac14=0.25. $$ The posterior mean of the group effect is $$ E(b_j\mid\mathbf{Y}_j)=0.25(-2)=-0.5 $$ response units. Consequently, the predicted group-specific mean is $8-0.5=7.5$ response units. The negative sign is retained, but the predicted deviation is smaller in magnitude than the raw deviation of $-2$, reflecting the substantial within-group measurement variance relative to the random-effect variance.

Worked Example: Computing the Weight From the Posterior Precision

Hypothetical data: Suppose a group has $n_j=8$, a raw deviation of $d_j=2.5$ response units, $\sigma^2=4$ response-units-squared, and $\tau^2=0.25$ response-units-squared.

Using precisions, the prior precision is $1/\tau^2=1/0.25=4$, and the data precision for the group mean is $n_j/\sigma^2=8/4=2$. Hence the posterior mean is $$ E(b_j\mid\mathbf{Y}_j) =\frac{2}{4+2}(2.5) =\frac13(2.5) =0.8333 $$ response units. The weight formula gives the same result: the group-mean variance is $4/8=0.5$, so $w_j=0.25/(0.25+0.5)=1/3$. The posterior variance is $1/(4+2)=1/6$ response-units-squared. The calculation shows how the two formulations—variance weighting and precision weighting—agree.

What the Weight Means—and What It Does Not

The denominator $\tau^2+\sigma^2/n_j$ is the sum of the prior variance of a group effect and the conditional variance of the observed group mean around that effect. A larger $\tau^2$, relative to $\sigma^2/n_j$, makes the prior distribution more diffuse relative to the measurement uncertainty, so the data receive more weight. A smaller $\tau^2$ makes effects more concentrated around zero, so the prediction is pulled more strongly toward zero.

The weight is a property of the variance assumptions and the amount of information in the group mean. The raw deviation $d_j$ determines the direction and size of the unshrunk signal; the weight scales it. It is not the probability that the group effect is nonzero, and it does not say that the observed mean is incorrect. It expresses how the conditional mean of the latent group effect is formed under the stated model.

Assumptions, Comparisons, and Failure Modes

The exact weighted-average result relies on a normal prior for $b_j$, a normal within-group error distribution, independent errors with common variance, and the simple random-intercept structure. It also treats the model parameters—especially $\beta_0$, $\tau^2$, and $\sigma^2$—as fixed and known for the derivation. If parameters are estimated and substituted, the formula is a plug-in prediction; the derivation alone does not account for uncertainty in those estimates.

The posterior mean is a natural alternative to either using the raw deviation $d_j$ unchanged or assigning every group the prior mean zero. The raw deviation ignores the prior distribution of group effects; assigning zero ignores the group’s data. In this Gaussian model, the posterior distribution is normal, so its mean and mode coincide. That is why the posterior mean and conditional mode give the same group-effect point prediction here. This equality is a feature of the Gaussian conditional distribution, not a general rule for every GLMM.

Degenerate variance cases: The formula for $w_j$ applies when $\tau^2+\sigma^2/n_j>0$. If $\tau^2=0$ and $\sigma^2>0$, the random effect is fixed at zero and the formula gives $w_j=0$. If $\tau^2>0$ and $\sigma^2=0$, a group mean observed without error identifies the effect exactly, and the formula gives $w_j=1$, provided the observations are consistent with the model. If both variances are zero, the ratio defining $w_j$ is undefined. In that fully degenerate model, $b_j=0$ and the errors are also zero, so the only possible observations satisfy $Y_{ij}=\beta_0$; the posterior effect is zero, but there is no defined shrinkage weight.

A practical check is to compare the data pattern with the model assumptions before interpreting the calculation. Strong within-group dependence, marked heterogeneity in error variances, or observations that are inconsistent with the assumed normal structure can make $\sigma^2/n_j$ a poor description of the uncertainty in the group mean. A calculation can be arithmetically correct and still be a poor prediction if its variance model is unsuitable. Also check units: $\tau^2$ and $\sigma^2/n_j$ must both be variances on the response scale before they are compared.

Common Mistakes

Reversing the weight. The multiplier on the raw deviation is $\tau^2/(\tau^2+\sigma^2/n_j)$, not the fraction with $\sigma^2/n_j$ in the numerator. The complementary fraction is the weight on zero in the weighted-average expression.
Using the observation-level variance as the group-mean variance. The relevant conditional variance is $\sigma^2/n_j$, because the predictor uses $\bar{Y}_j$, not one individual observation.
Calling the prediction the observed group mean. The group-specific predicted mean is $\beta_0+w_jd_j$; the raw group mean is $\beta_0+d_j$.
Applying the ratio when its denominator is zero. If $\tau^2=\sigma^2=0$, the ratio is undefined. State the degenerate model’s prediction directly instead of claiming that the weight is between zero and one in that case.
Interpreting shrinkage as proof that a group has no effect. Shrinkage is a conditional prediction under the model, not a test or a conclusion that $b_j=0$.

Key takeaway: In a nondegenerate Gaussian random-intercept model, the posterior mean is the raw group deviation multiplied by a weight equal to the prior variance divided by the sum of the prior variance and the group-mean sampling variance. The weighted-average formula is exact under the stated model and known-parameter assumptions.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A group has $\bar{Y}_j=13$, $\beta_0=11$, $n_j=4$, $\tau^2=4$, and $\sigma^2=16$. Calculate $d_j$, $w_j$, and the posterior mean of $b_j$.
In the model, what variance is used for the group mean’s sampling uncertainty, and why is it not simply $\sigma^2$?
If $\tau^2=0$ and $\sigma^2>0$, what is the posterior group effect? Does this case have a defined weight?
Suppose $\tau^2=0$ and $\sigma^2=0$. Is it valid to substitute these values into the weight ratio? What prediction does the model imply when the observations are consistent with it?

Solutions. (1) $d_j=13-11=2$. The group-mean variance is $16/4=4$, so $w_j=4/(4+4)=0.5$, and $E(b_j\mid\mathbf{Y}_j)=0.5(2)=1$ response unit. (2) The variance is $\sigma^2/n_j$, because the mean of $n_j$ independent errors, each with variance $\sigma^2$, has variance $n_j\sigma^2/n_j^2=\sigma^2/n_j$. (3) The effect is fixed at zero, so its posterior value is zero; the formula is defined and gives $w_j=0$, since its denominator is positive. (4) No. The ratio is $0/0$ and is undefined. Under the consistent degenerate model, $b_j=0$ and $Y_{ij}=\beta_0$ for each observation, so the predicted group effect is zero without assigning a shrinkage weight.