How Cluster Size Controls Random-Intercept Shrinkage

Why Cluster Size Matters

In a random-intercept model, each group has a latent deviation from a shared population level. We do not observe that deviation directly: we observe outcomes that also contain within-group noise. A group mean based on few observations is relatively noisy, so its deviation is pulled more strongly toward the population level. A group mean based on many observations is more precise and is pulled less. This is the role of cluster size in random-intercept shrinkage.

The key comparison is not simply “small groups versus large groups.” The amount of shrinkage depends on cluster size in relation to the within-group variance and the random-intercept variance. A large cluster can still be shrunk substantially if its observations are very noisy relative to the spread of group effects.

Model and Assumptions

Definition: Consider the Gaussian random-intercept model $$ Y_{ij}=\beta_0+b_j+\varepsilon_{ij},\qquad i=1,\ldots,n_j,\quad j=1,\ldots,J, $$ where $\beta_0$ is the common mean, $b_j\sim N(0,\tau^2)$ is group $j$'s random intercept, and $\varepsilon_{ij}\sim N(0,\sigma^2)$ is observation-level error. The random intercepts are independent across groups, the errors are independent across observations and groups, and all errors are independent of all random intercepts. The variances $\tau^2$ and $\sigma^2$ are positive. For the exact calculations below, treat $\beta_0,\tau^2,\sigma^2$ as known.

The group-specific mean is $\beta_0+b_j$. Write $\bar Y_j$ for the sample mean in group $j$, and define its observed deviation from the common mean as $d_j=\bar Y_j-\beta_0$. Averaging the model within group $j$ gives

d_j=b_j+\bar\varepsilon_j,\qquad \bar\varepsilon_j\sim N\left(0,\frac{\sigma^2}{n_j}\right).

Thus, under the assumptions, the sampling variance of the observed deviation decreases as $n_j$ increases. The group’s data inform its random intercept through its mean deviation; the deviations within the group do not provide additional information about $b_j$ under this model. In fitted applications, the parameters are usually estimated and substituted into the calculation. The exact results here describe the known-parameter model.

Deriving the Size-Dependent Prediction

The empirical Bayes prediction in this Gaussian model is the conditional mean of $b_j$ given the observed group data. The relevant conditional distribution can be derived from the prior distribution of $b_j$ and the sampling distribution of $d_j$. The earlier material on empirical Bayes prediction and the shrinkage weight establishes this conjugate-normal calculation; here the emphasis is how the weight changes with $n_j$.

b_j\sim N(0,\tau^2),\qquad d_j\mid b_j\sim N\left(b_j,\frac{\sigma^2}{n_j}\right).

The posterior density as a function of $b_j$ is proportional to the product of the prior and sampling densities. Ignoring factors that do not depend on $b_j$, its kernel is

\exp\left\{-\frac{1}{2} \left[\frac{b_j^2}{\tau^2}+ \frac{(d_j-b_j)^2}{\sigma^2/n_j}\right]\right\}.

Expanding the terms in brackets and collecting the terms involving $b_j$ gives a quadratic with precision $1/\tau^2+n_j/\sigma^2$. Completing the square yields a normal posterior with the following variance and mean:

\operatorname{Var}(b_j\mid d_j) =\left(\frac{1}{\tau^2}+\frac{n_j}{\sigma^2}\right)^{-1} =\frac{\tau^2\sigma^2}{\sigma^2+n_j\tau^2}, $$ $$ E(b_j\mid d_j) =\frac{n_j/\sigma^2}{1/\tau^2+n_j/\sigma^2}\,d_j =\frac{n_j\tau^2}{\sigma^2+n_j\tau^2}\,d_j.

Theorem: For the stated Gaussian random-intercept model, the empirical Bayes prediction of group $j$'s random intercept is $$ \widehat b_j^{\,EB}=\lambda_j(\bar Y_j-\beta_0),\qquad \lambda_j=\frac{n_j\tau^2}{\sigma^2+n_j\tau^2}. $$ The posterior variance is $\tau^2\sigma^2/(\sigma^2+n_j\tau^2)$. The weight $\lambda_j$ is between 0 and 1, so the prediction lies between the prior mean of zero and the observed deviation.

The weight increases with cluster size. Treating $n$ as a continuous quantity for this comparison,

\frac{d}{dn}\left(\frac{n\tau^2}{\sigma^2+n\tau^2}\right) =\frac{\tau^2\sigma^2}{(\sigma^2+n\tau^2)^2}>0.

Also, $\lambda_j$ approaches 0 as $n_j$ approaches 0 and approaches 1 as $n_j$ grows without bound. At $n_j=\sigma^2/\tau^2$, the data weight is $1/2$: the prediction is halfway between zero and the observed deviation. This threshold shows why sample size alone does not determine shrinkage. Increasing $\sigma^2$ raises the threshold, while increasing $\tau^2$ lowers it.

Worked Example: Same Deviation, Different Cluster Sizes

Suppose hypothetical groups have common mean $\beta_0=50$ points, random-intercept variance $\tau^2=4$ points squared, and residual variance $\sigma^2=9$ points squared. Group A has $n_A=2$ observations and mean $53$; group B has $n_B=18$ observations and mean $53$. Both observed deviations are $53-50=3$ points.

For group A,

\lambda_A=\frac{2(4)}{9+2(4)}=\frac{8}{17}=0.4706,\qquad \widehat b_A^{\,EB}=0.4706(3)=1.412\text{ points}.

For group B,

\lambda_B=\frac{18(4)}{9+18(4)}=\frac{72}{81}=0.8889,\qquad \widehat b_B^{\,EB}=0.8889(3)=2.667\text{ points}.

The raw deviations are identical, but the larger group’s prediction is closer to its observed deviation. The model assigns more weight to that mean because its sampling variance is lower. These are predictions of latent group deviations, not claims that either group’s outcomes were caused by its predicted intercept.

Precision, Not Size in Isolation

The weight can also be written as

\lambda_j=\frac{\tau^2}{\tau^2+\sigma^2/n_j}.

This form compares the prior variance of group effects, $\tau^2$, with the sampling variance of the observed group deviation, $\sigma^2/n_j$. When sampling variance is large relative to prior variance, the data are weak evidence about the particular group and the prediction is drawn toward zero. When sampling variance is small, the data receive more weight.

Worked Example: A Half-Weight Threshold

Suppose a hypothetical measurement process has $\tau^2=2.25$ units squared and $\sigma^2=9$ units squared. The cluster size at which the data weight is one half is

n=\frac{\sigma^2}{\tau^2}=\frac{9}{2.25}=4.

For a group of size 4 with observed deviation $d_j=4$ units,

\lambda_j=\frac{4(2.25)}{9+4(2.25)}=\frac{9}{18}=0.5,\qquad \widehat b_j^{\,EB}=0.5(4)=2\text{ units}.

For a group of size 16, with the same deviation,

\lambda_j=\frac{16(2.25)}{9+16(2.25)}=\frac{36}{45}=0.8,\qquad \widehat b_j^{\,EB}=0.8(4)=3.2\text{ units}.

The second group receives less shrinkage because its mean deviation has lower sampling variance, not because the model treats a large group as inherently more important in every respect. The comparison assumes the same residual and random-intercept variances for both groups.

Comparing Predictions and Their Precision

A larger weight means less shrinkage of the observed deviation, but it does not by itself describe all uncertainty in the prediction. Under the model, posterior variance decreases with cluster size:

\frac{\tau^2\sigma^2}{\sigma^2+n_j\tau^2}.

The square root of this variance is the posterior standard deviation of the latent intercept, conditional on the model parameters and observed data. It is not the standard deviation of a future individual outcome, which also includes observation-level error. The following comparison keeps the model parameters fixed and illustrates both the predicted deviations and posterior uncertainty.

Worked Example: Unequal Sizes and Opposite Deviations

For two hypothetical groups, suppose $\tau^2=9$ and $\sigma^2=16$, in squared measurement units. Group C has $n_C=4$ and observed deviation $d_C=-5$ units. Group D has $n_D=20$ and observed deviation $d_D=2.5$ units.

For group C,

\lambda_C=\frac{4(9)}{16+4(9)}=\frac{36}{52}=0.6923,\qquad \widehat b_C^{\,EB}=0.6923(-5)=-3.462\text{ units}.

Its posterior variance and standard deviation are

\operatorname{Var}(b_C\mid d_C)=\frac{9(16)}{16+4(9)} =\frac{144}{52}=2.769\text{ units}^2,\qquad SD(b_C\mid d_C)=\sqrt{2.769}=1.664\text{ units}.

For group D,

\lambda_D=\frac{20(9)}{16+20(9)}=\frac{180}{196}=0.9184,\qquad \widehat b_D^{\,EB}=0.9184(2.5)=2.296\text{ units}.

Its posterior variance and standard deviation are

\operatorname{Var}(b_D\mid d_D)=\frac{144}{196}=0.7347\text{ units}^2,\qquad SD(b_D\mid d_D)=\sqrt{0.7347}=0.8571\text{ units}.

Group C’s negative deviation remains negative after shrinkage, and group D’s positive deviation remains positive. Group D’s prediction is closer to its observed deviation and has lower posterior standard deviation. Both comparisons follow from the larger cluster size under the shared-variance model.

Alternatives, Diagnostics, and Limitations

The empirical Bayes prediction is a form of partial pooling: it combines the observed group deviation with the common prior center. Using each raw deviation $d_j$ without shrinkage is an unpooled alternative. It can be very noisy for small groups. Setting every group deviation to zero is complete pooling; it ignores the group-specific information. The empirical Bayes rule lies between these alternatives, with a different weight for each group size.

The size-based comparison depends on the model, not just on arithmetic. Check whether the assumed structure is plausible before interpreting the weights:

Within-group dependence: the variance $\sigma^2/n_j$ for the group mean relies on independent errors with common variance. Strong residual dependence can make the nominal cluster size overstate the information in the mean.
Unequal residual variances: if groups have different residual variances, their mean precisions differ for reasons beyond $n_j$. A single common weight formula may then be inappropriate.
Outliers or nonnormal distributions: one extreme observation can move a small group mean substantially. Normal-model shrinkage does not automatically protect against data errors or a poorly fitting outcome distribution.
Unusual cluster sizes: if cluster size is related to the group effect or to the outcome process, the assumption that sizes simply determine precision may not capture how groups entered the data.
Variance components near a boundary: if the estimated random-intercept variance is very small, the fitted weights can be small across groups. That behavior reflects the fitted model and should not be interpreted as evidence that every observed group difference is exactly absent.

A common mistake is to say, “The group with more observations must have a larger predicted effect.” That is not what the formula says. For fixed variances, a larger $n_j$ increases the weight on the observed deviation; the predicted effect still depends on the sign and magnitude of that deviation. A large group with $d_j=0$ has prediction zero, and a large group with a negative deviation has a negative prediction.

Key takeaway: Under the Gaussian random-intercept assumptions, cluster size controls the precision of the observed mean deviation. Larger clusters receive a larger data weight and a smaller posterior variance, while the predicted direction and magnitude still depend on the group’s observed deviation.

Check Your Understanding

Work through each calculation using the stated Gaussian random-intercept model. Solutions follow the questions.

With $\tau^2=4$, $\sigma^2=12$, $n_j=6$, and $d_j=3$ points, calculate the shrinkage weight and empirical Bayes prediction.
For $\tau^2=3$ and $\sigma^2=15$, find the cluster size at which the weight is one half. What happens to the weight when the cluster size is larger than this value?
Two groups have the same observed deviation of $-2$ units, with sizes 5 and 25. If $\tau^2=2$ and $\sigma^2=8$, calculate both predictions and state which is closer to the observed deviation.
Explain why a larger cluster does not necessarily have a larger predicted random intercept.

Solutions. (1) $\lambda_j=6(4)/(12+6(4))=24/36=0.6667$, so $\widehat b_j^{\,EB}=0.6667(3)=2.000$ points. (2) The half-weight size is $n_j=\sigma^2/\tau^2=15/3=5$. Since $\lambda_j$ increases with $n_j$, larger sizes give weights greater than one half. (3) For size 5, $\lambda=5(2)/(8+5(2))=10/18=0.5556$, giving $-1.111$ units. For size 25, $\lambda=25(2)/(8+25(2))=50/58=0.8621$, giving $-1.724$ units. The size-25 prediction is closer to $-2$. (4) Size increases the weight on the observed deviation; it does not determine that deviation’s sign or magnitude. A larger group with a negative observed deviation can have a negative prediction, and a group with zero deviation has prediction zero.