Deriving The Normal-Normal Shrinkage Weight

Why Derive the Shrinkage Weight?

In a basket trial, each subgroup has an observed estimate, but a hierarchical model does not generally treat that estimate as entirely separate from the others. Under a normal-normal hierarchy, the posterior mean for a subgroup lies between its observed mean and the hierarchical mean. The amount it moves toward the hierarchical mean is determined by two sources of information: the subgroup’s sampling precision and the between-subgroup variance.

The formula is useful because it makes that balance explicit. It also prevents a common ambiguity: the “shrinkage weight” may refer either to the weight on the subgroup data or to the weight on the hierarchical mean. Here, the shrinkage weight $w_j$ will mean the weight on the observed subgroup mean. The weight on the hierarchical mean is $1-w_j$.

Model and Assumptions

Definition: For subgroup $j$, suppose the observations $X_{ij}$, for $i=1,\ldots,n_j$, are conditionally independent and normally distributed with subgroup mean $\theta_j$ and known within-subgroup variance $\sigma_j^2$. The subgroup means follow a normal distribution with mean $\mu$ and known between-subgroup variance $\tau^2$: $$ X_{ij}\mid \theta_j \sim N(\theta_j,\sigma_j^2), \qquad \theta_j\mid\mu,\tau^2 \sim N(\mu,\tau^2). $$ Assume $\mu$, $\tau^2$, and each $\sigma_j^2$ are fixed and known for this conditional calculation. The subgroup observations are independent given their subgroup means, and subgroup means are conditionally independent given the shared parameters.

The model implies that the subgroup sample mean $\bar{x}_j$ has sampling distribution $N(\theta_j,\sigma_j^2/n_j)$. Write its sampling variance as $v_j=\sigma_j^2/n_j$. Thus, for inference about $\theta_j$, the subgroup data enter the normal-normal calculation through $\bar{x}_j$ and $v_j$. The known-variance assumption is important: the closed-form calculation below is exact under this model, not automatically exact when variances are estimated and substituted.

The earlier material on Normal-Normal Hierarchies For Continuous Outcomes establishes this hierarchical setup. The focus here is to derive the posterior mean’s weighting formula from the likelihood and prior rather than treating the formula as a rule to memorize.

Deriving the Posterior Mean

For a particular subgroup, suppress the index $j$ temporarily and write its sample mean as $\bar{x}$ and its sampling variance as $v=\sigma^2/n$. The sampling distribution and prior are $ \bar{x}\mid\theta\sim N(\theta,v) $ and $ \theta\mid\mu,\tau^2\sim N(\mu,\tau^2) $. By Bayes’ rule, the posterior density is proportional to the product of the sampling density and prior density. Ignoring factors that do not depend on $\theta$, its kernel is

\pi(\theta\mid\bar{x},\mu,\tau^2) \propto \exp\left\{ -\frac{1}{2} \left[ \frac{(\bar{x}-\theta)^2}{v} + \frac{(\theta-\mu)^2}{\tau^2} \right] \right\}.

To identify the posterior normal distribution, collect the terms involving $\theta$. The coefficient of $\theta^2$ inside the brackets is $1/v+1/\tau^2$, the posterior precision. The coefficient of the linear term is $-2(\bar{x}/v+\mu/\tau^2)$. Completing the square therefore gives the following posterior variance and mean:

V_{\text{post}} = \left(\frac{1}{v}+\frac{1}{\tau^2}\right)^{-1}, \qquad m_{\text{post}} = V_{\text{post}} \left(\frac{\bar{x}}{v}+\frac{\mu}{\tau^2}\right).

This is a precision-weighted average: the sample mean is weighted by its sampling precision $1/v$, and the hierarchical mean is weighted by its prior precision $1/\tau^2$. Rewriting the posterior mean as a weighted average makes the amount of shrinkage visible:

m_{\text{post}} = w\bar{x}+(1-w)\mu, \qquad w = \frac{1/v}{1/v+1/\tau^2} = \frac{\tau^2}{\tau^2+v}.

Returning to subgroup notation, the data weight and the prior-centering weight are $w_j=\tau^2/(\tau^2+v_j)$ and $1-w_j=v_j/(\tau^2+v_j)$. Since $v_j=\sigma_j^2/n_j$, the data weight can also be written in terms of the sample size and within-subgroup variance.

w_j = \frac{\tau^2}{\tau^2+\sigma_j^2/n_j} = \frac{n_j\tau^2}{n_j\tau^2+\sigma_j^2}.

Key result: The posterior mean for subgroup $j$ is $w_j\bar{x}_j+(1-w_j)\mu$. A larger sampling variance $v_j$ lowers the weight on the observed subgroup mean and increases shrinkage toward $\mu$. A larger between-subgroup variance $\tau^2$ raises the data weight and reduces shrinkage.

Interpreting the Formula

The formula compares variance scales, or equivalently precision scales. When $v_j$ is small relative to $\tau^2$, the subgroup mean is precise compared with the spread allowed among subgroup means. Then $w_j$ is near 1, so the posterior mean stays close to $\bar{x}_j$. When $v_j$ is large relative to $\tau^2$, the subgroup estimate is noisy compared with the assumed between-subgroup spread. Then $w_j$ is near 0, so the posterior mean lies closer to $\mu$.

The posterior variance also has a useful precision interpretation:

V_{\text{post},j} = \left(\frac{1}{v_j}+\frac{1}{\tau^2}\right)^{-1} = \frac{v_j\tau^2}{v_j+\tau^2}.

It is less than either variance when both are positive and finite, because the subgroup likelihood and prior each contribute precision. This is a conditional posterior variance: it treats $\mu$ and $\tau^2$ as known. If those quantities are uncertain, this expression alone does not represent the full posterior uncertainty.

Worked Example: A Precise Subgroup Estimate

Suppose a hypothetical continuous endpoint has $n_j=16$ participants in a subgroup, known within-subgroup standard deviation $\sigma_j=4$ outcome units, observed mean $\bar{x}_j=12$ units, hierarchical mean $\mu=10$ units, and between-subgroup standard deviation $\tau=2$ units. Calculate the posterior mean and variance.

The sampling variance of the mean is $v_j=\sigma_j^2/n_j=4^2/16=1$ squared outcome unit. The between-subgroup variance is $\tau^2=2^2=4$ squared outcome units. Thus,

w_j=\frac{4}{4+1}=0.8.

The posterior mean is $0.8(12)+0.2(10)=9.6+2.0=11.6$ outcome units. The posterior variance is $1/(1/1+1/4)=1/1.25=0.8$ squared outcome unit, giving posterior standard deviation $\sqrt{0.8}=0.8944$ outcome units. The observed mean receives most of the weight because its sampling variance is small relative to $\tau^2$.

Worked Example: A Noisier Subgroup Estimate

Now suppose a different hypothetical subgroup has $n_j=4$, $\sigma_j=6$ outcome units, $\bar{x}_j=15$ units, and the same $\mu=10$ units and $\tau=2$ units as above. Calculate the shrinkage weight and posterior mean.

Here, $v_j=6^2/4=9$ squared outcome units, while $\tau^2=4$ squared outcome units. Therefore,

w_j=\frac{4}{4+9}=\frac{4}{13}=0.3077.

The posterior mean is $(4/13)(15)+(9/13)(10)=60/13+90/13=150/13=11.54$ outcome units. It is closer to the hierarchical mean than the observed mean because the subgroup’s sampling variance is large. The posterior variance is $1/(1/9+1/4)=36/13=2.769$ squared outcome units. In this example, the assumed hierarchy pulls the subgroup estimate substantially, but does not replace it with the hierarchical mean.

Worked Example: Equal Variances Give Equal Weights

Consider a third hypothetical subgroup with $v_j=2.25$ squared outcome units, $\tau^2=2.25$ squared outcome units, $\bar{x}_j=8$ units, and $\mu=10$ units. Find the posterior mean and explain the role of the equal variance values.

The weight on the observed mean is $w_j=2.25/(2.25+2.25)=0.5$, so the hierarchical mean also receives weight $1-w_j=0.5$. The posterior mean is $0.5(8)+0.5(10)=9$ outcome units.

The equal weights arise because the subgroup sampling variance and between-subgroup variance are equal, or, equivalently, their precisions are equal. The posterior variance is $(1/2.25+1/2.25)^{-1}=1.125$ squared outcome units. The result is halfway between the two means; equality of weights does not imply that either source of information is itself an estimate of the other.

Why the Derivation Matters

The formula also gives a direct check on the direction and size of the posterior adjustment. Subtracting the subgroup mean from the posterior mean yields

m_{\text{post},j}-\bar{x}_j = (1-w_j)(\mu-\bar{x}_j).

Because $0<w_j<1$ for positive finite $v_j$ and $\tau^2$, the posterior mean lies between $\bar{x}_j$ and $\mu$. Its displacement from the observed mean is the prior-centering fraction $1-w_j$ times the original difference. For instance, if the observed subgroup mean is above $\mu$, the posterior mean moves downward; if it is below $\mu$, it moves upward. The formula does not pull the estimate past $\mu$.

As limiting checks, if $\tau^2$ becomes very large while $v_j$ stays fixed, $w_j$ approaches 1 and the prior allows substantial subgroup differences. If $v_j$ becomes very large while $\tau^2$ stays fixed, $w_j$ approaches 0 and the noisy subgroup mean contributes little. If $\tau^2$ approaches zero, the model concentrates subgroup means at $\mu$, and the posterior mean approaches $\mu$. These are mathematical implications of the assumed model, not conclusions that should be imposed without assessing whether its assumptions are credible.

Assumptions, Diagnostics, and Alternatives

The closed-form result depends on a normal likelihood for the subgroup mean with known sampling variance, a normal prior for the subgroup mean, and fixed values of $\mu$ and $\tau^2$. The hierarchy also needs to be substantively reasonable: it treats subgroup means as exchangeable draws from a shared distribution centered at $\mu$. If subgroups differ systematically in ways not represented by this model, the common center and shared variance may be poor descriptions.

Useful checks include confirming that the analysis uses the correct standard error for each subgroup mean, checking whether the normal model is plausible for the continuous outcome and sample size, and examining whether the observed subgroup estimates are consistent with the assumed spread around $\mu$. In application, strong sensitivity of posterior means to plausible choices of $\tau^2$ is a warning that conclusions depend on the assumed between-subgroup variation. A single calculated weight does not diagnose model fit, and it does not account for uncertainty in $\tau^2$ or $\mu$.

The formula clarifies the distinction between partial pooling and two limiting alternatives. With no pooling, each subgroup is analyzed separately, so the mean is $\bar{x}_j$ and the effective data weight is 1. With complete pooling, every subgroup mean is constrained to equal $\mu$, corresponding to the limiting case $\tau^2=0$, so the effective data weight is 0. The normal-normal model gives partial pooling between these extremes. These comparisons describe different assumptions about subgroup variation; they do not establish which assumption is appropriate for a particular trial.

A common mistake is to say that a large sample size always eliminates shrinkage. The relevant comparison is not $n_j$ alone: $w_j=n_j\tau^2/(n_j\tau^2+\sigma_j^2)$, so the weight also depends on within-subgroup variability and the assumed between-subgroup variance. Another mistake is to call $1-w_j$ the data weight. Under the convention used here, $w_j$ weights $\bar{x}_j$, while $1-w_j$ weights $\mu$; stating both weights avoids ambiguity.

Key takeaway: In the normal-normal model, the posterior mean is a precision-weighted compromise between a subgroup’s observed mean and the hierarchical mean. The subgroup’s sampling variance determines how noisy its estimate is, while $\tau^2$ determines how much subgroup variation the model allows.

Check Your Understanding

Use the normal-normal model and show the calculation for each answer. Assume known variances and fixed $\mu$ and $\tau^2$ unless stated otherwise.

A subgroup has $v_j=1$, $\tau^2=3$, $\bar{x}_j=14$ units, and $\mu=11$ units. Find the data weight and posterior mean.
If $v_j=\tau^2$, what are the weights on $\bar{x}_j$ and $\mu$?
For $n_j=9$, $\sigma_j=3$ units, and $\tau=2$ units, calculate $w_j$.
In the fixed-parameter model, what happens to $w_j$ as $v_j$ grows without bound while $\tau^2$ remains positive and fixed?

Solutions. (1) $w_j=3/(3+1)=0.75$. The posterior mean is $0.75(14)+0.25(11)=13.25$ units. (2) Equal variances give $w_j=\tau^2/(\tau^2+v_j)=1/2$; each mean has weight 0.5. (3) $v_j=\sigma_j^2/n_j=9/9=1$, and $\tau^2=4$, so $w_j=4/(4+1)=0.8$. (4) Since $w_j=\tau^2/(\tau^2+v_j)$, it approaches 0; the posterior mean approaches $\mu$, reflecting the diminishing information in an increasingly noisy subgroup estimate.