Quantifying Shrinkage For Each Basket

What Does Shrinkage Measure?

In a basket trial, a hierarchical model lets the estimate for one basket depend partly on information from other baskets. Shrinkage describes how much that basket’s estimate moves from its unpooled estimate under this borrowing. It is a comparison of estimates, not a separate measure of treatment benefit or evidence that the baskets have identical effects.

The comparison is meaningful only when both estimates target the same quantity, use the same scale, and are expressed in the same units. For example, compare treatment-effect estimates in percentage points with other estimates in percentage points—not a risk difference with a log odds ratio. The unpooled estimate should also be defined clearly. In the normal examples below, it is the basket’s likelihood estimate, which is also its posterior mean under a flat prior.

Definition: For basket $j$, let $y_j$ be its unpooled estimate and let $\hat{\theta}_j$ be its hierarchical posterior estimate, such as the posterior mean. The signed shift is $\hat{\theta}_j-y_j$, and the absolute shift is $|\hat{\theta}_j-y_j|$. Given a stated reference center $c$, a relative shrinkage index is $$ R_j=\frac{y_j-\hat{\theta}_j}{y_j-c},\qquad y_j\ne c. $$ When the estimate moves toward $c$, $R_j$ lies between 0 and 1: 0 indicates no movement, and 1 indicates movement all the way to $c$. Values below 0 indicate movement away from $c$; values above 1 indicate movement past it. If $y_j=c$, the ratio is undefined and the absolute shift should be reported instead.

There is no single relative shrinkage number that is meaningful without specifying its reference center and estimate. The absolute shift answers “how many units did the estimate move?” The relative index answers “what fraction of its distance from this chosen center did it move?” Reporting both, where appropriate, prevents a proportion from concealing a clinically important change in units.

A Normal Hierarchy and Its Shrinkage Formula

A simple model makes the role of precision explicit. Suppose $y_j$ is an approximately normal estimate of basket $j$’s treatment effect $\theta_j$, with known sampling variance $V_j$. Suppose the basket effects are exchangeable around a common center $\mu$, with between-basket variance $\tau^2$:

y_j\mid\theta_j\sim N(\theta_j,V_j),\qquad \theta_j\mid\mu,\tau^2\sim N(\mu,\tau^2).

For the formula below, treat $\mu$, $\tau^2$, and $V_j$ as known, and assume the sampling errors are conditionally independent across baskets. The normal likelihood and normal prior give a normal posterior for $\theta_j$. The posterior precision is the sum of likelihood and prior precisions, $1/V_j+1/\tau^2$, so the posterior mean is

E(\theta_j\mid y_j,\mu,\tau^2) =\frac{y_j/V_j+\mu/\tau^2}{1/V_j+1/\tau^2} =\frac{\tau^2}{V_j+\tau^2}y_j+ \frac{V_j}{V_j+\tau^2}\mu.

The posterior mean is a weighted average of the basket’s estimate and the common center. Subtracting it from $y_j$ gives

y_j-E(\theta_j\mid y_j,\mu,\tau^2) =\frac{V_j}{V_j+\tau^2}(y_j-\mu).

Thus, when $y_j\ne\mu$, the relative shrinkage toward $\mu$ is $V_j/(V_j+\tau^2)$. This is the prior weight in the posterior mean. Greater sampling variance $V_j$ gives the center more weight; greater between-basket variance $\tau^2$ gives the basket’s own estimate more weight. If $\tau^2$ approaches zero, the model approaches complete pooling at $\mu$. If $\tau^2$ becomes very large relative to $V_j$, the posterior mean approaches $y_j$, with little shrinkage.

Worked Example: Comparing Shrinkage in Two Baskets

Suppose treatment effects are measured in percentage points. For illustration, use a normal hierarchy with fixed center $\mu=2$ percentage points and between-basket variance $\tau^2=16$ squared percentage points. Basket A has unpooled estimate $y_A=10$ and sampling variance $V_A=9$; Basket B has $y_B=-4$ and $V_B=36$. Assume the normal likelihood and exchangeable normal effects are reasonable for these estimates, and that these variances and hyperparameters are treated as known.

For Basket A, the data weight is $16/(9+16)=0.64$, and the center weight is $9/25=0.36$. Its hierarchical posterior mean is

\hat{\theta}_A=0.64(10)+0.36(2)=6.40+0.72=7.12 \quad\text{percentage points}.

The signed shift is $7.12-10=-2.88$ percentage points: the estimate moves down toward 2. The relative shrinkage is $(10-7.12)/(10-2)=2.88/8=0.36$, or 36%.

For Basket B, the data weight is $16/(36+16)=16/52\approx0.3077$, and the center weight is $36/52\approx0.6923$. Therefore,

\hat{\theta}_B=\frac{16}{52}(-4)+\frac{36}{52}(2) =-1.2308+1.3846=0.1538 \quad\text{percentage points}.

The signed shift is $0.1538-(-4)=4.1538$ percentage points, toward 2. Its relative shrinkage is $( -4-0.1538)/(-4-2)\approx0.6923$, or 69.23%. Basket B has the less precise estimate and is consequently pulled more strongly toward the center. The model does not say that its true effect equals Basket A’s or equals 2; it says that, under the stated hierarchy, the estimate is less dominated by Basket B’s noisier data.

Allowing for Uncertain Hyperparameters

In a fitted hierarchical model, $\mu$ and $\tau^2$ are usually uncertain rather than fixed. The conditional formula still describes the posterior mean given particular hyperparameter values, but the final posterior estimate averages over their posterior distribution. In general, one cannot obtain the fully Bayesian estimate simply by inserting posterior mean values of $\mu$ and $\tau^2$ into the fixed-parameter formula. The averaging and the nonlinear weights need not commute.

E(\theta_j\mid\mathbf{y}) =\int E(\theta_j\mid y_j,\mu,\tau^2)\, p(\mu,\tau^2\mid\mathbf{y})\,d\mu\,d\tau^2.

Here $\mathbf{y}$ denotes the estimates from all baskets. In practice, posterior simulation can estimate this expectation by calculating the conditional mean for each posterior draw and averaging the results. The comparison with $y_j$ is then made using the resulting posterior estimate. If a relative index is reported, its reference $c$—for example, the posterior mean of the common center—must be identified. With uncertain hyperparameters, that index is a descriptive comparison; it need not equal a single conditional shrinkage weight or lie between 0 and 1.

Worked Example: Averaging Over Hyperparameter Uncertainty

Suppose a fitted model’s posterior is represented, for illustration, by two hyperparameter configurations. The first has posterior weight 0.25, $\mu=0$, and $\tau^2=4$; the second has weight 0.75, $\mu=6$, and $\tau^2=12$. For one basket, let $y=8$ and $V=4$. Treat this two-point representation as a toy approximation to the hyperparameter posterior, and assume the conditional normal model above.

Under the first configuration, the conditional posterior mean is

\frac{4}{4+4}(8)+\frac{4}{4+4}(0)=4.

Under the second configuration, it is

\frac{12}{4+12}(8)+\frac{4}{4+12}(6)=6+1.5=7.5.

Averaging over the two configurations gives the hierarchical posterior mean $0.25(4)+0.75(7.5)=6.625$. Relative to the unpooled estimate of 8, the signed shift is $-1.375$. The posterior mean of the center in this representation is $0.25(0)+0.75(6)=4.5$, so the descriptive relative index is $(8-6.625)/(8-4.5)=1.375/3.5\approx0.3929$, or 39.29%. The conditional shrinkage weights were 0.50 and 0.25, not 0.3929. This difference illustrates why an integrated posterior estimate should not be summarized as though it came from one fixed pair of hyperparameters.

Absolute Change and Relative Change

A relative index depends on distance from its reference center. Consequently, equal relative shrinkage can correspond to very different changes in clinical units. This matters when reporting basket-level results: a percentage alone may sound substantial or small without showing the estimate it changed.

Worked Example: The Same Fraction, Different Clinical Shifts

Consider two hypothetical baskets whose treatment effects are measured in percentage points. Let the common center be $\mu=0$, let $\tau^2=25$ squared percentage points, and let both sampling variances be $V=16$. Assume the known-parameter normal hierarchy applies. The data weight is $25/41$, and the relative shrinkage toward zero is $16/41\approx0.3902$, or 39.02%, for both baskets.

Basket C has $y_C=2$, giving

\hat{\theta}_C=\frac{25}{41}(2)=\frac{50}{41}\approx1.220, \qquad |2-1.220|\approx0.780 \quad\text{percentage points}.

Basket D has $y_D=12$, giving

\hat{\theta}_D=\frac{25}{41}(12)=\frac{300}{41}\approx7.317, \qquad |12-7.317|\approx4.683 \quad\text{percentage points}.

Both estimates move by the same fraction of their distance from zero, but Basket D’s absolute change is much larger. In a clinical report, the relative index should therefore be accompanied by the unpooled estimate, hierarchical estimate, and absolute shift in the treatment-effect units.

Assumptions, Diagnostics, and Alternatives

The closed-form weight is not a universal shrinkage formula. It relies on an approximately normal likelihood for the basket estimate, a normal exchangeable distribution of basket effects, conditional independence of sampling errors, and known $V_j$, $\mu$, and $\tau^2$. If the fitted model uses patient-level data, a non-normal likelihood, covariate adjustment, or uncertain hyperparameters, the fixed-parameter formula may not describe the final posterior estimate. The estimate comparison remains available, but its interpretation must match the fitted model.

Examine basket-level estimates and shifts alongside the model’s uncertainty and diagnostics. Large shifts concentrated in particular baskets may warrant checking whether their sampling variances are correctly specified, whether the common exchangeability assumption is plausible, and whether the model reproduces subgroup patterns. Posterior predictive checks across subgroups, leave-one-subgroup-out predictive diagnostics, and sensitivity to the hyperprior scale address related model-fit and borrowing concerns. A shrinkage calculation alone cannot diagnose those problems: a large shift is not proof of misspecification, and a small shift is not proof that borrowing is appropriate.

The main alternatives clarify what partial pooling is doing. An independent analysis estimates each basket without borrowing, so its hierarchical shift is zero by construction when the same estimate is used as the comparator. A shared-effect or complete-pooling analysis forces basket effects to be equal, effectively pulling all basket estimates to one common effect. A random-effects hierarchy lies between these extremes, allowing differences while borrowing according to the model. These approaches answer different modeling assumptions; the amount of shrinkage does not select the right assumption by itself.

Common Pitfalls

Calling the relative index “the percentage of information borrowed.” It measures movement in an estimate relative to a chosen center, not an effective sample size or a universal fraction of data contributed by other baskets.
Reporting a ratio without its center. A value such as 0.4 has no clear interpretation until the reference center and the unpooled estimate are specified.
Treating shrinkage as proof of equal effects. Partial pooling changes estimates without imposing equality; the posterior distribution can still reflect substantial basket differences.
Comparing estimates on different scales or from different estimands. A numerical shift is not interpretable if one estimate is a risk difference and the other is a log odds ratio.
Using the fixed-$\mu,\tau^2$ weight as an exact full-posterior result. With uncertain hyperparameters, integrate over their posterior distribution or use posterior draws.
Interpreting a large shift as a clinical conclusion. A large absolute shift is a change in the point estimate, not by itself evidence of benefit or harm.

Key takeaway: For each basket, report the unpooled estimate, the hierarchical posterior estimate, and their signed or absolute difference on the same scale. If giving a relative shrinkage index, state its reference center and explain that it is a descriptive estimate comparison, especially when hyperparameters are uncertain.

Check Your Understanding

Answer each question, then check the solutions below.

In the known-parameter normal hierarchy, $V_j=9$ and $\tau^2=36$. What is the relative shrinkage toward $\mu$, and which estimate receives more weight?
A basket has unpooled estimate $y_j=5$, hierarchical estimate $\hat{\theta}_j=4$, and reference center $c=0$. Find its signed shift, absolute shift, and relative index.
Why is a relative shrinkage index undefined when $y_j=c$, even though the absolute shift may still be reported?
Why can the fully Bayesian posterior estimate differ from the fixed-parameter formula evaluated at posterior mean hyperparameters?
In one sentence, explain why a 40% relative shrinkage index does not mean that 40% of the information came from other baskets.

Solutions. (1) The center weight is $V_j/(V_j+\tau^2)=9/45=0.20$, so relative shrinkage is 20%; the data estimate receives weight $36/45=0.80$, more than the center. (2) The signed shift is $4-5=-1$ unit, the absolute shift is 1 unit, and $R_j=(5-4)/(5-0)=0.20$, or 20%. (3) The denominator $y_j-c$ is zero, so the ratio is undefined; the difference $|\hat{\theta}_j-y_j|$ remains defined. (4) The conditional weights depend nonlinearly on the hyperparameters, so averaging conditional posterior means over their posterior distribution generally differs from substituting posterior mean hyperparameters into one conditional formula. (5) The index measures how far the estimate moved relative to its distance from a chosen center; it is not a measure of sample size or information contribution.