Why “Near Zero” Needs Careful Interpretation
In a basket trial, a hierarchical model allows treatment effects to vary across subgroups while borrowing information among them. The between-basket heterogeneity parameter controls how much variation the model allows. When its posterior is concentrated near zero, subgroup estimates tend to be pulled toward a shared center. That behavior can be useful, but its meaning is easy to overstate: a small estimated variance is not, by itself, evidence that the true effects are exactly identical.
The distinction depends partly on the model. A continuous prior for heterogeneity can put substantial posterior probability in a small interval near zero, but it assigns probability zero to the single value zero. A model that includes a separate point mass at zero can, in contrast, assign positive posterior probability to exact equality. These are different statements and require different calculations.
The model assumes that the basket effects are exchangeable given the shared parameters: their labels do not determine their distributions, and their variation is represented by the common distribution around \(\mu\). This is a modeling assumption, not a conclusion established by observing a small posterior value of \(\tau\). The interpretation also depends on the outcome scale. A small standard deviation on a log-odds scale need not correspond to the same practical differences as a small standard deviation on a risk-difference scale.
How Near-Zero Heterogeneity Produces Shrinkage
For a simple derivation, suppose the estimated effect in basket \(j\) is \(y_j\), with known sampling standard error \(s_j\), and use the normal sampling model \(y_j\mid\theta_j\sim N(\theta_j,s_j^2)\). Conditional on \(\mu\) and \(\tau^2\), combine this likelihood with the normal distribution for \(\theta_j\). Completing the square in the product of the two normal densities gives a normal conditional posterior for \(\theta_j\). Its mean is
The coefficient on \(y_j\), \(w_j=\tau^2/(\tau^2+s_j^2)\), is between zero and one. When \(\tau^2\) is small relative to \(s_j^2\), \(w_j\) is small, and the conditional estimate is close to \(\mu\). At \(\tau^2=0\), the model requires every \(\theta_j=\mu\), so the conditional estimate equals \(\mu\). This is the complete-pooling boundary of this particular hierarchy.
The derivation also shows why uncertainty in \(\tau^2\) matters. In a fitted model, the posterior estimate averages over plausible values of \(\mu\), \(\tau^2\), and \(\theta_j\); it is generally not obtained simply by plugging the posterior mean of \(\tau^2\) into the formula. A posterior concentrated near zero can therefore imply substantial shrinkage, but the amount and uncertainty in that shrinkage depend on the full posterior and the basket-specific sampling errors.
Worked Example: Conditional Shrinkage at a Small Variance
Consider three hypothetical baskets with effect estimates \(y_1=0.20\), \(y_2=0.50\), and \(y_3=0.80\) on a log-odds scale. Suppose each has sampling standard error \(s_j=0.20\). For illustration, condition on \(\mu=0.50\) and \(\tau^2=0.01\). This is a calculation at specified parameter values, not a claim that these values have been estimated from trial data.
The conditional weight on each basket’s estimate is \(w_j=0.01/(0.01+0.20^2)=0.01/0.05=0.20\). Thus the conditional posterior means are
The estimates are pulled toward the shared mean, and the units remain log-odds. The calculation does not establish that the three true effects are equal: it shows what the hierarchical model implies conditional on the specified small variance and shared mean. If \(\tau^2\) were uncertain, the final posterior summaries would also reflect that uncertainty.
Continuous Priors and Exact Equality Are Different Claims
Suppose the prior for \(\tau\) is continuous on the nonnegative numbers. Under ordinary continuous-prior updating, a single point has probability zero both before and after observing data: \(P(\tau=0\mid y)=0\). It remains possible for the posterior to put, for example, high probability on \(0\leq\tau<\epsilon\), where \(\epsilon\) is a prespecified, interpretable tolerance. This is evidence that the effects are likely to be similar on the modeled scale to within that tolerance; it is not posterior probability of exact equality.
To assign positive posterior probability to exact equality, use a model with a point mass at \(\tau=0\). For instance, let \(M_0\) specify \(\tau=0\), and let \(M_1\) specify a positive heterogeneity value or a distribution over positive values. Posterior model probabilities then depend on the prior probabilities of the models and their marginal likelihoods. A point-mass model comparison answers a different question from summarizing a continuous posterior near zero.
Worked Example: A Point-Mass Model Comparison
Suppose, hypothetically, that two independent Gaussian contrast coordinates have observed values \(x_1=0.20\) and \(x_2=0.20\). Under \(M_0\), each has mean zero and variance \(0.09\); under \(M_1\), each has mean zero and variance \(0.13\). Interpret the extra variance \(0.04\) under \(M_1\) as a fixed positive heterogeneity contribution to a baseline variance of \(0.09\). The normal sampling model and independence of the two coordinates are assumptions of this illustration.
The sum of squared coordinates is \(S=x_1^2+x_2^2=0.04+0.04=0.08\). The likelihood ratio for \(M_1\) against \(M_0\) is
With equal prior probabilities for \(M_0\) and \(M_1\), Bayes’ rule gives \(P(M_0\mid x)=1/(1+0.793761)\approx0.5575\), using the likelihood ratio before rounding. In this deliberately simple comparison, the data shift the posterior probability toward exact equality, but leave substantial probability on the specified positive-variance model. The result is conditional on these two models and their prior probabilities; it is not the posterior probability of equality under a continuous prior for \(\tau\).
Worked Example: The Boundary Can Affect a Likelihood Ratio
For another hypothetical illustration, suppose three independent, mean-zero Gaussian contrast coordinates have sum of squares \(S=0.08\). Compare a null variance \(v_0=0.09\) with an alternative variance \(v_1=0.13\). Assume the contrast coordinates are independent and have the same variance within each model. For \(n=3\) such coordinates, the log likelihood ratio is
Here the specified positive-variance alternative has lower likelihood than the null for these observations. This likelihood ratio alone is not a posterior model probability: prior model probabilities are also required. Nor does it establish that a continuous posterior for heterogeneity has no mass near zero. It illustrates why a comparison involving the boundary must be interpreted through its particular model and assumptions.
What the Boundary Does—and Does Not—Tell You
The parameter space for a standard deviation or variance has a boundary at zero: negative heterogeneity is not allowed. When posterior draws cluster near that boundary, the reported summary can be sensitive to the scale and prior used for heterogeneity. A prior specified on \(\tau\) and one specified on \(\tau^2\) are not interchangeable without accounting for the transformation and its density. Sparse information can also leave a broad posterior that includes both very small and appreciable heterogeneity, even if one summary such as the posterior mean appears small.
The boundary can also create computational challenges. For example, a sampler parameterizing \(\tau\) may explore very small positive values slowly or produce draws that are hard to distinguish numerically from zero. Numerical rounding of draws to zero does not create a genuine posterior point mass. Check parameterization, effective exploration of the near-zero region, and sensitivity to reasonable prior choices; do not interpret a chain’s rounded output as proof of exact homogeneity.
A useful interpretation separates three claims:
- Near-zero posterior heterogeneity: values close to zero have substantial posterior support under the chosen model and prior.
- Practical similarity: differences smaller than a clinically meaningful threshold are sufficiently plausible. This requires defining the threshold on an interpretable effect scale.
- Exact equality: all effects are exactly the same. This requires a model that assigns positive probability to equality if the claim is to receive positive posterior probability.
These distinctions matter for borrowing. Strong shrinkage can stabilize noisy basket estimates, particularly when their sampling errors are large, but it can also make distinct subgroup estimates look similar. A shared estimate is a consequence of the model’s structure and the data; it is not independent evidence that the biological effects are identical.
| Approach | Implication for subgroup effects | Interpretive caution |
|---|---|---|
| Complete pooling | Effects are constrained to be equal in the model. | Equality is assumed, not inferred from a continuous near-zero posterior. |
| No pooling | Each subgroup is estimated separately. | It avoids cross-basket borrowing but does not use the hierarchical assumption of exchangeability. |
| Partial pooling | Subgroup estimates are drawn toward a shared distribution, with the amount informed by heterogeneity and sampling precision. | Near-zero heterogeneity can yield strong shrinkage without proving exact equality. |
| Point-mass model comparison | Can assign posterior probability to an exact-equality model and to alternatives. | Results depend on the candidate models and their prior probabilities. |
Common Mistakes and Practical Checks
- Wrong: “The posterior mean of \(\tau^2\) is small, so the baskets have identical effects.” Correction: A small summary is not a point mass at zero. Examine the posterior distribution and distinguish near-zero values from exact equality.
- Wrong: “The basket estimates are close after fitting the model, so the data demonstrate similarity.” Correction: Shrinkage itself pulls estimates together. Interpret the result in light of the hierarchical assumptions, sampling errors, and the posterior uncertainty in heterogeneity.
- Wrong: “A posterior probability below a chosen \(\epsilon\) is the probability of exact equality.” Correction: \(P(\tau<\epsilon\mid y)\) concerns a range of values. Under a continuous prior, \(P(\tau=0\mid y)=0\).
- Wrong: “A model comparison likelihood ratio is a posterior probability.” Correction: Posterior model probabilities also require prior model probabilities, as in Bayes’ rule.
Before interpreting a near-zero result, check that the effect scale is clinically meaningful, the heterogeneity prior is appropriate for that scale, and posterior computation adequately explores small positive values. Also verify that the exchangeability assumption is plausible: a concentration near zero cannot repair a model that combines clinically incompatible baskets. These checks do not determine whether the effects are equal; they clarify what the fitted model can support.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- Under a continuous prior for \(\tau\), what is the distinction between \(P(\tau<0.05\mid y)\) and \(P(\tau=0\mid y)\)?
- In the conditional shrinkage formula, what happens to the weight on \(y_j\) as \(\tau^2\) approaches zero while \(s_j^2\) stays positive?
- Why is a posterior mean of \(\tau^2\) insufficient to establish practical similarity?
- What additional quantities are needed to turn a likelihood ratio between exact-equality and positive-heterogeneity models into a posterior model probability?
- Name one computational or modeling check relevant when posterior draws lie close to the zero boundary.
Solutions. (1) The first is posterior probability of an interval of small positive and possibly zero values; the second is probability at one exact value, which is zero under a continuous posterior. (2) It approaches zero, so the conditional estimate approaches \(\mu\). (3) A mean does not show how much posterior mass lies near zero or in clinically important larger values, and practical similarity needs a meaningful threshold on the effect scale. (4) The prior probabilities of the models, combined with their marginal likelihoods, are needed. (5) Check that computation explores very small positive values, assess sensitivity to a reasonable heterogeneity prior, or examine whether exchangeability is defensible.