Interpreting Heterogeneity Across Basket Indications

Why Heterogeneity Matters in a Basket Trial

A basket trial evaluates a treatment in several disease indications, often because the indications share a biological feature. A favorable overall response rate can be encouraging, but it does not establish that the treatment benefits every indication. Conversely, an uncertain result in a small indication does not by itself show that its response probability differs from the others. The central interpretive task is to distinguish evidence for benefit across indications from evidence that the response probabilities differ in clinically important ways.

The earlier material on partial response-rate pooling gives a way to estimate indication-specific rates while allowing information to be shared. Here the emphasis is on how to interpret those estimates and their posterior probabilities. The conclusions depend on what “benefit” and “important heterogeneity” mean clinically, so those quantities should be specified before looking at the results.

Specify the Estimand Before Interpreting Results

“Common benefit” can refer to different claims. One claim is that the response probability averaged over a defined target population exceeds a benchmark. Another is that every indication has a response probability above that benchmark. These are not equivalent: an average can be favorable even when one or more indications have little benefit. For a weighted target population with prespecified weights \(w_j\), where \(w_j\geq 0\) and \(\sum_j w_j=1\), the average response probability is \(\bar p_w=\sum_j w_jp_j\). The all-indications claim concerns the joint event that \(p_j>p_0\) for every \(j\).

Heterogeneity also requires a clinical scale. A nonzero difference can be statistically plausible yet too small to change practice. A useful event is \(p_j-p_k>\Delta\), where \(\Delta\) is a prespecified clinically meaningful difference. One can also examine the range, \(\max_j(p_j)-\min_j(p_j)\), and ask whether it exceeds \(\Delta\). These posterior probabilities answer different questions; none should be substituted for another without explanation.

Model and Assumptions

For a binary response endpoint, let \(y_j\) of \(n_j\) evaluable patients respond in indication \(j\). A standard sampling model is \(y_j\mid p_j\sim\operatorname{Binomial}(n_j,p_j)\). A partially pooled model may represent the indication probabilities as \(\operatorname{logit}(p_j)=\mu+u_j\), with \(u_j\sim N(0,\tau^2)\). Here \(\mu\) describes the center of the indication effects on the log-odds scale, and \(\tau\) describes their dispersion. When \(\tau\) is small, the model tends to pull indication estimates toward a common center; when it is larger, the data can support greater between-indication differences.

This model relies on more than the binomial formula. Within each indication, responses are treated as independent conditional on \(p_j\), and the response definition and assessment schedule must be comparable across indications. The hierarchical distribution assumes the indications are exchangeable after accounting for the model: their labels do not determine their distributions, and their effects are treated as draws from a shared distribution. This is a modeling assumption, not a consequence of using the same drug or biomarker. Different case mix, follow-up, or response assessment can make apparent heterogeneity difficult to interpret as a treatment-effect difference.

The priors on \(\mu\) and \(\tau\) matter, especially when each indication is small. A prior favoring small \(\tau\) encourages similarity; a prior allowing large \(\tau\) makes substantial differences more plausible. A posterior subgroup probability should therefore be described as conditional on the likelihood, the model, and the priors—not as a property of the data alone.

A Short Derivation for a Subgroup Probability

A simple beta-binomial calculation shows how a subgroup probability is formed. Suppose, for one indication only, \(p\sim\operatorname{Beta}(a,b)\), and \(y\) responses are observed among \(n\) patients. The binomial likelihood is proportional to \(p^y(1-p)^{n-y}\). Multiplying by the beta prior density gives a posterior density proportional to \(p^{a+y-1}(1-p)^{b+n-y-1}\). Thus \(p\mid y\sim\operatorname{Beta}(a+y,b+n-y)\). The probability of exceeding \(p_0\) is the posterior tail area:

This calculation treats the indication separately. A hierarchical model instead updates the indication probability jointly with the other indications and the hyperparameters. The definition of the posterior probability is unchanged, but its value can change because partial pooling changes the posterior distribution.

Reading a Basket’s Posterior Results

Consider a hypothetical basket trial with four indications. The observed data are shown below. They are simulated for teaching and are not results from an actual study.

Assessing Whether Heterogeneity Is Consequential

A disciplined interpretation separates three questions:

These questions guard against equating variation in observed proportions with variation in underlying response probabilities. Small indications can have widely spread observed rates by chance. Conversely, partial pooling can make subgroup estimates look more alike than their raw proportions, especially under a prior that concentrates \(\tau\) near zero. A posterior probability for a clinically defined contrast is more directly interpretable than a declaration that heterogeneity is simply “present” or “absent.”

Diagnostics, Alternatives, and Limitations

Before interpreting subgroup probabilities, check that computation and model fit are adequate. For simulation-based posterior calculations, inspect chain mixing and convergence diagnostics for \(\mu\), \(\tau\), and the \(p_j\) values. A small effective sample size for a reported tail probability can make its Monte Carlo estimate noisy. For example, an estimated probability near a decision threshold should not be reported with false precision if few effectively independent draws determine it.

Also examine whether the model reproduces the observed pattern. Posterior predictive checks can compare replicated response counts with the observed counts, including the spread across indications and unusually high or low indication-specific rates. If an indication is a clear outlier, investigate whether differences in eligibility, response assessment, follow-up, or patient characteristics provide a plausible explanation. Check sensitivity to reasonable prior choices for \(\tau\), since shrinkage can materially affect subgroup probabilities in small baskets.

A common-rate model is a simpler alternative, but it assumes all indications share one response probability and cannot directly describe indication-specific effects. Separate, unpooled beta-binomial analyses allow each indication its own rate, but can be imprecise with small samples and do not borrow information. A hierarchical model lies between these approaches: it permits differences while regularizing estimates toward a common distribution. This advantage depends on exchangeability and the chosen prior; it does not make the model automatically robust to structural differences among indications.

Basket data are often nonrandomized or single-arm. A response rate above a benchmark does not by itself establish a causal treatment effect, and cross-indication differences do not necessarily identify biological effect modification. Historical expectations, eligibility criteria, and measurement practices can differ. Finally, subgroup probabilities should not be treated as automatic approval or treatment rules: their use in decisions requires a prespecified clinical benchmark, a clear target population, and consideration of the consequences of false-positive and false-negative decisions.

Common Pitfall

• Wrong reasoning: “The pooled response rate is above the benchmark, so the treatment benefits every indication.” This confuses an average with a joint claim about all \(p_j\).
• Correction: State the pooled estimand separately, then examine indication-specific probabilities and clinically meaningful contrasts. If the intended claim is benefit in every indication, evaluate the joint posterior event \(P(p_1>p_0,\ldots,p_J>p_0\mid\text{data})\), not just the average.
• Wrong reasoning: “One indication has a higher observed rate, so the indications are truly different.” Raw proportions do not account for sampling variability.
• Correction: Quantify posterior contrasts, including uncertainty, and compare them with a prespecified clinical difference. Check sensitivity to pooling assumptions and model fit before making a consequential claim.