Patient-Level And Basket-Level Variation

Two Sources of Variation, Two Different Questions

A basket trial may enroll patients in several disease subgroups, or baskets. Outcomes vary among patients within a basket, even when they share the same underlying treatment effect. At the same time, the true treatment effect may differ from one basket to another. These are distinct sources of variation: patient-level sampling variation describes the randomness of observed patient outcomes given the model parameters, while between-basket treatment-effect variation describes differences among the basket-specific true effects.

Separating the two matters because a noisy estimate for one basket does not, by itself, imply that its true effect differs from other baskets. Conversely, a collection of precise basket estimates may still show genuine differences in treatment effect. A hierarchical model represents these sources in different components, rather than treating every observed difference as evidence of heterogeneity.

Where the Sources Appear in a Model

For a binary endpoint, let \(Y_{ij}\) be the response indicator for patient \(i\) in basket \(j\), with \(Y_{ij}=1\) for a response and \(Y_{ij}=0\) otherwise. Let \(x_{ij}\) indicate treatment assignment, and let \(\alpha_j\) represent the basket-specific baseline log odds. One illustrative model is

Conditional on \(p_{ij}\), the Bernoulli outcome is random. Its variance is \(p_{ij}(1-p_{ij})\), the patient-level component. The parameter \(\delta_j\) is the true treatment effect for basket \(j\), on the log-odds scale. The distribution of \(\delta_j\) across baskets has variance \(\tau^2\), the between-basket treatment-effect variance. The baseline term \(\alpha_j\) represents baseline risk differences; it is not the treatment-effect variance.

This specification assumes that outcomes are conditionally independent across patients given the model parameters, that the Bernoulli likelihood is appropriate, and that the basket effects follow the stated normal distribution. In a randomized trial, \(x_{ij}\) encodes assigned treatment; interpretation of \(\delta_j\) as a treatment contrast also relies on the trial’s design and the adequacy of the outcome model. Other link functions and outcome distributions can be used, but the distinction remains: the patient outcome distribution describes within-basket randomness, and the distribution of basket-specific effects describes between-basket variation.

For a continuous endpoint, a corresponding model might be

Here \(\sigma^2\) is the residual variance among patients conditional on their basket and treatment effect; \(\tau^2\) is the variance of true treatment effects across baskets. This model assumes independent, normally distributed residuals with common variance and normally distributed basket effects. The assumptions are modeling choices, not consequences of having multiple baskets. If residual variability differs by arm or basket, a common \(\sigma^2\) may be inadequate.

How Sampling Error and Heterogeneity Combine

Basket-specific estimates vary for two reasons: their underlying true effects may differ, and each estimate is noisy because it is based on a finite number of patients. A simple normal approximation makes the distinction explicit. Suppose an estimated basket effect \(\hat\delta_j\) has conditional sampling variance \(s_j^2\), and the basket effects have variance \(\tau^2\):

The first variance, \(s_j^2\), is the sampling variance of the estimate within basket \(j\). The second, \(\tau^2\), is the variance among true effects. To find the marginal variance of the estimate, use the variance decomposition \(\operatorname{Var}(A)=E\{\operatorname{Var}(A\mid B)\}+\operatorname{Var}\{E(A\mid B)\}\). Here \(A=\hat\delta_j\) and \(B=\delta_j\), so the conditional mean is \(\delta_j\) and the conditional variance is \(s_j^2\). Therefore,

when \(s_j^2\) is treated as fixed. Thus, a wide observed spread of estimates can reflect large sampling errors, substantial effect heterogeneity, or both. It is not valid to label the entire spread as between-basket variation without accounting for the precision of each estimate.

What a Variance Component Does—and Does Not—Say

The variance components answer different scientific and statistical questions. Patient-level variation concerns how much outcomes fluctuate among individuals under a given basket’s parameters. Between-basket treatment-effect variation concerns how widely true treatment effects differ across the modeled basket population. Posterior uncertainty about \(\delta_j\) or \(\tau\) is a third concept: it describes uncertainty about unknown parameters after observing data. A parameter can have a posterior distribution even if the model sets \(\tau=0\), and a positive \(\tau\) does not imply that every basket’s effect is estimated precisely.

For a binary endpoint, \(p_{ij}(1-p_{ij})\) is the conditional outcome variance for an individual response indicator. It is on the response-indicator scale, whereas \(\tau^2\) in the logistic model is on the squared log-odds scale. These variances should not be added directly: they are on different scales and refer to different random quantities. The simple addition \(s_j^2+\tau^2\) applies to estimates and effects expressed on the same scale under the stated normal approximation.

A further distinction is essential when baseline risks differ. Baskets can have different response probabilities even if their treatment effects are identical, because their baseline risks \(\alpha_j\) differ. Comparing raw response proportions across baskets therefore does not isolate treatment-effect heterogeneity. The model must represent baseline risk and treatment effects as separate quantities when both may vary.

Alternatives, Limitations, and Common Mistakes

These approaches are not interchangeable. A common-effect model rules out between-basket treatment-effect variation; an independent analysis does not estimate a common distribution of effects; and a hierarchical model introduces assumptions about that distribution. Sparse data may provide little information to distinguish the components, especially when each basket has few patients or there are few baskets. Results can then depend materially on modeling assumptions and prior information. A hierarchical model cannot manufacture information that the trial does not contain.

A common mistake is to see different observed response rates and conclude that treatment effects differ across baskets. That reasoning confuses observed outcomes with true effects and ignores both patient-level randomness and baseline-risk differences. The correction is to identify the modeled quantity first: patient outcomes vary through the likelihood, basket-specific effects vary through the random-effects distribution, and estimates remain uncertain because data are finite.

Another mistake is to interpret \(\tau\) as a patient-level standard deviation or as the standard error of a basket estimate. It is neither. Its scale is the scale of the basket-specific treatment effect, and it describes the spread of true effects under the hierarchical model. The estimate’s standard error instead depends on the within-basket data and the outcome model.