Worked Binary Basket Analysis With A Beta-Binomial Model

Why a Worked Binary Basket Analysis Matters

In a basket trial, patients with different diseases or biomarker-defined subgroups may receive the same treatment. A binary outcome, such as response by a prespecified assessment time, can be summarized in each basket by its number of responses and its sample size. The challenge is to estimate each basket’s response rate while allowing information to be shared across baskets without assuming their rates are identical.

The beta-binomial hierarchy gives a transparent model for this task. It represents basket-specific response rates as varying around a common distribution, then updates each rate with its own observed data. This tutorial works through that update, a summary of posterior heterogeneity, and predictions for future outcomes. It uses fixed beta hyperparameters; it does not estimate the common distribution’s hyperparameters from these four baskets.

Model and assumptions: For basket $j$, let $Y_j$ be the number of responses among $n_j$ patients, and let $\theta_j$ be that basket’s response probability. The beta-binomial hierarchy is $$ Y_j\mid \theta_j \sim \operatorname{Binomial}(n_j,\theta_j),\qquad \theta_j\sim \operatorname{Beta}(a,b). $$ The hyperparameters $a,b>0$ are fixed in this analysis and shared across baskets. Conditional on the basket rates, patient outcomes are Bernoulli and independent within and across baskets. The basket rates are independent draws from the common beta distribution before observing the data. The response definition, follow-up window, and patient populations must be sufficiently comparable for this exchangeability assumption to be defensible.

The model does not say that every basket has the same response probability. Rather, it gives the rates a shared distribution, so a basket with limited data is informed by the common beta prior while a basket with more data is informed more strongly by its own observations. With fixed $a,b$, the conditional update is conjugate: responses add to the first beta shape parameter, and nonresponses add to the second.

Updating the Basket-Specific Rates

Consider the following hypothetical data. Each basket has 10 evaluable patients, and the outcome is response by the same assessment time. Use a $\operatorname{Beta}(2,2)$ prior for each basket’s rate. This prior has mean $2/(2+2)=0.5$ and is symmetric about 0.5; it is an illustrative modeling choice, not a claim that the baskets truly have identical rates.

Basket	Responses $y_j$	Nonresponses $n_j-y_j$
A	2	8
B	5	5
C	8	2
D	1	9

For a beta prior and binomial likelihood, the posterior is $\theta_j\mid Y_j\sim\operatorname{Beta}(a+y_j,b+n_j-y_j)$. The posterior mean is the first shape parameter divided by the sum of the two shape parameters. The posterior variance for $\operatorname{Beta}(A,B)$ is $AB/[(A+B)^2(A+B+1)]$.

Worked Example: Updating Four Basket Response Rates

For Basket A, there are $y_A=2$ responses and $n_A-y_A=8$ nonresponses. Starting from $\operatorname{Beta}(2,2)$, its posterior is

\theta_A\mid Y_A\sim\operatorname{Beta}(2+2,\,2+8) =\operatorname{Beta}(4,10).

Thus its posterior mean response rate is $4/(4+10)=0.2857$, or 28.57%. Its posterior variance is $4(10)/[14^2(15)]=0.01361$. The same calculation for the other baskets gives:

\begin{aligned} \theta_B\mid Y_B&\sim\operatorname{Beta}(7,7), & E(\theta_B\mid Y_B)&=7/14=0.5000,\\ \theta_C\mid Y_C&\sim\operatorname{Beta}(10,4), & E(\theta_C\mid Y_C)&=10/14=0.7143,\\ \theta_D\mid Y_D&\sim\operatorname{Beta}(3,11), & E(\theta_D\mid Y_D)&=3/14=0.2143. \end{aligned}

All four posterior means are response probabilities, so they are unitless proportions (or can be reported as percentages). The prior has contributed two pseudo-counts of each outcome category to every basket. For these equal sample sizes, the observed rates $0.2, 0.5, 0.8,$ and $0.1$ move toward the prior mean of 0.5. This is partial pooling through the shared prior distribution, not a guarantee that baskets have equal rates.

The posterior means alone do not show how uncertain each rate is. For example, Basket B’s posterior is $\operatorname{Beta}(7,7)$, with variance $49/[14^2(15)]=0.01667$; Basket A’s is $\operatorname{Beta}(4,10)$, with variance $40/[14^2(15)]=0.01361$. A posterior interval or posterior probability about a rate should be computed from its full beta distribution, rather than inferred from the mean alone.

Summarizing Posterior Heterogeneity

A useful finite-set summary is the population variance of the four basket-specific rates, with divisor $J$, where $J=4$: $V=J^{-1}\sum_{j=1}^{J}(\theta_j-\bar{\theta})^2$, and $\bar{\theta}=J^{-1}\sum_j\theta_j$. This quantity describes how dispersed the four latent rates are. Its posterior expectation accounts both for differences among posterior means and for uncertainty within each basket.

Let $m_j=E(\theta_j\mid Y)$, $v_j=\operatorname{Var}(\theta_j\mid Y)$, and $\bar m=J^{-1}\sum_jm_j$. Under this model, the basket rates are posterior independent because their priors and likelihoods factor by basket. Consequently, $\operatorname{Var}(\bar{\theta}\mid Y)=J^{-2}\sum_jv_j$. A careful derivation uses the centered expansion:

\begin{aligned} \sum_j(\theta_j-\bar{\theta})^2 &=\sum_j(\theta_j-\bar m)^2-J(\bar{\theta}-\bar m)^2,\\ E\left[\sum_j(\theta_j-\bar{\theta})^2\mid Y\right] &=\sum_j\left[v_j+(m_j-\bar m)^2\right] -J\operatorname{Var}(\bar{\theta}\mid Y). \end{aligned}

For the second line, $E[(\theta_j-\bar m)^2\mid Y]=v_j+(m_j-\bar m)^2$, and $E[(\bar{\theta}-\bar m)^2\mid Y]=\operatorname{Var}(\bar{\theta}\mid Y)$. Dividing by $J$ gives the posterior expected $V$. The subtraction is the variance of the posterior mean of the rates, not the posterior mean of its square.

Worked Example: Calculating Expected Across-Basket Variance

The four posterior means are $2/7, 1/2, 5/7,$ and $3/14$, so $\bar m=3/7=0.4286$. Their squared deviations sum to $15/98=0.1531$. The posterior variances are $40/2940,\ 49/2940,\ 40/2940,$ and $33/2940$, which sum to $27/490=0.05510$.

Because the rates are posterior independent, $\operatorname{Var}(\bar{\theta}\mid Y)=(27/490)/4^2=27/7840$. Therefore:

\begin{aligned} E(V\mid Y) &=\frac{1}{4}\left[ \frac{15}{98}+\frac{27}{490}-4\left(\frac{27}{7840}\right) \right]\\ &=\frac{381}{7840}=0.04860. \end{aligned}

This is the posterior expected variance across these four basket rates, measured in squared probability units. It is not the variance of the beta distribution’s hyperprior, nor a probability that the baskets differ. To express uncertainty about $V$ itself, one would calculate $V$ for each joint posterior draw of the four rates and summarize those draws; the expectation alone does not give an interval or a threshold probability.

Predicting Future Responses

For a future group of $m$ patients in an observed basket, the posterior predictive distribution integrates over uncertainty in that basket’s rate. If its posterior is $\operatorname{Beta}(A,B)$, then the future response count has a beta-binomial distribution:

Posterior predictive result: For $Y_{\mathrm{new}}$ responses among $m$ future patients in the same basket, conditional on the observed data, $$ P(Y_{\mathrm{new}}=k\mid Y) =\binom{m}{k}\frac{B(A+k,B+m-k)}{B(A,B)},\qquad k=0,\ldots,m, $$ where $B(\cdot,\cdot)$ is the beta function and $\operatorname{Beta}(A,B)$ is that basket’s posterior. The predictive mean is $mA/(A+B)$, and the predictive variance is $mAB(A+B+m)/[(A+B)^2(A+B+1)]$.

Worked Example: Predicting Responses in Basket A

Suppose five additional patients in Basket A have the same response definition, and the aim is to predict whether at least two will respond. Basket A’s posterior is $\operatorname{Beta}(4,10)$, so the predictive count is beta-binomial with $m=5,A=4,B=10$. The probability of zero responses is

P(Y_{\mathrm{new}}=0\mid Y) =\frac{(10)_5}{(14)_5} =\frac{10\cdot11\cdot12\cdot13}{15\cdot16\cdot17\cdot18} =0.2337,

where $(x)_r=x(x+1)\cdots(x+r-1)$. The probability of exactly one response is

P(Y_{\mathrm{new}}=1\mid Y) =5\frac{(4)_1(10)_4}{(14)_5} =\frac{343200}{1028160} =0.3338.

Thus $P(Y_{\mathrm{new}}\ge2\mid Y)=1-0.2337-0.3338=0.4325$, approximately 43.25%. The predictive mean count is $5(4/14)=1.429$ responses. The predictive variance is $5(4)(10)(19)/[14^2(15)]=1.293$ responses squared. The prediction includes both binomial outcome variation and uncertainty about Basket A’s response rate.

Diagnostics, Alternatives, and Limitations

The calculations are only as credible as the model and the data. Before interpreting partial pooling, check whether response definitions and assessment times match across baskets, whether denominators represent comparable evaluable populations, and whether outcomes can reasonably be treated as conditionally independent. Examine subgroup response patterns and uncertainty, and use posterior predictive checks to assess whether replicated data under the model can resemble the observed subgroup counts. Strongly surprising subgroup patterns, systematic residual patterns, or sensitivity to plausible prior choices warrant investigation; none is repaired merely by calculating a posterior mean.

The fixed-$a,b$ model has an important limitation: it assumes the common distribution of basket rates is specified rather than learned from the basket data. The value of the prior shapes can materially affect estimates when baskets are small. Also, exchangeability is a substantive assumption, not a consequence of using a beta distribution. If baskets differ in ways that predict response, treating them as draws from one common distribution may be inadequate.

Independent basket analyses avoid borrowing but can leave small baskets with very imprecise estimates. Complete pooling instead imposes one common response rate, which can conceal real variation. The beta-binomial hierarchy is between these approaches: it allows rates to vary while giving them a common distribution. Here the hyperparameters are fixed, so the analysis should not be described as learning the degree of heterogeneity from the four baskets. A model that estimates or averages over uncertain hyperparameters is a different analysis and requires additional assumptions and computation.

Common mistake: Pooling all 16 responses and 24 nonresponses into one binomial calculation gives a single rate of $16/40=0.4000$. Calling this the posterior response rate for every basket silently assumes complete pooling. The basket-specific posterior means here range from 0.2143 to 0.7143. Report the model-based subgroup estimates and their uncertainty; use an overall summary only when its target and assumptions are stated.

Check Your Understanding

Answer each question using the model and assumptions stated above. Solutions follow the questions.

Under the $\operatorname{Beta}(2,2)$ prior, what is the posterior distribution and posterior mean for a basket with 3 responses among 8 patients?
Why does the heterogeneity derivation subtract $J\operatorname{Var}(\bar{\theta}\mid Y)$ from the centered sum of squared deviations?
For Basket B, what is the posterior predictive mean number of responses among 4 future patients?
What does a large posterior expected across-basket variance establish, and what does it not establish?

Solutions. (1) The posterior is $\operatorname{Beta}(2+3,2+5)=\operatorname{Beta}(5,7)$, with mean $5/12=0.4167$. (2) In the centered expansion, the subtracted term is $J(\bar{\theta}-\bar m)^2$; its posterior expectation is $J\operatorname{Var}(\bar{\theta}\mid Y)$, since $E(\bar{\theta}\mid Y)=\bar m$. (3) Basket B’s posterior is $\operatorname{Beta}(7,7)$, so the predictive mean is $4(7/14)=2$ responses. (4) It summarizes the expected dispersion of the basket rates under the posterior; by itself it does not give the probability that a specified pair differs by a chosen amount or establish a cause for any differences.