Exact Assurance for Beta-Binomial Trial Designs

Why Exact Assurance Is Available for Binary Trials

For a fixed-size trial with a binary endpoint, there are only finitely many possible pairs of event counts. This makes it possible to compute assurance by evaluating the design’s success rule for every possible pair and weighting each decision by its prior-predictive probability. The result is an exact finite sum under the specified model, rather than an estimate based on simulated trials.

Assurance averages the probability of meeting a prespecified success criterion over uncertainty in the event rates. In this tutorial, the same beta distributions describe that uncertainty before the trial and serve as the analysis priors. The focus is the calculation for this fixed-size, independent-arm model.

Definition: Let $p_T$ and $p_C$ be the treatment- and control-arm event probabilities. Conditional on these rates, suppose the event counts are independent, with $X_T\sim\operatorname{Binomial}(n_T,p_T)$ and $X_C\sim\operatorname{Binomial}(n_C,p_C)$. Assign independent priors $p_T\sim\operatorname{Beta}(a_T,b_T)$ and $p_C\sim\operatorname{Beta}(a_C,b_C)$, where all beta parameters are positive. For a deterministic Bayesian success rule $S(x_T,x_C)$, assurance is the prior-averaged probability that the rule declares success.

The model assumes a fixed number of participants in each arm, a common event probability within each arm, and conditionally independent binary outcomes. Independent priors across arms imply independent prior-predictive counts. The rule must be specified before calculating assurance: for example, success might mean that the posterior probability of a treatment benefit exceeds a threshold. A different stopping rule or dependence structure changes the outcome model and is not represented by the simple product formula below.

From Beta Priors to a Finite Sum

For one arm, a beta prior is conjugate to the binomial likelihood. If $X=x$ events are observed among $n$ participants and $p\sim\operatorname{Beta}(a,b)$, the posterior is $\operatorname{Beta}(a+x,b+n-x)$. Integrating the binomial probability of $x$ over the prior gives the beta-binomial predictive probability:

P(X=x) =\binom{n}{x}\frac{B(a+x,b+n-x)}{B(a,b)}, \qquad x=0,\ldots,n.

Here $B(u,v)$ is the beta function. This probability accounts for uncertainty in $p$: it is the chance of seeing $x$ events before observing the trial data, averaging over the beta prior. Apply the expression separately to the two arms, using each arm’s sample size and prior parameters.

Theorem: Under the independent-arm model, define $S(x_T,x_C)=1$ if the prespecified posterior decision rule declares success for counts $(x_T,x_C)$, and $S(x_T,x_C)=0$ otherwise. Then $$ \mathcal{A} =\sum_{x_T=0}^{n_T}\sum_{x_C=0}^{n_C} S(x_T,x_C) \left[\binom{n_T}{x_T}\frac{B(a_T+x_T,b_T+n_T-x_T)}{B(a_T,b_T)}\right] \left[\binom{n_C}{x_C}\frac{B(a_C+x_C,b_C+n_C-x_C)}{B(a_C,b_C)}\right]. $$ This is the assurance: the prior-predictive probability that the trial’s observed counts lead to a success decision.

The identity follows by conditioning on the two event counts. For each possible pair, the chance of that pair under the prior-predictive model is the product of its two arm-specific probabilities. The success indicator retains that probability if the pair meets the rule and contributes zero otherwise. Summing over all pairs therefore gives the probability of success.

Enumerate possible counts.
List $x_T=0,\ldots,n_T$ and $x_C=0,\ldots,n_C$.

Evaluate the posterior decision.
For each pair, update both beta distributions and apply the stated success rule.

Weight each decision.
Multiply the success indicator by the two beta-binomial predictive probabilities.

Sum and verify.
Add the retained weights and check that each arm’s predictive probabilities sum to one.

Worked Calculations

Worked Example: One Participant per Arm

Suppose $n_T=n_C=1$, and both event probabilities have independent $\operatorname{Beta}(1,1)$ priors. The design declares success if the posterior probability that $p_T>p_C$ exceeds $0.8$. Calculate assurance.

For a uniform prior and one observation, each arm’s predictive probabilities are $P(X=0)=P(X=1)=1/2$. Thus each of the four count pairs has prior-predictive probability $1/4$.

If $x_T=1$ and $x_C=0$, the posterior distributions are $p_T\mid x_T\sim\operatorname{Beta}(2,1)$ and $p_C\mid x_C\sim\operatorname{Beta}(1,2)$. The probability of superiority is

P(p_T>p_C\mid x_T=1,x_C=0) =\int_0^1 2t\{1-(1-t)^2\}\,dt =\frac{5}{6} \approx 0.8333.

This pair meets the rule. Reversing the counts gives posterior probability $1/6$, which does not meet it. Equal counts give identical posterior distributions and probability $1/2$, also below the strict $0.8$ threshold. Therefore only one of the four pairs is a success:

\mathcal{A}=\frac{1}{4}\times 1=0.25.

The assurance is 25%. It is a prior-averaged probability that this one-participant-per-arm design will meet its posterior decision rule, not a probability that the treatment is effective.

Worked Example: Nonuniform Priors and Unequal Predictive Weights

Let both arms have independent $\operatorname{Beta}(2,1)$ priors and let $n_T=n_C=2$. Declare success if the posterior probability that $p_T>p_C$ is greater than $0.5$. Find the exact assurance.

For either arm, the predictive count probability is

P(X=x)=\binom{2}{x}\frac{B(2+x,1+2-x)}{B(2,1)}.

Since $B(2,1)=1/2$, the three probabilities are

P(X=0)=\frac{B(2,3)}{B(2,1)}=\frac{1}{6},\qquad P(X=1)=2\frac{B(3,2)}{B(2,1)}=\frac{1}{3},\qquad P(X=2)=\frac{B(4,1)}{B(2,1)}=\frac{1}{2}.

They sum to $1/6+1/3+1/2=1$. For a given count $x$, the posterior is $\operatorname{Beta}(2+x,3-x)$. With equal priors and equal sample sizes, a larger treatment count gives a posterior distribution shifted toward larger event probabilities than the control posterior. Thus $x_T>x_C$ gives posterior probability greater than $0.5$; equal counts give $0.5$, and $x_T<x_C$ gives less than $0.5$. The successful pairs are $(1,0)$, $(2,0)$, and $(2,1)$. Their predictive weights yield

\mathcal{A} =\left(\frac{1}{3}\frac{1}{6}\right) +\left(\frac{1}{2}\frac{1}{6}\right) +\left(\frac{1}{2}\frac{1}{3}\right) =\frac{1}{18}+\frac{1}{12}+\frac{1}{6} =\frac{11}{36} \approx 0.3056.

The count pairs are not equally likely: the $\operatorname{Beta}(2,1)$ prior makes larger event counts more predictive. Replacing the beta-binomial weights with equal weights would therefore give the wrong assurance.

Worked Example: A More Demanding Posterior Threshold

Suppose both arms have independent $\operatorname{Beta}(1,1)$ priors, with two participants per arm. Success requires $P(p_T>p_C\mid x_T,x_C)>0.75$. Calculate assurance.

Each arm’s beta-binomial predictive distribution is uniform on $x=0,1,2$, so each of the nine count pairs has weight $1/9$. For counts $(1,0)$, the posteriors are $\operatorname{Beta}(2,2)$ and $\operatorname{Beta}(1,3)$. The control posterior CDF is $1-(1-t)^3$, giving

P(p_T>p_C\mid 1,0) =\int_0^1 6t(1-t)\{1-(1-t)^3\}\,dt =1-6B(2,5)=0.8.

For counts $(2,1)$, the posteriors are $\operatorname{Beta}(3,1)$ and $\operatorname{Beta}(2,2)$. Since the latter has CDF $3t^2-2t^3$,

P(p_T>p_C\mid 2,1) =\int_0^1 3t^2(3t^2-2t^3)\,dt =0.8.

For $(2,0)$, the posteriors are $\operatorname{Beta}(3,1)$ and $\operatorname{Beta}(1,3)$, and the probability is $1-3B(3,4)=0.95$. These three pairs exceed $0.75$. Ties give $0.5$, and reversed count pairs have probabilities below $0.5$, so none of them succeeds. Hence

\mathcal{A}=\frac{3}{9}=\frac{1}{3}\approx 0.3333.

The strict threshold matters: the $0.8$ posterior probabilities exceed $0.75$. If the success rule instead used a threshold of $0.8$ with a strict “greater than” sign, those two pairs would not qualify.

Diagnostics, Alternatives, and Limits

Exact enumeration is most useful when the outcome space is manageable and the sampling model matches the design. For two fixed sample sizes, the number of pairs is $(n_T+1)(n_C+1)$. The calculation is straightforward, but its correctness depends on both the predictive weights and the success rule being implemented accurately.

Check normalization. Each arm’s beta-binomial probabilities must be nonnegative and sum to one. A failure usually indicates an incorrect beta-function argument or omitted binomial coefficient.
Check the decision boundary. State whether the criterion is $>$ or $\geq$, and apply it consistently. Posterior probabilities exactly at the threshold can change the result.
Check the model structure. Multiplying the arm-specific predictive probabilities assumes independent priors and conditionally independent arm outcomes. Correlated rates require a joint predictive distribution instead.
Check the design. Fixed sample sizes and a common event probability within each arm are built into this calculation. Adaptive stopping, clustering, missing outcomes, or changing event probabilities require a model that represents those features.
Check computation. For large counts, evaluate beta-function ratios with stable log-gamma calculations rather than forming very small or large intermediate values directly.

The nested-integral expression for assurance averages conditional success probabilities over the event rates. Finite enumeration is an equivalent route here: integrate the likelihood for each count pair to obtain its beta-binomial predictive weight, then apply the success rule. Numerical integration can be useful for evaluating posterior probabilities involving a clinical margin, while simulation can handle more complicated designs. Those approaches may be necessary when the outcome space or model is more complex, but they introduce numerical approximation or Monte Carlo error. Exact enumeration avoids Monte Carlo error under the stated model; “exact” does not remove uncertainty about whether the model or priors are appropriate.

A common mistake is to average posterior success probabilities over the predictive count distribution. For a deterministic rule, the trial either meets the rule for a given count pair or does not: its contribution is the indicator $S(x_T,x_C)$, not the posterior probability used to define that rule. The posterior probability is compared with its threshold first; the resulting decision is then weighted by the predictive probability. Confusing these two stages can calculate a different operating characteristic.

Key takeaway: For a fixed-size two-arm binary trial with independent beta priors, exact assurance is a finite sum of beta-binomial predictive weights over the count pairs that satisfy the prespecified posterior success rule.

Check Your Understanding

Answer each question, then compare with the solutions below.

With independent $\operatorname{Beta}(1,1)$ priors and $n=2$ in one arm, what is the prior-predictive probability of observing one event?
In the second worked example, why is the predictive probability of two events not the same as the probability of zero events?
If the success rule is $P(p_T>p_C\mid\text{data})\geq 0.5$, what issue arises for count pairs giving posterior probability exactly $0.5$?
What assumption justifies multiplying the two arms’ beta-binomial predictive probabilities?

Solutions. (1) The beta-binomial distribution with a uniform prior is uniform over counts $0,1,2$, so the probability is $1/3$. (2) Under $\operatorname{Beta}(2,1)$, the prior favors larger event probabilities; the predictive probabilities are $P(X=0)=1/6$ and $P(X=2)=1/2$. (3) Tied posterior probabilities now meet the rule, so the success set and assurance can differ from those under a strict $>0.5$ rule. (4) Independence of the arm-specific priors, together with conditional independence of outcomes across arms, makes the prior-predictive counts independent.

Exact Assurance for Beta-Binomial Trial Designs

What You'll Learn