From a Terminal Rule to a Predictive Probability
In a single-arm phase 2 trial, the terminal analysis may declare success only if the posterior probability that the response rate exceeds a clinically relevant benchmark is sufficiently high. Before all participants have outcomes, a different question is useful: given the data observed so far, what is the probability that the eventual data will meet that terminal rule? Answering it requires looking ahead across all possible future response counts, not simply evaluating the current posterior.
The calculation has two layers. First, the terminal rule classifies each possible final response count as success or not. Second, the Bayesian predictive distribution assigns a probability to each possible future count. Adding the probabilities for the counts classified as successful gives the predictive probability of terminal success.
For the examples, let \(p\) denote the response probability and use a Beta prior, \(p\sim\text{Beta}(a,b)\), with \(a,b>0\). Assume each participant has a binary response, and that outcomes are conditionally independent with the same response probability \(p\). After \(x\) responses among \(n\) participants, the posterior is \(\text{Beta}(A,B)\), where \(A=a+x\) and \(B=b+n-x\). These assumptions mean that the future number of responses is fixed at \(m\), and that no change in response probability over time or across participants is built into the model.
Enumerating Future Counts
Conditional on \(p\), \(Y\) has a Binomial distribution with sample size \(m\) and response probability \(p\). The predictive distribution averages that Binomial probability over the current posterior distribution of \(p\). This is the beta-binomial predictive distribution, derived in the earlier material on the Beta-Binomial Predictive Distribution.
Here \(B(u,v)\) is the beta function. The formula includes both uncertainty about \(p\) and the variability of future binary outcomes conditional on \(p\). To convert it into a terminal-success probability, define \(D(y)=1\) if the final data \((x+y,n+m)\) meet the terminal criterion, and \(D(y)=0\) otherwise. Then
Equivalently, write \(S=\{y\in\{0,\ldots,m\}:D(y)=1\}\) for the set of successful future counts. The calculation is \(\sum_{y\in S}P(Y=y\mid x,n)\). The important practical step is to construct \(S\) from the terminal rule before summing. A count can have positive predictive probability but contribute nothing to this sum if its final posterior does not pass the rule.
Applying a Posterior Success Criterion
A common terminal rule declares success when the posterior probability that the response rate exceeds a benchmark \(p_0\) is at least \(\gamma\), where \(0<\gamma<1\). For a possible future count \(y\), the final posterior is \(\text{Beta}(A+y,B+m-y)\). Thus the count is classified as successful precisely when
The terminal decision is therefore recalculated for every possible \(y\), using the posterior that would result if that count occurred. It is not enough to check the current posterior or to compare the future count with an informal idea of “many responses.” When the terminal rule is monotone in the number of responses, the successful counts will form an upper tail, but the calculation should establish that boundary from the stated rule.
For integer posterior parameters, a useful way to evaluate the posterior tail in these examples is \(P(p>p_0)=P(K\le A-1)\), where \(p\sim\text{Beta}(A,B)\) and \(K\sim\text{Binomial}(A+B-1,p_0)\). This identity lets us check terminal decisions using finite binomial probabilities. It does not replace the predictive calculation: the identity decides whether a final count passes the rule, while the beta-binomial distribution supplies the probability of that count.
Worked Example: Only One Future Count Can Succeed
Suppose the prior is \(p\sim\text{Beta}(1,1)\), the benchmark is \(p_0=0.5\), and terminal success requires \(P(p>0.5\mid\text{all data})\ge 0.90\). There are 10 participants at the terminal analysis. Currently, 4 responses have been observed among 6 participants, so \(m=4\) outcomes remain. Calculate the predictive probability of terminal success.
Find the successful terminal counts. With \(k\) responses among all 10 participants, the posterior is \(\text{Beta}(k+1,11-k)\). For \(k=8\), the identity above gives \(P(p>0.5)=P(K\le 8)\) for \(K\sim\text{Binomial}(11,0.5)\). Therefore,
For \(k=7\), the corresponding posterior probability is
The posterior tail increases with the number of responses at this fixed terminal sample size. Thus the rule passes for \(k\ge8\), but not for \(k\le7\). Since \(x=4\), terminal success requires \(4+Y\ge8\), or \(Y=4\). Each of the four remaining outcomes must be a response.
Calculate the probability of that count. The current posterior is \(\text{Beta}(5,3)\). The probability of four future responses is the posterior predictive probability of all four outcomes being responses:
Therefore, conditional on the observed data and this model, the probability of meeting the terminal posterior success rule is about 0.2121, or 21.21%. This is a predictive probability of a future decision, not the current posterior probability that \(p>0.5\).
Summing Several Successful Counts
Often more than one future count satisfies the terminal criterion. Each qualifying count must be included, with its own predictive probability. The following example shows the enumeration explicitly.
Worked Example: Adding the Probabilities of Multiple Counts
Use the same prior, benchmark, and terminal rule as in the previous example: \(p\sim\text{Beta}(1,1)\), \(p_0=0.5\), and terminal success requires a posterior tail probability of at least 0.90. There are 10 participants in total. Five participants have been observed, with 5 responses, and the remaining \(m=5\) outcomes are yet to be observed.
The terminal rule passes when the final number of responses is at least 8, as established in the previous calculation. Since \(x=5\), the successful future counts are \(y=3,4,5\).
The current posterior is \(\text{Beta}(6,1)\). Applying the beta-binomial formula for \(m=5\) gives the following probabilities. For transparency, the successive unnormalized weights are proportional to \(1,6,21,56,126,252\); they sum to \(462\), so each weight is divided by \(462\).
| Future responses \(y\) | Final responses \(5+y\) | Terminal decision | Predictive probability |
|---|---|---|---|
| 0 | 5 | Not success | 1/462 |
| 1 | 6 | Not success | 6/462 |
| 2 | 7 | Not success | 21/462 |
| 3 | 8 | Success | 56/462 |
| 4 | 9 | Success | 126/462 |
| 5 | 10 | Success | 252/462 |
The predictive probabilities sum to \(462/462=1\), a useful arithmetic check. Summing only the successful rows gives
The probability is about 93.94%. This value is high because all three upper future counts qualify and, under the current posterior, those counts carry most of the predictive mass. The terminal rule is still applied to each final count; a high predictive probability does not alter the rule itself.
Checking Every Possible Final Decision
A compact enumeration is useful when the successful set has few values, but it is still important to check the full range \(0,\ldots,m\). This reveals whether the qualifying counts are contiguous, verifies that impossible counts have not been included, and makes clear which posterior decision each count produces.
Worked Example: A Two-Outcome Look Ahead
Suppose the prior is again \(\text{Beta}(1,1)\), terminal success requires \(P(p>0.5\mid\text{all data})\ge0.90\), and the terminal sample size is 10. At an interim point there are 6 responses among 8 participants, leaving \(m=2\) participants. Calculate the predictive probability of terminal success.
The current posterior is \(\text{Beta}(7,3)\). Terminal success requires at least 8 total responses, so only \(y=2\) qualifies: \(y=0\) gives 6 total responses and \(y=1\) gives 7. For \(m=2\), the beta-binomial denominator is \((A+B)(A+B+1)=10\cdot11=110\). The three future-count probabilities are
They sum to \(110/110=1\). The first two outcomes do not meet the rule: six or seven total responses have posterior tail probabilities below 0.90. With \(y=2\), there are eight total responses, and the terminal posterior tail is \(1981/2048\approx0.9673\), which passes. Therefore,
The chance of terminal success is about 50.91%, even though the current data already include six responses. The remaining sample size is small, and both future participants must respond for the terminal rule to pass.
Interpretation, Diagnostics, and Limitations
The predictive probability answers a forward-looking, model-based question: given the data observed now, what is the probability that the final data will satisfy the specified terminal rule? It is not the posterior probability that the treatment works, the probability that the response rate will exceed a benchmark, or a guarantee about the trial’s eventual outcome. The terminal posterior probability is recalculated for each possible final data set; the predictive probability averages the resulting pass-or-fail decisions over future data.
A closely related quantity is the current posterior probability \(P(p>p_0\mid x,n)\). That quantity assesses the current posterior evidence against the benchmark. It is not the probability of terminal success, because the latter also depends on the number of outcomes remaining and on how different future counts would affect the terminal posterior. Likewise, replacing the posterior predictive distribution with a Binomial distribution evaluated at a single estimate of \(p\) is not the same calculation: it omits the model’s posterior uncertainty about \(p\).
Several checks help identify mistakes:
- Check the final sample size. Every candidate count must use \(n+m\), not \(n\), in the terminal posterior.
- Check the count range. Enumerate integer values from 0 through \(m\); do not include counts greater than the number of remaining outcomes.
- Check the decision boundary. Apply the stated posterior threshold to final data for each count. Do not treat a count as successful merely because it seems large.
- Check the predictive distribution. The beta-binomial probabilities over all \(m+1\) counts must sum to 1, up to rounding.
- Check the conditioning. The result is conditional on the observed data, the prior, the likelihood assumptions, and the specified terminal rule.
The calculation is only as relevant as its assumptions. A common response probability and conditional independence may be poor descriptions if response rates vary substantially over time or across important subgroups. A Beta prior and a binary endpoint may also fail to represent the clinical problem adequately. In those cases, the predictive distribution must match the model actually used; the simple beta-binomial formula should not be applied automatically. The result also depends on the chosen prior, benchmark, posterior threshold, and remaining sample size, so these inputs should be stated alongside the probability.
The most common mistake is to report the current posterior tail probability as though it were the probability of eventual terminal success. For example, a current value of \(P(p>p_0\mid x,n)=0.90\) does not imply a 90% chance of passing the terminal rule. The correction is to enumerate possible future counts, determine the terminal decision for each, and sum their posterior predictive probabilities.
Check Your Understanding
Use the terminal rule and data stated in each question. Solutions are included below so you can check both the count classification and the probability calculation.
- Suppose terminal success is defined as at least 5 responses among 7 participants. There are currently 3 responses among 5 participants, and 2 outcomes remain. Which future counts qualify? Under a \(\text{Beta}(1,1)\) prior, calculate the predictive probability of terminal success.
- In the setting of the first worked example, why does the predictive probability of terminal success differ from the current posterior probability \(P(p>0.5\mid x=4,n=6)\)?
- If the terminal rule requires at least 8 responses among 10 participants, there are currently 6 responses among 8 participants. What is the successful set of future counts when \(m=2\)?
- Name two model assumptions needed to use the beta-binomial predictive distribution in this tutorial.
Solutions. (1) At least 5 total responses means \(3+Y\ge5\), so only \(Y=2\) qualifies. The current posterior is \(\text{Beta}(4,3)\). For \(m=2\), the probabilities for \(Y=0,1,2\) are \(12/56,24/56,20/56\), respectively: the denominator is \(7\cdot8=56\), and the numerators are \(3\cdot4\), \(2\cdot4\cdot3\), and \(4\cdot5\). Thus the predictive probability is \(20/56\approx0.3571\). (2) The current posterior tail evaluates the present evidence about \(p\); the predictive probability averages the terminal pass-or-fail decision across possible future outcomes. The future sample size and terminal rule therefore matter to the predictive quantity. (3) Only \(Y=2\) qualifies, because \(6+2=8\), while \(Y=0\) and \(Y=1\) give fewer than 8 total responses. (4) For example, the model assumes binary outcomes that are conditionally independent given \(p\), a common response probability across participants, and a Beta prior for \(p\).