Prior Sensitivity of Interim Stopping Decisions

Why Examine Prior Sensitivity?

An interim stopping rule can be mathematically precise and still depend on an uncertain modeling choice: the prior distribution for the response probability. If two clinically defensible priors produce different predictive probabilities, the same interim data may lead to different recommendations. Prior sensitivity analysis makes that dependence visible. It asks whether plausible alternatives change the probability enough to cross a prespecified efficacy or futility boundary.

The comparison is conditional on the same interim dataset, terminal success definition, remaining sample size, and decision rule. Changing any of those along with the prior would make it difficult to identify what caused an action to change. The aim is not to choose whichever prior gives the preferred result. It is to assess whether a decision is robust to reasonable prior uncertainty and to report any instability clearly.

Model and Decision Rule

Consider a hypothetical single-arm trial with a binary response endpoint. Let \(p\) be the response probability, \(n\) the number of patients with observed outcomes at an interim look, and \(x\) the number of responses among them. Suppose \(m\) additional patients remain, and let \(Y\) be their eventual number of responses. The terminal trial is successful if at least \(K\) responses are observed among all \(n+m\) patients. Thus, success requires \(Y\geq K-x\).

Assume outcomes are independent Bernoulli observations conditional on a common response probability \(p\), and place a beta prior on \(p\). This model assumes the response definition is consistent, outcomes are observed as specified, and the response probability does not change systematically across patients or time. If these assumptions are not credible, changing the prior alone will not repair the model.

A beta prior has mean \(a/(a+b)\); \(a+b\) is often described as its prior effective sample size, although the prior is a distribution rather than literal observed data. Alternatives can reflect reasonable uncertainty about the prior mean, its concentration, or both. The alternatives should be scientifically defensible and chosen or documented without selecting them to produce a desired interim action.

For clarity, use a two-boundary rule throughout the examples: declare early efficacy if the predictive probability is at least \(c_E\); declare futility if it is at most \(c_F\); otherwise continue. The cutoffs satisfy \(c_F<c_E\). The equality conventions are part of the rule and should be stated in an analysis plan. If the rule includes only a futility boundary, a probability above that boundary establishes only that futility is not triggered; it does not, by itself, establish continuation if a separate efficacy rule might apply.

Predictive Probability Under Each Prior

Write \(A=a+x\) and \(B=b+n-x\) for the interim posterior parameters. The predictive distribution of \(Y\), the response count among the \(m\) remaining patients, is beta-binomial:

where \(\mathrm{B}(u,v)=\int_0^1 p^{u-1}(1-p)^{v-1}\,dp\) is the beta function. Therefore, for \(0\leq K-x\leq m\), the predictive probability of success is

If \(K-x\leq 0\), terminal success is already assured by the observed responses and the predictive probability is 1. If \(K-x>m\), even the maximum possible future response count cannot reach the target, so the predictive probability is 0. These boundary cases should be checked before summing.

A Useful Special Case

When success requires every remaining patient to respond, \(K-x=m\), the event is \(Y=m\). Its predictive probability has a simple form:

To justify the last equality, use \(\mathrm{B}(u,v)=\Gamma(u)\Gamma(v)/\Gamma(u+v)\) and \(\Gamma(z+m)/\Gamma(z)=z(z+1)\cdots(z+m-1)\). For \(m=2\), this reduces to \(A(A+1)/[(A+B)(A+B+1)]\). It is the posterior expectation of \(p^2\), not the square of the posterior mean: future outcomes share uncertainty about \(p\), which is why integrating over the posterior matters.

When a Prior Change Triggers Efficacy

When Probabilities Differ but the Decision Does Not

How to Assess and Report Sensitivity

A disciplined comparison holds the data, terminal endpoint, remaining sample size, and cutoffs fixed. For each prior, calculate the posterior, the predictive probability, and the action. A compact table makes it harder to confuse a numerical change with a decision change.

One useful summary is the range of predictive probabilities across the specified priors, alongside the action under each. A narrow range is not automatically reassuring if it straddles a cutoff; a wider range may be decision-robust if all values remain on the same side of both cutoffs. Distance from the nearest boundary is therefore informative, but it does not replace the actual action classification.

Diagnostics, Alternatives, and Limitations

Check that each prior has the intended mean and concentration, that the posterior parameters add the observed responses and nonresponses correctly, and that the predictive event corresponds exactly to terminal success. For example, if \(K-x\) responses are needed from \(m\) patients, summing from \(K-x\) through \(m\) includes every successful future count and excludes every unsuccessful one. Exact enumeration or simulation can verify a calculation, but simulation error should not be mistaken for prior sensitivity; the exact beta-binomial sum is available under this model.

A simple alternative is to report the posterior mean response probability under each prior. That can describe how beliefs about \(p\) differ, but it is not generally the predictive probability of terminal success. The latter depends on the number of future patients and the terminal response threshold, and it accounts for uncertainty in \(p\). Another alternative is to report only the action. That hides how close a probability lies to a cutoff and can conceal a fragile decision.

Prior sensitivity analysis is conditional on the likelihood and decision rule. It does not test whether a common response probability is appropriate, account for unobserved or pending outcomes, or guarantee that the chosen alternatives span all scientifically reasonable beliefs. A decision that is stable across a limited set of priors should be described as stable across that set, not as universally insensitive to prior choice.

Common mistake: “The predictive probability is above the futility cutoff, so the decision is continuation.” This conclusion is incomplete if an efficacy boundary could also apply. The correction is to evaluate every part of the prespecified rule: check efficacy first or follow the explicitly stated ordering, check futility, and assign continuation only when neither stopping condition is met. Also report the cutoffs, so a reader can verify the classification.