Coherence of Sequential Bayesian Decisions

Why Coherence Matters in Sequential Decisions

A multi-arm trial may offer several opportunities to stop an arm, continue collecting data, or make a final treatment recommendation. Each decision uses the information available at that time. The central question is not whether every interim rule must give the same answer as the final rule; new data can rationally change a decision. Rather, the question is whether the whole sequence of decisions is consistent with one probability model, one account of the available actions, and one utility framework.

For example, a rule might say to stop an arm for futility when its posterior probability of benefit falls below a specified boundary. That can be a sensible rule if stopping means abandoning the arm and its expected future value is too low. But if the same model and utilities imply that adopting the arm immediately has positive expected utility, a rule that instead stops it without adoption may need justification. A boundary is not automatically coherent just because it is prespecified.

Definition: A sequential Bayesian decision rule is coherent when its posterior beliefs are updated from a common prior and a correctly specified joint model for the observed data, and its choices among stopping, continuing, and terminal actions are consistent with the specified utilities and available actions. In an optimal sequential policy, each choice maximizes expected utility conditional on the information then available.

Write $\mathcal{D}_t$ for all observed data and design information available by stage $t$, and let $\theta_j$ denote the unknown response parameter for arm $j$. The posterior distribution $P(\theta_1,\ldots,\theta_J\mid\mathcal{D}_t)$ summarizes current uncertainty. A stopping rule chooses an action using information in $\mathcal{D}_t$; it should not silently replace the prior, likelihood, utility, or action set used to interpret that posterior.

Posterior Updating Across Stages

Sequential updating means conditioning the current posterior on the additional data as they arrive. If the prior is $\pi(\theta)$, the stage-$t$ likelihood is $L_t(\theta)$, and the new data are $Y_{t+1}$, then, up to a normalizing constant,

\pi(\theta\mid\mathcal{D}_t,Y_{t+1}) \propto \pi(\theta\mid\mathcal{D}_t)\, p(Y_{t+1}\mid\theta,\mathcal{D}_t).

The likelihood for the new data must reflect the trial’s data-generating process. In a multi-arm study, that includes the arm assignments and any dependence induced by shared controls when those features affect the outcome likelihood. Adaptive allocation does not by itself invalidate Bayesian updating: the allocation process and outcomes must simply be represented correctly. If the stopping or selection mechanism depends on additional information not included in the model, the posterior may no longer represent the intended conditional beliefs.

Theorem: Under a coherent joint model and prior, if $Z$ is any integrable function of the unknown parameters, then its posterior expectation satisfies $$ E\!\left[E\{Z\mid\mathcal{D}_{t+1}\}\mid\mathcal{D}_t\right] = E\{Z\mid\mathcal{D}_t\}. $$ In particular, the posterior probability of an event $A$ has this property: $$ E\!\left[P(A\mid\mathcal{D}_{t+1})\mid\mathcal{D}_t\right] = P(A\mid\mathcal{D}_t). $$

This is a conditional-expectation identity, often described by saying that posterior expectations and posterior event probabilities are martingales as information accumulates. It does not say that the posterior probability will stay constant after new data arrive. It says that, before seeing the next data, the average of the possible updated probabilities—averaged over the model’s predictive distribution—is the current probability.

Justification: Why Posterior Probabilities Average Back to Their Current Value

Let $I_A$ equal 1 when event $A$ is true and 0 otherwise. The current posterior probability is $P(A\mid\mathcal{D}_t)=E(I_A\mid\mathcal{D}_t)$. By the law of iterated expectations,

\begin{aligned} E\!\left[P(A\mid\mathcal{D}_{t+1})\mid\mathcal{D}_t\right] &=E\!\left[E(I_A\mid\mathcal{D}_{t+1})\mid\mathcal{D}_t\right]\\ &=E(I_A\mid\mathcal{D}_t)\\ &=P(A\mid\mathcal{D}_t). \end{aligned}

The key assumption is that $\mathcal{D}_t$ is contained in the information available at stage $t+1$, and that both posterior calculations use the same coherent model. The identity concerns beliefs averaged over future data; it does not guarantee that an arbitrary stopping boundary is optimal, or that stopping has a particular frequentist error rate.

Worked Example: A Posterior Mean Before and After One More Observation

Suppose a hypothetical arm has a binary response probability $\theta$. Start with $\theta\sim\operatorname{Beta}(1,1)$. At an interim analysis, 7 of 10 participants have responded. Assume independent Bernoulli outcomes conditional on $\theta$, with complete outcome data.

The posterior is $\operatorname{Beta}(1+7,1+3)=\operatorname{Beta}(8,4)$, with mean $8/(8+4)=2/3$. The predictive probability that the next participant responds is therefore $2/3$.

If the next outcome is a response, the posterior becomes $\operatorname{Beta}(9,4)$, with mean $9/13$. If it is not a response, the posterior becomes $\operatorname{Beta}(8,5)$, with mean $8/13$. Averaging these possible updated means over the predictive outcomes gives

\frac{2}{3}\frac{9}{13}+\frac{1}{3}\frac{8}{13} =\frac{18+8}{39} =\frac{2}{3}.

Thus the updated mean changes with the next outcome, but its predictive average equals the current mean. This calculation relies on the stated beta-binomial model and on the next participant’s outcome following the same response process.

Stopping and Continuing Must Use the Same Decision Problem

Posterior probabilities are inputs to decisions, not decisions by themselves. To assess coherence, specify the actions available now and later, their utilities, and the cost of collecting more data. Let $U(a,\theta)$ be the utility of terminal action $a$ when the unknown state is $\theta$. The best expected utility from stopping now is

V_{\mathrm{stop}}(\mathcal{D}_t) = \max_{a\in\mathcal{A}_t} E\{U(a,\theta)\mid\mathcal{D}_t\},

where $\mathcal{A}_t$ is the set of terminal actions available at that point. If the trial can continue, its value is the expected best value after future data, less the cost of acquiring them. For a single possible next observation $Y_{t+1}$ with cost $c$, this is

V_{\mathrm{continue}}(\mathcal{D}_t) = -c+ E\!\left[ V_{\mathrm{stop}}(\mathcal{D}_t,Y_{t+1}) \mid\mathcal{D}_t \right].

When continuation itself can be followed by further stages, the same comparison is made recursively using the value of the best future policy. An optimal policy stops when the best stopping value is at least the continuation value; it continues when continuation has greater expected value. Prespecified boundaries can approximate this comparison, but the boundary’s rationale must be compatible with the model, utilities, and actions the protocol actually allows.

Worked Example: Paying for One More Signal

Consider a hypothetical arm with an unknown binary state $E$: it is either effective or ineffective. At an interim analysis, the posterior probability of effectiveness is $q=0.60$. The terminal actions are launch or do not launch. Launch gives utility $+1$ if the arm is effective and $-1$ if it is ineffective; not launching gives utility $0$. Assume these utilities and the state definition remain unchanged at the next decision.

Launching now has expected utility $0.60(1)+0.40(-1)=0.20$, while not launching has utility 0. Thus $V_{\mathrm{stop}}=0.20$.

The team can instead pay a cost of $0.10$ utility units for one more signal. Suppose the signal is positive with probability 0.80 if the arm is effective and 0.20 if it is ineffective. The predictive probability of a positive signal is $0.60(0.80)+0.40(0.20)=0.56$. After a positive signal, Bayes’ rule gives $P(E\mid+)=0.60(0.80)/0.56=6/7$. Launching then has expected utility $6/7-1/7=5/7$, which exceeds 0. After a negative signal, the predictive probability is $0.44$, and $P(E\mid-)=0.60(0.20)/0.44=3/11$. Launching then has expected utility $3/11-8/11=-5/11$, so not launching, with utility 0, is preferred.

The expected value after the signal, before its cost, is $0.56(5/7)+0.44(0)=0.40$. Therefore $V_{\mathrm{continue}}=0.40-0.10=0.30$, greater than the stop-now value 0.20. Under these assumptions, continuing is optimal. This is a coherent comparison because the same state, likelihood, terminal utilities, and available actions are used before and after the signal.

Arm Selection and Consistent Stopping Actions

In a multi-arm trial, coherence also concerns the consequences of selection. The posterior probability that an arm is best, the expected utility of selecting it, and the utility of stopping it are different quantities. A coherent rule must use the quantity corresponding to its stated objective. If an arm is dropped, its future data will not be collected; the value of that lost option should be represented when deciding whether to continue it. If selection changes which terminal actions are available, that change belongs in the decision problem rather than being treated as a harmless relabeling.

Worked Example: A Futility Boundary That Conflicts With the Terminal Action

Consider two hypothetical arms, A and B. For each arm, let $E_j$ indicate whether it meets the trial’s definition of effective. Before observing a signal, assume $P(E_j)=0.50$. A single positive signal is observed for each arm. Suppose the signal probabilities are $P(+\mid E_A)=0.62$, $P(+\mid E_A^c)=0.38$, $P(+\mid E_B)=0.48$, and $P(+\mid E_B^c)=0.52$. Assume these are the full likelihoods for the signals and the stated priors are used.

Bayes’ rule gives $P(E_A\mid+)=0.62$, because the prior odds are 1 and the signal likelihood ratio is $0.62/0.38$. Similarly, $P(E_B\mid+)=0.48$, since the likelihood ratio for B is $0.48/0.52$. Under the launch utility from the previous example, arm A has expected launch utility $0.62-0.38=0.24$, while arm B has expected launch utility $0.48-0.52=-0.04$. The terminal action rule therefore selects A for launch and does not launch B.

Now suppose an interim rule drops every arm with posterior effectiveness probability below 0.65, and dropping means it cannot be launched. It drops A at probability 0.62, even though the specified terminal action rule assigns A positive expected launch utility of 0.24. If stopping at this point offers only the choice of dropping the arm or launching it, dropping A is not utility-maximizing under the stated terminal utilities; the rule conflicts with those utilities. A higher futility boundary could still be justified by a model in which continuing has option costs or different consequences, but those must be specified. The numerical boundary alone does not establish coherence.

Diagnostics, Alternatives, and Limitations

A useful review of a sequential Bayesian policy traces a possible data path from the prior through each interim update to the final action. At every stage, verify that the posterior conditions on the data actually observed, that the likelihood includes relevant features of the design, and that the rule’s stopping action matches the action assumed in its utility calculation. Also check whether an arm can be selected, dropped, or reinstated: if the protocol does not permit an action, it cannot be counted as an available option in the decision calculation.

Inconsistent posterior calculations. Replacing the prior or likelihood at an interim look without explicitly defining a new model can make later beliefs incomparable with earlier ones.
Unmodeled adaptation. If allocation or selection depends on data, check whether that dependence affects the outcome model or the information available at the decision. Do not assume that adaptation is irrelevant in every design.
Boundary and action mismatch. A posterior probability boundary may not correspond to the utilities of launching, continuing, or stopping. State what happens when the boundary is crossed.
Ignoring continuation value. Comparing a posterior probability with a cutoff alone does not account for the value of future information or the cost of obtaining it.
Monte Carlo uncertainty near a boundary. If a posterior probability is estimated by simulation and is close to a decision threshold, numerical error can change the selected action. Check the simulation error and, when needed, improve the calculation before applying the rule.

A fixed-sample Bayesian decision rule makes its terminal choice after a planned data set and does not need to compare that choice with another interim opportunity to learn. A sequential optimal-stopping policy makes that comparison at each stage, usually accounting for the predictive distribution of future data and their cost. A prespecified threshold policy is a third practical approach: it can be easier to communicate and implement, but it is not automatically the optimal policy for the stated utilities. These approaches answer related but distinct questions.

Coherence also has limits. A mathematically coherent policy can still be sensitive to its prior, likelihood, utility scale, or definition of benefit; it can also have undesirable operating characteristics for particular data-generating scenarios. Coherence is a consistency requirement, not proof that the model is realistic, the utility elicitation is appropriate, or the policy meets a separate design objective. Those features require their own scrutiny.

Key takeaway: Bayesian updating keeps beliefs coherent when each stage uses the same prior and joint data model. Sequential actions are coherent when stopping and continuing are evaluated against the same utilities and available actions, with the value and cost of future information handled explicitly.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

At an interim analysis, a posterior probability of an event is 0.40. What does the martingale property say about the average posterior probability after the next data update?
A team uses one prior at the interim analysis and a different prior at the final analysis, without changing its stated model. What coherence concern arises?
Under the signal model in the worked example, what is the probability of a negative signal, and what is the posterior probability of effectiveness after a negative signal?
Why does a fixed posterior-probability boundary not by itself show that a sequential rule is optimal?

Solutions. (1) Averaged over the predictive distribution of the next data, the updated posterior probability is 0.40. Individual outcomes may move it above or below 0.40. (2) The posterior beliefs no longer follow sequential conditioning from one common prior and model; the change must be explicitly justified as a changed decision problem or model. (3) The negative-signal probability is $0.60(0.20)+0.40(0.80)=0.44$. The posterior probability is $0.60(0.20)/0.44=3/11$, approximately 0.273. (4) A boundary does not, on its own, compare the expected utility of stopping with the expected utility of continuing, including the value and cost of future data.