Distinguishing Bayesian Power and Assurance

Two Probabilities for Two Different Questions

A trial’s chance of meeting a Bayesian success criterion depends on what is treated as uncertain. One question fixes a particular value of the parameter that truly governs the trial and asks how often repeated trials would meet the criterion. Another question allows that parameter to vary according to a distribution representing uncertainty about plausible true values, and averages the success probability across those values. These are related but distinct operating characteristics: conditional Bayesian power and assurance.

The distinction matters for design decisions. A trial could have a high chance of success if the true treatment effect is a particular favorable value, but a lower assurance if many values considered plausible are less favorable. Reporting only one of these probabilities can therefore obscure the assumptions behind a sample-size justification.

Definition: Let $Y$ denote the trial data, $\theta$ the parameter governing the data-generating process, and $S(Y)$ the event that the trial meets its prespecified Bayesian success rule. The conditional Bayesian power at a fixed true parameter value $\theta$ is $$ \operatorname{CP}(\theta;n)=P_{\theta,n}\{S(Y)\}, $$ where $n$ denotes the design, including its sample size, and $P_{\theta,n}$ is the sampling distribution of the data when the true parameter is $\theta$. Given a design prior $\pi_D(\theta)$, the assurance is $$ A(n)=\int \operatorname{CP}(\theta;n)\,\pi_D(\theta)\,d\theta. $$ For a discrete design prior, the integral is a weighted sum.

The success event is defined by the analysis plan, not by the definition of power or assurance. For example, a trial might declare success when the posterior probability that a treatment effect exceeds a clinically meaningful threshold is at least $\gamma$. The posterior in that rule is calculated using the analysis prior, denoted $\pi_A$. The design prior $\pi_D$, by contrast, weights possible true parameter values when calculating assurance. They may be the same distribution, but they need not be; their roles must be specified separately.

What Is Fixed and What Is Averaged?

For conditional Bayesian power, the true parameter value is fixed for the calculation. Randomness comes from the data that would be observed if the trial were repeated under that value and design. Conditional Bayesian power is therefore a function of $\theta$: the same design can have different power at different true values.

Assurance averages that function over the design prior. It includes both the sampling variability represented by conditional power and the design-prior uncertainty about which true parameter value will apply. It is a probability under the joint predictive distribution induced by the sampling model and design prior. It is not the probability, after observing the trial data, that the treatment works; that would be a posterior probability calculated from the observed data and analysis prior.

Quantity	What is held fixed?	Where does uncertainty enter?	Question answered
Conditional Bayesian power	A specified true parameter value $\theta$	Repeated sampling of trial data under that value	At this true value, how often would the design meet its success rule?
Assurance	The trial design and design prior	Sampling variability and averaging over $\theta\sim\pi_D$	Before knowing the true value, what is the design-prior-averaged chance of meeting the rule?
Posterior probability of benefit	The observed data and analysis model	Posterior uncertainty about $\theta$ after observing data	Given these observed data, how much posterior probability supports benefit?

A Simple Decision Rule and Its Consequences

Consider a Bayesian trial with a parameter $\theta$ representing the response probability in the treatment arm. Suppose the prespecified success rule is that the posterior probability of a response probability above $0.5$ must be at least $0.7$. For a one-observation illustration, let $Y=1$ indicate a response and $Y=0$ indicate no response. Assume $Y\mid\theta\sim\operatorname{Bernoulli}(\theta)$, and use a uniform analysis prior, $\theta\sim\operatorname{Beta}(1,1)$. These assumptions define both the sampling model and the posterior calculation for this example.

If $Y=1$, the posterior is $\operatorname{Beta}(2,1)$, so the posterior probability that $\theta>0.5$ is $1-0.5^2=0.75$. If $Y=0$, the posterior is $\operatorname{Beta}(1,2)$, and that probability is $(1-0.5)^2=0.25$. Thus the rule is met exactly when $Y=1$. The posterior threshold determines which data outcomes count as success; conditional power then asks how likely those outcomes are under a specified true value.

Worked Example: Conditional Power at a Fixed Response Probability

Continue with the one-observation model and success rule above. Suppose the fixed true response probability is $\theta=0.8$. The analysis assumptions have already established that success occurs if and only if $Y=1$. Under the Bernoulli sampling model, the probability of that outcome is $0.8$.

Calculation. Since $S(Y)=\{Y=1\}$, $$ \operatorname{CP}(0.8;1)=P_{\theta=0.8}(Y=1)=0.8. $$

The conditional Bayesian power is therefore $0.800$, or 80.0%. In this hypothetical one-observation design, if the true response probability were 0.8 and the trial were repeated under the same conditions, 80.0% of repetitions would meet the specified posterior success rule in the long run. This is not a statement that there is an 80% posterior probability that $\theta=0.8$.

Assurance as an Average of Conditional Power

To calculate assurance, the design prior assigns weights to possible true values before the trial results are observed. The integral in the definition follows by conditioning on $\theta$: for each possible value, first calculate the chance of trial success under the sampling model, then average those chances using the design-prior weights.

\begin{aligned} A(n) &=P_{\pi_D,n}\{S(Y)\}\\ &=\int P_{\theta,n}\{S(Y)\}\,\pi_D(\theta)\,d\theta\\ &=\int \operatorname{CP}(\theta;n)\,\pi_D(\theta)\,d\theta. \end{aligned}

This is the law of total probability applied to the success event. It requires that the sampling model describe the distribution of $Y$ conditional on $\theta$, and that $\pi_D$ be a proper distribution over the parameter values being averaged. In a simulation study, the same logic applies: generate or evaluate parameter values according to the design prior, then estimate the success probability at each value and average with the appropriate weights.

Worked Example: Assurance Under a Discrete Design Prior

Use the one-observation design and posterior success rule from above. Suppose the design prior assigns probability $0.25$ to $\theta=0.3$, probability $0.50$ to $\theta=0.6$, and probability $0.25$ to $\theta=0.9$. Because success occurs exactly when $Y=1$, the conditional power at each value is $\operatorname{CP}(\theta;1)=\theta$.

Calculation. Weight each conditional power by its design-prior probability: $$ \begin{aligned} A(1) &=0.25(0.3)+0.50(0.6)+0.25(0.9)\\ &=0.075+0.300+0.225\\ &=0.600. \end{aligned} $$

The assurance is 60.0%. This is lower than the conditional power of 80.0% at $\theta=0.8$ in the preceding example because the design prior places weight on values below 0.8. Assurance does not contradict the conditional result: it answers a different question, averaging the success chance over a specified distribution of true values.

Worked Example: Changing the Design Prior Changes Assurance

Keep the same analysis model, success rule, and one-observation sampling model, but consider a different design prior: probability $0.5$ on $\theta=0.6$ and probability $0.5$ on $\theta=0.9$. The conditional power curve remains $\operatorname{CP}(\theta;1)=\theta$; neither the analysis rule nor the data model has changed.

Calculation. $$ \begin{aligned} A(1) &=0.50\operatorname{CP}(0.6;1)+0.50\operatorname{CP}(0.9;1)\\ &=0.50(0.6)+0.50(0.9)\\ &=0.300+0.450\\ &=0.750. \end{aligned} $$

The assurance is now 75.0%, rather than 60.0%, because this design prior gives no weight to $\theta=0.3$. The conditional power at any particular $\theta$ is unchanged. This illustrates why assurance cannot be interpreted without reporting the design prior: different defensible prior distributions can produce different prior-averaged probabilities for the same trial design.

Assumptions, Diagnostics, and Common Pitfalls

The calculation is meaningful only if the components of the design are explicit. Specify the sampling model, the estimand and its clinically relevant threshold, the analysis prior, the posterior success criterion, and—when reporting assurance—the design prior. Also state the sample size and other design features that affect the data distribution, such as allocation or follow-up, where relevant. The fixed value used for conditional power should be identified clearly, including its units or scale when applicable.

For a numerical calculation, check that the success event is implemented exactly as prespecified, that the parameter values used for conditional power cover the relevant range, and that the design-prior weights sum to one. If assurance is estimated by simulation, assess Monte Carlo uncertainty and verify that the simulated parameter values and trial data follow the intended design prior and sampling model. A plausible-looking average can still be wrong if the simulation accidentally averages equally over scenarios when the design prior assigns unequal weights.

A common mistake is to call conditional power “the probability of success given the current data.” That reverses the conditioning. Conditional Bayesian power is a repeated-sampling probability given a fixed true parameter value; it does not condition on the trial’s observed data. A second mistake is to call assurance “the posterior probability that the treatment is effective.” Assurance is calculated before observing trial data, averaging over a design prior. A posterior probability instead conditions on observed data and uses the analysis model.

Another pitfall is treating the analysis prior and design prior as interchangeable by default. The analysis prior determines the posterior used by the decision rule. The design prior describes how uncertainty about possible true parameter values is represented for assurance. They may coincide if that is a deliberate design choice, but using the same distribution for both does not make their purposes identical.

How the Quantities Relate to Alternatives

Conditional Bayesian power resembles conventional power in that both evaluate the probability of a decision outcome under a specified true state and repeated sampling. The difference is that a Bayesian trial’s success event is typically defined through a posterior decision rule, while conventional power is defined relative to a specified null hypothesis, alternative, and test procedure. The exact decision rule determines the relevant conditional probability; the word “Bayesian” does not remove the need to state that rule.

Assurance is the prior-averaged counterpart to conditional power. It is useful when the design question concerns uncertainty over plausible true values, but its answer depends on the design prior. Neither measure is universally preferable: conditional power makes the assumed true value transparent, while assurance summarizes performance across a specified distribution. Reporting both can clarify how performance varies across values and what average follows from the chosen design prior.

Both quantities have limitations. A misspecified sampling model or an inappropriate success criterion can make either estimate misleading. Assurance can conceal poor performance in an important region if the design prior assigns that region little weight; conditional power at a single value can conceal performance elsewhere. Consequently, a useful report presents the conditional operating-characteristic curve or selected scenario values alongside assurance, and explains why the design prior is appropriate. These summaries describe performance under stated assumptions; they do not guarantee the outcome of a future trial.

Key takeaway: Conditional Bayesian power fixes the true parameter and averages over trial-data variability. Assurance additionally averages conditional power over a design prior. The posterior success rule, sampling model, analysis prior, and design prior must be distinguished and stated.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

In one sentence, distinguish the uncertainty represented by conditional Bayesian power from that represented by assurance.
In the worked model, what is the conditional Bayesian power at $\theta=0.6$, and why?
Using the first discrete design prior, verify the assurance calculation and identify which quantity changes if only the design prior changes.
Does an assurance of 60% mean that, after observing trial data, there is a 60% posterior probability of treatment benefit? Explain.
Name the roles of the analysis prior and the design prior.

Solutions. (1) Conditional power fixes the true parameter and reflects trial-data sampling variability; assurance also averages over parameter uncertainty represented by the design prior. (2) The power is $0.600$, because success occurs exactly when $Y=1$, whose probability at $\theta=0.6$ is 0.6. (3) The weighted sum is $0.25(0.3)+0.50(0.6)+0.25(0.9)=0.600$. Changing only the design prior changes assurance, not the conditional power curve under the unchanged sampling model and decision rule. (4) No. Assurance is a prior-averaged, pre-data success probability; posterior probability of benefit conditions on observed data and uses the analysis model. (5) The analysis prior contributes to the posterior used in the decision rule; the design prior weights possible true parameter values when averaging conditional power.

Quantity	What is held fixed?	Where does uncertainty enter?	Question answered
Conditional Bayesian power	A specified true parameter value \(\theta\)	Repeated sampling of trial data under that value	At this true value, how often would the design meet its success rule?
Assurance	The trial design and design prior	Sampling variability and averaging over \(\theta\sim\pi_D\)	Before knowing the true value, what is the design-prior-averaged chance of meeting the rule?
Posterior probability of benefit	The observed data and analysis model	Posterior uncertainty about \(\theta\) after observing data	Given these observed data, how much posterior probability supports benefit?