Assurance as Prior-Averaged Success Probability

From a Fixed Effect to a Planning Distribution

A trial’s power depends on the treatment effect assumed to be true. Conditional Bayesian power answers a fixed-effect question: if the true effect were $\theta$, how often would the planned trial meet its Bayesian success rule? During planning, however, the true effect is unknown. A single assumed value can conceal how success probability changes across effects that are plausible before the trial.

Assurance summarizes that uncertainty by averaging conditional Bayesian power over a design prior. It is therefore a probability of success under a specified planning distribution for the treatment effect, not power evaluated at one selected effect. The averaging does not remove the need to state the trial’s success rule, the model, and the design prior; each is part of the quantity being reported.

Definition: Let $d$ denote a trial design, $Y$ the trial data, and $\theta$ the true treatment effect. Let $A_d(Y)$ be the event that the trial meets its prespecified Bayesian success rule under design $d$. The conditional Bayesian power at effect $\theta$ is $$ q(d,\theta)=P_{\theta,d}\{A_d(Y)\}, $$ the probability, under the data-generating model at that fixed effect, of meeting the rule. If $\pi_D(\theta)$ is a design prior for the treatment effect, the assurance is $$ \mathcal{A}(d)=E_{\theta\sim\pi_D}\{q(d,\theta)\}. $$ It is the design-prior average of conditional Bayesian power.

The subscript $D$ emphasizes the planning role of this distribution. The success event is defined by the analysis procedure—for example, a rule that declares success when the posterior probability of a beneficial effect exceeds a threshold. The design prior instead describes how the planner weights possible true effects when averaging conditional success probabilities. The two ingredients answer different questions, even though both involve probability distributions.

For a continuous design prior, the expectation can be written as an integral over effect values. For a finite set of possible effects $\theta_1,\ldots,\theta_J$, with design-prior weights $w_j=P(\theta=\theta_j)$, it is a weighted sum. In either case, assurance averages probabilities; it does not average effect sizes and then calculate power at the averaged effect.

Why the Average Is a Success Probability

The interpretation follows from the planning distribution. First, the design prior assigns probability to possible true effects. Conditional on any one of those effects, the trial’s sampling model assigns probability to data sets, and the fixed success rule determines whether each data set counts as a success. Averaging the conditional success probabilities gives the probability of success across this joint planning distribution.

For a finite design prior, this can be justified directly using the law of total probability. The joint predictive space consists of a value of $\theta$ together with trial data generated at that value. Partition that space according to the events $\{\theta=\theta_j\}$. Conditional on each event, the probability of success is $q(d,\theta_j)$, while the design prior assigns that event probability $w_j$. Therefore:

\mathcal{A}(d) = P\{A_d(Y)\} = \sum_{j=1}^{J}P\{A_d(Y)\mid\theta=\theta_j\}P(\theta=\theta_j) = \sum_{j=1}^{J}w_jq(d,\theta_j).

This argument concerns the joint predictive space, not a partition of trial outcomes by the value of the parameter. The parameter is not itself an outcome of the trial. Rather, the planning distribution assigns probabilities to possible parameter values, and data are generated conditionally on each value.

Worked Example: Averaging Across a Discrete Design Prior

Suppose a hypothetical trial measures the absolute difference in response rates, in percentage points, with positive values favoring treatment. Its prespecified success rule is that the posterior probability of a positive treatment effect must be at least 0.95. Under the planned model and trial design, suppose the conditional success probabilities have been evaluated at five possible true effects. The design prior and resulting conditional Bayesian power values are shown below.

True effect $\theta$ (percentage points)	Design-prior weight $w_j$	Conditional power $q(d,\theta_j)$
-10	0.10	0.10
0	0.20	0.25
10	0.40	0.50
20	0.20	0.75
30	0.10	0.90

The weights are nonnegative and sum to one: $0.10+0.20+0.40+0.20+0.10=1.00$. Thus they define a probability distribution over the listed effects. Applying the weighted-sum formula gives:

\begin{aligned} \mathcal{A}(d) &=\sum_{j=1}^{5}w_jq(d,\theta_j)\\ &=(0.10)(0.10)+(0.20)(0.25)+(0.40)(0.50) +(0.20)(0.75)+(0.10)(0.90)\\ &=0.010+0.050+0.200+0.150+0.090\\ &=0.500. \end{aligned}

The assurance is 0.500, or 50.0%. Under this design prior and the stated trial model and rule, the probability of meeting the success criterion is 50.0%. At a fixed true effect of 20 percentage points, conditional Bayesian power is 0.75, or 75.0%. That 75.0% is not the assurance: it conditions on one effect, whereas assurance averages across all five effects according to their design-prior weights.

The Design Prior Changes the Average, Not the Conditional Curve

Conditional power $q(d,\theta)$ describes the trial design and decision rule across fixed effect values. Once those are specified, changing the design prior does not change the conditional power at any particular $\theta$. It changes how much weight each value receives in the average, and therefore can change assurance.

Worked Example: Same Design, Different Planning Weights

Consider a hypothetical trial with the same endpoint, model, design, and posterior success rule as before. Suppose its conditional power is 0.15, 0.35, 0.60, and 0.80 at true effects of 0, 5, 10, and 15 percentage points, respectively. Two planning distributions assign different weights to these effects.

Under design prior A, let the weights be 0.10, 0.20, 0.40, and 0.30. The assurance is:

\begin{aligned} \mathcal{A}_A(d) &=(0.10)(0.15)+(0.20)(0.35)+(0.40)(0.60)+(0.30)(0.80)\\ &=0.015+0.070+0.240+0.240\\ &=0.565. \end{aligned}

Under design prior B, let the weights instead be 0.50, 0.25, 0.15, and 0.10. Then:

\begin{aligned} \mathcal{A}_B(d) &=(0.50)(0.15)+(0.25)(0.35)+(0.15)(0.60)+(0.10)(0.80)\\ &=0.075+0.0875+0.090+0.080\\ &=0.3325. \end{aligned}

Both sets of weights sum to one, and the conditional power values have not changed. The lower assurance under prior B reflects its greater weight on smaller effects, at which this particular design has lower conditional power. The difference is not evidence that the trial’s operating characteristics changed; it shows that the planning average depends on the planning distribution. The results should therefore be reported with the design prior, rather than as a standalone percentage.

Assurance for Comparing Planned Designs

Assurance can compare candidate designs when the same design prior and success definition are used. For each candidate, conditional power may differ because such features as sample size or allocation differ. The assurance for each candidate averages its own conditional power curve over the common design prior.

Worked Example: Comparing Two Hypothetical Sample Sizes

Suppose a hypothetical planning exercise compares 120 participants per arm with 180 participants per arm. The endpoint is again an absolute response-rate difference in percentage points, and success is defined by the same posterior rule for both designs. The design prior assigns weights 0.20, 0.50, and 0.30 to true effects of 0, 10, and 20 percentage points. Assume the conditional power values from the specified model and rule are as follows.

True effect (percentage points)	Weight	120 per arm	180 per arm
0	0.20	0.25	0.35
10	0.50	0.55	0.68
20	0.30	0.80	0.88

For 120 participants per arm, the assurance is:

\mathcal{A}_{120} =(0.20)(0.25)+(0.50)(0.55)+(0.30)(0.80) =0.050+0.275+0.240 =0.565.

For 180 participants per arm, it is:

\mathcal{A}_{180} =(0.20)(0.35)+(0.50)(0.68)+(0.30)(0.88) =0.070+0.340+0.264 =0.674.

Under the shared design prior and assumptions, the larger design has assurance 0.674 compared with 0.565 for the smaller design, an increase of 0.109, or 10.9 percentage points. This comparison summarizes the averaged probability of success; it does not say that the larger design has exactly 10.9 percentage points more power at every true effect. For example, at a true effect of 10 percentage points, the conditional powers are 0.55 and 0.68, a 13.0-point difference.

Assumptions, Checks, and Alternatives

Assurance is meaningful only relative to a well-defined planning calculation. At minimum, the following must be specified:

A fixed trial design and analysis rule. The sampling model, planned design, and event $A_d(Y)$ must determine what counts as success. If the rule changes, the conditional power values and assurance may change.
A conditional data-generating model. The probabilities $q(d,\theta)$ must be calculated under an appropriate model for trial data at each effect. Any other parameters that affect those probabilities must be fixed or handled according to the stated planning specification.
A design prior for the effect. Its support and weights should represent the planning distribution being used. For a discrete distribution, check that weights are nonnegative and sum to one. For a continuous distribution, it must be a proper probability distribution for the expectation to have the stated interpretation.
Consistent success definitions and conditioning. The same decision rule must be applied across effect values and candidate designs being compared. Conditional power is calculated at a fixed true effect; it is not the posterior probability of success after observing trial data.

Useful checks include inspecting the conditional power curve at the effects carrying the greatest prior weight and confirming that its values lie between zero and one. If increasing effects are expected to make success more likely, a sharply nonmonotone curve merits investigation. Monotonicity is not a universal mathematical requirement: the likelihood, endpoint, or decision rule may produce a more complex pattern. But unexpected irregularities can reveal calculation errors, inadequate numerical precision, or an unexamined model feature. For discrete calculations, independently verify the weighted products and that the prior weights sum to one.

The principal alternative is to report conditional Bayesian power at one clinically important effect. This is direct and transparent when that effect is the planning target, but it gives no average across other effects. A worst-case summary, such as the minimum conditional power over a stated range, answers a different question: how low can success probability be in that range? Assurance is an average, so high probabilities at effects with substantial prior weight can offset low probabilities elsewhere. These summaries are not interchangeable; the choice should follow the planning question, and more than one may be useful.

Assurance has limitations. It can be sensitive to the design prior, especially if conditional power varies greatly over its support. It is not a guarantee that a particular trial will succeed, and it is not the probability that the treatment is effective after data are observed. Nor does a high assurance by itself establish that the success rule is clinically appropriate or that the model is well calibrated. It reports a planning probability under stated assumptions. Sensitivity to plausible planning distributions and model assumptions is therefore important when the result will inform a consequential design choice.

Common Mistake

Common mistake: “The design has 75% assurance because conditional power is 75% at the selected effect.” This treats a probability conditional on one fixed effect as though it were an average over a design prior. The correction is to label the 75% as $q(d,\theta)$ at that effect, then calculate assurance by weighting conditional power across the full design prior. If the prior places all its weight on that one effect, the two quantities coincide; otherwise, they need not.

Key takeaway: Conditional Bayesian power asks how often a design succeeds at a specified true effect. Assurance averages those conditional probabilities over a stated design prior, so its value depends on both the trial’s conditional power curve and the planning distribution used to weight that curve.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A design prior assigns weights 0.25 and 0.75 to effects $\theta_1$ and $\theta_2$. Conditional powers are 0.20 and 0.60. What is the assurance?
If the conditional power at a true effect of 10 percentage points is 0.70, does that alone determine assurance? Explain.
For the same design and conditional power curve, a second design prior places more weight on effects with lower conditional power. What happens to assurance if those are the only weights changed?
State one important difference between assurance and the minimum conditional power over a specified effect range.

Solutions. (1) The weights sum to one, and $\mathcal{A}= (0.25)(0.20)+(0.75)(0.60)=0.05+0.45=0.50$, or 50.0%. (2) No. The value 0.70 is conditional power at one fixed effect. Assurance also requires the design prior and the conditional powers at the effects to which that prior assigns weight. (3) The assurance will generally decrease if more weight moves to effects with lower conditional power, while the conditional power curve itself remains unchanged. (4) Assurance is a prior-weighted average over effects; the minimum is the lowest conditional power in the stated range. They answer different planning questions and need not be close.