Constructing and Reading Assurance Curves

What an Assurance Curve Shows

A sample-size calculation often reports one number: the smallest design meeting a chosen assurance target. An assurance curve provides more context by showing how prior-averaged success probability changes across a range of sample sizes. It can reveal whether a target is reached gradually or quickly, how much assurance is attainable under the assumed design prior, and whether the curve levels off before reaching a desired target.

To construct the curve, specify the trial design and success rule, calculate the probability of success at each possible true parameter value, and average that probability over the design prior. The design prior represents uncertainty about the true effect for planning. It is distinct from the analysis prior used to obtain the posterior at the end of the trial.

Definition: Let $\theta$ denote the unknown true parameter, let $n$ denote sample size, and let $S$ denote the event that a prespecified Bayesian success rule is met. If $\pi_D(\theta)$ is the design-prior density, the assurance at sample size $n$ is $$ A(n)=\int P(S\mid \theta,n)\,\pi_D(\theta)\,d\theta. $$ The function $A(n)$, evaluated over a range of sample sizes, is the assurance curve. For a discrete design prior, the integral is replaced by a probability-weighted sum.

Assurance is therefore a probability averaged over two sources of uncertainty: the design prior over possible true effects, and the sampling distribution of trial data conditional on each effect. Its value depends on the success rule, the sampling model, the analysis prior, and the design prior. A curve is not an intrinsic property of a treatment effect alone.

A Normal-Endpoint Curve

Consider a hypothetical trial with $n$ participants in each of two independent arms. Assume a continuous endpoint has known variance 1 in each arm, and let $\delta$ be the treatment-minus-control mean difference, measured in endpoint units. The observed difference $Y$ then satisfies

Y\mid\delta,n\sim N\!\left(\delta,\frac{2}{n}\right).

For analysis, use a flat prior on $\delta$, so the posterior is $N(Y,2/n)$. Define success as a posterior probability greater than $q=0.95$ that the treatment effect is positive: $P(\delta>0\mid Y)>q$. This model assumes independent normally distributed outcomes, known equal variances, a fixed sample size, and no missing data or other departures from the stated design. The flat analysis prior gives the stated normal posterior; the design prior used below is proper.

Write $z_q$ for the $q$ quantile of the standard normal distribution. The posterior success rule is equivalent to $Y>z_q\sqrt{2/n}$. Thus, conditional on a particular true effect $\delta$, the probability of success is

P(S\mid\delta,n) =P\!\left(Y>z_q\sqrt{2/n}\mid\delta,n\right) =\Phi\!\left(\frac{\delta-z_q\sqrt{2/n}}{\sqrt{2/n}}\right),

where $\Phi$ is the standard normal cumulative distribution function. This conditional probability is the Bayesian power at a fixed effect. Assurance averages it over the design prior; it is not the same quantity as conditional power.

Suppose the design prior is $\delta\sim N(\mu_D,\tau_D^2)$. Under this prior and sampling model, the marginal distribution of $Y$ is $N(\mu_D,\tau_D^2+2/n)$. Since success is the event $Y>z_q\sqrt{2/n}$, the assurance curve has the closed form

A(n)=\Phi\!\left( \frac{\mu_D-z_q\sqrt{2/n}}{\sqrt{\tau_D^2+2/n}} \right).

This expression makes the roles of sample size, the threshold in the success rule, and design-prior assumptions explicit. For other endpoint models or success rules, the same prior-averaging definition applies, but the curve may require numerical integration rather than this closed form.

Worked Example: Calculating and Reading a Curve

Suppose the design prior is $N(0.20,0.30^2)$, with effects measured in the endpoint’s units, and retain the 0.95 posterior success threshold. Calculate assurance for $n=25$, $100$, and $400$ participants per arm.

Here $z_{0.95}=1.6449$, $\mu_D=0.20$, and $\tau_D=0.30$. For $n=25$, $\sqrt{2/n}=0.2828$, so $$ A(25)=\Phi\!\left( \frac{0.20-1.6449\sqrt{2/25}}{\sqrt{0.30^2+2/25}} \right) =\Phi\!\left(\frac{-0.26525}{0.41231}\right) =\Phi(-0.6433)\approx 0.260. $$

For $n=100$, $\sqrt{2/n}=0.1414$: $$ A(100)=\Phi\!\left( \frac{0.20-1.6449(0.1414)}{\sqrt{0.30^2+2/100}} \right) =\Phi\!\left(\frac{-0.0326}{0.3317}\right) =\Phi(-0.0983)\approx 0.461. $$

For $n=400$, $\sqrt{2/n}=0.07071$: $$ A(400)=\Phi\!\left( \frac{0.20-1.6449(0.07071)}{\sqrt{0.30^2+2/400}} \right) =\Phi\!\left(\frac{0.0837}{0.3082}\right) =\Phi(0.2715)\approx 0.607. $$

The curve rises across these sample sizes, but none reaches 0.70 assurance. The values refer to the probability of meeting the specified posterior rule, averaged over the stated design prior; they are not probabilities that the treatment is effective.

Target Crossings, Monotonicity, and Plateaus

A target crossing is a sample size at which the curve reaches a prespecified assurance level. In the example above, set the target to 0.60. Solving the closed-form curve for $A(n)=0.60$ gives a crossing at approximately $n=365$ participants per arm. Because sample sizes are integer-valued, the design would need to be checked at the adjacent integers; a practical design may also round upward to a convenient allocation. If the target is specified as a minimum, the selected sample size should be the first eligible value that meets it, not just an approximate crossing.

For this example, assurance increases with sample size. More generally, the curve need not be monotone for every design prior. In the normal model, put $s=\sqrt{2/n}$ and $x(s)=(\mu_D-z_qs)/\sqrt{\tau_D^2+s^2}$, so $A(n)=\Phi(x(s))$. Differentiation gives

\frac{dx}{ds} =-\frac{z_q\tau_D^2+\mu_Ds}{(\tau_D^2+s^2)^{3/2}}.

When $\mu_D>0$, this derivative is negative, and $s$ decreases as $n$ increases. Thus $x(s)$, and therefore assurance, increases with $n$ in this particular model. If $\mu_D<0$, the derivative can change sign; the assurance curve can then be nonmonotone. A target may be crossed more than once, so the curve must be checked over the full relevant range before identifying the smallest qualifying design.

The curve’s limiting value as $n$ grows is its plateau. In this normal example, as $n\to\infty$, sampling uncertainty vanishes and the formula gives

\lim_{n\to\infty}A(n)=\Phi\!\left(\frac{\mu_D}{\tau_D}\right) =P_{\pi_D}(\delta>0).

This is the probability, under the continuous normal design prior, of a positive true effect. The equality describes an average over that prior; it does not mean every conditional success probability converges to either zero or one. At a true effect $\delta>0$, the conditional success probability tends to 1; at $\delta<0$, it tends to 0. At $\delta=0$, however, the rule’s threshold and the sampling standard deviation shrink at the same rate:

P(S\mid \delta=0,n) =\Phi(-z_q)=1-q.

For $q=0.95$, this limit is 0.05, not zero. The prior-averaged limit is still $P_{\pi_D}(\delta>0)$ because a continuous design prior assigns probability zero to the single point $\delta=0$. A design prior with point mass at zero would retain a contribution of that mass times $1-q$ in the limiting assurance.

Worked Example: Checking the Plateau Carefully

For the design prior $N(0.20,0.30^2)$, the continuous-prior plateau is $$ \lim_{n\to\infty}A(n)=\Phi(0.20/0.30) =\Phi(0.6667)\approx 0.7475. $$

To see why this is not a claim that success becomes certain whenever the true effect is positive, consider the conditional success probability at $n=100$. Here $\sqrt{2/n}=0.1414$ and the success threshold is $1.6449(0.1414)=0.2326$. If $\delta=0$, then $$ P(S\mid\delta=0,n=100) =\Phi\!\left(\frac{0-0.2326}{0.1414}\right) =\Phi(-1.6449)=0.050. $$

For a fixed $\delta=0.20$, the same sample size gives $$ P(S\mid\delta=0.20,n=100) =\Phi\!\left(\frac{0.20-0.2326}{0.1414}\right) =\Phi(-0.2306)\approx 0.409. $$ At much larger sample sizes this conditional probability tends toward 1, while the conditional probability at $\delta=0$ remains 0.05. The overall assurance approaches 0.7475 because, under the design prior, about 74.75% of effects are positive; the zero-effect point has no probability mass under this continuous prior.

Comparing Design-Prior Assumptions

A useful assurance display overlays curves for plausible design priors while keeping the analysis model, success rule, and trial design fixed. The curves then show how planning conclusions depend on beliefs about the effect before the trial. The plateau is particularly informative: for a normal design prior it depends on the standardized mean $\mu_D/\tau_D$, not on the sample size once the limit is reached.

Worked Example: Prior Spread Can Change the Curve’s Ranking

Compare three hypothetical normal design priors with mean effect $0.20$ and standard deviations $0.15$, $0.30$, and $0.50$. At $n=100$, the shared numerator of the assurance formula is $0.20-1.6449\sqrt{0.02}=-0.0326$. For $\tau_D=0.15$, $$ A(100)=\Phi\!\left(\frac{-0.0326}{\sqrt{0.15^2+0.02}}\right) =\Phi(-0.1581)\approx 0.437. $$ For $\tau_D=0.30$, the calculation is $\Phi(-0.0983)\approx 0.461$, as above. For $\tau_D=0.50$, $$ A(100)=\Phi\!\left(\frac{-0.0326}{\sqrt{0.50^2+0.02}}\right) =\Phi(-0.0628)\approx 0.475. $$

At this sample size, assurance is higher for the more diffuse priors, but their limiting values are lower: respectively, $\Phi(1.333)\approx0.908$, $\Phi(0.6667)\approx0.747$, and $\Phi(0.400)\approx0.655$. Thus a wider prior can raise assurance at one sample size while lowering the plateau. Curves can cross; prior spread does not have a universally favorable direction.

Diagnostics, Alternatives, and Limitations

Before interpreting an assurance curve, verify that it represents the intended design and success rule. For numerical curves, use a sufficiently fine sample-size grid near a target and confirm apparent crossings with direct calculations. If a curve appears irregular, check the integration accuracy, the sample-size indexing, and the prior weights; a jagged line can reflect numerical error rather than a real change in assurance. If the model gives a nonmonotone curve, do not smooth it away or assume that the first crossing is the only crossing.

A curve is more informative than reporting only the selected sample size, but it still inherits the assumptions used to build it. The normal example assumes known variances and a specific posterior threshold. A different analysis prior, variance model, endpoint, or rule can change the entire curve. The plateau also does not represent a guarantee of success: it is the large-sample limit averaged under the specified design prior and rule.

The main alternative is to report conditional Bayesian power at selected fixed effect sizes. That answers how success probability changes with sample size if each particular effect were true, but does not average over design uncertainty. A single assurance value at one sample size is more compact, yet hides the curve’s shape and plateau. Reporting both the curve and a small set of conditional values can make the planning assumptions more transparent. The curve should be read as a design summary under stated assumptions, not as evidence that a future trial will achieve its planned outcome.

Mistake: assuming assurance must approach 1 as sample size grows. Correction: with a continuous design prior and a positive-effect rule, the plateau is the prior probability of a positive effect; it can be well below 1.
Mistake: treating the conditional success probability at $\delta=0$ as zero in the large-sample limit. Correction: under the stated normal rule it is $1-q$, because both the threshold and sampling standard deviation shrink proportionally.
Mistake: concluding that a wider design prior always increases or always decreases assurance. Correction: compare the curves over the sample-size range of interest; curves may cross.

Check Your Understanding

Answer each question using the definitions and normal-model results above. Brief solutions follow the questions.

In one sentence, distinguish conditional Bayesian power at a fixed effect from assurance.
For a continuous design prior $N(\mu_D,\tau_D^2)$ and the positive-effect rule with threshold $q$, what is the assurance plateau in the normal example?
Why does the conditional success probability at $\delta=0$ equal 0.05 when $q=0.95$, even as sample size grows?
If a target assurance exceeds the continuous-prior plateau, can any finite sample size meet it in this example?
Why should a nonmonotone curve be inspected across the full sample-size range rather than summarized by one crossing?

Solutions. (1) Conditional Bayesian power is the success probability at one specified true effect; assurance averages that probability over the design prior. (2) The plateau is $\Phi(\mu_D/\tau_D)=P_{\pi_D}(\delta>0)$. (3) At zero effect, the standardized success threshold remains $z_{0.95}$, so the success probability is $1-\Phi(z_{0.95})=0.05$. (4) For the worked example, where $\mu_D>0$, assurance increases toward the plateau, so a target above it cannot be met. In the general normal model, check the curve over the relevant sample sizes; a nonmonotone curve can exceed its plateau at finite $n$. (5) It may cross the target more than once, and identifying the smallest qualifying design requires checking all relevant sample sizes rather than assuming a single upward crossing.