Sample Size Assurance Under Prior Uncertainty

Why Average Success Over Prior Uncertainty?

A planned trial may have a high probability of success if the treatment effect equals one chosen value, yet a substantially different probability if the true effect is smaller or larger. When the treatment effect is uncertain at the design stage, a single fixed-effect calculation can give an incomplete picture. Assurance averages the trial’s probability of success over a design prior for the treatment effect. It therefore combines uncertainty about the effect with the sampling variability the future trial will have.

Assurance is a design quantity, not the probability of success conditional on interim data already observed. The latter is a predictive probability of success: it uses current trial data to update what is known about the effect. Assurance instead averages over the prior uncertainty before the planned trial’s outcomes are available.

Definition: Let $\delta$ be the treatment effect, with design prior density $\pi(\delta)$. Let $Y$ denote the future trial data, and let $S(Y)$ be the event that a prespecified Bayesian success rule is met. The assurance of the design is $$ A = P(S(Y)) = \int P(S(Y)\mid \delta)\,\pi(\delta)\,d\delta. $$ Here $P(S(Y)\mid\delta)$ is the probability of success under the sampling model when the treatment effect equals $\delta$. Assurance is the prior-weighted average of those conditional probabilities.

A design prior should represent uncertainty about plausible treatment effects before the planned trial, not merely encode the effect needed to justify a desired sample size. The prior, outcome model, success rule, and sample-size plan together determine assurance. The calculation is only as relevant as these inputs are to the trial’s intended population and conduct.

A Normal-Outcome Design Model

For a transparent calculation, suppose a future two-arm trial compares a treatment and a control using a continuous outcome. Let $\delta$ be the treatment-minus-control difference in means, measured in outcome units. Assume equal allocation, $n$ participants per arm, independent normally distributed outcomes in each arm, and a known common within-arm standard deviation $\sigma$. The difference in sample means, denoted $Y$, then has the sampling distribution

Y\mid\delta \sim N(\delta,V_n),\qquad V_n=\frac{2\sigma^2}{n}.

Assume the design prior is $\delta\sim N(m_0,\tau^2)$, and define success as a posterior probability of at least $q$ that the treatment effect is positive. In other words, success occurs when $P(\delta>0\mid Y)\ge q$, with $q>0.5$. This is a specified Bayesian decision rule for the examples; other trials may use a different posterior criterion.

Under the normal prior and sampling model, the posterior distribution is normal, with variance $v_n$ and mean $m_n(Y)$:

v_n=\left(\frac{1}{\tau^2}+\frac{1}{V_n}\right)^{-1} =\frac{\tau^2V_n}{\tau^2+V_n}, \qquad m_n(Y)=\frac{V_n m_0+\tau^2Y}{\tau^2+V_n}.

Because the posterior probability of a positive effect is $\Phi(m_n(Y)/\sqrt{v_n})$, where $\Phi$ is the standard normal cumulative distribution function, success is equivalent to a threshold on the posterior mean:

m_n(Y)\ge c_n,\qquad c_n=z_q\sqrt{v_n},

where $z_q$ is the $q$ quantile of the standard normal distribution. Thus the success rule translates into a rule for the future observed treatment-effect estimate.

Deriving the Assurance Formula

The prior predictive distribution of $Y$ follows by integrating its sampling distribution over the normal prior. It is $Y\sim N(m_0,\tau^2+V_n)$. Since $m_n(Y)$ is a linear transformation of $Y$, it is also normally distributed under this prior predictive distribution. Set $k_n=\tau^2/(\tau^2+V_n)$. Then $m_n(Y)=m_0+k_n(Y-m_0)$, giving mean $m_0$ and standard deviation $k_n\sqrt{\tau^2+V_n}$. The probability that this normally distributed posterior mean exceeds $c_n$ yields the assurance:

A(n)=P\{m_n(Y)\ge c_n\} =\Phi\left( \frac{m_0-z_q\sqrt{v_n}} {k_n\sqrt{\tau^2+V_n}} \right).

This result is specific to the normal model, normal prior, known sampling variance, and stated posterior-threshold success rule. It is an analytic shortcut: no simulation is needed to integrate over the future data and prior in this setting. In a more complex model, assurance can instead be estimated by repeatedly drawing an effect from the design prior, simulating trial data under that effect, and applying the success rule to each simulated trial.

Specify the design inputs.
Choose the design prior, outcome sampling model, allocation, candidate sample size, and the posterior criterion that defines success.

Calculate the conditional success probability.
For a given $\delta$, calculate $P(S(Y)\mid\delta)$ under the sampling model.

Average across prior uncertainty.
Integrate the conditional success probability over the design prior, or use an analytic formula when the model permits.

Compare plans.
Repeat the calculation across candidate sample sizes and report the achieved assurance alongside the target.

Worked Example: Assurance for a Candidate Sample Size

Suppose the treatment effect is measured in points on a clinical outcome. Use $\sigma=10$ points, a design prior $\delta\sim N(8,5^2)$ points, $n=50$ participants per arm, and the success rule $P(\delta>0\mid Y)\ge 0.975$. Calculate the assurance.

The sampling variance of the estimated difference is $V_n=2(10^2)/50=4$ points squared. The posterior variance is $v_n=(25\times4)/(25+4)=100/29=3.448$, so its standard deviation is $\sqrt{3.448}=1.857$ points. For $q=0.975$, $z_q\approx1.960$, giving the success cutoff $c_n=1.960(1.857)=3.640$ points.

The posterior-mean multiplier is $k_n=25/29=0.8621$. Under the prior predictive distribution, the posterior mean has standard deviation $k_n\sqrt{\tau^2+V_n}=(25/29)\sqrt{29}=4.642$ points. Therefore,

A(50)=\Phi\left(\frac{8-3.640}{4.642}\right) =\Phi(0.939)\approx0.826.

The planned trial has approximately 82.6% assurance under these assumptions and this success rule. This is a prior-averaged probability of meeting the posterior criterion, not a statement that the treatment has an 82.6% probability of being effective.

Prior Uncertainty and the Assurance Target

Assurance depends on both the prior mean and its spread. A prior centered farther above the success boundary will generally increase assurance, all else equal. A wider prior can reduce assurance when the prior mean is favorable but places more probability on small or negative effects. These effects are not interchangeable: a prior can be centered at the same effect while producing a different assurance because its uncertainty differs.

Sample size also matters, but a larger sample does not guarantee a dramatic increase in assurance. More precise data make the posterior more responsive to the trial results; they do not change the design prior’s assessment of which treatment effects are plausible. Assurance approaches a limit as sampling error becomes very small. Under this normal model, as $n$ increases without bound, $Y$ approaches $\delta$, and the success event approaches $\delta>0$. Thus assurance approaches $P(\delta>0)=\Phi(m_0/\tau)$, rather than necessarily approaching one.

Worked Example: Sensitivity to Prior Spread

Keep $\sigma=10$ points, $n=50$ per arm, and the $0.975$ posterior threshold from the previous calculation. Compare a prior $N(8,3^2)$ with the original prior $N(8,5^2)$. The prior means are identical, so the comparison isolates the effect of prior spread.

For the narrower prior, $\tau^2=9$, while $V_n=4$. Then $v_n=9(4)/(9+4)=36/13=2.769$, and $c_n=1.960\sqrt{2.769}=3.261$ points. Also $k_n=9/13$, so the prior predictive standard deviation of the posterior mean is $k_n\sqrt{13}=2.496$ points. The resulting assurance is

A(50)=\Phi\left(\frac{8-3.261}{2.496}\right) =\Phi(1.898)\approx0.971.

Under the broader $N(8,5^2)$ prior, the assurance was approximately 0.826. The narrower prior places less probability on treatment effects far below its favorable mean, leading to higher assurance with the same trial design. This result does not show that a narrow prior is preferable: its credibility must be justified using relevant evidence and clinical knowledge.

Worked Example: Comparing Sample Sizes for Two Assurance Targets

Use the prior $N(8,5^2)$, $\sigma=10$ points, and $q=0.975$. Compare candidate plans with $n=36$ and $n=75$ participants per arm. These plans illustrate the sample-size change associated with moving from an assurance target of about 0.80 to one of about 0.85.

For $n=36$, $V_n=200/36=5.556$ points squared. The posterior variance is $v_n=25(5.556)/(25+5.556)=4.545$, giving $c_n=1.960\sqrt{4.545}=4.179$ points. The posterior-mean predictive standard deviation is $(25/30.556)\sqrt{30.556}=4.523$ points. Thus

A(36)=\Phi\left(\frac{8-4.179}{4.523}\right) =\Phi(0.845)\approx0.801.

For $n=75$, $V_n=200/75=2.667$ points squared. Here $v_n=25(2.667)/(25+2.667)=2.410$, so $c_n=1.960\sqrt{2.410}=3.043$ points. The posterior-mean predictive standard deviation is $(25/27.667)\sqrt{27.667}=4.754$ points. Therefore,

A(75)=\Phi\left(\frac{8-3.043}{4.754}\right) =\Phi(1.043)\approx0.852.

The 36-per-arm plan is just above 0.80 assurance, and the 75-per-arm plan is above 0.85. These are candidate plans under the specified model, not universal sample-size recommendations. A sample-size search should evaluate the assurance at each feasible allocation and choose a plan that meets the target with appropriate allowance for practical constraints.

Interpretation, Diagnostics, and Alternatives

Assurance describes a probability under a joint design calculation: first regard the effect as distributed according to the design prior, then regard the future trial data as generated under the sampling model conditional on that effect. It is not the posterior probability that the treatment works, nor is it an unconditional guarantee that a trial will succeed. A design can have high assurance while still having a meaningful chance of failing its success rule.

A useful diagnostic is to examine the conditional success probability $P(S(Y)\mid\delta)$ over a range of clinically plausible effects, not just its average. That curve shows whether assurance is driven by plausible effects or by a small region of highly favorable effects. Also check that the chosen prior spread, outcome variance, treatment-effect scale, allocation, and success threshold match the planned analysis. If a numerical integration or simulation is used, check that increasing the number of draws does not materially change the estimate and that the simulated conditional success rates agree with the sampling model.

Assurance differs from common alternatives in purpose. Power evaluates success probability at a specified fixed effect; it does not average over prior uncertainty. A Bayesian predictive probability of success after an interim analysis averages over uncertainty updated by the interim data, and supports a decision during the ongoing trial. Assurance is calculated before the trial to assess a proposed design. These quantities answer different questions and should not be reported as if they were interchangeable.

The normal example has important limits: the outcome is assumed normally distributed, the variance is known and common between arms, and the design prior is normal. Real trials may have binary or time-to-event outcomes, uncertain variance, missing data, nonadherence, or other features that change the sampling distribution and success rule. In such settings the integral definition still applies, but the closed-form expression may not. Model-based simulation can estimate assurance, provided it reflects the planned trial and analysis.

Common Pitfall

Wrong reasoning: “The trial has 80% assurance, so there is an 80% probability that the treatment effect is positive.” Assurance is the probability that a trial meets its specified success rule, averaged over a design prior and the future data distribution; it is not the posterior probability of a positive effect.
Correction: State the success event and design assumptions when reporting assurance. For example: “Under the stated normal model and design prior, the probability that the posterior probability of a positive effect reaches 0.975 is 0.80.”
Another common error: Reporting power at the prior mean as though it were assurance. Power at one value uses $P(S(Y)\mid\delta=m_0)$; assurance integrates $P(S(Y)\mid\delta)$ across the entire prior.

Key takeaway: Assurance is the prior-averaged probability that a planned trial meets a prespecified success rule. Its value depends on the prior, sampling model, sample size, and rule, so all four should be reported and examined when comparing designs.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

In one sentence, distinguish assurance from power at a fixed treatment effect.
For the normal model, what is the sampling variance $V_n$ when $\sigma=8$ and $n=32$ per arm?
With a success rule $P(\delta>0\mid Y)\ge q$, what condition on the posterior mean defines success?
Why can a wider prior centered at the same favorable effect reduce assurance?
Does assurance of 0.85 mean there is an 85% posterior probability that the treatment is beneficial? Explain.

Solutions. (1) Power is conditional success probability at one fixed effect; assurance averages conditional success probability over the design prior. (2) $V_n=2(8^2)/32=4$ outcome-units squared. (3) Success occurs when $m_n(Y)\ge z_q\sqrt{v_n}$. (4) The wider prior assigns more probability to effects below the success boundary, which can lower the prior-averaged success probability. (5) No. Assurance is the prior-averaged probability that the trial meets its success rule; it is not the posterior probability that the effect is beneficial.