Assurance for Normal Outcomes

From a Fixed Effect to Assurance

A trial’s conditional Bayesian power answers a question at one specified treatment effect: if that effect were the truth, how often would the planned trial satisfy its Bayesian success rule? Assurance answers a different question. It averages that conditional success probability over a probability distribution for the treatment effect before the trial data are observed. For a normal endpoint, normal sampling model and normal design distribution, this average has a closed form.

The calculation has two layers. First, the analysis model and prior turn the success rule into a cutoff for the observed treatment-control difference. Second, the sampling distribution gives the probability of crossing that cutoff at each possible true effect. Assurance averages those probabilities over the design distribution. Keeping the analysis prior and design distribution distinct is essential: one determines how trial data are analyzed, while the other describes the effects over which planned performance is averaged.

Model and Decision Rule

Definition: Let $Y_{Ti}$ and $Y_{Ci}$ be continuous outcomes for the treatment and control groups, respectively. Assume independent observations with $Y_{Ti}\sim N(\mu_T,\sigma^2)$ and $Y_{Ci}\sim N(\mu_C,\sigma^2)$, where the common variance $\sigma^2$ is known. The treatment effect is $\Delta=\mu_T-\mu_C$, with larger values defined as beneficial. With $n$ participants per group, the observed difference $D=\bar{x}_T-\bar{x}_C$ has sampling distribution $D\mid\Delta\sim N(\Delta,v)$, where $v=2\sigma^2/n$.

For the analysis, suppose the prior distribution for the effect is $\Delta\sim N(m_a,\omega_a^2)$. The normal likelihood and normal prior give a normal posterior. Write its variance as $q$ and its mean as $m(D)$:

q=\left(\frac{1}{\omega_a^2}+\frac{1}{v}\right)^{-1}, \qquad m(D)=q\left(\frac{m_a}{\omega_a^2}+\frac{D}{v}\right).

Consider the decision rule that declares success when the posterior probability that the effect exceeds a clinically relevant threshold $\Delta_{\min}$ is at least $\gamma$, where $0<\gamma<1$. Under the normal posterior, this rule is equivalent to a threshold on $D$. Let $z_\gamma$ be the $\gamma$-quantile of the standard normal distribution, so that $\Phi(z_\gamma)=\gamma$. Then:

P(\Delta>\Delta_{\min}\mid D)\geq\gamma \quad\Longleftrightarrow\quad m(D)\geq\Delta_{\min}+z_\gamma\sqrt{q} \quad\Longleftrightarrow\quad D\geq c, $$ $$ c=\frac{v}{q}\left(\Delta_{\min}+z_\gamma\sqrt{q}-q\frac{m_a}{\omega_a^2}\right).

The cutoff $c$ depends on the sample size, known variance, analysis prior and success criterion. Once it is calculated, conditional Bayesian power at a fixed true effect $\Delta$ is the probability that the normally distributed $D$ exceeds $c$:

\operatorname{Power}(\Delta) =P(D\geq c\mid\Delta) =1-\Phi\left(\frac{c-\Delta}{\sqrt{v}}\right) =\Phi\left(\frac{\Delta-c}{\sqrt{v}}\right).

Averaging Over the Design Distribution

For this calculation, let the design distribution for the treatment effect be $\Delta\sim N(m_d,\omega_d^2)$. The assurance is the expected conditional power under that distribution. It is not the posterior probability of success after observing trial data; it is a pretrial probability of satisfying the decision rule, averaged across the effects represented by the design distribution.

Result: Under the model and decision rule above, with $\Delta\sim N(m_d,\omega_d^2)$ as the design distribution, assurance is $$ A=E_{\Delta}\left[\Phi\left(\frac{\Delta-c}{\sqrt{v}}\right)\right] =\Phi\left(\frac{m_d-c}{\sqrt{\omega_d^2+v}}\right). $$

This closed form requires the fixed-variance normal sampling model, a normal analysis prior, a posterior-probability decision rule of the stated form, and a normal design distribution. The design distribution’s variance $\omega_d^2$ is added to the sampling variance $v$ in the final expression. That addition reflects uncertainty about the true effect across the design distribution as well as variability in trial outcomes at any one effect.

Why the Closed Form Holds

Introduce a sampling error $E\sim N(0,v)$, independent of the design effect $\Delta$. Under the sampling model, the observed difference can be represented as $D=\Delta+E$. The event of success is $D\geq c$, so averaging the conditional success probability over the design distribution is equivalent to calculating the joint probability of this event:

A=E_{\Delta}\{P(D\geq c\mid\Delta)\} =P(\Delta+E\geq c).

Because independent normal variables add to a normal variable, $\Delta+E\sim N(m_d,\omega_d^2+v)$. Standardizing this distribution gives:

P(\Delta+E\geq c) =1-\Phi\left(\frac{c-m_d}{\sqrt{\omega_d^2+v}}\right) =\Phi\left(\frac{m_d-c}{\sqrt{\omega_d^2+v}}\right).

This argument also provides a useful check on the calculation: assurance must be between zero and one, and it increases as the design mean $m_d$ moves farther above the decision cutoff $c$. The formula does not say that a single future trial will achieve success with certainty; it gives the probability of success under the specified sampling and design distributions.

Worked Example: Conditional Power and Assurance

Suppose the endpoint is a hypothetical continuous measure recorded in units, with known standard deviation $\sigma=10$ units in both groups and $n=100$ participants per group. Use the analysis prior $\Delta\sim N(0,100)$, require posterior probability at least $0.95$ that $\Delta>0$, and use the design distribution $\Delta\sim N(2,2.25)$. Calculate the decision cutoff, conditional power at $\Delta=3$, and assurance.

The sampling variance of $D$ is $v=2(10^2)/100=2$ square units. The posterior variance is $q=(1/100+1/2)^{-1}=1.9608$ square units, so $\sqrt{q}=1.4003$ units. Since $m_a=0$, the posterior mean is $m(D)=qD/v=0.98039D$. For $\gamma=0.95$, $z_\gamma=1.6449$. The success rule requires a posterior mean above $0+1.6449(1.4003)=2.303$ units. Thus:

c=\frac{2.303}{0.98039}=2.349\text{ units}.

At the fixed true effect $\Delta=3$ units, the conditional power is:

\operatorname{Power}(3)=\Phi\left(\frac{3-2.349}{\sqrt{2}}\right) =\Phi(0.4603)=0.677.

For assurance, the design standard deviation is $\omega_d=1.5$ units. Therefore:

A=\Phi\left(\frac{2-2.349}{\sqrt{2.25+2}}\right) =\Phi(-0.1694)=0.433.

The conditional power at an effect of 3 units is about 0.677, whereas assurance is about 0.433 because the design distribution also assigns probability to effects below its mean of 2 units, including effects that are less likely to cross the success cutoff.

Worked Example: A Clinically Relevant Threshold

In a hypothetical trial, outcomes are measured in millimetres, $\sigma=8$ mm is known, and each group has $n=64$ participants. Use analysis prior $\Delta\sim N(1,16)$ mm², and define success as posterior probability at least $0.90$ that the effect exceeds $\Delta_{\min}=2$ mm. The design distribution is $N(4,4)$ mm². Find assurance.

The sampling variance is $v=2(8^2)/64=2$ mm². The posterior variance and standard deviation are:

q=(1/16+1/2)^{-1}=1.7778\text{ mm}^2, \qquad \sqrt{q}=1.3333\text{ mm}.

Here $z_{0.90}=1.2816$. The required posterior mean is at least $2+1.2816(1.3333)=3.7088$ mm. The posterior mean as a function of the observed difference is $m(D)=1.7778(1/16+D/2)=0.1111+0.8889D$. Hence the success cutoff is:

c=\frac{3.7088-0.1111}{0.8889}=4.047\text{ mm}.

The design mean is $m_d=4$ mm and the design variance is $\omega_d^2=4$ mm², so:

A=\Phi\left(\frac{4-4.047}{\sqrt{4+2}}\right) =\Phi(-0.0192)=0.492.

Under these assumptions, the planned trial has about 0.492 assurance. The relatively demanding threshold above the design mean helps explain why this probability is near one half, even though the design mean is a clinically relevant effect.

Worked Example: A More Stringent Success Criterion

Consider another hypothetical normal endpoint measured in mg/dL. Let the known standard deviation be $12$ mg/dL, with $n=144$ participants per group. Take the analysis prior as $N(0,400)$ (in squared mg/dL), require posterior probability at least $0.975$ that $\Delta>0$, and use a design distribution $N(3.5,4)$ (also in squared mg/dL). Calculate assurance and compare it with conditional power at the design mean.

The sampling variance is $v=2(12^2)/144=2$ (mg/dL)². The posterior variance is $q=(1/400+1/2)^{-1}=1.9900$ (mg/dL)², and its standard deviation is $1.4107$ mg/dL. With $z_{0.975}=1.9600$, the required posterior mean is $1.9600(1.4107)=2.765$ mg/dL. Since the prior mean is zero, $m(D)=qD/v=0.9950D$, giving:

c=\frac{2.765}{0.9950}=2.779\text{ mg/dL}.

At the design mean, the conditional power is $\Phi((3.5-2.779)/\sqrt{2})=\Phi(0.5098)=0.695$. Assurance includes the design variance of $4$ (mg/dL)²:

A=\Phi\left(\frac{3.5-2.779}{\sqrt{4+2}}\right) =\Phi(0.2944)=0.616.

The assurance is lower than the conditional power at the design mean because it averages over effects both below and above that mean. A stringent posterior threshold raises the observed-data cutoff; it does not make the assurance equal to the power at the design mean.

Assumptions, Diagnostics, and Alternatives

Before using the closed-form calculation, check that its assumptions represent the planned trial. The outcomes must be adequately modeled as independent normal observations with a common known variance, and the group sizes and allocation must be fixed as specified. The effect prior used in the analysis must be normal, and the decision must be expressible as a posterior probability crossing a fixed threshold. Finally, the design distribution must be normal for the stated closed-form assurance expression.

A practical diagnostic is to verify each link in the calculation: check that $v$ is the variance of the treatment-control difference, not the variance of an individual outcome; confirm that $q$ uses the analysis prior variance; and ensure the cutoff has the same units as $D$ and $\Delta$. As a numerical check, compute conditional power at several effects: it should increase with $\Delta$, because the event $D\geq c$ becomes more likely as the sampling distribution shifts upward. Assurance should agree with an average of those conditional powers over the design distribution.

When a variance is estimated rather than known, the sampling distribution and posterior calculations may no longer have the normal form used here. Unequal variances, non-normal outcomes, adaptive designs, or non-normal design distributions may also require numerical integration or simulation. Those methods can still implement the same principle—average conditional success probability over the design distribution—but the closed-form normal expression should not be applied without its assumptions.

Conditional Bayesian power is the main alternative when performance at a particular fixed effect is the question. Assurance is appropriate when performance across a distribution of plausible effects is the target. Neither replaces the other: a single assurance value can conceal how power varies over effect sizes, while a conditional power value does not average over uncertainty in the true effect. Report the decision rule, analysis prior, design distribution, sample size and variance assumptions alongside either result so its interpretation is clear.

Common Mistake

Common mistake: Calculating conditional power at the design mean and calling it assurance. That substitutes one effect value for an average over the design distribution. The correction is to evaluate $E_{\Delta}\{\operatorname{Power}(\Delta)\}$; under the normal assumptions here, this is $\Phi((m_d-c)/\sqrt{\omega_d^2+v})$, not $\Phi((m_d-c)/\sqrt{v})$.

Check Your Understanding

Answer each question, then compare your work with the solutions.

In this setting, what distinguishes conditional Bayesian power from assurance?
If $D\mid\Delta\sim N(\Delta,v)$ and the success rule is $D\geq c$, write the conditional power at a fixed $\Delta$.
Suppose $c=1$, $v=1$, and the design distribution is $N(2,1)$. Calculate assurance.
Name two conditions under which the closed-form normal assurance formula may not be appropriate.

Solutions. (1) Conditional power is the success probability at one fixed true effect; assurance averages that probability over a design distribution for the effect. (2) It is $P(D\geq c\mid\Delta)=\Phi((\Delta-c)/\sqrt{v})$. (3) Here $m_d=2$, $\omega_d^2=1$, so $A=\Phi((2-1)/\sqrt{1+1})=\Phi(0.7071)=0.760$. (4) Examples include an estimated rather than known variance, unequal group variances, non-normal outcomes, or a non-normal design distribution; such cases may require a different calculation or numerical integration.

What You'll Learn