Two Priors, Two Different Roles
Assurance is a probability of trial success averaged over uncertainty about the true treatment effect. That description involves two stages: the trial data are analyzed to decide whether the result meets a success rule, and the success probability is averaged over plausible true effects. A prior distribution used in the first stage need not be the same as the distribution used in the second.
The analysis prior enters the posterior distribution after data are observed. It therefore affects the posterior probability used in a Bayesian decision rule. The design prior describes uncertainty about the true parameter before the trial for purposes of planning; it weights the conditional success probabilities across possible true parameter values. Keeping these roles separate makes it possible to ask two different questions: “How will the data be interpreted?” and “Across which true effects are we averaging?”
The two distributions can coincide, but doing so is a modeling choice, not a requirement. If they differ, report both and explain their purposes. In particular, the design prior is not automatically the prior used to analyze a future trial, and an analysis prior is not automatically an appropriate distribution over true effects for planning.
Normal-Outcome Setup and Posterior Decision Rule
Suppose \(Y\) is an estimate of a treatment-control mean difference, measured in points, with known standard error \(s\). Assume the sampling model \(Y\mid\theta\sim N(\theta,s^2)\), with observations generated according to this model for each fixed true effect \(\theta\). For analysis, use the proper prior \(\theta\sim N(m_a,v_a)\), where \(v_a>0\). The normal likelihood and normal prior give a normal posterior.
For a one-sided success rule, let \(\delta\) be the minimum effect of interest and let \(\gamma\) be the required posterior probability, with \(0<\gamma<1\). The trial succeeds when \(P(\theta>\delta\mid Y)>\gamma\). Write \(q=s^2v_a/(s^2+v_a)\) for the posterior variance and \(z_\gamma\) for the \(\gamma\) quantile of the standard normal distribution. Since the posterior is normal, the rule is equivalent to requiring its mean to exceed \(\delta+z_\gamma\sqrt q\). The decision rule can therefore be written as a threshold on \(Y\):
Here \(c_a\) depends on the analysis prior and the specified posterior decision rule. It does not depend on the design prior. Once \(c_a\) is fixed, the conditional success probability at a fixed true effect is \(P_\theta(Y>c_a)\). If the design prior is \(g_d\), assurance averages that probability over \(g_d\). In this known-variance normal model, if \(\theta\sim N(m_d,v_d)\) under the design prior, then \(Y\) has the prior-predictive distribution \(N(m_d,v_d+s^2)\), giving:
Worked Examples: Separating the Roles in Practice
Worked Example: Calculate the Decision Threshold and Assurance
Hypothetical design. Suppose the estimated treatment effect \(Y\) is measured in points, \(s=0.50\) points, and the analysis prior is \(\theta\sim N(0,1)\), with prior variance \(1\) point². The trial is a success if \(P(\theta>0\mid Y)>0.95\). For planning, use a separate design prior \(\theta\sim N(0.50,0.16)\), with mean \(0.50\) points and variance \(0.16\) point². The normal sampling model with known standard error is assumed.
The posterior variance is \(q=(0.50^2\times1)/(0.50^2+1)=0.25/1.25=0.20\) point². Its standard deviation is \(\sqrt{0.20}=0.4472\) points, and the posterior mean is \(0.80Y\), because \(w=1/(1.25)=0.80\) and \(m_a=0\). With \(z_{0.95}=1.6449\), success requires \(0.80Y>1.6449\sqrt{0.20}\), so the data threshold is \(c_a=1.6449\sqrt{0.20}/0.80\approx0.9195\) points.
Under the design prior, the marginal distribution of \(Y\) is normal with mean \(0.50\) points and standard deviation \(\sqrt{0.16+0.25}=\sqrt{0.41}=0.6403\) points. Thus:
The assurance is approximately 25.6%. This is the design-prior-averaged probability that the future estimate exceeds the threshold imposed by the analysis rule. It is not the posterior probability that the treatment effect is beneficial.
Worked Example: Change the Design Prior, Hold the Analysis Rule Fixed
Keep the sampling model and analysis prior from the previous calculation, so the success threshold remains \(0.9195\) points. Now suppose a planning team uses the design prior \(\theta\sim N(0.80,0.25)\), with mean \(0.80\) points and variance \(0.25\) point². This is an alternative planning assumption; the analysis prior and posterior decision rule have not changed.
The marginal standard deviation of \(Y\) is \(\sqrt{0.25+0.25}=\sqrt{0.50}=0.7071\) points. Therefore:
The assurance is about 43.3%, larger than in the first example. The increase follows because the alternative design prior places more probability on larger positive effects, which are more likely to produce \(Y>0.9195\). The posterior decision rule itself has not become less stringent; only the distribution used to average its conditional success probability has changed.
Worked Example: Change the Analysis Prior, Hold the Design Prior Fixed
Return to the design prior \(\theta\sim N(0.50,0.16)\) and retain \(s=0.50\) points and the success rule \(P(\theta>0\mid Y)>0.95\). Change only the analysis prior to \(\theta\sim N(0,0.25)\), with variance \(0.25\) point². The normal model assumptions remain the same, but the posterior now gives the observed estimate less weight than it did under the \(N(0,1)\) analysis prior.
The posterior variance is \(q=(0.25\times0.25)/(0.25+0.25)=0.125\) point², and the posterior mean is \(0.50Y\). Thus success requires \(0.50Y>1.6449\sqrt{0.125}\), giving \(c_a=1.6449\sqrt{0.125}/0.50\approx1.1631\) points. The design-prior predictive standard deviation of \(Y\) remains \(\sqrt{0.16+0.25}=0.6403\) points, since the design prior and sampling model are unchanged.
The assurance is about 15.0%. The more concentrated analysis prior centered at zero pulls posterior estimates toward zero and raises the data threshold for meeting the same posterior-probability rule. The design prior still averages over the same distribution of true effects, but the event being averaged—crossing the analysis threshold—has changed.
Why the Nested Averaging Works
The calculation has a direct probability justification. Imagine first drawing a true effect \(\theta\) from the design prior and then generating trial data from the sampling model conditional on that effect. For each fixed \(\theta\), the chance of success is the conditional probability \(P_\theta(\text{success})\). Averaging these conditional probabilities over the design prior gives the overall probability of success under this planning model.
Choose an analysis prior, likelihood, and posterior decision rule. These determine which possible data sets count as success.
For a fixed true \(\theta\), calculate the probability that data generated under the sampling model meet the decision rule.
Weight the conditional probabilities by the design prior \(g_d(\theta)\) and integrate over \(\theta\).
For a general success event \(S\) determined by the observed data, the same logic is:
The indicator \(I\{y\in S\}\) equals one when the data meet the rule and zero otherwise. In the normal example, \(S\) is the event \(Y>c_a\), and the marginal normal distribution of \(Y\) gives a convenient closed-form calculation. The design prior appears in the outer averaging; the analysis prior determines the event \(S\) through the posterior rule.
Assumptions, Diagnostics, and Alternatives
The closed-form normal calculation relies on a known standard error that correctly describes the sampling variability, a normal sampling distribution for \(Y\) conditional on \(\theta\), and a normal analysis prior with positive variance. It also assumes that the success criterion is exactly the stated posterior-probability threshold and that the design prior represents the planning distribution over true effects. If the analysis prior is improper, the posterior may still be defined in some settings, but it cannot be treated as a proper probability distribution over possible effects for prior averaging without additional justification.
Before interpreting an assurance value, check that the treatment effect, prior means, and standard error use the same units; that prior variances use squared units; and that the posterior threshold has been calculated from the specified analysis prior and criterion. A useful diagnostic is to calculate \(c_a\) directly from the posterior rule and then verify that the assurance calculation uses that same threshold. Also inspect whether the design prior puts substantial probability on effects that are clinically implausible or inconsistent with the population the trial is intended to address.
| Planning approach | What is held fixed? | Question answered |
|---|---|---|
| Conditional Bayesian power | A particular true effect \(\theta\) | How likely is the rule to succeed if that fixed effect is true? |
| Assurance | A design prior over true effects | What is the success probability averaged over the specified distribution of true effects? |
| Posterior decision analysis | Observed data and the analysis prior | Given these data, what posterior probability supports the decision? |
A frequentist power calculation at a specified effect also conditions on a fixed parameter, but its rejection rule and error-rate interpretation need not match a Bayesian posterior decision rule. Assurance differs from conditional power because it averages over a distribution of effects; that averaging does not make it a posterior probability that the trial’s treatment effect is favorable. A major limitation is that assurance can be highly sensitive to the design prior, while the actual data analysis may use a different prior. A single value should therefore be accompanied by a clear account of both priors and the decision rule.
Common Mistake
Another error is to interpret assurance as the probability that the treatment works after observing trial data. Assurance is calculated before observing the future trial data under the assumed sampling model and design prior. It is an average probability of meeting a prespecified rule, not a claim that a particular future result or parameter value has that probability after the data are observed.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In one sentence each, state the role of the analysis prior and the design prior.
- If the design prior changes but the sampling model, analysis prior, and success rule remain fixed, which part of the normal assurance calculation changes?
- For \(Y\mid\theta\sim N(\theta,s^2)\), what event defines success when the posterior rule is \(P(\theta>\delta\mid Y)>\gamma\)?
- Why is an assurance of 25.6% not a 25.6% posterior probability that the treatment effect is positive?
Solutions. (1) The analysis prior is used to form the posterior and apply the decision rule; the design prior weights possible true effects when averaging conditional success. (2) The predictive distribution of \(Y\), and hence the prior average, changes; the threshold \(c_a\) remains fixed because the analysis rule is unchanged. (3) Success is the event \(Y>c_a\), where \(c_a=[\delta+z_\gamma\sqrt q-(1-w)m_a]/w\), \(q=s^2v_a/(s^2+v_a)\), and \(w=v_a/(s^2+v_a)\). (4) Assurance averages the chance that future data meet a rule before those data are observed; a posterior probability is conditional on observed data and the analysis prior.