Selecting Sample Size From a Target Assurance

From a Target to a Sample Size

A sample-size calculation based on assurance reverses the usual direction of a design evaluation. Instead of asking how likely a proposed trial is to succeed, the design question is: what is the smallest sample size whose assurance reaches a prespecified target? The answer depends on the success rule, the analysis prior, the design prior, and the candidate designs being considered. A target alone does not determine a sample size.

The introductory material on assurance defines it as a success probability averaged over uncertainty in the true parameter under a design prior. Here the central task is selection: calculate that probability for candidate sample sizes and compare it with the target. This comparison needs particular care when outcomes are discrete. A one-person increase in sample size can change the success cutoff, so assurance need not rise at every step.

Definition: For candidate sample size $n$, let $S_n$ be the set of possible data outcomes that satisfy the prespecified Bayesian success rule. If $\theta$ is the parameter and $\pi_D(\theta)$ is the design prior, the assurance is $$ A(n)=\int P_\theta(X\in S_n)\,\pi_D(\theta)\,d\theta. $$ For a discrete design prior with values $\theta_j$ and probabilities $w_j$, this becomes $$ A(n)=\sum_j w_j P_{\theta_j}(X\in S_n). $$ The selected sample size is the smallest allowable $n$ for which $A(n)$ is at least the target $A_{\mathrm{target}}$.

Specify the Decision Rule Before Comparing Designs

The success region $S_n$ is determined by the analysis rule, not chosen after seeing which sample size would pass. For example, a trial might declare success when the posterior probability that a response rate exceeds a clinically relevant benchmark is at least 0.90. For every candidate $n$, this rule is applied to each possible outcome. The resulting success region may be a threshold such as “at least $c_n$ responses,” but $c_n$ can change with $n$.

The following hypothetical single-arm trial illustrates the calculation. Let $p$ be the true response probability, and suppose the response count $X$ among $n$ independent participants satisfies $X\mid p\sim\operatorname{Binomial}(n,p)$. The analysis prior is $p\sim\operatorname{Beta}(1,1)$. The trial succeeds if $P(p>0.25\mid X)\geq 0.90$. For design, suppose $p=0.40$ with probability 0.60 and $p=0.80$ with probability 0.40. These values and probabilities are hypothetical.

The assumptions are that responses are independent conditional on $p$, all participants have the same response probability, the analysis prior and posterior rule are fixed in advance, and the design prior represents the uncertainty used for planning. There is no nuisance parameter in this illustration. If the design involved additional uncertain quantities, their uncertainty would also need to be included in the assurance calculation.

Worked Example: Finding the Success Cutoff for Seven Participants

For $n=7$, determine which response counts satisfy the posterior success rule. After observing $x$ responses, the posterior is $p\mid X=x\sim\operatorname{Beta}(x+1,8-x)$. For integer beta parameters, the beta-binomial identity gives $$ P\{\operatorname{Beta}(a,b)>t\}=P\{\operatorname{Binomial}(a+b-1,t)\leq a-1\}. $$ Here, $a=x+1$, $b=8-x$, and $t=0.25$, so the posterior probability is $P\{\operatorname{Binomial}(8,0.25)\leq x\}$.

At $x=3$, this probability is $P\{\operatorname{Binomial}(8,0.25)\leq3\}=0.8862$, which is below 0.90. At $x=4$, it is $P\{\operatorname{Binomial}(8,0.25)\leq4\}=0.9727$, which is at least 0.90. The posterior probability increases with $x$, so the success region is $S_7=\{4,5,6,7\}$: at least four responses are required.

This calculation shows why a cutoff must be established from the stated posterior rule. It cannot safely be carried over from another sample size.

Compute Assurance at Each Candidate

Once $S_n$ is known, calculate the probability of that region at each design-prior value, then average using the design-prior weights. In the example, if the cutoff is $c_n$, the conditional probability is a binomial upper tail: $$ P_p(X\in S_n)=P_p(X\geq c_n) =\sum_{x=c_n}^{n}{n\choose x}p^x(1-p)^{n-x}. $$ The assurance is $0.60P_{0.40}(X\geq c_n)+0.40P_{0.80}(X\geq c_n)$. It is not the success probability at the average response rate: averaging the conditional probabilities over the prior is the required operation.

Worked Example: Exact Assurance for Seven Participants

The cutoff just derived is four responses. For $p=0.40$, the conditional success probability is $$ P_{0.40}(X\geq4) ={7\choose4}(0.40)^4(0.60)^3 +{7\choose5}(0.40)^5(0.60)^2 +{7\choose6}(0.40)^6(0.60) +(0.40)^7 =0.289792. $$

For $p=0.80$, the corresponding probability is $$ P_{0.80}(X\geq4) ={7\choose4}(0.80)^4(0.20)^3 +{7\choose5}(0.80)^5(0.20)^2 +{7\choose6}(0.80)^6(0.20) +(0.80)^7 =0.966656. $$

Weight these probabilities by their design-prior probabilities: $$ A(7)=0.60(0.289792)+0.40(0.966656) =0.1738752+0.3866624 =0.5605376\approx0.5605. $$ Thus, under this hypothetical design prior and decision rule, the assurance for seven participants is about 0.5605, or 56.05%.

Select the Smallest Candidate, Not Just a Passing Candidate

To establish that a design is the smallest one meeting the target, every smaller allowable candidate must also be checked. For the same hypothetical trial, the posterior rule yields the cutoffs shown below. They are obtained by evaluating the posterior probability for each possible count and including outcomes for which it is at least 0.90. The conditional probabilities use the binomial model; the final column averages them using the design-prior weights 0.60 and 0.40.

Sample size $n$	Success cutoff	$P_{0.40}(X\geq c_n)$	$P_{0.80}(X\geq c_n)$	Assurance $A(n)$
1	At least 1	0.4000	0.8000	0.5600
2	At least 2	0.1600	0.6400	0.3520
3	At least 2	0.3520	0.8960	0.5696
4	At least 3	0.1792	0.8192	0.4352
5	At least 3	0.31744	0.94208	0.5673
6	At least 3	0.45568	0.98304	0.6666
7	At least 4	0.289792	0.966656	0.5605

Worked Example: Selecting a Sample Size for a 0.60 Target

Suppose the prespecified assurance target is $A_{\mathrm{target}}=0.60$, and every positive integer sample size is allowable. The table gives $A(1)=0.5600$, $A(2)=0.3520$, $A(3)=0.5696$, $A(4)=0.4352$, and $A(5)=0.5673$; all are below 0.60. At $n=6$, $$ A(6)=0.60(0.45568)+0.40(0.98304) =0.273408+0.393216 =0.666624. $$ Since $0.666624\geq0.60$, $n=6$ meets the target, and the checks of all smaller positive integers establish that it is the smallest sample size that does so.

The result also shows why checking only a convenient candidate or assuming assurance rises with $n$ is unsafe: $A(6)=0.6666$, but $A(7)=0.5605$. For seven participants the posterior rule requires four responses rather than three. That discrete change in the success region more than offsets the increased chance of observing responses in this example.

Selection rule: Define the allowable candidate set and the success rule first. Compute $A(n)$ for each candidate, then choose the smallest $n$ satisfying $A(n)\geq A_{\mathrm{target}}$. If candidate sizes are restricted, “smallest” means smallest among those allowable designs.

Diagnostics, Alternatives, and Limitations

Before reporting a selected sample size, check that the posterior rule was applied to every possible outcome for each candidate and that the assurance weights sum to one. Recalculate at least one binomial tail directly, and verify that each weighted average lies between its conditional probabilities. In the seven-participant calculation, $0.5605$ must lie between $0.289792$ and $0.966656$. Such checks can catch a wrong cutoff, a missing outcome in a tail, or an incorrect design-prior weight.

A common mistake is to interpolate between candidate sample sizes or to stop at the first candidate that looks promising without checking smaller ones. With discrete data, the success region can jump as $n$ changes, so assurance need not be monotone. The correction is to evaluate the allowable integer candidates in order, using the actual success rule at each one. If no candidate meets the target within the practical range, report that fact rather than silently extending or changing the range.

This exact enumeration is preferable when the outcome distribution and success region can be evaluated directly. A normal approximation can be faster for some large-sample designs, but it may be inaccurate near a discrete cutoff or for rare outcomes; it should be checked against exact calculations when feasible. Simulation is useful for complex models where exact evaluation is impractical, but a simulated estimate has Monte Carlo error and should not be treated as an exact assurance. These approaches are alternatives for evaluating a candidate, not substitutes for defining the decision rule and searching the candidate set.

The answer is conditional on the chosen analysis prior, design prior, model, success criterion, and allowable designs. A sample size meeting an assurance target does not guarantee that the trial will succeed, and it does not establish that the design assumptions are clinically appropriate. If assurance is close to the target, numerical precision and uncertainty in estimated assurance matter: a rounded value just above the target should be checked using unrounded calculations. A design may also need to satisfy operational or ethical constraints beyond assurance.

Check Your Understanding

Answer each question, then compare with the solutions.

Why must the success region be determined separately for each candidate sample size?
In the seven-participant example, why is the assurance not $P_{0.40}(X\geq4)$ alone?
For a target of 0.60 in the hypothetical trial, why is $n=7$ not selected even though it has more participants than $n=6$?
If only sample sizes that are multiples of two are allowable, which of $n=4$ and $n=6$ is the smallest candidate meeting a target of 0.60?

Solutions. (1) The posterior success rule is applied to the possible data at that $n$; discrete outcomes can make the resulting response-count cutoff change with $n$. (2) Assurance averages conditional success probabilities over the design prior: $0.60P_{0.40}(X\geq4)+0.40P_{0.80}(X\geq4)=0.5605$. (3) At $n=7$, the cutoff rises to four responses and the computed assurance is 0.5605, below 0.60; assurance is not guaranteed to increase at each integer step. (4) $n=4$ has assurance 0.4352 and fails, whereas $n=6$ has assurance 0.6666 and passes, so $n=6$ is the smallest allowable candidate among the two.