Why Variance Uncertainty Matters
A sample-size calculation uses a planned within-subject standard deviation, but that value is rarely known precisely. It may come from a small pilot study, a historical study, or a pooled estimate that is only partly comparable to the planned study. If the true variability is higher than the planning value, the standard error will be larger and power will be lower. Thus, a study can have adequate power at the assumed variance while still having a meaningful risk of falling short of its target.
This tutorial focuses on that risk for average bioequivalence assessed on a log scale. Let \(D\) denote a subject's test-minus-reference log outcome for a crossover comparison, and let \(\sigma_D\) be the population standard deviation of those within-subject differences. For simplicity, the worked calculations use a centered true effect, \(E(D)=0\), and a normal approximation to the two one-sided tests (TOST). They isolate uncertainty in \(\sigma_D\); they do not account for uncertainty in the true treatment effect, dropout, or other design features.
Power as a Function of Within-Subject Variance
Let the equivalence limits for the mean log difference be \(-\Delta\) and \(+\Delta\), with \(\Delta>0\), and let \(\alpha\) be the one-sided significance level for each TOST component. Under the centered-effect assumption, a normal-approximation calculation with \(n\) evaluable subjects gives
where \(z_{1-\alpha}\) is the \(1-\alpha\) standard normal quantile. This expression applies when the acceptance interval is symmetric about the true mean. It is a planning approximation: small-sample calculations commonly use a \(t\) distribution and may have power that differs from this formula.
The formula makes the direction of the effect clear. Increasing \(\sigma_D\) reduces \(\Delta\sqrt{n}/\sigma_D\), so conditional power falls. A useful diagnostic is the variance threshold at which power equals a target \(P^*\). Rearranging the power expression gives
For a centered effect, conditional power is below \(P^*\) whenever \(\sigma_D\) exceeds this threshold. This translates a power target into a variance question: how plausible is it, under the planning information, that the true standard deviation is above the threshold?
Worked Example: Scenario-Based Underpowering Risk
Suppose a hypothetical design uses \(\alpha=0.05\), and its planned margin and sample size give \(\Delta\sqrt{n}=0.77284\). Consider three plausible values of \(\sigma_D\), with planning weights 0.20, 0.50, and 0.30 for 0.20, 0.25, and 0.30 log units, respectively. These weights are stipulated for this illustration; they are not estimated probabilities from a clinical study.
For each scenario, use \(z_{0.95}=1.64485\). At \(\sigma_D=0.20\), the normal-score argument is \(0.77284/0.20-1.64485=2.21935\), so power is \(2\Phi(2.21935)-1\approx0.9735\). At \(\sigma_D=0.25\), the argument is \(0.77284/0.25-1.64485=1.44651\), giving power \(2\Phi(1.44651)-1\approx0.8520\). At \(\sigma_D=0.30\), the argument is \(0.77284/0.30-1.64485=0.93128\), giving power \(2\Phi(0.93128)-1\approx0.6483\).
If the target is 80% power, only the third scenario falls below target. The scenario-weighted underpowering risk is therefore \(0.30\), or 30%. The weighted average conditional power is \(0.20(0.9735)+0.50(0.8520)+0.30(0.6483)=0.81519\), approximately 0.8152, or 81.52%. That average is above 80%, but it does not remove the 30% scenario-based risk of falling below the target. These are different summaries and should not be substituted for one another.
Using a Pilot Estimate to Represent Variance Uncertainty
A pilot standard deviation is itself noisy. Under the assumption that independent pilot differences are normally distributed with common variance \(\sigma_D^2\), the sample variance \(s_D^2\) satisfies
where \(\nu\) is the degrees of freedom for the variance estimate (for a simple sample of independent differences, \(\nu=n_{\mathrm{pilot}}-1\)). This relationship provides a confidence limit for \(\sigma_D\). If \(\chi^2_{q,\nu}\) denotes the \(q\) quantile of the chi-square distribution with \(\nu\) degrees of freedom, a one-sided \(100(1-\gamma)\%\) upper confidence limit is
The lower-tail quantile appears in the denominator because a small chi-square value corresponds to a relatively large compatible population variance. This upper limit is a frequentist confidence bound, not a statement that the unknown standard deviation has a \(1-\gamma\) probability of lying below it. Choosing it for planning is a conservative decision rule: calculate power at the bound and assess whether that power is acceptable.
Worked Example: Planning at an Upper Confidence Limit
Suppose a hypothetical pilot has \(n_{\mathrm{pilot}}=21\) independent within-subject log differences and \(s_D=0.25\) log units. Thus, \(\nu=20\). For a one-sided 90% upper confidence limit, use \(\gamma=0.10\) and \(\chi^2_{0.10,20}\approx12.443\).
The estimated variance is \(s_D^2=0.25^2=0.0625\). The upper limit is \[ \sigma_{D,U}=\sqrt{\frac{20(0.0625)}{12.443}} =\sqrt{0.10046}\approx0.3170 \] log units. Under the normal-approximation design in the preceding example, power at this variance is \(2\Phi(0.77284/0.3170-1.64485)-1\). The argument is approximately \(0.793\), giving power of about \(2\Phi(0.793)-1=0.572\), or 57.2%.
At the pilot point estimate of 0.25, the corresponding power was about 85.20%. The large difference shows how a small pilot can leave substantial uncertainty even when its point estimate appears reassuring. It does not prove that the planned study has 57.2% power: the calculation is conditional on using the upper confidence limit as the planning value. Rather, it shows the consequence of that conservative choice under the stated model.
From a Target Power to a Conservative Sample Size
For a chosen variance value, the normal-approximation formula can be rearranged to find the sample size needed for target power \(P^*\):
Because sample size must be an integer, a planning result is rounded up. The formula also shows why variance uncertainty matters disproportionately: holding other quantities fixed, the required sample size is proportional to \(\sigma_D^2\). A 10% increase in the planning standard deviation therefore increases the unrounded sample size by about 21%, not 10%.
Worked Example: Comparing Point-Estimate and Conservative Planning
Continue with the hypothetical pilot above. Suppose the log equivalence limits are \(\pm\Delta\), where \(\Delta=\ln(1.25)=0.22314\), and the target is \(P^*=0.80\) at \(\alpha=0.05\). Since \(z_{0.95}=1.64485\) and \(z_{0.90}=1.28155\), their sum is 2.92640.
Planning at the pilot estimate \(\sigma_D=0.25\) gives \[ n\approx\left[\frac{0.25(2.92640)}{0.22314}\right]^2 =\left[\frac{0.73160}{0.22314}\right]^2 \approx10.75. \] Rounding up gives 11 evaluable subjects under this approximation. Planning instead at the 90% upper confidence limit \(\sigma_{D,U}\approx0.3170\) gives \[ n\approx\left[\frac{0.3170(2.92640)}{0.22314}\right]^2 =\left[\frac{0.9277}{0.22314}\right]^2 \approx17.28, \] so the conservative choice gives 18 evaluable subjects. These are normal-approximation calculations; an operational design should use the appropriate small-sample method and account separately for expected nonevaluable profiles or dropout.
The comparison quantifies the design cost of protecting against a higher variance. It does not establish that the upper-limit choice is always preferable. The confidence level, pilot relevance, budget, and consequences of missing the power target all matter. A sensitivity table over several defensible variance values can make that trade-off more transparent than a single chosen value.
Assumptions, Diagnostics, and Alternative Choices
The calculations above rely on a clearly defined variance: the standard deviation of the within-subject log differences relevant to the planned endpoint and design. The pilot and planned study should be sufficiently comparable in population, formulation, endpoint definition, sampling, and analysis. The simple chi-square interval additionally assumes independent, approximately normal pilot differences with a common variance. If those assumptions are doubtful, its nominal coverage may not be a useful description of uncertainty.
Check that the variance estimate is on the log scale and corresponds to the exact endpoint being powered. AUC and Cmax may have different within-subject variability, so one variance should not be transferred between them without justification. Inspect the pilot differences for outliers, skewness, unusual period or sequence patterns, and data-quality issues. A single extreme value can inflate \(s_D\); removing it without a prespecified, scientifically defensible reason can instead understate variability. Also verify that the sample size refers to evaluable subjects, not merely subjects enrolled.
Three common planning choices answer different questions:
| Choice | What it does | Main limitation |
|---|---|---|
| Plug-in estimate | Calculates power or sample size at the pilot or historical point estimate. | Does not directly represent estimation uncertainty; a small estimate can give false reassurance. |
| Upper confidence limit | Uses a selected upper bound as a conservative planning variance. | Conservatism depends on the chosen confidence level and the relevance of the data supporting the bound. |
| Scenario analysis | Reports conditional power across plausible variance values and, if justified, a weighted underpowering risk. | Results depend on scenario selection and any assigned weights; sparse scenarios can miss important values. |
A useful practice is to show both a central estimate and a sensitivity range, then state which variance value drives the final sample size and why. The introductory material on evaluating variance estimates for sample-size planning covers the assessment of candidate sources; here the emphasis is on propagating uncertainty in a selected variance into power and sample-size consequences. These calculations also differ from sample-size re-estimation: they concern planning under uncertainty before the study, rather than updating a design during a study.
Common Mistakes and Practical Limits
- Treating the point-estimate power as guaranteed. Power calculated at \(s_D\) is conditional on that value being the true standard deviation. It does not summarize the uncertainty in the estimate.
- Calling average power the probability of meeting the target. An average such as 81.52% is an average of conditional powers under stipulated weights. The probability that conditional power is below 80% is a separate calculation; in the scenario example it is 30%.
- Using an upper confidence limit as though it were a probability distribution. The limit is a frequentist bound with a coverage interpretation across repeated pilot studies. Treating it as a planning value is a conservative choice, not a claim about the probability that the true variance equals that value.
- Applying a pilot variance from a different endpoint or design. A variance is meaningful for planning only when its measurement scale and source are relevant to the intended analysis.
- Assuming the normal approximation is exact. Small samples, nonzero true effects, or more complex designs can require a different power calculation. Use a suitable exact or model-based method when the approximation is inadequate.
No single variance choice eliminates planning risk. A conservative bound may increase cost and still fail to protect against variance values beyond that bound; a point estimate may be efficient but expose the study to substantial underpowering risk. The decision should make the uncertainty visible, use assumptions that can be defended, and distinguish the consequences of plausible variance values from the uncertainty in the treatment effect.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In the centered-effect normal approximation, what happens to conditional power as \(\sigma_D\) increases while \(n\) remains fixed?
- A design has conditional power 88% at one variance scenario and 62% at another. The scenarios have weights 0.75 and 0.25, and the target is 80%. What are the weighted average power and scenario-based underpowering risk?
- Why does a one-sided upper confidence limit for \(\sigma_D\) use a lower-tail chi-square quantile in its denominator?
- Why does a 10% increase in the planned standard deviation require more than a 10% increase in sample size, approximately?
Solutions. (1) Power decreases because \(\Delta\sqrt{n}/\sigma_D\) decreases. (2) Weighted average power is \(0.75(0.88)+0.25(0.62)=0.815\), or 81.5%; only the second scenario is below 80%, so the underpowering risk is 25%. (3) A smaller chi-square value is compatible with a larger population variance in \(\nu s_D^2/\sigma_D^2\); dividing by the lower quantile therefore gives an upper bound for \(\sigma_D\). (4) Required sample size is proportional to \(\sigma_D^2\), so multiplying the standard deviation by 1.10 multiplies the unrounded sample size by \(1.10^2=1.21\), an increase of about 21%.