Diagnosing Implausible Sample Size Results

Why a Plausible-Looking Output May Be Wrong

A sample-size program can return a precise integer even when one of its inputs is on the wrong scale. For a bioequivalence study, an especially small result may reflect a variance entered as a standard deviation, a coefficient of variation treated as a log-scale variance, or an assumed test-to-reference ratio set too close to the centre of the equivalence interval. A large result can arise from the converse errors, an overly narrow margin, or an inappropriate degrees-of-freedom calculation. The output is not self-validating: the design, assumptions, and meaning of the number must all be checked.

This tutorial uses a standard two-period, two-sequence crossover as a diagnostic reference. The checks apply more broadly, but the variance and degrees of freedom depend on the design and analysis model. A rough calculation can reveal scale errors; it does not replace a power calculation matched to the planned analysis.

Start with the Scale of Variability

A coefficient of variation (CV) is usually reported as a proportion or percentage on the original measurement scale. It is not the variance of the log-transformed endpoint. Under a lognormal model, if the original-scale CV is \(CV\), the corresponding log-scale variance and standard deviation are

The CV must be entered as a proportion: 30% is \(0.30\), not \(30\). For small CVs, \(\ln(1+CV^2)\) is approximately \(CV^2\), but this is a shortcut, not the exact conversion. The approximation becomes less accurate as CV increases. Also check the software field name: some procedures request a variance, others a standard deviation, and others a CV. Entering the correct number in the wrong field can change the result substantially.

A Screening Calculation and Its Assumptions

For a quick diagnostic in a balanced two-period crossover, assume independent participants, a common within-subject residual variance, no missing evaluable outcomes, and a treatment-difference estimator whose variance is \(2\sigma_w^2/n\). Assume the log-scale equivalence limits are fixed, and that the planned test uses two one-sided tests at significance level \(\alpha\). A useful normal-approximation screen for target power \(1-\beta\) is

Here \(z_q\) is the \(q\)-quantile of the standard normal distribution, and \(d\) is the distance from the assumed \(\theta\) to the nearer limit. This approximation focuses on the nearer boundary: it asks that the estimated difference clear that boundary with the specified power, while using a normal critical value. It is a screening calculation, not an exact finite-sample solution to the joint two-one-sided-test power problem. Its assumptions and simplifications explain why its result can differ from a design-specific calculation.

A short justification

The estimated difference has approximate standard error \(SE=\sqrt{2\sigma_w^2/n}\). At the nearer boundary, the assumed effect is \(d\) log units inside the limit. To obtain approximate power \(1-\beta\) for that limiting one-sided test, the effect in standard-error units must be about \(z_{1-\alpha}+z_{1-\beta}\). Thus

and squaring and rearranging gives the screening formula. The farther limit is not used to calculate \(d\); the minimum distance is the limiting one. If \(\theta\) is on or outside either equivalence limit, \(d\leq 0\), and this formula cannot give a meaningful positive-power planning result for the assumed effect.

Check the Margins and the Assumed Ratio

The equivalence limits are assessed on the log scale, not by treating a percentage difference as a symmetric distance around every possible assumed ratio. With limits \(L\) and \(U\), calculate both \(\theta-L\) and \(U-\theta\), then use the smaller distance. An assumed ratio closer to one margin than the other reduces \(d\), increasing the required sample size. If the assumed ratio is outside the limits, a sample size cannot make that assumed true effect equivalent.

Audit Degrees of Freedom, Design, and Enrollment

The normal screen uses normal quantiles and does not account for the finite-sample \(t\) distribution. In a common balanced two-sequence, two-period crossover analysis, residual degrees of freedom are often \(n-2\), but this is not universal: the analysis model, sequence allocation, missing observations, and design determine the relevant degrees of freedom. A result that reports an \(n\)-dependent \(df\) inconsistent with the planned model deserves investigation. Small studies are particularly sensitive to the difference between a normal critical value and a \(t\) critical value.

For final planning, calculate power using the analysis model and variance structure intended for the study. For a simple crossover, this generally means using the appropriate finite-sample test distribution and determining power for both one-sided tests, rather than treating the screening formula as exact. A noncentral-\(t\) calculation or simulation can accommodate the finite-sample test and model details; simulation is especially useful when the design or analysis is more complex. The result should also be checked by evaluating power at the proposed integer sample size and at the preceding feasible size.

Finally, distinguish evaluable participants from participants to enroll. If 22 evaluable participants are required and the planned attrition fraction is 15%, a basic inflation is

Twenty-six also preserves equal allocation to the two sequences. This inflation targets an expected evaluable count under the assumed attrition rate; it does not guarantee that 22 participants will complete the study. The attrition assumption should be justified, and the resulting enrollment should be rounded to a number compatible with the design.

Diagnostic Checklist and Common Pitfalls

When an output appears implausibly small or large, audit it in a fixed order. Recalculate key inputs independently rather than changing them until the answer looks familiar.

A common mistake is to reason, “The CV is 30%, so the variance input is 0.30.” That confuses a percentage-scale variability measure with a log-scale variance. The correction is to express the CV as \(0.30\), apply \(\sigma_w^2=\ln(1+0.30^2)=0.08618\), and enter the result only if the procedure requests log variance. If it requests log standard deviation, use \(0.2936\); if it requests CV, use \(0.30\).

A useful cross-check is directional: holding other inputs fixed, greater variability, less power, a stricter significance level, or a smaller distance to the nearest limit should not reduce the required sample size. A change in the opposite direction may indicate a coding, scale, or parameterization error. This check can reveal errors but cannot establish that the model is appropriate.

What the Screen Can and Cannot Establish

The screening formula assumes a fixed variance, a specified true effect, independent participants, and the stated crossover variance structure. It does not by itself handle uncertainty in the variance estimate, nonadherence, unequal or missing data, or an analysis different from the assumed one. A suspicious result is a reason to diagnose inputs and assumptions, not evidence that a particular sample size is inherently right or wrong.