Type I Error Control in Adaptive Sample Size Re-Estimation

Why Adaptation Changes the Error-Control Question

A sample-size re-estimation may change only the planned number of subjects, yet it can change the sampling distribution of the final test statistic. The key question is therefore not simply whether the adaptation used a blinded variance estimate. It is whether the complete procedure—including the rule for choosing the sample size and the final analysis—controls the chance of declaring equivalence when equivalence is false.

For average bioequivalence, let \(\theta\) be the population mean difference between test and reference on the log scale, and let \(L\) and \(U\) be the lower and upper equivalence limits, with \(L<U\). The alternative is that the mean difference lies inside the limits. In the usual two one-sided tests procedure (TOST), equivalence is declared only if both component null hypotheses are rejected.

The component hypotheses are \(H_{01}:\theta\leq L\) versus \(H_{a1}:\theta>L\), and \(H_{02}:\theta\geq U\) versus \(H_{a2}:\theta<U\). With a correctly calibrated fixed-sample test, each one-sided test is conducted at level \(\alpha\). Since equivalence requires rejecting both, the probability of declaring equivalence under either part of the null is no greater than the rejection probability of the corresponding component test, and thus no greater than \(\alpha\).

What Must Hold After Sample-Size Re-Estimation

Let \(A\) denote the outcome of the adaptation rule: for example, the selected sample size or a category of interim variance estimates. After adaptation, the final rejection rule may have a different conditional distribution for each value of \(A\). A direct sufficient condition for type I error control is that, for every possible adaptation outcome, the conditional probability of declaring equivalence is at most \(\alpha\) throughout each component of the null.

If the conditional probability inside the integral is at most \(\alpha\) for all \(a\) and every \(\theta\) in the null region, its average is also at most \(\alpha\). A less restrictive sufficient route is to establish conditional size at the relevant boundary and also establish monotonicity: for every adaptation outcome, the lower component test’s conditional rejection probability must be nondecreasing as \(\theta\) moves from the lower null region toward its alternative; for the upper component, it must be nonincreasing as \(\theta\) moves from the upper null region toward its alternative. Then the largest conditional rejection probability over each null component occurs at its boundary. Boundary control alone, without this monotonicity or control throughout the component, is not enough.

These are conditions on the conditional test, not merely on the fixed-sample test used after selecting a sample size. Conditioning on an adaptation outcome can alter the distribution of the statistic. A nominal critical value that is valid for a fixed sample size need not remain valid conditional on how that sample size was chosen.

Why Blinding Is Not a Complete Guarantee

A blinded variance-based rule avoids using the observed treatment difference as an explicit input to the sample-size decision. That can reduce the risk of directly selecting a larger study when interim results look favorable. It does not, by itself, prove type I error control. The relevant questions include whether the variance statistic is independent of the treatment-effect statistic under the model, whether the final statistic has its assumed distribution after adaptation, and whether the analysis accounts for any dependence created by using interim data both to adapt and to test.

For example, an independently conducted variance-only pilot can select the size of a subsequent confirmatory study. If the pilot outcome is independent of all confirmatory treatment-effect data, then conditional on the selected size the confirmatory data still follow the planned fixed-sample model. A different situation arises if the interim subjects contribute to the final estimate and their data also determine the sample size. In that case, validity needs to be established for the combined adaptive procedure rather than inferred from blinding.

A Boundary Check Is Not Always Enough

The monotonicity requirement matters because an adaptation outcome can select data with unusual properties. The following hypothetical calculation shows the logical gap; it is not a model of a particular bioequivalence test.

Methods for Protecting Type I Error

Several approaches can make an adaptive analysis defensible. The appropriate choice depends on what data inform the adaptation, what data enter the final estimate, and whether the planned test’s conditional distribution can be derived.

• Use an independent nuisance-parameter estimate. A separate pilot or genuinely independent variance source can select the confirmatory sample size while leaving the confirmatory test distribution unchanged, conditional on the selected size.
• Derive and use a conditionally calibrated test. The critical value or p-value is chosen to control the conditional error given the interim information. This requires a valid model for the conditional distribution, not simply reusing the fixed-sample critical value.
• Use a prespecified combination or sequential procedure. When interim data contribute to the final test, a method designed for the stages and decision rule can account for the information used at adaptation. Its error control must match the equivalence hypotheses and the actual adaptation rule.
• Keep the original fixed-sample design. This avoids adaptation-related distribution changes but may leave the study underpowered if the planning variance was inaccurate.

These approaches are not interchangeable shortcuts. A combination method or conditional-error method must be specified for the relevant test, adaptation timing, and data structure. An adjustment designed for a single one-sided test should not be assumed to control a two-component equivalence procedure without checking both components.

Diagnostics, Failure Modes, and Limitations

Before adopting a variance-based re-estimation, specify the adaptation rule, the maximum and minimum sample sizes, the final analysis, and the intended type I error argument. Then check whether the assumptions behind that argument match the actual design. In particular, distinguish an independent variance pilot from an interim estimate calculated using subjects whose treatment-effect information will later be analyzed.

Simulation under null parameter values is a useful diagnostic when analytic conditional distributions are difficult. Simulate the entire process—including the adaptation rule, sample-size caps, any rounding or allocation constraints, missing data handling, and final test—at the lower boundary, the upper boundary, and representative points farther into both null components. Estimate the probability of declaring equivalence at each value. Simulation can reveal inflation or implementation errors, but a finite simulation is not proof of control: its Monte Carlo uncertainty must be considered, and it cannot establish behavior for unexamined parameter values without additional reasoning.

Common failure modes include using an unadjusted fixed-sample test after an effect-informed sample-size decision; checking error only at the two boundaries without establishing conditional monotonicity; and treating blinding as a mathematical guarantee of independence. Another failure is to simulate only the boundary values while the procedure has a non-monotone rejection probability in the null interior. If these checks do not support a valid argument, the design or analysis needs adjustment rather than an assertion that the nominal level is preserved.