Adaptive Allocation and Null Rejection

Why Allocation Changes Matter

An interim review may lead a trial team to assign more future participants to one treatment arm than another. This can be useful operationally, but it changes the path by which the final data are generated. If the allocation decision responds to interim outcomes, the final sample sizes and the interim results are statistically dependent. A familiar test statistic may therefore no longer have the null distribution it would have had under a fixed allocation plan.

The relevant question is not simply whether the final treatment groups have similar sample sizes, or whether the final analysis uses all observations. It is whether the rejection rule has a controlled probability of rejecting a true null under the actual adaptive design. This tutorial focuses on a single treatment-effect hypothesis; multiplicity across several claims requires its own error control.

A Two-Stage Model and Its Assumptions

Consider a two-arm trial with a continuous outcome. In a simple working model, \(Y_{ij}=\mu+\tau A_{ij}+\varepsilon_{ij}\), where \(A_{ij}=1\) denotes treatment, \(A_{ij}=0\) denotes control, \(\mu\) is a common mean under the null, and \(\tau\) is the treatment effect. The null hypothesis is \(H_0:\tau=0\). For the calculations below, assume independent normally distributed errors with mean zero and known variance \(\sigma^2\). These assumptions make the conditional distribution calculations transparent; they are not a claim that all clinical outcomes follow this model.

Assume the first-stage participants are randomized as planned, the interim allocation decision is a prespecified measurable function of the interim data, and second-stage outcomes are independent of the first-stage outcomes conditional on the allocation decision. Under \(H_0\), treatment assignment does not change the outcome distribution. We also assume the endpoint, outcome definition, and null hypothesis remain fixed. The results below concern the effect of changing allocation, not changes to those other design features.

A fixed-design test is calibrated for a specified allocation and its resulting sampling distribution. Once sample sizes depend on interim data, conditioning on the chosen allocation also conditions on information about the outcomes. Consequently, the final statistic’s conditional distribution need not be the fixed-design distribution. Even if a statistic has mean zero over all possible trial paths under \(H_0\), it does not follow that its standardized version has the reference distribution used by a conventional test.

Conditional Error Gives a Validity Criterion

The conditional error of an original design at interim data \(x\) is the probability, under a true null, that the original design would reject after continuing according to its planned rule:

where \(\mathcal{F}\) denotes the information available at interim. A continuation design may change allocation, but it must account for the null rejection probability of that changed continuation. A sufficient condition for preserving the original Type I error bound is that, for every possible interim state \(x\), the new continuation’s conditional rejection probability is at most \(A(x)\). More generally, it may be bounded by another valid conditional error limit \(A^*(x)\), provided \(E_0[A^*(\mathcal{F})]\leq\alpha\).

The justification is the law of total probability. The event of rejection can be partitioned by the interim information, so its probability is the average of its conditional probabilities. Applying the conditional bound and then the bound on its expectation gives the result:

This argument does not require the final test statistic to have the same distribution it would have had under fixed allocation. It requires a valid bound on conditional rejection probability under the actual continuation. The conditional error must be calculated for the original rule and the relevant null; it is not automatically equal to \(\alpha\) at every interim outcome.

Randomization-Based Alternatives and Their Scope

One alternative to applying a fixed-allocation reference distribution is to base inference on the randomization mechanism actually used. For example, suppose participant \(i\) is assigned treatment with probability \(\pi_i\), determined from information available before that assignment. Under a sharp null of no treatment effect for any participant, the outcome \(Y_i\) is unaffected by assignment. If outcomes are centered at a known null mean \(\mu_0\), define the assignment score

At each assignment, \(A_i\) is randomized with conditional probability \(\pi_i\), so \(E_0(A_i-\pi_i\mid\text{past})=0\). Under the sharp null, the current outcome is unaffected by the current assignment. Thus each score increment has conditional mean zero, and \(U\) is a sum of zero-mean increments under the stated assumptions. This does not make \(U\) normally distributed or by itself provide a valid p-value. A randomization test must use the known assignment mechanism to determine the null distribution, including how later probabilities depend on earlier observed data.

Randomization-based inference can be attractive because it incorporates the assignment probabilities directly. Its scope matters: the score argument above uses a sharp null and a known null mean. A weak null such as a zero average treatment effect does not imply that every individual outcome is unchanged by treatment, so the same randomization argument does not automatically establish validity for that null. The conditional-error approach instead starts from a valid original test and controls the conditional rejection probability of its continuation.

Diagnostics, Pitfalls, and Limitations

Before interpreting a final rejection after an allocation change, check that the analysis reflects the design actually used. Useful diagnostics include confirming the interim decision rule and allocation probabilities from trial records, checking that each probability was determined before the corresponding assignment, and verifying that the final test or conditional-error calculation used the selected allocation path. If a branch is rarely used or has very small assignment probabilities, normal approximations and estimated standard errors may also be unreliable; that is a reason to scrutinize the null calibration, not evidence that calibration is preserved.

• Using final sample sizes as if they were fixed in advance. This ignores the dependence between interim data and the allocation selected. The fixed-size standard error alone does not establish the reference distribution.
• Assuming mean zero is enough. A test calibrated from a standard normal distribution requires the relevant distributional approximation or exact result, not merely a zero null expectation.
• Checking only the overall average allocation. Two trials with the same average allocation can have different adaptive paths and different null distributions.
• Treating a conditional bound as a universal pass. The conditional error must correspond to the actual null and original rejection rule, and the new continuation must meet its bound in every relevant interim state.

Conditional-error calculations can be conservative, depend on the chosen original test, or be difficult to compute for complex models. Randomization-based methods require the allocation mechanism to be fully specified and reproduced in the reference distribution. Neither approach repairs violations such as an unrecorded allocation rule, outcome-dependent exclusions, or a changed endpoint. Adaptive allocation also does not, by itself, guarantee improved precision or ethical advantage; those are separate design questions.