Prespecifying Weights and Adaptation Algorithms

From a test name to a reproducible rule

An adaptive design is not reproducible if its protocol says only “re-estimate the sample size at the interim analysis” or “combine the stagewise evidence.” An independent analyst needs to know exactly which data enter each stagewise test, how the stage-two sample size is selected, whether the trial can stop early, and what numerical rule determines rejection. These details are also part of the statistical justification: a valid stagewise test does not by itself guarantee that the complete adaptive procedure controls Type I error.

The focus here is a fixed-weight inverse-normal combination test with a sample-size rule determined by the stage-one p-value. The weights and all branches of the rule are specified before observing trial data. The sample-size adaptation may depend on stage one, but stage two uses new, independent observations and a conditionally valid stagewise p-value.

A fully specified two-stage rule

Consider a one-sided comparison of two treatment groups on a continuous outcome. Let \(\delta=\mu_T-\mu_C\), where the means are measured in the outcome’s units. Test \(H_0:\delta\leq 0\) against \(H_a:\delta>0\), at one-sided significance level \(\alpha=0.025\). For the concrete rule below, assume independent observations within and between groups, a common known standard deviation \(\sigma\), and no reuse of stage-one observations in the stage-two test. The known-\(\sigma\) model makes the stagewise p-values explicit; other models require stagewise tests that are valid under their own assumptions.

Stage one enrolls 50 participants per group. Its statistic and upper-tail p-value are

Apply this adaptation and stopping algorithm, with \(n_2\) denoting the number added to each group in stage two:

If the trial continues, compute \(Z_2\) and \(p_2=1-\Phi(Z_2)\) using only the new stage-two observations and the same one-sided test. Combine the stagewise results using weights fixed in the protocol, \(w_1=0.6\) and \(w_2=0.8\). Since \(w_1^2+w_2^2=1\), the final statistic is \(C=0.6\Phi^{-1}(1-p_1)+0.8\Phi^{-1}(1-p_2)\). Reject \(H_0\) only if the trial continued and \(C\geq\Phi^{-1}(0.975)\), approximately 1.960. There is no early efficacy rejection.

For a protocol to be operational, it should also state how to handle missing or unusable stagewise data, the precise analysis population, the outcome definition, and any permitted rounding of the sample size. Those details are not interchangeable implementation choices: changing them may change the test or the distribution used to justify it.

Why fixed weights support the final threshold

The key condition is not that the two stage sample sizes are fixed. It is that the stage-two p-value remains valid conditional on the stage-one data and the selected sample size. Under the boundary null \(\delta=0\), suppose each stagewise p-value is continuous and uniform under its stage-specific null distribution. Because stage two uses fresh independent data, \(p_2\), conditional on stage one and the adaptation decision, is still uniform under the null. Thus its transformed statistic is conditionally standard normal.

For the no-futility version of the rule, the justification can be written directly. Under the boundary null, \(Z_1=\Phi^{-1}(1-p_1)\) is standard normal. Conditional on stage one, \(Z_2=\Phi^{-1}(1-p_2)\) is standard normal, with a conditional distribution that does not depend on stage one. It follows that \(Z_1\) and \(Z_2\) are independent standard normal variables. With fixed weights satisfying \(w_1^2+w_2^2=1\), their weighted sum is standard normal:

Therefore the one-sided rule \(C\geq\Phi^{-1}(1-\alpha)\) has rejection probability \(\alpha\) at the boundary null. For \(\delta<0\), the one-sided stagewise evidence shifts away from rejection, so the test is conservative for the composite null \(H_0:\delta\leq0\), under the stated model and test-validity conditions.

The sample-size rule may depend on \(p_1\) because, for every permitted stage-two sample size, the conditional null distribution of \(p_2\) remains uniform. The futility rule can only remove possible rejections: the adaptive procedure rejects on a subset of the rejection events from the corresponding procedure that continues after every stage-one result. Its Type I error is therefore no greater than \(\alpha\). This reasoning would fail if an adaptation caused the stage-two p-value to be invalid conditional on stage one.

Worked applications of the algorithm

Writing and checking the protocol

A useful prespecification makes every branch executable without an investigator choosing among options after seeing the interim data. The following sequence is a compact audit framework.

A practical diagnostic is to ask whether two independent analysts, given the same stage-one data and protocol, must choose the same sample size and stopping action. If not, the adaptation algorithm is incomplete. Check as well that the software uses the specified stagewise p-values, weights, and critical value; a coding error in any of these can implement a different test.

The main alternative approaches answer related but different design needs. A conditional-error approach expresses the remaining permissible Type I error after interim data and specifies a continuation test that respects that bound. A fixed-weight combination test instead specifies how stagewise evidence is combined. These approaches can be related, but a combination rule cannot be replaced by an informal conditional-error argument. A naive pooled test is different again: after a data-dependent change in the design, its original fixed-design calibration need not apply. The appropriate choice depends on the intended adaptation and the validity conditions that can be justified.

Common mistakes and limitations

• Choosing weights after observing interim evidence. The standard normal critical value above relies on fixed weights. If the weights are selected using \(p_1\), the combined statistic need not have the stated null distribution. A data-dependent weighting rule needs its own calibration; it is not covered by the fixed-weight justification.
• Assuming any sample-size change is harmless. Adaptation is justified only when the stage-two test remains valid conditional on stage one and the selected sample size. Reusing stage-one observations in \(p_2\), or changing the test without accounting for that change, can violate this condition.
• Leaving stopping boundaries vague. “Stop if the interim result is unpromising” is not an algorithm. Give a numerical criterion and state whether stopping leads to rejection, non-rejection, or a further analysis.
• Treating a futility stop as an efficacy result. A futility stop in this rule means only that the trial stops without rejecting \(H_0\); it is not proof that the treatments are equivalent.

The fixed-weight procedure also has limitations. It does not guarantee a desired power or make the chosen adaptation efficient; those are separate design questions. It requires suitable stagewise p-values and a justified dependence structure, and a one-sided test addresses only the direction specified in advance. If the outcome model, missing-data process, or stagewise analysis differs from the prespecified assumptions, the stated error-control argument may not apply without further justification.