Preserving Conditional Error When Arms Are Dropped

Why a Dropped Arm Changes the Boundary Problem

At an interim analysis, a MAMS trial may stop some treatment arms and continue others. Dropping arms changes the set of later rejection opportunities, but it does not by itself say how much later-stage evidence should be required for the retained arms. Keeping every original boundary is a simple conservative option. A different option is to use the error that the original design would have spent after the observed interim history to set new boundaries for the arms that remain.

The relevant quantity is the conditional error: the probability, under a null hypothesis, that the original design would reject later, conditional on the interim information and on having reached that analysis without an earlier rejection. A modified design can preserve error control if its conditional probability of later rejection does not exceed that original amount, for each null hypothesis or intersection test that must be protected.

Suppose an adaptation based on \(\mathcal{F}\) drops some arms and uses a new later rejection event \(R_{\mathrm{new,later}}\). The central requirement is $$ P_{H_I}(R_{\mathrm{new,later}}\mid\mathcal{F},S) \le CE_{\mathrm{old},I}(\mathcal{F}) $$ for every relevant surviving history and every null intersection \(H_I\) for which the original design supplied a valid local test. If the design uses closed testing, this comparison must protect the local tests required by that procedure, not only the test of the global null. Earlier material on closed testing and conditional error provides the broader familywise-error context; here the focus is how to set later critical values after arms are dropped.

Recalibrating a Later Boundary

For a single retained arm, suppose its future test statistic \(Z\), conditional on the interim information, has a continuous distribution with cumulative distribution function \(G_{\mathcal{F}}\) under the null. A one-sided new rejection rule \(Z>b(\mathcal{F})\) has conditional rejection probability \(1-G_{\mathcal{F}}(b(\mathcal{F}))\). To spend the full original conditional error, choose \(b(\mathcal{F})\) to satisfy $$ 1-G_{\mathcal{F}}(b(\mathcal{F})) = CE_{\mathrm{old},I}(\mathcal{F}). $$ A smaller conditional rejection probability is also valid, but is more conservative. In a multi-arm test, the later rejection event may be a union or another joint boundary-crossing region. Then the new boundary or boundaries are calibrated so the probability of that region under the conditional joint distribution is no greater than the original conditional error.

This is a history-specific calculation. The interim data can affect both the conditional distribution of future statistics and the original design’s conditional error. A single unconditional critical value will not generally preserve the required bound at every surviving history. Nor does the number of surviving arms alone determine the correct new boundary: the original boundary-crossing rule, the conditional distribution, and the observed history all matter.

History-Specific Calibration and Its Justification

A common case has a normally distributed future statistic whose conditional mean depends on the interim statistic. For example, if \(Z_1\) is a standardized interim statistic and \(Z_2\) is a standardized later statistic from a joint standard normal model with correlation \(\rho\), then $$ Z_2\mid Z_1=z,\ H_0 \sim N\!\left(\rho z,\,1-\rho^2\right). $$ If the original later boundary is \(c\), its conditional error on that history is $$ 1-\Phi\!\left( \frac{c-\rho z}{\sqrt{1-\rho^2}} \right). $$ A new one-sided boundary \(b(z)\) for that same future statistic can preserve the conditional error by equating the two upper-tail probabilities. Under this same conditional distribution, that gives \(b(z)=c\). In other settings—particularly when dropping arms changes a joint rejection region—the new boundary can differ. The calculation must use the actual conditional distribution and the actual old and new rejection events.

The preservation argument is an averaging argument, but it must account for survival. Let \(R_{\mathrm{early}}\) be the event of an earlier rejection. The original overall rejection probability under \(H_I\) can be written as $$ P_{H_I}(R_{\mathrm{early}}) + E_{H_I}\!\left[ \mathbf{1}_S P_{H_I}(R_{\mathrm{old,later}}\mid\mathcal{F},S) \right]. $$ If the new later rule satisfies the conditional-error inequality on every surviving history, then $$ \begin{aligned} P_{H_I}(R_{\mathrm{early}}) + E_{H_I}\!\left[ \mathbf{1}_S P_{H_I}(R_{\mathrm{new,later}}\mid\mathcal{F},S) \right] &\le P_{H_I}(R_{\mathrm{early}}) + E_{H_I}\!\left[ \mathbf{1}_S CE_{\mathrm{old},I}(\mathcal{F}) \right]\\ &= P_{H_I}(R_{\mathrm{early}}) + P_{H_I}(S\cap R_{\mathrm{old,later}}). \end{aligned} $$ The right-hand side is the original probability of an early or later rejection, with the later rejection counted only among trials that survived to that stage. This is why the survival indicator cannot be omitted.

Requirements, Diagnostics, and Alternatives

Conditional-error preservation relies on a valid original design and a valid conditional calculation. In particular, the original local tests must have the intended error control; the interim selection and dropping rule must be based on information included in \(\mathcal{F}\); and the conditional distribution used for calibration must correctly describe future statistics under the null. If allocation, measurement, or analysis procedures change in ways not represented in that distribution, a nominal conditional-error calculation may no longer be reliable.

For strong familywise control, verify the condition for each relevant intersection null. Preserving the error for the global intersection alone does not automatically establish control for every possible configuration of true and false nulls. The local tests must also remain properly defined for the retained hypotheses. The adaptation described here drops arms; changing the hypotheses or redefining the testing procedure requires its own justification.

Useful diagnostics include checking, over a grid of plausible surviving interim histories, that the calculated new conditional rejection probability never exceeds the old conditional error. For a multi-arm rule, evaluate the joint tail probability using the conditional dependence among the retained statistics, not separate marginal tails unless the construction justifies that simplification. Also check boundary calculations numerically against direct probability evaluation and verify that the implemented decision rule matches the rule used in calibration.

There are two practical approaches. One is to retain the original later boundaries for all surviving arms. This avoids recalibration, and dropping rejection opportunities often makes the resulting test conservative, but it may forgo some usable conditional error. The other is to recalibrate later boundaries to use some or all of the original conditional error. This can improve efficiency, but requires reliable history-specific calculations and validation for each protected intersection. Simply dividing a nominal level by the number of remaining arms is not a substitute: it need not reproduce the original design’s conditional error.

Common Pitfalls

• Ignoring survival. Averaging a later conditional error over all interim histories without restricting to \(S\) does not account correctly for early rejection. Separate the early-rejection probability and average the later conditional probability with the survival indicator.
• Using only the number of retained arms. The correct error budget depends on the old rejection event and the conditional joint distribution, not just the count of arms left.
• Checking only one null configuration. A conditional bound for the global null alone may not establish strong familywise control. Check the required intersection tests.
• Treating a conditional error as an unconditional alpha level. Conditional error varies with interim history. Calibrate using the observed history and the appropriate conditional model.
• Assuming marginal calculations capture joint crossing. When retained statistics are dependent, compute the probability of their joint rejection region with that dependence represented.