Why Condition on the Interim Data?
In a multi-arm multi-stage (MAMS) trial, interim results can determine which arms continue. An unconditional familywise type I error rate averages over every possible interim result and every resulting selection. After a particular interim result is observed, however, a different question becomes relevant: under the original testing plan, what is the probability of a future false rejection, conditional on the data actually observed? That probability is the conditional type I error.
It is a property of a specified testing plan, a specified null hypothesis (or intersection of null hypotheses), and the interim information being conditioned on. It is not itself a new test, and it is not the probability that a null hypothesis is true. Here the focus is its calculation after interim arm selection, not how to redesign a future testing plan to use or preserve that quantity.
The probability is evaluated under the statistical model for the true-null arms and the plan's prespecified boundaries, multiplicity procedure, and stopping rules. If a composite null leaves nuisance parameters or the joint distribution unspecified, the conditional error may depend on them; it is not automatically a single known number. The examples below use canonical normal models with specified covariance structures and known null distributions.
Conditional Distribution Under the Canonical Model
Suppose \(Z_{i,j}\) is the standardized statistic for arm \(i\) at stage \(j\), and \(t_j\) is its cumulative information fraction, with \(0<t_1<\cdots<t_J=1\). Under a true null, the canonical model gives standard normal statistics at each stage, with correlations reflecting the nested information. For one arm, \(Z_i(t)\) has correlation \(\sqrt{s/t}\) with \(Z_i(s)\) when \(s\leq t\). For multiple arms, the cross-arm correlations must also be specified; shared-control designs, for example, generally induce dependence.
For a single arm under the null, the canonical model can be represented as \(Z(t)=B(t)/\sqrt{t}\), where \(B(t)\) is Brownian motion. Independent increments imply that, after observing \(Z(t)=z\), the unstandardized future increment \(B(1)-B(t)\) is independent of the interim data and has variance \(1-t\). Since \(B(t)=\sqrt{t}\,z\), the final statistic satisfies
Consequently, if the plan rejects this arm at the final look when \(Z(1)\geq c\), its conditional error is the upper-tail probability of that conditional normal distribution. The formula applies to a one-sided upper boundary; a two-sided boundary or a multistage plan requires the probability of the corresponding full rejection event.
Worked Example: One Arm and One Remaining Look
Hypothetical setup. There is one true null arm, an interim look at \(t=0.5\), and a single final rejection boundary \(c=2.0\). The observed interim statistic is \(z=1.0\). Assume the canonical null model and no additional stopping or testing rule.
The conditional final distribution has mean \(\sqrt{0.5}(1.0)=0.7071\) and variance \(1-0.5=0.5\), so its standard deviation is \(\sqrt{0.5}=0.7071\). The conditional error is
Thus, for this observed interim statistic, the original plan has about a 3.37% probability of rejecting this true null at the final look. This is a conditional probability under the stated model, not an unconditional type I error rate for the whole trial.
Selection and the Joint Distribution Across Arms
Let \(S(x)\) be the set of arms selected to continue by the prespecified rule when the interim data are \(x\). Once the full interim data are conditioned on, \(S(x)\) is known. For a familywise error calculation under \(H_I\), the relevant future event concerns the true-null arms in \(I\) that remain eligible for rejection. The conditional error is the probability of that event under the plan's full rejection logic; it is not necessarily the probability that any selected statistic crosses a single unadjusted boundary.
For a multivariate normal model, the conditional law of future statistics can be obtained from the joint covariance matrix. If \(X\) is the vector of interim statistics and \(Y\) the vector of future statistics, with joint mean vector \((\mu_X,\mu_Y)\) and covariance blocks \(\Sigma_{XX},\Sigma_{YY},\Sigma_{YX}\), then
The conditional error is the probability, under this distribution, of the future rejection region. Selection does not remove the need to specify the joint model: conditioning on the selected arm's statistic alone is justified only when the model shows that the other observed interim statistics do not add information about its future statistic.
Worked Example: Two Independently Evolving Arms
Hypothetical setup. Two true-null arms have independent canonical processes. Both are observed at \(t=0.5\), and each has a final upper boundary of \(2.0\). The observed interim statistics are \(z_A=1.0\) and \(z_B=0.5\); the selection rule retains both arms. Assume there are no other boundaries or stopping rules. Independence of the entire arm processes is part of the model, so conditioning on both interim values leaves the two future statistics conditionally independent.
For arm A, the standardized conditional boundary is \((2.0-\sqrt{0.5}(1.0))/\sqrt{0.5}=1.8284\), giving a conditional crossing probability of about \(0.03374\). For arm B, it is
Because the future statistics are conditionally independent, the probability that at least one crosses is one minus the probability that neither crosses:
The conditional probability of at least one false rejection is therefore about 4.335%. Adding the two armwise probabilities would give about 4.369%, which is not the exact familywise probability because it counts the event that both cross twice.
Worked Example: Selection With Correlated Arms
Hypothetical setup. Two true-null arms follow a joint canonical model with cross-arm correlation \(\rho=0.4\) at a common information fraction \(t=0.5\). The selection rule keeps only the arm with the larger interim statistic. The observed values are \(z_A=0.8\) and \(z_B=1.4\), so arm B is selected. Its final upper boundary is \(2.0\). Assume the canonical covariance structure across both arms and stages.
Write \(X=(Z_A(0.5),Z_B(0.5))^\mathsf T\) and \(Y=Z_B(1)\). Under the null, the interim covariance matrix is \(\Sigma_{XX}=\begin{pmatrix}1&0.4\\0.4&1\end{pmatrix}\). The covariance vector between \(Y\) and \(X\) is \(\Sigma_{YX}=\sqrt{0.5}(0.4,1)\), and \(\operatorname{Var}(Y)=1\). Applying the conditional normal formula, the conditional mean is
The equality follows because \((0.4,1)\) is the second row of \(\Sigma_{XX}\). The conditional variance is \(1-0.5=0.5\), so the boundary in conditional standard deviation units is
Thus the conditional error for the selected true-null arm is about 7.66% for this observed interim vector. In this particular canonical model, the result equals the calculation based on \(z_B\) alone, but that simplification is justified by the specified covariance structure, not by selection itself. A different joint model could make \(Z_A(0.5)\) informative about \(Z_B(1)\) even after conditioning on \(Z_B(0.5)\), changing the answer.
Multistage Rejection Events, Checks, and Limitations
With more than one remaining look, conditional error is the probability of the plan's entire future rejection event. For a single arm, for example, crossing at stage \(k+1\) or at a later stage contributes to the same event; a path that crosses at an earlier future look should not be counted again as a distinct rejection. For multiple arms, the event may also include a maximum-statistic boundary, step-down rule, closed-testing decision, or other prespecified multiplicity procedure. The conditional multivariate distribution must be integrated over that rejection region. The earlier material on recursive probability calculation for boundary crossing provides a way to evaluate such sequential events; conditioning changes the starting distribution to the law given the interim data.
Useful checks before trusting a calculation include verifying that the observed data correspond to a path that actually reaches the interim look, confirming the selection rule and the true-null set, and checking that the covariance matrix matches the analysis model. Conditional probabilities should lie between zero and one. In the one-arm canonical example, as the observed interim statistic increases, the conditional mean increases and the probability of crossing a fixed upper boundary cannot decrease. A violation of that monotonicity is a warning to check the signs, standardization, or conditioning calculation.
The result also depends on what is conditioned on. If selection depends on the complete interim data, conditioning only on the selected arm's statistic can be insufficient. Conversely, if the joint canonical model establishes that the other interim statistics provide no extra information about that arm's future statistic, the simpler calculation is valid. Conditioning on a selection label alone is not the same as conditioning on the observed interim vector: the label usually leaves a range of possible interim values, which must be averaged over to obtain a probability conditional only on that label.
Conditional error and unconditional type I error answer different questions. The former is evaluated at a fixed interim data realization; the latter averages rejection probabilities over possible data and selection outcomes under the null. A small unconditional rate does not imply a small conditional error for every observed interim result. Nor is conditional error a p-value: it is a probability of a future rejection under the specified null model and plan, conditional on interim data, rather than a tail probability calculated to summarize evidence against a null using an observed test statistic.
A common mistake is to condition on two arms' interim statistics but use only one arm's marginal model to calculate that arm's future probability. The calculation is justified only if the full joint model implies that the other interim observation adds no information about the future statistic, as in the canonical independent-increments example above. Otherwise, use the full conditional multivariate normal distribution. Another mistake is to calculate armwise conditional probabilities and add them as if they were the probability of any rejection. That is exact only in special cases; generally the dependence and overlap of rejection events must be accounted for.
Check Your Understanding
Try the questions first, then check the solutions below.
- Under a one-arm canonical null model, \(t=0.25\), observed \(z=0\), and a final upper boundary \(c=1.96\). What is the conditional error at the final look?
- Why is a marginal model for arm B alone generally insufficient when conditioning on the interim statistics of both arms?
- Two independent true-null arms each have conditional crossing probability 0.02. What is the conditional probability that at least one crosses?
- A selection rule retains the arm with the larger interim statistic. Does the selected arm's interim value necessarily equal the conditional distribution obtained by conditioning only on the selection label? Explain.
Solutions.
1. The conditional mean is \(\sqrt{0.25}(0)=0\) and the conditional standard deviation is \(\sqrt{1-0.25}=\sqrt{0.75}\). Thus the conditional error is \(1-\Phi(1.96/\sqrt{0.75})=1-\Phi(2.263)\approx0.0118\).
2. The other arm's interim statistic may be correlated with arm B's future statistic. The conditional mean and variance can therefore depend on both observed values. A marginal calculation is valid only if the specified joint model shows that conditioning on the other interim data adds no information.
3. Independence gives \(1-(1-0.02)^2=1-0.9604=0.0396\), or 3.96%. The answer would require the joint crossing probability if the future statistics were dependent.
4. No. Conditioning on the selection label averages over all interim vectors that lead to that label. Conditioning on the observed interim vector fixes the actual statistics, so the conditional future distribution and error can differ.