Why Order the Boundaries?
In a multi-arm trial, the evidence for each experimental arm is examined over time, and the trial may stop an arm when its statistic crosses an efficacy boundary. If the same boundary is applied separately to every arm and every look, the chance of at least one false rejection can exceed the nominal significance level. A maximum-statistic procedure accounts for this multiplicity by calibrating against the joint distribution of the statistics. A step-down procedure can then use the observed ordering: the most extreme statistic faces a threshold for the full family, while later statistics face thresholds calibrated for progressively smaller subsets.
The key is that a boundary must account for the whole planned path of a statistic, not just its value at one look. The calibration also needs the correct dependence across arms and looks. Shared controls and repeated measurements generally make these statistics dependent, so independent-test calculations are not automatically suitable.
For continuous null distributions, this definition gives \(P(M_S>c_S)=\alpha\). If the distribution has atoms, use the quantile convention that ensures \(P(M_S>c_S)\leq\alpha\). Calibration is subset-specific: the distribution used for \(c_S\) must represent the joint statistics of the arms in \(S\), not a collection of unrelated single-arm distributions.
The Step-Down Rule
At a given look, consider the hypotheses not previously rejected; call that active set \(A\). Order their running maxima from largest to smallest, so that \(M_{(1)}(t)\geq M_{(2)}(t)\geq\cdots\). At rank \(r\), let \(S_r\) contain the arms at ranks \(r\) through \(|A|\). Compare \(M_{(r)}(t)\) with \(c_{S_r}\). Reject the hypothesis at rank \(r\) if \(M_{(r)}(t)>c_{S_r}\), then proceed to the next rank. At the first failure to exceed its threshold, stop the step-down sequence for that look; do not reject that hypothesis or any lower-ranked one at that look. At a later look, update the running maxima and repeat for the hypotheses still active.
Every \(c_{S_r}\) is calibrated for the maximum across all planned looks, even when the comparison is made before the final look. This matters: repeating a comparison at several looks is not the same as having several independent chances at level \(\alpha\). Instead, the path maximum has already included those opportunities. A failure at one look is not, by itself, a futility decision; whether the trial continues is a separate design choice.
Worked Example: Calibrating Independent Arm Paths
Consider a hypothetical three-arm trial with two planned looks, one-sided \(\alpha=0.05\), and six independent standard-normal statistics under the global null: two looks for each of three arms. For a subset of \(m\) arms, its path maximum is the maximum of \(2m\) independent standard normals. This independence assumption is deliberately simple; it makes the calculation transparent and would not describe shared-control data without further justification.
For any real \(c\), the probability that all \(2m\) statistics are at most \(c\) is \(\Phi(c)^{2m}\). Setting that probability to \(0.95\) gives \[ c_m=\Phi^{-1}\left(0.95^{1/(2m)}\right). \] For three arms, \(c_3=\Phi^{-1}(0.95^{1/6})\approx2.386\). For two arms, \(c_2=\Phi^{-1}(0.95^{1/4})\approx2.234\). For one arm, \(c_1=\Phi^{-1}(0.95^{1/2})\approx1.955\). These values are decreasing as arms are removed, as required by the nested-maximum property.
For example, the full-family calibration satisfies \(P(M_{\{1,2,3\}}>2.386)=0.05\) under the stated global null. It does not mean that each arm separately has a 0.05 chance of crossing 2.386; nor does it imply that the same values apply when the statistics are correlated.
Worked Example: Applying the Ordered Boundaries
Use the hypothetical independent calibration above. At a particular look, suppose the three running maxima are \(M_A(t)=2.50\), \(M_B(t)=2.30\), and \(M_C(t)=2.00\). The order is A, B, C. The running maxima are compared with path-calibrated thresholds, not with thresholds recalculated as if this were the only look.
At rank 1, \(S_1=\{A,B,C\}\), so compare \(2.50\) with \(c_3=2.386\). Since \(2.50>2.386\), reject \(H_A\). At rank 2, \(S_2=\{B,C\}\); compare \(2.30\) with \(c_2=2.234\). Since \(2.30>2.234\), reject \(H_B\). At rank 3, compare \(2.00\) with \(c_1=1.955\). Since \(2.00>1.955\), reject \(H_C\). Under the assumptions used for this example, all three hypotheses cross their step-down boundaries.
If instead the ordered maxima were \(2.50, 2.20, 2.00\), the first comparison would still reject \(H_A\), but the second would fail because \(2.20<2.234\). The sequence stops there, so neither \(H_B\) nor \(H_C\) is rejected at this look. It would be incorrect to continue to test \(H_C\) against \(c_1\): the step-down rule stops at the first failed rank.
Why Subset Pivotality Gives Strong Control
Controlling the probability of any rejection when all hypotheses are null is called weak familywise error rate control. Strong control requires the same bound when some hypotheses are false. The step-down argument uses subset pivotality: the joint distribution of the statistics for any subset of true null hypotheses is the same whether or not hypotheses outside that subset are false. In practice, the relevant joint distribution may be specified by a canonical joint-normal model; the important point here is that the true-null subset has the null distribution used to calibrate its path maximum.
Here is the justification. Let \(I_0\) be the set of true null hypotheses. If no true hypothesis is rejected, the familywise error event does not occur. Otherwise, consider the first true hypothesis rejected. Before that rejection, only false hypotheses can have been rejected, so every true hypothesis is still in the active set. At the rank of the first true rejection, the remaining subset \(S\) therefore contains \(I_0\). The rejected true hypothesis has a running maximum exceeding \(c_S\); hence \(M_{I_0}>c_S\). Since \(I_0\subseteq S\), nested quantiles give \(c_{I_0}\leq c_S\), so \(M_{I_0}>c_{I_0}\). Therefore, any familywise error implies that the maximum over the true-null paths exceeds their own calibrated critical value.
Subset pivotality says that \(M_{I_0}\) has the joint-null distribution used to calibrate \(c_{I_0}\). The calibration then gives \(P(M_{I_0}>c_{I_0})\leq\alpha\), and consequently the probability of at least one false rejection is at most \(\alpha\), regardless of which other hypotheses are false.
Worked Example: The First True Rejection
Suppose three hypotheses are tested, with \(H_A\) false and \(H_B,H_C\) true. Assume the procedure first rejects \(H_A\). At a subsequent comparison, suppose \(B\) is ranked ahead of \(C\), so the remaining subset at B’s comparison is \(S=\{B,C\}\). If B is rejected, then \(M_{\{B,C\}}>c_{\{B,C\}}\). Since \(M_{\{B,C\}}\) includes both true-null paths, this event implies \(M_{I_0}>c_{\{B,C\}}\), where \(I_0=\{B,C\}\). Here \(c_{\{B,C\}}=c_{I_0}\), and calibration bounds the probability of this event by \(\alpha\). If C were the first true rejection instead, the same argument would apply at C’s comparison. A false rejection of A does not consume an informal “share” of alpha; the guarantee follows from the calibrated maximum for the true-null subset.
More generally, if the active set at the first true rejection contains extra false hypotheses, it is a superset \(S\) of \(I_0\). The inequality \(c_{I_0}\leq c_S\) is what allows the argument to go through. This is why defining critical values as actual quantiles, rather than as arbitrary thresholds that merely happen to satisfy separate tail bounds, is important.
Dependence, Diagnostics, and Alternatives
With shared controls, statistics from different arms are correlated; repeated looks also induce correlation within an arm. The critical values should therefore be obtained from the joint null distribution that reflects the planned design. A common approach is to compute or simulate the path maxima under the specified joint model for each relevant subset. The introductory material on Dunnett-type boundaries covers the role of shared-control dependence for a single maximum; the step-down procedure needs the corresponding calibration for the nested subsets and sequential paths.
A useful diagnostic is to compare the assumed and intended covariance structure: check which arms share controls, how information accumulates across looks, and whether the planned statistics have the stated null means, variances, and correlations. Also verify the calibration itself: for each subset, the estimated probability of its path maximum exceeding \(c_S\) should be no greater than \(\alpha\), within appropriate simulation error. Check that \(S\subseteq R\) implies \(c_S\leq c_R\), and test the actual stopping and ranking code on tied values and on cases where a comparison fails. If the model ignores dependence, uses incorrect look correlations, or calibrates only a single look, the claimed error control may not hold.
Step-down maxT is often less conservative than a Bonferroni procedure because it uses the joint distribution and the observed ordering. Bonferroni instead allocates error across tests using a union bound and can provide control without a detailed dependence model, though often at a cost in power. A single-step maximum-statistic procedure compares each arm with the critical value for the full family; it is simpler but does not take advantage of progressively smaller subsets after the most extreme hypotheses are rejected. Dunnett-type procedures account for shared-control dependence, but a step-down sequential version still requires appropriate subset and path calibration. These methods are not interchangeable unless their calibration and stopping rules match the design.
The guarantee has limits. The family of hypotheses, direction of testing, planned looks, statistics, and dependence model must be specified consistently with calibration. Strong control relies on subset pivotality; if the distribution of true-null statistics changes with effects on other arms, the proof does not apply without additional justification. Data-dependent selection or dropping of arms can also change the relevant testing process. Such a rule must be included in the procedure’s calibration or handled by a valid error-control argument; simply removing an arm and reusing fixed boundaries does not establish control. Finally, efficacy boundaries do not by themselves specify futility stopping, estimation after selection, or conditional error after an unplanned change.
Check Your Understanding
Answer each question using the definitions and rules above. Short solutions follow the questions.
- If \(S\subseteq R\), which critical value is at least as large: \(c_S\) or \(c_R\)? Why?
- In the independent two-look example, what is the critical value for two arms at \(\alpha=0.05\), rounded to three decimals?
- At one look, ordered running maxima are 2.45, 2.15, and 1.90, with \(c_3=2.386\), \(c_2=2.234\), and \(c_1=1.955\). Which hypotheses are rejected by the step-down rule?
- State the role of subset pivotality in the strong familywise error argument.
- Why can the independent-normal critical values not automatically be used for arms sharing a control?
Solutions. (1) \(c_R\geq c_S\), because the maximum over \(R\) is at least the maximum over \(S\) for every realization, and the critical values are the corresponding upper quantiles. (2) \(c_2=\Phi^{-1}(0.95^{1/4})\approx2.234\). (3) The first statistic exceeds \(2.386\), so reject the first hypothesis. The second, 2.15, is below \(2.234\), so stop; do not reject the second or third hypothesis at this look. (4) It ensures that the joint distribution of the true-null statistics is the one used to calibrate their subset path maximum, even when other hypotheses are false. (5) Shared controls induce dependence between arm statistics, changing the distribution of the maximum; calibration must reflect that joint dependence.