Worked Calibration of Two-Stage Three-Arm Boundaries

Why Calibrate Both Stages Together?

In a multi-arm, multi-stage trial, each treatment is compared with a common control at interim and final analyses. Looking at several arms at more than one stage creates several opportunities to reject a true null hypothesis. The familywise error rate (FWER) is the probability of making at least one such false rejection among the hypotheses under consideration. A boundary that is appropriate for one comparison at one analysis does not automatically control this probability across all comparisons and stages.

This tutorial works through a transparent calibration for three treatment arms, one shared control, and two stages. It uses a prespecified allocation of the FWER across stages and then across arms. The resulting boundaries are conservative: they guarantee control without requiring independence among the test statistics. That feature is useful when the shared control makes the statistics correlated.

Model and Calibration Rule

Definition: Let \(H_i\) be the one-sided null hypothesis that treatment arm \(i\) is not better than control, for \(i=1,2,3\). At stage \(k\), let \(Z_{ik}\) be the standardized statistic comparing arm \(i\) with the shared control, with larger values favouring treatment. Under every parameter value in a true \(H_i\), assume \(P(Z_{ik}\geq c)\leq 1-\Phi(c)\) for each one-sided critical value \(c\) used at that analysis; equality holds at the boundary null for an exact normal test. A two-stage design rejects \(H_i\) if \(Z_{i1}\geq c_1\), or, if the arm continues, if \(Z_{i2}\geq c_2\).

The null-tail condition is the key requirement for the calibration below. For a known-variance normal outcome, if the treatment effect is \(\delta\leq 0\), then \(Z_{ik}\sim N(\delta/SE,1)\), so \(P(Z_{ik}\geq c)\leq 1-\Phi(c)\); equality holds at the boundary \(\delta=0\). For other test statistics, the same null-tail bound must be justified, for example by an appropriate large-sample approximation. The design must also prespecify the direction of testing, analysis times, stopping rules, and the error allocation. The statistics must be valid for each true-null arm even when other arms are effective; for example, the comparisons should not be distorted by interference between treatment groups.

Let the target FWER be \(\alpha\). Choose nonnegative stage allocations \(\alpha_1\) and \(\alpha_2\) such that \(\alpha_1+\alpha_2=\alpha\). At stage \(k\), allocate \(\alpha_k/3\) to each arm and set the common one-sided boundary to

c_k=\Phi^{-1}\left(1-\frac{\alpha_k}{3}\right),

where \(\Phi\) is the standard normal cumulative distribution function. This allocates error by a union bound: it does not claim that stagewise rejection events are independent or disjoint. If an arm has already stopped, its later test is not performed; omitting that possible rejection can only reduce the error probability.

Worked Calibration for Three Arms

Worked Example: Choosing Interim and Final Boundaries

Consider a hypothetical two-stage trial with three treatment arms sharing a control. The target one-sided FWER is \(0.05\). The design team allocates \(0.02\) to the interim stage and \(0.03\) to the final stage. Assume each true-null statistic has upper-tail probabilities no greater than the corresponding standard normal tail at its planned boundary, with equality at the boundary null for an exact normal test, and use the same boundary for each arm within a stage.

At the interim analysis, the per-arm allocation is \(0.02/3=0.0066667\). Therefore,

c_1=\Phi^{-1}(1-0.02/3) =\Phi^{-1}(0.9933333) \approx 2.475.

At the final analysis, the per-arm allocation is \(0.03/3=0.01000\), giving

c_2=\Phi^{-1}(1-0.03/3) =\Phi^{-1}(0.99000) \approx 2.326.

The rule is to reject an arm’s null if its interim statistic is at least \(2.475\), or, if it continues to the final analysis, if its final statistic is at least \(2.326\). The final boundary is lower because the design assigned more error to that stage. These are marginal-tail boundaries; they are not obtained by assuming independence between arms or stages.

Why the Guarantee Holds

Let \(S\) be the set of truly null treatment hypotheses in a particular configuration. A familywise error occurs only if at least one hypothesis in \(S\) crosses a boundary. For each true-null arm, the probability of crossing its interim boundary is at most \(\alpha_1/3\), and the probability of crossing its final boundary is at most \(\alpha_2/3\), under the stated null-tail calibration. The event of a false rejection is contained in the union of these arm-stage events.

\begin{aligned} \mathrm{FWER} &=P(\text{at least one true null is rejected})\\ &\leq \sum_{i\in S}\left\{ P(Z_{i1}\geq c_1)+P(Z_{i2}\geq c_2) \right\}\\ &\leq |S|\left(\frac{\alpha_1}{3}+\frac{\alpha_2}{3}\right)\\ &\leq \alpha_1+\alpha_2=\alpha, \end{aligned}

The first inequality is the union bound. It does not need independence. The last inequality follows from \(|S|\leq 3\). Because the argument applies to any set \(S\) of true nulls, it gives strong FWER control: the guarantee holds under the global null and under partial-null configurations, provided the marginal null distributions remain valid.

In a design with arm-specific continuation, a final rejection is possible only if that arm reaches stage two. Its rejection event is therefore contained in the event \(Z_{i2}\geq c_2\). This containment is enough for the bound above; no assumption that continuation is independent of interim results is needed. The final statistic and its null marginal must, however, be valid under the actual prespecified design and analysis.

Shared Control and Stage Dependence

With a common control, the arm statistics are generally correlated. For example, suppose a continuous outcome has known standard deviation \(\sigma\), with equal independent sample sizes \(n\) in each treatment arm and in control at an analysis. The treatment-control mean-difference statistic for arm \(i\) has variance \(\sigma^2(1/n+1/n)\). Two different treatment-control contrasts share the control mean, so their covariance is \(\sigma^2/n\). Thus their correlation is

\operatorname{Corr}(Z_{i},Z_{j}) =\frac{\sigma^2/n}{2\sigma^2/n} =\frac{1}{2}, \qquad i\ne j.

Stage one and stage two statistics for the same arm are also typically correlated because stage-two data include stage-one information. Under the canonical information model, their correlation is the square root of the ratio of the earlier to the later information. Neither correlation is needed to justify the conservative union-bound calibration. They do matter if a design seeks less conservative boundaries by calculating a joint crossing probability from the multivariate distribution.

Worked Example: Checking the Shared-Control Correlation

Suppose, hypothetically, that at an analysis the control group and each treatment group have \(50\) independent observations, and the outcome standard deviation is known to be \(10\) units. For each treatment-control difference, the standard error is

SE=10\sqrt{\frac{1}{50}+\frac{1}{50}} =10\sqrt{0.04} =2.000 \text{ units}.

If two treatment contrasts use this same control group, their covariance is \(10^2/50=2.000\) square units. Each contrast has variance \(2.000^2=4.000\) square units, so their correlation is \(2.000/4.000=0.5000\). The comparisons are not independent. The critical values in the preceding example remain valid for the stated FWER guarantee because the union-bound proof uses their marginal probabilities, not independence.

Applying the Boundaries

Worked Example: Interpreting Hypothetical Trial Results

Suppose a hypothetical trial uses \(c_1=2.475\) and \(c_2=2.326\). At the interim analysis, the three statistics are \(Z_{11}=2.60\), \(Z_{21}=2.10\), and \(Z_{31}=1.75\). Arm 1 crosses the interim boundary and its null is rejected. Arms 2 and 3 do not cross, so they continue under the design’s stopping rules.

At the final analysis, suppose the statistics for the continuing arms are \(Z_{22}=2.40\) and \(Z_{32}=2.20\). For arm 2, \(2.40\geq2.326\), so its null is rejected. For arm 3, \(2.20<2.326\), so its null is not rejected. The design rejects the null hypotheses for arms 1 and 2 and does not reject the null for arm 3.

These decisions follow the prespecified test rule. They do not establish that a treatment is clinically important; clinical interpretation also depends on estimated effects, uncertainty, and the outcome’s units. The FWER guarantee concerns the probability of one or more false rejections, not the probability that any particular null is true.

Calibration Choices, Diagnostics, and Limitations

The stage allocations determine the trade-off between early and late evidence. A larger \(\alpha_1\) lowers the interim boundary and can make early stopping more likely, while leaving less error budget for the final stage. A larger \(\alpha_2\) has the opposite effect. The allocations should be chosen before observing trial outcomes, in accordance with the design’s objectives.

A more efficient alternative is to use the joint multivariate distribution of the arm- and stage-specific statistics and solve directly for boundaries that give the desired crossing probability. Such calibration can account for shared-control and across-stage correlation and may yield less conservative boundaries. It depends on correctly specifying the joint distribution and on the null configuration used to calibrate the procedure. The stagewise Bonferroni approach here is simpler and robust to dependence, at the cost of potentially requiring stronger evidence than a joint calibration.

Before using the calculated boundaries, check that the actual design matches the calibration inputs. In particular, verify the number of arms, the one-sided direction, the stage allocation, and the marginal null distribution at each analysis. If variance estimation, unequal allocation, adaptation, or dropping arms changes the statistic’s distribution or the set of hypotheses, the original marginal tail probabilities may no longer apply. Recalibration or a valid adjustment is then needed.

Wrong number of tests. Dividing a stage allocation by two arms rather than three gives too much error to each arm. Divide by the number of hypotheses protected at that stage.
Ignoring the stage split. Applying the full \(0.05\) separately at both stages can exceed the target. The allocations must sum to no more than \(0.05\).
Assuming independence. Shared control induces correlation. Do not multiply marginal non-rejection probabilities as if the tests were independent unless that assumption is justified.
Using a boundary after changing the design. A new arm count, variance model, or adaptation can invalidate the calibration. Check the test-statistic distributions and error allocation again.

Key takeaway: With three arms and two stages, preallocate the familywise error across stages, divide each stage allocation across the three arms, and use the corresponding one-sided marginal quantiles. The union bound then controls FWER without requiring independent shared-control or interim/final statistics, provided the true-null marginal tests are valid.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

For three arms and a stage allocation of \(0.015\), what one-sided per-arm tail probability is assigned at that stage?
If the two stage allocations are \(0.02\) and \(0.03\), what is the upper bound on FWER from the union-bound calibration?
Why does positive correlation between two shared-control comparisons not invalidate this calibration?
With a final boundary of \(2.326\), an arm’s final statistic is \(2.30\). Does it cross the boundary?
Name one design change that could require recalibration, and explain why.

Solutions. (1) \(0.015/3=0.005000\) per arm. (2) The bound is \(0.02+0.03=0.05\). (3) The proof bounds the probability of a union by the sum of marginal probabilities and does not assume independence. (4) No: \(2.30<2.326\). (5) For example, changing the number of tested arms changes the error allocation per arm; the original boundaries may then no longer ensure that the stagewise allocations sum to at most the target FWER.