Inverse-Normal Boundaries Across Arms and Stages

Why combine evidence across arms and stages?

In a multi-arm, multi-stage trial, each treatment arm has its own efficacy hypothesis, but evidence for that hypothesis may arrive in several stages. A stagewise p-value summarizes the evidence from one stage; an inverse-normal combination statistic puts those p-values onto a common scale and combines them within an arm. A second calibration is then needed: testing several arms, and possibly looking more than once, increases the chance of at least one false rejection.

The goal here is to separate these two tasks. First construct a valid arm-specific combination statistic. Then choose rejection boundaries that control the probability of falsely rejecting one or more true arm hypotheses. The introductory material on stagewise familywise error boundaries provides useful context; the calculations below focus on inverse-normal combination and its joint calibration.

The standard exact calibration assumes that, under \(H_j\), each stagewise p-value is uniformly distributed on \([0,1]\), and that the stagewise p-values are independent across stages. Then each \(Z_{jl}\) has a standard normal distribution, and the weighted sum \(Z_j\) is standard normal because its mean is zero and its variance is \(\sum_l w_l^2=1\). If stagewise p-values are only super-uniform, or the stages are dependent, this exact normal calibration need not hold; a valid null distribution or a conservative procedure is needed instead.

The arm statistics need not be independent. Shared controls, for example, can induce positive correlation between treatment-versus-control statistics. That dependence affects the probability of at least one rejection across arms, even when each individual \(Z_j\) has the correct standard normal null distribution.

Combining stagewise evidence within an arm

The weights describe how stagewise evidence contributes to the final statistic. Equal weights across two stages are \(w_1=w_2=1/\sqrt{2}\). Unequal weights are also possible, but must be chosen before examining the stagewise p-values for the analysis at hand. A larger weight gives that stage more influence, while the sum-of-squares condition preserves unit variance under the stated independence assumptions.

Stage independence is a substantive assumption, not a consequence of applying the inverse-normal transformation. It is most plausible when stages use nonoverlapping data and stagewise statistics are constructed without reusing earlier observations. If the same patients contribute to more than one stagewise test, or if a later analysis uses information from an earlier one in a way that compromises independent increments, the variance and null distribution of the weighted sum may differ from the standard normal distribution.

Joint calibration across arms

For a set \(I\) of arms whose hypotheses form an intersection null, a joint max test rejects that intersection when at least one corresponding statistic exceeds a critical value. If the joint null distribution is known, the critical value \(c_I\) can be calibrated to satisfy \(P(\max_{j\in I} Z_j>c_I\mid H_I)=\alpha\), where \(H_I\) states that every hypothesis in \(I\) is true. This controls the probability of falsely rejecting that intersection at level \(\alpha\).

A single common critical value is convenient when the statistics have the same marginal null distribution. Its value depends on their joint dependence. With independent arms, the probability that all \(|I|\) statistics are at most \(c_I\) is \(\Phi(c_I)^{|I|}\). With correlated arms, calibration instead uses their joint multivariate normal distribution, provided that model is justified. A numerical calibration should use the correct correlation matrix rather than treating shared-control statistics as independent.

Closed testing supplies that broader guarantee. For every nonempty subset \(I\) of arm hypotheses, test the intersection \(H_I\) at level \(\alpha\). Reject an individual \(H_j\) only if every intersection \(H_I\) containing \(H_j\) is rejected. If each local intersection test has type I error no greater than \(\alpha\) whenever its intersection null is true, the closed procedure strongly controls familywise error: whenever any true hypothesis is rejected, at least one true intersection containing that hypothesis must also have been rejected incorrectly.

For the local test, a max statistic over \(j\in I\) is natural. Its critical value may differ across subsets because the subset's size and correlation structure can differ. Under subset pivotality, the joint null distribution of statistics for a true subset is unaffected by which hypotheses outside that subset are false, so its global-null subset distribution can be used for local calibration. If that assumption is not justified, the local test must instead be calibrated for the relevant intersection null.

Boundaries across repeated stages

A trial may also permit efficacy rejection at an interim look. Let \(q_t=\sum_{l=1}^t w_l^2\), the cumulative weight variance through stage \(t\), and define the standardized partial statistic \(U_{jt}=\sum_{l=1}^t w_l Z_{jl}/\sqrt{q_t}\). Under independent standard normal stagewise scores, each \(U_{jt}\) is standard normal under \(H_j\). A sequential rule can reject when \(U_{jt}\) crosses a boundary at any stage.

The repeated looks are correlated: \(U_{jt}\) and \(U_{js}\) share early-stage evidence. If the stagewise cross-arm correlation at stage \(l\) is \(R_{l,ij}\), then under independent stages the covariance is \(\operatorname{Cov}(U_{it},U_{js})=\sum_{l\leq\min(t,s)}w_l^2R_{l,ij}/\sqrt{q_tq_s}\). Thus a boundary for a maximum over arms and stages must account for both sources of dependence, not just the number of arms.

A simple simultaneous-boundary construction uses a common boundary \(b_I\) for the statistics in an intersection: \(P(\max_{j\in I,\,t\leq L} U_{jt}\geq b_I\mid H_I)=\alpha\). This probability can be evaluated from the joint multivariate normal distribution when its covariance assumptions are appropriate. More flexible stage-specific boundaries \(b_{It}\) can be calibrated to a specified error-spending plan, but their joint crossing probability must still be controlled. In either case, local tests for all intersections can be combined by closed testing for strong familywise control.

Diagnostics, alternatives, and limitations

Before using an inverse-normal boundary, check the ingredients that determine its null distribution. Confirm that stagewise p-values are valid for the stated hypotheses, that the weighting rule was specified without using the current stage's p-values, and that the assumed independence across stages is defensible. For joint calibration, inspect whether the arm correlation matrix reflects the design, including shared controls and any planned adaptations. A boundary calibrated under independence can be too liberal if the true dependence differs in a way that increases the relevant maximum-tail probability.

Adaptation requires particular care. If arms are selected or dropped using interim evidence, simply treating the surviving stagewise p-values as if the original fixed set had been tested independently can invalidate the combination or its joint calibration. Stagewise p-values must remain valid under the adaptation, and the joint rejection rule must account for the selection procedure. Conditional-error methods provide one established way to preserve error after adaptations; the inverse-normal weights alone do not repair invalid selection.

A common mistake is to calibrate each arm at level \(\alpha\) and conclude that the familywise error is also \(\alpha\). That only controls each arm's marginal error; it does not generally control the chance of at least one false rejection across arms or repeated looks. Another mistake is to combine p-values and then ignore their dependence when calibrating the maximum. The correction is to define the joint rejection event explicitly and calibrate its probability under each relevant intersection, or use a dependence-robust allocation such as Bonferroni. These procedures can lose power, and exact joint calibration can be sensitive to covariance misspecification; neither limitation is removed merely by using inverse-normal weights.