Multivariate-Normal Calibration of Stagewise Boundaries

Why Calibrate Boundaries Jointly?

In a multi-arm, multi-stage design, evidence for different treatments is observed repeatedly. The test statistics are generally dependent: statistics from different arms can share a control group, and statistics at different stages use overlapping data. Consequently, setting each boundary from a single-test tail probability does not, by itself, determine the probability that the design makes at least one false efficacy declaration.

Joint multivariate-normal calibration accounts for this dependence directly. It chooses stagewise boundaries so that the probability of crossing any efficacy boundary by each planned stage matches a prespecified cumulative error target. The introductory material on alpha-spending functions motivates those targets; here the focus is evaluating the path probabilities that determine the boundary values.

The Joint-Normal Model and the No-Rejection Region

For a fixed set of arms and planned looks, collect the statistics into a vector \(Z\), with one component for each arm-stage combination. Under the null hypothesis being calibrated, assume \(Z\sim N(\mu,\Sigma)\), where \(\mu\) is the null mean vector and \(\Sigma\) is the covariance matrix implied by the design. In a standard one-sided efficacy test under the null, the relevant components have mean zero and unit variance. The covariance matrix must reflect both shared controls and repeated use of accumulating data; it is not generally an identity matrix.

The following calibration requires a prespecified design, a specified covariance matrix under the null, upper efficacy boundaries, and a target cumulative error \(A_k\) at each planned stage. The normal model may be exact under a suitable sampling model or an approximation justified by the design. If these assumptions are not reasonable, the calibrated probabilities need not describe the actual error rate.

The calibration equation for stage \(k\) is $$ P(S_k)=1-A_k. $$ If only one boundary at stage \(k\) is unknown, the other boundaries and the covariance matrix are fixed, and this equation can be solved for that boundary. If several boundaries are unknown, one needs additional prespecified rules, such as equal boundaries across arms or an allocation of the stagewise error among arms; the cumulative FWER target alone may not identify a unique set of values.

First-Rejection Contributions and a Short Derivation

For interpreting stagewise contributions, define \(R_k\) as the event that no arm crosses at any stage before \(k\), and at least one arm crosses at stage \(k\). The events \(R_1,\ldots,R_K\) are disjoint: a path can have only one first stage at which any arm crosses. Because a crossing by stage \(K\) must have a first crossing at some stage no later than \(K\), $$ \operatorname{FWER}_K=\sum_{k=1}^{K}P(R_k). $$ This is a design-wide partition. In contrast, events defined separately as “arm \(j\) first crosses at stage \(k\)” can overlap across arms, so summing their probabilities would generally count some paths more than once.

There is another useful expression for each stage’s contribution. Since \(S_k\) is contained in \(S_{k-1}\), a path is in \(S_{k-1}\) but not \(S_k\) exactly when it has its first design-wide rejection at stage \(k\). Thus $$ P(R_k)=P(S_{k-1})-P(S_k), $$ where \(P(S_0)=1\). Substituting into the sum gives $$ \operatorname{FWER}_K =\sum_{k=1}^{K}\{P(S_{k-1})-P(S_k)\} =1-P(S_K). $$ The intermediate survival probabilities cancel. This establishes why integrating the no-rejection region through the final stage suffices to obtain cumulative FWER, while the differences between successive survival probabilities give disjoint stagewise contributions.

Calibrating a Later Boundary by Integration

With repeated looks, a later-stage crossing contributes to FWER only on paths that did not cross earlier. For one arm with two looks, let the null statistics be jointly standard normal with correlation \(\rho\). If the first boundary \(b_1\) is fixed and the second boundary \(b_2\) is being calibrated, the cumulative crossing probability is $$ q(b_2)=P(Z_1\ge b_1)+P(Z_1<b_1,\ Z_2\ge b_2). $$ The second probability is a pathwise contribution: it excludes paths that already crossed at the first look. For a multivariate-normal vector, it can be evaluated by integrating the joint density over the indicated region, or equivalently by using a multivariate-normal probability routine. A one-dimensional root-finder can then solve \(q(b_2)=A_2\), provided \(q\) is evaluated accurately enough for the required boundary precision.

In a multi-arm design the same principle applies, but the event at the later stage is that at least one arm crosses, conditional on all arms having remained below their earlier boundaries. The integration region includes every relevant arm-stage component. Correlations across arms matter: one must evaluate the joint probability rather than multiply armwise probabilities unless independence is justified.

Checking Boundaries and Comparing Alternatives

Joint-normal calibration is useful because it targets the design-wide probability directly, rather than requiring an unnecessarily conservative bound on it. It also makes the dependence assumptions explicit. The result is only as credible as the null distribution and covariance matrix used in the integration. In particular, a covariance matrix that omits shared-control dependence or misrepresents overlap between looks can yield boundaries with incorrect error control.

A simpler alternative is Bonferroni calibration. If there are \(m\) arms and \(K\) looks, assigning a one-sided null tail probability no greater than \(\alpha/(mK)\) to each arm-stage test guarantees a cumulative FWER no greater than \(\alpha\), by the union bound. This approach does not require the joint dependence structure, but it can be conservative because it does not use the fact that crossings may occur together. Armwise boundaries or single-stage maximum-statistic calibration are also alternatives, but they do not automatically solve the full repeated-look path probability problem.

For a practical calibration, first verify that the covariance matrix is symmetric and positive semidefinite, and that its entries correspond to the design’s actual stage information and shared-control structure. Then check that increasing an upper boundary decreases the probability of crossing it; this monotonicity is a useful numerical diagnostic when root-finding. Finally, recompute the cumulative FWER at the reported, rounded boundaries. Rounding can move the probability away from the target, so the unrounded solution should be retained for verification when possible.

Common errors include using independent-arm calculations despite a shared control, counting later crossings on paths already rejected, and summing overlapping arm-specific crossing probabilities as if they were disjoint. The correction is to define the full no-rejection event through each stage and evaluate its joint probability under the null covariance model. Calibration also has limits: misspecified normality or covariance can invalidate nominal error control, and a global-null calibration alone does not establish strong control for every intersection hypothesis. The appropriate null hypothesis and testing framework must be specified for the error rate being claimed.