Calibrating a Maximum-Statistic Boundary Under the Global Null

Why Calibrate the Maximum?

In a multi-arm, multi-stage trial, there are several opportunities to declare efficacy: an arm might cross a boundary at an interim look, or at the final look. If each arm-stage statistic is compared with a single-arm cutoff, the probability of at least one false efficacy declaration can exceed the nominal level. The relevant event under the global null is that at least one statistic crosses its boundary.

A common maximum-statistic boundary gives a direct way to calibrate that event when every planned test uses the same one-sided cutoff. The cutoff depends not only on the number of statistics but also on their joint distribution. Shared controls and repeated observations generally make the statistics correlated, so treating them as independent can give the wrong crossing probability.

Definition: In a two-arm, two-stage design, let \(Z_{ij}\) be the standardized, one-sided efficacy statistic for arm \(i\) at stage \(j\), where \(i=1,2\) and \(j=1,2\). Define \(M=\max(Z_{11},Z_{21},Z_{12},Z_{22})\). A common critical value \(c\) rejects at least one arm-stage null hypothesis if \(M\geq c\). Under the global null \(H_{\mathrm{global}}\), all treatment effects are zero. The global-null crossing probability is \(P(M\geq c\mid H_{\mathrm{global}})\).

Model and Calibration Equation

For the calibration below, assume that under \(H_{\mathrm{global}}\) the four statistics have a continuous, jointly normal distribution with mean zero, unit marginal variances, and a known positive-definite correlation matrix \(R\). The statistics are oriented so that large positive values favor efficacy. The test uses an upper-tail boundary, and the target global-null crossing probability is \(\alpha\). The design and the set of four planned testing opportunities are fixed before observing the outcomes.

These assumptions make the joint distribution, and therefore the crossing probability, determined by \(R\). For illustration, suppose both experimental arms share a control and the interim look occurs at one half of the final information. Use an arm-to-arm correlation of 0.50 at a given look and a same-arm correlation of \(\sqrt{0.5}\) between looks. Under this idealized shared-control model, cross-arm, cross-look correlations are the product \(0.50\sqrt{0.5}\). In the ordering \((Z_{11},Z_{21},Z_{12},Z_{22})\), the matrix is

R= \begin{pmatrix} 1 & 0.5000 & 0.7071 & 0.3536\\ 0.5000 & 1 & 0.3536 & 0.7071\\ 0.7071 & 0.3536 & 1 & 0.5000\\ 0.3536 & 0.7071 & 0.5000 & 1 \end{pmatrix}.

This matrix is a model input, not a universal MAMS covariance matrix. In a real design it must reflect the allocation, control sharing, information at each look, and the definitions of the statistics. In particular, the correlation matrix must be valid: it must be symmetric, have ones on the diagonal, and be positive semidefinite.

Let \(\Phi_4(c\mathbf{1};R)\) denote the probability that all four components of a mean-zero, unit-variance normal vector with correlation matrix \(R\) are at most \(c\). Since the complement of “all four are below \(c\)” is “at least one is at or above \(c\),” the calibration equation is

P(M\geq c\mid H_{\mathrm{global}}) =1-\Phi_4(c\mathbf{1};R)=\alpha.

Because the joint distribution is continuous, the probability of a statistic being exactly \(c\) is zero, so using \(>\) rather than \(\geq\) does not change this equation. The left-hand side decreases continuously as \(c\) increases. For a nondegenerate joint normal distribution it ranges from 1 to 0, so for \(0<\alpha<1\) there is a unique cutoff solving the equation. The multivariate normal probability is evaluated numerically; a one-dimensional root-finding method then finds \(c\).

Worked Example: Calibrating a Two-Stage Boundary

Suppose a hypothetical trial uses the matrix \(R\) above and requires a one-sided global-null crossing probability of \(\alpha=0.025\). Find the common cutoff.

For a trial value \(c\), evaluate \(g(c)=1-\Phi_4(c\mathbf{1};R)\). The target is \(g(c)=0.025\). Numerical multivariate-normal integration followed by one-dimensional root-finding gives \(c\) approximately \(2.42\). At the rounded value \(c=2.420\), evaluating the crossing probability gives approximately \(0.0250\), to four significant figures.

Thus, under the stated global-null model, the chance that at least one of the four planned statistics reaches 2.420 or higher is about 2.50%. This is a probability of a joint testing event under the specified null and covariance model; it is not a probability that the null hypothesis is true.

The numerical check matters. For this same matrix, evaluating the crossing probability at \(c=2.450\) gives approximately \(0.0233\), which is below the 0.025 target. Therefore 2.450 is too stringent for this calibration. A boundary should be reported with enough digits to reproduce the target probability at the stated precision.

Reading the Boundary in a Trial

The common-boundary rule compares each eligible arm-stage statistic with the same calibrated \(c\). An arm is declared efficacious if its statistic crosses at either stage. For example, with the illustrative cutoff, a statistic of 2.43 crosses and a statistic of 2.41 does not. If an efficacy crossing ends the trial, later statistics need not be observed: the event being controlled is whether any planned look would have crossed.

The calibration concerns the global null, where both experimental treatments have no effect. Under that null, every efficacy declaration is false, so the probability of any crossing is the weak familywise error rate. This calculation alone does not establish control for every configuration in which some treatments are effective and others are not. Nor does it automatically cover a different data-dependent selection or testing rule: the event used for calibration must match the rule actually used.

Worked Example: Applying the Calibrated Cutoff

Suppose the four hypothetical standardized statistics from a trial are \(Z_{11}=1.86\), \(Z_{21}=2.31\), \(Z_{12}=2.18\), and \(Z_{22}=1.74\). Use the rounded common boundary \(c=2.420\).

The observed maximum is

M=\max(1.86,2.31,2.18,1.74)=2.31.

Since \(2.31<2.420\), no statistic crosses the boundary, and this rule makes no efficacy declaration. This decision uses the maximum across both arms and both looks; comparing only a favored arm or only the final-stage values would not implement the stated rule.

If instead one statistic were 2.45, then \(M\geq2.420\), and at least one efficacy declaration would be made. That observation-level decision does not alter the calibrated global-null crossing probability: calibration is a property of the joint null distribution and the pre-specified rule, not of the particular values observed in one trial.

Why Dependence Changes the Cutoff

A simple benchmark shows why the joint distribution cannot be ignored. If four statistics were independent standard normal variables, the probability that all were below \(c\) would be \(\Phi(c)^4\). An exact independent-statistic cutoff for \(\alpha=0.025\) would therefore solve \(1-\Phi(c)^4=0.025\). This is a different model from the shared-control example.

Worked Example: Comparing Independence and Bonferroni

For four independent standard normal statistics, solve

1-\Phi(c)^4=0.025, \qquad c=\Phi^{-1}\!\left((1-0.025)^{1/4}\right).

The probability inside the inverse normal quantile is \(0.975^{1/4}\approx0.99369\), giving \(c\approx2.492\). In contrast, Bonferroni assigns \(0.025/4=0.00625\) to each one-sided test, yielding \(c=\Phi^{-1}(1-0.00625)\approx2.498\). Both calculations are correct for their stated procedures and assumptions, but neither is the shared-control calibration: the independent calculation assumes zero correlations, while Bonferroni controls the overall crossing probability without using the correlations.

For the illustrative correlated matrix, the calibrated common cutoff is about 2.42. Positive dependence tends to make simultaneous extreme values more clustered, changing the probability of at least one crossing compared with independence. The direction and size of the change should be calculated from the actual joint distribution rather than inferred from a rule of thumb.

Diagnostics, Failure Modes, and Limits

Before using a computed cutoff, check that the covariance model and numerical calculation match the design. A plausible-looking value can still be wrong if a correlation is omitted, the order of statistics is mismatched to the matrix, or the probability being evaluated is not the intended crossing event.

Check the hypotheses and tail. The calculation above is for one-sided, upper-tail efficacy tests under the global null. A lower-tail or two-sided rule has a different event and requires its own calibration.
Check the looks and arms. Include every planned testing opportunity covered by the rule. If an arm can be dropped or the set of tested hypotheses depends on interim data, the maximum event must represent that selection process; a fixed four-statistic calculation may no longer describe it.
Check the covariance matrix. Confirm the matrix is symmetric and positive semidefinite, and that its entries match the design’s control sharing and information fractions. Small numerical errors in a matrix can produce invalid or unstable probability calculations.
Check the achieved probability. After solving, substitute the rounded cutoff back into the probability calculation. Report the resulting crossing probability and, if using simulation, its Monte Carlo uncertainty.

For example, a simulation check based on one million independent draws under the global-null model would estimate a probability near 0.025 with approximate Monte Carlo standard error \(\sqrt{0.025(0.975)/1{,}000{,}000}\approx0.000156\). This is useful as a diagnostic, but its random error is larger than the numerical integration error typically sought for a precise cutoff; simulation alone may not verify the final digits.

The maximum-statistic approach is efficient when the planned rule really is a common cutoff over a specified set of jointly normal statistics. Its limitations are equally important: it depends on the model for the joint distribution, it addresses the stated global-null event, and it does not by itself validate a different boundary pattern or a data-dependent procedure. Bonferroni is a simpler alternative that does not require the correlations, often at the cost of a more conservative cutoff. Independence calibration is appropriate only when independence is a defensible assumption.

Key takeaway: A common maximum-statistic cutoff is found by solving \(1-\Phi_m(c\mathbf{1};R)=\alpha\) for the planned joint-null distribution. The covariance matrix, testing event, and achieved probability must all be checked against the design.

Check Your Understanding

Answer each question using the definitions and calculations above.

Why is the global-null crossing event written as \(1-\Phi_4(c\mathbf{1};R)\)?
In the hypothetical trial, what conclusion follows if the four statistics are 2.40, 1.90, 2.02, and 1.77?
What independence assumption is required for the cutoff near 2.492?
Why should the calculated crossing probability be checked after rounding the critical value?

Solutions. (1) \(\Phi_4(c\mathbf{1};R)\) is the probability that all four statistics are below \(c\); its complement is the event that at least one reaches or exceeds \(c\). (2) Their maximum is 2.40, which is below 2.420, so the common-boundary rule makes no efficacy declaration. (3) All four statistics must be mutually independent standard normal variables under the global null for the equation \(1-\Phi(c)^4=0.025\) to apply. (4) Rounding changes the cutoff and hence the crossing probability; substitution verifies that the reported boundary still achieves the target to the stated precision.