Why a Boundary-Crossing Probability Needs the Full History
In a multi-arm, multi-stage test, a rejection can occur if any arm crosses its efficacy boundary at any analysis. The probability of at least one such crossing depends not only on each arm’s marginal distribution, but also on dependence between arms and between stages. After an analysis at which no boundary is crossed, the observed statistics have been restricted to a particular region. That restriction changes the distribution of the statistics at later analyses whenever the stages are dependent.
The recursive calculation below keeps the joint density of all stage statistics along histories that have survived. Integrating that density gives the probability of no crossing yet; subtracting from one gives the probability of at least one crossing. This is useful both for evaluating a proposed set of boundaries and for calibrating boundaries to a target familywise error rate.
Model, Boundaries, and Surviving Histories
Suppose there are \(J\) arms and \(T\) analyses. Under the global null hypothesis, let \(Z_t=(Z_{1t},\ldots,Z_{Jt})^\mathsf{T}\) be the vector of standardized test statistics at stage \(t\). Stack the stage vectors as \(Z_{1:T}=(Z_1^\mathsf{T},\ldots,Z_T^\mathsf{T})^\mathsf{T}\). Assume this stacked vector has a specified, nonsingular multivariate-normal distribution under the null. The means are zero, and the covariance matrix includes the dependence between arms and stages. The covariance structure must reflect the design and the way the statistics are calculated.
For an upper-boundary test, write \(b_{jt}\) for the boundary for arm \(j\) at stage \(t\). A crossing occurs at stage \(t\) if at least one \(Z_{jt}\geq b_{jt}\). Define the stage-\(t\) non-crossing region \(R_t=\{z_t:z_{jt}<b_{jt}\text{ for every }j\}\). The complete region of histories with no crossing through stage \(t\) is \(A_t=R_1\times\cdots\times R_t\).
The conditional density of \(Z_t\) given \(Z_1=z_1,\ldots,Z_{t-1}=z_{t-1}\) is determined by the joint multivariate-normal model. Denote it by \(f_t(z_t\mid z_1,\ldots,z_{t-1})\). For a multivariate normal vector, this conditional distribution is also multivariate normal, with a conditional mean that can depend on the preceding statistics and a conditional covariance determined by the joint covariance matrix. The recursion uses that conditional density, rather than treating stages or arms as independent.
The Survivor-Density Recursion
At stage 1, the density is retained only in the non-crossing region. At each later stage, extend each surviving history by the conditional density of the new stage, then discard extensions that cross a boundary. This gives
The recursion is a density on the entire history \((z_1,\ldots,z_t)\). In particular, it retains the previous-stage coordinates; it does not integrate them out while defining \(g_t\). The survival probability and the probability of a first crossing at stage \(t\) are
The familywise error through stage \(t\) is therefore \(1-S_t\), and by the final analysis it is \(1-S_T\). These statements concern the global null; strong familywise error control across configurations with some true and some false null hypotheses requires the appropriate testing framework and assumptions, such as those addressed by subset pivotality and closed testing.
Why the Recursion Is Correct
The joint density of \((Z_1,\ldots,Z_t)\) factors into the density of the first stage and the successive conditional densities. The indicator of survival through stage \(t\) is the product of the indicators that each stage lies in its non-crossing region. Consequently, the joint density multiplied by that indicator is exactly the subdensity above. Equivalently, the recursion follows by induction: it is true by definition at stage 1; multiplying \(g_{t-1}\) by the conditional density appends stage \(t\) to each surviving history, and multiplying by \(\mathbf{1}\{z_t\in R_t\}\) retains precisely the extensions that survive stage \(t\). Integrating over \(A_t\) therefore gives the probability of the event defining \(S_t\).
Since surviving through stage \(t\) implies surviving through stage \(t-1\), \(S_t\leq S_{t-1}\). The difference \(S_{t-1}-S_t\) is thus nonnegative and counts histories that survived to the start of stage \(t\) but crossed at that stage. Summing these first-crossing probabilities telescopes:
This identity is a useful check on a computation: the stage-specific crossing probabilities must sum to the cumulative crossing probability.
Worked Example: One-Stage Familywise Error With Independent Arms
Suppose two standardized statistics are independent standard normal variables under the global null, and each has the same one-sided boundary \(b=2.3263\). Assume both tests use the same analysis and a crossing by either arm counts as a familywise rejection. Check the model assumptions: the example specifies standard normal null distributions, independence between arms, and no earlier stages.
For one arm, \(P(Z_{j1}<2.3263)\approx0.9900\). Independence gives the probability that both arms remain below their boundaries:
Therefore the familywise error under the global null is \(1-S_1=1-0.9801=0.01990\), or about 1.99%. This is larger than the 1.00% one-sided error for either individual arm because either arm can produce a crossing. The calculation relies on the stated independence; it would not generally be valid with shared-control correlation.
Worked Example: Two Correlated Looks at One Arm
Consider one arm with two standard normal statistics under the null, with correlation \(0.5\), and boundary zero at both stages. This is a deliberately simple numerical illustration of the recursion, not a realistic efficacy-boundary choice. The specified bivariate-normal model is nonsingular, and the same boundary is applied at each look.
The first-stage survival probability is \(S_1=P(Z_1<0)=0.5\). The second-stage survival probability is the probability that both coordinates are below zero. For a standard bivariate normal pair with correlation \(\rho\), symmetry gives
For completeness, this identity follows by differentiating the bivariate normal probability at the origin with respect to \(\rho\): its derivative is \(1/(2\pi\sqrt{1-\rho^2})\). At \(\rho=0\) the probability is \(1/4\); integrating the derivative from 0 to \(\rho\) gives the displayed expression. Substituting \(\rho=0.5\) gives
Thus the probability of a first crossing at stage 1 is \(S_0-S_1=1-0.5=0.5\). The probability of surviving stage 1 but first crossing at stage 2 is \(S_1-S_2=0.5-1/3=1/6\). The probability of any crossing is \(0.5+1/6=2/3\), which also equals \(1-S_2\). Multiplying the two marginal non-crossing probabilities would give \(0.5^2=0.25\), not \(1/3\); that shortcut incorrectly assumes independence between looks.
Using the Recursion to Calibrate Boundaries
Boundary calibration reverses the calculation: choose boundary values so that the final familywise error under the global null equals a specified target \(\alpha\). For fixed covariance and a family of boundaries parameterized by a common value, the recursion or an equivalent multivariate-normal integral can be evaluated repeatedly while searching for the value that satisfies \(1-S_T=\alpha\). With stage-specific boundaries, calibration may instead solve for several values under additional design constraints, such as a chosen spending pattern.
Worked Example: Calibrating a Common One-Stage Boundary
Suppose two independent standard normal arm statistics are tested once, and the target global-null familywise error is \(\alpha=0.05\). Use a common one-sided boundary \(b\). Independence and the normal model are explicit assumptions for this example.
The target requires \(1-\Phi(b)^2=0.05\), where \(\Phi\) is the standard normal cumulative distribution function. Rearranging,
Each arm’s marginal upper-tail probability is approximately \(1-0.974679=0.02532\). The probability of at least one crossing is \(1-(0.974679)^2=0.05000\), up to rounding. Thus the common boundary achieves the target familywise error under the stated null model; it is not the same as assigning 0.05 to each arm.
Computation, Alternatives, and Diagnostics
A direct multivariate-normal probability calculation over the complete non-crossing region is an alternative to recursive integration. It can be convenient for a modest number of stages, but the dimension grows with the number of arms and stages. The survivor-density recursion expresses the same probability as successive conditional integrations and can make the first-crossing probabilities available along the way. Both approaches require the same correct joint null model. Monte Carlo simulation is another alternative: simulate complete histories from that model, record whether any boundary is crossed, and estimate the crossing proportion. Simulation is flexible, but its estimates have Monte Carlo error and may require many simulations for small error probabilities.
A practical implementation may store or approximate the surviving density rather than evaluate every history exactly. Regardless of method, check the following:
- Covariance validity. The joint covariance matrix must be symmetric and positive semidefinite; a nonsingular Gaussian density requires positive definiteness. Check that its correlations represent both shared-control and repeated-look dependence in the design.
- Probability bounds and monotonicity. Verify \(0\leq S_t\leq S_{t-1}\leq1\). A negative stage-specific crossing probability signals an integration, indexing, or numerical-approximation error.
- Agreement of cumulative and incremental calculations. Check that the sum of the stage-specific first-crossing probabilities equals \(1-S_T\), to numerical tolerance.
- Numerical integration accuracy. Repeat calculations with finer integration settings or an independent method. In simulation, report or assess Monte Carlo uncertainty; a small estimated error rate can be imprecise even when no crossings are observed.
- Boundary convention. For continuous statistics, using \(Z_{jt}\geq b_{jt}\) for crossing or \(Z_{jt}>b_{jt}\) makes no probability difference. For discrete statistics, boundary equality may have positive probability and the convention must be applied consistently.
Bonferroni boundaries offer a simpler alternative that controls familywise error without using the joint dependence to its full extent: allocate a total error budget across tests and use a valid bound. This can be conservative compared with calibration using the correct joint distribution. Conversely, the recursive calculation is only as trustworthy as its specified model and boundary rule. It does not repair a misspecified covariance structure, automatically establish strong control for every configuration of true and false nulls, or account for selection or design changes not included in the assumed model.
Common Pitfall
A related error is to multiply marginal non-crossing probabilities as if all arm-stage statistics were independent. That is valid only under an independence assumption that applies to the entire set of statistics involved. Shared controls and repeated looks commonly induce dependence, so use the joint model rather than a product of marginal probabilities.
Check Your Understanding
Try each question, then check the concise solutions below.
- In the recursion for \(g_t\), which variables does the density \(g_t\) depend on, and what does its integral over \(A_t\) equal?
- If \(S_1=0.92\) and \(S_2=0.88\), what is the probability of a first crossing at stage 2, and what is the cumulative crossing probability through stage 2?
- Why is it generally wrong to calculate multi-stage survival by multiplying the marginal survival probabilities at each stage?
- What condition should the joint covariance matrix satisfy for the specified nonsingular multivariate-normal model to be valid?
Solutions. (1) \(g_t\) depends on the full history \((z_1,\ldots,z_t)\); its integral over \(A_t\) is \(S_t\), the probability of no crossing through stage \(t\). (2) The stage-2 first-crossing probability is \(S_1-S_2=0.92-0.88=0.04\); cumulative crossing is \(1-S_2=0.12\). (3) Stage statistics may be dependent, so survival at a later stage is affected by the earlier values and by conditioning on earlier survival. (4) The covariance matrix must be symmetric and positive definite for a nonsingular multivariate-normal density.