Unequal Information Increments Across Stages

Why Unequal Information Increments Matter

In a multi-arm, multi-stage trial, an interim statistic summarizes the evidence accumulated so far. Its dependence on later statistics is determined by how much information was added between looks. If stages contribute unequal increments, treating the looks as equally spaced can give the wrong joint distribution, even when the marginal statistic at each look is correctly standardized. Since boundary calibration depends on the probability of crossing at least one boundary, this can distort familywise type I error.

The key calculation is to build cumulative statistics from stage-specific score increments and then derive their covariance. The resulting covariance matrix—not the labels “first,” “second,” or “final” look by themselves—is the input to a multivariate-normal calibration. The earlier material on multivariate-normal calibration provides the general calibration framework; here the focus is how unequal information increments determine its dependence structure.

Score Increments and the Joint Distribution

Let \(i=1,\ldots,J\) index treatment comparisons and \(k=1,\ldots,K\) index stages. For comparison \(i\), let \(\Delta I_{ik}>0\) be the information added at stage \(k\), and let \(I_{ik}=\sum_{r=1}^{k}\Delta I_{ir}\) be cumulative information. Write \(Y_{ik}\) for the stage-\(k\) score increment, on a scale for which its variance is the information increment under the null. The cumulative standardized statistic is

For the canonical joint model used here, assume the increments have mean zero under the relevant null hypotheses, are jointly multivariate normal, and are independent across stages. Within stage \(r\), let \(\operatorname{Cov}(Y_{ir},Y_{jr})=C_{ij,r}\), with \(C_{ii,r}=\Delta I_{ir}\). These assumptions describe a fixed, prespecified information plan and score process; if the analysis or data collection changes the joint law, the covariance must be justified for that design rather than borrowed unchanged.

In particular, for a single arm comparison, \(C_{ii,r}=\Delta I_{ir}\), so \(\operatorname{Corr}(Z_{ik},Z_{i\ell})=\sqrt{I_{ik}/I_{i\ell}}\) when \(k<\ell\). The correlation is therefore the square root of the earlier information divided by the later information, not necessarily the square root of the look-number ratio. Across comparisons, the covariances \(C_{ij,r}\) reflect shared data and design features such as a common control group. They should come from the specified score covariance model.

Justifying the Covariance Formula

The formula follows directly by expanding the covariance of the two cumulative sums. If \(k\leq \ell\), then the first statistic contains increments \(1,\ldots,k\), while the second contains increments \(1,\ldots,\ell\). Stage independence makes every covariance between distinct stages zero. The only contributions are the matching stages through \(k\):

The same result with \(\min(k,\ell)\) applies when the order of the looks is reversed. Since the increments are jointly normal and the statistics are linear combinations of them, the full vector of \(Z\)-statistics is multivariate normal under the model. This derivation also supplies a practical diagnostic: a proposed covariance matrix should be symmetric, have unit diagonal, and be positive semidefinite. A failure indicates an arithmetic error or an incompatible increment-covariance specification.

From the Joint Law to Familywise-Valid Boundaries

Suppose each hypothesis is tested in the upper direction, with rejection at look \(k\) for comparison \(i\) if \(Z_{ik}\geq b_{ik}\). Under a specified intersection null, define the crossing event as the event that at least one true-null statistic crosses its boundary. A multivariate-normal calibration chooses the boundaries so that the probability of this event is at most \(\alpha\). In integral form, for a set \(S\) of hypotheses treated as true, the calibration condition is

where the probability is computed using the joint null distribution for the statistics in \(S\). Equality is often used when calibrating a common boundary or a prespecified family of boundaries, but a conservative inequality is also valid. For strong familywise control, this condition must hold for every possible set of true null hypotheses, or be combined with a valid closed-testing procedure. Calibrating only under the complete null is not automatically enough: its validity for partial-null configurations requires an additional property, such as the appropriate subset-pivotal joint distribution.

The calibration is a numerical probability calculation. With a fixed covariance matrix, the probability of no crossing is the multivariate-normal probability of the rectangle \(Z_{ik}<b_{ik}\) for all included looks and comparisons; crossing probability is one minus that rectangle probability. Recursive probability calculation or numerical multivariate-normal integration can evaluate it. Unequal increments enter through the covariance matrix, so replacing that matrix by one based on equal information fractions changes the probability being calibrated.

Allocation, Diagnostics, and Limits

A boundary need not be constant across looks. One can allocate error over comparison-look pairs, use prespecified stagewise spending, or numerically calibrate a joint boundary pattern. Whatever choice is made, the allocation or calibration rule should be fixed as part of the design and should account for all opportunities to reject a true null. In particular, using a nominal level \(\alpha\) separately at every look does not control the familywise error at \(\alpha\).

Before using a calibrated boundary, check the assumptions and the calculation:

• Confirm that \(\Delta I_{ik}>0\), that cumulative information is updated by summation, and that every statistic is standardized by the square root of its own cumulative information.
• Check that increment covariances are appropriate for the design and that the resulting joint covariance matrix is symmetric and positive semidefinite.
• Verify that the marginal null distributions used by the calibration are standard normal, or use a justified alternative distribution and calibration method.
• Recompute crossing probabilities with the final boundary values and check the error budget, including the relevant true-null subsets.

A common mistake is to set the correlation between looks to \(\sqrt{k/\ell}\) merely because one statistic is from look \(k\) and the other from look \(\ell\). That expression is appropriate only when information increments are equal across those looks for the comparison in question. In the worked example, the actual within-comparison correlations are \(0.6325\) and \(0.5000\), not a common value obtained from look numbers. A second mistake is to infer strong control from a complete-null calibration alone. The correction is to establish the necessary subset validity or use a procedure that controls error for every relevant intersection.

The canonical calculation has limits. Stage independence and a correct score covariance model are substantive assumptions, not consequences of unequal information arithmetic. If increments are dependent across stages, the numerator of the covariance must include the nonzero cross-stage covariances as well. If information is altered using interim data, or the null distribution changes under alternative configurations, a fixed covariance calibration may not be valid without further justification. The central lesson remains: boundaries are only as reliable as the joint distribution used to calibrate them.