When Cumulative Statistics Are Not Independent

Why Dependence Between Looks Matters

In a group sequential design, a test statistic is evaluated at more than one planned look. A tempting shortcut is to treat the statistics at different looks as independent because each look produces a separate number. That shortcut is generally wrong for cumulative statistics: a later statistic includes information already represented in the earlier one. The shared information creates covariance, which affects conditional probabilities and the joint behavior of the monitoring statistics.

The issue is not whether separate calculations are performed at separate times. It is whether the random quantities being calculated share data or score information. Under the canonical model, the covariance can be derived directly from the information times. Once it is known, the bivariate normal distribution gives a precise account of the dependence.

Definition: Let \(0<t_1<t_2\) be fixed information times. Write \(U(t)\) for the cumulative score-like statistic through information time \(t\), and define the standardized cumulative statistic \(Z(t)=U(t)/\sqrt{t}\). Under the canonical null model, \(U(t)\) has mean zero, variance \(t\), and cumulative values at two times have covariance \(\operatorname{Cov}(U(t_1),U(t_2))=t_1\). The pair \((Z(t_1),Z(t_2))\) is jointly normal with mean zero and unit marginal variances.

This model is exact for the usual Gaussian canonical construction. In regular models it may instead be an asymptotic approximation for the score statistics. The information times are treated as fixed, the increments over disjoint information intervals have mean zero and variances equal to the interval lengths under \(H_0\), and the cumulative statistics use the same endpoint and analysis definition at both looks. A covariance calculation based on this model should not be transferred automatically to statistics with different definitions or materially data-dependent information times.

Deriving the Two-Look Covariance

The later cumulative statistic contains the contribution accumulated by the first look. Thus \(U(t_2)\) can be written as \(U(t_1)\) plus the increment from \(t_1\) to \(t_2\). Under the canonical model, that increment is independent of \(U(t_1)\) and has mean zero. Consequently, its covariance with \(U(t_1)\) is zero, leaving the covariance of the two cumulative scores equal to the variance accumulated by the first look.

\begin{aligned} \operatorname{Cov}\{Z(t_1),Z(t_2)\} &=\frac{\operatorname{Cov}\{U(t_1),U(t_2)\}}{\sqrt{t_1t_2}}\\ &=\frac{t_1}{\sqrt{t_1t_2}}\\ &=\sqrt{\frac{t_1}{t_2}}. \end{aligned}

Each standardized statistic has variance one, so its covariance with the other statistic is also their correlation. For \(0<t_1<t_2\), the correlation is strictly between zero and one. In particular, it is not zero, and the statistics cannot be independent. The implication uses only the general fact that independent random variables with finite second moments have zero covariance; joint normality also lets us describe their conditional distribution explicitly.

This covariance does not mean that the two test statistics are numerically identical or that the later look adds no new information. It quantifies how strongly their random variation is connected. The amount of connection is determined by the fraction of total information already present at the first look, \(t_1/t_2\).

Worked Example: Covariance at Two Equally Spaced Information Levels

Suppose the first look occurs at information \(t_1=0.5\), and the second at \(t_2=1\). Assume the canonical null model holds: the cumulative score has variance equal to information, and the increment from \(0.5\) to \(1\) is independent of the earlier score.

The covariance and correlation of the standardized cumulative statistics are

\operatorname{Cov}\{Z(0.5),Z(1)\} =\operatorname{Corr}\{Z(0.5),Z(1)\} =\sqrt{\frac{0.5}{1}} =\sqrt{0.5} \approx 0.7071.

For a direct check, take independent increments \(Y_1\sim N(0,0.5)\) and \(Y_2\sim N(0,0.5)\), so \(U(0.5)=Y_1\) and \(U(1)=Y_1+Y_2\). Then

\operatorname{Cov}\left\{\frac{Y_1}{\sqrt{0.5}},\frac{Y_1+Y_2}{\sqrt{1}}\right\} =\frac{\operatorname{Var}(Y_1)+\operatorname{Cov}(Y_1,Y_2)}{\sqrt{0.5}} =\frac{0.5+0}{\sqrt{0.5}} \approx 0.7071.

The calculation checks the model assumptions by specifying independent, mean-zero normal increments with the stated variances. Since the resulting correlation is nonzero, the two cumulative statistics are dependent, despite the independence of the increments.

Conditional Behavior at the Later Look

Under the canonical null model, the joint normal distribution and the covariance determine the conditional distribution at the second look. If \(\rho=\sqrt{t_1/t_2}\), then

Z(t_2)\mid Z(t_1)=z \sim N\left(\rho z,\;1-\rho^2\right).

The first argument is the conditional mean and the second is the conditional variance. The conditional mean changes with the observed first-look statistic: a positive first-look value shifts the conditional distribution at the later look upward, while a negative value shifts it downward. The conditional variance is below one because knowing the first-look value provides information about the later cumulative statistic.

Worked Example: A Conditional Upper-Tail Probability

Suppose \(t_1=0.25\) and \(t_2=1\), and the canonical null model is appropriate. Consider the conditional probability that the later statistic exceeds \(1.96\), given that the first-look statistic is \(2\).

Here \(\rho=\sqrt{0.25/1}=0.5\). Therefore the conditional mean is \(0.5(2)=1\), and the conditional variance is \(1-0.5^2=0.75\). The conditional standard deviation is \(\sqrt{0.75}\). Standardizing the boundary gives

\frac{1.96-1.00}{\sqrt{0.75}} =\frac{0.96}{0.866025\ldots} \approx 1.109.

Thus

P\{Z(t_2)>1.96\mid Z(t_1)=2\} =1-\Phi(1.109) \approx 0.134.

This is a conditional probability under the canonical null model, evaluated at the specified first-look value. If the statistics were independent, the conditional probability would instead equal the marginal upper-tail probability \(P\{N(0,1)>1.96\}\approx 0.0250\). The independence assumption would therefore substantially understate the conditional chance of crossing this later boundary in this example.

Conditioning on \(Z(t_1)=2\) refers to the conditional distribution given that value, not to an event with positive probability of an exact equality. The reported probability describes the model’s later-look behavior at that conditioning value. It is not a probability that \(H_0\) is true, and the example does not say that observing a first-look value of 2 causes a later crossing.

Dependence Under a Fixed Alternative

The same covariance structure also applies under the canonical fixed-alternative model. In that model the cumulative statistic has a drift, while its covariance structure remains the canonical one. A nonzero mean does not remove dependence: covariance is calculated after centering the variables by their means.

Worked Example: Dependence With a Nonzero Drift

Consider information times \(t_1=0.4\) and \(t_2=1\). Suppose a fixed alternative gives \(U(t)\) mean \(\theta t\), with \(\theta=0.3\), and the centered cumulative process has the canonical covariance. The standardized means are

E\{Z(0.4)\}=\frac{0.3(0.4)}{\sqrt{0.4}}\approx 0.1897, \qquad E\{Z(1)\}=\frac{0.3(1)}{\sqrt{1}}=0.3000.

After subtracting these means, the covariance is

\operatorname{Cov}\{Z(0.4),Z(1)\} =\frac{\operatorname{Cov}\{U(0.4),U(1)\}}{\sqrt{0.4}} =\frac{0.4}{\sqrt{0.4}} =\sqrt{0.4} \approx 0.6325.

The model assumptions here are a fixed information schedule, the stated linear drift, and canonical covariance for the centered process. The nonzero covariance shows that the two statistics remain dependent under this alternative as well. Their means differ from the null means, but their dependence is not eliminated by that shift.

How to Diagnose an Independence Error

Before using a joint or conditional calculation, write down what each statistic accumulates. If the later statistic contains contributions already included in the earlier one, independent marginal calculations cannot by themselves justify independence. Under the canonical model, calculating \(\sqrt{t_1/t_2}\) is a direct diagnostic: any positive value confirms nonzero covariance.

A useful numerical check is to simulate independent increments under the assumed model, form the cumulative statistics at both looks, and calculate their empirical correlation. It should be near \(\sqrt{t_1/t_2}\), with deviations attributable to simulation error. A correlation near zero would prompt checks of the statistic construction, information times, and whether the simulation actually reuses the earlier increment in the later cumulative value. Simulation is a check on implementation, not a replacement for the covariance derivation.

Approach	What it assumes or uses	Consequence
Canonical joint calculation	Uses both marginal variances and \(\operatorname{Corr}\{Z(t_1),Z(t_2)\}=\sqrt{t_1/t_2}\).	Supports correct conditional probabilities and joint monitoring calculations under the model.
Independent-statistics approximation	Sets the covariance to zero without deriving it from the statistic definitions.	Can miscalculate conditional behavior and the probability of events involving both looks.
Increment-based construction	Models the separate contributions over disjoint information intervals, then forms cumulative statistics.	Preserves the model’s covariance when the cumulative statistics are assembled.

The correct alternative to a mistaken independence assumption is not to discard the sequential structure. Use the joint distribution with its covariance, or calculate from the increments and then form the cumulative statistics. A product of marginal probabilities is justified for two events only when the events are independent; nonzero covariance here rules out independence of the jointly normal statistics. For more than two looks, the same principle requires using the full covariance matrix rather than treating each pair or each marginal test as isolated.

Common Mistake and Limitations

Wrong reasoning: “The tests are performed at different looks, so their statistics are independent.” Separate analysis times do not establish independence; cumulative statistics may reuse earlier information.
Correction: Specify the statistics and calculate their covariance. In the two-look canonical model it is \(\sqrt{t_1/t_2}\), which is positive whenever \(0<t_1<t_2\).
Wrong reasoning: “The increments are independent, so the cumulative statistics must be independent.” Independence of disjoint increments is not a claim that sums containing shared increments are independent.
Correction: Derive the covariance of the cumulative quantities, or construct them from the increments explicitly as in the first worked example.

The result depends on the specified canonical model. If the statistics are not jointly normal, zero covariance alone would not establish independence; conversely, covariance can still diagnose dependence when it is nonzero and finite. If the information schedule is random, the endpoint or score definition changes between looks, or the increments do not follow the assumed covariance structure, the formula may not apply as written. Those conditions call for a model-specific joint calculation rather than an automatic independence assumption or automatic use of the canonical formula.

Key takeaway: Under the canonical two-look model, standardized cumulative statistics have correlation \(\sqrt{t_1/t_2}\), not zero. Use this dependence in conditional and joint calculations; do not infer independence merely because the looks occur at different times.

Check Your Understanding

Answer each question using the canonical model unless a question states otherwise. Brief solutions follow.

At information times \(t_1=0.36\) and \(t_2=1\), what are the covariance and correlation of the standardized cumulative statistics?
At \(t_1=0.25\), \(t_2=1\), and \(\rho=0.5\), what is the conditional mean of \(Z(t_2)\) if \(Z(t_1)=-1.2\)?
Why does a nonzero covariance establish that the two statistics are not independent?
Suppose two statistics are marginally standard normal. Is that enough to justify multiplying their upper-tail probabilities to obtain the probability that both exceed their boundaries?

Solutions. (1) The covariance and correlation are \(\sqrt{0.36/1}=0.6000\). (2) The conditional mean is \(\rho z=0.5(-1.2)=-0.6000\); the conditional variance is \(1-0.5^2=0.75\). (3) Independent variables with finite second moments have covariance zero, so a nonzero covariance contradicts independence. (4) No. Marginal standard normal distributions do not imply independence; under the canonical model the positive correlation must be included in the joint probability calculation.