Correlations of Nested Z Statistics

Why Nested Z Statistics Are Correlated

At an interim analysis, a cumulative statistic uses the observations available by that look. At a later analysis, the cumulative statistic reuses those observations and adds new information. The two statistics therefore share part of their random variation. Standardizing each statistic to have variance one does not remove that shared variation: the resulting Z statistics are generally correlated.

Knowing this correlation matters because joint calculations across looks depend on the joint distribution, not just on the marginal distribution at each look. In the canonical joint normal model, the correlation is determined by how much information the two cumulative statistics share. The result below makes that relationship explicit.

Definition: Let \(U_k\) be a cumulative, unstandardized statistic at look \(k\), and let \(I_k\) be its information, with \(I_k>0\). Its information-standardized statistic is \(Z_k=U_k/\sqrt{I_k}\). For two nested looks, label the earlier look 1 and the later look 2, so \(0<I_1\leq I_2\). In the canonical model, \(\operatorname{Var}(U_k)=I_k\) and \(\operatorname{Cov}(U_1,U_2)=I_1\). Thus the earlier statistic's information is the information shared by the two cumulative statistics.

These covariance assumptions describe the canonical joint normal setting: the cumulative statistics are jointly normal, their variances equal their information levels, and the earlier data contribute to both looks. Under the null hypothesis, their means are zero; the correlation calculation itself follows from the covariance structure. The same correlation holds under a canonical fixed alternative when the mean changes but this covariance structure remains in force. The formula is not a universal property of any pair of statistics called Z statistics; it depends on these assumptions.

Deriving the Square-Root Information Ratio

Correlation is covariance divided by the product of the standard deviations. Since \(Z_1=U_1/\sqrt{I_1}\) and \(Z_2=U_2/\sqrt{I_2}\), scaling the two statistics gives their covariance, while the variance assumptions give each standardized statistic variance one.

\begin{aligned} \operatorname{Cov}(Z_1,Z_2) &=\operatorname{Cov}\left(\frac{U_1}{\sqrt{I_1}},\frac{U_2}{\sqrt{I_2}}\right)\\ &=\frac{\operatorname{Cov}(U_1,U_2)}{\sqrt{I_1I_2}} =\frac{I_1}{\sqrt{I_1I_2}} =\sqrt{\frac{I_1}{I_2}}. \end{aligned}

Also, \(\operatorname{Var}(Z_k)=\operatorname{Var}(U_k)/I_k=1\) for \(k=1,2\). Therefore the covariance of \(Z_1\) and \(Z_2\) is their correlation. Equivalently, applying the definition of correlation directly gives

\operatorname{Corr}(Z_1,Z_2) =\frac{\operatorname{Cov}(U_1,U_2)} {\sqrt{\operatorname{Var}(U_1)\operatorname{Var}(U_2)}} =\frac{I_1}{\sqrt{I_1I_2}} =\sqrt{\frac{I_1}{I_2}}.

Theorem: For two nested cumulative statistics in the canonical model, with \(0<I_1\leq I_2\), the correlation of their information-standardized statistics is \(\sqrt{I_1/I_2}\). It is positive and no greater than one. It equals one when the information levels are equal and decreases as the later information becomes larger relative to the earlier information.

The square root appears because correlation uses standard deviations, not variances. The information ratio \(I_1/I_2\) is the ratio of variances for the unstandardized statistics; taking the square root converts that ratio to the scale of their standardized correlation. When information doubles, for example, the correlation is not \(1/2\), but \(1/\sqrt{2}\).

Worked Example: Information Levels of 50 and 200

Suppose two nested looks have information \(I_1=50\) and \(I_2=200\), in consistent information units. Assume the canonical covariance structure: \(\operatorname{Var}(U_1)=50\), \(\operatorname{Var}(U_2)=200\), and \(\operatorname{Cov}(U_1,U_2)=50\). Find the correlation of the standardized statistics.

The standard deviations of the unstandardized statistics are \(\sqrt{50}\) and \(\sqrt{200}\). Thus

\operatorname{Corr}(Z_1,Z_2) =\frac{50}{\sqrt{50}\sqrt{200}} =\frac{50}{\sqrt{10000}} =\frac{50}{100} =0.500.

The later look has four times the information of the earlier one, so the correlation is \(\sqrt{50/200}=\sqrt{1/4}=0.500\). The exact square roots were retained in the calculation; no rounded standard deviations were treated as exact. This is the model-based correlation across repeated realizations, not a correlation estimated from one pair of observed statistics.

Worked Example: Closely Spaced Information Levels

Consider two nested looks with \(I_1=120\) and \(I_2=180\). Assume the canonical model, so the covariance of the cumulative unstandardized statistics is 120 and their variances are 120 and 180. Calculate and interpret the correlation.

Using the covariance definition,

\operatorname{Corr}(Z_1,Z_2) =\frac{120}{\sqrt{120}\sqrt{180}} =\sqrt{\frac{120}{180}} =\sqrt{\frac{2}{3}} \approx 0.8165.

The standardized statistics have a strong positive correlation of about 0.8165. The later look adds information, but the earlier information is still two-thirds of the later total. Consequently, substantial random variation is shared. This does not mean that the observed values must be close in every trial; it describes their joint distribution over repeated realizations under the model.

Checking the Result in a Normal-Mean Setting

A simple model illustrates why the shared-information covariance equals the earlier information. Suppose \(X_1,X_2,\ldots\) are independent observations with \(X_r\sim N(\mu,\sigma^2)\), where \(\sigma\) is known. For testing a specified null mean \(\mu_0\), define the cumulative score-like statistic at sample size \(n\) as

U_n=\frac{1}{\sigma^2}\sum_{r=1}^{n}(X_r-\mu_0), \qquad I_n=\frac{n}{\sigma^2}.

Under \(H_0:\mu=\mu_0\), each summand in \(U_n\) has mean zero and variance \(1/\sigma^2\). Independence therefore gives \(\operatorname{Var}(U_n)=n/\sigma^2=I_n\). If \(n_1\leq n_2\), the two sums share exactly the first \(n_1\) observations. The remaining terms in the later sum are independent of those shared terms, so the covariance is the variance of the shared sum, \(n_1/\sigma^2=I_{n_1}\). Standardizing yields the stated information-ratio formula. This argument verifies the assumptions in this particular model; other applications need their own justification for the covariance structure.

Worked Example: Nested Normal Samples

For a hypothetical normal-mean analysis, suppose \(X_r\) are independent with known standard deviation \(\sigma=12\) units, the null mean is \(\mu_0=100\) units, and the looks occur at \(n_1=36\) and \(n_2=144\). Assume the model above. Find the correlation, and calculate the two observed Z statistics if the first 36 observations sum to 24 units above the null total and the next 108 sum to 60 units below the null total.

The information levels are \(I_1=36/12^2=0.25\) and \(I_2=144/12^2=1\), in reciprocal squared outcome units. The observations are independent and the second sample contains the first, so the covariance of the cumulative statistics is \(I_1=0.25\). Hence

\operatorname{Corr}(Z_1,Z_2) =\frac{0.25}{\sqrt{0.25}\sqrt{1}} =0.500.

The first cumulative sum of deviations is 24 units, and the full sum at the second look is \(24-60=-36\) units. Using \(Z_n=\sum_{r=1}^n(X_r-\mu_0)/(\sigma\sqrt{n})\), the observed statistics are

Z_1=\frac{24}{12\sqrt{36}}=\frac{24}{72}=0.3333, \qquad Z_2=\frac{-36}{12\sqrt{144}}=\frac{-36}{144}=-0.2500.

These realized values happen to have opposite signs, but that does not contradict the positive model-based correlation of 0.500. Correlation describes the tendency across repetitions, not a requirement that every realized pair move in the same direction.

When the Formula Applies—and When It Does Not

The essential ingredients are nested cumulative statistics, a variance equal to information at each look, and covariance equal to the earlier information. In practice, check that the later statistic really reuses the earlier data in the specified way, and that the information values correctly represent the variances under the model. In a model with changing weights, dependence among observations, estimated nuisance parameters, or a different variance structure, the covariance may not equal \(I_1\). The square-root ratio should not be used merely because two looks are ordered in time.

A useful check is to calculate correlation from the covariance directly. If the assumed quantities are \(\operatorname{Cov}(U_1,U_2)=C\), \(\operatorname{Var}(U_1)=V_1\), and \(\operatorname{Var}(U_2)=V_2\), then the standardized correlation is \(C/\sqrt{V_1V_2}\). The square-root information ratio follows only after substituting \(C=I_1\), \(V_1=I_1\), and \(V_2=I_2\). Also verify that the resulting value lies between \(-1\) and 1; a value outside that range signals inconsistent covariance or variance inputs.

Approach	What it uses	When it is appropriate
Square-root information ratio	\(I_1\), \(I_2\), and the canonical nested covariance assumptions	Nested canonical cumulative statistics with covariance equal to earlier information
Direct covariance calculation	The actual covariance and variances of the two unstandardized statistics	Whenever those quantities are known, including settings where the canonical assumptions need checking
Independent-statistics approximation	Assumes correlation zero	Not appropriate for nested cumulative statistics with positive shared information

The direct covariance calculation is the general route; the information-ratio result is its simplified form under canonical assumptions. Treating nested statistics as independent is not a harmless shortcut: when their correlation is positive, it changes joint probabilities and can distort calculations involving repeated looks. Conversely, using the formula with inappropriate information values can be just as misleading.

Common Mistakes and Practical Diagnostics

Using the information ratio without the square root. The ratio \(I_1/I_2\) is not the correlation. Correlation is \(I_1/\sqrt{I_1I_2}=\sqrt{I_1/I_2}\).
Using the later information in the covariance. Under the nested canonical model, the shared information is the earlier information \(I_1\), not \(I_2\).
Assuming that standardization makes looks independent. Each statistic may have variance one while the pair remains correlated through shared data.
Inferring correlation from one observed pair. A single pair of values cannot establish the model-based correlation; use the covariance structure or repeated data under a suitable model.
Ignoring departures from the model. If the information does not match the variance or the covariance is not the earlier information, calculate correlation from the actual covariance and variances instead.

The formula also has clear boundary behavior. If \(I_2=I_1>0\), there is no additional information between the looks and the standardized statistics coincide under the canonical cumulative construction, giving correlation 1. As \(I_2/I_1\) grows, the correlation decreases toward zero, though it remains positive for finite positive \(I_1\). A look with \(I_1=0\) cannot produce an information-standardized statistic, so the formula is defined only for positive information.

Key takeaway: For nested canonical cumulative statistics, shared covariance equals the earlier information. Dividing by the standard deviations therefore gives \(\operatorname{Corr}(Z_1,Z_2)=\sqrt{I_1/I_2}\). Confirm the covariance assumptions before applying the shortcut.

Check Your Understanding

Answer each question, then compare with the brief solutions.

Two nested looks have information 25 and 100. Under the canonical model, what is the correlation?
Why is the correlation not \(I_1/I_2\)?
For information levels \(I_1=80\) and \(I_2=200\), what covariance between the unstandardized statistics is required by the canonical nested model, and what is the standardized correlation?
Two standardized statistics have observed values 1.1 and -0.2. Does this pair show that their model-based correlation is negative? Explain.
If two ordered analyses use nonnested data, can the square-root information ratio automatically be used? What quantity should be checked?

Solutions. (1) \(\sqrt{25/100}=0.500\). (2) The ratio is a variance ratio; correlation divides covariance by the product of standard deviations, yielding its square root. (3) The required covariance is \(I_1=80\), and the correlation is \(\sqrt{80/200}=\sqrt{0.4}\approx0.6325\). (4) No. One realized pair does not determine the joint model's correlation; positively correlated variables can have opposite signs in a particular realization. (5) No. Check the actual covariance and both variances, then calculate correlation as covariance divided by the product of standard deviations.