A Worked Three-Look Increment Example

Why Work With Increments?

At three planned analyses, a cumulative statistic at a later look contains information from earlier observations as well as new information. The cumulative statistics are therefore correlated. In the canonical model, subtracting the earlier cumulative score from the later one isolates the contribution from the intervening information increase. Dividing by the square root of that increase gives a standardized increment.

This tutorial works through that conversion at three looks and checks the arithmetic against the information schedule. The key distinction is between a cumulative statistic, which summarizes all information collected so far, and an increment, which summarizes only the information added between adjacent looks.

Definition: Let \(U_k\) be the cumulative score statistic at look \(k\), with cumulative information \(I_k\), for \(k=1,2,3\). Set \(I_0=0\) and \(U_0=0\), and define the information increase \(\Delta I_k=I_k-I_{k-1}\). The raw increment is \(\Delta U_k=U_k-U_{k-1}\). Its standardized increment is \(W_k=\Delta U_k/\sqrt{\Delta I_k}\), provided \(\Delta I_k>0\).

Model and Increment Result

Assume the canonical joint model: the information levels are fixed, strictly increasing values, and the cumulative statistics have a joint normal distribution with \(E(U_k)=\theta I_k\) and \(\operatorname{Cov}(U_j,U_k)=\min(I_j,I_k)\). Here \(\theta\) is the parameter measuring drift from the null; under \(H_0\), \(\theta=0\). This is the standard canonical representation of cumulative score statistics. It applies when the score process has the stated normal increments and the information schedule is treated as fixed. The earlier material on the Brownian motion representation gives the same covariance structure in information time.

For \(j<k\), the raw increments use disjoint information intervals. Their covariance is zero:

\begin{aligned} \operatorname{Cov}(\Delta U_j,\Delta U_k) &=\operatorname{Cov}(U_j-U_{j-1},U_k-U_{k-1})\\ &=\min(I_j,I_k)-\min(I_j,I_{k-1})\\ &\quad-\min(I_{j-1},I_k)+\min(I_{j-1},I_{k-1})\\ &=I_j-I_j-I_{j-1}+I_{j-1}=0. \end{aligned}

The simplification uses \(j<k\), so \(I_j\leq I_{k-1}\), and \(I_{j-1}\leq I_j\). Each raw increment has mean \(\theta\Delta I_k\) and variance \(\Delta I_k\). Because the increments are jointly normal, zero covariance implies independence. Standardizing each one therefore gives the result below.

Theorem: Under the canonical joint model, the three standardized increments are independent and \(W_k\sim N(\theta\sqrt{\Delta I_k},1)\) for \(k=1,2,3\). In particular, under \(H_0:\theta=0\), each \(W_k\sim N(0,1)\). The cumulative statistic is recovered by \(U_k=\sum_{r=1}^{k}\sqrt{\Delta I_r}\,W_r\), and its standardized form is \(Z_k=U_k/\sqrt{I_k}\).

The theorem concerns increments of \(U_k\), not differences of the standardized cumulative statistics \(Z_k\). The factor \(\sqrt{\Delta I_k}\) is essential when converting between the standardized increment and its raw score contribution.

Worked Three-Look Conversion

Worked Example: Recovering Increments From Three Cumulative Statistics

Suppose a hypothetical normal-mean analysis has known standard deviation \(\sigma=1\) and looks after \(n=2,5,8\) observations. For this model, cumulative information is \(I_k=n_k/\sigma^2\), so \(I_1=2\), \(I_2=5\), and \(I_3=8\). Thus the information increases are \(2,3,3\). Suppose the observed cumulative standardized statistics are \(Z_1=0.4\), \(Z_2=0.7\), and \(Z_3=0.5\). These hypothetical values are consistent with normal observations: the cumulative score at each look is the cumulative sum because \(\sigma=1\).

First recover the cumulative raw scores using \(U_k=Z_k\sqrt{I_k}\):

U_1=0.4\sqrt{2}=0.5657,\qquad U_2=0.7\sqrt{5}=1.5652,\qquad U_3=0.5\sqrt{8}=1.4142.

Now subtract adjacent raw scores and divide by the square root of the corresponding information increase:

\begin{aligned} W_1&=\frac{U_1-U_0}{\sqrt{2}} =\frac{0.565685\ldots}{\sqrt{2}}=0.4000,\\ W_2&=\frac{U_2-U_1}{\sqrt{3}} =\frac{1.565247\ldots-0.565685\ldots}{\sqrt{3}} =\frac{0.999562\ldots}{1.732051\ldots}=0.5771,\\ W_3&=\frac{U_3-U_2}{\sqrt{3}} =\frac{1.414214\ldots-1.565248\ldots}{\sqrt{3}} =\frac{-0.151034\ldots}{1.732051\ldots}=-0.08720. \end{aligned}

The raw second increment is about \(0.9996\), not the standardized increment: its standardized value is about \(0.5771\). Reconstructing the cumulative scores checks the calculation: \(U_1=\sqrt{2}(0.4000)\approx0.5657\), \(U_2=U_1+\sqrt{3}(0.5771)\approx1.5652\), and \(U_3=U_2+\sqrt{3}(-0.08720)\approx1.4142\), subject to rounding.

The information increments are positive, as required, and the known-variance normal-mean model supplies the canonical normal score process. The theorem describes the sampling distribution of the increments under that model; it does not claim that these three observed values are themselves independent in every possible data-generating process.

Checking the Information Schedule

The information schedule determines both the variance of each raw increment and the scale used to standardize it. It is not enough to verify only that the cumulative information increases. For each interval, the model requires \(\operatorname{Var}(\Delta U_k)=\Delta I_k\), and dividing by \(\sqrt{\Delta I_k}\) must produce variance one.

Worked Example: Verifying Means and Variances Under a Fixed Drift

Use the schedule \(I=(2,5,8)\) and suppose the hypothetical fixed alternative has \(\theta=0.2\). The information increases are \(2,3,3\). For the first raw increment, the model gives mean \(0.2(2)=0.4\) and variance \(2\); for the second and third, it gives mean \(0.2(3)=0.6\) and variance \(3\). Standardization gives:

\begin{aligned} W_1&\sim N(0.2\sqrt{2},1)=N(0.2828,1),\\ W_2&\sim N(0.2\sqrt{3},1)=N(0.3464,1),\\ W_3&\sim N(0.2\sqrt{3},1)=N(0.3464,1). \end{aligned}

The cumulative means also agree with the schedule: \(E(U_1)=0.4\), \(E(U_2)=0.4+0.6=1.0\), and \(E(U_3)=1.0+0.6=1.6\), which equal \(\theta I_1,\theta I_2,\theta I_3\). This checks the drift as well as the increment variances. The independent standardized increments need not have equal means under an alternative when the information increases differ.

Reconstructing Cumulative Statistics From Increments

The conversion works in both directions. Given the standardized increments and the information increases, multiply each increment by \(\sqrt{\Delta I_k}\) to recover its raw score contribution, add contributions up to the current look, and divide by \(\sqrt{I_k}\) if a cumulative standardized statistic is wanted.

Worked Example: Building Three Cumulative Statistics From Increments

Consider a hypothetical schedule \(I_1=1\), \(I_2=3\), \(I_3=6\), giving increases \(1,2,3\). Suppose the observed standardized increments are \(W_1=0.6\), \(W_2=-0.4\), and \(W_3=1.1\). Their raw score contributions are:

\Delta U_1=0.6\sqrt{1}=0.6000,\quad \Delta U_2=-0.4\sqrt{2}=-0.5657,\quad \Delta U_3=1.1\sqrt{3}=1.9053.

Adding them successively gives \(U_1=0.6000\), \(U_2=0.6000-0.5657=0.03431\), and \(U_3=0.03431+1.9053=1.9396\). Therefore the cumulative standardized statistics are:

Z_1=\frac{0.6000}{\sqrt{1}}=0.6000,\qquad Z_2=\frac{0.03431}{\sqrt{3}}=0.01981,\qquad Z_3=\frac{1.9396}{\sqrt{6}}=0.7918.

The small value of \(Z_2\) results from partial cancellation between the first two raw increments. It does not contradict the fact that \(W_1\) and \(W_2\) are independent under the canonical model; independence describes their joint sampling distribution, not whether particular realized values add or cancel.

Diagnostics, Alternatives, and Limitations

A reliable calculation has three checks. First, confirm that \(I_1<I_2<I_3\), so every information increase is positive. Second, check the raw variances against the increases: \(\operatorname{Var}(\Delta U_k)=\Delta I_k\), hence \(\operatorname{Var}(W_k)=1\). Third, reconstruct the cumulative scores from the increments and verify that they reproduce the starting values, allowing for rounding. A failure in any check often signals that a cumulative score was confused with a standardized statistic, that an incorrect information difference was used, or that rounding was applied too early.

One alternative is to work directly with the joint covariance matrix of the cumulative statistics and verify the canonical covariance pattern. The increment method is usually more transparent for three looks: it identifies which data contribution belongs to each interval and makes the variance calculation explicit. Direct covariance work remains a useful check, especially if the observed statistics or the information schedule are supplied in a different form.

The result depends on the canonical model, including its normality and covariance assumptions. If increments are not jointly normal, zero covariance alone does not establish independence. If information levels are not fixed as assumed, or the score process has a different covariance structure, the standardized increments may not have unit variance or be independent. These are reasons to check the model rather than treating the conversion as an automatic property of any three statistics.

Mistake: Subtract the standardized cumulative statistics and call the differences independent increments. Correction: Recover \(U_k=Z_k\sqrt{I_k}\), subtract the raw cumulative scores, then divide by \(\sqrt{\Delta I_k}\).
Mistake: Treat the raw increment \(\Delta U_k\) as having variance one. Correction: Its variance is \(\Delta I_k\); it is \(W_k=\Delta U_k/\sqrt{\Delta I_k}\) that has variance one.
Mistake: Infer independence from zero covariance without checking the distributional assumption. Correction: Here independence follows because the increments are jointly normal and uncorrelated under the canonical model.

Key takeaway: At each look, convert the cumulative standardized statistic to a raw score using \(U_k=Z_k\sqrt{I_k}\). Then subtract adjacent raw scores and scale by the square root of the information increase. Under the canonical joint normal model, the resulting increments are independent with unit variance.

Check Your Understanding

Answer each question, then check the brief solutions below.

If \(I=(2,6,9)\), what are the three information increases, and what is the variance of the second raw increment under the canonical model?
For \(I=(1,4,9)\), suppose \(Z_1=0.5\), \(Z_2=0.25\), and \(Z_3=0.4\). Find the raw cumulative scores and the three standardized increments.
Under a fixed alternative \(\theta=0.1\) with information increase \(\Delta I=4\), state the mean and variance of the standardized increment.
Why does zero covariance establish independence for the increments here, and what assumption is needed?
What is wrong with standardizing a raw increment by \(\sqrt{I_k}\) rather than \(\sqrt{\Delta I_k}\)?

Solutions. (1) The increases are \(2,4,3\); the second raw increment has variance \(4\). (2) The raw scores are \(U_1=0.5\), \(U_2=0.25(2)=0.5\), and \(U_3=0.4(3)=1.2\). Thus \(W_1=0.5\), \(W_2=(0.5-0.5)/\sqrt{3}=0\), and \(W_3=(1.2-0.5)/\sqrt{5}=0.3130\). (3) The distribution is \(N(0.1\sqrt{4},1)=N(0.2,1)\). (4) The increments are jointly normal, so zero covariance implies independence; without joint normality that implication need not hold. (5) A raw increment’s variance is the information increase, not the cumulative information. Dividing by \(\sqrt{\Delta I_k}\) gives unit variance under the model.