Unequal Information Increments

Why Unequal Information Gains Matter

Sequential looks need not add the same amount of statistical information. A look after a modest increase in information has a different conditional relationship with the preceding look than one after a much larger increase. The relevant quantity is not simply how many participants or calendar days separate the looks, but how much information the new data contribute.

The earlier material on independent increments distinguishes the successive score contributions from the cumulative standardized statistics. Here the information gains can vary from one interval to another. The key calculation is that a score increment has variance equal to its information gain, while the conditional variance of the later cumulative Z statistic is that gain divided by the later cumulative information.

Definition: Let \(I_1<I_2<\cdots<I_K\) be the cumulative information levels at successive looks, and let \(S_k\) be the cumulative score statistic at look \(k\). The information increment between looks \(k-1\) and \(k\) is \(\Delta I_k=I_k-I_{k-1}\). Define \(I_0=0\), so the first increment is \(\Delta I_1=I_1\). The cumulative standardized statistic is \(Z_k=S_k/\sqrt{I_k}\).

The canonical Gaussian model assumes that the score increment \(S_k-S_{k-1}\) is independent of the scores from earlier increments and has distribution \(N(\theta\Delta I_k,\Delta I_k)\). Here \(\theta\) is the fixed canonical drift: under \(H_0\), \(\theta=0\); under the specified fixed alternative, \(\theta\) is constant across looks. Thus, the model treats information as additive, and the increment variance is positive when \(\Delta I_k>0\). In regular models, this is typically a normal or asymptotic normal approximation, not a claim that every finite-sample score is exactly Gaussian.

Deriving the Conditional Transition

For two looks \(i<j\), write \(\Delta I_{i,j}=I_j-I_i\). By the independent-increment model, the new score contribution \(S_j-S_i\) is independent of \(S_i\) and has mean \(\theta\Delta I_{i,j}\) and variance \(\Delta I_{i,j}\). Because \(S_i=Z_i\sqrt{I_i}\), the later standardized statistic can be written as

Z_j = \sqrt{\frac{I_i}{I_j}}Z_i + \frac{S_j-S_i}{\sqrt{I_j}}.

Conditional on \(Z_i=z_i\), only the new increment remains random. Dividing its mean by \(\sqrt{I_j}\) and its variance by \(I_j\) gives the transition law:

Z_j\mid Z_i=z_i \sim N\left( \sqrt{\frac{I_i}{I_j}}z_i + \theta\frac{I_j-I_i}{\sqrt{I_j}}, \frac{I_j-I_i}{I_j} \right).

Under \(H_0\), set \(\theta=0\). The conditional mean is then the earlier statistic multiplied by \(\sqrt{I_i/I_j}\); the conditional variance is the fraction of the later information supplied by the new increment. A larger increment, relative to \(I_j\), gives a larger conditional variance because more of \(Z_j\) comes from data not present at look \(i\).

The same model also explains why the \(Z\) statistics are not independent. Since \(S_i\) is part of \(S_j\), their covariance is

\operatorname{Cov}(Z_i,Z_j) = \frac{\operatorname{Cov}(S_i,S_j)}{\sqrt{I_iI_j}} = \frac{I_i}{\sqrt{I_iI_j}} = \sqrt{\frac{I_i}{I_j}}, \qquad i<j.

This covariance does not depend on \(\theta\). Unequal information gains change the correlation through the cumulative information levels, but they do not make the cumulative statistics independent. The independence assumption applies to the score increments, not to the \(Z\) statistics.

Worked Example: Unequal Gains Across Three Looks

Suppose the cumulative information levels are \(I_1=80\), \(I_2=200\), and \(I_3=500\). The successive gains are \(80\), \(120\), and \(300\) information units, so the later intervals are not equal in information. Under \(H_0\), calculate the conditional distribution at look 2 given \(Z_1=1.20\).

The null transition has conditional mean \(\sqrt{I_1/I_2}Z_1\) and variance \((I_2-I_1)/I_2\). Substituting the values,

E(Z_2\mid Z_1=1.20,H_0) = \sqrt{\frac{80}{200}}(1.20) = \sqrt{0.4}(1.20) \approx 0.7589,

\operatorname{Var}(Z_2\mid Z_1,H_0) = \frac{200-80}{200} = 0.6000.

Therefore \(Z_2\mid Z_1=1.20,H_0\sim N(0.7589,0.6000)\), where the second normal parameter is the variance. Its conditional standard deviation is \(\sqrt{0.6000}\approx0.7746\).

At the next transition, if \(Z_2=0.50\), the information gain is \(500-200=300\). The null conditional mean is \(\sqrt{200/500}(0.50)\approx0.3162\), and the conditional variance is \(300/500=0.6000\). Thus \(Z_3\mid Z_2=0.50,H_0\sim N(0.3162,0.6000)\). Although the absolute gain grew from 120 to 300 units, its fraction of the later cumulative information is the same at both transitions, so the two conditional variances are equal.

Interpreting Score Increments and Conditional Residuals

For a given transition, the new score contribution is \(S_j-S_i\), not \(Z_j-Z_i\). Under \(H_0\), its mean is zero and its variance is \(I_j-I_i\). Its standardized form, \((S_j-S_i)/\sqrt{I_j-I_i}\), is standard normal under the canonical model and independent of earlier score increments. This provides a direct check on calculations made using the conditional distribution of \(Z_j\).

In particular, the standardized conditional residual can be calculated from the two observed Z statistics as

\frac{ Z_j-\sqrt{I_i/I_j}Z_i-\theta(I_j-I_i)/\sqrt{I_j} }{ \sqrt{(I_j-I_i)/I_j} }.

Under the model, this residual is standard normal and independent of the earlier increments. Under \(H_0\), omit the drift term involving \(\theta\). This is an algebraic diagnostic for whether a reported transition calculation has used the appropriate information gain and conditional variance; a single residual is not, by itself, a comprehensive model check.

Worked Example: Recovering the New Score Contribution

Suppose \(I_1=100\), \(I_2=300\), \(Z_1=1.50\), and \(Z_2=1.10\). Work under \(H_0\). First recover the cumulative scores using \(S_k=Z_k\sqrt{I_k}\):

S_1=1.50\sqrt{100}=15.00, \qquad S_2=1.10\sqrt{300}\approx19.05.

The new score contribution is \(S_2-S_1\approx4.05\), and the information gain is \(300-100=200\). Its null standard deviation is \(\sqrt{200}\approx14.14\), so the standardized increment is

\frac{S_2-S_1}{\sqrt{200}} \approx \frac{4.05}{14.14} \approx 0.2866.

Now check this using the conditional transition. Given \(Z_1=1.50\), the null conditional mean for \(Z_2\) is \(\sqrt{100/300}(1.50)\approx0.8660\), and its conditional standard deviation is \(\sqrt{200/300}\approx0.8165\). Therefore,

\frac{1.10-0.8660}{0.8165}\approx0.2866.

The two calculations agree. The observed later statistic is above its conditional null mean, but the standardized residual is only about 0.29 standard deviations above that mean.

Worked Example: A Conditional Upper-Tail Probability

Suppose \(I_1=100\), \(I_2=250\), and the observed first-look value is \(Z_1=0.80\). Under a fixed alternative with \(\theta=0.10\), calculate the conditional probability that \(Z_2\) exceeds 1.96. The information gain is \(150\). The conditional mean and variance are

\mu_{2\mid1} = \sqrt{\frac{100}{250}}(0.80) + 0.10\frac{150}{\sqrt{250}} \approx0.5060+0.9487 = 1.4547,

\sigma^2_{2\mid1} = \frac{250-100}{250} = 0.6000, \qquad \sigma_{2\mid1}=\sqrt{0.6000}\approx0.7746.

Standardizing the threshold gives \((1.96-1.4547)/0.7746\approx0.6525\). Hence the requested conditional probability is \(1-\Phi(0.6525)\approx0.2571\), where \(\Phi\) is the standard normal cumulative distribution function. In context, under this fixed-drift model and given \(Z_1=0.80\), the probability that the second-look statistic exceeds 1.96 is about 25.7%. This is a conditional probability under the specified model, not an unconditional probability of crossing a boundary.

Assumptions, Diagnostics, and Alternatives

These transition formulas require \(0<I_i<I_j\), so the added information is positive. They also require the canonical increment model: information adds across intervals, score increments have the stated means and variances, and increments are independent (exactly in the stipulated Gaussian model or approximately under the relevant asymptotic conditions). The information levels must correspond to the score statistics being analyzed. A fixed alternative must have the same canonical drift \(\theta\) across looks for the stated fixed-drift transition.

Before using a transition, check the following:

Information increases: verify \(I_j-I_i>0\). A zero or negative gain makes the stated transition variance zero or invalid.
Scale consistency: use the same definition of cumulative information and score at both looks. The formula \(Z_k=S_k/\sqrt{I_k}\) must apply to each look.
Variance check: confirm that the conditional variance \((I_j-I_i)/I_j\) lies strictly between 0 and 1 when \(0<I_i<I_j\).
Increment check: when both statistics are available, compute the standardized score increment and the conditional residual. They must agree algebraically.
Model check: substantial departures from the assumed score behavior, dependence across new data contributions, or a poor normal approximation can undermine the transition, even if the arithmetic is correct.

A common mistake is to treat unequal increments as though each later statistic were an independent new Z statistic, or to use the same conditional variance at every transition. Both are wrong. The later cumulative statistic retains the earlier score, so it is correlated with the earlier Z statistic; its conditional variance is specifically the new information divided by the later cumulative information. Another frequent error is to use \(Z_j-Z_i\) as the standardized increment. The correct score increment is \(S_j-S_i=Z_j\sqrt{I_j}-Z_i\sqrt{I_i}\), and it must be divided by \(\sqrt{I_j-I_i}\) to standardize it under \(H_0\).

Compared with working directly with cumulative Z statistics, working with score increments makes the independence structure visible and is often simpler for verifying variances. The conditional-transition representation is more convenient when the question concerns the distribution of a later statistic given an earlier one. Equal information gains are a special case of the same formulas; unequal gains require no new probability principle, but each transition must use its own \(I_j-I_i\). These methods remain limited by the validity of the canonical model and do not repair misspecified information levels or invalid independence assumptions.

Key takeaway: For a transition from \(I_i\) to \(I_j\), the new score increment has variance \(I_j-I_i\). Consequently, the conditional variance of the later Z statistic is \((I_j-I_i)/I_j\), and its conditional mean combines the carried-forward earlier statistic with the drift contributed by the new information.

Check Your Understanding

Try each question, then compare your reasoning with the solutions below.

At two looks, \(I_1=50\) and \(I_2=200\). Under \(H_0\), find the conditional mean and variance of \(Z_2\) given \(Z_1=0.60\).
For \(I_i=120\) and \(I_j=300\), what is the variance of the new score increment? What is the conditional variance of \(Z_j\) given \(Z_i\)?
Explain why independent score increments do not imply independent cumulative Z statistics.
Under \(H_0\), an analyst calculates the conditional variance as \(I_i/I_j\). Identify the error and give the correct variance.

Solutions. (1) The conditional mean is \(\sqrt{50/200}(0.60)=0.3000\), and the conditional variance is \((200-50)/200=0.7500\). Thus \(Z_2\mid Z_1=0.60,H_0\sim N(0.3000,0.7500)\). (2) The score increment variance is \(300-120=180\); the conditional Z variance is \(180/300=0.6000\). (3) The later cumulative score contains the earlier score, so \(\operatorname{Cov}(Z_i,Z_j)=\sqrt{I_i/I_j}>0\) when \(i<j\). Independence belongs to the nonoverlapping score increments, not to the cumulative statistics. (4) \(I_i/I_j\) is the fraction of later information already available at the earlier look; the conditional variance comes from the new information, so it is \((I_j-I_i)/I_j=1-I_i/I_j\).

What You'll Learn