Information-Standardized Test Statistics

Why Standardize a Cumulative Score?

At an interim look, a cumulative test statistic summarizes evidence from the observations collected so far. Its raw magnitude, however, depends on the scale of the data and on how much information has accumulated. A score of 4 could be substantial when its standard deviation is 2, but modest when its standard deviation is 20. Dividing a cumulative score by the square root of its null information puts the statistic on a common, dimensionless scale.

This standardization is useful for interpreting evidence at different amounts of information. It does not, by itself, make statistics from different looks independent, establish a normal distribution, or provide a multiple-testing adjustment. Those are separate questions.

Definition: Let $U_k$ be a cumulative score statistic at look $k$, evaluated under the null hypothesis $H_0$. Its null information $I_k$ is the variance of $U_k$ under $H_0$, when that variance is defined and finite. The information-standardized statistic is $$ Z_k=\frac{U_k}{\sqrt{I_k}}, \qquad I_k>0. $$ Under $H_0$, this statistic has variance one. It is exactly standard normal only when the null distribution of $U_k$ is normal; otherwise, a standard normal distribution may be an approximation.

In likelihood-based settings, the score is the derivative of the log likelihood with respect to a parameter, evaluated at its null value. Under standard regularity conditions, its expectation under $H_0$ is zero, and its variance is the Fisher information. At interim looks, the cumulative score uses all observations available by that look, and the corresponding information is calculated for that same accumulated data under $H_0$. The score and its information must refer to the same parameterization and data.

Why the Standardized Statistic Has Unit Variance

The basic justification is a direct variance calculation. Suppose $I_k=\operatorname{Var}_0(U_k)$, where the subscript 0 indicates that the variance is evaluated under $H_0$. The denominator is a fixed positive quantity under the null model. Therefore,

\operatorname{Var}_0(Z_k) =\operatorname{Var}_0\left(\frac{U_k}{\sqrt{I_k}}\right) =\frac{\operatorname{Var}_0(U_k)}{I_k} =\frac{I_k}{I_k} =1.

If also $E_0(U_k)=0$, then $E_0(Z_k)=0$. These two facts give mean zero and variance one, but they do not determine the full distribution. A discrete statistic, for example, remains discrete after scaling and cannot become exactly normal merely because its variance is one.

For a regular likelihood, the score’s expected information is often used in place of its exact null variance. When the model is correctly specified and the usual regularity conditions hold, these agree in standard settings. In large samples, the standardized score is often approximately $N(0,1)$ under $H_0$. The approximation can be poor with little information, sparse outcomes, or substantial model misspecification.

Worked Example: Normal Observations With Known Variance

Suppose, hypothetically, that $n=25$ independent observations come from $N(\mu,\sigma^2)$, with known $\sigma=2$. Test $H_0:\mu=10$. The observed sample mean is $\bar{x}=10.7$. Assume observations are independent and the stated normal model and known variance are correct.

For one observation, the score for $\mu$, evaluated at $\mu_0=10$, is $(x_i-\mu_0)/\sigma^2$. Thus the cumulative score and its null information are

U=\frac{n(\bar{x}-\mu_0)}{\sigma^2} =\frac{25(10.7-10)}{2^2} =\frac{17.5}{4} =4.375, \qquad I=\frac{n}{\sigma^2} =\frac{25}{4} =6.25.

The standardized statistic is

Z=\frac{4.375}{\sqrt{6.25}} =\frac{4.375}{2.5} =1.750.

The same value follows from $Z=\sqrt{n}(\bar{x}-\mu_0)/\sigma$: $5(10.7-10)/2=1.750$. Under the stated null model, this statistic is exactly standard normal. The value $1.750$ is a standardized score; a decision about evidence would still require a specified testing rule.

Examples With Discrete Outcomes

The same calculation applies when the score is not normally distributed. In such cases, standardization still gives unit null variance if the information is the exact null variance, but a normal approximation may need separate justification.

Worked Example: A Binomial Proportion

Imagine 100 independent hypothetical Bernoulli observations, each with success probability $p$. Test $H_0:p=0.20$. Suppose 27 successes are observed. Assume the observations are independent, have a common success probability, and are measured without missingness or dependence.

Under $H_0$, the success count $X$ has mean $np_0=20$ and variance $np_0(1-p_0)=16$. The score for $p$ at $p_0$ is $U=(X-np_0)/[p_0(1-p_0)]$, and its information is $I=n/[p_0(1-p_0)]$. Substitution gives

U=\frac{27-100(0.20)}{0.20(0.80)} =\frac{7}{0.16} =43.75, \qquad I=\frac{100}{0.20(0.80)} =625.

Therefore,

Z=\frac{43.75}{\sqrt{625}} =\frac{43.75}{25} =1.750.

Equivalently, $Z=(X-np_0)/\sqrt{np_0(1-p_0)}=7/4=1.750$. Its null variance is exactly one. Its null distribution is nevertheless discrete, so exact unit variance does not imply an exact standard normal distribution. Here the expected success and failure counts are 20 and 80, respectively; whether a normal approximation is adequate should still be judged for the intended inference.

Worked Example: A Poisson Rate

Suppose, hypothetically, that 40 independent observation units have Poisson counts with common rate $\lambda$ per unit. Test $H_0:\lambda=3$, and suppose the total observed count is 132. Assume the counts are independent, the exposure is one unit for each observation, and a common Poisson rate is appropriate.

For each count $Y_i$, the score for $\lambda$ at $\lambda_0=3$ is $(Y_i-\lambda_0)/\lambda_0$. Summing the scores gives $U$, and the information is $I=n/\lambda_0$. Thus,

U=\frac{132-40(3)}{3} =\frac{12}{3} =4, \qquad I=\frac{40}{3} =13.333\ldots.

The standardized score is

Z=\frac{4}{\sqrt{40/3}} =\frac{4}{3.651} \approx 1.095.

Under the assumed null model, $U$ has mean zero and variance $40/3$, so $Z$ has mean zero and variance one. The count total is discrete, and a normal approximation to $Z$ is not guaranteed by the variance calculation alone. Also, if the counts are overdispersed relative to the Poisson model, the stated information may understate the actual null variance.

Information Must Match the Score

A standardization is only as sound as the variance used in its denominator. The information must be calculated under the null model and for the same cumulative score. For independent observations with individual score contributions $u_i$, each having null mean zero and variance $I_i$, the cumulative score is $U_k=\sum_{i=1}^k u_i$. Independence then gives

\operatorname{Var}_0(U_k) =\sum_{i=1}^k \operatorname{Var}_0(u_i) =\sum_{i=1}^k I_i.

If contributions are dependent, the variance also includes their covariances; simply adding individual information values can then be wrong. Similarly, using the variance of a different statistic, information evaluated at an unrelated parameter value, or information for only part of the accumulated data will generally not produce unit null variance.

In group sequential work, a statistic is computed at more than one look. Each $Z_k$ is standardized using the information accumulated by that look. This places the marginal statistic at each look on a unit-variance scale when the variance calculation is correct. It does not make repeated looks independent, nor does it control the probability of at least one false rejection across looks. The monitoring rule still needs appropriate calibration.

Diagnostics, Alternatives, and Limitations

Before using an information-standardized statistic, check the ingredients rather than relying on its familiar $Z$ label.

Check the null mean and variance. Confirm that the score has null expectation zero and that the stated information is its null variance, exactly or under clearly stated regularity conditions.
Check the data structure. Independence, common distributions, exposure amounts, and other model assumptions must fit the setting. Dependence or overdispersion can invalidate a variance that assumes independent observations.
Check the amount of information. Very small expected counts or sparse outcomes can make a normal approximation unreliable even when the standardized statistic has unit variance.
Check that information is positive. If $I_k=0$, the standardized statistic is undefined; there is no positive null variance by which to scale.
Keep standardization separate from sequential calibration. Unit variance at each look does not account for the chance of crossing a boundary at any of several looks.

A raw score retains the model-specific scale of $U_k$, which can make its magnitude hard to compare across sample sizes or parameterizations. The information-standardized score expresses the score in units of its null standard deviation. A Wald statistic instead uses an estimate and standard error for an estimated parameter, commonly comparing that estimate with its null value. These approaches can agree approximately in regular large samples, but they need not agree in finite samples, and they rely on different calculations. A likelihood-ratio statistic is another approach: it compares likelihoods under the null and fitted alternatives rather than scaling a score by its null variance.

Common pitfall: “The statistic has variance one, so it is standard normal and can be compared with a standard normal cutoff at every look.” The first part does not imply the second: many nonnormal distributions have variance one. And even if the statistic is approximately standard normal at each look, applying a single-look cutoff repeatedly does not automatically control the overall false-positive probability.

Check Your Understanding

Try each question, then compare your reasoning with the brief solutions.

If $E_0(U_k)=0$ and $\operatorname{Var}_0(U_k)=I_k=9$, what are the null mean and variance of $Z_k=U_k/\sqrt{I_k}$? Does this establish that $Z_k$ is standard normal?
For a hypothetical binomial test with $n=50$, $p_0=0.40$, and $X=24$, calculate the standardized score.
Why can a statistic have exactly unit null variance but still fail to have an exact standard normal distribution?
Two interim statistics are each standardized to have unit null variance. Does that imply they are independent or that repeated testing is calibrated? Explain.
Name one model failure that could make a Poisson information calculation inappropriate.

Solutions. (1) The mean is $0/\sqrt{9}=0$, and the variance is $9/9=1$. These moments alone do not establish normality. (2) The null variance of $X$ is $50(0.40)(0.60)=12$. Thus $Z=(24-20)/\sqrt{12}=4/3.464=1.155$. (3) Standardization fixes the mean and variance, not the distribution’s shape; a discrete score remains discrete. (4) No. Unit marginal variances say nothing by themselves about dependence between looks or the calibration of a repeated-testing rule. (5) Overdispersion, dependence among counts, or a nonconstant rate could invalidate the assumed Poisson variance.