Score Statistics in Regular Models

Why Scores Lead to a Canonical Model

A score statistic measures how rapidly the log likelihood would change if a parameter moved away from its null value. Fisher information scales that change by the amount of information in the data. In a regular model, this standardized score is approximately normal. At several nested analyses, the cumulative scores share observations, so their covariance is determined by the information accumulated at each look. Together these facts yield the canonical joint normal approximation used in group sequential methods.

The approximation is not automatic merely because a statistic is called a score. It relies on a correctly specified, sufficiently smooth model, finite information, and enough independent information for a central limit approximation. The derivation below uses a scalar parameter first, then gives the standard adjustment when nuisance parameters are present.

Regularity Conditions and the One-Look Result

For the scalar derivation, suppose observations are independent and the model is correctly specified. Assume the parameter is an interior point of its parameter space; the density is differentiable in \(\theta\); differentiation can be interchanged with integration; the score has finite, positive variance; and no small collection of observations dominates the total information. These conditions support the score identities and a central limit theorem. For a sequence of looks, assume each look adds observations and that cumulative information grows. The score and information are evaluated under the null unless stated otherwise.

Under these regularity conditions, the score has mean zero at the true parameter and variance equal to Fisher information. The standardized score therefore has an approximately standard normal distribution under \(H_0\), provided information is sufficiently large. A two-sided score test compares \(|Z|\) with the standard normal critical value; a one-sided test uses the direction specified by \(H_a\).

The information identity follows directly from the density \(f(x;\theta)\). Since \(\int f(x;\theta)\,dx=1\), differentiating once gives \(E_\theta[U(\theta)]=0\). Differentiating twice gives \(E_\theta[\partial^2\ell(\theta)/\partial\theta^2]+E_\theta[U(\theta)^2]=0\). Thus expected negative curvature and score variance agree when the differentiations are justified. For independent observations, log likelihoods add, so both score and information add across observations.

From Cumulative Scores to the Joint Normal Form

Let \(U_k\) be the cumulative score at look \(k\), evaluated at \(\theta_0\), and let \(I_k\) be the corresponding cumulative Fisher information under \(H_0\), with \(I_1\leq \cdots \leq I_K\). Define \(Z_k=U_k/\sqrt{I_k}\). Under the null, each component is approximately standard normal. For looks \(j\leq k\), the first \(j\) observations are contained in the data at look \(k\). Independence of observations and the zero-mean score imply that the covariance of the cumulative scores is the information in their shared observations: \(\operatorname{Cov}_0(U_j,U_k)\approx I_j\). Consequently,

Thus the vector of standardized scores is approximately multivariate normal with mean zero, unit marginal variances, and the canonical correlation matrix whose \((j,k)\) entry is \(\sqrt{I_j/I_k}\) for \(j\leq k\). The approximation depends on the cumulative information, not on equal spacing of calendar times or sample sizes. The covariance calculation explains the joint distribution; it does not say that the cumulative \(Z\) statistics themselves are independent.

Local Alternatives and Canonical Drift

The same score construction describes local alternatives, which approach the null as total information increases. Write the alternative as \(\theta=\theta_0+h/\sqrt{I_K}\), where \(h\) is a fixed, dimensionless shift and \(I_K\) is the information at the final look under the null. A first-order expansion of the expected null score gives \(E_\theta[U_k(\theta_0)]\approx I_k(\theta-\theta_0)\). Its variance is approximately \(I_k\) for these local shifts. Hence the standardized score has approximate mean \(h\sqrt{I_k/I_K}\) at look \(k\), while its covariance structure remains approximately the canonical one.

The notation \(\dot{\sim}\) indicates an asymptotic approximation. This drift is the local-alternative form of the canonical model. If a fixed alternative is not close to the null, the score’s mean and variance under that alternative need not follow this first-order expression. One should not treat \(h\) as a fixed parameter displacement: the displacement is \(h/\sqrt{I_K}\), which shrinks as information increases.

For justification, differentiate the expected null score with respect to the true parameter at \(\theta_0\). Under the regularity conditions, \(\left.\partial E_\theta[U_k(\theta_0)]/\partial\theta\right|_{\theta=\theta_0} =E_{\theta_0}[U_k(\theta_0)^2]=I_k\). A first-order Taylor expansion therefore gives \(E_\theta[U_k(\theta_0)]\approx I_k(\theta-\theta_0)\). Substituting the local displacement and dividing by \(\sqrt{I_k}\) yields the displayed drift. Higher-order terms vanish in the usual regular large-information limit.

Nuisance Parameters and Efficient Information

When the parameter of interest \(\theta\) is accompanied by nuisance parameters \(\gamma\), the raw score for \(\theta\) can include variation that is also explained by changes in \(\gamma\). Let the Fisher information matrix have blocks \(I_{\theta\theta}\), \(I_{\theta\gamma}\), \(I_{\gamma\theta}\), and \(I_{\gamma\gamma}\). Under regularity and nonsingular nuisance information, the score component adjusted for nuisance directions and its information are

The standardized efficient score is \(U_{\theta\cdot\gamma}/\sqrt{I_{\theta\theta\cdot\gamma}}\). It removes, to first order, the part of the target score aligned with nuisance scores. In sequential use, the efficient score and its information must be defined consistently at each look; a changing nuisance parameterization or unstable nuisance estimation can spoil the simple approximation.

How the Score Approach Compares and Where It Can Fail

A score test evaluates the likelihood’s local slope at the null and uses information at that null. A Wald test instead scales an estimated parameter’s distance from the null, while a likelihood-ratio test compares maximized likelihoods. Under regular large-sample conditions these approaches often agree to first order, but their finite-sample results can differ. The score form can be useful when fitting the unrestricted model is difficult, although sequential score analyses still require valid information estimates and consistent nuisance adjustment.

The canonical approximation is most vulnerable when its regularity assumptions fail. Boundary parameters, nonidentifiability, sparse data, strong dependence, model misspecification, or a small number of highly influential observations can make the score nonnormal or the information calculation misleading. Diagnostics should therefore include checking whether the likelihood is smooth near the null, whether fitted information is positive and numerically stable, whether the model’s independence and variance assumptions are plausible, and whether expected counts or effective information are too small for a normal approximation. If these checks fail, a more appropriate finite-sample method or a carefully justified simulation may be needed.

• Wrong reasoning: “Each \(Z_k\) is approximately standard normal, so the \(Z_k\) values at different looks are independent.”
• Correction: Marginal normality does not imply independence. Nested scores reuse observations; their canonical null correlation is approximately \(\sqrt{I_j/I_k}\) for \(j\leq k\).
• Wrong reasoning: “The canonical local drift is valid for any fixed alternative.”
• Correction: The displayed drift is derived for local alternatives whose distance from the null shrinks at rate \(1/\sqrt{I_K}\). A fixed alternative may require a different mean and variance calculation.

The central practical lesson is to specify the model and its information before standardizing a score. In regular settings, score variance and Fisher information put cumulative statistics on a common scale; shared information then determines the joint covariance, and local alternatives determine the canonical drift. These components make the approximation interpretable, while also making its assumptions visible.