Why Score Increments Are Asymptotically Independent

Why Score Increments Become Independent

At successive analyses, the data collected since the previous analysis contribute a new score increment. In a regular model, that increment has mean zero under the null and variance described by the information accumulated in that block. When blocks contain enough information, standardized increments are approximately normal. If the blocks are independent, their joint approximation is a collection of independent normal variables.

The distinction between independence and asymptotic normality matters. With independent observations assigned to fixed, nonoverlapping blocks, the block scores are already independent at finite sample sizes. What is usually asymptotic is their normal approximation. In sequential settings where increments need not be exactly independent, a martingale central limit argument can give asymptotically independent Gaussian increments under suitable conditions.

The usual regularity conditions ensure that each score has mean zero and that its variance agrees with the expected negative second derivative of the log likelihood. For independent observations, disjoint blocks then have independent score increments. The central limit theorem supplies the approximate normal distribution when each block's information grows and no individual observation dominates that block.

For a fixed number of blocks, a useful version of the conditions is that the observations are independent, the expected information in each block tends to infinity, and a Lindeberg condition holds within each block. In words, the condition rules out a material share of the block's score variation coming from rare, exceptionally large contributions. The model must also be identifiable and differentiable near \(\theta_0\), with finite, positive information. These conditions are not automatic merely because a score can be calculated.

Derivation for Independent Blocks

Under \(H_0\), regularity gives \(E_{\theta_0}\{s_i(\theta_0)\}=0\). If observations are independent, scores from different observations are independent. Consequently, score increments formed from disjoint blocks are independent, and $$ E_{\theta_0}(\Delta U_k)=0, \qquad \operatorname{Var}_{\theta_0}(\Delta U_k)=\Delta I_k. $$ The central limit theorem, applied separately to each block, gives \(Z_k\) an approximate standard normal distribution. Independence of the blocks makes the joint characteristic function factor into the product of the block characteristic functions. Since each factor converges to the characteristic function of a standard normal variable, the joint limit is a vector of independent standard normal variables.

The right-hand side is the joint characteristic function of independent \(N(0,1)\) variables. Thus, for a fixed number \(K\) of blocks, the standardized score increments converge jointly to independent standard normal variables. This argument uses more than zero covariance: it uses independence of the blocks at finite sample sizes, followed by a normal limit for each block.

There is another route when observations arrive sequentially and score increments are martingale differences rather than independent variables. A martingale difference has conditional mean zero given the past. If its accumulated conditional variance, after standardization, converges to a nonrandom value, and a martingale Lindeberg condition prevents large contributions, a martingale central limit theorem yields a Gaussian limit. For several consecutive blocks, the same argument applied jointly gives a Gaussian vector whose components have zero cross-covariances when the limiting information increments are nonoverlapping. In a Gaussian vector, zero cross-covariance implies independence. This is asymptotic independence; the original increments need not be independent.

Heterogeneous Information and Sequential Scores

Equal block sizes are not required. If observations are independent but have different information contributions, the variance of a block score is the sum of those contributions. Standardizing by the block's own \(\Delta I_k\), rather than by its number of observations, is what puts increments on a common scale. The large-block requirement concerns information and the distribution of score contributions, not just a raw count.

The same reasoning applies to an efficient score when a regular nuisance parameter is present. For a scalar target parameter \(\psi\) and scalar nuisance parameter \(\lambda\), let the information matrix for one observation at the null be $$ I= \begin{pmatrix} I_{\psi\psi} & I_{\psi\lambda}\\ I_{\lambda\psi} & I_{\lambda\lambda} \end{pmatrix}. $$ If \(I_{\lambda\lambda}>0\), the efficient score contribution for \(\psi\) is \(s_{\psi,i}-(I_{\psi\lambda}/I_{\lambda\lambda})s_{\lambda,i}\). Its variance is the efficient information \(I_{\psi\psi}-I_{\psi\lambda}^2/I_{\lambda\lambda}\). Using that score and its information in each block preserves the block argument when the nuisance parameter is handled under the same regular model.

Diagnostics, Alternatives, and Limits

The independent-increment approximation is most credible when the data are collected in fixed, nonoverlapping blocks from an independent-observation model, or when a suitable martingale formulation applies. Check that the score is evaluated under the stated null model, the information is positive and correctly calculated, every block has enough information for its normal approximation, and no single observation or cluster dominates. If the data are dependent, or if a block boundary is selected using earlier outcomes, ordinary independence of block scores cannot simply be assumed. A sequential model and its conditional information structure must justify the alternative argument.

A common mistake is to argue, “The score increments have zero covariance, so they are independent.” Zero covariance alone does not establish independence for general, nonnormal variables. The correction is to justify independence from independent data blocks, or to establish a joint Gaussian limit and then use the fact that uncorrelated components of a Gaussian vector are independent. A related mistake is to call the increments asymptotically independent without checking that each block has enough information for the required limit.

Two useful approaches answer related but different questions. A direct covariance calculation identifies covariance under a specified model, but zero covariance alone may not settle independence. The score-process approach explains why a joint Gaussian approximation can arise and when its increments are independent. Alternatively, a canonical joint normal model specifies the joint distribution directly; the score argument provides a regular-model justification for that kind of approximation rather than replacing its assumptions with a universal guarantee.

The conclusion is limited to the model and asymptotic regime used in the argument. Small or sparse blocks, heavy-tailed score contributions, non-identifiability, boundary parameter values, or misspecified information can invalidate the normal approximation. Shared nuisance-parameter estimates or data-dependent changes to the analysis can also complicate the block structure. In these cases, the issue is not fixed merely by dividing each score by the square root of an information estimate.