Leverage-Based Lindeberg Conditions

Why Leverage Enters a Score Limit

In random-design regression, the score that drives the least-squares coefficient error is a sum of terms \(X_i\varepsilon_i\). A central limit approximation for that sum requires more than a stable average covariance: no small set of unusually influential terms should retain a substantial share of the total variation. The conditional Lindeberg condition makes this requirement precise after conditioning on the observed regressors.

The key quantity is a variance-weighted version of leverage. It measures how much each observation contributes to the covariance of the score, rather than only how unusual its regressor is. A useful sufficient condition combines vanishing maximum weighted leverage with uniform control of the tails of standardized errors.

Conditional Score Contributions and Leverage

For each sample size \(n\), let \(X_{ni}\in\mathbb{R}^p\) be the regressor vector and let \(\varepsilon_{ni}\) be the regression error, for \(i=1,\ldots,n\), with fixed \(p\). Condition on the complete design \(\mathcal X_n=(X_{n1},\ldots,X_{nn})\). Assume that, given \(\mathcal X_n\), the errors are independent, have conditional means zero, and have finite, strictly positive conditional variances \(\sigma_{ni}^2=E[\varepsilon_{ni}^2\mid\mathcal X_n]\). Define the conditional score covariance per observation by

and assume \(Q_n\) is positive definite. The normalized score and its covariance-standardized individual contributions are

Here \(Q_n^{-1/2}\) is the symmetric inverse square root. Conditional on the design, the \(V_{ni}\) are independent, have mean zero, and their covariance matrices sum to the identity. Their squared lengths in expectation define the variance-weighted leverages.

In the homoskedastic case, \(\sigma_{ni}^2=\sigma^2\), this definition reduces to the ordinary diagonal leverage \(h_{ni}=X_{ni}'(\sum_jX_{nj}X_{nj}')^{-1}X_{ni}\). Thus weighted leverage extends familiar design leverage to account for differences in conditional error variance.

The Conditional Lindeberg Condition

The condition says that the total conditional variance contributed by score terms larger than any fixed threshold tends to zero. Convergence in probability is appropriate when the design, and therefore the conditional quantities, are random. In this setup, write \(\eta_{ni}=\varepsilon_{ni}/\sigma_{ni}\), so \(E[\eta_{ni}^2\mid\mathcal X_n]=1\). Directly from the definitions,

Consequently, the Lindeberg sum is exactly

This expression shows why small leverage alone is not the whole argument: an observation with small weighted leverage can still contribute to the Lindeberg sum if its standardized error has sufficiently heavy tails.

A Leverage and Tail Bound

Proof. Fix \(\delta>0\), and let \(m_n=\max_i\lambda_{ni}\). On the event \(m_n\leq a\), where \(a>0\), the indicator \(\{\sqrt{\lambda_{ni}}|\eta_{ni}|>\delta\}\) implies \(|\eta_{ni}|>\delta/\sqrt a\). Therefore,

Given any positive tolerance, choose \(a\) sufficiently small that the last bound is below that tolerance; this is possible because \(\rho(t)\to0\). Since \(m_n\xrightarrow{p}0\), the probability that \(m_n>a\) tends to zero. Thus the Lindeberg sum tends to zero in probability. This proves the result. \(\square\)

A standard conditional Lindeberg–Feller theorem now gives a conditional normal limit for the whitened score: under the stated conditional independence and mean-zero assumptions, the conditional distribution of \(\sum_i V_{ni}=Q_n^{-1/2}S_n\) approaches \(N(0,I_p)\) in probability. If, in addition, \(Q_n\xrightarrow{p}Q\) for a deterministic positive-definite matrix \(Q\), the corresponding unwhitened score has limiting distribution \(N(0,Q)\). The theorem above supplies a practical way to check the Lindeberg hypothesis; it does not replace the covariance convergence needed for the latter limit.

Worked Examples

Using the Condition in Regression Limits

In least squares with a correctly specified conditional mean, the coefficient error has a score representation involving \(S_n=n^{-1/2}\sum_iX_{ni}\varepsilon_{ni}\) and the empirical Gram matrix. A conditional normal approximation for the score is one ingredient in an asymptotic normality argument; a stable, nonsingular Gram matrix is another. Leverage-based Lindeberg checks address whether individual score terms are negligible, not whether the design is identified or the covariance matrix converges.

A useful proof strategy is to compute the covariance-weighted leverage, establish that its maximum vanishes, and then verify a uniform tail bound for the standardized errors. Under homoskedasticity, ordinary maximum hat-matrix leverage is the relevant leverage measure. Under heteroskedasticity, the variance weights matter: raw design leverage can be small while one observation still dominates the score covariance.

Common Mistake

A tempting but incomplete argument is: “Every ordinary hat-matrix leverage tends to zero, so the score satisfies Lindeberg.” This overlooks two separate requirements. First, with heteroskedastic errors, the relevant quantity is variance-weighted leverage. Second, even small weighted leverage needs suitable control of the standardized error tails.

For example, take \(X_{ni}=1\) for all \(i\), let \(\sigma_{n1}^2=n\), and let \(\sigma_{ni}^2=1\) for \(i\geq2\). Ordinary leverage is \(1/n\) for every observation, but the weighted leverage of the first observation is

If the standardized errors are Rademacher, the same calculation as in the preceding example shows that Lindeberg fails. The correction is to assess leverage using the conditional score covariance and then check the standardized error tails; neither check can be silently omitted.