Lyapunov Conditions for Regression Asymptotics

Why Higher Moments Enter an OLS Limit

For random-design least squares, the central random quantity is not a regressor or an error considered separately. It is their product: the score contribution \(X_i\varepsilon_i\). A central limit theorem for the sum of these contributions gives the limiting distribution of the coefficient estimator after the empirical Gram matrix is accounted for.

Lyapunov’s condition is a convenient way to verify that central limit theorem. It bounds moments of order greater than two, ensuring that unusually large score contributions do not dominate the sum. The condition is useful, but it is a sufficient condition rather than a necessary one; finite variance can sometimes be enough even when no higher moment exists.

Lyapunov’s Condition for a Sum

Definition: Let \(Z_{n1},\ldots,Z_{nk_n}\) be independent, mean-zero random variables in row \(n\), and define \(s_n^2=\sum_{i=1}^{k_n}\operatorname{Var}(Z_{ni})\), with \(0<s_n^2<\infty\). For some \(\delta>0\), the Lyapunov condition is

\frac{\sum_{i=1}^{k_n}E[|Z_{ni}|^{2+\delta}]}{s_n^{2+\delta}}\longrightarrow 0.

If the condition holds, then \(\sum_i Z_{ni}/s_n\xrightarrow{d}N(0,1)\).

The numerator measures the total contribution of moments above order two; the denominator is the total variance raised to the matching power. For identically distributed summands \(Z_i\) with mean zero, variance \(\sigma^2>0\), and \(E[|Z_i|^{2+\delta}]<\infty\), the ratio for a sum of \(n\) terms is

\frac{nE[|Z_1|^{2+\delta}]}{(n\sigma^2)^{1+\delta/2}} = \frac{E[|Z_1|^{2+\delta}]}{(\sigma^2)^{1+\delta/2}}n^{-\delta/2} \longrightarrow 0.

Thus, for an identically distributed sequence, any finite moment of order \(2+\delta\) makes the Lyapunov ratio vanish. The argument below explains why the condition works.

Why Lyapunov Implies a Central Limit Theorem

The Lindeberg condition for this array requires that for every \(\epsilon>0\),

\frac{1}{s_n^2}\sum_{i=1}^{k_n} E\!\left[Z_{ni}^2\mathbf{1}\{|Z_{ni}|>\epsilon s_n\}\right] \longrightarrow 0.

It says that the fraction of total variance coming from terms larger than \(\epsilon s_n\) vanishes. Under the independent-array hypotheses stated above, the Lindeberg–Feller central limit theorem says that this condition implies \(\sum_i Z_{ni}/s_n\xrightarrow{d}N(0,1)\). Lyapunov’s condition implies Lindeberg’s condition directly.

Result: For every \(\epsilon>0\),

\frac{1}{s_n^2}\sum_{i=1}^{k_n} E\!\left[Z_{ni}^2\mathbf{1}\{|Z_{ni}|>\epsilon s_n\}\right] \leq \frac{1}{\epsilon^\delta} \frac{\sum_{i=1}^{k_n}E[|Z_{ni}|^{2+\delta}]}{s_n^{2+\delta}}.

Therefore the Lyapunov condition implies the Lindeberg condition and hence the stated central limit theorem.

To verify the inequality, on the event \(|Z_{ni}|>\epsilon s_n\) we have \(1<|Z_{ni}|^\delta/(\epsilon s_n)^\delta\). Multiplying by \(Z_{ni}^2\), taking expectations, summing over \(i\), and dividing by \(s_n^2\) gives

\frac{1}{s_n^2}\sum_i E[Z_{ni}^2\mathbf{1}\{|Z_{ni}|>\epsilon s_n\}] \leq \frac{\sum_i E[|Z_{ni}|^{2+\delta}]}{\epsilon^\delta s_n^{2+\delta}}.

The right side tends to zero by Lyapunov’s condition. The Lindeberg–Feller theorem then yields the claimed normalized normal limit. This proof identifies the role of the higher moment: it controls the variance carried by the large terms.

Applying the Condition to Random-Design OLS

Consider a fixed-dimensional random-design linear regression model with observations \((Y_i,X_i)\), where \(X_i\) is a \(p\)-dimensional column vector and

Y_i=X_i'\beta+\varepsilon_i,\qquad i=1,\ldots,n.

Assume the pairs \((X_i,\varepsilon_i)\) are identically distributed and independent across \(i\), \(E[X_i\varepsilon_i]=0\), and \(E[\|X_i\|^2]<\infty\). Define the population Gram matrix \(Q=E[X_iX_i']\) and score covariance matrix \(\Omega=E[X_iX_i'\varepsilon_i^2]\). For the result below, assume \(Q\) and \(\Omega\) are positive definite, and assume that \(E[\|X_i\varepsilon_i\|^{2+\delta}]<\infty\) for some \(\delta>0\). The positive definiteness of \(Q\) supplies identification; the positive definiteness of \(\Omega\) gives a nondegenerate limiting distribution.

Let \(Q_n=n^{-1}\sum_iX_iX_i'\) and let \(\hat{\beta}\) be the least-squares estimator. When \(Q_n\) is invertible, its exact error representation is

\sqrt{n}(\hat{\beta}-\beta) = Q_n^{-1}\frac{1}{\sqrt{n}}\sum_{i=1}^nX_i\varepsilon_i.

The law of large numbers gives \(Q_n\xrightarrow{p}Q\). To obtain a limit for the sum, it suffices to establish a central limit theorem for every scalar projection of the score vector. This is the Cramér–Wold device: a vector converges in distribution to a specified multivariate normal distribution if every fixed scalar projection has the corresponding univariate normal limit.

Theorem: Under the assumptions above,

\frac{1}{\sqrt{n}}\sum_{i=1}^nX_i\varepsilon_i \xrightarrow{d}N(0,\Omega), \qquad \sqrt{n}(\hat{\beta}-\beta) \xrightarrow{d}N(0,Q^{-1}\Omega Q^{-1}).

Proof. Fix a nonzero vector \(a\in\mathbb{R}^p\) and set \(W_i=a'X_i\varepsilon_i\). These variables are independent and identically distributed, and \(E[W_i]=a'E[X_i\varepsilon_i]=0\). Their variance is

\sigma_a^2=E[W_i^2]=a'\Omega a>0.

Also, \(|W_i|\leq \|a\|\|X_i\varepsilon_i\|\), so \(E[|W_i|^{2+\delta}]<\infty\). The iid Lyapunov ratio therefore equals

\frac{nE[|W_i|^{2+\delta}]}{(n\sigma_a^2)^{1+\delta/2}} = \frac{E[|W_i|^{2+\delta}]}{(\sigma_a^2)^{1+\delta/2}}n^{-\delta/2} \longrightarrow 0.

Lyapunov’s theorem gives \(\frac{1}{\sqrt{n}}\sum_i W_i\xrightarrow{d}N(0,\sigma_a^2)\). Since this holds for every nonzero \(a\), the Cramér–Wold device gives the vector limit for \(n^{-1/2}\sum_iX_i\varepsilon_i\). The law of large numbers gives \(Q_n\xrightarrow{p}Q\); since \(Q\) is positive definite, \(Q_n\) is invertible with probability tending to one and \(Q_n^{-1}\xrightarrow{p}Q^{-1}\). Applying Slutsky’s theorem to the exact OLS representation gives the coefficient limit, with covariance \(Q^{-1}\Omega(Q^{-1})'=Q^{-1}\Omega Q^{-1}\), since \(Q\) is symmetric. \(\square\)

No independence between \(X_i\) and \(\varepsilon_i\) is required here. What matters for this argument is independence across observations, the zero-mean score condition, and the stated moments of the score products. Nor does the result require homoskedasticity: \(\Omega\) allows the conditional variance of the error to depend on the regressors.

Worked Example: A Scalar Regression With Normal Regressors and Errors

Consider scalar regression through the origin, with \(Y_i=\beta X_i+\varepsilon_i\), where \(X_i\sim N(0,1)\) and \(\varepsilon_i\sim N(0,1)\) are independent. Then \(Q=E[X_i^2]=1\), \(E[X_i\varepsilon_i]=0\), and the score is \(W_i=X_i\varepsilon_i\).

Its variance is \(E[W_i^2]=E[X_i^2]E[\varepsilon_i^2]=1\). For \(\delta=2\), independence and the fourth moment of a standard normal give

E[|W_i|^4]=E[X_i^4]E[\varepsilon_i^4]=3\cdot 3=9.

The Lyapunov ratio for the sum of \(n\) scores is

\frac{nE[|W_i|^4]}{(n\operatorname{Var}(W_i))^2} = \frac{9n}{n^2} = \frac{9}{n} \longrightarrow 0.

The score sum divided by \(\sqrt{n}\) therefore converges to \(N(0,1)\). Since \(Q_n\xrightarrow{p}1\), the exact OLS representation gives \(\sqrt{n}(\hat{\beta}-\beta)\xrightarrow{d}N(0,1)\).

Worked Example: A Two-Coefficient Model With Bounded Regressors

Let \(X_i=(1,R_i)'\), where \(R_i\) takes values \(-1\) and \(1\) with equal probability, and let \(\varepsilon_i\sim N(0,1)\) be independent of \(R_i\). Then \(E[X_i\varepsilon_i]=0\). Direct calculation gives

Q=E[X_iX_i'] = \begin{pmatrix}1&0\\0&1\end{pmatrix}, \qquad \Omega=E[X_iX_i'\varepsilon_i^2] = \begin{pmatrix}1&0\\0&1\end{pmatrix}.

For a fixed vector \(a=(a_0,a_1)'\), the scalar score projection is \(W_i=(a_0+a_1R_i)\varepsilon_i\). Its variance is \(a_0^2+a_1^2\), while its fourth moment is

$$ E[|W_i|^4] = 3E[(a_0+a_1R_i)^4] = 3(a_0^4+6a_0^2a_1^2+a_1^4). $$

The last equality follows by averaging the two values \((a_0+a_1)^4\) and \((a_0-a_1)^4\); the odd powers of \(R_i\) cancel. For every nonzero \(a\), the variance is positive and the fourth moment is finite, so the Lyapunov ratio is

\frac{nE[|W_i|^4]}{(n(a_0^2+a_1^2))^2} = \frac{E[|W_i|^4]}{n(a_0^2+a_1^2)^2} \longrightarrow 0.

The vector score thus has limiting covariance \(I_2\), and the OLS coefficient vector satisfies \(\sqrt{n}(\hat{\beta}-\beta)\xrightarrow{d}N(0,I_2)\). This calculation uses score moments, not a requirement that the regressors themselves be normally distributed.

Lyapunov Is Sufficient, Not Necessary

The higher-moment condition is often easy to check, especially when the design is bounded or familiar moment calculations are available. But it is stronger than the Lindeberg condition and can exclude score distributions for which the ordinary central limit theorem still holds. A finite variance alone does not guarantee a Lyapunov condition for any \(\delta>0\).

Worked Example: A Finite-Variance Score With No Higher Moment

Consider an intercept-only model \(Y_i=\beta+\varepsilon_i\), so \(X_i=1\) and \(\hat{\beta}\) is the sample mean. Define a symmetric discrete error distribution by

P(\varepsilon_i=k)=P(\varepsilon_i=-k) = \frac{C}{2k^3(\log(k+1))^2}, \qquad k=1,2,\ldots,

where \(C\) is the positive normalizing constant that makes the probabilities sum to one. Symmetry gives \(E[\varepsilon_i]=0\). The second moment is finite because

E[\varepsilon_i^2] = C\sum_{k=1}^{\infty}\frac{1}{k(\log(k+1))^2} <\infty.

However, for every \(\delta>0\),

E[|\varepsilon_i|^{2+\delta}] = C\sum_{k=1}^{\infty} \frac{k^{\delta-1}}{(\log(k+1))^2} = \infty.

For \(\delta>0\), the summands in the last series eventually dominate a constant multiple of \(k^{-1+\delta/2}\), whose series diverges. Thus no Lyapunov condition based on a moment of order \(2+\delta\) is available. Nevertheless, the iid central limit theorem applies because the errors are independent, identically distributed, mean zero, and have finite positive variance. Consequently, \(\sqrt{n}(\hat{\beta}-\beta)\) converges to a centered normal distribution with variance \(E[\varepsilon_i^2]\).

Common Pitfall and Practical Use

Wrong reasoning: “The Lyapunov moment condition fails, so OLS cannot have an asymptotic normal distribution.”
Correction: Lyapunov is one sufficient route to a central limit theorem. Its failure only means this particular verification is unavailable. A Lindeberg argument or another valid central limit theorem may still apply.
Another mistake: Checking moments of \(X_i\) and \(\varepsilon_i\) separately and assuming that settles the score condition. The relevant quantity is \(X_i\varepsilon_i\); separate moment bounds imply a product bound only under suitable additional assumptions, such as independence and appropriately chosen moment orders.
Interpretation: A Lyapunov proof can be convenient and transparent, but the extra moment should be reported as an assumption used for the proof, not as a universal requirement for asymptotic normality.

The practical sequence is to identify the score contribution, verify its mean and covariance, and then check a moment bound on its scalar projections. Once the score central limit theorem is established, the empirical Gram matrix and the OLS representation translate that limit into the coefficient distribution.

Check Your Understanding

Work through these questions to check the conditions and logic of the argument.

For iid mean-zero scalar scores with variance \(\sigma^2>0\), derive the Lyapunov ratio when \(E[|W_i|^{2+\delta}]<\infty\). Why does it vanish?
State the inequality that shows Lyapunov’s condition implies the Lindeberg condition, and identify where the threshold \(\epsilon s_n\) enters the bound.
In random-design OLS, why is the moment condition naturally imposed on \(X_i\varepsilon_i\) rather than on the error alone?
Does the regression result require \(X_i\) and \(\varepsilon_i\) to be independent within an observation? Which score condition is used instead?
If the score has finite variance but no moment of order \(2+\delta\), what does failure of the Lyapunov condition establish—and what does it not establish?