Moment Conditions for Random-Design OLS

Why Moment Conditions Matter

In random-design regression, both the regressors and the errors vary across observations. Large-sample arguments therefore depend on moments of their joint distribution. A finite second moment for the regressors makes the entries of the population regressor second-moment matrix finite. A central limit theorem for the regression score requires more: the score must have finite covariance. These are distinct requirements, and assumptions that ensure one do not automatically ensure the other.

Let the number of regressors, including any intercept, be a fixed integer \(k\). For observation \(i\), write the model as \(Y_i=x_i^\top\beta+u_i\), where \(x_i\) is a \(k\)-dimensional random vector and \(u_i\) is the error. Assume that the pairs \((x_i,u_i)\) are independent and identically distributed, and that \(\beta\) is fixed. No independence between \(x_i\) and \(u_i\) is assumed. For the moment, the concern is whether relevant expectations exist, not whether the population matrix is nonsingular.

The matrix \(G\) is a second-moment matrix; it is not necessarily a covariance matrix, since the regressors need not be centered. In particular, when \(x_i\) includes an intercept, its first entry is one. The score \(s_i\) is the random vector whose sample average appears in the OLS estimating equations. Its variability depends on the joint behavior of regressors and errors, not just on their separate distributions.

Conditions for Finite Regressor Moments and Score Variance

Use the Euclidean norm \(\|x\|=(\sum_{j=1}^k x_j^2)^{1/2}\). A convenient condition for \(G\) to be finite is \(E\|x_i\|^2<\infty\). The direct condition for finite score covariance is \(E[\|x_i\|^2u_i^2]<\infty\), along with a finite score mean. The latter mean exists under this second condition: by Cauchy–Schwarz, \(E\|x_i u_i\|\leq (E[\|x_i\|^2u_i^2])^{1/2}\).

Proof. For any coordinates \(j,\ell\), \(|x_{ij}x_{i\ell}|\leq (x_{ij}^2+x_{i\ell}^2)/2\leq\|x_i\|^2\). Thus \(E|x_{ij}x_{i\ell}|<\infty\), so the corresponding entry of \(G\) is finite. For the score, its \(j\)th coordinate is \(x_{ij}u_i\), and

Consequently each score coordinate has a finite second moment, and hence a finite first absolute moment by Cauchy–Schwarz. Products of score coordinates are integrable because \(|x_{ij}x_{i\ell}u_i^2|\leq \|x_i\|^2u_i^2\). The covariance entries, which subtract products of finite means from these finite second moments, are therefore finite. \(\square\)

The score condition is deliberately joint: it concerns the product of regressor size and error size. Requiring \(E\|x_i\|^4<\infty\) and \(E|u_i|^4<\infty\) is a familiar sufficient alternative, but can be stronger than necessary. Indeed,

This inequality follows from Cauchy–Schwarz applied to \(\|x_i\|^2\) and \(u_i^2\); it does not require independence. Another useful sufficient condition is \(E[u_i^2\mid x_i]\leq C\) almost surely for some finite constant \(C\), together with \(E\|x_i\|^2<\infty\). The tower property then gives \(E[\|x_i\|^2u_i^2]=E[\|x_i\|^2E(u_i^2\mid x_i)]\leq C E\|x_i\|^2\).

From Moments to Laws of Large Numbers and Central Limit Theorems

Under the iid assumption, the strong law of large numbers applies to each integrable coordinate. Thus \(E\|x_i\|^2<\infty\) ensures that the sample average of \(x_i x_i^\top\) converges almost surely, entry by entry, to \(G\). If \(E[x_i u_i]=0\) and \(E[\|x_i\|^2u_i^2]<\infty\), the score coordinates are integrable, so their sample average converges almost surely to zero. The finite-score-covariance condition also permits the iid multivariate central limit theorem for the centered score.

Justification. For the first convergence, each matrix entry \(x_{ij}x_{i\ell}\) is integrable by the bound in the preceding proof. The iid strong law applies to each of the finitely many entries. For the second, finite score second moments imply finite first absolute moments, and the iid strong law applies to each score coordinate; the stated zero mean gives the limit. Finally, the score vectors are iid, have mean zero, and have finite covariance \(\Omega\), so the iid multivariate central limit theorem gives the displayed distributional convergence. These are applications of the standard iid laws; they do not require normal regressors or normal errors. \(\square\)

The distinction between the two limit arguments is important. A law of large numbers needs integrability of the quantity being averaged. A central limit theorem for the score needs finite score covariance. Finite \(E\|x_i\|^2\) alone says nothing about whether the score variance is finite. Likewise, a finite score variance does not itself imply a zero score mean: that is a separate moment restriction, often ensured by the conditional mean assumption \(E[u_i\mid x_i]=0\), when the score is integrable.

Common Mistake and Practical Interpretation

A common mistake is to reason, “The regressors have finite variance and the errors have finite variance, so the OLS score has finite variance.” That conclusion is not valid without additional information about their joint distribution. The correction is to check \(E[\|x_i\|^2u_i^2]\) directly, or to verify a stated sufficient condition such as finite fourth moments for both quantities or a bounded conditional error second moment. Conversely, imposing fourth moments automatically can be needlessly restrictive when the direct product moment is finite.

These moment conditions support basic averages and score limits; they do not establish every other requirement for OLS asymptotics. In particular, finite moments do not imply that \(G\) is nonsingular, nor do they address dependence across observations or changing distributions. Keep the roles separate: integrability supports laws of large numbers, finite score covariance supports the usual iid central limit theorem, and a zero score mean is an additional centering condition.