Why OLS Limit Arguments Need an Audit
A random-design OLS argument often reduces to a matrix average multiplied by a sum of score terms. That familiar structure does not make every limit valid: the population design must identify the coefficients, no single observation can dominate the score sum, the score must have enough moments for the proposed central limit theorem, and its expectation must be zero at the claimed parameter. Checking these conditions is part of the argument, not a formality.
Let \(x_i\) be a \(p\)-dimensional regressor vector and consider \(y_i=x_i'\beta+\varepsilon_i\), with fixed \(p\). In the usual independent and identically distributed random-design setting, a standard sufficient set of conditions for the familiar normal limit is: \(E[x_i x_i']=Q\) exists and is positive definite; \(E[x_i\varepsilon_i]=0\); \(E[\|x_i\varepsilon_i\|^2]<\infty\); and the observations are independent and identically distributed. These conditions imply the law of large numbers and multivariate central limit theorem needed below. They are sufficient conditions, not a claim that every one is necessary for every possible limit theorem.
When \(X'X\) is invertible, substitution of the model into the OLS formula gives the exact identity
Under the stated conditions, the first factor converges in probability to \(Q^{-1}\), while the second has a centered normal limit with covariance \(\Omega=E[\varepsilon_i^2 x_i x_i']\). This decomposition shows where the potential failures enter. The examples below each break one part of the reasoning.
Failure 1: A Singular Population Design
A sample cross-product matrix can be invertible in some samples without a well-defined, uniquely identified population coefficient vector. If \(Q\) is singular, there is a nonzero vector \(v\) such that \(Qv=0\). Since \(v'Qv=E[(v'x_i)^2]=0\), it follows that \(v'x_i=0\) almost surely. Adding any multiple of \(v\) to a coefficient vector therefore leaves its fitted linear predictor unchanged.
Worked Example: Two Regressors That Are Exact Multiples
Let \(x_i=(1,Z_i,2Z_i)'\), where \(Z_i\) is nondegenerate and has a finite second moment. There are three coefficients, but the second and third regressors are exact multiples.
Take \(v=(0,-2,1)'\). For every observation, \(v'x_i=-2Z_i+2Z_i=0\). Consequently, \(Qv=E[x_i(x_i'v)]=0\), so \(Q\) is singular. Moreover, for any scalar \(t\), the coefficients \(\beta\) and \(\beta+tv\) give the same predictor:
Thus the data can identify the intercept and the combined effect of the two \(Z_i\) columns, but not their separate coefficients. An argument that writes \(Q^{-1}\), or reports a unique limit for all three coefficients, is invalid. More observations do not resolve an exact population collinearity.
The key diagnostic is not merely whether an inverse appears in a derivation. Check that the matrix being inverted converges to a positive-definite population matrix. Near-singularity is a different issue: it can produce unstable finite-sample approximations even when the population matrix is technically invertible.
Failure 2: One Observation Does Not Become Negligible
An ordinary central limit theorem for the score sum relies on the contributions becoming suitably small relative to the sum's scale. In random-design OLS, a high-leverage observation can carry a nonvanishing share of the normalized design. A finite population second moment alone does not rule out this phenomenon for a sequence of designs.
Worked Example: A Triangular Design With Persistent Leverage
For each sample size \(n\), fit a regression through the origin with scalar regressors \(x_1=\sqrt n\) and \(x_i=1\) for \(i=2,\ldots,n\). Let the errors be independent Rademacher variables: \(P(\varepsilon_i=1)=P(\varepsilon_i=-1)=1/2\). The model is \(y_i=\beta x_i+\varepsilon_i\). The design sum of squares and first OLS leverage are
The first observation's leverage does not vanish. The OLS error is exactly
By the central limit theorem, \(\sum_{i=2}^n\varepsilon_i/\sqrt n\xrightarrow{d}Z\), where \(Z\sim N(0,1)\); this limit is independent of \(\varepsilon_1\). Therefore
This is not normally distributed: its characteristic function is \(\cos(t/2)e^{-t^2/8}\), which has zeros, unlike the characteristic function of a normal distribution. The regressor values in each row of the design are known, and the errors have finite moments of every order; nevertheless, the usual normal limit fails because one score contribution remains influential. A leverage condition or another suitable negligibility condition is needed before applying a central limit theorem to the full score sum.
Failure 3: The Score Has Too Few Moments
The central limit theorem for the score sum is not available merely because the regressors have finite second moments. Its relevant random vector is \(x_i\varepsilon_i\), so a common sufficient condition for the ordinary square-root-\(n\) limit is a finite second moment for that product. The next example has a perfectly regular constant design but errors with infinite variance.
Worked Example: Infinite-Variance Errors With a Constant Regressor
Use a regression through the origin with \(x_i=1\), so \(\hat\beta-\beta=n^{-1}\sum_i\varepsilon_i\). Let the errors be independent with the symmetric Pareto density
This is a probability density because its integral over the two tails is \(\alpha\int_1^\infty u^{-\alpha-1}\,du=1\). Symmetry gives \(E[\varepsilon_i]=0\), and \(E[|\varepsilon_i|]=\alpha/(\alpha-1)<\infty\). But \(E[\varepsilon_i^2]=\infty\), since the tail integral for the second moment is proportional to \(\int_1^\infty u^{1-\alpha}\,du\), which diverges for \(\alpha=3/2\).
For this Pareto distribution, its characteristic function \(\phi(t)=E[e^{it\varepsilon_i}]\) satisfies
where \(c_\alpha=\alpha\int_0^\infty(1-\cos u)u^{-\alpha-1}\,du\) is finite and positive. Finiteness follows because near zero the integrand is bounded by a constant times \(u^{1-\alpha}\), and in the tail it is bounded by \(2u^{-\alpha-1}\). It follows that the characteristic function of \(n^{-1/\alpha}\sum_i\varepsilon_i\) converges to \(\exp(-c_\alpha|t|^\alpha)\), the characteristic function of a nondegenerate symmetric stable random variable \(S\). Hence
Here \(1/\alpha-1/2=1/6>0\). The expression in parentheses converges in distribution to \(S\), which has a continuous distribution and no atom at zero. Multiplying it by \(n^{1/6}\) makes the sequence non-tight: for each fixed \(M\), the probability that its absolute value is at most \(M\) tends to zero. Thus a square-root-\(n\) normal approximation cannot be justified in this example. The lesson is to check moments of the score, not just moments of the regressor.
Failure 4: The Score Is Not Centered at the Claimed Coefficient
The condition \(E[x_i\varepsilon_i]=0\) is a population condition at the proposed coefficient. If it fails, the normalized score sum contains a nonzero mean term. The issue is not repaired by applying a central limit theorem to its centered fluctuations: the mean accumulates at order \(n\) before the sum is divided by \(\sqrt n\).
Worked Example: A Correlated Error Shifts the OLS Target
Let \(x_i\) independently take values \(-1\) and \(1\), each with probability \(1/2\). Set \(\varepsilon_i=x_i\), and define \(y_i=\beta x_i+\varepsilon_i\). All variables have finite moments, and \(E[x_i^2]=1\), but the score is not centered:
In fact, \(y_i=(\beta+1)x_i\) for every observation. Whenever the sample contains a nonzero regressor—which it always does—OLS through the origin gives \(\hat\beta=\beta+1\) exactly. The estimator does not converge to \(\beta\). Its population least-squares target is \(E[x_i y_i]/E[x_i^2]=\beta+1\).
The false step would be to write the OLS expansion and treat \(n^{-1/2}\sum_i x_i\varepsilon_i\) as a centered fluctuation around zero. Here it equals \(\sqrt n\) exactly, so it does not have a finite centered normal limit. Before using a score-based limit at \(\beta\), verify its expectation is zero under the stated model.
A Practical Diagnostic Sequence
A useful audit separates identification, averaging, and centering. It prevents a valid-looking matrix manipulation from concealing a failed probability limit.
Define \(Q=E[x_i x_i']\). Verify it exists and is positive definite before replacing the sample inverse by \(Q^{-1}\).
At the coefficient being studied, calculate \(E[x_i\varepsilon_i]\). Do not infer it from the notation or from \(E[\varepsilon_i]=0\) alone.
Establish the moment and dependence conditions for the particular central limit theorem being used, applied to \(x_i\varepsilon_i\).
For changing designs or triangular arrays, examine whether the largest normalized score or leverage contribution vanishes. Fixed-row intuition is not a substitute for this check.
Once the matrix limit, score limit, and centering have been justified, use the OLS expansion and the appropriate limit theorem.
Common Mistake
Wrong reasoning: “The regressors are independent across observations and the sample size is large, so the OLS coefficient is asymptotically normal.” Independence alone neither ensures that the population coefficients are identified nor verifies a central limit theorem for the score. It also does not ensure that a score is centered or that a single row of a changing design is negligible.
Correction: Start from the exact OLS error identity. Check that its matrix factor converges to an invertible limit, and separately justify the score sum's centering, moments, and negligibility conditions. If any check fails, the proposed limit may be undefined, shifted, nonnormal, or not tight at the stated rate.
Check Your Understanding
For each question, identify which part of the OLS limit argument is at issue.
- If \(Q\) has a nonzero null vector \(v\), what does \(v'x_i=0\) almost surely imply about identifying \(\beta\)?
- In the triangular-design example, why does a finite-variance error distribution not by itself guarantee a normal limit?
- For the constant-regressor Pareto example, which random quantity lacks a finite second moment, and what scaling describes the score sum?
- Why is \(E[\varepsilon_i]=0\) not enough to ensure \(E[x_i\varepsilon_i]=0\) in a regression without an intercept?
- In the OLS error identity, which two separate limits must be justified before combining them to obtain a coefficient limit?