Conditional and Unconditional Regression Limits

Why distinguish conditional and unconditional limits?

In random-design regression, the regressors are random before data collection. After observing them, however, an analyst may want to describe the sampling uncertainty in the coefficient estimator while treating that particular design as fixed. This is a conditional question. A different question averages over both the errors and the random regressors, giving an unconditional limit.

These two viewpoints need not agree automatically. The conditional variance can depend on the observed design, and an unconditional distribution can be a mixture of different conditional distributions. Under a stable iid design and suitable moment and central limit conditions, the design-based conditional covariance converges to a fixed matrix. It then agrees with the covariance in the unconditional normal limit. The distinction is about the conditions that justify this agreement, not about two different formulas for OLS.

Setup and precise meanings

Let \(X_i\) be a \(p\)-dimensional regressor vector, and suppose \(Y_i=X_i'\beta+u_i\), for \(i=1,\ldots,n\). The pairs \((X_i,u_i)\) are iid. Write \(X_1,\ldots,X_n\) for the observed design, and suppose the conditional errors are independent given these regressors. Define \(\sigma^2(x)=E[u_i^2\mid X_i=x]\).

Assume \(E[u_i\mid X_i]=0\), \(E\|X_i u_i\|^2<\infty\), and that \(A=E[X_iX_i']\) is positive definite. Put \(B=E[X_iX_i'u_i^2]\), and define \(A_n=n^{-1}\sum_{i=1}^n X_iX_i'\). The conditional Lindeberg condition below is one convenient precise condition for the conditional central limit step:

Also require that the conditional score covariance \(B_n=n^{-1}\sum_{i=1}^n X_iX_i'\sigma^2(X_i)\) converges in probability to \(B\). Under iid sampling this follows from the law of large numbers when \(E\|X_i u_i\|^2<\infty\). The conditional Lindeberg condition follows, for example, if \(E\|X_i u_i\|^{2+\delta}<\infty\) for some \(\delta>0\): the expectation of the nonnegative left side is \(E[\|X_i u_i\|^2\mathbf{1}\{\|X_i u_i\|>\epsilon\sqrt n\}]\), which tends to zero by dominated convergence, so the left side tends to zero in probability by Markov's inequality.

Conditional and unconditional OLS limits

When \(A_n\) is invertible, OLS satisfies the exact identity \(\hat\beta-\beta=A_n^{-1}n^{-1}\sum_i X_i u_i\). The following result makes the two limiting viewpoints precise.

Unconditional derivation. The score vectors \(X_i u_i\) are iid. Their mean is zero because \(E[X_i u_i]=E[X_iE[u_i\mid X_i]]=0\), and their covariance is \(B\). The multivariate central limit theorem therefore gives \(n^{-1/2}\sum_i X_i u_i\xrightarrow{d}N(0,B)\). The law of large numbers gives \(A_n\xrightarrow{p}A\); because \(A\) is positive definite, \(A_n^{-1}\xrightarrow{p}A^{-1}\). Substitution into the exact OLS identity and Slutsky's theorem give the claimed unconditional limit, with covariance \(A^{-1}BA^{-1}\).

Conditional derivation. Given the regressors, the score terms \(X_i u_i\) are independent, have conditional means zero, and have conditional covariance matrices \(X_iX_i'\sigma^2(X_i)\). The conditional Lindeberg condition and \(B_n\xrightarrow{p}B\) give the conditional Lindeberg–Feller limit: the conditional law of \(n^{-1/2}\sum_i X_i u_i\) converges in probability to \(N(0,B)\). Also \(A_n\xrightarrow{p}A\), so \(A_n^{-1}\xrightarrow{p}A^{-1}\). Applying this convergence to the exact OLS identity yields the conditional limit \(N(0,A^{-1}BA^{-1})\).

This also verifies that the unconditional limit follows from the conditional one. For bounded continuous \(g\), conditional convergence in probability implies convergence in \(L^1\) of the conditional expectations of \(g\): their differences are bounded, and a bounded sequence converging to zero in probability has expectations tending to zero. Taking expectations over the design then gives convergence of the unconditional expectations to those under \(N(0,V)\). Thus the two limits agree under the assumptions; this does not assert that the finite-sample conditional law is normal.

Where the covariance formula comes from

The conditional covariance calculation is particularly direct. Conditional on the design, the scaled score has covariance \(B_n\). Thus the conditional covariance of the scaled OLS error is \(A_n^{-1}B_nA_n^{-1}\), whenever \(A_n\) is invertible. Its probability limit is the same \(V\) as in the unconditional central limit argument.

This agreement does not require homoskedasticity. If instead \(E[u_i^2\mid X_i]=\sigma^2\) is constant, then \(B=\sigma^2 A\), so the common limit simplifies to \(V=\sigma^2 A^{-1}\). With heteroskedastic errors, \(B\) retains the variation of the conditional error variance across regressor values.

There is an important qualification about the word “covariance.” The theorem identifies the covariance matrix of the limiting normal distribution and the probability limit of the conditional covariance matrix. It does not, by itself, prove that the expectation of the finite-sample conditional covariance converges to \(V\). That stronger assertion needs sufficient integrability of those random covariance matrices. In particular, convergence in probability alone does not justify exchanging a limit and an expectation.

When agreement can fail

The stable iid design assumptions matter. If the design is governed by a random quantity shared across the whole sample, conditioning can reveal a persistent feature that does not average away. The conditional limiting covariance may then remain random, and the unconditional limit may be a mixture rather than a single normal distribution.

Common mistake

• Wrong reasoning: “Once the design is observed, OLS has covariance \(\sigma^2(X'X)^{-1}\), so that must be the right covariance in every model.” This uses the homoskedastic formula even when the conditional error variance changes with the regressors.
• Correction: Under conditional heteroskedasticity, use the conditional score covariance. For the scaled estimator it is \(A_n^{-1}B_nA_n^{-1}\), which converges to \(A^{-1}BA^{-1}\) under the stated conditions. Also distinguish this probability limit from the expectation of the finite-sample conditional covariance; the latter requires additional control of integrability.