OLS Under Nonlinear Conditional Means

Why OLS Can Have a Limit Under a Nonlinear Mean

A linear regression can be useful even when the conditional mean of the outcome is curved, has a threshold, or gradually saturates. In that setting, the key question is not whether OLS recovers a nonexistent straight-line conditional mean. It is which linear approximation the data and the distribution of the regressor select.

The limit depends on both the response function and how often different regressor values occur. Consequently, the OLS slope need not equal the derivative of the conditional mean at a typical value of the regressor. The derivation below identifies the population quantity OLS estimates, and the examples calculate it for three distinct response shapes.

Population Linear Approximation and Assumptions

Let \((Y_i,X_i)\), \(i=1,\ldots,n\), be independent and identically distributed observations, where \(X_i\) is a scalar regressor. Write the conditional mean as \(m(x)=E[Y_i\mid X_i=x]\), and suppose the outcome can be expressed as \(Y_i=m(X_i)+u_i\), with \(E[u_i\mid X_i]=0\). The function \(m\) is not required to be linear.

OLS with an intercept fits the sample regression of \(Y_i\) on \(1\) and \(X_i\). For its probability limit, assume \(E[Y_i^2]<\infty\), \(E[X_i^2]<\infty\), and that the matrix \(E[Z_iZ_i']\) is nonsingular, where \(Z_i=(1,X_i)'\). Nonsingularity rules out a regressor that is constant with probability one. These conditions allow the sample moments to converge to their population counterparts and ensure a unique population linear projection.

Definition: The population linear projection of \(Y\) on an intercept and \(X\) is the linear function \(\beta_0+\beta_1X\) whose coefficients minimize \(E[(Y-\beta_0-\beta_1X)^2]\). Its coefficients are a best linear approximation under the population distribution of \(X\), not necessarily parameters of the conditional mean.

The projection theorem gives the normal equations \(E[Z(Y-Z'\beta)]=0\), where \(\beta=(\beta_0,\beta_1)'\). Equivalently, \(\beta=E[ZZ']^{-1}E[ZY]\). When the conditional mean is nonlinear, this definition still applies: it selects the intercept and slope that minimize the average squared prediction error among straight lines.

Deriving the Probability Limit

The sample OLS estimator can be written using sample averages. Under the stated assumptions, the law of large numbers makes those averages converge to their population expectations. The conditional-mean restriction then shows why the resulting limit is the linear projection of \(m(X)\).

\begin{aligned} \hat{\beta} &=\left(\frac{1}{n}\sum_{i=1}^n Z_iZ_i'\right)^{-1} \left(\frac{1}{n}\sum_{i=1}^n Z_iY_i\right)\\ &\xrightarrow{p} E[ZZ']^{-1}E[ZY]\\ &=E[ZZ']^{-1}E[Z\{m(X)+u\}]\\ &=E[ZZ']^{-1}E[Zm(X)]. \end{aligned}

For the last equality, \(Z\) is determined by \(X\), so \(E[Zu]=E[Z E[u\mid X]]=0\). Thus OLS converges to the coefficients of the population linear projection of the conditional mean. This conclusion does not require \(m(X)\) itself to be linear. It does require the stated moment and nonsingularity conditions, as well as the conditional-mean restriction used to remove \(u\).

For a scalar regressor with finite, positive variance, the same result has a useful form. The slope is the covariance of \(X\) and \(Y\) divided by the variance of \(X\); since \(E[u\mid X]=0\), this is also the covariance of \(X\) and \(m(X)\) divided by the variance of \(X\). The intercept makes the fitted line pass through the point of means.

\beta_1=\frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(X)} =\frac{\operatorname{Cov}(X,m(X))}{\operatorname{Var}(X)}, \qquad \beta_0=E[Y]-\beta_1E[X].

The slope formula makes the role of the regressor distribution explicit. Changing the distribution of \(X\) can change the covariance and variance, and therefore change the linear approximation even if the response function \(m\) stays fixed. A projection slope is generally not a derivative: it summarizes how the response varies across the regressor values represented in the population.

Worked Linear Approximations

Worked Example: A Quadratic Conditional Mean

Suppose \(X\) is uniformly distributed on \([0,2]\), and the conditional mean is \(m(x)=1+2x+3x^2\). Assume \(Y=m(X)+u\), with \(E[u\mid X]=0\) and finite second moments. The population variance of this uniform regressor is \(\operatorname{Var}(X)=1/3\).

The needed moments are \(E[X]=1\), \(E[X^2]=4/3\), and \(E[X^3]=2\). Therefore, \(E[m(X)]=1+2(1)+3(4/3)=7\). The covariance is

\begin{aligned} \operatorname{Cov}(X,m(X)) &=\operatorname{Cov}(X,1+2X+3X^2)\\ &=2\operatorname{Var}(X)+3\{E[X^3]-E[X]E[X^2]\}\\ &=2(1/3)+3\{2-(1)(4/3)\}=8/3. \end{aligned}

Thus \(\beta_1=(8/3)/(1/3)=8\), and \(\beta_0=7-8(1)=-1\). The probability limit of the OLS line is \(-1+8X\). The true conditional-mean derivative is \(m'(x)=2+6x\), which varies from 2 to 14 over the support. The projection slope of 8 is not a derivative at every \(x\); it is the slope of the best linear approximation for this uniform distribution.

Worked Example: A Threshold Response

Let \(X\) be uniform on \([0,1]\), and suppose \(m(x)=1+2x+4\mathbf{1}\{x>0.5\}\), where the indicator equals 1 when \(x>0.5\) and 0 otherwise. Assume again that \(E[u\mid X]=0\) and the required moments are finite. Here \(E[X]=0.5\) and \(\operatorname{Var}(X)=1/12\).

The mean response is \(E[m(X)]=1+2(0.5)+4(0.5)=4\). To calculate the slope, use the covariance formula. The linear part contributes \(2\operatorname{Var}(X)=1/6\). For the threshold term,

E[(X-0.5)\mathbf{1}\{X>0.5\}] =\int_{0.5}^{1}(x-0.5)\,dx =\frac{1}{8}.

The jump of size 4 therefore contributes \(4(1/8)=1/2\) to the covariance. In total, \(\operatorname{Cov}(X,m(X))=1/6+1/2=2/3\), so \(\beta_1=(2/3)/(1/12)=8\). The intercept is \(\beta_0=4-8(0.5)=0\). The probability limit is the line \(8X\).

The conditional mean has slope 2 on either side of the threshold and a jump at \(0.5\); it does not have a single slope that equals 8. The projection slope also reflects the jump and the uniform distribution’s weighting of values above and below the threshold.

Worked Example: A Saturating Conditional Mean

Suppose \(X\) is uniform on \([0,2]\), and \(m(x)=1+5x/(1+x)\). This response rises as \(x\) increases but its derivative, \(5/(1+x)^2\), declines. Assume \(Y=m(X)+u\), \(E[u\mid X]=0\), and finite second moments. As before, \(E[X]=1\) and \(\operatorname{Var}(X)=1/3\).

For a uniform distribution on \([0,2]\), expectations are integrals divided by 2. First,

E\left[\frac{X}{1+X}\right] =\frac{1}{2}\int_0^2\frac{x}{1+x}\,dx =1-\frac{\ln 3}{2}.

Next, since \(x^2/(1+x)=x-1+1/(1+x)\),

E\left[\frac{X^2}{1+X}\right] =\frac{1}{2}\int_0^2\frac{x^2}{1+x}\,dx =\frac{\ln 3}{2}.

It follows that \(\operatorname{Cov}(X,X/(1+X))=\ln 3-1\). The constant 1 in \(m(x)\) has zero covariance with \(X\), so the OLS slope limit is \(5(\ln 3-1)/(1/3)=15(\ln 3-1)\), approximately 1.479. The mean response is \(6-(5/2)\ln 3\), approximately 3.253. Hence the intercept is \(6-(5/2)\ln 3-15(\ln 3-1)=21-(35/2)\ln 3\), approximately 1.774. The limiting line is approximately \(1.774+1.479X\).

The fitted slope is an average linear summary over \(X\in[0,2]\), not the derivative at a particular regressor value. Changing the range or distribution of \(X\) would generally change the projection, even though the saturating response function remained the same.

What the Limit Does—and Does Not—Mean

Under the assumptions above, nonlinearity does not prevent OLS from converging. It determines what the limit represents: a population linear projection, rather than coefficients from a correctly linear conditional mean. The projection is defined relative to a particular distribution of \(X\), so it is a summary of the response over that distribution. It does not, by itself, establish a causal effect.

The projection’s residual \(Y-\beta_0-\beta_1X\) is orthogonal to the intercept and \(X\): its population mean and its covariance with \(X\) are zero. But when the conditional mean is nonlinear, the conditional expectation of that residual given \(X\) need not be zero. This distinction matters: unconditional projection conditions still identify the best linear approximation, while a linear conditional-mean assumption is a different claim.

A common mistake is to read the OLS slope as the derivative of the conditional mean at the average regressor value. In the quadratic example, \(E[X]=1\) and the derivative there is \(m'(1)=8\), coincidentally the same as the projection slope. That equality is specific to this function and distribution; it does not hold in general. For the threshold response, for instance, the within-segment derivative is 2 while the projection slope is 8. The correction is to identify the estimand first: under a nonlinear conditional mean, OLS estimates the best linear projection for the distribution at hand, and the slope need not be a local derivative.

Key takeaway: With a nonlinear conditional mean and \(E[u\mid X]=0\), OLS converges to the population linear projection. Its slope depends on the response function and the distribution of the regressor; it is not automatically a derivative or a causal effect.

Check Your Understanding

Use the population projection formulas and distinguish the projection from the conditional mean.

State the moment and nonsingularity conditions used to derive the OLS probability limit in this tutorial.
Why does \(E[Zu]=0\) follow from \(E[u\mid X]=0\) when \(Z=(1,X)'\)?
For the quadratic example, what feature of the distribution of \(X\) helps determine the projection slope?
In the threshold example, why is the projection slope not the within-segment derivative?
Explain why changing the distribution of \(X\) can change the OLS limit even when \(m(x)\) is unchanged.