Local Constant Regression and Its Interior Bias

From a Local Fit to a Regression Estimate

In nonparametric regression, the estimate at a point \(x\) is built from observations near \(x\), with nearby observations receiving more weight. A local constant fit makes the simplest possible local approximation: near \(x\), represent the regression function by a single number \(a\). Choosing that number by weighted least squares gives the Nadaraya–Watson estimator.

The fitted value is not generally centered on the true regression function \(m(x)=E[Y\mid X=x]\). Even away from boundaries, its leading smoothing bias depends on how curved \(m\) is and on how the design points are distributed nearby. The density contribution matters because local weighting averages responses over the observed design points, not over an evenly spaced grid.

The Degree-Zero Fit and Its Formula

Write \(K_h(t)=K(t/h)/h\). The local constant criterion is

Here \(a\) is the only coefficient: the local approximation has degree zero. The bandwidth determines the scale of the neighborhood, while the kernel determines how weights vary within it. Multiplying all weights by the same positive constant does not change the minimizer.

To verify the result, differentiate the criterion with respect to \(a\):

When the weights are nonnegative and their sum is positive, \(Q_x''(a)>0\), so the criterion is strictly convex. Setting \(Q_x'(a)=0\) gives \(a\sum_i K_h(X_i-x)=\sum_i K_h(X_i-x)Y_i\), which yields the stated minimizer. The common factor \(1/h\) cancels from numerator and denominator. \(\square\)

If all weights vanish, the criterion is constant in \(a\) and does not define a fitted value. In practice one reports the estimate only where the denominator is positive. The usual setting assumes a nonnegative kernel, so the fitted value is a weighted average of the responses with positive weights.

The Interior Bias Expansion

To describe the leading smoothing bias, first consider the population version of the weighted average. Let \(X\) have density \(f\), let \(m(t)=E[Y\mid X=t]\), and suppose \(f(x)>0\). For a random observation, the conditional mean response at design value \(t\) is \(m(t)\). The population smoother at \(x\) is therefore

This ratio is the population target of the local constant weighted average. Its difference from \(m(x)\) is the smoothing bias. It is not, for finite \(n\), exactly the expectation of the sample ratio \(\hat m_{\mathrm{NW}}(x)\): taking the expectation of a ratio is not the same as taking the ratio of expectations.

The compact support and interior-point condition ensure that, for sufficiently small \(h\), the kernel neighborhood stays within the region where the stated smoothness conditions hold. Symmetry gives \(\int uK(u)\,du=0\); normalization and the second moment determine the remaining terms of this order.

For a proof, substitute \(t=x+hu\) in the numerator and denominator, cancelling the common factor \(h\). Define \(q(t)=m(t)f(t)\). Taylor expansion at \(x\), with the continuous second derivatives giving remainders uniform over the bounded support of \(K\), gives

The first-order terms vanish because the kernel is symmetric. Since \(f(x)>0\), divide the expansions using \(1/(f(x)+\delta)=1/f(x)-\delta/f(x)^2+o(\delta)\) as \(\delta\to0\). This yields

Now \(q(x)/f(x)=m(x)\), and differentiating \(q=mf\) gives \(q''=m''f+2m'f'+mf''\). Substituting and cancelling the terms involving \(mf''\) leaves \(m''+2m'f'/f\), proving the expansion. \(\square\)

The first part of the bracket, \(m''(x)\), is the contribution from curvature in the regression function. The second part, \(2m'(x)f'(x)/f(x)\), records how the design density changes across the local neighborhood. It vanishes when the density is locally flat or the regression function has zero slope at \(x\). This is an interior expansion; near a boundary, the effective kernel neighborhood need not be symmetric, so the argument and leading terms can change.

Worked Bias Calculations

Interpretation and a Common Pitfall

The formula shows why a local constant estimate can respond to the design even when the target \(m\) itself is unchanged: observations in denser parts of the neighborhood contribute more to the weighted average. A calculation that retains only \(m''(x)\) is valid only when the density term vanishes, for example when \(f'(x)=0\) or \(m'(x)=0\) at the point of interest. The logarithmic derivative \(f'(x)/f(x)\) describes the local relative change in density, so it is the relevant quantity rather than \(f'(x)\) alone.

• Wrong reasoning: “The kernel is symmetric, so the local average has no first-order effect and its leading bias is just the curvature term.”
• Correction: Kernel symmetry removes the first kernel moment, but it does not make the design density locally constant. Expanding both the numerator and denominator of the population ratio produces the additional term \(2m'(x)f'(x)/f(x)\).

A second distinction is also important: the derived expression is the leading bias of the population smoothing target. The sample estimator is a ratio with a random denominator, so its finite-sample expectation does not equal that target exactly. In applications, the displayed expansion describes the leading smoothing component; sampling variability and finite-sample ratio effects are separate considerations.