Common Errors in Local Polynomial Analysis

Why Small Errors Change the Analysis

Local polynomial methods are built from weighted least squares, but their formulas involve several conventions at once: the scale used for the kernel, the coordinate in which the polynomial is written, and the location at which its moments are evaluated. A calculation can look familiar and still be wrong if one of these conventions changes unnoticed. The errors are consequential: they can alter a derivative estimate, a boundary correction, or a claimed conditional expectation.

This tutorial sets out a consistent notation and uses short calculations to diagnose five frequent mistakes. The emphasis is not on deriving the full bias and variance theory again, but on checking the assumptions and scaling needed to use those results correctly.

A Common Setup and the Kernel Convention

Suppose \(Y_i=m(X_i)+\varepsilon_i\), and we estimate \(m(x)\) at a target \(x\) with bandwidth \(h>0\). For a degree-\(p\) local polynomial, write the fitted polynomial in the scaled coordinate \(u=(X_i-x)/h\):

(\hat\beta_0,\ldots,\hat\beta_p) \in \arg\min_{b_0,\ldots,b_p} \sum_{i=1}^n K\!\left(\frac{X_i-x}{h}\right) \left[Y_i-\sum_{j=0}^p b_j\left(\frac{X_i-x}{h}\right)^j\right]^2.

Here \(K\) is a nonnegative kernel, and the fit is assumed to have a unique solution whenever a coefficient is interpreted as a single value. The fitted value at the target is \(\hat m(x)=\hat\beta_0\). The bandwidth-scaled kernel is often written \(K_h(v)=K(v/h)/h\). Replacing each weight \(K((X_i-x)/h)\) by \(K_h(X_i-x)\) multiplies every weight by the same positive factor \(1/h\), so it does not change the minimizer.

Definition: The \(r\)th kernel moment, when it exists, is \(\mu_r(K)=\int u^rK(u)\,du\). In formulas that use a normalized kernel, the convention is \(\mu_0(K)=1\). Moments over a restricted support, such as the part of the kernel window available at a boundary, must instead be computed over that restricted region.

Two different operations are easily confused. Multiplying all weights in a weighted least-squares objective by the same positive constant leaves the fit unchanged. But moment expressions and asymptotic constants are numerical quantities: their values depend on the actual scaling of \(K\). A formula derived with \(\mu_0(K)=1\) cannot be used unchanged with an unnormalized kernel.

Worked Example: A Kernel Rescaling Leaves the Fit Alone

Let \(K_1(u)=\tfrac12\) for \(-1\leq u\leq 1\) and zero elsewhere, and let \(K_2(u)=1\) for \(-1\leq u\leq 1\) and zero elsewhere. Thus \(K_2=2K_1\). For any observations and any \(x,h\), each observation inside the window receives weight \(K_2((X_i-x)/h)=2K_1((X_i-x)/h)\), while observations outside receive zero under both kernels.

The objective based on \(K_2\) is therefore exactly twice the objective based on \(K_1\). Multiplying an objective by 2 does not change its minimizing coefficient vector, so both kernels give the same local polynomial fit, assuming the minimizer is unique.

Their moments are not the same: \(\mu_0(K_1)=\int_{-1}^1\tfrac12\,du=1\), whereas \(\mu_0(K_2)=\int_{-1}^1 1\,du=2\). For example, \(\mu_2(K_1)=\tfrac12\int_{-1}^1u^2du=1/3\), while \(\mu_2(K_2)=\int_{-1}^1u^2du=2/3\). The fit is invariant to this rescaling; a formula involving kernel moments is not invariant unless its other terms are adjusted consistently.

Bandwidth Powers and the Meaning of the Coefficients

The coefficients \(b_j\) in the displayed objective multiply powers of the scaled coordinate \(u=(X_i-x)/h\), not powers of \(X_i-x\). This choice keeps the design coordinates near order one in the fitting window, but it means the coefficients are not directly the derivatives of \(m\).

Key relation: If \(m\) has \(p\) derivatives at \(x\), its Taylor polynomial in the scaled coordinate is \(\sum_{j=0}^p [h^j m^{(j)}(x)/j!]u^j\). Thus the population Taylor coefficient for \(u^j\) is \(h^j m^{(j)}(x)/j!\), and the corresponding derivative is obtained by multiplying that coefficient by \(j!/h^j\).

This relation explains the bandwidth powers. For instance, an estimate of \(m'(x)\) based on the coefficient of \(u\) is \(\hat\beta_1/h\); an estimate of \(m''(x)\) based on the coefficient of \(u^2\) is \(2\hat\beta_2/h^2\). These are not optional conventions: they follow from the chain rule and the Taylor expansion.

Worked Example: Checking the Scaled Taylor Coefficients

Take \(m(x)=2+3x+4x^2\), target \(x=1/2\), and bandwidth \(h=1/5\). Put \(X=x+hu\). Substitution gives

\begin{aligned} m(1/2+u/5) &=2+3(1/2+u/5)+4(1/2+u/5)^2\\ &=2+\frac32+\frac35u+1+\frac45u+\frac{4}{25}u^2\\ &=\frac92+\frac75u+\frac{4}{25}u^2. \end{aligned}

Therefore the coefficients in the \(u\)-coordinate are \(\beta_0=9/2\), \(\beta_1=7/5\), and \(\beta_2=4/25\). The original-scale derivatives are \(m(1/2)=9/2\), \(m'(1/2)=3+8(1/2)=7\), and \(m''(1/2)=8\). The coefficient relations check exactly: \(\beta_1/h=(7/5)/(1/5)=7\), and \(2\beta_2/h^2=2(4/25)/(1/25)=8\).

A common error is to report \(\hat\beta_1\) itself as an estimate of the derivative. In this parameterization, that would report \(7/5\) where the derivative is 7. The discrepancy is precisely the missing factor \(1/h\).

Bandwidth powers also appear in variance orders. Under the usual one-dimensional random-design conditions—positive design density near \(x\), finite conditional error variance, and a regular local fit—the effective number of observations in a window is of order \(nh\). A derivative of order \(j\) rescales a coefficient by \(h^{-j}\), so its variance has order \(1/(nh^{2j+1})\). This order is a scaling guide, not a replacement for the conditions or constants in a particular variance formula.

Boundary Moments Are One-Sided

At an interior point with a symmetric kernel, odd moments vanish because the integration region is symmetric and the integrand \(u^rK(u)\) is odd for odd \(r\). At a boundary, the observed design may supply only one side of the kernel window. Then symmetry no longer applies, and using interior moments can invalidate a boundary calculation.

Worked Example: Local Linear Moments at a Boundary

Let the design support begin at 0, take target \(x=0\), and use the uniform kernel \(K(u)=1/2\) on \([-1,1]\). For positive bandwidth, the available scaled design coordinates are \(u\in[0,1]\), not \([-1,1]\). The restricted moments are

\mu_0^+=\int_0^1\frac12\,du=\frac12,\qquad \mu_1^+=\int_0^1\frac12u\,du=\frac14,\qquad \mu_2^+=\int_0^1\frac12u^2\,du=\frac16.

In particular, the first moment is not zero. For a local linear fit, the equivalent weight function on the available half-window is proportional to \(K(u)(\mu_2^+-\mu_1^+u)/D\), where \(D=\mu_0^+\mu_2^+-(\mu_1^+)^2\). Here \(D=(1/2)(1/6)-(1/4)^2=1/48\), so the weight function is

L(u)= \frac{(1/2)(1/6-u/4)}{1/48} =4-6u,\qquad 0\leq u\leq1.

Its moments verify the local linear reproduction conditions: \(\int_0^1L(u)\,du=4-3=1\) and \(\int_0^1uL(u)\,du=2-2=0\). The negative values for \(u>2/3\) do not mean the original kernel weights were negative; they arise in the equivalent weights after fitting and correcting for the one-sided design.

The practical check is straightforward: write the actual range of \(u=(X_i-x)/h\) first, then integrate over that range. At an interior point, the full kernel support may be available; near a boundary it may be truncated. Do not set odd moments to zero merely because the untruncated kernel is symmetric.

Random-Design Conditioning Requires the Full Design

A local polynomial estimator can be written, conditional on the design locations, as a linear smoother in the responses whenever the weighted least-squares solution is unique. Let \(S(X_1,\ldots,X_n)\) be its smoothing matrix, so the fitted vector is \(S Y\). In general, the conditional mean of this fit depends on the conditional mean of the error vector given all observed locations, not just on each location separately.

Result: Suppose the weighted fit is uniquely defined and its weights and design matrix are functions of \(X_1,\ldots,X_n\). If \(Y_i=m(X_i)+\varepsilon_i\) and \(E[\varepsilon_i\mid X_1,\ldots,X_n]=0\) for every \(i\), then \(E[SY\mid X_1,\ldots,X_n]=S(m(X_1),\ldots,m(X_n))^\mathsf{T}\).

Proof. Conditional on the full design, \(S\) and the vector \(m_X=(m(X_1),\ldots,m(X_n))^\mathsf{T}\) are fixed. Since \(Y=m_X+\varepsilon\), linearity of conditional expectation gives

E[SY\mid X_1,\ldots,X_n] =S E[m_X+\varepsilon\mid X_1,\ldots,X_n] =S m_X+S E[\varepsilon\mid X_1,\ldots,X_n] =S m_X.

The final equality uses the stated mean-zero condition for every component conditional on all locations. This proves the result. \(\square\)

Worked Example: Why Conditioning on Each Own Location Is Not Enough

Let \(X_1,X_2\) be independent Bernoulli random variables with probability \(1/2\) of being 1. Set \(m\equiv0\), \(\varepsilon_1=X_2-1/2\), and \(\varepsilon_2=0\). Independence gives \(E[\varepsilon_1\mid X_1]=E[X_2-1/2]=0\), and \(E[\varepsilon_2\mid X_2]=0\). Yet \(E[\varepsilon_1\mid X_1,X_2]=X_2-1/2\), which is not always zero.

Consider the local constant fit with equal positive weights on these two observations. Its fitted value is \((Y_1+Y_2)/2=(X_2-1/2)/2\), so its conditional expectation given both locations is \((X_2-1/2)/2\), not zero in general. The separate conditions \(E[\varepsilon_i\mid X_i]=0\) therefore do not justify replacing the conditional mean of the fit by the fit to the regression values. A sufficient familiar setting for the full-design condition is independent sampling of the pairs \((X_i,Y_i)\), together with \(E[\varepsilon_i\mid X_i]=0\).

Three Checks Before Trusting a Formula

Identify the coordinate.
Check whether the polynomial uses \(X_i-x\) or \((X_i-x)/h\). Convert coefficients to derivatives using the corresponding powers of \(h\) and factorials.

Write the integration region.
Determine which scaled locations are available. Compute moments on that region rather than borrowing symmetric interior moments at a boundary.

Condition on what is random.
When taking a conditional expectation of a fitted value, state whether the expectation is conditional on the full design and check the error-mean assumption at that conditioning level.

The interpretation of \(\hat\beta_j\) also depends on the basis and target. In the scaled-coordinate Taylor interpretation, \(\hat\beta_0\) estimates the function value at \(x\), while \(\hat\beta_j\) estimates a bandwidth-scaled derivative coefficient, not the derivative itself. In finite samples, the fitted coefficient also reflects the weighted observations and need not equal the Taylor coefficient exactly. A reliable analysis states the parameterization and reports any required conversion explicitly.

Common Mistakes and Their Corrections

“A kernel can be rescaled, so its moments do not matter.” Rescaling all least-squares weights by one common positive constant leaves the minimizer unchanged, but changes the numerical moments. Keep the kernel normalization consistent between the objective and the formulas.
“The coefficient of \(u^j\) estimates \(m^{(j)}(x)\).” In the scaled coordinate \(u=(X-x)/h\), the Taylor coefficient is \(h^jm^{(j)}(x)/j!\). Convert it by \(j!/h^j\).
“A symmetric kernel has zero odd moments everywhere.” That statement applies to integration over a symmetric region. At a boundary, use the available one-sided region and recompute the moments.
“Mean-zero errors given their own covariates imply mean-zero errors given the entire design.” Not without additional assumptions. For the conditional linear-smoother calculation, state the full-design condition or justify it from the sampling model.
“Equivalent weights must be nonnegative because the kernel is nonnegative.” Local polynomial fitting can produce negative equivalent weights, as the boundary calculation showed. Check reproduction moments rather than assuming positivity.

Key takeaway: Most local polynomial bookkeeping errors can be caught by naming the coordinate, recording the actual kernel integration region, and matching every conditional expectation to its stated assumptions. These checks are especially important when moving from an interior formula to a boundary or derivative calculation.

Check Your Understanding

For each question, state the convention or calculation you would check before accepting the claim.

If a kernel is multiplied by 3 in a weighted least-squares objective, which quantities are unchanged and which may change?
A quadratic fit uses \(u=(X_i-x)/h\) and has coefficient \(\hat\beta_2\). What is the corresponding estimate of the second derivative?
Why is the first kernel moment generally nonzero when only the right half of a symmetric kernel window is available?
What conditional mean assumption on the errors justifies taking the full-design conditional expectation of a linear smoother?
For the boundary equivalent weight \(L(u)=4-6u\) on \([0,1]\), verify its zeroth and first moments and explain their role.