From a Local Fit to Observation Weights
A local polynomial estimator is defined by a weighted least-squares fit near the target point. Yet the fitted value at that point can also be written as a weighted average of the observed responses. The weights in this representation are called equivalent weights; their continuous, population analogue is the equivalent kernel. This change of viewpoint makes an estimator’s local averaging and polynomial-reproduction properties explicit.
The focus here is an interior target point \(x\): the kernel window is not truncated by the edge of the design space. Fix a bandwidth \(h>0\), write \(u_i=(X_i-x)/h\), and define the degree-\(p\) polynomial vector \(r_p(u)=(1,u,\ldots,u^p)^\mathsf{T}\). The local polynomial coefficients are obtained by minimizing a kernel-weighted sum of squared residuals. We assume a nonnegative kernel \(K\); the finite-sample fit also requires enough positively weighted design points to identify the polynomial coefficients.
Because \(r_p(0)=e_0\), the intercept coefficient is the fitted polynomial evaluated at the target. Define the weighted moment matrix \(S_n=\sum_{i=1}^n K(u_i)r_p(u_i)r_p(u_i)^\mathsf{T}\). If \(S_n\) is invertible, the normal equations give the exact weights.
Derivation and Exact Moment Identities
To derive the weights, differentiate the weighted sum of squares with respect to \(\beta\). Setting the gradient to zero yields \(S_n\hat\beta=\sum_i K(u_i)r_p(u_i)Y_i\). Invert \(S_n\), take the first coordinate, and collect the coefficient multiplying each \(Y_i\):
The same formula implies exact reproduction of every polynomial of degree at most \(p\) on the observed design. In particular, the equivalent weights satisfy \(\sum_i w_i(x)u_i^j=1\) for \(j=0\) and \(0\) for \(1\leq j\leq p\). These identities are exact when the matrix is invertible; they do not depend on an asymptotic approximation.
where \(e_j\) denotes the coordinate vector for the power \(u^j\), with coordinates indexed from \(0\) to \(p\).
Indeed, by substituting the weight formula and factoring the sum,
For a polynomial response \(Y_i=q(u_i)\) of degree at most \(p\), write \(q(u)=\sum_{j=0}^p a_j u^j\). The fitted value is then \(\sum_j a_j\sum_i w_i u_i^j=a_0=q(0)\), so the fit recovers the polynomial’s value at the target exactly. This is the algebraic reason the moment identities matter.
Worked Example: Exact Local Linear Weights at Three Design Points
Suppose the scaled design points are \(u_1=-1,u_2=0,u_3=1\), with kernel weights \(K(u_1)=1,K(u_2)=2,K(u_3)=1\), and responses \(Y_1=1,Y_2=2,Y_3=5\). Fit a local linear polynomial, so \(r_1(u)=(1,u)^\mathsf{T}\).
The moment matrix is \(S_n=\begin{pmatrix}1+2+1&-1+1\\-1+1&1+1\end{pmatrix} =\begin{pmatrix}4&0\\0&2\end{pmatrix}\). Thus \(e_0^\mathsf{T}S_n^{-1}=(1/4,0)\), and the equivalent weight at \(u_i\) is \(K(u_i)/4\). The three weights are therefore \(1/4,1/2,1/4\).
The fitted value is \(\hat m_1(x)=(1/4)(1)+(1/2)(2)+(1/4)(5)=1/4+1+5/4=5/2\). The moment checks are \(\sum_i w_i=1/4+1/2+1/4=1\) and \(\sum_i w_i u_i=(-1/4)+0+(1/4)=0\), as required for a local linear fit.
The Population Equivalent Kernel
For a continuous description, define the kernel moment matrix \(S=\int K(u)r_p(u)r_p(u)^\mathsf{T}\,du\). Assume these integrals exist and \(S\) is positive definite, so its inverse exists. In an interior setting with a locally stable design, the local polynomial’s leading weight shape is described by the equivalent kernel below. It is a population-level function, not the exact finite-sample weight for any one observation.
Its moment identities follow directly from the definition. For each \(j=0,\ldots,p\), the vector integral \(\int K(u)r_p(u)u^j\,du\) equals \(Se_j\), since its coordinates are the moments \(\int K(u)u^{k+j}\,du\). Consequently,
Thus the equivalent kernel integrates to one and has zero moments for powers one through \(p\). If \(g\) is any polynomial of degree at most \(p\), these identities give \(\int L_p(u)g(u)\,du=g(0)\). Moments of orders above \(p\) are not generally zero; they are not part of the exact polynomial-reproduction requirement.
Worked Example: The Local Linear Equivalent Kernel for an Asymmetric Kernel
Take \(p=1\) and \(K(u)=(1+u)/2\) for \(-1\leq u\leq1\), with \(K(u)=0\) elsewhere. This is a nonnegative kernel with integral one. Its first two moments are \(\mu_1=\int uK(u)\,du=1/3\) and \(\mu_2=\int u^2K(u)\,du=1/3\); also \(\mu_0=1\).
The moment matrix and its inverse are \(S=\begin{pmatrix}1&1/3\\1/3&1/3\end{pmatrix}\) and \(S^{-1}=\begin{pmatrix}3/2&-3/2\\-3/2&9/2\end{pmatrix}\). The determinant of \(S\) is \(1/3-1/9=2/9>0\), confirming invertibility. The first row of the inverse gives \(L_1(u)=K(u)(3/2-3u/2)\), hence \(L_1(u)=3(1-u^2)/4\) on \([-1,1]\), and zero elsewhere.
Direct verification gives \(\int L_1(u)\,du=(3/4)(2-2/3)=1\). Since \(L_1\) is even, \(\int uL_1(u)\,du=0\). Although the original kernel \(K\) is asymmetric, the local linear correction produces an equivalent kernel with the required zero first moment.
Worked Example: A Quadratic Equivalent Kernel Can Be Negative
Let \(K(u)=1/2\) on \([-1,1]\), zero elsewhere, and fit degree \(p=2\). The odd moments vanish, while \(\mu_0=1,\mu_2=1/3,\mu_4=1/5\). In the order \(1,u,u^2\), the moment matrix is \(S=\begin{pmatrix}1&0&1/3\\0&1/3&0\\1/3&0&1/5\end{pmatrix}\). The determinant of its block for the constant and quadratic terms is \(1/5-1/9=4/45>0\), and the remaining diagonal block is \(1/3>0\), so \(S\) is positive definite.
The first row of \(S^{-1}\) is \((9/4,0,-15/4)\). Therefore \(L_2(u)=K(u)(9/4-15u^2/4)=9/8-15u^2/8\) on \([-1,1]\), and zero elsewhere. Its moments are \(\int L_2=9/4-(15/4)(1/3)=1\), \(\int uL_2=0\) by symmetry, and \(\int u^2L_2=(9/4)(1/3)-(15/4)(1/5)=0\).
For \(u^2>3/5\), this equivalent kernel is negative. The original kernel is nonnegative, but the polynomial correction need not be. Negative equivalent weights are therefore compatible with a valid local polynomial fit; they do not mean that the original kernel-weighted least-squares objective used negative weights.
Why the Equivalent Kernel Matters
The equivalent-kernel representation separates the local polynomial correction from the raw kernel weighting. Its moment identities explain why a degree-\(p\) fit reproduces polynomials up to that degree, while its higher moments describe how it responds to departures from those polynomials. It also makes clear that the equivalent kernel need not itself be a probability density: it can be negative, even when \(K\) is nonnegative.
A common mistake is to treat the equivalent kernel as simply the original \(K\), or to assume its weights must all be nonnegative. The formula \(L_p(u)=K(u)e_0^\mathsf{T}S^{-1}r_p(u)\) shows the missing ingredient: a polynomial correction determined by the moment matrix. For a local constant fit the correction has a particularly simple form, but for higher degrees it enforces additional zero moments and may change the sign of weights.
Check Your Understanding
Use the definitions and moment identities to answer the following.
- What condition on the finite-sample moment matrix is needed to derive the exact equivalent weights?
- For a degree-two fit, what are the required equivalent-kernel moments of orders zero, one, and two?
- Why does the local polynomial estimate reproduce a response that is a polynomial of degree at most \(p\)?
- In the asymmetric-kernel example, which correction makes the local linear equivalent kernel have zero first moment?
- Why does a negative value of a quadratic equivalent kernel not imply that the original weighted least-squares fit used negative kernel weights?