Kernel Moment Conditions and Kernel Choice

Why Kernel Choice Is About More Than Shape

A kernel assigns relative weight to observations according to their distance from the target point. Its shape matters, but the consequences are expressed more precisely through its moments and its squared-kernel integral. Moments determine which terms appear in a local-polynomial bias expansion; the squared integral contributes to the leading variance. Support also matters computationally and at boundaries, although it does not by itself determine bias or variance.

A key distinction is between the raw kernel \(K\), used to weight observations in the local fit, and the equivalent kernel \(L\), which describes the resulting fit after the polynomial regression has adjusted those weights. For a local polynomial of degree greater than zero, it is generally the moments of \(L\), not the raw moments of \(K\) alone, that express the bias cancellations.

The moments have different roles. Normalization gives \(\mu_0(K)=1\); symmetry removes odd raw moments; and even moments such as \(\mu_2(K)\) and \(\mu_4(K)\) enter bias constants. The value \(R(K)\) controls the variance constant for a simple kernel smoother. For a local-polynomial fit, the corresponding variance constant is instead the squared integral of its equivalent kernel.

Four Common Symmetric Kernels

The table compares four normalized kernels. Their supports differ: the first three vanish outside \([-1,1]\), whereas the Gaussian kernel has unbounded support. For compactly supported kernels, the listed moments follow by integrating over \([-1,1]\); for the Gaussian, they are the familiar moments of \(N(0,1)\).

The numerical comparison assumes the displayed scaling. Rescaling a kernel changes its moments and squared integral, and can be offset by changing the bandwidth. Kernel comparisons are therefore meaningful only with the bandwidth convention made clear. In practice, the leading bias and variance constants of the estimator, not an isolated kernel number, are the relevant comparison.

Equivalent-Kernel Moments and Bias Cancellation

Consider estimating the \(\nu\)th derivative of a regression function at an interior point \(x\), using a local polynomial of degree \(p\), where \(0\leq\nu\leq p\). Define \(r_p(u)=(1,u,\ldots,u^p)^\mathsf{T}\), and suppose the matrix

is finite and nonsingular. The equivalent kernel for the derivative estimate is

where \(e_{\nu+1}\) is the coordinate vector with a one in position \(\nu+1\) and zeros elsewhere. With this convention, its leading population action on a smooth function is \(h^{-\nu}\int L_{p,\nu}(u)m(x+hu)\,du\).

Proof. The vector \(\int r_p(u)u^jK(u)\,du\) is column \(j+1\) of \(S_p\), because its \(k\)th entry is \(\int u^{k-1+j}K(u)\,du\), the \((k,j+1)\) entry of \(S_p\). It is therefore \(S_pe_{j+1}\). Substituting into the equivalent-kernel definition gives

The final inner product is one if \(j=\nu\) and zero otherwise, proving the claim. \(\square\)

These identities explain polynomial reproduction: all terms through degree \(p\) are handled exactly, with the \(\nu\)th derivative correctly scaled. They do not say that every bias term above degree \(p\) vanishes. The first such term depends on higher moments of \(L_{p,\nu}\), and symmetry can cause further cancellations.

Proof. Under the change \(u\mapsto -u\), each polynomial coordinate \(u^k\) is multiplied by \((-1)^k\). Since \(K\) is symmetric, the entries of \(S_p\) with opposite coordinate parity are zero: the corresponding integrand is an odd function. Thus \(S_p\), and its inverse, separate into even- and odd-power blocks. The linear combination \(e_{\nu+1}^\mathsf{T}S_p^{-1}r_p(u)\) contains only powers with the same parity as \(\nu\). Multiplication by the even function \(K(u)\) preserves that parity, so \(L_{p,\nu}(-u)=(-1)^\nu L_{p,\nu}(u)\). If \(j\) has the opposite parity, \(u^jL_{p,\nu}(u)\) is odd; its integral over a symmetric domain is zero. \(\square\)

How the Squared Integral Enters Variance

Under random design, suppose the observations \((X_i,Y_i)\) are independent and identically distributed, \(Y_i=m(X_i)+\varepsilon_i\), \(E[\varepsilon_i\mid X_i]=0\), and \(\operatorname{Var}(\varepsilon_i\mid X_i)=\sigma^2(X_i)\). Assume that the design density \(f\) is continuous and positive at the interior point \(x\), that \(\sigma^2\) is continuous at \(x\), and that the local polynomial fit is well-defined. With \(nh\to\infty\), the leading conditional variance, given the design points, is

The approximation follows from the equivalent weights: each observation contributes approximately \(L_{p,\nu}((X_i-x)/h)/(n f(x)h^{\nu+1})\). Squaring these weights and summing conditional error variances yields the displayed factor. Indeed, replacing the local sum by its density-weighted integral gives

where the last step uses \(t=x+hu\) and continuity at \(x\). This is a conditional variance calculation; it does not treat variation in the design points as if it were conditional error variance.

Bias Expansions and Support Conditions

For compactly supported \(L_{p,\nu}\), a Taylor expansion of \(m(x+hu)\) can be integrated over its support when \(m\) has the required derivatives in a neighborhood of \(x\). For example, if \(m\) has a continuous derivative of order \(p+1\) there, the remainder after its degree-\(p+1\) Taylor polynomial is \(o(h^{p+1}|u|^{p+1})\), uniformly for \(u\) in the bounded support. Combining this with the equivalent-kernel moments gives, when the displayed higher moment is nonzero,

If symmetry makes that moment zero, the next potentially nonzero term has the same parity as \(\nu\), and a higher-order smoothness expansion is needed to identify it. The formula should not be used to claim a nonzero leading term when its coefficient vanishes.

An unbounded-support equivalent kernel, such as a polynomial times a Gaussian, requires more care. A Taylor expansion valid uniformly only on bounded \(u\) cannot simply be integrated over the whole real line. One sufficient alternative is to assume the derivative of order \(p+1\) is bounded globally and that \(\int |u|^{p+1}|L_{p,\nu}(u)|\,du<\infty\). The normalized Taylor remainder is then bounded by a constant times \(|u|^{p+1}\), and tends to zero for each fixed \(u\); dominated convergence justifies integrating the remainder. Another route is to truncate the integral and separately bound the tail using explicit global growth and kernel-tail conditions. Without such conditions, a local smoothness assumption near \(x\) alone does not control contributions from arbitrarily large \(u\).

Choosing a Kernel and Avoiding a Common Mistake

Kernel choice affects constants, support, and practical computation. A compactly supported kernel uses only observations within a finite bandwidth radius and makes local Taylor arguments straightforward. A Gaussian kernel uses observations at every distance, with distant points receiving very small weights; its polynomial-Gaussian equivalent kernel also has unbounded support. At a boundary, symmetry around the target is lost for the observations available to the fit, so interior moment cancellations cannot be assumed unchanged.

For a fixed local-polynomial degree and suitable regularity, changing among standard kernels generally changes leading constants rather than the basic rate. A kernel with a smaller \(\mu_2(K)\) need not have a smaller risk: variance depends on the squared integral of the relevant equivalent kernel, while bias depends on the appropriate higher equivalent-kernel moments and derivatives of \(m\).

• Mistake: “The raw kernel has zero odd moments, so every odd bias term of the local-polynomial fit is zero.” Symmetry does imply parity cancellations, but the relevant moments are those of \(L_{p,\nu}\), and the fitted degree also determines which moments are reproduced.
• Correction: Derive or use the equivalent-kernel moment conditions for the chosen degree and derivative. Then check parity and the first higher moment that is actually nonzero.
• Mistake: “A Taylor expansion near \(x\) can be integrated against any kernel, including a Gaussian.” Local validity does not control an unbounded tail.
• Correction: Restrict the argument to compact support, or provide global remainder bounds and integrable tail conditions that justify the integration.

The practical comparison is therefore between full estimator constants under a specified bandwidth convention and set of assumptions. Raw kernel moments help explain those constants, but equivalent-kernel moments and squared integrals reveal how the local-polynomial procedure actually uses the observations.