Why Rescale Distances Around the Target?
In local polynomial estimation, observations contribute according to how far their predictors are from the point \(x\) where the regression function is being estimated. The bandwidth \(h\) sets the distance scale: an observation \(h\) units from \(x\) is one bandwidth away, whether \(h\) is measured in years, millimetres, or another predictor unit.
The standardization \(u=(X-x)/h\) turns distances measured in the predictor’s original units into dimensionless distances. This makes it possible to describe the shape of the weighting scheme using a fixed function \(K(u)\), while bandwidth scaling accounts for how the neighborhood changes as \(h\) changes. Keeping these two roles separate is essential when calculating weighted sums, moments, and variance orders.
The factor \(h^{-1}\) in \(K_h\) is useful when treating the kernel as a rescaled density: if \(\int K(u)\,du=1\), then \(\int K_h(t)\,dt=1\). But it is important not to confuse this normalization with the raw weights used in local weighted least squares. Using \(K_h(X_i-x)\) instead of \(K((X_i-x)/h)\) multiplies every observation’s weight by the same factor \(h^{-1}\), so it leaves the weighted least-squares minimizer unchanged.
Kernel Moments and Squared-Kernel Integrals
A kernel moment records the integral of a power of standardized distance against the kernel. These integrals are fixed numbers determined by \(K\), not by \(x\) or \(h\). They are the constants that appear when a local calculation is expressed in standardized coordinates.
For a symmetric kernel, \(K(-u)=K(u)\), every existing odd moment is zero: the integrand \(u^jK(u)\) is odd when \(j\) is odd, and its integral over the real line is zero. The second moment and \(R(K)\) are different quantities: \(\mu_2(K)\) integrates \(u^2K(u)\), whereas \(R(K)\) integrates \(K(u)^2\). Both may matter, but they answer different questions.
Worked Example: Moments of the Uniform Kernel
Consider the uniform kernel \(K(u)=\frac12\) for \(|u|\leq 1\), and \(K(u)=0\) otherwise. Its support is \([-1,1]\), so an observation receives zero weight when \(|X_i-x|>h\).
The zeroth moment is \(\mu_0(K)=\int_{-1}^{1}\frac12\,du=\frac12(2)=1\). The first moment is \(\mu_1(K)=\int_{-1}^{1}\frac12u\,du=0\), because the integrand is odd.
The second moment and squared-kernel integral are \(\mu_2(K)=\frac12\int_{-1}^{1}u^2\,du=\frac12(2/3)=1/3\) and \(R(K)=\int_{-1}^{1}(1/2)^2\,du=(1/4)(2)=1/2\). The moment \(\mu_2(K)\) describes a weighted squared standardized distance; \(R(K)\) describes the integral that enters squared-weight calculations.
Worked Example: Gaussian Kernel Moments
For the standard Gaussian kernel \(K(u)=(2\pi)^{-1/2}\exp(-u^2/2)\), the zeroth moment is 1 because \(K\) is the \(N(0,1)\) density. Symmetry gives \(\mu_1(K)=0\), and the second moment is \(\mu_2(K)=1\).
To compute \(R(K)\), square the kernel and integrate: \(R(K)=\int (2\pi)^{-1}\exp(-u^2)\,du\). Since \(\int \exp(-u^2)\,du=\sqrt{\pi}\) over the real line, \(R(K)=\sqrt{\pi}/(2\pi)=1/(2\sqrt{\pi})\). Unlike the uniform kernel, the Gaussian kernel has no finite support. Its weights decay with distance rather than becoming exactly zero beyond a fixed number of bandwidths.
How Bandwidth Scaling Appears in Weighted Sums
Let \(X_1,\ldots,X_n\) be independent and identically distributed with density \(f\), and consider a fixed target point \(x\). The substitution \(u=(X_i-x)/h\) in an integral introduces a Jacobian factor \(h\). Consequently, the expectation of a raw kernel-weighted term is typically of order \(h\), not order 1. Summing over \(n\) observations gives order \(nh\).
Proof. A change of variables gives \(E[g((X_i-x)/h)]=\int g((t-x)/h)f(t)\,dt =h\int g(u)f(x+hu)\,du\). For each fixed \(u\), continuity of \(f\) at \(x\) implies \(f(x+hu)\to f(x)\). If \(\|f\|_\infty\) is a finite bound for the density, then \(|g(u)f(x+hu)|\leq\|f\|_\infty|g(u)|\), an integrable bound. The dominated convergence theorem therefore gives the stated expectation limit.
For the empirical sum, its normalized expectation has just been shown to converge to \(f(x)\int g\). Independence gives \(\operatorname{Var}(\frac{1}{nh}\sum_i g((X_i-x)/h)) =\frac{1}{nh^2}\operatorname{Var}(g((X_i-x)/h))\). Bounding variance by the second moment and changing variables again yields \(\operatorname{Var}(\frac{1}{nh}\sum_i g((X_i-x)/h)) \leq\frac{\|f\|_\infty}{nh}\int g(u)^2\,du\), which tends to zero when \(nh\to\infty\). Chebyshev’s inequality now gives convergence in probability to the limiting expectation. \(\square\)
Apply this result with \(g(u)=K(u)u^j\). If both \(\int |K(u)u^j|\,du\) and \(\int K(u)^2u^{2j}\,du\) are finite, then \[ \frac{1}{nh}\sum_{i=1}^n K((X_i-x)/h)((X_i-x)/h)^j \xrightarrow{p} f(x)\mu_j(K), \] provided \(f\) is bounded and continuous at \(x\), and \(nh\to\infty\). Thus the normalized sum converges in probability to \(f(x)\mu_j(K)\); when this limit is nonzero, the sum is asymptotically equivalent to \(nh f(x)\mu_j(K)\), and when it is zero, the sum is \(o_p(nh)\).
For a kernel supported on \([-A,A]\), only observations satisfying \(|X_i-x|\leq Ah\) can receive positive weight. If \(f\) is continuous at \(x\), the probability of this interval is approximately \(2Ah f(x)\), so the number of observations in it has expected size approximately \(2A nh f(x)\). This makes “effective neighborhood size of order \(nh\)” literal for compactly supported kernels. With a Gaussian kernel, there is no hard cutoff; any fixed standardized interval \(|(X_i-x)/h|\leq c\) still has probability approximately \(2ch f(x)\), while more distant observations contribute through decaying weights.
Worked Example: Effective Neighborhood Size at a Fixed Design Point
Suppose \(X_i\) has a density continuous at \(x\), with \(f(x)=2\) in the predictor’s units. Take \(n=1000\), \(h=0.1\), and the uniform kernel, which is supported on \([-1,1]\). The nonzero weights come from \([x-h,x+h]\), an interval of width \(0.2\).
For small bandwidth, the probability that one observation falls in this interval is approximately \(0.2f(x)=0.2(2)=0.4\). Thus the expected number of observations with nonzero weight is approximately \(1000(0.4)=400\). The same calculation in scale form is \(2nhf(x)=2(1000)(0.1)(2)=400\). This is a local approximation based on continuity of the density at \(x\); it is not an exact count for a realized sample.
Bandwidth Powers in Polynomial Terms and Noise
Write \(u_i=(X_i-x)/h\). Then \(X_i-x=hu_i\), so a polynomial term of degree \(r\) in the original distance becomes \(h^r u_i^r\). This identity explains the powers of bandwidth in local expansions without yet requiring a particular regression estimator or a particular bias formula.
Worked Example: Tracking a Polynomial Term Through a Weighted Sum
Suppose a response contribution has the form \(c_r(X_i-x)^r\), where \(c_r\) is fixed. Multiplying by the raw kernel weight gives \(K((X_i-x)/h)c_r(X_i-x)^r=c_rh^rK(u_i)u_i^r\).
Summing over observations and dividing by \(nh\), the resulting quantity is \(c_rh^r\frac{1}{nh}\sum_iK(u_i)u_i^r\). Under the conditions of the weighted-moment theorem, the factor after \(c_rh^r\) converges in probability to \(f(x)\mu_r(K)\). Thus the contribution carries a factor \(h^r\), multiplied by a quantity with a finite limit. The factor \(h^r\) comes from the change of distance scale; it does not come from the kernel moment itself.
Squared weights lead to a related but distinct bandwidth calculation. Consider \(S_n=\sum_{i=1}^n K((X_i-x)/h)\varepsilon_i\). Assume the pairs \((X_i,\varepsilon_i)\) are independent and identically distributed, \(X_i\) has a globally bounded density continuous at \(x\), \(E[\varepsilon_i\mid X_i]=0\), and \(E[\varepsilon_i^2\mid X_i]=\sigma^2\) for a constant \(\sigma^2>0\). Suppose also that \(0<R(K)=\int K(u)^2\,du<\infty\). These conditions ensure the mean is zero, distinct summands have zero covariance, and the design-density limit applies to \(K^2\).
Proof. Conditional mean zero implies \(E[S_n]=0\). Independence across observations makes the variance the sum of the individual variances, so \(\operatorname{Var}(S_n)=nE[K((X_i-x)/h)^2\varepsilon_i^2]\). Taking conditional expectation given \(X_i\) and using the constant conditional variance gives \(\operatorname{Var}(S_n)=n\sigma^2E[K((X_i-x)/h)^2]\). The change of variables \(u=(t-x)/h\), followed by dominated convergence using boundedness of \(f\) and integrability of \(K^2\), gives \(\frac1hE[K((X_i-x)/h)^2]\to f(x)R(K)\). Combining these equalities proves the limit after division by \(nh\). Because \(\sigma^2>0\), \(f(x)>0\), and \(R(K)>0\), the limiting coefficient is positive, which justifies the asymptotic-equivalence notation in the second statement. \(\square\)
Worked Example: Squared-Weight Variance Scale for a Gaussian Kernel
Suppose the design density is globally bounded and continuous at \(x\), with \(f(x)=2\), and the errors satisfy the theorem’s conditions with \(\sigma^2=4\). Use the Gaussian kernel and take \(n=1000\), \(h=0.1\). From the Gaussian calculation, \(R(K)=1/(2\sqrt{\pi})\).
The theorem gives the leading variance scale \(\operatorname{Var}(S_n)\sim nh\sigma^2f(x)R(K)\). Here \(nh=100\), so the coefficient is \(100(4)(2)/(2\sqrt{\pi})=400/\sqrt{\pi}\), approximately \(225.7\). This is an asymptotic approximation as \(h\) becomes small and \(nh\) grows, not an exact variance for the particular values \(n=1000\) and \(h=0.1\). The assumptions about the density matter even for the Gaussian kernel, whose weights extend beyond every fixed neighborhood.
Common Mistakes and Why the Scaling Matters
- Dropping the Jacobian factor. Substituting \(u=(t-x)/h\) gives \(dt=h\,du\). Omitting this factor incorrectly changes an expectation of a raw kernel weight from order \(h\) to order 1.
- Mixing up \(K_h\) and raw weights. The density-rescaled function \(K_h(t)=h^{-1}K(t/h)\) includes a factor not present in \(K((X_i-x)/h)\). The common factor cancels in a local weighted least-squares fit, but it changes the scale of a raw sum if left unaccounted for.
- Treating \(nh\) as a universal exact count. It is an order-of-magnitude description under local density conditions. Constants depend on \(f(x)\), the kernel, and how “effective neighborhood” is defined.
- Using a variance limit without design conditions. The limit for \(E[K^2((X_i-x)/h)]/h\) requires conditions such as continuity of \(f\) at \(x\), boundedness to justify domination for kernels with unbounded support, and finite positive \(R(K)\). Independence and an error variance alone do not establish it.
- Confusing a moment with a squared-kernel integral. \(\mu_2(K)\) and \(R(K)\) are different integrals, so one cannot substitute one for the other in a variance calculation.
These calculations provide the bookkeeping needed for later local polynomial results. A weighted design sum typically contributes a factor \(nh\); a term of degree \(r\) in distance contributes \(h^r\); and a sum of squared kernel-weighted errors has variance of order \(nh\) under the stated regularity conditions. Keeping the standardized coordinate, the Jacobian, and the square of the kernel distinct makes those powers traceable rather than memorized.
Check Your Understanding
Use the definitions and scaling arguments above to answer the following questions.
- For \(K(u)=\frac12\) on \([-1,1]\), calculate \(\mu_0(K)\), \(\mu_1(K)\), \(\mu_2(K)\), and \(R(K)\).
- Explain why the expected number of observations within \(Ah\) of \(x\) is approximately \(2Anh f(x)\) when \(f\) is continuous at \(x\).
- If a weighted sum contains a term proportional to \((X_i-x)^3\), what power of \(h\) appears after writing \(u_i=(X_i-x)/h\)?
- Why is continuity of \(f\) at \(x\) relevant to the limit of \(E[K^2((X_i-x)/h)]/h\)?
- State the additional assumptions that make the asymptotic-equivalence version of the squared-weight variance result valid.