Plug-In Bandwidth Selection for Local Polynomials

Why a Plug-In Bandwidth Needs Pilot Fits

A local polynomial fit uses a bandwidth to balance two opposing effects: a small bandwidth reduces smoothing bias but uses fewer observations, increasing variance. The pointwise MSE-optimal bandwidth depends on quantities that are usually unknown, including the regression curve's curvature and the conditional noise variance. A plug-in bandwidth replaces those unknown quantities by pilot estimates and substitutes them into the MSE-optimal formula.

To keep the bias coefficient precise, this tutorial derives the formula for an interior local linear fit with a symmetric kernel. Higher-degree fits have their own leading bias coefficients. In random-design regression, those coefficients must account for the design density and its derivatives when they contribute at the relevant order; they cannot always be obtained from a regression derivative alone.

Pointwise Bias, Variance, and the Oracle Bandwidth

Write \(u_i=(X_i-x)/h\). The local linear fit at \(x\) minimizes \(\sum_{i=1}^n K(u_i)\{Y_i-a-bu_i\}^2\) over \(a,b\); its estimate of \(m(x)\) is the fitted intercept \(\hat m_h(x)=\hat a\). Define \(R(K)=\int K(u)^2\,du\). At an interior point, the leading bias and variance are

The local-linear intercept reproduces an exactly linear regression curve, regardless of the local distribution of \(X\). For a curved regression, the leading intercept bias is the local quadratic term after projection onto the constant and linear terms. In the limit, symmetry gives zero first kernel moment, so the coefficient is \(\mu_2(K)m''(x)/2\). Variation in the design density does not add a term of order \(h^2\) to this local-linear bias. This cancellation should not be assumed for other degrees or for a local-constant fit.

The variance coefficient has two components: \(\sigma^2(x)\), the local noise level, and \(1/f(x)\), which reflects how many observations are available near \(x\). The kernel contributes \(R(K)\). Squaring the leading bias and adding the leading variance gives the pointwise MSE approximation

To verify the minimization, differentiate the approximating MSE with respect to \(h>0\). Its derivative is \(4B(x)^2h^3-V(x)/(nh^2)\). Setting this to zero and multiplying by \(h^2\) gives \(4B(x)^2h^5=V(x)/n\), yielding the stated expression. The second derivative is \(12B(x)^2h^2+2V(x)/(nh^3)>0\), so this stationary point is the unique minimum. The two terms balance there: \(V(x)/(nh_{\mathrm{or}})=4B(x)^2h_{\mathrm{or}}^4\). \(\square\)

Building the Plug-In Estimate

For this local-linear setup, estimate the regression curvature, density, and conditional variance at \(x\). Denote the resulting estimates by \(\widehat{m''}(x)\), \(\hat f(x)\), and \(\hat\sigma^2(x)\). Form

A standard curvature pilot is a local quadratic fit with its own bandwidth \(b\). In the coordinate \(v=(X_i-x)/b\), fit a quadratic by weighted least squares and write its fitted polynomial as \(\hat\beta_0+\hat\beta_1v+\hat\beta_2v^2\). The curvature estimate is \(2\hat\beta_2/b^2\); the division by \(b^2\) is essential because the polynomial was fitted in a rescaled coordinate. Under sufficient smoothness, positive design density, and regular kernel and moment conditions, a typical consistency regime is \(b\to0\) and \(n b^5\to\infty\). The first condition controls approximation error, while the second allows the derivative estimate's variance to vanish.

The density can be estimated by a kernel density pilot with bandwidth \(a\), for example \(\hat f(x)=(n_fa)^{-1}\sum_{i=1}^{n_f}J((X_i-x)/a)\), where \(J\) integrates to one. Consistency requires \(a\to0\) and \(n_fa\to\infty\), along with continuity of \(f\) at \(x\). For the conditional variance, one may smooth squared residuals \(\{Y_i-\hat m_{\mathrm{pilot}}(X_i)\}^2\) near \(x\). A clean implementation can use an independent pilot sample for \(\hat m_{\mathrm{pilot}}\) and a separate sample for residual smoothing; with consistent regression pilots, a consistent local average of those squared residuals estimates \(\sigma^2(x)\). Finite conditional fourth moments in a neighborhood are a usual sufficient moment condition for this variance estimation.

Indeed, the ratio is \(\{\hat V(x)/V(x)\}^{1/5}\{B(x)^2/\hat B(x)^2\}^{1/5}\). Consistency gives \(\hat V(x)/V(x)\xrightarrow{p}1\) and \(\hat B(x)/B(x)\xrightarrow{p}1\). The map taking positive ratios to their one-fifth powers is continuous at one, so the product converges in probability to one by the continuous mapping theorem. This proves the result. \(\square\)

The result is a relative-consistency statement: if the pilots consistently estimate the two coefficients and the leading bias is nonzero, their plug-in bandwidth approaches the oracle bandwidth proportionally. It does not promise that a particular finite-sample pilot is accurate. In particular, squaring an imprecise curvature estimate can make the selected bandwidth highly unstable.

Choosing Pilots and Interpreting the Result

The final bandwidth and the pilot bandwidths solve different problems. The final \(h\) smooths the regression estimate at the desired point. The derivative pilot \(b\) estimates curvature, and derivative estimation is especially sensitive to noise: its variance typically has order \(1/(n b^5)\), rather than the order \(1/(n b)\) associated with estimating a function value. If \(b\) is too small, the curvature estimate is noisy; if it is too large, local polynomial approximation error can dominate. Density and variance pilots also have their own bandwidths and consistency requirements. A single numerical bandwidth should not automatically be reused for every pilot task.

The plug-in formula applies when the displayed \(h^2\) bias is genuinely the leading bias. If \(m''(x)=0\), then \(B(x)=0\) and the formula divides by a quantity tending to zero; the next nonzero bias term may determine a different rate and requires its own valid expansion. Likewise, a curvature estimate near zero can produce an extremely large or erratic bandwidth. In practice, report the pilot estimates alongside the selected bandwidth and inspect whether the leading-bias assumptions are plausible.

• Mistake: Use a local-constant bias formula for a local-linear fit, or assume a higher-degree bias coefficient is always just a regression derivative times a kernel moment. Correction: Derive the coefficient for the estimator and design setting being used; density-derivative terms can enter random-design expansions.
• Mistake: Use the local-quadratic coefficient on the rescaled coordinate directly as \(m''(x)\). Correction: If the coordinate is \(v=(X-x)/b\), multiply the fitted quadratic coefficient by \(2/b^2\).
• Mistake: Treat the plug-in value as universally optimal or stable. Correction: It estimates a pointwise leading-MSE optimum and depends on consistent pilots and a nonzero leading bias coefficient.