Polynomial Degree and Local Smoothness

Why Degree and Smoothness Work Together

A local polynomial fit estimates a regression function near a target point by fitting a polynomial to nearby observations. Its degree determines which polynomial shapes it can reproduce exactly. The target function’s smoothness determines how closely it can be approximated by such a polynomial over a small neighborhood. Together, these facts determine the order of the fit’s approximation bias as the bandwidth \(h\) becomes small.

The key conclusion is an order bound: with a degree-\(p\) fit and local smoothness \(\beta\), the bias is typically at most of order \(h^{\min(\beta,p+1)}\), under suitable local-design and smoothness conditions. This is not automatically an exact bias formula. The actual bias can be smaller, or even zero, because of symmetry or other cancellations.

Local Polynomial Fit and Its Reproduction Property

Fix a target point \(x\). For observations at locations \(X_i\) within bandwidth \(h\), write \(u_i=(X_i-x)/h\). A degree-\(p\) local polynomial fit uses the regressors \(1,u_i,\ldots,u_i^p\) and minimizes a weighted sum of squared residuals. The fitted value at \(x\) is the fitted intercept.

For a precise bias statement, consider fixed design locations and nonnegative normalized weights \(q_i\), with \(\sum_i q_i=1\). Let \(r_p(u)=(1,u,\ldots,u^p)^\mathsf{T}\), and suppose the scaled locations with positive weight satisfy \(|u_i|\leq A\) for a fixed finite \(A\). Define the weighted design matrix

Assume \(G\) is invertible. With responses \(Y_i=f(X_i)+\varepsilon_i\), suppose \(E[\varepsilon_i\mid X_1,\ldots,X_n]=0\). The fitted coefficient vector is \(G^{-1}\sum_i q_i r_p(u_i)Y_i\), and the fitted value at \(x\) is its first component.

Proof. Let the response at each scaled location be \(g(u_i)=r_p(u_i)^\mathsf{T}a\), where \(a\) is its coefficient vector. Then

The fitted coefficient vector is therefore \(G^{-1}Ga=a\). Its first component is \(g(0)\), since the constant basis function is the first component of \(r_p(u)\). This proves exact reproduction. \(\square\)

Worked Calculations

From Smoothness to a Bias-Order Bound

A smoothness condition describes how well \(f\) can be approximated near \(x\). For noninteger \(\beta=m+\alpha\), where \(m\) is a nonnegative integer and \(0<\alpha<1\), a standard local Hölder condition is that \(f\) has \(m\) continuous derivatives near \(x\) and its \(m\)th derivative is locally \(\alpha\)-Hölder. This gives a Taylor polynomial through degree \(m\) with remainder bounded by a constant times \(|t-x|^\beta\). For integer \(\beta\), assume the derivatives through order \(\beta\) exist locally, are continuous as needed for Taylor’s theorem, and the derivative of order \(\beta\) is bounded; the Taylor polynomial through degree \(\beta-1\) then has remainder bounded by a constant times \(|t-x|^\beta\).

The following bound makes the role of degree explicit. It uses the design and weight assumptions already stated, including invertibility of \(G\). Let \(s=\min(\beta,p+1)\).

Proof. The fitted intercept is a linear function of the responses. Let \(e_0=(1,0,\ldots,0)^\mathsf{T}\), and let \(T(u)\) be a polynomial of degree at most \(p\) approximating the function \(u\mapsto f(x+hu)\). Polynomial reproduction shows that replacing the responses \(f(x+hu_i)\) by \(T(u_i)\) gives fitted value \(T(0)\).

By the smoothness condition, choose \(T\) so that, for \(|u|\leq A\),

for a constant \(C\) independent of small \(h\). If \(\beta\leq p+1\), use the Taylor polynomial through degree \(\lfloor\beta\rfloor\) for noninteger \(\beta\), or through degree \(\beta-1\) for integer \(\beta\); its remainder is \(O(h^\beta)\). If \(\beta>p+1\), use the Taylor polynomial through degree \(p\); the bounded derivative of order \(p+1\) gives remainder \(O(h^{p+1})\). In either case its degree is at most \(p\) and \(s=\min(\beta,p+1)\).

Write \(\delta_i=f(x+hu_i)-T(u_i)\). The difference between the fitted intercept for the true means and \(T(0)\) is

Because the inverses of the design matrices are uniformly bounded as \(h\to0\), \(\|r_p(u_i)\|\) is bounded for \(|u_i|\leq A\), and \(\sum_iq_i=1\), the absolute value of this difference is at most a constant times \(\max_i|\delta_i|\), hence at most a constant times \(h^s\). The conditional mean-zero error assumption makes the conditional expectation of the fitted value equal to the fit to the means. Therefore its bias relative to \(f(x)=T(0)\) is \(O(h^s)\). \(\square\)

What the Order Does—and Does Not—Say

The expression \(h^{\min(\beta,p+1)}\) is an upper-order description under the stated conditions, not a promise that the leading bias coefficient is nonzero. In the cubic-target example, the degree-two fit has a nonzero \(h^3\) bias. But a symmetric design can cancel particular terms, and if \(f\) itself is a polynomial of degree at most \(p\), polynomial reproduction gives zero bias exactly.

The degree and the smoothness therefore answer different questions. Degree \(p\) sets the highest polynomial order reproduced exactly and contributes the \(p+1\) ceiling in the generic remainder bound. Smoothness \(\beta\) limits how small the approximation remainder can be, even when the fitted polynomial has higher degree. Neither one alone determines an exact bias constant.

Common Mistake

• Wrong reasoning: “A cubic fit always has bias of order \(h^4\), because its degree is three.” This ignores the possibility that the target is less smooth than order four.
• Correction: The bound is \(O(h^{\min(\beta,p+1)})\). For a cubic fit and a \(5/2\)-smooth target, the bound is \(O(h^{5/2})\), as the worked example demonstrates.
• Another caution: An order bound does not assert that the bias is exactly that order. Polynomial reproduction or design symmetry may remove terms and make the bias smaller.