Deriving the Interior Bias Expansion

Why the Interior Bias Depends on Degree and Symmetry

A local polynomial fit estimates a regression function near a target point by fitting a polynomial to nearby observations. Its degree determines which Taylor terms are reproduced exactly; the kernel moments determine how the first unreproduced terms affect the fitted intercept. At an interior point, symmetry can make that first contribution vanish, so the bias may be of higher order than a simple degree count suggests.

We derive the bias for a univariate regression function $m(x)=E[Y\mid X=x]$. The focus is the intercept estimate at an interior point $x$, not the variance or boundary behavior. We first use the population weighted least-squares fit, which isolates the deterministic bias calculation. We then state a direct condition under which the same moment calculations apply to sample weights.

Population Local Polynomial Fit

Definition: Let $K$ be a nonnegative, symmetric kernel supported on $[-1,1]$, let $h>0$, and write $r_p(u)=(1,u,\ldots,u^p)^T$. The population degree-$p$ local polynomial coefficient vector $\beta(h,x)$ minimizes $$ \int K(u)\left\{m(x+hu)-r_p(u)^T\beta\right\}^2 f(x+hu)\,du, $$ where $f$ is the design density. The population local polynomial estimate of $m(x)$ is the fitted intercept $\beta_0(h,x)=e_0^T\beta(h,x)$, where $e_0=(1,0,\ldots,0)^T$.

Assume $x$ is interior to the support of $X$, $f$ is positive and continuous at $x$, and the moment matrix $$ M_0=\int K(u)r_p(u)r_p(u)^T\,du $$ is nonsingular. These conditions make the limiting local polynomial fit well-defined. Write $\mu_j=\int u^jK(u)\,du$. For any integer $s\geq 0$, define $q_s$ as the vector whose $j$th entry, indexed from $j=0$ to $p$, is $\mu_{j+s}$. The coefficient $$ c_s=e_0^TM_0^{-1}q_s $$ is the intercept produced by projecting $u^s$ onto polynomials of degree at most $p$ under the kernel-weighted inner product.

Polynomial reproduction: If $m(x+hu)$ is a polynomial in $u$ of degree at most $p$, the population fit reproduces it exactly. In particular, its fitted intercept is $m(x)$.

To verify this, if $m(x+hu)=r_p(u)^T\gamma$ for some vector $\gamma$, then choosing $\beta=\gamma$ makes the integrated squared error zero. Nonsingularity of the moment matrix makes that minimizer unique. The constant coefficient of the polynomial in $u$ is $m(x)$, proving the claim.

Worked Example: Computing the Degree-One Bias Coefficient

Take the uniform kernel $K(u)=1/2$ on $[-1,1]$, and use a degree-one fit. The odd kernel moments are zero, while $\mu_0=1$ and $\mu_2=1/3$. Thus $$ M_0= \begin{pmatrix} 1&0\\ 0&1/3 \end{pmatrix}, \qquad q_2= \begin{pmatrix} \mu_2\\ \mu_3 \end{pmatrix} = \begin{pmatrix} 1/3\\ 0 \end{pmatrix}. $$

Since $M_0^{-1}$ has first row $(1,0)$, $c_2=e_0^TM_0^{-1}q_2=1/3$. Taylor expansion gives the quadratic term $m''(x)h^2u^2/2$. Its contribution to the fitted intercept is therefore $$ \frac{m''(x)}{2}h^2c_2=\frac{m''(x)}{6}h^2. $$ The degree-one fit reproduces the constant and linear terms; the displayed term is its leading bias when $m''(x)$ is nonzero.

The General Interior Expansion

Theorem: Suppose $K$ is symmetric, supported on $[-1,1]$, and has finite moments through order $2p+2$. Suppose $M_0$ is nonsingular, $f$ is continuous and positive at the interior point $x$, and $m$ has $p+1$ continuous derivatives in a neighborhood of $x$. Then, as $h\to0$, $$ \beta_0(h,x)-m(x) = \frac{m^{(p+1)}(x)}{(p+1)!}\,c_{p+1}h^{p+1} +o(h^{p+1}). $$ If $p$ is even, symmetry implies $c_{p+1}=0$, so this expansion states that the bias is $o(h^{p+1})$; it does not by itself specify the next nonzero term.

Proof. The population normal equations have matrix and right-hand side $$ M(h)=\int K(u)r_p(u)r_p(u)^Tf(x+hu)\,du, \qquad b(h)=\int K(u)r_p(u)m(x+hu)f(x+hu)\,du. $$ Continuity of $f$ at $x$, compact support of $K$, and boundedness of the integrands on the integration region imply $M(h)\to f(x)M_0$. The limit is nonsingular, so $M(h)^{-1}\to f(x)^{-1}M_0^{-1}$.

Taylor expansion uniformly for $|u|\leq1$ gives $$ m(x+hu)=\sum_{s=0}^{p}\frac{m^{(s)}(x)}{s!}h^su^s +\frac{m^{(p+1)}(x)}{(p+1)!}h^{p+1}u^{p+1} +o(h^{p+1}). $$ The first sum is a polynomial of degree at most $p$, so its fit reproduces its value at $u=0$, namely $m(x)$. The remaining Taylor term has right-hand side, after division by $h^{p+1}$, tending to $f(x)m^{(p+1)}(x)q_{p+1}/(p+1)!$. Multiplication by $M(h)^{-1}$ and extraction of the intercept yield the stated coefficient. The Taylor remainder contributes $o(h^{p+1})$: its absolute value is uniformly $o(h^{p+1})$, and the kernel integrals and inverse matrices remain bounded after their common factor $f(x)$ is accounted for.

For even $p$, the integer $p+1$ is odd. Symmetry makes $M_0$ block diagonal between even and odd powers: its $(j,k)$ entry is zero when $j+k$ is odd. The first row of $M_0^{-1}$ therefore has zero entries at odd indices. For every even index $j$, the corresponding entry of $q_{p+1}$ is $\mu_{j+p+1}=0$, because $j+p+1$ is odd. Hence $c_{p+1}=0$. $\square$

When an Even Degree Raises the Bias Order

For even $p$, a sharper expansion requires one more derivative of $m$ and information about how the design density changes near $x$. Define matrices $M_1$ and vectors $q_{s,1}$ by replacing every moment $\mu_j$ in $M_0$ and $q_s$ with $\mu_{j+1}$. In particular, the $(j,k)$ entry of $M_1$ is $\mu_{j+k+1}$, and the $j$th entry of $q_{s,1}$ is $\mu_{j+s+1}$. Let $f_0=f(x)$ and $f_1=f'(x)$.

Theorem: If $p$ is even, $m$ has $p+2$ continuous derivatives near $x$, $f$ has a continuous first derivative near $x$, and the preceding kernel and nonsingularity assumptions hold, then $$ \beta_0(h,x)-m(x) = h^{p+2}\left\{ \frac{m^{(p+2)}(x)}{(p+2)!}c_{p+2} + \frac{m^{(p+1)}(x)}{(p+1)!}\frac{f_1}{f_0}d_{p+1} \right\} +o(h^{p+2}), $$ where $$ d_s=e_0^TM_0^{-1}\left(q_{s,1}-M_1M_0^{-1}q_s\right). $$

Derivation. Taylor expansion of the density gives $M(h)=f_0M_0+hf_1M_1+o(h)$. For the Taylor monomial $u^s$, its right-hand-side moment vector is $f_0q_s+hf_1q_{s,1}+o(h)$. Factoring out $f_0$ and expanding the inverse matrix gives $$ M(h)^{-1} = \frac{1}{f_0}\left\{ M_0^{-1} -h\frac{f_1}{f_0}M_0^{-1}M_1M_0^{-1} +o(h) \right\}. $$ Consequently, the fitted intercept for the monomial $u^s$ is $$ e_0^TM(h)^{-1}\{f_0q_s+hf_1q_{s,1}+o(h)\} =c_s+h\frac{f_1}{f_0}d_s+o(h). $$ Apply this to the Taylor terms of orders $p+1$ and $p+2$. Since $p$ is even, $c_{p+1}=0$, as shown above. The order-$p+1$ Taylor term therefore contributes at order $h^{p+2}$ through the first density correction; the order-$p+2$ term contributes its coefficient $c_{p+2}$. The Taylor remainder is $o(h^{p+2})$, and the displayed formula follows. $\square$

Worked Example: Local Constant Bias with a Changing Design Density

Let $p=0$ and use the uniform kernel $K(u)=1/2$ on $[-1,1]$. Then $M_0=(1)$, $\mu_2=1/3$, and $c_2=\mu_2=1/3$. Also $M_1=(\mu_1)=(0)$, $q_1=(\mu_1)=(0)$, and $q_{1,1}=(\mu_2)=(1/3)$, so $d_1=1/3$. The even-degree expansion becomes $$ \beta_0(h,x)-m(x) = h^2\left\{\frac{m''(x)}{6} +\frac{m'(x)}{3}\frac{f'(x)}{f(x)}\right\} +o(h^2). $$

For a numerical check, take $x=0$, $m(t)=t+t^2$, and density $f(t)=e^t/(e-e^{-1})$ on $[-1,1]$. The point $0$ is interior; $m'(0)=1$, $m''(0)=2$, and $f'(0)/f(0)=1$. The curvature contribution is $2/6=1/3$, and the density contribution is $1/3$. Thus the leading bias is $2h^2/3$, with remainder $o(h^2)$. Omitting the design-density term would give the wrong leading coefficient.

Worked Example: Degree Two and the Vanishing Cubic Term

Use the same uniform kernel and fit a degree-two polynomial with constant design density. For $p=2$, the order-$h^3$ coefficient is zero by symmetry. To calculate the next coefficient, consider the even-power block for projecting $u^4$ onto $1$ and $u^2$: $$ \begin{pmatrix} 1&1/3\\ 1/3&1/5 \end{pmatrix} \begin{pmatrix}a\\b\end{pmatrix} = \begin{pmatrix}1/5\\1/7\end{pmatrix}. $$ The determinant is $1/5-1/9=4/45$. Therefore $$ a= \frac{(1/5)(1/5)-(1/3)(1/7)}{4/45} = \frac{1/25-1/21}{4/45} = \frac{-4/525}{4/45} =-\frac{3}{35}. $$ Thus $c_4=-3/35$. At $x=0$, let $m(t)=t^4$. Then $m^{(4)}(0)=24$, and the leading bias is $h^4(24/24)c_4=-3h^4/35$. This verifies both the vanished cubic contribution and the next nonzero term for this example.

Interpreting the Expansion for Sample Fits

For observations $(X_i,Y_i)$, define $t_i=(X_i-x)/h$, $r_p(t_i)=(1,t_i,\ldots,t_i^p)^T$, and $$ \Gamma_n=\frac{1}{nh}\sum_{i=1}^n K(t_i)r_p(t_i)r_p(t_i)^T. $$ When $\Gamma_n$ is invertible, the sample local polynomial estimate is $e_0^T\Gamma_n^{-1}(nh)^{-1}\sum_i K(t_i)r_p(t_i)Y_i$. It is linear in the responses, with weights $$ W_i(x)=\frac{1}{nh}K(t_i)e_0^T\Gamma_n^{-1}r_p(t_i). $$ The normal equations imply exact sample reproduction: $\sum_iW_i(x)t_i^j=1$ for $j=0$ and $0$ for $1\leq j\leq p$.

For a sample-level bias expansion, the relevant condition is on the weighted moments themselves. For example, a degree-$p$ leading-term calculation requires $\sum_iW_i(x)t_i^{p+1}$ to converge to the appropriate intercept moment $c_{p+1}$, together with a bounded weighted Taylor remainder. One must not assume that an individual sample weight, after multiplying by $h$, is itself an equivalent-kernel value: under random design, the weight scale also depends on the sample size and design density. For even $p$, obtaining the sharper order-$h^{p+2}$ formula requires control of the order-$h$ moment correction as well; convergence to a limit alone does not guarantee that rate.

Common Mistake and Practical Importance

A tempting shortcut is: “A degree-$p$ fit reproduces terms through degree $p$, so its bias must always be order $h^{p+1}$.” The first part is correct, but the conclusion can fail. It identifies the first Taylor term not reproduced exactly, not whether that term changes the fitted intercept. With a symmetric kernel and even $p$, the intercept moment for that odd-power term is zero. The bias is then smaller, and its next coefficient can depend on the derivative of the design density.

This distinction matters when comparing degrees, describing bias orders, or reasoning about bandwidth choices. State the coefficient and the conditions under which it is nonzero rather than reporting an order based on polynomial degree alone. Also distinguish the population expansion from a finite-sample conditional bias expansion: random empirical moments can leave a nonzero contribution even when the corresponding population moment vanishes.

Key takeaway: Taylor expansion identifies the first unreproduced regression term; kernel moment equations determine its intercept contribution. Symmetry makes the order-$h^{p+1}$ contribution vanish for even $p$, and the next-order bias can involve both regression curvature and the local design-density slope.

Check Your Understanding

Use the moment definitions and parity argument above to answer these questions.

Why does a degree-$p$ local polynomial fit reproduce every polynomial in the scaled coordinate $u$ of degree at most $p$?
For a symmetric kernel, why is $c_{p+1}=0$ when $p$ is even?
For a degree-one fit, which kernel moment determines the leading interior bias coefficient?
In the local constant example, what term would be omitted by treating the design density as constant when $f'(x)\ne0$?
Why does convergence of sample weighted moments to their limits not, by itself, establish the order-$h^{p+2}$ sample bias expansion for even $p$?