Integrated Risk and Boundary Contributions

From Pointwise Error to Integrated Risk

A pointwise mean squared error describes estimation accuracy at one fixed location. In regression, however, a fitted curve is usually used across a range of locations. Its integrated mean squared error (IMSE) measures average squared error over that range and makes the contribution of boundaries visible.

Consider independent observations \((X_i,Y_i)\), \(i=1,\ldots,n\), on the interval \([0,1]\), with \(Y_i=m(X_i)+\varepsilon_i\), \(E[\varepsilon_i\mid X_i]=0\), and \(\operatorname{Var}(\varepsilon_i\mid X_i)=\sigma^2(X_i)\). Let \(\hat m_h(x)\) be a local constant or local linear estimator using bandwidth \(h\). We use Lebesgue measure for integrated risk:

The design density \(f\) determines how observations are distributed, but this definition gives equal weight to every location in the interval. A risk integrated with respect to the design distribution instead has \(f(x)\,dx\) as its measure and generally has different constants.

Conditions and Boundary Expansions

For the expansions below, take a symmetric, nonnegative kernel \(K\), supported on \([-1,1]\), with \(\int K(u)\,du=1\), and write \(\mu_2=\int u^2K(u)\,du\) and \(R(K)=\int K(u)^2\,du\). Assume \(m\) has at least three continuous derivatives, \(f\) has at least two and is bounded away from zero, and \(\sigma^2\) is bounded and continuous. The usual local polynomial moment matrices are assumed nonsingular throughout the boundary layer. We consider \(h\to0\), \(nh^3\to\infty\), and estimators defined whenever the local fit exists; standard concentration conditions can make the probability of a missing local fit negligible.

These conditions give the familiar interior bias expansions. For local constant regression under random design, \(B_h(x)=h^2b_0(x)+o(h^2)\), where \(b_0(x)=\frac{\mu_2}{2}\{m''(x)+2m'(x)f'(x)/f(x)\}\). For local linear regression, \(B_h(x)=h^2b_1(x)+o(h^2)\), where \(b_1(x)=\frac{\mu_2}{2}m''(x)\). The density-gradient term in the local constant bias is not present in the leading local linear bias.

At a boundary, local constant regression has a different leading behavior. For \(x=ht\), \(0\leq t\leq1\), define \(\mu_j(t)=\int_{-t}^1u^jK(u)\,du\). To first order, the local constant weights use this truncated kernel. Expanding \(m(hu)\) and \(m(ht)\) about zero gives

The term of order \(h\) arises because the truncated weights do not have zero first moment about the target. For local linear regression, the fitted line reproduces constants and linear functions even with one-sided data. Its boundary bias is therefore \(O(h^2)\), under the stated smoothness and nonsingularity conditions. One-sided equivalent kernels can have negative weights, but their zeroth and first moments still reproduce those two polynomial terms.

How the Boundary Changes Integrated Squared Bias

A boundary layer has width \(h\). For local constant regression, its leading bias is of size \(h\) when the endpoint slope is nonzero. Squaring gives size \(h^2\), and integrating over a region of width \(h\) gives size \(h^3\). More precisely, with the symmetric kernel above, the leading boundary coefficient is

The integral in this coefficient is positive for a nondegenerate symmetric kernel with positive mass on the positive half of its support: at \(t=0\), the truncated first moment is positive. Thus a nonzero endpoint slope makes the leading \(h^3\) contribution positive. If both endpoint slopes vanish, this term disappears; one must then examine the next bias terms rather than assert that the squared bias is exactly of order \(h^3\).

For local linear regression, the boundary bias is \(O(h^2)\), so its squared contribution over a width-\(h\) strip is \(O(h^5)\). The interior squared bias is usually \(O(h^4)\) after integration over a region of order one. Consequently, when the interior leading bias function is not identically zero, it is the interior—not the boundary strip—that determines the leading integrated squared bias for local linear regression.

Integrated Variance and a Risk Decomposition

For either estimator, the variance has a leading contribution from the regression errors. In the interior, the local equivalent kernel is \(K\) for both local constant and local linear fits with a symmetric kernel. The leading conditional-noise variance is therefore \(\sigma^2(x)R(K)/(nhf(x))\). Integrating over the interval gives the constant

provided \(C_V>0\). The boundary strip does not change this leading integrated term: its pointwise variance is of order \(1/(nh)\), but the strip has width \(h\), so its integrated contribution is only of order \(1/n\). This argument assumes regular boundary moment matrices, which keep the boundary variance constant bounded.

The variance statement requires care when the conditional errors have little or no variance. For random design, the estimator can vary because the observed \(X_i\)'s vary, even if every \(\varepsilon_i=0\). The exact variance decomposition is \(\operatorname{Var}(\hat m_h(x))=E[\operatorname{Var}(\hat m_h(x)\mid X_1,\ldots,X_n)]+\operatorname{Var}(E[\hat m_h(x)\mid X_1,\ldots,X_n])\). The first term is the conditional-noise contribution above; the second is design variation and may remain when \(C_V=0\). In ordinary smooth settings its interior scale can be \(h/n\), rather than \(1/(nh)\). It cannot be discarded merely by writing down the noise-variance formula.

Two Integrated-Risk Regimes

Combining the expansions gives useful bandwidth balances, but only when their leading coefficients are positive. For local constant regression, if at least one endpoint slope is nonzero and \(C_V>0\), then

The balance \(h^3\asymp1/(nh)\) yields \(h\asymp n^{-1/4}\). This is a consequence of the first-order boundary bias, not a universal local constant rate. If both endpoint slopes vanish, the \(h^3\) coefficient is zero and the next nonzero integrated bias term must be identified before choosing a rate.

For local linear regression, if \(B_1=\int_0^1b_1(x)^2dx>0\) and \(C_V>0\), then

Balancing these terms gives \(h\asymp n^{-1/5}\). If \(b_1\) is identically zero, this balance is not justified: higher-order bias terms or other sources of error may determine the risk. Similarly, if \(C_V=0\), the remaining design-variation variance must be assessed before deriving a bandwidth rate.

Common Pitfall and Practical Interpretation

A common mistake is to say that local constant squared bias is “of order \(h^4\)” everywhere because its interior bias is of order \(h^2\). That ignores the boundary strip: when an endpoint slope is nonzero, local constant bias there is order \(h\), and its integrated square is order \(h^3\). The correction is to separate interior and boundary regions before integrating.

A second mistake is to treat \(C_V/(nh)\) as the full leading variance without checking \(C_V\). It is the leading conditional-noise term. When \(C_V>0\), it dominates the usual smaller design-variation term and supports the balances above. If it vanishes, the total variance need not vanish, nor need it have order \(1/(nh)\); a separate variance calculation is required.