Undersmoothing and Bias Correction for Honest Bands

Why Smoothing Bias Matters for a Band

A regression confidence band is meant to cover the entire regression function, not merely its value at one chosen point. Smoothing makes estimation possible, but it also averages function values from nearby locations. If the function is curved, that averaging generally shifts the estimator away from the target. A band calibrated only for random error can therefore miss systematically, even when its standard errors are calculated correctly.

Two common ways to address this problem are undersmoothing, which reduces the bandwidth until the bias is negligible relative to the band width, and explicit bias correction, which modifies the estimator to cancel leading bias terms. Both can support honest coverage under suitable assumptions. Neither is free: undersmoothing increases variance, while correction typically adds variability and requires enough smoothness to control the remaining bias.

Setup: Stochastic Error and Smoothing Bias

Consider fixed design points $x_1,\ldots,x_n$, responses $Y_i=m(x_i)+\varepsilon_i$, and a target interval $\mathcal X$. Here $m$ is the regression function and $\varepsilon_i$ are mean-zero errors. For a bandwidth $h$, write a linear smoother as

\hat m_h(x)=\sum_{i=1}^n w_{i,h}(x)Y_i, \qquad \sum_{i=1}^n w_{i,h}(x)=1.

The weights depend on the design, $x$, and $h$. The estimator's bias is $b_h(x)=E[\hat m_h(x)]-m(x)$, while its variance is $\operatorname{Var}(\hat m_h(x))$. A simultaneous band has the form $\hat m_h(x)\pm r_h(x)$, for all $x\in\mathcal X$. Its radius $r_h(x)$ must account for the largest stochastic error over the interval; in many standard settings this introduces a factor of order $\sqrt{\log n}$ beyond a pointwise standard error.

Definition: A sequence of bands $C_n(x)$ is asymptotically honest over a function class $\mathcal M$ at level $1-\alpha$ if $$ \liminf_{n\to\infty}\inf_{m\in\mathcal M} P_m\{m(x)\in C_n(x)\text{ for every }x\in\mathcal X\}\ge 1-\alpha. $$ Honesty is a uniform coverage requirement. A band with the right coverage for one smooth function need not be honest over a larger class.

Undersmoothing: Make the Bias Small Enough

For a kernel-type estimator in a one-dimensional design with regular design density, suppose $m$ has smoothness $s$, and the leading bias is of order $h^s$. A typical pointwise standard error is of order $(nh)^{-1/2}$. A simultaneous stochastic radius is therefore of order $\sqrt{\log n/(nh)}$, up to constants and design factors. To make the bias negligible compared with that radius, require

\frac{h^s}{\sqrt{\log n/(nh)}}\longrightarrow 0, \qquad\text{equivalently}\qquad \frac{nh^{2s+1}}{\log n}\longrightarrow 0.

This condition explains the name: the bandwidth must be smaller than the scale at which bias and simultaneous stochastic error are comparable. The bandwidth must still be large enough for the estimator and its stochastic approximation to behave well; in particular, one usually also needs $nh$ to grow. The exact conditions depend on the design, kernel, error distribution, and method used to calibrate the supremum.

Worked Example: Choosing an Undersmoothed Bandwidth

Suppose the leading bias is $O(h^2)$, the simultaneous standard-error scale is $\sqrt{\log n/(nh)}$, and choose $h_n=(\log n/n)^{0.24}$. Check whether the bias is negligible relative to the band radius.

The bias-to-radius ratio has order $$ \frac{h_n^2}{\sqrt{\log n/(nh_n)}} =\sqrt{\frac{n}{\log n}}h_n^{5/2} =\sqrt{\frac{n}{\log n}}\left(\frac{\log n}{n}\right)^{0.60} =\left(\frac{\log n}{n}\right)^{0.10}. $$

This tends to zero. Also $nh_n=n^{0.76}(\log n)^{0.24}$ tends to infinity, so the choice does not make the local sample size vanish. By comparison, the balance $nh^5/\log n$ of constant order occurs at $h=(\log n/n)^{1/5}$; the chosen exponent $0.24$ gives a smaller bandwidth. Its standard-error scale is consequently larger, which is the price paid for making the bias negligible.

Result: Suppose a band is formed from a stochastic approximation whose simultaneous error is calibrated at level $1-\alpha$, and suppose its radius $r_n(x)$ satisfies $$ P_m\left\{\sup_{x\in\mathcal X} \frac{|\hat m_h(x)-E[\hat m_h(x)]|}{r_n(x)} \le 1+o(1)\right\}\ge 1-\alpha-o(1) $$ uniformly over the stated model class. If $\sup_{x\in\mathcal X}|b_h(x)|/r_n(x)\to0$ uniformly over that class, then the band with radius $r_n(x)$, with an asymptotically negligible inflation if needed, has asymptotic coverage at least $1-\alpha$.

Proof. For every $x$, the triangle inequality gives $$ \frac{|\hat m_h(x)-m(x)|}{r_n(x)} \le \frac{|\hat m_h(x)-E[\hat m_h(x)]|}{r_n(x)} +\frac{|b_h(x)|}{r_n(x)}. $$ On the event that the first term is at most $1+o(1)$ uniformly in $x$, the second term is uniformly $o(1)$ by assumption. Thus the entire estimation error is at most $1+o(1)$ times the radius simultaneously. Inflating the radius by a factor tending to one but exceeding this remainder gives the claimed coverage. Uniformity of both assumptions yields the infimum-over-$m$ coverage statement. $\square$

Explicit Bias Correction: Cancel Leading Moments

Instead of shrinking $h$, one can alter the weights so that the estimator reproduces low-order polynomials at the target. For an estimator at zero with weights $a_j$ at locations $x_j$, the moment conditions

\sum_j a_j=1,\qquad \sum_j a_jx_j=0,\qquad \sum_j a_jx_j^2=0

preserve constants and cancel the first- and second-order Taylor terms. The negative weights that may be needed are important: bias cancellation does not generally come from averaging with nonnegative weights. Taylor's theorem then identifies the leading remaining bias, while independence of the errors gives the variance as the sum of squared weights times their variances.

Worked Example: Deriving a Second-Order Bias Correction

Let $Y_t=m(t)+\varepsilon_t$ at locations $t=-2h,-h,0,h,2h$, with independent errors of common variance $\sigma^2$. Begin with the three-point estimator $T_h=(Y_{-h}+Y_0+Y_h)/3$. Define $$ \hat D_2=\frac{Y_{-2h}-2Y_0+Y_{2h}}{4h^2}, \qquad \tilde m_h(0)=T_h-\frac{h^2}{3}\hat D_2. $$ Expanding the subtraction gives $$ \tilde m_h(0)= \frac13Y_{-h}+\frac12Y_0+\frac13Y_h -\frac1{12}Y_{-2h}-\frac1{12}Y_{2h}. $$

The weights sum to $1/3+1/2+1/3-1/12-1/12=1$. Their first moment is zero by symmetry. Their second moment is $$ 2\left(\frac13\right)h^2 +2\left(-\frac1{12}\right)(2h)^2 =\frac23h^2-\frac23h^2=0. $$ Thus constant, linear, and quadratic terms are reproduced exactly at zero. If $m$ is four times continuously differentiable near zero, Taylor expansion through order four gives a fourth-order contribution $$ \frac{m^{(4)}(0)}{24} \left\{2\left(\frac13\right)h^4+ 2\left(-\frac1{12}\right)(2h)^4\right\} =-\frac{h^4}{12}m^{(4)}(0). $$ The odd-order terms cancel by symmetry, and the Taylor remainder is $o(h^4)$. Therefore $E[\tilde m_h(0)]-m(0)=-h^4m^{(4)}(0)/12+o(h^4)$.

Independence and equal error variances give $$ \operatorname{Var}(\tilde m_h(0)) =\sigma^2\left\{2\left(\frac13\right)^2+ \left(\frac12\right)^2+ 2\left(\frac1{12}\right)^2\right\} =\frac{35}{72}\sigma^2. $$ The original three-point estimator has variance $\sigma^2/3$, so this correction multiplies variance by $(35/72)/(1/3)=35/24$. The bias order improves from $h^2$ to $h^4$, but the noise is amplified.

This calculation is a local illustration, not a universal band construction. For a band, the correction must be applied across the target interval, and its covariance and supremum distribution must be calibrated. The remaining bias also needs a bound uniform over the function class. If the smoothness assumption only supports a second-order expansion, the fourth-order remainder claim above is not available.

Worked Example: Checking Coverage and Width in a Normal Model

Use the preceding five-point design with $m(t)=t^2$, $h=0.3$, and independent $N(0,0.2^2)$ errors. Compare nominal 95% normal intervals at zero for the uncorrected and corrected estimators. Take the critical value as $1.96$.

For $T_h$, the expectation is $\{h^2+0+h^2\}/3=2h^2/3=0.06$, so its bias relative to $m(0)=0$ is $0.06$. Its standard deviation is $0.2/\sqrt{3}\approx0.11547$, giving radius $1.96(0.11547)\approx0.2263$. Its actual coverage is $$ P(-0.2263\le T_h\le0.2263) =\Phi\left(\frac{0.2263-0.06}{0.11547}\right) -\Phi\left(\frac{-0.2263-0.06}{0.11547}\right) \approx \Phi(1.440)-\Phi(-2.479)\approx0.919. $$ Thus a nominal 95% interval undercovers in this example because it ignores the smoothing bias.

For $\tilde m_h(0)$, the quadratic bias is exactly zero because its weights have zero second moment. Its standard deviation is $0.2\sqrt{35/72}\approx0.13944$, and its 95% radius is approximately $1.96(0.13944)=0.2733$. The corrected estimate is exactly normal with mean zero, so this interval has coverage $0.95$. Its radius is larger, as expected from the variance calculation.

How the Two Strategies Trade Off

Undersmoothing avoids changing the estimator's weights but uses more conservative bandwidths. Explicit correction can retain a larger bandwidth and remove leading bias, but it generally increases variance and may make the estimator more sensitive to errors or boundary effects. Both approaches require a valid stochastic calibration for the band and a bias argument uniform over the target set.

Strategy	Bias control	Main cost or requirement
Undersmoothing	Choose $h$ so the uncorrected bias is negligible relative to the simultaneous radius	Smaller $h$ increases variance and band width
Explicit correction	Choose weights or a correction term that cancels leading bias moments	Variance may increase; a bound on the residual bias is still required

In either case, a bias bound can be used directly instead of forcing bias to be negligible. If $|b_h(x)|\le B_h(x)$, a stochastic radius $q\,s_h(x)$ can be enlarged to $q\,s_h(x)+B_h(x)$. On the event that the stochastic error is at most $q\,s_h(x)$ everywhere, the triangle inequality ensures simultaneous coverage with the enlarged radius. The bound must be justified over the class of functions under consideration; estimating or choosing a bound without accounting for its uncertainty does not by itself establish honesty.

The earlier material on self-similarity and bias bounds concerns assumptions that can make such uniform bias control possible. The practical choice between undersmoothing and correction is then a choice about how to use that control while maintaining adequate stochastic precision. If the function class permits substantial bias that cannot be bounded or removed, no choice of standard errors alone fixes the coverage problem.

Common Pitfall

Wrong reasoning: “The pointwise standard error is correct, so the confidence band covers the function.” This ignores both smoothing bias and the fact that simultaneous coverage concerns a supremum over $x$, not a single point.
Correction: Separate the centered stochastic error from $E[\hat m_h(x)]-m(x)$. Calibrate the stochastic error for the whole interval, then either make the bias negligible relative to the radius or include a valid uniform bias bound.
Another warning: Cancelling a leading Taylor term does not mean the corrected estimator is unbiased. It leaves a higher-order remainder, whose size depends on smoothness and must be controlled.

Key takeaway: Honest bands must control bias and stochastic error simultaneously. Undersmoothing controls bias by accepting a larger variance; explicit correction cancels leading bias by altering the estimator, usually at a variance cost. The chosen method is credible only when its remaining bias and its simultaneous stochastic fluctuations are both accounted for.

Check Your Understanding

Work through these questions by separating bias control from stochastic calibration.

For bias of order $h^s$ and simultaneous radius of order $\sqrt{\log n/(nh)}$, derive the condition required for bias to be negligible.
Why can a band with correct pointwise standard errors still fail to have simultaneous coverage?
For the corrected five-point estimator, verify that the weights sum to one and have zero second moment.
Explain why negative weights can reduce bias but increase variance.
If a uniform bias bound is $B_h(x)$, how can it be incorporated into a stochastic band radius?

Strategy	Bias control	Main cost or requirement
Undersmoothing	Choose \(h\) so the uncorrected bias is negligible relative to the simultaneous radius	Smaller \(h\) increases variance and band width
Explicit correction	Choose weights or a correction term that cancels leading bias moments	Variance may increase; a bound on the residual bias is still required