Honesty and Width of Nonparametric Confidence Bands

Why Coverage and Width Must Be Described Separately

A confidence band is a random range of plausible values for an entire regression function. Its reliability depends on where coverage is required: at one specified input, or at every input in a domain at once. Its size is a separate feature. A very wide band can cover reliably but convey little information, while a narrow band can be informative when it covers and fail too often.

These distinctions matter especially in nonparametric regression, where the target is a function rather than a single parameter. A statement that intervals have 95% coverage at each fixed input does not, by itself, say that the whole function lies inside the collection of intervals with probability 95%. The distinction can be made precisely through coverage events and width measures.

Regression Bands and Coverage

Let $T$ be a compact regression domain, such as $[0,1]$, and let $f:T\to\mathbb{R}$ be the unknown regression function. The data have distribution $P_f$ when the regression function is $f$. A band is a pair of data-dependent endpoint functions $L_n(x)$ and $U_n(x)$, with $L_n(x)\le U_n(x)$, defining the interval $[L_n(x),U_n(x)]$ at each $x\in T$. Assume the target and band endpoints have continuous paths, so the simultaneous coverage event is measurable. The probability $P_f$ includes any randomness used by the procedure.

Definition: At a fixed input $x$, the band has pointwise coverage $1-\alpha$ over a function class $\mathcal{F}$ if $$ \inf_{f\in\mathcal{F}} P_f\{L_n(x)\le f(x)\le U_n(x)\}\ge 1-\alpha. $$ It has simultaneous coverage $1-\alpha$ over $T$ if $$ \inf_{f\in\mathcal{F}} P_f\{L_n(x)\le f(x)\le U_n(x)\text{ for every }x\in T\}\ge 1-\alpha. $$ The infimum over $f$ requires the guarantee to hold uniformly across the stated class.

For a sequence of bands, asymptotic uniform honesty at level $1-\alpha$ means

\liminf_{n\to\infty}\ \inf_{f\in\mathcal{F}} P_f\{L_n(x)\le f(x)\le U_n(x)\text{ for every }x\in T\} \ge 1-\alpha.

This is an asymptotic lower bound on the worst-case simultaneous coverage; it does not assert that every finite-sample coverage probability is at least $1-\alpha$. The analogous pointwise honesty condition replaces the event involving every $x$ with the event at a specified $x$. When the function class is understood, the class is sometimes omitted from the wording, but it is essential to the guarantee.

Worked Example: A Constant Regression Function

Suppose $T=[0,1]$, $f(x)=\theta$ for every $x$, and observations satisfy $Y_i=\theta+\varepsilon_i$, where the $\varepsilon_i$ are independent $N(0,\sigma^2)$ variables and $\sigma$ is known. Then $\bar Y\sim N(\theta,\sigma^2/n)$. Set $z_{0.975}=1.959964$ and use the band $$ L_n(x)=\bar Y-z_{0.975}\frac{\sigma}{\sqrt n}, \qquad U_n(x)=\bar Y+z_{0.975}\frac{\sigma}{\sqrt n}, \quad x\in[0,1]. $$

For any one fixed $x$, coverage is $P_\theta\{|\bar Y-\theta|\le z_{0.975}\sigma/\sqrt n\} =P\{|Z|\le1.959964\}=0.95$, where $Z\sim N(0,1)$. Because both the target and band are constant in $x$, the event that the band covers at one point is exactly the event that it covers the entire domain. Its simultaneous coverage is therefore also 0.95.

For $n=16$ and $\sigma=2$, the standard error is $2/\sqrt{16}=0.5$. The half-width is $1.959964(0.5)=0.979982$, so the width is $1.959964$, approximately $1.96$. Here pointwise and simultaneous coverage agree because of the constant-function structure; that equality is not a general rule.

What Uniform Honesty Guarantees

Simultaneous coverage is a stronger event than coverage at any one point. The resulting implication holds with no independence assumption and for any function class.

Theorem: If a band has simultaneous coverage at least $1-\alpha$ over $\mathcal{F}$, then it has pointwise coverage at least $1-\alpha$ at every fixed $x\in T$, over the same class. The same implication holds for asymptotic uniform honesty.

Proof. For each $f$ and fixed $x$, let $A_f$ be the event that the band covers $f$ at every point of $T$, and let $B_{f,x}$ be the event that it covers $f(x)$. By definition, $A_f\subseteq B_{f,x}$. Thus $P_f(B_{f,x})\ge P_f(A_f)$. Taking the infimum over $f\in\mathcal{F}$ preserves the inequality, establishing the finite-sample claim. For a sequence of bands, the same inequality holds at each $n$; taking the lower limit in $n$ gives the asymptotic claim. $\square$

The converse fails: good coverage at every fixed point need not give good coverage over the whole domain. As the next example shows, even exact 95% coverage at each of two points can yield less than 95% probability of covering both.

Worked Example: Pointwise Intervals That Miss Simultaneously

Let the domain consist of two points, $T=\{x_1,x_2\}$, with unrestricted target values $f(x_1)=\theta_1$ and $f(x_2)=\theta_2$. Observe $Y_j=\theta_j+\varepsilon_j$, where $\varepsilon_1,\varepsilon_2$ are independent $N(0,1)$ variables. At each point use the interval $[Y_j-1.959964,Y_j+1.959964]$.

At either point, the probability of coverage is $P(|Z|\le1.959964)=0.95$. The two coverage events are independent, since they depend on independent errors. Consequently, the probability of covering both target values is $0.95\times0.95=0.9025$, not 0.95.

To guarantee at least 0.95 simultaneous coverage, use at each point the wider interval $[Y_j-2.2414,Y_j+2.2414]$. The standard normal probability within $2.2414$ standard deviations is approximately $0.975$, so the exact simultaneous coverage here is approximately $0.975^2=0.950625$. Each interval has width $2(2.2414)=4.4828$. This adjustment illustrates the coverage-versus-width trade-off; it is not required to be the narrowest possible simultaneous band.

Band Width: Pointwise and Global Measures

The width at $x$ is the random quantity $w_n(x)=U_n(x)-L_n(x)$. Its expectation $E_f[w_n(x)]$ describes the average width at that particular input under regression function $f$. For a band intended to cover the whole domain, a useful global measure is the maximal width $$ W_n=\sup_{x\in T}\{U_n(x)-L_n(x)\}. $$

The expected global width under $f$ is $E_f[W_n]$. For a class-wide summary, one can take $\sup_{f\in\mathcal{F}}E_f[W_n]$, the worst-case expected maximal width. This differs from the width at one selected $x$: a band may be narrow in most of the domain and wide in a small region. Also, in general, $E_f[\sup_x w_n(x)]$ and $\sup_x E_f[w_n(x)]$ are not the same quantity; the former is at least the latter.

Coverage and width answer different questions. Coverage is a probability that the target is contained in the band; width measures the range the band reports. Neither alone captures both reliability and informativeness. For example, the band $(-\infty,\infty)$ has coverage one but infinite width, while a narrow band may have low coverage.

Worked Example: Random Width and Two Coverage Probabilities

Again take $T=\{x_1,x_2\}$, with independent observations $Y_j=\theta_j+\varepsilon_j$, where $\varepsilon_j\sim N(0,1)$. Independently of both errors, draw a fair coin. Set a common radius $R=1$ for heads and $R=2$ for tails, and use the band $[Y_j-R,Y_j+R]$ at both points. Its global width is $W=2R$, so $E[W]=0.5(2)+0.5(4)=3$.

Write $p(r)=P(|Z|\le r)=2\Phi(r)-1$, where $\Phi$ is the standard normal distribution function. At either specified point, the coverage probability averages over the coin: $0.5p(1)+0.5p(2)$. Using $p(1)\approx0.682689$ and $p(2)\approx0.954500$, this is approximately $0.818595$.

Conditional on $R=r$, the two errors are independent, so the probability of simultaneous coverage is $p(r)^2$. Averaging over the coin gives $0.5(0.682689^2)+0.5(0.954500^2)\approx0.688567$. Thus this band has expected global width 3, pointwise coverage about 0.819, and simultaneous coverage about 0.689. The expected width alone does not determine either coverage probability.

A Useful Coverage Bound

For a finite set of inputs, the union bound gives a direct sufficient condition for simultaneous coverage. No independence among the pointwise coverage events is needed.

Theorem: Suppose $T=\{x_1,\ldots,x_m\}$, and for every $f\in\mathcal{F}$ and $j$, the probability of missing $f(x_j)$ at $x_j$ is at most $\alpha/m$. Then the band covers $f$ simultaneously at all $m$ inputs with probability at least $1-\alpha$, uniformly over $\mathcal{F}$.

Proof. For each $j$, let $M_j=\{L_n(x_j)>f(x_j)\text{ or }U_n(x_j)<f(x_j)\}$ be the miss event. Failure of simultaneous coverage is $\bigcup_{j=1}^m M_j$. The union bound yields $$ P_f\left(\bigcup_{j=1}^m M_j\right) \le \sum_{j=1}^m P_f(M_j) \le m\frac{\alpha}{m}=\alpha. $$ Taking complements gives simultaneous coverage at least $1-\alpha$ for each $f$, hence also after taking the infimum over $\mathcal{F}$. $\square$

For a continuous domain, one cannot apply this finite union bound directly to infinitely many inputs. A valid argument then needs additional structure that controls coverage across the domain. The key point here is conceptual: a collection of marginal error probabilities is not itself a simultaneous guarantee.

Common Mistake

Wrong reasoning: “The interval has 95% coverage at every $x$, so the band covers the whole function with probability 95%.” This treats the separate pointwise events as if they were one event. In the independent two-point example, the actual probability of both events was $0.9025$.

Correction: State explicitly whether the target is covered at a fixed $x$ or at every $x\in T$. For a band, report the coverage event and the function class over which its guarantee holds. Separately identify the width measure being used, such as expected width at a point or expected maximal width over the domain.

Key takeaway: Uniform honesty is a lower bound on coverage probability across a specified function class. Simultaneous honesty concerns the entire function and implies pointwise honesty, but pointwise coverage does not generally imply simultaneous coverage. Width is a separate measure of the band’s size.

Check Your Understanding

Answer each question using the coverage event or width measure that is relevant.

What event defines simultaneous coverage of a regression function on a domain $T$?
Why does simultaneous coverage imply coverage at each fixed input, and why does the reverse implication fail in general?
If two independent pointwise intervals each cover with probability $0.90$, what is the probability that both cover?
Define the global maximal width of a band and explain how its expected value differs from expected width at one fixed input.
What does the infimum over $f\in\mathcal{F}$ require in a statement of uniform honesty?