Pointwise Versus Integrated Squared-Error Rates

Why the Loss Measure Matters

A regression estimator can be judged at one specified covariate value or across a region. These questions lead to different risk functionals: pointwise risk emphasizes accuracy at a particular location, while integrated risk averages errors over a domain. Under regular conditions, their minimax squared-error rates have the same exponent. Their constants, however, can reflect different features of the design and of the region being evaluated.

We make this comparison for isotropic Hölder smoothness in random-design regression. The result is about rates, not universal sharp constants: a rate describes how risk scales with sample size, while its multiplicative constant may depend on the smoothness radius, noise level, domain, design density, and loss measure.

Model, Smoothness, and Risk

Let \(X_i\) take values in \(\mathcal X=[0,1]^d\), and observe independent pairs satisfying \(Y_i=m(X_i)+\varepsilon_i\), for \(i=1,\ldots,n\). Assume the \(X_i\) are identically distributed with density \(q\), and that \(E[\varepsilon_i\mid X_i]=0\). For the upper bound below, assume the errors are conditionally sub-Gaussian with a uniformly bounded variance proxy. For the lower bounds, take the more specific model \(\varepsilon_i\sim N(0,\sigma^2)\), independent of the covariates. Suppose throughout that

For clarity, take \(\beta>0\) noninteger, write \(r=\lfloor\beta\rfloor\) and \(\alpha=\beta-r\), and use the usual isotropic Hölder ball \(\mathcal H^ \beta(L)\): its functions have bounded derivatives through order \(r\), and derivatives of order \(r\) are \(\alpha\)-Hölder, with the relevant norms bounded by \(L\). When \(r=0\), this means that \(m\) itself is \(\beta\)-Hölder. The restriction to noninteger \(\beta\) avoids endpoint conventions; it does not change the rate calculation.

The design-density bounds make Lebesgue and design-weighted integrated risks comparable: for any nonnegative integrand \(g\),

Thus these two integrated losses have the same rate under the stated assumptions, although their constants need not match.

Balancing Approximation and Sampling Error

Partition the domain into regular cubes of side length \(h\), and fit a polynomial of degree \(r\) separately within each cube by least squares. This is a local polynomial sieve estimator. Hölder smoothness gives a polynomial approximation error of order \(h^\beta\) throughout each cube. Squaring this error contributes order \(h^{2\beta}\) to squared risk.

A cube contains, on average, order \(nh^d\) observations because its probability is of order \(h^d\) under the density bounds. A fixed-dimensional polynomial fit therefore has variance of order \((nh^d)^{-1}\). Consequently the risk bound has the form

The first term decreases as the cubes become smaller; the second increases because fewer observations fall in each cube. Balancing them gives \(h^{2\beta}\asymp (nh^d)^{-1}\), hence \(h\asymp n^{-1/(2\beta+d)}\). Substitution yields squared-error risk of order \(n^{-2\beta/(2\beta+d)}\).

Proof. Use a regular partition with \(h\) comparable to \(n^{-1/(2\beta+d)}\). In any cell, rescale coordinates to a unit cube and use a fixed basis for polynomials of degree at most \(r\). The population Gram matrix of this basis has eigenvalues bounded above and away from zero: the rescaled design density is bounded above and below, and the polynomial space is fixed and finite-dimensional. The cell count has expectation comparable to \(nh^d\). Boundedness of the rescaled basis and concentration for sums of bounded random matrices imply that, except on an event of probability at most \(C_1\exp(-c_1nh^d)\), the empirical Gram matrix is also well-conditioned. Here \(C_1,c_1>0\) do not depend on the cell or \(n\).

On that event, Taylor approximation in the cell has uniform residual at most \(C_2Lh^\beta\). Stability of the finite-dimensional least-squares projection therefore bounds its pointwise approximation error by \(C_3Lh^\beta\). Conditional on the covariates, the fitted value is a linear combination of the responses; well-conditioning and the sub-Gaussian noise assumption bound its variance by \(C_4/(nh^d)\). Its conditional squared bias is at most \(C_5h^{2\beta}\). On the exceptional event, define the cell estimator to be zero; since \(|m|\le L\), its squared error is bounded by \(L^2\). The exceptional-event contribution is at most \(C_1L^2\exp(-c_1nh^d)\), which is bounded by a constant times \((nh^d)^{-1}\) when \(nh^d\) is sufficiently large.

Combining these bounds gives a uniform pointwise risk bound \(C_6\{h^{2\beta}+(nh^d)^{-1}\}\) in every cell. Integrating this pointwise bound over a domain of volume one gives the same order for integrated risk. Finally, with \(h\asymp n^{-1/(2\beta+d)}\), both terms have order \(n^{-2\beta/(2\beta+d)}\). \(\square\)

Why the Matching Rate Is Also a Lower Bound

The upper bound alone does not show that the rate is unavoidable. A pointwise lower bound can be obtained by comparing two functions that differ only in a small neighborhood of the evaluation point. For integrated risk, many separated local perturbations are needed: a single bump would occupy too little of the domain to give the full integrated lower rate.

Proof for pointwise risk. Choose a smooth function \(\psi\) supported on the unit ball with \(\psi(0)=1\). For small \(h\), compare \(m_0(x)=0\) with \(m_1(x)=a h^\beta\psi((x-x_0)/h)\), where \(a>0\) is small enough that both functions lie in the Hölder ball. Their values at \(x_0\) differ by \(\Delta=a h^\beta\). The Kullback–Leibler divergence between the two \(n\)-sample distributions is

Set \(h=c_0 n^{-1/(2\beta+d)}\) and choose \(c_0\) (or \(a\)) small enough that this divergence is bounded by a fixed small constant. Pinsker’s inequality then bounds the total variation distance between the two distributions away from one. Any estimator induces a test choosing whichever of \(m_0(x_0)\) and \(m_1(x_0)\) is closer to its estimate. A testing error forces squared estimation error at least \(\Delta^2/4\); the sum of the two testing error probabilities is at least one minus the total variation distance. Thus the maximum risk under the two functions is at least a positive constant times \(\Delta^2\), which is \(h^{2\beta}\asymp n^{-2\beta/(2\beta+d)}\).

Proof for integrated risk. Place \(M\asymp h^{-d}\) disjoint copies of the same scaled bump inside the domain, and form a hypercube of functions by independently switching each bump on or off. Adjacent functions differ in one cell. Their Kullback–Leibler divergence is at most \(C_8na^2h^{2\beta+d}\), so the same choice \(h\asymp n^{-1/(2\beta+d)}\), with small enough \(a\), keeps adjacent distributions difficult to distinguish. The squared Lebesgue distance contributed by one bump is a constant times \(a^2h^{2\beta+d}\). The Assouad hypercube bound sums the testing errors across the \(M\) disjoint cells, giving a minimax integrated risk at least

This proves the lower bound for Lebesgue integrated risk. The bounds on \(q\) make design-weighted and Lebesgue integrated risks comparable, so the same exponent holds for design-weighted risk. \(\square\)

Interpreting the Constants and the Loss

The rate exponent is shared, but pointwise and integrated risks answer different questions. At a fixed \(x_0\), local information matters: a larger \(q(x_0)\) generally supplies more observations near that point and can reduce the variance constant. The pointwise lower-bound calculation shows why density near \(x_0\) enters the difficulty of distinguishing local alternatives. Under continuity of \(q\), the local divergence is approximately proportional to \(q(x_0)n h^{2\beta+d}\), which makes this dependence especially transparent.

For integrated risk, errors across the domain are accumulated. A density-weighted loss emphasizes locations where \(q\) is large; Lebesgue loss gives equal weight per unit volume. If \(q\) is bounded above and away from zero, this weighting changes constants but not the rate. If those bounds fail, the two losses need not be comparable, and the integrated rate may depend on how quickly the density approaches zero. That is a different design regime from the one treated here.

A common mistake is to conclude that integrated risk must have a faster rate because it averages errors over many points. The lower-bound argument explains the flaw: a function class can contain many independent local perturbations, and integrated loss adds their contributions. The number of distinguishable regions offsets the smaller contribution of each individual bump. Another frequent mistake is to report \(n^{-\beta/(2\beta+d)}\) as the squared-error rate. That is the root-mean-squared-error rate; squared-error risk has exponent \(2\beta/(2\beta+d)\).