Minimax Rates Under Integrated Absolute Loss

Why Integrated Absolute Loss Has Its Own Rate

A density estimate can be close at most locations while still making substantial errors across the whole domain. Integrated absolute loss measures the total size of those errors rather than the error at a single point or the largest error anywhere. Its minimax rate reflects a balance: smoothing reduces variation in the estimate but introduces bias, while using less smoothing reduces bias but makes the estimate noisier.

Here the domain is the \(d\)-dimensional unit torus, represented by \([0,1]^d\) with opposite faces identified. This removes boundary effects from the rate calculation. Assume the observations \(X_1,\ldots,X_n\) are independent with common density \(f\), and that \(f\) belongs to a bounded Hölder class of smoothness \(\beta>0\). For the main statements, take \(\beta\) noninteger; the corresponding integer-smoothness result holds under the usual Hölder convention. The class is assumed to have fixed bounds on the density and its required derivatives.

A rate statement \(n^{-\beta/(2\beta+d)}\) means that, up to positive constants independent of \(n\), the minimax risk is bounded above and below by this power. It is not a claim that every estimator, or every density in the class, has exactly that risk.

Worked Examples: Reading the Rate

An Upper Bound: Bias Plus Integrated Fluctuation

A kernel estimator makes the bias–fluctuation balance explicit. Use a compactly supported kernel \(K\) with integral one, finite squared integral, and vanishing moments through order \(\lfloor\beta\rfloor\). Take a periodic version of the kernel estimator, with bandwidth \(h\). The moment conditions remove the lower-order terms in a Taylor expansion, so for a density in the stated Hölder class the smoothing bias is bounded in absolute value by \(C h^\beta\), uniformly in \(x\). Here and below \(C\) denotes a finite constant that may depend on the class, kernel, \(\beta\), and \(d\), but not on \(n\) or \(h\).

Proof. Write \(\hat f_h-E_f[\hat f_h]=\hat f_h-E_f[\hat f_h]\) and apply the triangle inequality inside the integral. The expected integrated error is at most the integrated absolute bias plus the expected integrated absolute centered fluctuation. The uniform bias bound gives an integrated bias at most \(C h^\beta\), since the domain has volume one.

At each \(x\), independence of the observations gives \(\operatorname{Var}_f(\hat f_h(x))\leq n^{-1}E_f[K_h^{\mathrm{per}}(x-X_1)^2]\), where \(K_h^{\mathrm{per}}\) is the periodic kernel. Boundedness of \(f\), compact support, and the finite squared integral of \(K\) imply \(E_f[K_h^{\mathrm{per}}(x-X_1)^2]\leq C h^{-d}\). Thus the pointwise variance is at most \(C/(nh^d)\). By Cauchy–Schwarz, \(E_f|\hat f_h(x)-E_f\hat f_h(x)|\leq\sqrt{\operatorname{Var}_f(\hat f_h(x))}\leq C(nh^d)^{-1/2}\). Integrating over a domain of volume one gives the same order for the integrated fluctuation. These integrations and expectation exchanges are valid for the nonnegative absolute-error integrands by Tonelli’s theorem.

Finally, set the two orders equal: \(h^\beta=(nh^d)^{-1/2}\) implies \(h^{2\beta+d}=n^{-1}\), hence \(h=n^{-1/(2\beta+d)}\). Substitution gives \(h^\beta=n^{-\beta/(2\beta+d)}\), and the fluctuation has that same order. This proves the bound. \(\square\)

A Matching Lower Bound: Many Hard-to-Distinguish Densities

An upper bound alone does not establish a minimax rate: a different estimator might perform better. For a lower bound, construct a collection of densities that are smooth but differ in many small, separated regions. The sample must contain enough information to distinguish the alternatives in each region. At the optimal scale, it cannot reliably distinguish all of them, so every estimator incurs integrated error on many regions.

Proof. Choose a nonzero smooth function \(\psi\) supported strictly inside the unit cube, with \(\int\psi=0\). For an integer \(q\), put \(h=1/q\) and divide the torus into \(m=q^d=h^{-d}\) cubes. Index the cubes by \(j\), with lower corners \(x_j\). For each vector \(\theta\in\{-1,1\}^m\), define \(f_\theta(x)=1+a h^\beta\sum_{j=1}^m\theta_j\psi((x-x_j)/h)\), interpreting each rescaled bump on its cube and as zero elsewhere.

Every bump integrates to zero after rescaling, so \(f_\theta\) integrates to one. The supports are disjoint. Choose the fixed amplitude \(a>0\) small enough that \(a\|\psi\|_\infty\leq1/2\); then \(f_\theta\geq1/2\). Rescaling a smooth bump by \(h\) and multiplying by \(h^\beta\) keeps its Hölder-\(\beta\) norm uniformly bounded. Consequently these are densities in a fixed Hölder class when its radius is chosen to include them.

Consider two vectors \(\theta,\theta'\) differing only in coordinate \(j\). Their densities differ on that cube by \(2a h^\beta\psi((x-x_j)/h)\). For one observation, using \(\log u\leq u-1\) gives the standard bound \(D(p\|q)\leq\int(p-q)^2/q\) for densities \(p,q\) with \(q>0\). Since \(f_{\theta'}\geq1/2\), this yields \[ D(f_\theta\|f_{\theta'})\leq 2\int(f_\theta-f_{\theta'})^2 =8a^2 h^{2\beta+d}\int\psi(u)^2\,du. \] For \(n\) independent observations, the Kullback–Leibler divergence is \(n\) times the one-observation divergence. Choose \(q=\lceil n^{1/(2\beta+d)}\rceil\), so \(nh^{2\beta+d}\leq1\), and reduce \(a\), if necessary, so neighboring sample distributions have divergence at most \(1/8\). Pinsker’s inequality then bounds their total variation distance by \(\sqrt{(1/8)/2}=1/4\).

To connect testing difficulty to loss, let \(g\) be any integrable estimator and define, for each cube, \[ T_j(g)=\int_{\text{cube }j}(g(x)-1)\operatorname{sign}\!\left(\psi((x-x_j)/h)\right)\,dx. \] Under \(f_\theta\), this functional equals \(\theta_j a h^{\beta+d}\|\psi\|_1\), where \(\|\psi\|_1=\int|\psi|\); the zero-integral bumps in other cubes do not contribute. Decode the \(j\)th sign from the sign of \(T_j(g)\), assigning either sign in a tie. If the decoded sign is wrong, then the difference between \(T_j(g)\) and its value under \(f_\theta\) is at least \(a h^{\beta+d}\|\psi\|_1\). Since the sign function has absolute value at most one, that difference is no larger than \(\int_{\text{cube }j}|g-f_\theta|\). Thus each wrongly decoded coordinate forces at least that much local integrated error.

For a pair of neighboring sample distributions, any test’s average probability of error under equal prior probabilities is at least \((1-\mathrm{TV})/2\), hence at least \(3/8\). Averaging uniformly over all sign vectors and pairing them along coordinate \(j\) shows that the average probability of decoding coordinate \(j\) incorrectly is at least \(3/8\). Summing the local errors over the disjoint cubes, the maximum risk over the alternatives is at least their average risk: \[ \sup_\theta E_\theta\int|g-f_\theta| \geq \frac{3}{8}m\,a h^{\beta+d}\|\psi\|_1 =\frac{3a\|\psi\|_1}{8}h^\beta. \] Because \(q=\lceil n^{1/(2\beta+d)}\rceil\), \(h\) is bounded below by a positive constant times \(n^{-1/(2\beta+d)}\) for sufficiently large \(n\). The lower bound is therefore a positive constant times \(n^{-\beta/(2\beta+d)}\). It applies to every estimator \(g\), proving the theorem. \(\square\)

What the Rate Means—and a Common Pitfall

The two arguments explain why the exponent has this form. The upper bound balances a smoothing bias \(h^\beta\) against integrated fluctuation \((nh^d)^{-1/2}\). The lower bound shows that at the corresponding spatial scale, a large collection of smooth density alternatives remains difficult to distinguish from the observations. Therefore the rate is not just a consequence of one estimator’s calculation.

• Pitfall: using pointwise fluctuation without checking the integrated loss. Pointwise variance is useful, but the risk here is an integral of absolute error. The proof must bound the expected absolute fluctuation at each location and integrate it. In this setting, that calculation gives order \((nh^d)^{-1/2}\); for a regular partition it is also \(\sqrt{m/n}\), where \(m=h^{-d}\).
• Pitfall: confusing the loss with its square. Integrated absolute loss is \(\int|\hat f-f|\), not \(\int(\hat f-f)^2\). The exponents and the relevant bias–fluctuation calculation must match the chosen loss.
• Pitfall: calling a balance an optimality proof. Balancing terms supplies an upper bound for a chosen estimator. A matching lower bound is needed to establish that no estimator can improve the rate over the class.