Common Errors In Nonparametric Rate Exponents

Why Rate Exponents Are Easy to Misread

A nonparametric rate is not just an exponent attached to \(n\). It summarizes how an estimator performs under a particular loss, for a particular smoothness class, dimension, and set of assumptions. Two correct statements about the same problem can therefore display different exponents if one reports squared risk and the other reports its square root. A formula can also be wrong because it suppresses dimension, treats unequal smoothness as isotropic, or omits a logarithmic factor required by the loss.

The goal here is to audit the rate before using it. We will state a small set of rate conventions, derive the transformations that commonly cause errors, and work through examples where the mistakes produce visibly different answers. The rates below are canonical orders under the stated settings, not claims that every estimator or every model has these rates.

First Identify the Risk Being Reported

Let \(g\) be a regression function on \([0,1]^d\), and let \(\hat g\) estimate it from \(n\) observations. For integrated squared error, define the risk at \(g\) by \(E_g\|\hat g-g\|_2^2\), where the norm is with respect to Lebesgue measure on \([0,1]^d\). The minimax risk over a function class \(\mathcal G\) is the infimum over estimators of the worst-case risk on that class.

For example, under the usual regularity conditions for nonparametric regression—an isotropic Hölder class of smoothness \(\beta>0\), fixed dimension \(d\), a suitably regular design density, and noise with controlled variance—the canonical integrated squared-error minimax order is \(n^{-2\beta/(2\beta+d)}\). The root minimax risk then has order \(n^{-\beta/(2\beta+d)}\). These are two descriptions of the same performance, not two competing minimax results.

Proof. By the meaning of \(\asymp\), there are constants \(0<c\le C<\infty\) such that \(ca_n\le R_n\le Ca_n\) for all sufficiently large \(n\). Taking square roots preserves the inequalities because the square-root function is increasing on nonnegative numbers. Thus \(\sqrt c\,\sqrt{a_n}\le \sqrt{R_n}\le \sqrt C\,\sqrt{a_n}\), which is precisely \(\sqrt{R_n}\asymp\sqrt{a_n}\). Since \(\sqrt{n^{-q}}=n^{-q/2}\), the stated power follows. \(\square\)

Keep Dimension in the Isotropic Exponent

For an isotropic smoothness class, the standard integrated squared-error exponent contains \(d\) in the denominator. Dimension records how many directions must be resolved as the sample grows. Dropping it is not a harmless simplification: it changes the rate, often substantially. A useful check is to substitute the actual dimension only after writing the entire formula, rather than relying on a remembered one-dimensional expression.

This comparison is an exponent check, not a proof that every three-dimensional model has exactly this rate. The design, loss, noise, and function class must still match the conditions under which the canonical rate applies.

Do Not Substitute an Isotropic Formula for Anisotropic Smoothness

An anisotropic class allows different smoothness levels in different coordinate directions. Write them as \(\beta_1,\ldots,\beta_d\), with every \(\beta_j>0\). For the usual anisotropic Hölder regression setting and integrated squared error, the canonical squared-risk order is \(n^{-2/(2+\sum_{j=1}^d 1/\beta_j)}\). Its exponent depends on all the coordinate smoothness levels, not just one selected \(\beta_j\).

The same expression can be written using the effective smoothness \(\bar\beta=d/(\sum_{j=1}^d 1/\beta_j)\). Substitution gives \(2/(2+\sum_j1/\beta_j)=2\bar\beta/(2\bar\beta+d)\). This is a harmonic-type average: a less smooth direction contributes a larger reciprocal and can lower the effective smoothness.

Derivation. Set \(S=\sum_{j=1}^d1/\beta_j\) and \(t=n^{-1/(2+S)}\), and choose \(h_j=t^{1/\beta_j}\). Then \(h_j^{2\beta_j}=t^2\) for every \(j\), so the sum of bias terms is \(dt^2\). Also, \(\prod_j h_j=t^S\); since \(n=t^{-(2+S)}\), the variance term is \(1/(nt^S)=t^2\). Thus the displayed bound is of order \(t^2=n^{-2/(2+S)}\).

This balance gives the order of the stated bound. To check that no different resolutions make that bound smaller in order, write \(A=\sum_j h_j^{2\beta_j}\). If \(A\ge t^2\), the bound is already at least \(t^2\). If \(A<t^2\), each \(h_j^{2\beta_j}<t^2\), so \(\prod_jh_j<t^S\), and the variance term is greater than \(1/(nt^S)=t^2\). In either case the bound is at least \(t^2\), while the chosen resolutions attain a constant multiple of \(t^2\). \(\square\)

Check Whether a Logarithm Belongs to the Loss

A power of \(n\) is not always the whole rate. For example, in standard settings the supremum-norm rate over an isotropic Hölder class includes a logarithmic factor: a typical root supremum-error order is \((\log n/n)^{\beta/(2\beta+d)}\). The corresponding squared supremum-error order is \((\log n/n)^{2\beta/(2\beta+d)}\). The factor arises in the supremum-loss setting; it should not be inserted into or removed from a rate without checking the loss and assumptions.

Proof. Squaring the positive expression gives \(\big((\log n)/n\big)^{2a}\) by the exponent rule \((x^a)^2=x^{2a}\). Equivalently, its factors are \((\log n)^{2a}n^{-2a}\). Leaving the logarithm at power \(a\) while doubling only the power of \(n\) would not be squaring the rate. \(\square\)

A Practical Rate-Audit Checklist

Before comparing two rate statements or reporting one, write down what is being estimated and what quantity the rate describes. Then check the dimensions, smoothness parameters, and any logarithms against that target. The following sequence catches the most frequent exponent errors.

Common Mistakes

• Calling a root-risk exponent a squared-risk exponent. If squared risk is \(n^{-q}\), its square root is \(n^{-q/2}\), not \(n^{-q}\).
• Dropping \(d\) from an isotropic formula. The expression \(2\beta/(2\beta+d)\) depends on the dimension being estimated over.
• Using one smoothness value for an anisotropic class. The reciprocal sum \(\sum_j1/\beta_j\) is needed for the stated anisotropic rate.
• Squaring only the power of \(n\). Squaring \((\log n/n)^a\) produces \((\log n/n)^{2a}\), not \((\log n)^a n^{-2a}\).
• Comparing rates for different targets as if they were the same. A rate depends on the loss and assumptions; a correct exponent in one setting need not apply in another.