Effective Smoothness In Anisotropic Rates

Why Coordinate Smoothness Changes the Rate

When a regression function varies at different rates along different coordinates, a single isotropic resolution is inefficient. A surface may be very smooth in one direction but change rapidly in another. Rectangular approximation lets us use short intervals where the function is rough and longer intervals where it is smooth. The resulting squared-error rate depends on all coordinate smoothnesses, but through their reciprocals.

We derive that dependence for integrated squared-error risk. The basic mechanism is a balance: finer rectangles reduce approximation bias but increase the number of fitted coefficients, and hence variance. A matching lower-bound argument explains why this balance is not just a property of one estimator.

Setting and Anisotropic Smoothness

Let \(X_i\) be independent and uniform on \([0,1]^d\), and suppose \(Y_i=f(X_i)+\varepsilon_i\), where the errors are independent of the design and are distributed as \(N(0,\sigma^2)\). We measure performance by the expected integrated squared-error risk \(E\|\hat f-f\|_2^2\), where \(\|g\|_2^2=\int_{[0,1]^d}g(x)^2\,dx\). The rate below is for a fixed dimension \(d\), fixed smoothness indices, and fixed positive class radius and noise variance.

The coordinatewise definition makes clear what each \(\beta_j\) controls: regularity as \(x_j\) varies with the other coordinates fixed. For such classes, rectangular local-polynomial approximation can use a width \(h_j\) in coordinate \(j\) and achieve squared integrated approximation error bounded by a constant times \(\sum_{j=1}^d h_j^{2\beta_j}\). This is the approximation property needed for the rate calculation.

Deriving the Upper-Rate Exponent

For coordinate widths \(h_1,\ldots,h_d\), the number of rectangular cells, and thus the order of the number of local polynomial coefficients, is \(K\asymp\prod_{j=1}^d h_j^{-1}\). The approximation property above gives squared bias of order at most \(\sum_j h_j^{2\beta_j}\). Under a regular design and a stable local-polynomial or sieve estimator, the usual integrated-risk bias–variance bound is \(C\{\sum_j h_j^{2\beta_j}+K/n\}\), for a constant \(C\) independent of \(n\) and the chosen widths in the relevant range.

To balance the coordinate biases, set \(h_j=t^{1/\beta_j}\). Then every \(h_j^{2\beta_j}=t^2\), while the number of coefficients is determined by the single sum \(S=\sum_j1/\beta_j\). The next result completes the calculation.

Derivation. Equate the two contributions: \(t^2=n^{-1}t^{-S}\). Multiplying by \(t^S\) gives \(t^{2+S}=n^{-1}\), so \(t=n^{-1/(2+S)}\). Substitution yields \(t^2=n^{-2/(2+S)}\). Also \(K\asymp t^{-S}=n^{S/(2+S)}\), and therefore \(K/n=n^{-2/(2+S)}\), as required. The corresponding widths are \(h_j=n^{-1/[\beta_j(2+S)]}\). Thus each direction gets its own resolution, even though all directions share the same balanced bias scale. \(\square\)

Why the Same Exponent Is a Lower Bound

The balancing calculation gives an achievable rate. To see why the exponent is intrinsic to the class, consider a family of localized smooth perturbations. Let \(\psi\) be a nonzero, infinitely differentiable function supported strictly inside the unit cube. For a small scale \(t\), use boxes with side lengths proportional to \(h_j=t^{1/\beta_j}\), separated so their interiors do not overlap. There are \(M\asymp\prod_j h_j^{-1}=t^{-S}\) such boxes.

On box \(k\), place a rescaled copy of \(\psi\) with amplitude \(a=c_0t\), and choose independently whether its sign is positive or negative. Here \(c_0>0\) is a sufficiently small fixed constant. These functions lie in a fixed-radius coordinatewise anisotropic Hölder class: in coordinate \(j\), the amplitude divided by \(h_j^{\beta_j}\) is \(c_0t/t=c_0\), so the relevant Hölder bound stays controlled. Let \(f_\omega\) denote the function indexed by the vector of signs \(\omega\in\{-1,1\}^M\).

Lower-bound derivation. The squared \(L^2\) distance between two functions that differ only in the sign on one box is a constant times \(a^2\prod_j h_j=c_1t^{2+S}\), where \(c_1>0\) depends on the bump and the fixed scale constants. For two such neighboring functions, the Kullback–Leibler divergence between the distributions of the \(n\) observations is \(\frac{n}{2\sigma^2}\|f_\omega-f_{\omega'}\|_2^2\). This identity follows by conditioning on the uniform design and using the Gaussian mean-shift divergence \((\mu-\mu')^2/(2\sigma^2)\) for each response. Hence the divergence is at most \(c_2nt^{2+S}\).

Choose \(t=c_3n^{-1/(2+S)}\), with \(c_3\) small enough that the divergence for neighboring sign vectors is at most \(1/8\). Pinsker's inequality then bounds their total variation distance by at most \(1/4\). For any estimator, decide the sign on each box by the sign of its \(L^2\) inner product with that box's bump. If this decision is wrong, the squared estimation error on that box is at least a constant times \(a^2\prod_jh_j\): the true bump coefficient has magnitude \(a\|\psi_k\|_2\), and a wrong sign makes the coefficient error at least that large. For each sign, the average probability of error when distinguishing the two neighboring distributions is at least \((1-\operatorname{TV})/2\), hence at least \(3/8\). Summing over the \(M\) disjoint boxes gives risk at least a constant times \(M t^{2+S}\asymp t^{-S}t^{2+S}=t^2=n^{-2/(2+S)}\). This matches the upper exponent and proves the claimed order. \(\square\)

Interpreting Effective Smoothness

The quantity \(S=\sum_j1/\beta_j\) plays the role of an effective dimension in the exponent. If all coordinates have common smoothness \(\beta\), then \(S=d/\beta\), and the formula reduces to \(n^{-2\beta/(2\beta+d)}\). With unequal smoothnesses, one can define the effective smoothness \(\bar\beta=d/S\); the exponent then has the same algebraic form \(2\bar\beta/(2\bar\beta+d)\). This is a rate summary, not a claim that the function has identical smoothness in every direction.

A rough coordinate has a large reciprocal \(1/\beta_j\), so it contributes more to the effective dimension and reduces the exponent. A very smooth coordinate has a small reciprocal, so its contribution is modest. The widths \(h_j=n^{-1/[\beta_j(2+S)]}\) express the same idea geometrically: lower smoothness requires a finer partition in that direction.

Common Mistake

A tempting but incorrect calculation is to average the smoothness indices and then substitute that average into the isotropic formula. For \((1/2,1,2)\), the arithmetic mean is \(7/6\), which would produce \(2(7/6)/(2(7/6)+3)=7/16\). The correct exponent is \(4/11\), because the anisotropic partition count depends on \(1/\beta_1+1/\beta_2+1/\beta_3\), not on the arithmetic mean of the \(\beta_j\). The correction is to count resolution levels: each coordinate contributes \(h_j^{-1}\), and with \(h_j=t^{1/\beta_j}\), their product is \(t^{-\sum_j1/\beta_j}\).