Anisotropic Smoothness and Dimension

Why Smoothness Across Coordinates Matters

A multivariate density need not be equally smooth in every direction. For example, a density may change gradually with one coordinate but have sharper variation with another. A single bandwidth applied to all coordinates ignores this difference: it may smooth too little in a rough direction or too much in a smooth one. An anisotropic estimator instead assigns a bandwidth to each coordinate and chooses those bandwidths in light of their separate smoothness orders.

The central calculation balances two contributions to integrated squared error (ISE). Smoothing creates bias, whose size depends on both each bandwidth and its coordinate’s smoothness. Sampling creates variance, which grows as the product of the bandwidths shrinks. The resulting rate shows how dimension and directional smoothness jointly determine the cost of estimating a density.

Directional Smoothness and the Estimator

For a precise rate calculation, consider a density \(f\) on the \(d\)-dimensional unit torus, so that boundary effects do not enter the calculation. Assume observations \(X_1,\ldots,X_n\) are independent and identically distributed from \(f\). Let \(\beta_j>0\) be the smoothness order in coordinate \(j\), for \(j=1,\ldots,d\). We use noninteger \(\beta_j\) here. Write \(m_j=\lfloor\beta_j\rfloor\) and \(\alpha_j=\beta_j-m_j\), so \(0<\alpha_j<1\).

For orders above one, the corresponding Taylor remainder in direction \(j\) is bounded in \(L_2\) by a constant times \(|t|^{\beta_j}\). This is the feature of the smoothness condition used below. Take a one-dimensional kernel \(K_j\) for each coordinate, with integral one, finite \(L_2\) norm and finite absolute \(\beta_j\)-moment. Require its moments of orders \(1,\ldots,m_j\) to vanish. The product kernel estimator, with bandwidths \(h_1,\ldots,h_d\), is

where kernels are periodized on the torus. The moment conditions remove the lower-order terms in the directional Taylor expansion. They can require a signed kernel when higher-order moments must vanish; the estimator need not be nonnegative. That issue is distinct from the bandwidth-rate calculation.

Bias and Variance with Separate Bandwidths

Let \(T_j\) denote convolution with the \(j\)th scaled kernel in coordinate \(j\). The directional Taylor remainder and the kernel moment conditions imply \(\|(T_j-I)f\|_2\le C_j h_j^{\beta_j}\), where \(I\) is the identity operator. The product smoothing operator is \(T_1\cdots T_d\). Subtracting \(f\) and telescoping the product gives a sum of directional smoothing differences. Each other convolution operator has \(L_2\) operator norm at most the \(L_1\) norm of its kernel, so these factors change only the constant. Consequently, for a constant \(C_1\) independent of the bandwidths,

Squaring and using \((\sum_j a_j)^2\le d\sum_j a_j^2\) yields a squared-bias bound proportional to \(\sum_j h_j^{2\beta_j}\). For variance, independence of the observations gives a factor \(1/n\). Integrating the pointwise variance and dropping its nonnegative squared-mean term bounds it by \(n^{-1}\) times the squared \(L_2\) norm of the product kernel. For sufficiently small bandwidths, that norm squared is a constant times \(\prod_j h_j^{-1}\). Thus:

To see the decomposition, write \(\hat f_h-f=(\hat f_h-\mathbb{E}\hat f_h)+(\mathbb{E}\hat f_h-f)\). The cross term has expectation zero at every \(x\); integration therefore gives the sum of integrated variance and squared bias. Applying the two bounds above proves the result. This derivation makes clear why every coordinate affects the rate: the bias terms are directional, while the variance depends on the volume \(\prod_j h_j\).

Balancing the Terms

The bias contribution for coordinate \(j\) is \(h_j^{2\beta_j}\), while making any bandwidth smaller increases the variance. For a rate calculation, set \(h_j=n^{-a_j}\). If all the squared-bias terms and the variance have the same order \(n^{-r}\), then \(2\beta_j a_j=r\) for every \(j\), and \(1-\sum_j a_j=r\). Solving these equations gives the bandwidth allocation and rate.

Indeed, the proposed bandwidth has \(a_j=1/(\beta_jD)\), so \(h_j^{2\beta_j}=n^{-2/D}\). Also, \(\sum_j a_j=(D-2)/D\); hence the variance term is \(n^{-1+(D-2)/D}=n^{-2/D}\). The bias sum has only \(d\) terms, so its order is also \(n^{-2/D}\). Substitution into the risk bound proves the stated upper rate. This is a rate-optimal allocation for that bias-variance bound: any power allocation balancing all terms must satisfy the equations above, which uniquely determine \(r\) and the \(a_j\). \(\square\)

Worked Bandwidth Allocations

Dimension, Interpretation, and a Common Pitfall

The sum \(\sum_j1/\beta_j\) is the key measure of anisotropic complexity. A low smoothness order contributes a large reciprocal and slows the rate; a smoother coordinate contributes less. If all coordinates have common order \(\beta\), then \(D=2+d/\beta\), giving rate \(n^{-2\beta/(2\beta+d)}\). Thus the familiar dimension penalty is recovered, while unequal smoothness replaces \(d/\beta\) with the sum of the coordinate-specific reciprocals. One can summarize this using the effective smoothness \(\beta_{\mathrm{eff}}=d/(\sum_j1/\beta_j)\), for which \(D=2+d/\beta_{\mathrm{eff}}\).

Common mistake: use one bandwidth exponent for every coordinate. A tempting argument is that there are \(d\) coordinates, so the optimal bandwidth should have the same exponent in each, regardless of smoothness. That allocation generally fails to balance the directional bias terms: if \(h_j=n^{-a}\), then coordinate \(j\)'s squared bias has exponent \(2\beta_j a\), which differs across coordinates when the \(\beta_j\) differ. The correction is to choose \(a_j=1/(\beta_jD)\). Rough coordinates then receive smaller bandwidths, while smoother coordinates can use larger ones. The product of all bandwidths must still be included when calculating variance.