Common Errors in Density-Risk Calculations

Why Density-Risk Calculations Go Wrong

Many mistakes in density estimation are not errors in algebra so much as errors in scope. A formula derived at an interior point may fail at a support boundary; a density estimator normalized on the real line may not remain normalized after restriction; and a rate for squared error is not the rate for its square root. The calculations are manageable once the estimand, domain, and loss are stated explicitly.

We use the kernel density estimator based on independent observations \(X_1,\ldots,X_n\) with common density \(f\):

Here \(K\) is an integrable kernel with \(\int K(u)\,du=1\). It need not be nonnegative: higher-order kernels may take negative values, so their estimators need not be densities in the nonnegative sense. The normalization and risk calculations below must therefore be read as statements about integrals and losses, not as automatic claims that every estimate is a probability density.

Normalization: Specify the Domain

If \(K\) is integrable with unit integral, each kernel contribution integrates to one over the entire real line. In particular, for every observed value \(X_i\), changing variables \(u=(x-X_i)/h\) gives

Averaging over observations therefore gives \(\int_{\mathbb R}\hat f_h(x)\,dx=1\), including when \(K\) is signed. This does not imply that the estimator integrates to one over a restricted domain \(D\), such as the support \([0,1]\). Nor does it guarantee nonnegativity when \(K\) can be negative.

Boundary Bias Is Not an Interior Approximation

For a symmetric kernel, the familiar second-order bias calculation uses cancellation of the first moment, \(\int uK(u)\,du=0\), and a Taylor expansion of \(f(x-hu)\). It applies at points where the full kernel window lies in a region on which the density is smooth. At a support boundary, part of that window lies outside the support, where the density is zero; the same cancellation argument cannot simply be reused.

Pointwise and Integrated Risk Are Different Quantities

For any estimator with finite second moments, adding and subtracting its expectation gives the exact pointwise decomposition

This identity follows because \(E[\hat f_h(x)-E\hat f_h(x)]=0\), so the cross-product vanishes after squaring and taking expectation. If \((\omega,x)\mapsto(\hat f_h(x,\omega)-f(x))^2\) is measurable, Tonelli's theorem also gives

The nonnegativity of the squared integrand is what permits this exchange even before finiteness is established; a finite MISE is then an additional property of the estimator and model. The identity does not make the two risks interchangeable: one is evaluated at a specified point, while the other accumulates risk across a domain.

Kernel Variance: Track Both Factors of the Bandwidth

A common scaling error is to lose one factor of \(h\) when computing the variance. Put \(Y_i=K((x-X_i)/h)\). Independence gives \(\operatorname{Var}(\hat f_h(x))=\operatorname{Var}(Y_i)/(nh^2)\). If \(f\) is a density and \(K\) is square-integrable, a change of variables yields the exact formula

The first term has scale \(1/(nh)\), not \(1/n\): the change of variables in \(E[Y_i^2]\) contributes a factor \(h\), while the estimator itself has the prefactor \(1/h\). If \(f\) is bounded and continuous at an interior point \(x\), and \(K\) is square-integrable with suitable integrable support or tail control, the leading variance term is \(f(x)R(K)/(nh)\), where \(R(K)=\int K(u)^2du\). The subtracted term is of order \(1/n\) under the corresponding regularity conditions and is smaller than \(1/(nh)\) as \(h\to0\).

Squared-Error Rates and Error Rates

Under an interior, twice-smooth setting, a symmetric second-order kernel with finite second moment has bias of order \(h^2\); the leading bias is \(\tfrac12 h^2 f''(x)\int u^2K(u)\,du\) when the relevant Taylor expansion is valid. Together with variance of order \(1/(nh)\), this gives pointwise MSE of order \(h^4+1/(nh)\). For whole-line integrated risk, the same order is obtained under suitable smoothness and integrability conditions, for example when the density and its second derivative are square-integrable and the boundary does not introduce a larger bias term. These are not boundary-free guarantees for every support or estimator.

When both leading constants are positive, balancing the two terms gives \(h\) of order \(n^{-1/5}\), and the optimized squared-error risk is of order \(n^{-4/5}\). The corresponding root-MSE is of order \(n^{-2/5}\), because it is the square root of the MSE. In general, taking a square root changes the exponent of a squared-error rate; it does not leave the rate unchanged.

Common Mistakes and a Reliable Check

Before accepting a density-risk calculation, identify the domain, the loss, the location relative to the boundary, and whether the expression is exact or asymptotic. The following errors are especially consequential:

• Assuming restriction preserves normalization. Unit kernel mass on the real line does not imply unit mass on the support. Integrate over the stated domain.
• Using an interior bias expansion at a boundary. The kernel window is truncated there; check its expectation directly or use a justified boundary correction.
• Equating pointwise and integrated risk. One evaluates at a point and the other integrates over a domain. Neither generally controls a supremum without additional conditions.
• Dropping a bandwidth factor in the variance. The estimator has a factor \(1/h\), and the change of variables in the second moment supplies another \(h\); the leading variance scale is \(1/(nh)\).
• Reporting a squared-error exponent as an error exponent. MSE of order \(n^{-4/5}\) corresponds to root-MSE of order \(n^{-2/5}\).
• Treating a leading approximation as exact. The variance expression includes a subtracted term; it may be lower order, but it is still part of the exact formula.