Choosing the Series Truncation Dimension

Why the Truncation Dimension Matters

An orthonormal series density estimator uses only a finite number of basis functions to represent an unknown density. The truncation dimension \(m\) controls a direct trade-off: a small \(m\) may leave important structure out of the estimate, while a large \(m\) requires estimating more coefficients from the same sample. Choosing \(m\) means balancing these two sources of integrated error, rather than maximizing the number of terms or minimizing approximation error alone.

This tutorial derives that balance under a Hölder-smoothness approximation bound. The resulting dimension is a rate-level choice: constants, the basis, and the sample size can affect the best finite-sample integer. The main conclusions concern how the dimension and risk scale with \(n\), smoothness \(\beta\), and ambient dimension \(d\).

Setup: Approximation Error and Variance

Let \(X_1,\ldots,X_n\) be independent observations from a density \(f\) on a \(d\)-dimensional domain, and let \(\{\phi_j\}_{j\geq 1}\) be an orthonormal basis for the relevant \(L^2\) space. Write \(\theta_j=\int \phi_j(x)f(x)\,dx\) and estimate each coefficient by \(\hat\theta_j=n^{-1}\sum_{i=1}^n\phi_j(X_i)\). The series estimate truncated at \(m\) terms is \(\hat f_m=\sum_{j=1}^m\hat\theta_j\phi_j\). Its integrated squared-error risk is \(E[\|\hat f_m-f\|_2^2]\).

The earlier material on projection bias and series variance gives the guiding decomposition: integrated risk is the squared error from projecting \(f\) onto the first \(m\) basis functions plus the variance from estimating their coefficients. For the basis and smoothness class under consideration, assume the following bounds hold uniformly over the candidate dimensions:

Here \(\beta>0\) is the Hölder smoothness, \(d\geq1\) is the ambient dimension, and \(A,B>0\) do not depend on \(m\) or \(n\). The approximation bound describes how well the first \(m\) terms can represent densities in the class; the variance bound describes the cost of estimating \(m\) coefficients. Thus an upper bound for the risk is

This bound is the object we optimize. If it is only an upper bound, its minimizer optimizes the bound, not necessarily the exact finite-sample risk. When the bias and variance are also bounded below by constant multiples of these expressions, the same calculation describes the risk order of an optimal choice.

Deriving the Optimal Continuous Dimension

First treat \(m\) as a positive real number. This simplifies the optimization; an actual series estimator must use an integer dimension, which will be handled afterward. Define the bound \(G_n(m)=A m^{-\alpha}+Bm/n\). Its derivative is

The derivative is strictly increasing in \(m>0\), since \(G_n''(m)=\alpha(\alpha+1)A m^{-\alpha-2}>0\). It is negative for sufficiently small \(m\) and positive for sufficiently large \(m\), so its unique zero is the global minimizer.

Proof. Setting the derivative equal to zero gives

Taking the positive \((\alpha+1)\)st root yields \(m_c=(\alpha A n/B)^{1/(\alpha+1)}\). Strict positivity of the second derivative shows this stationary point is the unique global minimum. At the stationary point, the derivative equation also gives \(Bm_c/n=\alpha A m_c^{-\alpha}\). Therefore

Finally, \(\alpha/(\alpha+1)=2\beta/(2\beta+d)\), so with the other quantities fixed this expression is a constant times \(n^{-2\beta/(2\beta+d)}\). Substituting \(\alpha=2\beta/d\) into the expression for \(m_c\) gives the stated dimension formula. \(\square\)

The two contributions at the optimum are therefore comparable, but they are not generally equal: their ratio is set by \(\alpha\). In rate calculations, one often writes \(m\asymp n^{d/(2\beta+d)}\), suppressing fixed positive constants. Substituting any such dimension into the two terms gives the same risk order \(n^{-2\beta/(2\beta+d)}\).

From a Real-Valued Optimum to an Integer Choice

The formula for \(m_c\) is not itself an estimator dimension unless it happens to be an integer. Rounding is harmless for the rate, provided the candidate dimensions are available and \(m_c\) is not confined to a small-sample boundary. The next result makes this precise for all positive integer dimensions.

Derivation. Let \(k=\lfloor m_c\rfloor\). Since \(m_c\geq2\), \(k\geq m_c/2\), and \(k\leq m_c\). Consequently

Adding these inequalities shows \(G_n(k)\leq\max(2^\alpha,1)G_n(m_c)\), which is no larger than the displayed bound. The integer minimizer has a bound no greater than \(G_n(k)\). Alternatively, \(q=\lceil m_c\rceil\) satisfies \(q\geq m_c\) and \(q\leq2m_c\), so \(Aq^{-\alpha}\leq A m_c^{-\alpha}\) and \(Bq/n\leq2Bm_c/n\), giving the same constant-factor conclusion. This proves the result. \(\square\)

In the usual asymptotic setting, \(m_c\) grows with \(n\), so \(m_c\geq2\) eventually. If only a restricted sequence of dimensions is available, such as a set of block sizes, the rounding argument requires that an available dimension be within a fixed multiplicative factor of \(m_c\). The rate conclusion then follows by the same comparison. For very small samples, constraints such as a minimum dimension may instead determine the practical choice.

How Smoothness and Dimension Affect the Rates

The optimal dimension and risk have different exponents:

For fixed \(d\), increasing \(\beta\) decreases \(q\), since

Thus greater smoothness improves the risk exponent but makes the optimal dimension grow more slowly with sample size. This does not assert that the optimal finite-sample dimension must decrease in every comparison: \(A\), \(B\), the basis, and integer rounding also matter.

For fixed \(\beta\), increasing \(d\) has the opposite effect on the dimension exponent and the risk exponent:

The dimension therefore grows faster with \(n\) as \(d\) increases, while the risk rate becomes slower. This reflects the fact that representing a density with the same smoothness becomes more demanding in a higher-dimensional domain.

Practical Interpretation and Common Pitfall

A useful selection rule is to choose an available dimension on the order of \(n^{d/(2\beta+d)}\), with constants informed by the approximation and variance scales when they are known. The smoothness \(\beta\) is often unknown in practice, so this formula alone does not supply an adaptive data-driven rule. It describes the target balance when the smoothness class and constants are specified.

In the usual bias–variance terminology for this truncation problem, too few retained terms create excessive approximation error and are called oversmoothing. Too many terms retain noisy coefficient estimates and are called undersmoothing. The words can be used differently in other contexts, so the underlying error description is more important than the label.

• Pitfall: “Greater smoothness always means a larger optimal dimension.” This does not follow from the rate formula. For fixed \(d\), the exponent \(d/(2\beta+d)\) decreases as \(\beta\) increases. The exact finite-sample choice can also depend on constants, so avoid asserting a universal ordering of dimensions.
• Pitfall: “Higher dimension makes the optimal dimension grow more slowly with \(n\).” The exponent \(d/(2\beta+d)\) increases with \(d\). What becomes slower is the risk rate, because \(2\beta/(2\beta+d)\) decreases.
• Pitfall: Reversing the smoothing labels. In this series-truncation setting, too few terms mean oversmoothing and excess bias; too many mean undersmoothing and excess variance.
• Pitfall: Treating a bound's optimizer as an exact finite-sample oracle. The calculation optimizes \(A m^{-2\beta/d}+Bm/n\). If this only bounds the risk from above, the actual risk can have different constants or a different best integer.