Bandwidth Asymptotics And The Bias-Variance Tradeoff

Why Bandwidth Matters

A kernel density estimator replaces each observed loss with a smooth contribution and adds those contributions to estimate the underlying density. The bandwidth $h$ controls how widely each observation is smoothed. A small bandwidth preserves local detail but can produce a variable, jagged estimate; a large bandwidth reduces sampling variation but can smooth away genuine features. The asymptotic bias-variance tradeoff gives a principled way to describe this tension.

The goal here is to derive the bandwidth that minimizes the leading approximation to the estimator’s integrated mean squared error. The result is a rate, not a universal numerical choice: its constant depends on the kernel and on the unknown density.

Definition: Let $X_1,\ldots,X_n$ be independent and identically distributed observations from a density $f$, and let $K$ be a kernel. The kernel density estimator with bandwidth $h>0$ is $$ \widehat f_h(x)=\frac{1}{nh}\sum_{i=1}^n K\!\left(\frac{x-X_i}{h}\right). $$ Write $R(g)=\int_{-\infty}^{\infty}g(x)^2\,dx$ and $\mu_2(K)=\int_{-\infty}^{\infty}u^2K(u)\,du$. The mean integrated squared error is $\operatorname{MISE}(h)=E\!\left[\int(\widehat f_h(x)-f(x))^2\,dx\right]$.

Conditions and the AMISE Approximation

The standard leading-order result assumes a symmetric, second-order kernel with $\int K(u)\,du=1$, $\int uK(u)\,du=0$, finite $\mu_2(K)$, and finite $R(K)$. Assume also that $f$ has a sufficiently smooth second derivative, $R(f'')<\infty$, and that the density and kernel satisfy conditions allowing the asymptotic expansions to be integrated. The bandwidth regime is $h\to0$ and $nh\to\infty$: smoothing becomes increasingly local while the effective number of observations contributing near a point grows.

For an interior point $x$, or for a density smooth on the whole real line without a boundary complication, the leading bias and variance are

\operatorname{Bias}\{\widehat f_h(x)\} \approx \frac{h^2\mu_2(K)}{2}f''(x), \qquad \operatorname{Var}\{\widehat f_h(x)\} \approx \frac{R(K)f(x)}{nh}.

Integrating squared bias over $x$ gives a term proportional to $h^4$, while integrating variance gives a term proportional to $1/(nh)$. The resulting asymptotic mean integrated squared error (AMISE) is

\operatorname{AMISE}(h) = \frac{h^4\mu_2(K)^2}{4}R(f'') +\frac{R(K)}{nh}.

This approximation keeps the leading contributions as $n$ grows and $h$ shrinks. It is not the exact finite-sample MISE, and it does not promise that the AMISE-minimizing bandwidth is optimal for every local feature of the density. In particular, ordinary symmetric-kernel expansions need modification at a support boundary. The expressions here describe the standard smooth, interior or whole-line setting.

Deriving the Asymptotically Optimal Bandwidth

Set $A=\mu_2(K)^2R(f'')/4$ and $B=R(K)$, so that $\operatorname{AMISE}(h)=Ah^4+B/(nh)$. The first term increases with $h$, while the second decreases. Differentiating and setting the derivative to zero gives the minimizing positive bandwidth:

\frac{d}{dh}\left(Ah^4+\frac{B}{nh}\right) =4Ah^3-\frac{B}{nh^2}=0 \quad\Longrightarrow\quad h^5=\frac{B}{4An} =\frac{R(K)}{n\mu_2(K)^2R(f'')}.

Theorem: Under the stated regularity conditions, the bandwidth minimizing the AMISE is $$ h_{\mathrm{AMISE}} = \left\{\frac{R(K)}{n\mu_2(K)^2R(f'')}\right\}^{1/5}. $$ For fixed kernel and density, $h_{\mathrm{AMISE}}$ has order $n^{-1/5}$. At this bandwidth, the leading squared-bias contribution has order $n^{-4/5}$, the integrated-variance contribution has order $n^{-4/5}$, and the AMISE has order $n^{-4/5}$.

The fifth root is a consequence of balancing terms with different powers of $h$: the integrated squared-bias term is proportional to $h^4$, and integrated variance is proportional to $h^{-1}$. At the minimizing bandwidth, the variance contribution is four times the squared-bias contribution. Indeed, the derivative equation implies $B/(nh)=4Ah^4$, so the leading AMISE equals $5Ah^4$, or $5B/(4nh)$. This relation is also a useful arithmetic check.

Worked Example: Calculating the AMISE-Optimal Bandwidth

Suppose $n=500$ independent losses are sampled from a smooth density $f$. A symmetric second-order kernel has $\mu_2(K)=1$ and $R(K)=0.2$, and the density has $R(f'')=0.1$. Calculate the AMISE-optimal bandwidth and the two leading AMISE contributions. Treat the stated quantities as the model inputs; no interest or mortality assumptions enter this density-estimation calculation.

Method. Substitute the inputs into the AMISE bandwidth formula:

h_{\mathrm{AMISE}} = \left\{\frac{0.2}{500(1)^2(0.1)}\right\}^{1/5} =(0.004)^{1/5} \approx 0.3314.

The integrated squared-bias term is $h^4\mu_2(K)^2R(f'')/4=0.025h^4$. Using the unrounded fifth root, $h^4\approx0.01207$, so this contribution is approximately $0.025(0.01207)=0.0003017$. The integrated-variance term is

\frac{R(K)}{nh} = \frac{0.2}{500(0.331445\ldots)} \approx 0.0012068.

Thus the AMISE is approximately $0.0003017+0.0012068=0.0015085$. The variance contribution is approximately four times the squared-bias contribution, as the derivative condition requires. The bandwidth is reported to four decimal places; the intermediate computations retain more precision.

Common mistake avoided: taking a fourth root because the bias-squared term contains $h^4$. The variance term also depends on $h$; differentiating the sum gives a fifth-power equation, not a fourth-power equation.

What the Rate Says About Sample Size

If the kernel and density remain fixed, the constants in the optimal-bandwidth formula do not change with $n$. Multiplying the sample size by $c$ therefore multiplies the optimal bandwidth by $c^{-1/5}$. The bandwidth shrinks, but slowly: increasing the sample size does not justify shrinking $h$ in direct proportion to $1/n$. At the AMISE optimum, the leading error scales as $n^{-4/5}$, since both terms balance at that order.

These statements concern asymptotic orders. They do not assert that every finite-sample density estimate improves uniformly at every $x$. A bandwidth suitable for integrated error can still obscure a narrow peak or show too much local variation, depending on the estimation objective.

Worked Example: Effect of Multiplying the Sample Size

Use the model and kernel inputs of the previous calculation, but increase the independent sample size from $500$ to $16{,}000$. Find the new optimal bandwidth and the ratio of the new AMISE to the original AMISE. Assume the same underlying density and kernel, so $R(f'')$, $\mu_2(K)$, and $R(K)$ are unchanged.

The sample size is multiplied by $16{,}000/500=32=2^5$. Because the optimal bandwidth has order $n^{-1/5}$, the new bandwidth is the old one multiplied by $32^{-1/5}=1/2$:

h_{\mathrm{new}}\approx \frac{0.331445}{2}=0.1657225\approx0.1657.

The leading AMISE has order $n^{-4/5}$, so its ratio is $32^{-4/5}=2^{-4}=1/16$. Using the original AMISE of approximately $0.0015085$, the new AMISE is approximately $0.0015085/16=0.00009428$.

This result can also be checked term by term. Halving $h$ divides the squared-bias term by $2^4=16$. For variance, the denominator $nh$ is multiplied by $32(1/2)=16$, so that term is also divided by 16. Both leading contributions therefore have the claimed ratio. Rounding the bandwidth to four decimal places, the result is $0.1657$.

Common mistake avoided: saying that a 32-fold sample-size increase divides the bandwidth by 32. The fifth-root relation divides it by 2; it is the leading AMISE, not the bandwidth, that is divided by 16.

Comparing Bandwidths Around the Optimum

The optimum results from balancing the two contributions, not from minimizing either one alone. A bandwidth below the optimum reduces leading squared bias but increases variance; a bandwidth above it reduces variance but increases squared bias. The following comparison shows why checking both terms matters.

Worked Example: Choosing Among Three Candidate Bandwidths

Suppose $n=100$ independent observations come from a smooth density, with $R(f'')=0.16$. Use a symmetric second-order kernel with $\mu_2(K)=1$ and $R(K)=0.5$. Compare the AMISE at $h=0.25$, $h=0.5$, and $h=1$. Assume the standard smooth whole-line approximation applies.

Here the squared-bias coefficient is $\mu_2(K)^2R(f'')/4=0.04$, and the variance term is $0.5/(100h)=0.005/h$. Thus $\operatorname{AMISE}(h)\approx0.04h^4+0.005/h$. The formula gives

h_{\mathrm{AMISE}} = \left\{\frac{0.5}{100(1)^2(0.16)}\right\}^{1/5} =(0.03125)^{1/5}=0.5,

because $0.5^5=0.03125$. Now calculate each candidate:

At $h=0.25$, $h^4=0.00390625$. The squared-bias contribution is $0.04(0.00390625)=0.00015625$, the variance contribution is $0.005/0.25=0.02$, and AMISE is $0.02015625$.
At $h=0.5$, $h^4=0.0625$. The squared-bias contribution is $0.04(0.0625)=0.0025$, the variance contribution is $0.005/0.5=0.01$, and AMISE is $0.0125$.
At $h=1$, the squared-bias contribution is $0.04(1)=0.04$, the variance contribution is $0.005/1=0.005$, and AMISE is $0.045$.

Among these candidates, $h=0.5$ gives the smallest AMISE, consistent with the derivative calculation. At that bandwidth the variance contribution $0.01$ is four times the squared-bias contribution $0.0025$.

Common mistake avoided: selecting the narrowest bandwidth on the grounds that it has the smallest bias. At $h=0.25$, the squared-bias contribution is indeed smaller than at $h=0.5$, but its much larger variance makes the total AMISE worse.

Exam Traps and Interpretation

Using the wrong root. Because the variance term contains $h^{-1}$, differentiation balances $h^3$ against $h^{-2}$ and produces $h^5$, not $h^4$.
Dropping a constant. The squared-bias coefficient is $\mu_2(K)^2R(f'')/4$. Omitting the factor $1/4$ changes the bandwidth constant.
Confusing orders with exact values. The statements $h_{\mathrm{AMISE}}$ has order $n^{-1/5}$ and AMISE has order $n^{-4/5}$ describe how quantities scale with sample size, not their numerical values without the kernel and density constants.
Applying the expansion outside its conditions. The standard expression assumes adequate smoothness and no unaddressed boundary effect. A density supported only on nonnegative losses may require special care near zero.
Rounding too early. Keep the fifth root unrounded when calculating the two AMISE terms; round only the reported result.

The asymptotic result is aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus. Its practical value is that it makes the competing effects explicit and provides a benchmark for reasoning about smoothing. It is not a substitute for checking whether the assumptions suit the data and the estimation target.

Check Your Understanding

For each question, use the standard smooth-density AMISE expressions where applicable. Retain adequate precision in intermediate calculations.

For $n=1{,}000$, $\mu_2(K)=1$, $R(K)=0.4$, and $R(f'')=0.2$, calculate $h_{\mathrm{AMISE}}$ to four decimal places. (Answer: $0.2885$)
If the sample size is multiplied by 4 while the kernel and density remain fixed, by what factor does the AMISE-optimal bandwidth change? By what factor does the leading AMISE change? (Answer: bandwidth factor $4^{-1/5}\approx0.7579$; AMISE factor $4^{-4/5}\approx0.3299$)
For $n=200$, $\mu_2(K)=2$, $R(K)=0.3$, and $R(f'')=0.05$, calculate the AMISE-optimal bandwidth to four decimal places. (Answer: $(0.0075)^{1/5}\approx0.3758$)
A proposed sequence of bandwidths is $h=n^{-1/3}$. State the orders of the leading integrated squared-bias term and integrated-variance term, and identify which dominates as $n$ grows. (Answer: squared bias has order $n^{-4/3}$, variance has order $n^{-2/3}$, and variance dominates; the sequence satisfies $h\to0$ and $nh\to\infty$, but is not the AMISE-optimal rate.)
At the AMISE-optimal bandwidth, suppose the squared-bias contribution is $C$, where $C>0$. What is the leading AMISE after doubling the bandwidth, expressed in terms of $C$, and what is the ratio to the optimal leading AMISE? (Answer: $18C$; the optimal leading AMISE is $5C$, so the ratio is $18/5=3.6$)

What You'll Learn