Smoothing Prior Measures Without Losing Risk Bounds

Why Smooth a Prior Measure?

In admissibility arguments, integrated risk is often evaluated against a prior measure. A measure may be discrete, singular, or otherwise inconvenient when a proof needs a prior density. Smoothing replaces it with a nearby measure that has a smooth density. The key question is not merely whether the smoothed measure is close in some sense: it is whether the integrated risk changes by a controlled amount.

The central issue is the risk function's behavior under small changes in the parameter. Continuity gives pointwise control, while an integrable bound permits that control to pass through the integral. The integrability requirement is essential: a small difference between two formal integrands does not make either integral well-defined.

Convolution Smoothing

Definition: Let $\mu$ be a finite Borel measure on $\mathbb{R}^d$. Let $\kappa$ be a probability density on $\mathbb{R}^d$. For $\varepsilon>0$, the smoothed measure $\mu_\varepsilon$ is the pushforward of the product measure $\mu(d\theta)\kappa(u)\,du$ under the map $(\theta,u)\mapsto\theta+\varepsilon u$. Equivalently, for every nonnegative Borel function $h$, $$ \int h(x)\,\mu_\varepsilon(dx) = \int_{\mathbb{R}^d}\int_{\mathbb{R}^d} h(\theta+\varepsilon u)\,\kappa(u)\,du\,\mu(d\theta). $$

When $\kappa$ is a $C^\infty$ density with compact support, $\mu_\varepsilon$ has the density

q_\varepsilon(x) = \int_{\mathbb{R}^d} \varepsilon^{-d}\kappa\!\left(\frac{x-\theta}{\varepsilon}\right)\mu(d\theta).

This density is smooth: each derivative of the kernel is bounded, and differentiating the displayed integral is justified by domination with a constant integrable under the finite measure $\mu$. The total mass is preserved, since Tonelli's theorem gives $\int q_\varepsilon(x)\,dx=\mu(\mathbb{R}^d)$. Thus a probability prior remains a probability prior. Smoothing does not discard mass; it spreads the mass over a neighborhood whose scale is set by $\varepsilon$.

For parameter spaces restricted to a subset of $\mathbb{R}^d$, the convolution can place mass outside that subset. Any application must therefore ensure that the model and risk are defined on the resulting support, or use a smoothing procedure that stays within the parameter space.

A Risk-Convergence Theorem

For a fixed decision rule, write $R(\theta)$ for its risk at parameter $\theta$. The integrated risk under a finite measure $\mu$ is $\int R\,d\mu$, provided this integral is defined. The following result gives explicit sufficient conditions for smoothing to preserve it.

Theorem: Let $\mu$ be a finite Borel measure on $\mathbb{R}^d$, and let $\kappa$ be a probability density supported on the unit ball. Let $R:\mathbb{R}^d\to\mathbb{R}$ be Borel measurable, continuous at $\mu$-almost every $\theta$, and satisfy $R\in L^1(\mu)$. Suppose there are $\varepsilon_0>0$ and a nonnegative $G\in L^1(\mu)$ such that, for every $0<\varepsilon\leq\varepsilon_0$, $$ |R(\theta+\varepsilon u)-R(\theta)|\leq G(\theta) $$ for $\mu$-almost every $\theta$ and $\kappa(u)\,du$-almost every $u$. Then the smoothed risks are finite and $$ \lim_{\varepsilon\downarrow0}\int R(x)\,\mu_\varepsilon(dx) = \int R(\theta)\,\mu(d\theta). $$

Proof. By the definition of $\mu_\varepsilon$, the smoothed integral is the integral of $R(\theta+\varepsilon u)$ over the product measure $\mu(d\theta)\kappa(u)\,du$. The hypotheses imply

|R(\theta+\varepsilon u)| \leq |R(\theta)|+G(\theta).

The right side is integrable under the product measure: its integral is $\int |R|\,d\mu+\int G\,d\mu<\infty$, because $\kappa$ integrates to one. Thus each smoothed integral is absolutely finite. For $\mu$-almost every $\theta$, continuity at $\theta$ implies $R(\theta+\varepsilon u)\to R(\theta)$ as $\varepsilon\downarrow0$ for every $u$. The difference is bounded by $G(\theta)$, which is integrable under the product measure. The dominated convergence theorem therefore yields

\int\!\!\int \bigl(R(\theta+\varepsilon u)-R(\theta)\bigr) \,\kappa(u)\,du\,\mu(d\theta) \longrightarrow 0.

The left side is the difference between the smoothed and original integrated risks. This proves the claim. $\square$

A Quantitative Bound for Lipschitz Risks

A convenient special case replaces the envelope condition with a global modulus of continuity. A function $R$ is $L$-Lipschitz if $|R(x)-R(y)|\leq L\|x-y\|$ for all $x,y$. If $R\in L^1(\mu)$, $\mu$ is finite, and $\kappa$ has finite first moment, then

\left| \int R\,d\mu_\varepsilon-\int R\,d\mu \right| \leq L\varepsilon\,\mu(\mathbb{R}^d) \int \|u\|\kappa(u)\,du.

Indeed, express the difference as the product-measure integral of $R(\theta+\varepsilon u)-R(\theta)$, take its absolute value, and apply the triangle inequality for integrals. The Lipschitz condition bounds the integrand by $L\varepsilon\|u\|$, so

\left|\int\!\!\int \bigl(R(\theta+\varepsilon u)-R(\theta)\bigr) \,\kappa(u)\,du\,\mu(d\theta)\right| \leq \int\!\!\int L\varepsilon\|u\|\,\kappa(u)\,du\,\mu(d\theta),

which is the stated bound. It also establishes absolute integrability of the difference. Since $R\in L^1(\mu)$, the smoothed integral is finite as well. For a kernel supported on the unit ball, $\int\|u\|\kappa(u)\,du\leq1$, so the risk change is at most $L\varepsilon\mu(\mathbb{R}^d)$.

Worked Examples

Worked Example: Smoothing a Two-Point Prior

Consider the normal location model $X\sim N(\theta,1)$, the constant estimator $\delta(X)=1$, and squared-error loss. Its risk is $R(\theta)=(1-\theta)^2$, since the estimation error is $1-\theta$. Let $\mu=0.4\delta_0+0.6\delta_2$, a probability measure on $\mathbb{R}$.

Choose any symmetric $C^\infty$ probability density $\kappa$ supported on $[-1,1]$, and let $U$ have this density. Then $E[U]=0$. Under the original prior,

\int R(\theta)\,\mu(d\theta) =0.4(1-0)^2+0.6(1-2)^2=1.

Under the smoothed prior, the parameter is $\theta+\varepsilon U$. Expanding the squared term and using $E[U]=0$,

\begin{aligned} \int R(x)\,\mu_\varepsilon(dx) &=E\!\left[\{1-(\Theta+\varepsilon U)\}^2\right]\\ &=E[(1-\Theta)^2] -2\varepsilon E[(1-\Theta)]E[U] +\varepsilon^2E[U^2]\\ &=1+\varepsilon^2E[U^2]. \end{aligned}

Since $|U|\leq1$, $0\leq E[U^2]\leq1$, so the increase is at most $\varepsilon^2$. In particular, the smoothed risk converges to the original integrated risk. The resulting prior has a smooth density even though the original prior has only two atoms.

Worked Example: Smoothing an Irregular Prior

Let $\mu$ be the Cantor probability measure on $[0,1]$, which is singular with respect to Lebesgue measure. Consider the nonnegative continuous risk function $R(\theta)=1+\sin^2(\theta)$, and smooth $\mu$ using a probability density $\kappa$ supported on the unit ball in $\mathbb{R}$.

\left|\int R\,d\mu_\varepsilon-\int R\,d\mu\right| \leq \varepsilon.

Here $R$ is bounded, so it is integrable under $\mu$, and all the displayed integrals are finite. The bound proves convergence without requiring a density for the original prior or an explicit formula for its integral. The smoothed prior has a density, while the risk difference is controlled directly.

Worked Example: An Exact Change Under a Uniform Prior

In the normal location model $X\sim N(\theta,1)$, use the constant estimator $\delta(X)=0$ and squared-error loss. Its risk is $R(\theta)=\theta^2$. Let $\mu$ be uniform on $[0,1]$, and smooth it using a symmetric probability density $\kappa$ supported on $[-1,1]$. Write $m_2=E[U^2]$, so $0\leq m_2\leq1$.

The original integrated risk is

\int_0^1\theta^2\,d\theta =\left[\frac{\theta^3}{3}\right]_0^1 =\frac{1}{3}.

For the smoothed parameter $\Theta+\varepsilon U$, independence and symmetry give $E[U]=0$. Therefore

\begin{aligned} E[(\Theta+\varepsilon U)^2] &=E[\Theta^2]+2\varepsilon E[\Theta]E[U] +\varepsilon^2E[U^2]\\ &=\frac{1}{3}+\varepsilon^2m_2. \end{aligned}

The absolute change is $\varepsilon^2m_2\leq\varepsilon^2$, and hence tends to zero. The normal location model is defined for all real parameter values, so the small amount of smoothed mass outside $[0,1]$ causes no problem in this example.

Common Pitfall: A Difference Bound Is Not Enough

It is tempting to assume that continuity and a bound on $|R(\theta+\varepsilon u)-R(\theta)|$ suffice for convergence of integrated risks. They do not: the original signed integral must itself be defined. For example, let $\mu$ be the standard Cauchy probability measure on $\mathbb{R}$, $R(\theta)=\theta$, and $\kappa$ be supported on $[-1,1]$. Then $R$ is continuous and

|R(\theta+\varepsilon u)-R(\theta)| =\varepsilon|u|\leq1 \qquad (0<\varepsilon\leq1).

Nevertheless, $\int \theta\,\mu(d\theta)$ is undefined as a Lebesgue integral: its positive and negative parts both have infinite integral. Thus the difference of integrated risks in the proposed conclusion is not meaningful. The correction is to require $R\in L^1(\mu)$, as in the theorem. With that condition, the envelope also gives $|R(\theta+\varepsilon u)|\leq|R(\theta)|+G(\theta)$, ensuring the smoothed integrals are finite before dominated convergence is applied.

In an admissibility argument, this distinction matters when a smoothed prior is used to transfer an integrated-risk comparison from one measure to another. Verify that the risks being integrated are finite, then establish either an integrable envelope or a quantitative continuity bound. Smoothing makes the prior regular; it does not by itself make an ill-defined risk integral valid.

Key takeaway: Convolution with a smooth probability density can turn a discrete or singular finite prior into a smooth density while preserving integrated risk in the small-bandwidth limit. The essential safeguards are integrability of the original risk and sufficient control of its change under small parameter shifts.

Check Your Understanding

Use the definitions and bounds above to answer the following questions.

Why does convolution with a probability density preserve the total mass of a finite prior measure?
In the risk-convergence theorem, where are $R\in L^1(\mu)$ and $G\in L^1(\mu)$ used in the proof?
If $R$ is $L$-Lipschitz, $\mu$ has mass $M$, and $\kappa$ is supported on the unit ball, what upper bound follows for the absolute integrated-risk change?
For the two-point prior example, why does symmetry of the smoothing density remove the term linear in $\varepsilon$?
Why does a uniform bound on $|R(\theta+\varepsilon u)-R(\theta)|$ not by itself guarantee that the original integrated risk exists?