Diffuse Prior Scaling on Euclidean Parameter Spaces

Why Scaling a Prior Changes an Integrated Risk Gap

In admissibility arguments, a prior can be spread over an increasingly large parameter space so that it remains proper while its mass becomes diffuse. The normalization needed to keep total mass equal to one matters: it can make the prior mass of every fixed bounded region tend to zero. Consequently, a strict risk improvement on a fixed region may produce an integrated risk gap that vanishes, even though the improvement itself has not disappeared.

The key calculation is a change of scale. It identifies the density's local order, explains how much mass remains in a growing region, and shows why the behavior of the risk difference in the tails cannot be ignored.

Scaling a Proper Density

Definition: Let $p$ be a positive integer and let $g$ be a probability density on $\mathbb{R}^p$, with respect to Lebesgue measure. For a scale $a>0$, define the scaled prior density $$ \pi_a(\theta)=a^{-p}g(\theta/a), \qquad \theta\in\mathbb{R}^p. $$ This is a proper density. A sequence with $a\to\infty$ is called a diffuse scaling of $g$.

The factor $a^{-p}$ compensates for expanding each of the $p$ coordinates by a factor of $a$. In fact, the substitution $u=\theta/a$ gives

\int_{\mathbb{R}^p}\pi_a(\theta)\,d\theta =\int_{\mathbb{R}^p}a^{-p}g(\theta/a)\,d\theta =\int_{\mathbb{R}^p}g(u)\,du =1.

For a measurable risk difference $D(\theta)$, its integrated risk difference under this prior is

I_a(D)=\int_{\mathbb{R}^p}D(\theta)\pi_a(\theta)\,d\theta =\int_{\mathbb{R}^p}D(au)g(u)\,du,

whenever the integral exists. We use $D(\theta)=R(\theta,\delta_0)-R(\theta,\delta_1)$ when positive values represent a risk improvement by $\delta_1$ over $\delta_0$. The last expression makes an important feature visible: as $a$ grows, the prior evaluates the risk difference at parameter values $au$. For almost every fixed $u\ne 0$, these values move out into the tails.

Fixed Regions Lose Mass at a Predictable Rate

Suppose $g$ is continuous at the origin. On any fixed bounded region, $\pi_a(\theta)$ is approximately $a^{-p}g(0)$ for large $a$. More precisely, if $K\subset\mathbb{R}^p$ is bounded and Lebesgue measurable, then

a^pP_{\pi_a}(\theta\in K) =\int_K g(\theta/a)\,d\theta \longrightarrow g(0)\,|K|,

where $|K|$ denotes its Lebesgue measure. To verify the limit, boundedness of $K$ implies $\sup_{\theta\in K}\|\theta/a\|\to 0$. Continuity at zero therefore gives uniform convergence of $g(\theta/a)$ to $g(0)$ on $K$. Integrating this uniform convergence over the finite-measure set $K$ proves the displayed limit.

A corresponding result applies directly to a localized risk difference. If $D$ is measurable, integrable, and zero outside a bounded set, then

Theorem: If $g$ is a probability density continuous at zero and $D$ is Lebesgue integrable with bounded support, then $$ a^p I_a(D)\longrightarrow g(0)\int_{\mathbb{R}^p}D(\theta)\,d\theta. $$

Proof. Choose a bounded measurable set $K$ outside which $D=0$. For every $a>0$,

a^p I_a(D)=\int_K D(\theta)g(\theta/a)\,d\theta.

For all sufficiently large $a$, the arguments $\theta/a$, $\theta\in K$, lie in a fixed neighborhood of zero on which $g$ is bounded, since $g$ is continuous there. Thus the integrands are bounded in absolute value by a constant times $|D(\theta)|$, an integrable function. Also, $g(\theta/a)\to g(0)$ for every $\theta\in K$. The dominated convergence theorem yields the stated limit. $\square$

Thus, if the integral of $D$ is finite, a compactly supported gain typically contributes at order $a^{-p}$ to the normalized integrated risk difference. This is a statement about prior weighting, not about whether $D$ is positive or negative at any particular parameter value.

Worked Example: Uniform Scaling and a Compact Improvement

Consider the one-dimensional prior uniform on $[-a,a]$, so $g(u)=\tfrac12\mathbf{1}_{[-1,1]}(u)$ and $\pi_a(\theta)=\tfrac{1}{2a}\mathbf{1}_{[-a,a]}(\theta)$. In a no-observation decision problem with two actions, let the losses be $L(\theta,\delta_0)=1$ and $L(\theta,\delta_1)=1-\mathbf{1}_{[-1,1]}(\theta)$. Both losses are nonnegative, and the gain $D=R(\cdot,\delta_0)-R(\cdot,\delta_1)$ equals $\mathbf{1}_{[-1,1]}$.

For $a\geq 1$, the prior contains the full improvement region, whose length is two. Therefore

I_a(D)=\int_{-1}^{1}\frac{1}{2a}\,d\theta =\frac{2}{2a} =\frac{1}{a}.

At $a=10$, this integrated gain is $0.1$; at $a=100$, it is $0.01$. The rule $\delta_1$ still improves risk by one throughout $[-1,1]$. The smaller integrated gain comes entirely from the prior's decreasing density there.

Tail Mass Can Determine the Limit

A fixed compact region has vanishing prior mass, so the prior's remaining mass must lie outside it. For a measurable set $A$, the exact mass identity is

P_{\pi_a}(\theta\in A)=\int_{A/a}g(u)\,du, \qquad P_{\pi_a}(\theta\notin A)=1-\int_{A/a}g(u)\,du.

Here $A/a=\{u:au\in A\}$. In particular, for a ball of radius $r$ centered at zero, its mass is the probability that a random vector with density $g$ lies in a ball of radius $r/a$. If the radius itself grows as $r_a=ca$, for fixed $c>0$, the enclosed mass is instead the fixed probability that the vector lies within radius $c$. An expanding prior does not make every expanding region negligible.

This rescaling also gives a tail-limit result for bounded risk differences. Suppose $D$ is measurable and bounded, and $D(\theta)\to d_\infty$ as $\|\theta\|\to\infty$. Then

Theorem: For a diffuse scaling of any probability density $g$ on $\mathbb{R}^p$, $$ I_a(D)\longrightarrow d_\infty. $$

Proof. Write $I_a(D)=\int D(au)g(u)\,du$. For every $u\ne 0$, $\|au\|\to\infty$, so $D(au)\to d_\infty$. The singleton $\{0\}$ has Lebesgue measure zero, and hence probability zero under the density $g$. Since $D$ is bounded, the integrands are bounded in absolute value by a constant, which is integrable with respect to the probability density $g$. The dominated convergence theorem therefore gives

\lim_{a\to\infty}I_a(D) =\int_{\mathbb{R}^p}d_\infty g(u)\,du =d_\infty.

$\square$

A compactly supported $D$ has $d_\infty=0$, consistent with the previous theorem. But if the risk difference approaches a nonzero constant in the tails, the diffuse integrated difference approaches that constant, rather than zero.

Worked Example: A Gaussian Prior and a Smooth Local Gain

Let $p=1$, take $g$ to be the $N(0,1)$ density, and use the scaled prior $N(0,a^2)$. In a no-observation problem, set $L(\theta,\delta_0)=1$ and $L(\theta,\delta_1)=1-e^{-\theta^2}$. The losses are nonnegative, and the gain is $D(\theta)=e^{-\theta^2}$, which is positive everywhere and tends to zero in the tails.

The integrated gain can be calculated exactly:

I_a(D) =\frac{1}{\sqrt{2\pi}a}\int_{\mathbb{R}} \exp\!\left(-\theta^2-\frac{\theta^2}{2a^2}\right)d\theta =\frac{1}{\sqrt{1+2a^2}}.

For the last equality, combine the exponents as $-\tfrac12(2+a^{-2})\theta^2$ and integrate the resulting centered Gaussian kernel. At $a=10$, the gain is $1/\sqrt{201}$, approximately $0.0705$. Moreover, $aI_a(D)=a/\sqrt{1+2a^2}\to 1/\sqrt{2}$, agreeing with the compact-local scaling calculation's constant $g(0)\int e^{-\theta^2}d\theta=(1/\sqrt{2\pi})\sqrt{\pi}=1/\sqrt{2}$. The gain is not compactly supported, but its integrable tails yield the same first-order constant.

Worked Example: Tail Probability Under an Expanding Two-Dimensional Prior

Let $\theta\in\mathbb{R}^2$ have prior $N(0,a^2I_2)$, where $I_2$ is the two-dimensional identity matrix. Suppose a decision rule has gain $D(\theta)=\mathbf{1}_{\{\|\theta\|\leq 1\}}$, as in the two-action loss construction above. The integrated gain is the prior probability of the unit disk.

The radial density of this centered isotropic Gaussian gives $P(\|\theta\|\leq r)=1-\exp(-r^2/(2a^2))$. Substituting $r=1$,

I_a(D)=1-\exp\!\left(-\frac{1}{2a^2}\right).

At $a=5$, this is $1-e^{-0.02}$, approximately $0.0198$. As $a\to\infty$, the expansion $1-e^{-x}\sim x$ at zero gives $I_a(D)\sim 1/(2a^2)$, matching the $a^{-p}$ rate with $p=2$. In contrast, the mass inside the disk of radius $a$ is $1-e^{-1/2}$, approximately $0.3935$, and the mass outside it is $e^{-1/2}$, approximately $0.6065$. The substantial tail mass at this expanding radius is not a contradiction: its radius grows proportionally to the prior's scale.

Interpreting Diffuse-Prior Risk Gaps

The calculations distinguish two questions that are easy to conflate. First, how large is the risk improvement at a parameter value? Second, how much prior mass is placed where that improvement occurs? A fixed local improvement can remain strict while its normalized integrated contribution tends to zero. Conversely, a risk difference that remains substantial far into the tails can dominate the diffuse-prior limit because most prior mass escapes every fixed bounded region.

When bounding a gap, splitting the integral makes this dependence explicit. For any measurable $K$,

|I_a(D)| \leq \int_K |D(\theta)|\pi_a(\theta)\,d\theta +\int_{K^c}|D(\theta)|\pi_a(\theta)\,d\theta.

The first term can be small because fixed $K$ has small prior mass. That fact alone says nothing about the second term. To control the tail contribution, one needs information about $D$ there, such as a bound that decreases with $\|\theta\|$, or a direct bound on its prior-weighted tail integral. The appropriate bound depends on the risk difference and the prior family; normalization by itself does not provide it.

For a compactly supported gain, the normalized integral is typically of order $a^{-p}$.
For a bounded gain with a limit at infinity, the diffuse integrated gap converges to that tail limit.
For a growing region, compute its mass after rescaling its radius by $a$; a fixed-radius calculation does not describe it.

Common Mistake

A tempting but incorrect argument says: “The prior density is small on every fixed bounded set, so the integrated risk difference tends to zero.” This conclusion ignores the integral over the complement. For example, if $D(\theta)=1$ for every $\theta$, then $I_a(D)=1$ for every $a$, since $\pi_a$ integrates to one. The correct argument separates the fixed-region contribution from the tail contribution and controls both. For compactly supported $D$, the tail term is exactly zero; for a risk difference that persists in the tails, it may be decisive.

Key takeaway: Diffuse scaling preserves total prior mass but sends the mass of each fixed bounded region toward zero. A vanishing integrated risk gap on such priors can reflect this normalization, so its meaning depends on the risk difference's behavior outside the region where it is locally informative.

Check Your Understanding

Use the scaling identity and distinguish fixed-region mass from tail behavior.

For $\pi_a(\theta)=a^{-p}g(\theta/a)$, verify by substitution that $\pi_a$ integrates to one.
If $g$ is continuous at zero and $K$ is bounded with finite positive volume, what is the limit of $a^pP_{\pi_a}(\theta\in K)$?
In the one-dimensional uniform example, what is the integrated gain for $a\geq 1$, and why does it shrink although the gain on $[-1,1]$ is unchanged?
Suppose $D$ is bounded and tends to a constant $d_\infty$ as $\|\theta\|\to\infty$. What does $I_a(D)$ converge to, and which convergence theorem justifies the result?
Why does knowing that a fixed compact set has vanishing prior mass not, by itself, establish that the full integrated risk difference vanishes?