Weak Convergence of Prior Sequences and Risk Integrals

Why Convergence of Priors Is Not Enough by Itself

In admissibility arguments, a sequence of priors can approximate a limiting prior while the integrated risks are used to compare decision rules. The key question is whether convergence of the priors also forces convergence of those risk integrals. The answer depends on the risk function: weak convergence directly controls bounded continuous functions, but it does not automatically control discontinuities or large values in the tails.

Let \(\Theta\) be a metric parameter space, let \(P_\theta\) describe the distribution of the data at parameter \(\theta\), and let \(L(\theta,a)\) be a nonnegative loss for taking action \(a\). For a measurable decision rule \(\delta\), its risk is

R(\theta,\delta)=E_\theta[L(\theta,\delta(X))].

Assume this risk is measurable as a function of \(\theta\). For a prior probability measure \(\pi\), its integrated risk is

r(\pi,\delta)=\int_\Theta R(\theta,\delta)\,\pi(d\theta),

when the integral is defined. The results below concern how this integral changes as \(\pi\) varies; the decision rule is fixed unless stated otherwise.

Definition: A sequence of probability measures \(\pi_n\) on \(\Theta\) converges weakly to a probability measure \(\pi\), written \(\pi_n\Rightarrow\pi\), if \(\int g\,d\pi_n\to\int g\,d\pi\) for every bounded continuous real-valued function \(g\) on \(\Theta\). Weak convergence tests priors using exactly this class of functions.

Bounded Continuous Risks

For a fixed rule, if its risk is bounded and continuous in the parameter, the definition of weak convergence applies directly. No separate tail argument is needed: boundedness ensures the risk is an allowed test function, and continuity is the other required property.

Theorem: Suppose \(\pi_n\Rightarrow\pi\) on a metric space \(\Theta\), and \(R(\cdot,\delta)\) is bounded and continuous. Then \(r(\pi_n,\delta)\to r(\pi,\delta)\).

Proof. Write \(g(\theta)=R(\theta,\delta)\). By hypothesis, \(g\) is a bounded continuous real-valued function. The definition of weak convergence therefore gives

r(\pi_n,\delta) =\int_\Theta g(\theta)\,\pi_n(d\theta) \longrightarrow \int_\Theta g(\theta)\,\pi(d\theta) =r(\pi,\delta).

This is a direct application of the definition, not a general guarantee for every measurable risk function. \(\square\)

Worked Example: A Bounded Continuous Risk

Take \(\Theta=\mathbb{R}\), and for integers \(n\ge 2\) define \(\pi_n=(1-1/n)\delta_0+(1/n)\delta_n\), where \(\delta_x\) is a point mass at \(x\). First check the weak limit. For every bounded continuous \(g\),

\int g\,d\pi_n =\left(1-\frac1n\right)g(0)+\frac1n g(n) \longrightarrow g(0),

because \(g(n)\) is bounded and \(1/n\to0\). Thus \(\pi_n\Rightarrow\delta_0\). Now take the bounded continuous risk \(R(\theta,\delta)=1/(1+\theta^2)\). Its integrated risk is

r(\pi_n,\delta) =\left(1-\frac1n\right)\cdot 1+ \frac1n\cdot\frac{1}{1+n^2} \longrightarrow 1 =r(\delta_0,\delta).

The prior puts some mass at increasingly distant parameter values, but that mass vanishes. For this bounded continuous risk, the weak-convergence calculation still determines the limit.

Why Continuity and Boundedness Matter

The definition only tests bounded continuous functions. A risk can fail to belong to that class in two different ways. If it is discontinuous, it may distinguish a limiting parameter from nearby parameters. If it is unbounded, a small amount of prior mass can contribute a large amount to the integral. Either issue can prevent integrated risks from converging.

Worked Example: A Bounded but Discontinuous Risk

Let \(\pi_n=\delta_{1/n}\) on \(\mathbb{R}\). For every bounded continuous \(g\), \(g(1/n)\to g(0)\), so \(\pi_n\Rightarrow\delta_0\). Consider the bounded measurable function \(R(\theta,\delta)=\mathbf{1}_{\{0\}}(\theta)\), which is discontinuous at \(0\). Then

r(\pi_n,\delta)=\int \mathbf{1}_{\{0\}}(\theta)\,\delta_{1/n}(d\theta)=0 \quad\text{for every }n,

whereas

r(\delta_0,\delta)=\int \mathbf{1}_{\{0\}}(\theta)\,\delta_0(d\theta)=1.

Thus the integrated risks do not converge to the limiting integrated risk. The issue is not unboundedness: it is that weak convergence does not require convergence of integrals for functions discontinuous at a point receiving positive mass under the limit.

Worked Example: A Continuous Risk With Uncontrolled Tails

Use \(\pi_n=(1-1/n)\delta_0+(1/n)\delta_n\) again, so that \(\pi_n\Rightarrow\delta_0\), and take the continuous but unbounded risk \(R(\theta,\delta)=\theta^2\). For every \(n\),

r(\pi_n,\delta) =\left(1-\frac1n\right)0^2+\frac1n n^2 =n, \qquad r(\delta_0,\delta)=0.

The prior mass at \(n\) is only \(1/n\), but its contribution to the risk integral is \(n\). In particular, weak convergence alone cannot rule out an increasingly large contribution from the tails.

Uniform Integrability Controls Unbounded Risks

For an unbounded risk, a useful extra condition is uniform integrability of its values under the prior sequence. For a nonnegative measurable function \(R\), the relevant tail condition is

\lim_{M\to\infty}\sup_{n\ge1} \int_\Theta R(\theta)\mathbf{1}_{\{R(\theta)>M\}}\,\pi_n(d\theta)=0.

It says that the contribution to the risk integral from values above a large threshold is uniformly small across the sequence. The limit prior must also have a vanishing tail contribution. This extra control is separate from weak convergence.

Theorem: Suppose \(\pi_n\Rightarrow\pi\), and \(R:\Theta\to[0,\infty)\) is continuous. Assume \[ \lim_{M\to\infty}\sup_{n\ge1}\int R\mathbf{1}_{\{R>M\}}\,d\pi_n=0 \quad\text{and}\quad \lim_{M\to\infty}\int R\mathbf{1}_{\{R>M\}}\,d\pi=0. \] Then \(\int R\,d\pi_n\to\int R\,d\pi\).

Proof. For \(M>0\), set \(R_M(\theta)=\min\{R(\theta),M\}\). Since \(R\) is continuous, \(R_M\) is continuous; it is also bounded. Weak convergence therefore gives

\int R_M\,d\pi_n\longrightarrow\int R_M\,d\pi \qquad\text{for each fixed }M.

For nonnegative \(R\), the truncation error satisfies \(0\le R-R_M\le R\mathbf{1}_{\{R>M\}}\). Consequently,

\left|\int R\,d\pi_n-\int R\,d\pi\right| \le \left|\int R_M\,d\pi_n-\int R_M\,d\pi\right| +\int R\mathbf{1}_{\{R>M\}}\,d\pi_n +\int R\mathbf{1}_{\{R>M\}}\,d\pi.

For fixed \(M\), the first term tends to zero as \(n\to\infty\). Taking the limit superior in \(n\) bounds the remaining error by the two tail terms, with the first bounded by its supremum over \(n\). By the assumed tail conditions, both bounds tend to zero as \(M\to\infty\). Hence the integrated risks converge. \(\square\)

Worked Example: Normal Priors and a Quadratic Risk

Let \(\pi_n=N(0,1+1/n)\) and \(\pi=N(0,1)\), with the second argument denoting variance. To verify weak convergence, let \(Z\sim N(0,1)\). For bounded continuous \(g\), dominated convergence gives

\int g\,d\pi_n =E[g(\sqrt{1+1/n}\,Z)] \longrightarrow E[g(Z)] =\int g\,d\pi.

Thus \(\pi_n\Rightarrow\pi\). Take \(R(\theta,\delta)=\theta^2\), which is continuous and unbounded. Directly,

\int \theta^2\,d\pi_n=1+\frac1n\longrightarrow 1=\int \theta^2\,d\pi.

The tail condition also holds. If \(V_n\sim N(0,1+1/n)\), then \(E[V_n^4]=3(1+1/n)^2\le12\) for \(n\ge1\). On the event \(V_n^2>M\), \(V_n^2\le V_n^4/M\), so

\sup_{n\ge1}E[V_n^2\mathbf{1}_{\{V_n^2>M\}}] \le \frac{12}{M}\longrightarrow0.

The limit distribution has fourth moment \(3\), giving the same bound with \(3/M\). Here the unbounded risks converge because their tail contributions are controlled, not merely because the priors converge weakly.

Using the Distinction in Risk Comparisons

When integrated risks appear in prior-sequence arguments, first identify the function being integrated. For a fixed rule, it is its risk \(R(\theta,\delta)\). If that risk is bounded and continuous, weak convergence is sufficient. If it is unbounded, establish a tail condition such as uniform integrability before passing to the limit. If it is discontinuous, weak convergence alone does not generally settle the integral; additional facts about the discontinuity set or the particular sequence are needed.

A common mistake is to argue that \(\pi_n\Rightarrow\pi\), so \(\int R\,d\pi_n\to\int R\,d\pi\) for any risk \(R\). That is the definition of weak convergence with the crucial words “bounded continuous” omitted. The point-mass example with \(R(\theta)=\theta^2\) shows why the omission matters: a vanishing amount of prior mass can have a non-vanishing, or even diverging, risk contribution. Before taking an integrated-risk limit, check the regularity and tail behavior of the risk itself.

Key takeaway: Weak convergence passes integrated risks to the limit for bounded continuous risks. For unbounded continuous risks, control the tails; for discontinuous risks, verify an appropriate additional condition rather than relying on weak convergence alone.

Check Your Understanding

For each question, distinguish what weak convergence guarantees from what requires an additional assumption.

State the definition of weak convergence used for a sequence of prior probability measures.
Why does a bounded continuous risk have convergent integrated risks under weakly convergent priors?
For \(\pi_n=\delta_{1/n}\) and \(R(\theta)=\mathbf{1}_{\{0\}}(\theta)\), calculate the integrals under \(\pi_n\) and under the weak limit.
For \(\pi_n=(1-1/n)\delta_0+(1/n)\delta_n\) and \(R(\theta)=\theta^2\), calculate the integrated risk and explain why weak convergence does not control it.
What tail condition allows the truncation proof to extend convergence of integrated risks to an unbounded nonnegative continuous risk?