Why Risk Geometry Depends on Randomization
In a finite decision problem, each decision rule has a risk at every parameter value. Listing those risks as a vector turns comparison of procedures into a geometric problem. But the set of vectors from deterministic rules need not be convex: a line segment between two attainable risks may contain points that no deterministic rule attains. A lottery over rules fills in that segment.
This matters for Wald-type complete-class arguments because separation is naturally applied to convex sets. Randomization supplies convexity in the attainable risks; it is not merely a device for making a procedure seem less certain. The central point here is precise: mixtures combine risks linearly, and a boundary point produced by that combination may be unavailable to every deterministic rule.
Finite Decision Problems and Risk Vectors
A randomized rule can be implemented by a random choice between deterministic rules, made independently of the unknown parameter and the observation. For example, choose \(\delta_1\) with probability \(\lambda\) and \(\delta_2\) with probability \(1-\lambda\), where \(0\leq\lambda\leq1\). The resulting risk at each parameter is the same weighted average of the two risks.
Proof. First draw an independent random variable \(J\) that equals 1 with probability \(\lambda\) and 2 with probability \(1-\lambda\). After observing \(X\), take action \(\delta_J(X)\). At any fixed parameter value \(\theta\), the expected loss is
The equality follows by conditioning on the two possible values of \(J\), which is independent of \(X\). It holds at every \(\theta\), so it holds coordinate by coordinate for the risk vectors. \(\square\)
Thus the randomized attainable risk set contains every line segment between deterministic risk vectors. In particular, it is convex if we allow lotteries over deterministic rules. In a finite problem, this description is exact, not merely a lower bound.
Proof. The first theorem shows that every finite lottery over deterministic rules has a risk vector in the randomized attainable set. Conversely, let \(q(a\mid x)\) be any randomized rule. Construct a random deterministic rule \(\Delta\) as follows: independently for each \(x\in\mathcal X\), draw \(\Delta(x)\) from the distribution \(q(\cdot\mid x)\). Because \(\mathcal X\) and \(\mathcal A\) are finite, this defines a probability distribution over the finite set of deterministic rules.
For each fixed observation \(x\), the probability that \(\Delta(x)=a\) is \(q(a\mid x)\). Therefore, at every parameter value, the expected loss of the random rule \(\Delta\) is the same as the expected loss of the original randomized rule: condition first on \(X=x\), and then use this equality of action probabilities. Averaging over \(X\) gives equality of risks at every parameter. The risk vector of \(q\) is consequently a finite weighted average of deterministic risk vectors, hence lies in their convex hull. Together, the two inclusions prove the claim. \(\square\)
A Two-State Boundary That Deterministic Rules Miss
Worked Example: A Supporting Point Only a Mixture Attains
There are two parameter states, \(\theta_1\) and \(\theta_2\), no informative observation (the observation space contains only one point), and two actions, \(A\) and \(B\). The loss table is:
| Parameter | Loss for \(A\) | Loss for \(B\) |
|---|---|---|
| \(\theta_1\) | 0 | 4 |
| \(\theta_2\) | 4 | 0 |
There are only two deterministic rules, since no observation can distinguish cases. Choosing \(A\) gives risk vector \((0,4)\); choosing \(B\) gives \((4,0)\). Neither deterministic rule attains \((2,2)\).
Now choose \(A\) with probability \(1/2\) and \(B\) with probability \(1/2\). At \(\theta_1\), the expected loss is \((1/2)0+(1/2)4=2\). At \(\theta_2\), it is \((1/2)4+(1/2)0=2\). The randomized risk vector is therefore \((2,2)\).
The line \(r_1+r_2=4\) supports the attainable risk region at this point. To see the geometry precisely, the randomized risk vectors are \((4(1-p),4p)\) for \(0\leq p\leq1\), and every such vector has coordinate sum 4. The coordinatewise upper risk region is the set of nonnegative vectors \(r\) with \(r_1+r_2\geq4\); its lower boundary includes \((2,2)\). With deterministic rules alone, the upper risk region is the union of the regions above \((0,4)\) and \((4,0)\), and it does not contain \((2,2)\). Thus this supporting boundary point is attainable by randomization but not by a deterministic rule.
What Changes When Observations Are Available?
A deterministic rule can use the data to change its action. That is different from randomizing: observing informative data can itself produce risk vectors between, or beyond, the risks of constant-action rules. The finite convex-hull result still applies, but its comparison is with all deterministic rules, including rules that depend on the observation.
Worked Example: A Randomized Rule With an Informative Observation
Let \(X\in\{0,1\}\), with \(P_{\theta_1}(X=1)=0.75\) and \(P_{\theta_2}(X=1)=0.25\). Use the same two-action loss table as above. Consider a randomized rule that chooses \(A\) with probability 0.8 when \(X=1\), and with probability 0.2 when \(X=0\).
At \(\theta_1\), the probability of choosing \(A\) is \((0.75)(0.8)+(0.25)(0.2)=0.60+0.05=0.65\). The loss is 4 exactly when the rule chooses \(B\), so \(R(\theta_1)=4(1-0.65)=1.4\).
At \(\theta_2\), the probability of choosing \(A\) is \((0.25)(0.8)+(0.75)(0.2)=0.20+0.15=0.35\). Here the loss is 4 exactly when the rule chooses \(A\), so \(R(\theta_2)=4(0.35)=1.4\). The risk vector is \((1.4,1.4)\).
This particular randomized rule can also be represented as a lottery over deterministic rules, as the finite convex-hull theorem guarantees. The observation-dependent probabilities do not invalidate the representation: a random deterministic rule can be drawn in advance, with its action at each possible observation drawn according to the specified conditional probabilities.
Randomization and Weighted Risk Boundaries
Supporting lines make the role of convexity especially visible with two parameter states. A linear criterion \(w_1r_1+w_2r_2\), with nonnegative weights \(w_1,w_2\), assigns a score to each risk vector. Its level sets are lines, and minimizing the criterion seeks a boundary of the attainable risk region. Randomization can fill a gap between deterministic risk vectors and create boundary points with a more balanced distribution of risk.
Worked Example: A Mixture Improves the Worst-Case Risk
Consider one observation, two states, and three actions with risk vectors \(A=(0,6)\), \(B=(6,0)\), and \(C=(4,4)\). The deterministic worst-case risks, found by taking the larger coordinate, are 6, 6, and 4. The best deterministic worst-case risk is therefore 4, attained by \(C\).
A 50–50 lottery between \(A\) and \(B\) has risk vector \((1/2)(0,6)+(1/2)(6,0)=(3,3)\). Its worst-case risk is 3, which is strictly less than 4. No deterministic rule attains \((3,3)\), since the only deterministic vectors are the three listed.
The weighted sum \(r_1+r_2\) equals 6 at \(A\), \(B\), and their mixture \((3,3)\), but equals 8 at \(C\). The line \(r_1+r_2=6\) supports the lower boundary at \((3,3)\). The mixture balances the state-specific risks without improving both coordinates relative to every deterministic rule: \(C\), for example, has risk 4 in each state, whereas the mixture has risk 3 in each.
Why This Matters in Wald-Type Arguments
In finite-dimensional complete-class arguments, one studies risk vectors and uses convex geometry, including separation, to identify procedures that can be improved or excluded. If the starting set consists only of deterministic rules, it may be nonconvex. A separating-hyperplane argument cannot simply be applied to that nonconvex set as though every line segment between its points were attainable.
Allowing randomized rules repairs this particular geometric obstacle: finite mixtures are attainable, so the risk set is convex. This does not by itself prove a complete-class theorem. Such a theorem needs its own hypotheses and further argument. The point here is narrower and essential: convexity needed in a risk-space argument may be a property of the randomized class even when it fails for the deterministic class.
Common Mistake
A common error is to say, “The midpoint of two risk vectors is attainable because risks can be averaged.” Risks are numerical summaries, not procedures. The midpoint is attained only after specifying an actual randomized rule, such as drawing one of the two procedures with the required probabilities, and then verifying that the draw is independent of the parameter and that its expected loss gives the stated coordinates.
A related mistake is to conclude that randomization is unnecessary because a deterministic rule can sometimes use the data to balance risk. It may do so when the data are informative, but the singleton-observation example shows that no deterministic data-dependent choice is available in every problem. The correct claim is that randomization convexifies the deterministic risk vectors; it does not claim every randomized risk vector is already the risk vector of a deterministic rule.
Check Your Understanding
Use the definitions and calculations above to answer these questions.
- For two deterministic rules with risk vectors \((1,5)\) and \((5,1)\), what risk vector results from choosing each with probability \(1/2\)?
- In the two-state loss-table example, why can no deterministic rule attain \((2,2)\)?
- State the finite-space condition used to show that every randomized rule is a lottery over deterministic rules.
- Does the convex-hull result say that every randomized risk vector is attained by one deterministic rule? Explain.
- Why is it unsafe to apply a separation argument directly to a nonconvex set of deterministic risk vectors?