Comparing Rules by Their Risks
A decision rule is judged by how it performs across the possible parameter values, not by the loss it happens to incur on one observed sample. The risk \(R(\theta,\delta)\) of a rule \(\delta\) is its expected loss when the true parameter is \(\theta\). To compare two rules, we therefore compare their risks at each parameter value in the parameter space.
The central distinction is between being no worse everywhere and being strictly better somewhere. Equality at some parameter values is allowed in a strict domination comparison; worsening at even one parameter value is not. The word “strict” can be used in different ways in informal discussion, so the definitions below make the convention explicit.
Here risk is the expectation of the loss under the sampling distribution at \(\theta\). For a nonrandomized rule \(\delta\), this is \(R(\theta,\delta)=E_\theta[L(\theta,\delta(X))]\), when the expectation is defined and finite. The same risk comparison applies if rules are randomized, with the expectation also averaging over the rule’s randomization.
A Precise Test for Strict Domination
For a pair of rules, it is useful to separate parameter values where the first rule improves from those where it ties or worsens. Define the risk difference $$ \Delta(\theta)=R(\theta,\eta)-R(\theta,\delta). $$ Positive values mean \(\delta\) has lower risk, zero means equal risk, and negative values mean \(\delta\) has higher risk.
Proof. Suppose first that \(\delta\) strictly dominates \(\eta\). By definition, \(R(\theta,\delta)\leq R(\theta,\eta)\) for every \(\theta\). Subtracting the left risk from the right gives \(\Delta(\theta)\geq 0\) everywhere. Strict domination also requires at least one parameter value \(\theta_0\) for which \(R(\theta_0,\delta)<R(\theta_0,\eta)\), so \(\Delta(\theta_0)>0\). Conversely, if \(\Delta(\theta)\geq 0\) everywhere, then \(R(\theta,\delta)\leq R(\theta,\eta)\) everywhere. If also \(\Delta(\theta_0)>0\) for some \(\theta_0\), then \(R(\theta_0,\delta)<R(\theta_0,\eta)\). These are exactly the two conditions for strict domination. \(\square\)
This criterion distinguishes three useful patterns. If \(\Delta\) is zero everywhere, the risks tie everywhere, so there is no strict domination. If \(\Delta\) is positive everywhere, the first rule is strictly better everywhere and therefore strictly dominates. If \(\Delta\) is nonnegative everywhere and positive only on part of the parameter space, it still strictly dominates: equality elsewhere does not disqualify it.
Worked Risk Comparisons
Worked Example: Equal Risks at Every Parameter Value
Let \(\Theta=\{0,1\}\), let \(X\) have a Bernoulli distribution with success probability \(1/2\) under either parameter value, and take the action space to be \(\{0,1\}\). Use the zero-one loss \(L(\theta,a)=1\) if \(a\ne\theta\), and \(0\) otherwise. Compare the rules \(\delta(X)=X\) and \(\eta(X)=1-X\).
At \(\theta=0\), \(X\) equals 0 and 1 with probability \(1/2\) each. Rule \(\delta\) makes an error when \(X=1\), so \(R(0,\delta)=1/2\). Rule \(\eta\) makes an error when \(X=0\), so \(R(0,\eta)=1/2\).
At \(\theta=1\), rule \(\delta\) makes an error when \(X=0\), while rule \(\eta\) makes an error when \(X=1\). Each event has probability \(1/2\), giving \(R(1,\delta)=R(1,\eta)=1/2\). Thus the risk difference is zero at both parameter values. Neither rule strictly dominates the other, even though the rules make different decisions for each observed value of \(X\).
Worked Example: Strict Improvement at Every Parameter Value
Let \(\Theta=\{0,1\}\), let \(X\sim\operatorname{Bernoulli}(\theta)\), and estimate \(\theta\) under squared-error loss \(L(\theta,a)=(a-\theta)^2\). Compare the constant rules \(\delta(X)=1\) and \(\eta(X)=2\). Because both rules are constant, their risks are their squared errors.
At \(\theta=0\), $$ R(0,\delta)=(1-0)^2=1,\qquad R(0,\eta)=(2-0)^2=4. $$ At \(\theta=1\), $$ R(1,\delta)=(1-1)^2=0,\qquad R(1,\eta)=(2-1)^2=1. $$
The first risk is lower at both parameter values, so \(\delta\) strictly dominates \(\eta\) and is strictly better everywhere. This example also checks why the comparison must use risk: the distribution of \(X\) does not affect either rule’s constant action or its loss.
Worked Example: Improvement at Only One Parameter Value
Let \(\Theta=\{0,\tfrac12\}\), let \(X\sim\operatorname{Bernoulli}(\theta)\), and use squared-error loss for estimating \(\theta\). Compare \(\delta(X)=\tfrac12 X\) with the constant rule \(\eta(X)=0\).
At \(\theta=0\), \(X=0\) with probability one. Both rules choose 0, so $$ R(0,\delta)=R(0,\eta)=0. $$ At \(\theta=\tfrac12\), \(X=0\) and \(X=1\) each have probability \(1/2\). Rule \(\delta\) chooses 0 when \(X=0\), with squared error \((0-\tfrac12)^2=\tfrac14\), and chooses \(\tfrac12\) when \(X=1\), with squared error 0. Therefore $$ R(\tfrac12,\delta)=\tfrac12\cdot\tfrac14+\tfrac12\cdot 0=\tfrac18. $$ The constant rule always chooses 0, so \(R(\tfrac12,\eta)=(0-\tfrac12)^2=\tfrac14\).
Consequently, \(\delta\) ties \(\eta\) at \(\theta=0\) and has lower risk at \(\theta=\tfrac12\). It strictly dominates \(\eta\), although the improvement is not strict everywhere. The equality at one parameter value is fully consistent with strict domination.
Why the Distinction Matters
In admissibility questions, a rule is ruled out as inadmissible when another rule strictly dominates it under the stated convention. The comparison requires two checks: the proposed alternative must have no greater risk at every parameter value, and it must have lower risk at least once. A gain at one parameter value is not enough if the alternative has higher risk somewhere else.
The most common mistake is to read “strict domination” as “strictly lower risk at every parameter value.” That condition is sufficient, but it is not necessary. The opposite mistake is to treat any observed improvement in a risk table as domination. An improvement at one parameter value cannot compensate for a risk increase at another: the no-worse condition is pointwise across the whole parameter space.
A second pitfall is to compare realized losses for a particular sample instead of expected losses under each parameter value. Domination is a statement about risk functions. A rule may incur more loss on one observation and still have lower risk after averaging; conversely, a favorable realized loss does not establish lower risk.
For every \(\theta\), determine whether \(R(\theta,\delta)\) is below, equal to, or above \(R(\theta,\eta)\).
If \(R(\theta,\delta)>R(\theta,\eta)\) for even one \(\theta\), then \(\delta\) does not dominate \(\eta\).
If there is no worsening and there is a parameter value with lower risk, then \(\delta\) strictly dominates \(\eta\). If all risks tie, there is no strict domination.
Check Your Understanding
For each comparison, focus on the risks over the entire parameter space.
- If \(R(\theta,\delta)=R(\theta,\eta)\) for every \(\theta\), does either rule strictly dominate the other? Explain.
- Suppose \(\delta\) has lower risk than \(\eta\) at every parameter value. What kind of comparison does this establish?
- Suppose \(\delta\) has equal risk to \(\eta\) at three parameter values and lower risk at a fourth, with no parameter value where its risk is higher. Does it strictly dominate \(\eta\)?
- Why does a lower realized loss on one observed sample not by itself establish that a rule dominates another?
- What condition fails if a proposed rule improves risk at one parameter value but has higher risk at another?