Risk-Equivalent Rules and Minimal Representatives

Why Identify Rules by Their Risks?

Two decision rules can prescribe different actions for some observations yet perform identically, in risk, for every parameter value. Keeping both rules in a description of a class adds procedural detail without adding any attainable risk function. The relevant reduction is therefore not necessarily a reduction in the number of actions or in the number of rules’ pointwise differences: it is a reduction in the number of distinct risks.

This distinction matters when studying complete classes. Completeness is about whether a class contains a rule whose risk is no greater than the risk of each rule under consideration. It is therefore unchanged when a rule is replaced by another rule with exactly the same risk function. Minimal completeness asks a further question: can any member be removed while retaining completeness? Risk equivalence helps remove duplicates, but it does not by itself settle that further question.

Risk Equivalence

Fix a statistical decision problem with parameter space $\Theta$, an observation $X$ with distribution indexed by $\theta$, action space $\mathcal A$, and loss $L(\theta,a)$. A (nonrandomized) decision rule $\delta$ is a measurable function from the sample space to $\mathcal A$. Its risk at $\theta$ is

R(\theta,\delta)=E_\theta[L(\theta,\delta(X))],

where the expectation is allowed to be an extended nonnegative value if the loss is nonnegative. The risk function records the expected loss at every parameter value; it does not record which action the rule takes for each possible observation.

Definition: Two decision rules $\delta_1$ and $\delta_2$ are risk-equivalent if $$ R(\theta,\delta_1)=R(\theta,\delta_2)\quad\text{for every }\theta\in\Theta. $$ Write $\delta_1\sim_R\delta_2$ for this relation. It is an equivalence relation: every rule is equivalent to itself, equivalence is symmetric, and equality of risk functions is transitive.

Risk equivalence is weaker than equality of rules. The rules may give different actions on observations that have probability zero under every parameter value. They may also differ on observations with positive probability, with the differences averaging out to the same risk. In either case, the definition requires equality of the whole risk functions, not merely equality at one parameter value or equality of maximum risk.

For a class $C$ of rules, its attainable risk set is

\mathcal R(C)=\{R(\,\cdot\,,\delta):\delta\in C\}.

This set contains distinct risk functions, not distinct rules. Consequently, two rules in $C$ that are risk-equivalent contribute only one element to $\mathcal R(C)$.

Worked Example: Different Rules with the Same Risk

Let $\Theta=\{\theta_1,\theta_2\}$ and let the observation $X$ take values $0$ and $1$, each with probability $1/2$ under both parameter values. The action space is $\{A,B\}$, with losses

Parameter	$L(\theta,A)$	$L(\theta,B)$
$\theta_1$	0	2
$\theta_2$	3	1

Define $\delta_1$ to choose $A$ at $X=0$ and $B$ at $X=1$. Define $\delta_2$ to choose $B$ at $X=0$ and $A$ at $X=1$. These rules differ at both possible observations. Under $\delta_1$, each action is chosen with probability $1/2$, so

R(\theta_1,\delta_1)=\tfrac12(0)+\tfrac12(2)=1,\qquad R(\theta_2,\delta_1)=\tfrac12(3)+\tfrac12(1)=2.

Under $\delta_2$, the actions again each occur with probability $1/2$. Thus

R(\theta_1,\delta_2)=\tfrac12(2)+\tfrac12(0)=1,\qquad R(\theta_2,\delta_2)=\tfrac12(1)+\tfrac12(3)=2.

Therefore $R(\,\cdot\,,\delta_1)=R(\,\cdot\,,\delta_2)=(1,2)$, even though the rules are not the same function of the observation. They are distinct procedures but a single risk-equivalence class.

Representative Reduction Preserves Attainable Risks

Within a class $C$, divide the rules into equivalence classes under $\sim_R$. A representative set $C_0$ contains exactly one rule from each nonempty equivalence class in $C$. It keeps one procedure for each risk function present in $C$.

Theorem: If $C_0$ is a representative set for the risk-equivalence classes in $C$, then $$ \mathcal R(C_0)=\mathcal R(C). $$ In particular, the reduction preserves every risk function attainable in $C$.

Proof. Since $C_0\subseteq C$, every risk function attained by a member of $C_0$ is attained by a member of $C$. Hence $\mathcal R(C_0)\subseteq\mathcal R(C)$. Conversely, take any $r\in\mathcal R(C)$. There is some $\delta\in C$ with $r=R(\,\cdot\,,\delta)$. The representative $\delta_0\in C_0$ of $\delta$’s equivalence class satisfies $R(\,\cdot\,,\delta_0)=R(\,\cdot\,,\delta)$, so $r\in\mathcal R(C_0)$. Thus $\mathcal R(C)\subseteq\mathcal R(C_0)$, proving equality. $\square$

Completeness is preserved for the same reason. Use the pointwise risk definition: a class $C\subseteq D$, where $D$ is the class of rules under consideration, is complete if for every $\eta\in D$, there exists $\delta\in C$ such that $R(\theta,\delta)\leq R(\theta,\eta)$ for every $\theta\in\Theta$. If $C$ is complete, its representative set $C_0$ is complete as well. For any $\eta\in D$, completeness gives such a $\delta\in C$; the representative of $\delta$’s equivalence class has exactly the same risk and therefore satisfies the same inequalities. Conversely, if $C_0$ is complete, then $C$ is complete because $C_0\subseteq C$.

Worked Example: Reducing a Complete Class to Risk Representatives

Return to the two-parameter decision problem above, and let $D$ consist of all four deterministic rules. Write a rule as its actions at $X=0$ and $X=1$. The rules $AA$ and $BB$ have risks

R(\,\cdot\,,AA)=(0,3),\qquad R(\,\cdot\,,BB)=(2,1).

The other two rules, $AB$ and $BA$, have the same risk $(1,2)$, as calculated in the preceding example. Thus the four-rule class $D$ has three attainable risk functions:

\mathcal R(D)=\{(0,3),(1,2),(2,1)\}.

Choose $C_0=\{AA,AB,BB\}$, retaining $AB$ and omitting its risk-equivalent rule $BA$. The omitted rule’s risk, $(1,2)$, is still attained by $AB$, so $\mathcal R(C_0)=\mathcal R(D)$. Moreover, $C_0$ is complete relative to $D$: for each rule in $D$, either the rule itself is in $C_0$, or its risk is matched by $AB$. This reduction changes neither the attainable risks nor the class’s completeness.

Minimal Complete Classes

A complete class may contain members that can be removed while leaving a complete class. The relevant notion is inclusion, not cardinality.

Definition: A complete class $C\subseteq D$ is minimal complete (under inclusion) if no proper subset of $C$ is complete relative to $D$. Minimal means that no member can be discarded while retaining completeness; it does not mean that $C$ is the smallest complete class by number of rules.

Risk-equivalent duplicates cannot occur in a minimal complete class. If $\delta_1,\delta_2\in C$ and $\delta_1\sim_R\delta_2$, then removing $\delta_1$ does not lose its risk. Any comparison in which $\delta_1$ supplied a risk no greater than a target risk can instead use $\delta_2$, which has exactly that risk. Thus $C\setminus\{\delta_1\}$ remains complete, contradicting minimality.

Theorem: A complete class $C\subseteq D$ is minimal complete if and only if, for every $\delta\in C$, there exists some $\eta_\delta\in D$ such that no $\gamma\in C\setminus\{\delta\}$ satisfies $$ R(\theta,\gamma)\leq R(\theta,\eta_\delta)\quad\text{for every }\theta\in\Theta. $$

Proof. Suppose first that $C$ is minimal complete. Removing any $\delta\in C$ leaves a proper subset that is not complete. By the definition of completeness, failure means that some $\eta_\delta\in D$ has no rule in $C\setminus\{\delta\}$ whose risk is pointwise no greater than $R(\,\cdot\,,\eta_\delta)$. This gives the stated condition.

Conversely, suppose the condition holds for every $\delta\in C$. Let $C'$ be any proper subset of $C$, and choose $\delta\in C\setminus C'$. The condition supplies an $\eta_\delta$ not covered by any member of $C\setminus\{\delta\}$, and hence not covered by any member of $C'$. Therefore $C'$ is not complete. Since $C$ is complete by assumption and no proper subset is complete, $C$ is minimal complete. $\square$

The condition makes the distinction between duplicate removal and minimality precise. Risk-equivalent rules are always redundant in a complete class. But a rule with a unique risk function can also be redundant if another member’s risk is pointwise no greater for every target it needs to cover.

Worked Example: A Minimal Complete Representative Class

In the four-rule problem, the representative class $C_0=\{AA,AB,BB\}$ has risks $(0,3)$, $(1,2)$, and $(2,1)$. Each rule in $D$ has one of these three risks, so $C_0$ is complete. None of the three risk vectors is pointwise no greater than another: moving from $(0,3)$ to $(1,2)$ raises the first coordinate, while moving from $(1,2)$ to $(2,1)$ raises it again. In the opposite direction, the second coordinate increases.

If $AA$ is removed, neither remaining vector is pointwise no greater than $(0,3)$, because both have a positive first coordinate. If $AB$ is removed, neither $(0,3)$ nor $(2,1)$ is pointwise no greater than $(1,2)$. If $BB$ is removed, neither remaining vector is pointwise no greater than $(2,1)$, because each has a second coordinate greater than $1$. Thus removing any one member makes the class incomplete relative to $D$, and $C_0$ is minimal complete.

In this example the reduction first removes a duplicate risk, then the three distinct risks are all necessary. The conclusion depends on the risk comparisons in the specified decision problem; it does not follow merely from having selected one rule per equivalence class.

Common Pitfalls and Why the Distinction Matters

Calling two rules equivalent because they agree at one parameter. Risk equivalence requires equality at every $\theta$. Equality of maximum risk, or equality at a selected parameter value, is not enough.
Confusing identical rules with risk-equivalent rules. Rules can differ on observations with positive probability, as in the first example, and still have identical risk functions.
Assuming distinct risks imply minimal completeness. Consider a decision problem with one parameter and three available rules whose risks are $0$, $1$, and $2$, and let $D$ consist of those rules. The class containing the rules with risks $0$ and $1$ is complete: risk $0$ is no greater than the risk of every rule in $D$. Its members have distinct risks, but removing the risk-$1$ rule leaves a complete class. It is not minimal.
Confusing minimal with smallest. Minimal completeness means no proper subset is complete. It does not compare the size of one minimal complete class with that of a different minimal complete class.

The practical benefit of risk-equivalence reduction is a cleaner analysis: the class retains every risk function it could attain, while distinctions that have no effect on risk are suppressed. This gives an exact, risk-preserving simplification. Establishing that the resulting class is minimal complete still requires checking whether each remaining risk is needed to cover some rule in the full class.

Check Your Understanding

Use equality of full risk functions, and keep completeness relative to the stated class of rules.

What condition on $R(\theta,\delta_1)$ and $R(\theta,\delta_2)$, for all $\theta$, defines risk equivalence?
Why does choosing one representative from each risk-equivalence class leave the attainable risk set unchanged?
Can two rules that differ at observations of positive probability be risk-equivalent? Explain briefly.
Why can a minimal complete class not contain two distinct risk-equivalent rules?
Does having a unique risk function guarantee that a member of a complete class is necessary? Give the relevant distinction.

Parameter	\(L(\theta,A)\)	\(L(\theta,B)\)
\(\theta_1\)	0	2
\(\theta_2\)	3	1