Domination and Admissibility for General Decision Rules

Why Decision Rules Are Compared by Risk

A decision rule takes observed data and produces an action. The action might be an estimate, a treatment choice, or a recommendation; it need not be a number. To assess a rule, we ask how much loss it incurs under each possible state of nature. The resulting risk function is what makes it possible to compare procedures across the parameter space.

A statistical model describes how the data are generated. A rule describes how data lead to actions, and a loss function describes the consequences of those actions. Risk combines all three. It is therefore not a property of the sampling model alone, nor is it a single number attached to a rule regardless of the decision problem.

Decision Problems and Rules

Let $\Theta$ be a parameter space, $(\mathcal X,\mathcal A_{\mathcal X})$ a measurable sample space, and $(\mathcal D,\mathcal A_{\mathcal D})$ a measurable action space. A statistical model is a family $\{P_\theta:\theta\in\Theta\}$ of probability distributions on the sample space. A loss function $L(\theta,a)$ records the loss from taking action $a$ when the parameter is $\theta$.

For the general definitions below, assume $L$ is a nonnegative, jointly measurable function on $\Theta\times\mathcal D$, and that the model and decision rules have the required measurable-kernel structure. Nonnegativity allows risks to be defined even when they are infinite. Signed losses require additional integrability conditions to make their expectations well-defined.

Definition: A deterministic decision rule is a measurable map $\delta:\mathcal X\to\mathcal D$. A randomized decision rule is a Markov kernel $\delta(da\mid x)$ from the sample space to the action space: for each observation $x$, it specifies a probability distribution over actions, measurably in $x$. A deterministic rule is the special case that assigns probability one to $\delta(x)$.

The decision rule is chosen before $\theta$ is known. Its input is the data, not the true parameter. When the rule is randomized, its additional randomness is represented by the conditional distribution over actions given the data.

Definition: The risk function of a rule $\delta$, under model $\{P_\theta\}$ and loss $L$, is $$ R(\theta,\delta) = \int_{\mathcal X}\int_{\mathcal D} L(\theta,a)\,\delta(da\mid x)\,P_\theta(dx), \qquad \theta\in\Theta. $$ For a deterministic rule, this reduces to $R(\theta,\delta)=E_\theta[L(\theta,\delta(X))]$. Risk is a function of $\theta$ and can take any value in $[0,\infty]$.

The notation $E_\theta$ means expectation when the data have distribution $P_\theta$. In the randomized case, the inner integral averages over the action distribution conditional on the observed data, and the outer integral averages over the data. If the sample or action space is continuous, these integrals are understood with respect to the relevant probability measures; no density need exist.

Worked Risk Calculations

Worked Example: A Rule Can Have Zero Risk

Suppose $\Theta=\{0,1\}$, $X$ takes values in $\{0,1\}$, and $P_\theta(X=\theta)=1$. The action space is $\mathcal D=\{0,1\}$, with loss $L(\theta,a)=\mathbf{1}\{a\ne\theta\}$. Consider the rule $\delta(X)=X$.

At $\theta=0$, the observation is $X=0$ with probability one, so the rule takes action 0 and incurs loss 0. Thus $R(0,\delta)=0$. At $\theta=1$, the observation is $X=1$ with probability one, so the rule takes action 1 and again incurs loss 0. Hence $$ R(0,\delta)=0,\qquad R(1,\delta)=0. $$

The zero risk at both parameter values also establishes admissibility in this particular problem. Every rule has nonnegative risk, so no rule can have risk below zero at either parameter value, and no rule can improve strictly at any parameter value. This argument relies on the stated model and loss, not just on the formula $\delta(X)=X$.

Worked Example: The Same Rule Under Two Losses

Let $X\sim N(\theta,1)$ for $\theta\in\mathbb R$, take the action space to be $\mathbb R$, and use the rule $\delta(X)=X$. Under squared-error loss, $L_2(\theta,a)=(a-\theta)^2$, the risk is $$ R_2(\theta,\delta)=E_\theta[(X-\theta)^2]=\operatorname{Var}_\theta(X)=1. $$

Now keep the model, action space, and rule unchanged, but use absolute-error loss $L_1(\theta,a)=|a-\theta|$. Since $X-\theta\sim N(0,1)$, its density is $\phi(z)=(2\pi)^{-1/2}e^{-z^2/2}$. Therefore $$ R_1(\theta,\delta) =\int_{-\infty}^{\infty}|z|\phi(z)\,dz =2\int_0^\infty z\frac{e^{-z^2/2}}{\sqrt{2\pi}}\,dz =\sqrt{\frac{2}{\pi}}\int_0^\infty z e^{-z^2/2}\,dz =\sqrt{\frac{2}{\pi}}. $$

The integral in the final step equals 1, since the antiderivative of $z e^{-z^2/2}$ is $-e^{-z^2/2}$. Both risks are constant in $\theta$, but they have different values because the loss changed. The sampling model did not change.

Worked Example: Risk for a Randomized Rule

Let $\Theta=\{\theta_1,\theta_2\}$ and $X\in\{0,1\}$, with $P_{\theta_1}(X=1)=0.75$ and $P_{\theta_2}(X=1)=0.25$. The actions are $a$ and $b$, and the losses are $L(\theta_1,a)=0$, $L(\theta_1,b)=2$, $L(\theta_2,a)=3$, and $L(\theta_2,b)=0$.

Consider the rule that chooses $a$ with probability 0.6 when $X=1$, and with probability 0.2 when $X=0$. The probability of choosing $b$ is consequently 0.4 when $X=1$ and 0.8 when $X=0$. At $\theta_1$, loss occurs only when the rule chooses $b$, and that loss is 2. Thus $$ R(\theta_1,\delta) =2\{P_{\theta_1}(X=1)(0.4)+P_{\theta_1}(X=0)(0.8)\} =2\{0.75(0.4)+0.25(0.8)\}=1. $$

At $\theta_2$, loss occurs only when the rule chooses $a$, and that loss is 3. Therefore $$ R(\theta_2,\delta) =3\{P_{\theta_2}(X=1)(0.6)+P_{\theta_2}(X=0)(0.2)\} =3\{0.25(0.6)+0.75(0.2)\}=0.9. $$

This calculation uses both sources of randomness: the observation and the rule’s action conditional on that observation. The risk is still indexed by the parameter, even though the rule itself does not know which parameter value is true.

Domination and Admissibility

Risk makes it possible to compare two rules for the same decision problem. The comparison must use the same model and loss for both; otherwise the two risk functions do not answer the same decision question.

Definition: A rule $\delta_1$ dominates a rule $\delta_0$ if $R(\theta,\delta_1)\le R(\theta,\delta_0)$ for every $\theta\in\Theta$, with strict inequality for at least one $\theta$. A rule is admissible if no other rule in the specified class dominates it.

The class of available rules is part of the definition. For example, a question may compare deterministic rules only, or it may allow randomized rules as well. A rule’s admissibility can depend on that choice, just as it depends on the model, loss, parameter space, and action space.

Theorem: Suppose $\Theta$, $\mathcal X$, and $\mathcal D$ are measurable spaces, $P_\theta$ is a probability kernel from $\Theta$ to $\mathcal X$, $\delta(da\mid x)$ is a probability kernel from $\mathcal X$ to $\mathcal D$, and $L(\theta,a)$ is nonnegative and jointly measurable. Then $R(\theta,\delta)$ is a measurable function of $\theta$, with values in $[0,\infty]$.

Proof. First define $$ H(\theta,x)=\int_{\mathcal D}L(\theta,a)\,\delta(da\mid x). $$ The parameterized-kernel integration theorem implies that $H$ is measurable in $(\theta,x)$. For clarity, this fact follows first for nonnegative indicator functions of measurable rectangles in $(\theta,x,a)$, then for nonnegative simple functions by finite addition, and then for all nonnegative measurable functions by increasing simple-function approximation and monotone convergence. Applying the same integration theorem to $P_\theta(dx)$ gives $$ R(\theta,\delta)=\int_{\mathcal X}H(\theta,x)\,P_\theta(dx) $$ as a measurable function of $\theta$. Nonnegativity ensures the integrals exist, possibly with value $+\infty$. $\square$

Theorem: If two randomized rules $\delta_1$ and $\delta_2$ are mixed with a fixed probability $\lambda\in[0,1]$, the resulting rule $\delta_\lambda(da\mid x)=\lambda\delta_1(da\mid x)+(1-\lambda)\delta_2(da\mid x)$ has risk $$ R(\theta,\delta_\lambda) =\lambda R(\theta,\delta_1)+(1-\lambda)R(\theta,\delta_2). $$

Proof. By the definition of risk and linearity of integration for nonnegative functions, $$ \begin{aligned} R(\theta,\delta_\lambda) &=\int_{\mathcal X}\int_{\mathcal D}L(\theta,a) \{\lambda\delta_1(da\mid x)+(1-\lambda)\delta_2(da\mid x)\}P_\theta(dx)\\ &=\lambda\int_{\mathcal X}\int_{\mathcal D}L(\theta,a)\delta_1(da\mid x)P_\theta(dx) +(1-\lambda)\int_{\mathcal X}\int_{\mathcal D}L(\theta,a)\delta_2(da\mid x)P_\theta(dx)\\ &=\lambda R(\theta,\delta_1)+(1-\lambda)R(\theta,\delta_2). \end{aligned} $$ The identity holds for extended nonnegative risks, with the endpoint cases $\lambda=0$ and $\lambda=1$ giving the corresponding unmixed rule. $\square$

What Risk Comparisons Do—and Do Not—Say

Admissibility is a statement about the absence of a uniformly better alternative in the chosen class of rules. It is not a claim that a rule has the smallest risk at every parameter value among all rules; such a claim would compare the rule separately with every competitor at each parameter. Nor is admissibility a property of the distribution of the data alone. It is defined through the risk functions induced by the full decision problem.

A common mistake is to treat a risk as though it were determined only by the sampling model—for example, to say that a normal model “has risk one.” The normal example shows why that is incomplete: the risk of the rule $X$ is 1 under squared-error loss and $\sqrt{2/\pi}$ under absolute-error loss. To state a risk precisely, identify the model, the rule, and the loss, then give the resulting function of $\theta$.

Another pitfall is to compare rules at only one parameter value and infer a global conclusion. Domination requires the risk inequality to hold throughout the parameter space, as well as a strict improvement somewhere. A calculation at one point can establish only what happens at that point; it cannot by itself establish the required comparison everywhere.

Key takeaway: A decision problem specifies a model, actions, and losses; a rule maps data to actions; and its risk records expected loss at each parameter value. Domination and admissibility are defined by comparisons of these risk functions within a specified class of rules.

Check Your Understanding

Use the definitions to answer each question.

What three ingredients, in addition to the parameter value, determine the risk of a decision rule?
How does the risk formula for a deterministic rule differ in appearance from the formula for a randomized rule?
In the randomized-rule example, why is the risk at $\theta_1$ twice the probability of choosing $b$ under that parameter?
State the two risk conditions required for one rule to dominate another.
Why does changing the loss while holding the model and rule fixed potentially change the risk function?