Why Order Risk Functions?
A statistical procedure does not usually have one risk value: its risk depends on the unknown parameter. Comparing procedures therefore means comparing functions across the parameter space, not just comparing two numbers at one selected parameter value. The pointwise order gives a precise meaning to the statement that one risk profile is no larger than another anywhere in the model.
There is a small but important technical issue. A risk profile can be described under different labels, even when its value at every parameter is the same. Antisymmetry belongs to an order on the profiles themselves, not to an order on arbitrary labels for them. We make this precise by identifying profiles that agree everywhere and ordering the resulting equivalence classes.
Risk Profiles and Pointwise Order
The order compares the two values at each parameter separately. A comparison at just one parameter is not enough to establish \(r\preceq s\); the inequality must hold throughout \(\Theta\). The symbol \(\preceq\) emphasizes that this is an order between functions, rather than the ordinary order between single real numbers.
Let \(\mathcal R\) be any collection of risk functions under consideration. Define a relation \(\sim\) on \(\mathcal R\) by $$ r\sim s \quad\Longleftrightarrow\quad r(\theta)=s(\theta)\ \text{for every }\theta\in\Theta. $$ This is equality of profiles at every parameter value. It is an equivalence relation: each function agrees with itself, agreement is symmetric, and if \(r\) agrees everywhere with \(s\) and \(s\) agrees everywhere with \(t\), then \(r\) agrees everywhere with \(t\). Write \([r]\) for the equivalence class of \(r\), consisting of all profiles that agree with it everywhere.
Before calling this a partial order, two things must be checked. First, the definition cannot depend on which representative is chosen from either class. Second, the relation must be reflexive, transitive, and antisymmetric. Antisymmetry says that if two classes are ordered in both directions, then they are the same class.
Worked Comparisons
Worked Example: Comparing Finite Risk Profiles
Let \(\Theta=\{\theta_1,\theta_2,\theta_3\}\), and suppose three risk profiles have the following values:
| Profile | \(\theta_1\) | \(\theta_2\) | \(\theta_3\) |
|---|---|---|---|
| \(r\) | 1 | 2 | 0 |
| \(s\) | 2 | 2 | 3 |
| \(t\) | 3 | 4 | 3 |
To verify \(r\preceq s\), check all three parameter values: \(1\leq2\), \(2\leq2\), and \(0\leq3\). Thus \(r\preceq s\). Similarly, \(s\preceq t\), since \(2\leq3\), \(2\leq4\), and \(3\leq3\). Comparing \(r\) directly with \(t\) gives \(1\leq3\), \(2\leq4\), and \(0\leq3\), so \(r\preceq t\) as well.
The equality at \(\theta_2\) in the first comparison does not prevent the pointwise inequality: the definition permits equality at some parameter values. What matters is that none of the three required inequalities fails.
Worked Example: Risks in a Deterministic Observation Model
Suppose the parameter space and action space are both the real line, and the observation is deterministic: \(X=\theta\) with probability one under parameter \(\theta\). Use squared-error loss \(L(\theta,a)=(a-\theta)^2\). Compare the constant rule \(\delta_0(X)=0\) with the rule \(\delta_1(X)=X\).
For the constant rule, the action is always zero. Its risk is $$ R_{\delta_0}(\theta)=E_\theta[(0-\theta)^2]=\theta^2. $$ For the rule that reports the observation, \(X=\theta\) with probability one, so its risk is $$ R_{\delta_1}(\theta)=E_\theta[(X-\theta)^2]=E_\theta[0]=0. $$
Since \(\theta^2\geq0\) for every real \(\theta\), \(R_{\delta_1}(\theta)\leq R_{\delta_0}(\theta)\) throughout the parameter space. This verifies the pointwise comparison directly from the risk definitions; it does not rely on checking just a few selected values of \(\theta\).
Worked Example: A Normal-Mean Risk Comparison
Let \(X\sim N(\theta,1)\), with \(\theta\in\mathbb R\), and use squared-error loss. Compare \(\delta_1(X)=X\) and \(\delta_2(X)=2X\). To calculate their risks, write \(X=\theta+Z\), where \(E_\theta[Z]=0\) and \(E_\theta[Z^2]=1\).
For \(\delta_1\), the estimation error is \(X-\theta=Z\). Therefore $$ R_{\delta_1}(\theta)=E_\theta[Z^2]=1. $$ For \(\delta_2\), the error is \(2X-\theta=2(\theta+Z)-\theta=\theta+2Z\). Expanding the square and taking expectations gives $$ R_{\delta_2}(\theta)=E_\theta[(\theta+2Z)^2] =\theta^2+4\theta E_\theta[Z]+4E_\theta[Z^2] =\theta^2+4. $$ Thus, for every \(\theta\in\mathbb R\), $$ R_{\delta_1}(\theta)\leq R_{\delta_2}(\theta) \quad\text{because}\quad (\theta^2+4)-1=\theta^2+3>0. $$
The final inequality has been checked for the entire parameter space by an algebraic expression that is positive for every real \(\theta\). In a pointwise comparison, this universal check is what establishes the order.
Why the Relation Is a Partial Order
Proof of well-definedness. Suppose \([r]=[r']\) and \([s]=[s']\). By the definition of the equivalence classes, \(r(\theta)=r'(\theta)\) and \(s(\theta)=s'(\theta)\) for every \(\theta\in\Theta\). If \(r\preceq s\), then for each \(\theta\), $$ r'(\theta)=r(\theta)\leq s(\theta)=s'(\theta). $$ Hence \(r'\preceq s'\). The pointwise comparison therefore has the same truth value for every choice of representatives, so the order on classes is well-defined.
Reflexivity. Take any class \([r]\). For every \(\theta\), the real or extended-real inequality \(r(\theta)\leq r(\theta)\) holds. Consequently, \(r\preceq r\), and by definition \([r]\preceq[r]\).
Transitivity. Suppose \([r]\preceq[s]\) and \([s]\preceq[t]\). By the definition of the order, for every \(\theta\in\Theta\), $$ r(\theta)\leq s(\theta)\quad\text{and}\quad s(\theta)\leq t(\theta). $$ Transitivity of the ordinary order on \([0,\infty]\) gives \(r(\theta)\leq t(\theta)\) for every \(\theta\). Thus \(r\preceq t\), and \([r]\preceq[t]\).
Antisymmetry. Suppose \([r]\preceq[s]\) and \([s]\preceq[r]\). For every \(\theta\), these two assumptions give $$ r(\theta)\leq s(\theta)\quad\text{and}\quad s(\theta)\leq r(\theta). $$ Antisymmetry of the ordinary numerical order implies \(r(\theta)=s(\theta)\) at each \(\theta\). Hence \(r\sim s\), so \([r]=[s]\). This proves antisymmetry and completes the proof. \(\square\)
All three properties use pointwise numerical comparisons. The quotient matters specifically for antisymmetry: the conclusion is equality of the profiles at every parameter, which is exactly equality of their equivalence classes.
A Common Pitfall and Why the Distinction Matters
A common mistake is to treat procedure labels as the objects being ordered and then claim antisymmetry from two opposite inequalities. From \(R_{\delta_1}(\theta)\leq R_{\delta_2}(\theta)\) and \(R_{\delta_2}(\theta)\leq R_{\delta_1}(\theta)\) for every \(\theta\), the valid conclusion is that the two risk functions have equal values everywhere. That conclusion is equality of risk profiles; it is not, by itself, a claim that two labels or descriptions of procedures are identical. The order proved here is an order on risk profiles, represented formally by their equivalence classes.
A second error is to establish an inequality only at selected parameter values and then call it pointwise. If \(\Theta\) is infinite, checking several values does not establish the inequality at every remaining value. A proof must cover the whole parameter space, for example by deriving a formula for the difference and showing its sign for all \(\theta\), as in the normal-mean example.
This order supplies a clean language for risk comparisons: it separates the numerical profile from the way that profile is named or represented. It also makes the logical properties explicit. Whenever both directions of a pointwise inequality hold, the profiles agree everywhere; whenever a comparison is asserted, it must hold throughout the stated parameter space.
Check Your Understanding
Answer each question using the definition of pointwise order and the proof above.
- For \(\Theta=\{a,b\}\), suppose \(r(a)=1,\ r(b)=4,\ s(a)=1,\ s(b)=5\). Is \(r\preceq s\)? State the coordinate checks.
- Why does \(r\preceq s\) and \(s\preceq r\) imply \([r]=[s]\)?
- Which part of the partial-order proof requires identifying profiles that agree at every parameter value?
- For \(X\sim N(\theta,1)\) and squared-error loss, calculate the risk of the rule \(\delta(X)=X-1\).
- If an inequality between two risk profiles has been checked at five parameter values, what additional fact is needed to establish their pointwise order on an infinite parameter space?