Finite-Parameter Squared-Error Estimation

Why Finite Risk Comparisons Help

When the parameter space and the candidate estimator class are finite, an admissibility question can be made concrete: calculate each candidate’s risk at each parameter value, then compare the resulting lists of numbers. The comparison is not based on which estimator looks more reasonable or performs best on average. A rule is dominated only when another candidate has no greater risk at every parameter value and strictly smaller risk at at least one.

This tutorial focuses on that calculation and comparison under squared-error loss. Keeping the parameter values as separate coordinates is essential: an estimator that performs better at one value may perform worse at another.

Risk Vectors Under Squared-Error Loss

Definition: Let the parameter space be $\Theta=\{\theta_1,\ldots,\theta_m\}$, and let $\delta$ be an estimator of $\theta$. Under squared-error loss, its risk at $\theta_i$ is $$ R(\theta_i,\delta)=E_{\theta_i}\!\left[(\delta(X)-\theta_i)^2\right]. $$ If all these risks are finite, the risk vector of $\delta$, in the stated parameter order, is $$ \mathbf R(\delta)=\bigl(R(\theta_1,\delta),\ldots,R(\theta_m,\delta)\bigr). $$

For a finite observation space $\mathcal X$, write $p_{\theta_i}(x)=P_{\theta_i}(X=x)$. If $\delta(x)$ is the estimate reported after observing $x$, then each risk is obtained by summing squared errors weighted by their probabilities:

R(\theta_i,\delta)=\sum_{x\in\mathcal X}p_{\theta_i}(x)\bigl(\delta(x)-\theta_i\bigr)^2.

The formula uses the distribution at the particular parameter value $\theta_i$. Even when the same estimator is used at every parameter value, its risk generally changes because both the target $\theta_i$ and the distribution of $X$ can change.

Definition: Within a specified class of estimators, $\delta_1$ dominates $\delta_2$ if $R(\theta_i,\delta_1)\leq R(\theta_i,\delta_2)$ for every $i$, with strict inequality for at least one $i$. An estimator is admissible in that class if no estimator in the class dominates it.

The class matters: a rule may be admissible among the candidates under review but dominated by an estimator outside that class. In the comparisons below, admissibility always means admissibility relative to the explicitly listed finite class.

From Risk Calculations to Coordinate Comparisons

Theorem: For a finite parameter space and a finite class of estimators with finite risks, a candidate is dominated within the class exactly when another candidate’s risk vector is no larger in every coordinate and is strictly smaller in at least one coordinate.

Proof. By definition, domination requires the risk inequality at every parameter value and a strict inequality at some parameter value. The $i$th coordinate of a risk vector is precisely the risk at $\theta_i$. Thus the definition is equivalent to the stated coordinatewise comparison. In the other direction, if one risk vector satisfies those inequalities, translating each coordinate back to its parameter value gives exactly the conditions for domination. $\square$

This theorem gives a direct audit: write the parameter order once, calculate each vector in that same order, and check every pair. A single coordinate where the proposed dominator has greater risk is enough to rule out domination in that direction.

Worked Example: One Rule Dominated, Two Rules Admissible

Suppose $\Theta=\{0,1,2\}$, and the observation is $X=\theta$ with probability one. Under squared-error loss, consider three estimators, specified by their values at $X=0,1,2$:

\delta_A=(1,2,3),\qquad \delta_B=(1,1,1),\qquad \delta_C=(0.2,1.8,2.2).

Because the observation equals the parameter, each risk is just the squared difference between the estimator’s value at that observation and $\theta$. For $\delta_A$, the estimates exceed the parameter by one at all three values, so

\mathbf R(\delta_A)=\bigl((1-0)^2,(2-1)^2,(3-2)^2\bigr)=(1,1,1).

For the constant estimator $\delta_B=1$, the three squared errors are

\mathbf R(\delta_B)=\bigl((1-0)^2,(1-1)^2,(1-2)^2\bigr)=(1,0,1).

For $\delta_C$, they are

\mathbf R(\delta_C)=\bigl((0.2-0)^2,(1.8-1)^2,(2.2-2)^2\bigr)=(0.04,0.64,0.04).

Comparing $\delta_C$ with $\delta_A$, each coordinate is strictly smaller: $0.04<1$, $0.64<1$, and $0.04<1$. Therefore $\delta_C$ dominates $\delta_A$. But $\delta_C$ and $\delta_B$ are incomparable: $\delta_C$ has lower risk at $\theta=0$ and $\theta=2$, while $\delta_B$ has lower risk at $\theta=1$, since $0<0.64$. Neither dominates the other. Thus $\delta_A$ is inadmissible, while $\delta_B$ and $\delta_C$ are admissible within this three-estimator class.

Risk Calculations in a Random Model

Worked Example: Incomparable Rules in a Bernoulli Model

Let $X$ be Bernoulli, with $P_{\theta=0}(X=1)=0.1$ and $P_{\theta=1}(X=1)=0.9$. Compare $\delta_A(X)=X$, $\delta_B(X)=1-X$, and the constant estimator $\delta_C(X)=0.5$, using squared-error loss. The parameter order for the risk vectors is $(0,1)$.

At $\theta=0$, estimator $\delta_A$ reports 0 with probability 0.9 and 1 with probability 0.1. Its risk is

R(0,\delta_A)=0.9(0-0)^2+0.1(1-0)^2=0+0.1=0.1.

At $\theta=1$, $X=1$ with probability 0.9, so $\delta_A$ reports 1 with probability 0.9 and 0 with probability 0.1:

R(1,\delta_A)=0.9(1-1)^2+0.1(0-1)^2=0+0.1=0.1.

For $\delta_B=1-X$, at $\theta=0$ it reports 1 with probability 0.9 and 0 with probability 0.1. At $\theta=1$ it reports 0 with probability 0.9 and 1 with probability 0.1. Hence

R(0,\delta_B)=0.9(1-0)^2+0.1(0-0)^2=0.9,\qquad R(1,\delta_B)=0.9(0-1)^2+0.1(1-1)^2=0.9.

The constant estimator $\delta_C=0.5$ has squared error $0.25$ at either parameter value, regardless of $X$. Therefore

\mathbf R(\delta_A)=(0.1,0.1),\qquad \mathbf R(\delta_B)=(0.9,0.9),\qquad \mathbf R(\delta_C)=(0.25,0.25).

The risk vector of $\delta_A$ is strictly smaller in both coordinates than the other two vectors, so $\delta_A$ dominates both $\delta_B$ and $\delta_C$. Consequently, only $\delta_A$ is admissible in this class.

Worked Example: Checking Every Risk Coordinate

Keep the Bernoulli model from the preceding example and add the estimator $\delta_S$, which reports $0.1$ when $X=0$ and $0.9$ when $X=1$. At $\theta=0$, $X=0$ with probability $0.9$ and $X=1$ with probability $0.1$. Thus

R(0,\delta_S)=0.9(0.1-0)^2+0.1(0.9-0)^2 =0.9(0.01)+0.1(0.81)=0.009+0.081=0.09.

At $\theta=1$, $X=0$ with probability $0.1$ and $X=1$ with probability $0.9$. Therefore

R(1,\delta_S)=0.1(0.1-1)^2+0.9(0.9-1)^2 =0.1(0.81)+0.9(0.01)=0.081+0.009=0.09.

Its risk vector is $(0.09,0.09)$, strictly lower in both coordinates than $\delta_A$'s $(0.1,0.1)$, $\delta_B$'s $(0.9,0.9)$, and $\delta_C$'s $(0.25,0.25)$. Thus $\delta_S$ dominates each of those rules and is the only admissible estimator in the expanded class. The conclusion follows from the computed risks; it does not depend on whether the estimator’s form initially seems intuitively attractive.

What the Comparison Does—and Does Not—Say

A finite risk table gives an exact answer for the listed estimators and parameter values. It does not rank all estimators by a single overall score. In particular, if one rule has smaller risk at one parameter and larger risk at another, neither rule dominates the other under the pointwise definition. Such a comparison is inconclusive about dominance, not a reason to average the coordinates without an explicitly chosen weighting criterion.

There is also an important distinction between equality of risk vectors and domination. Under the definition used here, strict improvement at at least one parameter is required for domination. Therefore two different rules with identical risk vectors do not dominate one another; each remains admissible unless some other rule dominates it. Admissibility is about the available risk comparisons, not necessarily uniqueness of the estimator.

Common pitfall: A squared-error contribution is multiplied by the probability of the observation that produces it. For example, under $\theta=0$ in the Bernoulli model, $\delta_A(X)=X$ is wrong only when $X=1$, an event of probability $0.1$. Its risk is $0.1(1-0)^2=0.1$, not $0.9$. The correction is to list the possible observations, their probabilities, and the corresponding squared errors before adding the terms.

A reliable comparison proceeds in a fixed order: state the parameter order, calculate one risk at a time from its sampling distribution, assemble the risk vectors, and test each proposed domination across all coordinates. This makes both arithmetic errors and logical errors easier to catch.

Check Your Understanding

Use the stated parameter order in each question, and justify comparisons using the risk coordinates.

In the deterministic model $X=\theta$, where $\Theta=\{0,1\}$, calculate the risk vector of the constant estimator $\delta(X)=0.5$.
In the Bernoulli model of the worked examples, why must the risk of $\delta_A(X)=X$ at $\theta=0$ use the probability $P_0(X=1)=0.1$ for its nonzero squared-error term?
Suppose three rules have risk vectors $(1,2)$, $(2,1)$, and $(1,1)$. Which rules are dominated, and which are admissible within this class?
Two distinct estimators have the same risk vector. Does either dominate the other under the definition in this tutorial? Explain.
Can a rule with lower risk at one parameter value but higher risk at another dominate its competitor? State why or why not.