Unbiasedness Restrictions and Class-Relative Admissibility

Why the Comparison Class Matters

An estimator is called admissible only after specifying which competing rules are allowed. If the search is limited to unbiased estimators, a rule may have no admissible competitor in that restricted set even though a biased estimator has smaller risk. This distinction matters whenever unbiasedness is imposed as a constraint: it changes the set of possible dominators, not the risk function used to compare rules.

We will use squared-error loss for estimating a scalar parameter. The central example has a finite parameter space and a single Bernoulli observation, so every estimator and every risk can be calculated directly.

Definition: Let \(\mathcal C\) be a specified class of estimators, and let \(R(\theta,\hat\theta)\) denote the risk of estimator \(\hat\theta\) at parameter value \(\theta\). An estimator \(\hat\theta\in\mathcal C\) is admissible relative to \(\mathcal C\) if there is no \(\tilde\theta\in\mathcal C\) such that \(R(\theta,\tilde\theta)\leq R(\theta,\hat\theta)\) for every \(\theta\), with strict inequality for at least one \(\theta\). Such a \(\tilde\theta\) is said to dominate \(\hat\theta\) in \(\mathcal C\).

For an estimator of \(\theta\), unbiasedness means \(E_\theta[\hat\theta(X)]=\theta\) at every parameter value under consideration. Write \(\mathcal U\) for the class of unbiased estimators and \(\mathcal C\) for a larger class, such as all estimators in a specified decision problem. Relative admissibility always requires that the estimator being assessed belongs to the class named in the definition.

What Restricting the Class Can and Cannot Do

The basic relationship between the two notions follows from set inclusion. A smaller comparison class gives fewer potential dominators. It can therefore make admissibility easier to satisfy, but not harder.

Theorem: Suppose \(\mathcal U\subseteq\mathcal C\) and \(\hat\theta\in\mathcal U\). If \(\hat\theta\) is admissible relative to \(\mathcal C\), then it is admissible relative to \(\mathcal U\). The converse need not hold.

Proof. Suppose, to the contrary, that \(\hat\theta\) is inadmissible relative to \(\mathcal U\). Then there is some \(\tilde\theta\in\mathcal U\) whose risk is no greater than the risk of \(\hat\theta\) at every parameter value and is strictly smaller at at least one parameter value. Since \(\mathcal U\subseteq\mathcal C\), this same \(\tilde\theta\) belongs to \(\mathcal C\) and dominates \(\hat\theta\) there. That contradicts admissibility relative to \(\mathcal C\). Hence \(\hat\theta\) is admissible relative to \(\mathcal U\). The reverse implication does not follow: a dominator in \(\mathcal C\) may lie outside \(\mathcal U\). \(\square\)

Worked Example: Checking the Class-Inclusion Implication

In the Bernoulli model used below, let \(\mathcal C\) be all deterministic real-valued estimators based on one observation, and let \(\mathcal U\) be the unbiased estimators. The rule \(X\) belongs to \(\mathcal U\), so it also belongs to \(\mathcal C\). If \(X\) were admissible relative to \(\mathcal C\), the theorem would imply that it is admissible relative to \(\mathcal U\).

The implication runs only this way. A proof that \(X\) is admissible relative to \(\mathcal U\) rules out dominators in \(\mathcal U\); it says nothing about a biased estimator in \(\mathcal C\setminus\mathcal U\). The next examples calculate exactly such a case.

A Finite-Sample Bernoulli Example

Let \(X\sim\operatorname{Bernoulli}(\theta)\), where the parameter space is \(\Theta=\{2/5,3/5\}\). There is one observation, and the action is a real-valued estimate of \(\theta\), assessed under squared-error loss \(L(\theta,a)=(a-\theta)^2\). An estimator is a deterministic function of \(X\), so it is specified by two real numbers \(d_0=\hat\theta(0)\) and \(d_1=\hat\theta(1)\). Its risk is \(R(\theta,\hat\theta)=E_\theta[(\hat\theta(X)-\theta)^2]\).

Worked Example: Finding Every Unbiased Estimator

For an estimator with values \(d_0,d_1\), its expectation is \(E_\theta[\hat\theta(X)]=(1-\theta)d_0+\theta d_1\). Unbiasedness at both permitted parameter values requires

\frac{3}{5}d_0+\frac{2}{5}d_1=\frac{2}{5}, \qquad \frac{2}{5}d_0+\frac{3}{5}d_1=\frac{3}{5}.

Multiplying each equation by 5 gives \(3d_0+2d_1=2\) and \(2d_0+3d_1=3\). Subtracting the first equation from the second yields \(-d_0+d_1=1\), so \(d_1=d_0+1\). Substitution into the first equation gives \(3d_0+2(d_0+1)=2\), hence \(5d_0=0\) and \(d_0=0\). Therefore \(d_1=1\).

Thus the unique deterministic unbiased estimator is \(\hat\theta(X)=X\), and \(\mathcal U=\{X\}\). A class containing only one estimator has no distinct member that can dominate it, so \(X\) is admissible relative to \(\mathcal U\).

Worked Example: A Biased Rule Dominates the Unbiased Estimator

In the full class \(\mathcal C\), the constant estimator \(\tilde\theta(X)=1/2\) is available. At either allowed parameter value, its squared-error risk is

R(\theta,\tilde\theta)=\left(\frac12-\theta\right)^2 =\left(\frac{1}{10}\right)^2=\frac{1}{100}.

For the unbiased estimator \(X\), squared-error risk equals its variance because \(E_\theta[X]=\theta\). Since \(X\) is Bernoulli,

R(\theta,X)=\operatorname{Var}_\theta(X) =\theta(1-\theta) =\frac{2}{5}\frac{3}{5} =\frac{6}{25} =\frac{24}{100}.

These calculations hold at both \(\theta=2/5\) and \(\theta=3/5\). Consequently, \(R(\theta,\tilde\theta)=1/100<24/100=R(\theta,X)\) at every point in \(\Theta\). The constant estimator strictly dominates \(X\) in \(\mathcal C\), so \(X\) is inadmissible relative to all deterministic estimators even though it is admissible relative to the unbiased class.

The domination comparison does not contradict unbiasedness of \(X\): the competing rule is biased. In fact, its bias is \(E_\theta[\tilde\theta]-\theta=1/2-\theta\), which is \(1/10\) at \(2/5\) and \(-1/10\) at \(3/5\).

Bias, Variance, and a Larger Rule Family

The constant estimator is one member of a useful family that makes the risk trade-off explicit. For a real number \(c\), define \(\hat\theta_c(X)=cX+(1-c)/2\). At \(c=1\), this is the unbiased rule \(X\); at \(c=0\), it is the constant rule \(1/2\). Intermediate values combine the observation with the midpoint of the parameter space.

Worked Example: Calculating Risk Along a Shrinkage Family

First calculate the expectation: \(E_\theta[\hat\theta_c]=c\theta+(1-c)/2\). Subtracting \(\theta\), the signed bias is

E_\theta[\hat\theta_c]-\theta =c\theta+\frac{1-c}{2}-\theta =(c-1)\left(\theta-\frac12\right) =(1-c)\left(\frac12-\theta\right).

At \(\theta=2/5\), this bias is \((1-c)/10\); at \(\theta=3/5\), it is \(-(1-c)/10\). Thus its square is \((1-c)^2/100\) at either parameter value. Since \(\hat\theta_c\) is an affine function of \(X\), its variance is \(c^2\operatorname{Var}_\theta(X)=6c^2/25\). The squared-error risk is variance plus squared bias, giving the same risk at both parameter values:

R(\theta,\hat\theta_c) =\frac{6}{25}c^2+\frac{1}{100}(1-c)^2 =\frac{1}{100}(25c^2-2c+1).

For example, take \(c=1/25\). Then

R\left(\theta,\hat\theta_{1/25}\right) =\frac{1}{100}\left(\frac{1}{25}-\frac{2}{25}+1\right) =\frac{1}{100}\frac{24}{25} =\frac{6}{625} =0.0096.

This is less than \(1/100=0.01\), the risk of the constant estimator, and far less than \(6/25=0.24\), the risk of \(X\). The signed bias changes sign across the two parameter points, but its square—and hence the risk contribution—is the same at both.

The example also demonstrates that the unbiased estimator is not the only possible rule with an attractive risk profile. Bias can reduce variance enough to lower total risk. Here, the full estimator class admits a rule with uniformly much smaller risk than the unique unbiased estimator. The unbiasedness restriction removes that rule from the set being searched; it does not make the rule’s risk comparison disappear.

Common Pitfalls and Interpretation

Leaving the comparison class implicit. Saying that a rule is “admissible” without stating which estimators are allowed can obscure the claim. In this example, \(X\) is admissible relative to \(\mathcal U\), but inadmissible relative to \(\mathcal C\).
Forgetting class membership. Admissibility relative to \(\mathcal U\) is defined for rules in \(\mathcal U\). A statement comparing \(\mathcal U\) with \(\mathcal C\) must identify which class contains the estimator being assessed.
Treating unbiasedness as a risk guarantee. Unbiasedness sets the mean of the estimator; it does not ensure low variance or low risk. Under squared-error loss, risk includes both variance and squared bias.
Reversing the bias sign. Bias is expectation minus the target. For this family it is \((c-1)(\theta-1/2)=(1-c)(1/2-\theta)\). Reversing the sign does not change squared bias, but it gives an incorrect statement about the direction of bias at each parameter value.
Assuming a restricted-class result transfers upward. A rule admissible among unbiased estimators may be dominated by a biased estimator outside that class. The class-inclusion theorem only guarantees that admissibility in the larger class implies admissibility in the smaller one, provided the rule belongs to the smaller class.

Key takeaway: Admissibility is always relative to a specified collection of rules. Restricting comparisons to unbiased estimators can make a rule class-relative admissible, even when a biased estimator strictly dominates it among all estimators.

Check Your Understanding

Use the definitions and calculations above to answer the following questions.

Why does admissibility relative to a larger class imply admissibility relative to a smaller class containing the estimator?
In the Bernoulli example, what two equations must the values \(d_0,d_1\) of an unbiased estimator satisfy?
Calculate the risk of the constant estimator \(1/2\) at \(\theta=3/5\), and compare it with the risk of \(X\).
For the family \(\hat\theta_c\), what are the signed biases at \(\theta=2/5\) and \(\theta=3/5\)?
Does admissibility relative to the unbiased class imply admissibility among all deterministic estimators? Explain using the example.