Unbiased but Inconsistent Estimators

Unbiasedness Does Not Guarantee Consistency

An estimator can have the correct expected value for every sample size and still fail to become reliable as more observations are collected. Unbiasedness is a statement about the estimator’s average over repeated samples at a fixed sample size. Consistency is a statement about what happens to its probability of being far from the parameter as the sample size increases. These are different properties, and an estimator sequence can have one without the other.

The key diagnostic is often the sampling variance. For an unbiased estimator, a variance that tends to zero is enough to establish consistency. But variance that remains positive or grows is a warning, not a proof of inconsistency by itself. To prove failure of consistency, examine the probability of a fixed-size estimation error.

Definition: Let $T_n$ be an estimator of a parameter $\theta$, based on a sample of size $n$. The sequence is unbiased if $E_\theta[T_n]=\theta$ for every $n$ and every $\theta$ in the parameter space. It is consistent for $\theta$ if, for every $\varepsilon>0$, $P_\theta(|T_n-\theta|>\varepsilon)\to 0$ as $n\to\infty$. It is inconsistent if this convergence fails for at least one $\varepsilon>0$.

A sequence is not made consistent merely by writing it with a subscript $n$. It must use the growing sample in a way that controls estimation error. An estimator that always uses only the first observation, for example, can remain unbiased while retaining the variability of one observation forever.

Worked Example: An Unbiased Estimator That Always Uses One Observation

Suppose $X_1,\ldots,X_n$ are independent Bernoulli observations with success probability $p$. Consider $T_n=X_1$ for every $n$. Determine whether the sequence is unbiased and consistent when $p=0.4$, and calculate its variance.

Method. Check the expectation and variance under the stated Bernoulli model, then evaluate the probability of a fixed estimation error.

Since $E_p[X_1]=p$, we have $E_p[T_n]=p$ for every sample size. Thus $T_n$ is unbiased. A Bernoulli random variable with success probability $p$ has variance $p(1-p)$, so

\operatorname{Var}_p(T_n)=p(1-p)=0.4(0.6)=0.24

The variance does not depend on $n$. To check consistency directly, take $\varepsilon=0.3$. If $X_1=0$, then $|T_n-p|=|0-0.4|=0.4>0.3$. If $X_1=1$, then $|T_n-p|=|1-0.4|=0.6>0.3$. These are the only possible outcomes, so

$$ P_p(|T_n-p|>0.3)=P_p(X_1=0)+P_p(X_1=1)=0.6+0.4=1 $$

This probability is 1 for every $n$, not a quantity tending to zero. Therefore $T_n$ is inconsistent. Common mistake avoided: more observations are available as $n$ increases, but this estimator never uses them; availability of data alone does not make an estimator consistent.

Check. Its expectation is $0.4$, its variance is $0.24$, and its probability of error greater than $0.3$ is exactly 1, as required by the model.

Why Vanishing Variance Is a Useful Sufficient Condition

For an unbiased estimator, the mean squared error equals its variance. More importantly for consistency, Chebyshev’s inequality converts a variance bound into a probability bound. This gives a practical route from a calculation about sampling variability to a conclusion about convergence in probability.

Theorem: Suppose $E_\theta[T_n]=\theta$ for every $n$, and $\operatorname{Var}_\theta(T_n)\to 0$. Then $T_n$ is consistent for $\theta$. In particular, for every $\varepsilon>0$, $$ P_\theta(|T_n-\theta|>\varepsilon)\leq \frac{\operatorname{Var}_\theta(T_n)}{\varepsilon^2}\to 0. $$

Here is the justification. Since the estimator is unbiased, $E_\theta[T_n-\theta]=0$, and the variance of $T_n-\theta$ is $\operatorname{Var}_\theta(T_n)$. Chebyshev’s inequality says that a random variable with finite variance satisfies $P(|Z-E[Z]|>\varepsilon)\leq\operatorname{Var}(Z)/\varepsilon^2$. Apply it to $Z=T_n$ to obtain the displayed bound. If the variance tends to zero, the bound tends to zero for every fixed positive $\varepsilon$, which is precisely consistency.

The argument is a sufficient condition, not an equivalence. A positive limiting variance does not automatically settle consistency: a sequence can have rare, increasingly extreme values that keep its variance from vanishing even as the probability of a fixed-size error tends to zero. For a proof of inconsistency, the decisive test is whether some fixed $\varepsilon>0$ has an error probability that does not tend to zero.

Worked Example: An Unbiased Normal Estimator With Increasing Variance

Suppose $X_1,\ldots,X_n$ are independent with $X_i\sim N(12,4)$, so the target mean is $\mu=12$ and the variance of each observation is $4$. Consider $T_n=\overline{X}_n+\sqrt{n}(X_1-\overline{X}_n)$. Find its expectation and variance, and determine whether it is consistent.

Method. Use linearity of expectation and the variance of a sum, including the covariance between the sample mean and the centered term.

First, $E[\overline{X}_n]=12$ and $E[X_1-\overline{X}_n]=12-12=0$. Therefore

E[T_n]=E[\overline{X}_n]+\sqrt{n}\,E[X_1-\overline{X}_n]=12

Thus the estimator is unbiased at every $n$. For the variance calculation, independence gives $\operatorname{Var}(\overline{X}_n)=4/n$. Also,

\operatorname{Var}(X_1-\overline{X}_n) =4+\frac{4}{n}-2\operatorname{Cov}(X_1,\overline{X}_n) =4+\frac{4}{n}-2\left(\frac{4}{n}\right) =4\left(1-\frac{1}{n}\right)

The covariance of $\overline{X}_n$ and $X_1-\overline{X}_n$ is zero, because $\operatorname{Cov}(\overline{X}_n,X_1)=4/n$ and $\operatorname{Var}(\overline{X}_n)=4/n$. Hence

\begin{aligned} \operatorname{Var}(T_n) &=\operatorname{Var}(\overline{X}_n) +n\operatorname{Var}(X_1-\overline{X}_n)\\ &=\frac{4}{n}+4n\left(1-\frac{1}{n}\right) =4\left(n-1+\frac{1}{n}\right). \end{aligned}

At $n=4$, the variance is $4(4-1+1/4)=4(3.25)=13$. As $n$ increases, the variance grows without bound. More directly, $T_n$ is normally distributed with mean 12 and variance $4(n-1+1/n)$, since it is a linear combination of jointly normal observations. For any fixed $\varepsilon>0$,

P(|T_n-12|>\varepsilon) =2\left[1-\Phi\left(\frac{\varepsilon}{2\sqrt{n-1+1/n}}\right)\right]\longrightarrow 1,

where $\Phi$ is the standard normal cumulative distribution function. The argument of $\Phi$ tends to zero, so the probability tends to $2[1-\Phi(0)]=1$. Thus this estimator is not merely inconsistent: its probability of being more than any fixed positive distance from the target tends to 1. Common mistake avoided: the term involving $X_1-\overline{X}_n$ has mean zero, but its variance is multiplied by $n$; a zero-mean adjustment need not be negligible.

Check. The expectation remains 12, the variance at $n=4$ is 13, and the exact normal-tail expression has limit 1. These conclusions agree with the estimator’s increasing sampling variability.

Build an Unbiased Sequence With Persistent Variability

A useful construction starts with an unbiased estimator and adds a term whose expected value is zero. The added term must be handled carefully: if its variability remains substantial as $n$ grows, it can prevent consistency. Independent observations make it possible to calculate that variability and, in some models, to give a direct lower bound on the estimation-error probability.

Worked Example: A Centered Difference That Prevents Consistency

Let $X_1,\ldots,X_n$ be independent Bernoulli observations with $p=0.5$, and use the estimator $T_n=\overline{X}_n+X_1-X_2$, for $n\geq2$, to estimate $p$. Establish unbiasedness, calculate the variance, and prove inconsistency.

Method. The added difference has expectation zero. Calculate its covariance with the sample mean before adding variances, then use a fixed-error event to establish failure of consistency.

Because $E[X_1-X_2]=0.5-0.5=0$ and $E[\overline{X}_n]=0.5$, the estimator is unbiased. Write $\sigma^2=p(1-p)$, which is $0.25$ here. Independence gives

\operatorname{Var}(\overline{X}_n)=\frac{\sigma^2}{n}, \qquad \operatorname{Var}(X_1-X_2)=2\sigma^2.

Furthermore, $\operatorname{Cov}(\overline{X}_n,X_1)=\sigma^2/n$ and $\operatorname{Cov}(\overline{X}_n,X_2)=\sigma^2/n$, so $\operatorname{Cov}(\overline{X}_n,X_1-X_2)=0$. It follows that

\operatorname{Var}(T_n) =\frac{\sigma^2}{n}+2\sigma^2 =0.25\left(\frac{1}{n}+2\right).

For example, at $n=2$, this is $0.25(0.5+2)=0.625$, and as $n\to\infty$ it tends to $0.5$. To prove inconsistency, consider the event $X_1\ne X_2$. Its probability is $P(X_1=1,X_2=0)+P(X_1=0,X_2=1)=0.25+0.25=0.5$. On this event, $X_1-X_2$ is either 1 or $-1$. Since $0\leq\overline{X}_n\leq1$, if the difference is 1 then $T_n=\overline{X}_n+1$ and $T_n-0.5\geq0.5$; if the difference is $-1$, then $T_n=\overline{X}_n-1$ and $0.5-T_n\geq0.5$. Therefore

P(|T_n-0.5|>0.4)\geq P(X_1\ne X_2)=0.5

for every $n\geq2$. The probability cannot tend to zero, so $T_n$ is inconsistent. Common mistake avoided: a term that is centered, such as $X_1-X_2$, does not necessarily become small just because the sample mean uses more observations.

Check. The variance formula gives $0.625$ at $n=2$ and tends to $0.5$; the event argument gives an error probability of at least $0.5$ for the chosen threshold at every sample size.

How to Diagnose an Unbiased but Inconsistent Sequence

A reliable analysis separates the calculation of bias from the analysis of convergence. The following steps keep the logic clear.

Check unbiasedness for each sample size.
Compute $E_\theta[T_n]$ under the stated model. To claim unbiasedness, the result must equal $\theta$ for every $n$, not just in the limit.

Calculate the sampling variance.
Use the model’s variance and covariance rules. Do not discard covariance terms unless independence or a direct calculation justifies doing so.

Use a vanishing variance as a sufficient proof.
If the estimator is unbiased and its variance tends to zero, Chebyshev’s inequality establishes consistency.

For inconsistency, find a fixed-error probability.
Identify an $\varepsilon>0$ and show that $P_\theta(|T_n-\theta|>\varepsilon)$ does not tend to zero. A positive limiting variance alone does not complete this step.

The examples use two common constructions: ignoring all but a fixed observation, and adding a centered term whose variability persists or grows. Both preserve the target expectation, but neither ensures that the estimator concentrates around the target. The central exam distinction is therefore between an average property, $E_\theta[T_n]=\theta$, and a probability statement about increasingly accurate estimation.

Key takeaway: Unbiasedness controls the center of an estimator’s sampling distribution, not its spread. Vanishing variance is sufficient for consistency when the estimator is unbiased; to prove inconsistency, show that the probability of some fixed-size error fails to vanish.

Check Your Understanding

For each problem, check unbiasedness separately from consistency, and use a fixed-error probability when the question asks you to establish inconsistency.

Let $X_1,\ldots,X_n$ be independent Bernoulli observations with $p=0.3$, and set $T_n=X_1$. Find its expectation and variance, then calculate $P(|T_n-p|>0.2)$. (Answer: $E[T_n]=0.3$, $\operatorname{Var}(T_n)=0.21$, and the probability is $1$ for every $n$; the sequence is unbiased but inconsistent.)
Suppose $X_1,\ldots,X_n$ are independent with mean $\mu$ and variance $16$. Define $T_n=\overline{X}_n+X_1-\overline{X}_n$. Find its expectation and variance and state whether it is consistent. (Answer: $T_n=X_1$, so $E[T_n]=\mu$, $\operatorname{Var}(T_n)=16$, and it is not consistent in general.)
For independent observations with mean $\mu$ and variance $\sigma^2$, define $T_n=\overline{X}_n+c(X_1-X_2)$, where $c$ is a fixed constant and $n\geq2$. Find its expectation and variance. (Answer: $E[T_n]=\mu$ and $\operatorname{Var}(T_n)=\sigma^2/n+2c^2\sigma^2$, because $\operatorname{Cov}(\overline{X}_n,X_1-X_2)=0$.)
Let the observations be Bernoulli with $p=0.25$ and $T_n=\overline{X}_n+X_1-X_2$. Show that the sequence is inconsistent using $\varepsilon=0.5$. (Answer: $E[T_n]=0.25$, and on $X_1=1,X_2=0$, $T_n-0.25=\overline{X}_n+0.75>0.5$; this event has probability $0.25(0.75)=0.1875$ for every $n$, so the sequence is inconsistent.)
A sequence of random variables equals $\theta$ except with probability $1/n^2$; with probability $1/(2n^2)$ each, it equals $\theta+n$ or $\theta-n$. Is it unbiased? Is it consistent? What is its variance? (Answer: It is unbiased, consistent because $P(|T_n-\theta|>\varepsilon)=1/n^2\to0$ for every fixed $\varepsilon>0$, and its variance is $1$ for every $n$.)

What You'll Learn