Consistency and Convergence in Probability

Why Consistency Is a Long-Run Property

An estimator is judged not only by how it performs at one sample size, but also by what happens as the amount of data grows. A procedure can be unbiased and still have errors that do not become less likely as the sample size increases. Conversely, an estimator can have some bias at every finite sample size while its errors become increasingly unlikely to exceed any fixed tolerance.

Consistency formalizes this long-run idea. It describes whether an estimator concentrates near the parameter it is intended to estimate as the sample size increases. The key is not whether an estimator happens to be close in one sample, but how its sampling distribution behaves across increasing sample sizes.

Definition of Consistency

Let $\theta$ be a fixed parameter in a statistical model, and let $\hat{\theta}_n$ be an estimator based on a sample of size $n$. Probabilities for $\hat{\theta}_n$ are calculated under the model when the true parameter is $\theta$. The sequence $\hat{\theta}_n$ is consistent for $\theta$ if the probability of an error larger than any fixed positive tolerance approaches zero.

Definition: The estimator sequence $\hat{\theta}_n$ is consistent for $\theta$ if, for every $\varepsilon>0$, $$ \lim_{n\to\infty}P_\theta\!\left(\left|\hat{\theta}_n-\theta\right|>\varepsilon\right)=0. $$ This is also written $\hat{\theta}_n\xrightarrow{P}\theta$, meaning that $\hat{\theta}_n$ converges in probability to $\theta$.

The tolerance $\varepsilon$ is fixed while the sample size grows. The definition requires the condition to hold for every positive tolerance, not just one convenient choice. Equivalently, for any tolerance $\varepsilon>0$ and any desired probability bound $\alpha>0$, there is a sample size $N$ such that, for every $n\geq N$, the probability of an error exceeding $\varepsilon$ is less than $\alpha$.

Consistency is a property of a sequence of estimators, not a description of one estimate from one sample. A calculation at $n=100$, for example, can show how an estimator performs at $n=100$, but cannot by itself establish what happens as $n$ increases without a model for the sequence.

Worked Example: Error Probabilities Under a Normal Sampling Model

Consider a hypothetical estimator $\hat{\theta}_n$ of a claim-severity parameter $\theta$, measured in thousands of dollars. Suppose its sampling distribution is specified as $\hat{\theta}_n\sim N(\theta,4/n)$. The distribution is normal with mean $\theta$ and variance $4/n$, so its standard deviation is $2/\sqrt n$. This is a stipulated statistical model, not observed industry data. No interest rate or mortality assumption is involved because the example concerns a sampling distribution rather than discounted payments or survival.

Find the probability that the absolute estimation error exceeds $0.1$ when $n=400$, and when $n=1600$. Then assess whether the sequence is consistent.

Method. Standardize the estimation error using the specified normal distribution. Let $Z\sim N(0,1)$, and let $\Phi$ denote its cumulative distribution function.

At $n=400$. The variance is $4/400=0.01$, so the standard deviation is $\sqrt{0.01}=0.1$. Therefore,

P_\theta\!\left(\left|\hat{\theta}_{400}-\theta\right|>0.1\right) =P(|Z|>0.1/0.1) =2[1-\Phi(1)] =2(0.158655\ldots) \approx 0.3173.

At $n=1600$. The variance is $4/1600=0.0025$, and the standard deviation is $\sqrt{0.0025}=0.05$. Thus,

P_\theta\!\left(\left|\hat{\theta}_{1600}-\theta\right|>0.1\right) =P(|Z|>0.1/0.05) =2[1-\Phi(2)] =2(0.022750\ldots) \approx 0.0455.

The probability falls from $0.3173$ to $0.0455$ as the sample size increases from 400 to 1600. These two calculations illustrate the change, but consistency requires the limiting statement for every fixed tolerance. The normal-model argument for that statement is given below. A common calculation trap is to round a normal tail too early: using $2(0.1587)$ gives $0.3174$, whereas retaining the more precise tail gives $0.3173$. Both are close, but intermediate rounding should be reported consistently.

Worked Example: Unbiasedness Does Not Guarantee Consistency

Suppose a hypothetical estimator sequence for a parameter $\theta$ has $\hat{\theta}_n=\theta+Z_n$, where $Z_n\sim N(0,1)$ for every $n$. The sequence might represent a procedure whose effective measurement error does not improve when more observations are available. This example specifies the estimator's sampling distribution directly; no interest rate or mortality assumption applies.

Check unbiasedness. Since $E[Z_n]=0$,

E_\theta[\hat{\theta}_n]=E_\theta[\theta+Z_n]=\theta.

Thus, the estimator is unbiased for every sample size. Its variance is also $1$ for every $n$, since adding the constant $\theta$ does not change the variance.

Check consistency. Use the fixed tolerance $\varepsilon=0.5$. For every $n$,

P_\theta\!\left(\left|\hat{\theta}_n-\theta\right|>0.5\right) =P(|Z_n|>0.5) =2[1-\Phi(0.5)] =2(0.308537\ldots) \approx 0.6171.

This probability is the same for all $n$, so its limit is $0.6171$, not zero. The sequence is therefore unbiased but inconsistent. The mistake to avoid is concluding that an estimator is consistent merely because its expected value equals the target. Unbiasedness describes its center; consistency concerns the limiting probability of errors larger than every fixed tolerance.

Worked Example: A Biased but Consistent Estimator Sequence

For a second hypothetical estimator sequence, suppose $\hat{\theta}_n$ equals $\theta+3$ with probability $1/n$, and equals $\theta$ with probability $1-1/n$, for positive integer $n$. This is an abstract sampling model chosen to display the definition clearly. It does not model interest, mortality, or actual insurance data.

Find the bias. The expectation is

E_\theta[\hat{\theta}_n] =(\theta+3)\frac{1}{n}+\theta\left(1-\frac{1}{n}\right) =\theta+\frac{3}{n}.

The bias is $E_\theta[\hat{\theta}_n]-\theta=3/n$. It is nonzero for every finite $n$, so the estimator is biased at every finite sample size.

Calculate selected error probabilities. For the tolerance $\varepsilon=2$, the estimator's absolute error is either $0$ or $3$. Only the latter exceeds 2. Hence,

P_\theta\!\left(\left|\hat{\theta}_n-\theta\right|>2\right)=\frac{1}{n}.

At $n=50$, this probability is $1/50=0.0200$. At $n=200$, it is $1/200=0.0050$.

Verify consistency for every tolerance. If $0<\varepsilon<3$, the error exceeds $\varepsilon$ exactly when the estimator takes the value $\theta+3$, so the probability is $1/n$, which approaches zero. If $\varepsilon\geq 3$, the error never exceeds $\varepsilon$, so the probability is zero for every $n$. The defining limit is therefore zero for every $\varepsilon>0$. This sequence is consistent, despite being biased at every finite sample size.

A Normal-Model Justification

The first example belongs to a common general pattern. Suppose that, for each $n$, the estimator has sampling distribution $\hat{\theta}_n\sim N(\theta,\sigma^2/n)$, where $\sigma^2>0$ is fixed. This model has mean $\theta$ and standard deviation $\sigma/\sqrt n$. For a fixed $\varepsilon>0$, standardization gives

P_\theta\!\left(\left|\hat{\theta}_n-\theta\right|>\varepsilon\right) =P\!\left(|Z|>\frac{\varepsilon\sqrt n}{\sigma}\right) =2\left[1-\Phi\!\left(\frac{\varepsilon\sqrt n}{\sigma}\right)\right], \qquad Z\sim N(0,1).

As $n$ increases, $\varepsilon\sqrt n/\sigma$ increases without bound because $\varepsilon$ and $\sigma$ are fixed and positive. The standard normal upper-tail probability $1-\Phi(x)$ approaches zero as $x$ approaches infinity. Therefore the displayed error probability approaches zero for every $\varepsilon>0$, which proves consistency for this specified normal model. This justification uses the estimator's full sampling distribution; it is not a general proof based only on unbiasedness or variance.

Consistency, Unbiasedness, and Variance

These properties answer different questions and should be kept separate. Unbiasedness concerns the expectation at a given sample size. Variance measures spread around the expectation at that sample size. Consistency concerns a limit of error probabilities as the sample size grows.

Property	Question it answers	What it does not establish by itself
Unbiasedness	Is $E_\theta[\hat{\theta}_n]=\theta$ at the stated sample size?	That large errors become unlikely as $n$ grows
Variance	How variable is the estimator around its expectation at the stated sample size?	That the estimator is centered at $\theta$, or that its error probabilities tend to zero
Consistency	Does $P_\theta(\|\hat{\theta}_n-\theta\|>\varepsilon)$ tend to zero for every $\varepsilon>0$?	That the estimator is unbiased or optimal at any particular finite sample size

In particular, a small variance is not enough to infer consistency unless the estimator's position relative to $\theta$ is also addressed. An estimator can have little variation around the wrong target. Similarly, an unbiased estimator may retain a substantial probability of a large error, as the normal example demonstrated.

Common Exam Trap and Practical Interpretation

A frequent error is to check a convenient fact about one estimator—such as unbiasedness, or a small error probability at one $n$—and treat it as a limiting result. The definition demands more: fix any positive tolerance, examine the probability of exceeding it as $n$ grows, and verify that this probability tends to zero. A sequence that has an error probability of $0.01$ at one sample size could still have that same probability at every sample size.

For actuarial work, consistency is useful when considering whether an estimation procedure improves as more observations become available. For instance, an analyst might estimate a parameter in a hypothetical claim model from increasingly large portfolios. Consistency says that the probability of being farther than a chosen amount from the true parameter eventually becomes small. It does not promise that each estimate is close, prescribe how large the sample must be for a particular application, or rank procedures by their finite-sample performance.

When assessing a proposed estimator sequence, use this sequence of questions:

Identify the target and sampling model.
State the parameter $\theta$, the estimator $\hat{\theta}_n$, and the distribution or probability calculation that applies for each $n$.

Choose a fixed tolerance.
Write the error event $|\hat{\theta}_n-\theta|>\varepsilon$, with $\varepsilon>0$ held fixed as $n$ increases.

Examine the limit.
Determine whether the event's probability tends to zero for every positive tolerance. A single finite-$n$ result is not enough.

Key takeaway: Consistency means convergence in probability to the target parameter. It is a long-run statement about error probabilities—not a synonym for unbiasedness, low variance at one sample size, or guaranteed accuracy in every sample.

Check Your Understanding

For each problem, use the stated sampling model and distinguish a finite-sample calculation from a claim about consistency. Probabilities are rounded to four decimals.

A hypothetical estimator has $\hat{\theta}_n\sim N(\theta,9/n)$. Find $P_\theta(|\hat{\theta}_{100}-\theta|>0.3)$, and state whether the specified sequence is consistent. (Answer: The probability is $2[1-\Phi(1)]\approx 0.3173$; the sequence is consistent because the standardized tolerance $0.3\sqrt n/3$ tends to infinity.)
Suppose $\hat{\theta}_n=\theta+Z_n/\sqrt n$, where $Z_n\sim N(0,1)$. Calculate the probability of an error greater than $0.2$ at $n=25$ and $n=100$, and classify the sequence. (Answer: $0.3173$ at $n=25$, $0.0455$ at $n=100$; consistent.)
For $n\geq2$, an estimator equals $\theta+4$ with probability $2/n$, and equals $\theta$ otherwise. Find its bias at $n=80$, and find the probability that its absolute error exceeds 3 at $n=80$. Is it consistent? (Answer: Bias $=4(2/80)=0.1000$; error probability $=2/80=0.0250$; consistent.)
An estimator has $\hat{\theta}_n=\theta+2$ or $\theta-2$, each with probability $1/2$, for every $n$. Check unbiasedness and evaluate the error probability for tolerance $\varepsilon=1$. (Answer: Unbiased; $P_\theta(|\hat{\theta}_n-\theta|>1)=1$ for every $n$, so it is not consistent.)
Suppose $\hat{\theta}_n\sim N(\theta+1/\sqrt n,1/n)$. At $n=100$, find the bias and $P_\theta(|\hat{\theta}_{100}-\theta|>0.2)$, then assess consistency. (Answer: Bias $=0.1$; probability $=\Phi(-3)+1-\Phi(1)\approx0.1600$; consistent, since the bias tends to zero and the specified normal error probabilities tend to zero.)

Property	Question it answers	What it does not establish by itself
Unbiasedness	Is \(E_\theta[\hat{\theta}_n]=\theta\) at the stated sample size?	That large errors become unlikely as \(n\) grows
Variance	How variable is the estimator around its expectation at the stated sample size?	That the estimator is centered at \(\theta\), or that its error probabilities tend to zero
Consistency	Does \(P_\theta(\|\hat{\theta}_n-\theta\|>\varepsilon)\) tend to zero for every \(\varepsilon>0\)?	That the estimator is unbiased or optimal at any particular finite sample size

What You'll Learn