Convergence in Probability and Probability Limits

Why Probability Limits Matter

In econometrics, many quantities are estimated from data whose sample size can grow: a sample average, a regression coefficient, or a measured frequency. To describe what happens as the sample size increases, we need a precise way to say that a random quantity gets close to a fixed target. Convergence in probability gives that meaning. It focuses on the probability of a deviation larger than any fixed tolerance, rather than requiring every possible outcome to be close.

The notation $\operatorname{plim} X_n=c$ says that the sequence of random variables $X_n$ converges in probability to the constant $c$. The index $n$ often represents sample size, but the definition applies to any sequence of random variables.

Definition: A sequence of random variables $X_n$ converges in probability to a constant $c$ if, for every $\varepsilon>0$, $$ P(|X_n-c|>\varepsilon)\longrightarrow 0 \quad \text{as } n\longrightarrow\infty. $$ When this holds, $c$ is called the probability limit of $X_n$, and we write $\operatorname{plim} X_n=c$.

The tolerance $\varepsilon$ is fixed before $n$ grows. The definition requires that the chance of being more than that tolerance away from $c$ becomes arbitrarily small. It does not require $X_n=c$ with high probability for any particular finite $n$, nor does it say that all outcomes eventually stay close forever. The latter is a different, stronger type of convergence.

For a deterministic sequence $x_n$, there is no sampling uncertainty: it can be viewed as a sequence of constant random variables $X_n=x_n$. Then the probability in the definition is either zero or one. Convergence in probability reduces to the familiar requirement that, for every $\varepsilon>0$, eventually $|x_n-c|\leq\varepsilon$.

Reading the Epsilon Bound

To verify a proposed probability limit, start with an arbitrary $\varepsilon>0$, express the event $|X_n-c|>\varepsilon$ in a useful way, and show that its probability tends to zero. The argument may use an exact probability, or a bound that becomes small. A single calculation at one tolerance is not enough: the definition requires the result for every positive tolerance.

Specify the proposed target.
State the constant $c$ and identify the sequence $X_n$ whose behavior is being studied.

Fix an arbitrary tolerance.
Let $\varepsilon>0$. Rewrite or bound the event $|X_n-c|>\varepsilon$.

Take the limit in $n$.
Show that the probability of the deviation event tends to zero for this arbitrary $\varepsilon$. That establishes convergence in probability.

Worked Example: A Deterministic Sequence

Consider the sequence $X_n=5+2/n$. Show that its probability limit is $5$, treating each $X_n$ as a constant random variable.

Fix any $\varepsilon>0$. The deviation from 5 is $2/n$, so

P(|X_n-5|>\varepsilon) = \begin{cases} 1, & 2/n>\varepsilon,\\ 0, & 2/n\leq\varepsilon. \end{cases}

Once $n\geq 2/\varepsilon$, the second case applies, and the probability is exactly zero. Thus, for every fixed positive tolerance, the deviation probability is eventually zero. Therefore $\operatorname{plim}X_n=5$. The cutoff depends on $\varepsilon$, which is permitted: the definition does not require one cutoff to work for every tolerance at once.

Probability Limits of Random Sample Averages

A sample average is itself a random variable when its observed values are random. To establish its probability limit, we apply the same epsilon definition to the average; being a sample average does not, by itself, establish convergence. The next examples use deliberately specified data-generating sequences, so their probability statements can be checked directly without relying on a general law for averages.

Worked Example: An Average With a Shared Random Shock

Let $Z$ have a uniform distribution on $(-1,1)$, and define observations $X_i=\theta+Z/i$, where $\theta$ is a fixed quantity. For each $n$, form the sample average $\bar{X}_n=(1/n)\sum_{i=1}^n X_i$. Find its probability limit.

Define $H_n=\sum_{i=1}^n 1/i$. Substituting the observations into the average gives

\bar{X}_n = \frac{1}{n}\sum_{i=1}^n\left(\theta+\frac{Z}{i}\right) = \theta+\frac{Z H_n}{n}.

Since $|Z|<1$, the absolute deviation is bounded by $H_n/n$. Also, $H_n\leq 1+\ln n$, and $(1+\ln n)/n$ tends to zero. For any fixed $\varepsilon>0$, choose $n$ large enough that $H_n/n\leq\varepsilon$. For every such $n$, $|\bar{X}_n-\theta|\leq\varepsilon$ for every possible value of $Z$. Therefore

P(|\bar{X}_n-\theta|>\varepsilon)=0

for all sufficiently large $n$, and $\operatorname{plim}\bar{X}_n=\theta$. This example is intentionally not an independent sample: every observation contains the same $Z$. The conclusion follows from the displayed bound, not from an independence assumption.

Worked Example: An Alternating Random Sample Average

Let $V$ be uniform on $(-2,2)$, and define $X_i=\theta+(-1)^iV$. The observations share the same random variable $V$, with alternating signs. Show that their sample average converges in probability to $\theta$.

In an even-sized sample, the $V$ terms cancel in pairs, so $\bar{X}_n=\theta$. In an odd-sized sample, one negative $V$ remains because the first sign is negative. Hence

\bar{X}_n-\theta = \begin{cases} 0, & n\text{ even},\\ -V/n, & n\text{ odd}. \end{cases}

Since $|V|<2$, for every $n$ we have $|\bar{X}_n-\theta|\leq 2/n$. Given any $\varepsilon>0$, when $n\geq 2/\varepsilon$, the deviation cannot exceed $\varepsilon$. Thus $P(|\bar{X}_n-\theta|>\varepsilon)=0$ for all sufficiently large $n$, and $\operatorname{plim}\bar{X}_n=\theta$. The example shows that exact independence is not part of the definition of convergence in probability. It does not imply that dependence is harmless in other settings; here the bound is what justifies the conclusion.

Probability Limits Are Unique

A sequence cannot converge in probability to two different constants. If it did, then for large $n$ it would have to be close to both constants with probability near one. When the constants are distinct, those two requirements are incompatible.

Justification: Suppose $X_n$ converges in probability to both $c$ and $d$, with $c\ne d$, and let $\delta=|c-d|>0$. The events $\{|X_n-c|<\delta/3\}$ and $\{|X_n-d|<\delta/3\}$ are disjoint: if both occurred, the triangle inequality would imply $|c-d|<2\delta/3$, contradicting $|c-d|=\delta$. But convergence to either constant makes its corresponding event have probability tending to one. Two disjoint events cannot both have probabilities tending to one. The supposition is impossible, so the probability limit, when it exists, is unique.

What a Probability Limit Does Not Say

Convergence in probability describes tail probabilities around a target. It does not, without further conditions, ensure that expectations converge to the same target. For example, let $X_n=n$ with probability $1/n$ and $X_n=0$ otherwise. For any fixed $\varepsilon>0$, once $n>\varepsilon$,

P(|X_n-0|>\varepsilon)=\frac{1}{n}\longrightarrow 0, \qquad E[X_n]=n\left(\frac{1}{n}\right)=1.

Thus $X_n$ converges in probability to zero, although its expectation remains one. Rare values far from zero become less likely, but their size grows enough to keep the expectation from approaching zero. A claim about a probability limit must therefore be justified using the deviation probabilities, not just by examining an expectation.

Another important distinction is between a finite-sample statement and an asymptotic one. For a particular $n$, a deviation probability may be small, large, or even zero. Convergence in probability concerns what happens as $n$ grows for every fixed positive tolerance. A result at one sample size, or at one chosen tolerance, cannot replace that limiting argument.

Common Mistakes

Using a changing tolerance without justification. The definition asks about every fixed $\varepsilon>0$. Showing a result only for a tolerance that changes with $n$ does not directly verify the definition.
Assuming the average must converge because it is an average. A sample average is random, and its behavior depends on the sequence of observations. Give an argument about its deviation probability.
Equating a probability limit with an expectation. The example above has probability limit zero but expectation one. They are different summaries, and one does not automatically determine the other.
Interpreting convergence as exact equality for large samples. The definition permits deviations at finite $n$. It says their probability becomes small for each fixed tolerance, not that the random variable equals its limit.

Key takeaway: To establish $\operatorname{plim}X_n=c$, fix an arbitrary $\varepsilon>0$ and show that $P(|X_n-c|>\varepsilon)$ tends to zero as $n$ grows. The conclusion is about the probability of a fixed-size deviation, not about exact equality or expectation.

Check Your Understanding

For each question, focus on the deviation probability required by the definition.

State the formal epsilon-bound definition of $X_n$ converging in probability to $c$.
For $X_n=3+1/n$, what is the proposed probability limit? Explain how the deviation event behaves for a fixed $\varepsilon>0$.
In the shared-shock sample-average example, which bound ensures that the deviation from $\theta$ becomes small?
Why does a calculation showing a small deviation probability for one value of $n$ not, by itself, prove convergence in probability?
Can a sequence converge in probability to zero while its expectation does not tend to zero? Explain using the rare-large-value example.