Markov And Chebyshev Bounds For Convergence

Why Moment Bounds Matter

In an econometric problem, the distribution of an estimator or measured quantity may be difficult to characterize fully. Sometimes, however, its expectation or variance is known or can be bounded. Markov’s and Chebyshev’s inequalities turn that limited information into a bound on the probability of a large value or deviation. The bounds may not be sharp, but they can be enough to show that a sequence of random variables approaches a target in probability.

The key is to keep the conclusion proportional to the information available. A bound such as $P(|X-\mu|\geq t)\leq 0.25$ does not tell us the exact probability of that event; it says only that the probability cannot exceed $0.25$. For convergence in probability, a sequence of such upper bounds is useful when it tends to zero for every fixed positive tolerance.

Markov’s Inequality

Theorem (Markov’s inequality): If $X$ is a nonnegative random variable and $E[X]$ is finite, then for every $a>0$, $$ P(X\geq a)\leq \frac{E[X]}{a}. $$ The same upper bound also applies to $P(X>a)$, since $\{X>a\}$ is contained in $\{X\geq a\}$.

Nonnegativity is essential: the result concerns a variable whose values cannot be below zero. The inequality says that if $X$ has a small expectation relative to a threshold $a$, it cannot often be at least $a$. It does not require a particular distributional form.

The argument is a pointwise comparison. When $X\geq a$, the value of $X$ is at least $a$; when $X<a$, the indicator of that event is zero. Consequently, for every outcome, $$ X\geq a\,\mathbf{1}\{X\geq a\}, $$ where $\mathbf{1}\{X\geq a\}$ equals one when $X\geq a$ and zero otherwise. Taking expectations preserves the inequality. Since the expectation of an indicator is the probability of its event, $$ E[X]\geq a E[\mathbf{1}\{X\geq a\}] =aP(X\geq a). $$ Dividing by $a>0$ proves Markov’s inequality.

Worked Example: Bounding a Large Nonnegative Cost

Suppose a hypothetical monthly administrative cost $C$ is nonnegative and has expectation $E[C]=\$120$. What can be said about the probability that the cost is at least $\$600$?

The conditions hold: $C$ is nonnegative, its expectation is finite, and the threshold is positive. Markov’s inequality gives $$ P(C\geq \$600)\leq \frac{E[C]}{\$600} =\frac{\$120}{\$600}=0.20. $$

Thus the probability of a monthly cost of at least $\$600$ is no greater than $0.20$, or 20%. This is an upper bound, not a claim that the probability equals 20%. The expectation alone does not determine the exact tail probability.

Chebyshev’s Inequality

Markov’s inequality can be applied to a squared deviation. This produces a bound on how far a random variable can lie from its mean, using its variance.

Theorem (Chebyshev’s inequality): If $X$ has finite mean $\mu=E[X]$ and finite variance $\sigma^2=E[(X-\mu)^2]$, then for every $t>0$, $$ P(|X-\mu|\geq t)\leq \frac{\sigma^2}{t^2}. $$

To derive the result, let $Y=(X-\mu)^2$. This variable is nonnegative, and its expectation is the variance, $E[Y]=\sigma^2$. Also, the event $|X-\mu|\geq t$ is exactly the event $Y\geq t^2$. Applying Markov’s inequality to $Y$ at threshold $t^2>0$ yields $$ P(|X-\mu|\geq t) =P(Y\geq t^2) \leq \frac{E[Y]}{t^2} =\frac{\sigma^2}{t^2}. $$ This derivation shows why the variance appears in the numerator and why the distance from the mean is squared in the denominator. A larger variance permits a larger upper bound; increasing the deviation threshold reduces it.

Worked Example: Bounding a Deviation From a Mean

Suppose a hypothetical weekly index $Z$, measured in index points, has mean $40$ points and variance $25$ squared index points. Bound the probability that it differs from its mean by at least $10$ points.

The mean and variance are finite, so Chebyshev’s inequality applies with $t=10$ points. The variance has units of squared points, matching the squared threshold: $$ P(|Z-40|\geq 10) \leq \frac{25\ \text{points}^2}{(10\ \text{points})^2} =\frac{25}{100}=0.25. $$

The probability that the index is at least 10 points from its mean is at most $0.25$, or 25%. The inequality does not state that the probability is 25%; it may be lower. If the computed upper bound exceeds one, it remains mathematically valid but gives no useful improvement over the basic probability bound $P(A)\leq 1$.

From Bounds to Convergence in Probability

A sequence $X_n$ converges in probability to a constant $c$ if, for every fixed $\varepsilon>0$, the probability that $X_n$ is more than $\varepsilon$ from $c$ tends to zero:

Definition: $X_n$ converges in probability to $c$, written $X_n\xrightarrow{p}c$, if for every $\varepsilon>0$, $$ P(|X_n-c|>\varepsilon)\longrightarrow 0 \quad\text{as }n\longrightarrow\infty. $$

Markov’s inequality gives a useful sufficient condition. If $E[|X_n-c|]$ is finite, then $|X_n-c|$ is nonnegative, so $$ P(|X_n-c|>\varepsilon) \leq P(|X_n-c|\geq\varepsilon) \leq \frac{E[|X_n-c|]}{\varepsilon}. $$ Therefore, if $E[|X_n-c|]\to 0$, the right-hand side tends to zero for every fixed $\varepsilon>0$, and $X_n\xrightarrow{p}c$. The bound proves a sufficient condition; it does not say that this moment condition is necessary.

Chebyshev’s inequality gives another sufficient condition when the sequence’s means may vary. Write $\mu_n=E[X_n]$. If $\mu_n\to c$ and $\operatorname{Var}(X_n)\to 0$, then $X_n\xrightarrow{p}c$. To see this, fix any $\varepsilon>0$. For all sufficiently large $n$, $|\mu_n-c|\leq\varepsilon/2$. The triangle inequality then implies that $|X_n-c|>\varepsilon$ can occur only if $|X_n-\mu_n|>\varepsilon/2$. Hence $$ P(|X_n-c|>\varepsilon) \leq P(|X_n-\mu_n|\geq \varepsilon/2) \leq \frac{4\operatorname{Var}(X_n)}{\varepsilon^2} \longrightarrow 0. $$ The conditions matter: vanishing variance alone controls deviations from each $X_n$’s own mean, but does not ensure that those means approach $c$.

Worked Example: Markov’s Inequality Establishes Convergence

For each positive integer $n$, consider a hypothetical nonnegative random variable $X_n$ that equals $\sqrt{n}$ with probability $1/n$ and equals $0$ otherwise. Establish that $X_n$ converges in probability to $0$.

First compute its absolute first moment: $$ E[|X_n-0|] =\sqrt{n}\left(\frac{1}{n}\right)+0\left(1-\frac{1}{n}\right) =\frac{1}{\sqrt{n}}. $$ For any fixed $\varepsilon>0$, Markov’s inequality gives $$ P(|X_n|>\varepsilon) \leq P(|X_n|\geq\varepsilon) \leq \frac{E[|X_n|]}{\varepsilon} =\frac{1}{\varepsilon\sqrt{n}} \longrightarrow 0. $$ Since this holds for every positive $\varepsilon$, the definition gives $X_n\xrightarrow{p}0$. The bound is sufficient even though it need not equal the actual tail probability.

Worked Example: Chebyshev’s Inequality for a Sequence

Suppose a hypothetical sequence of estimators $T_n$ targets $c=5$ units and satisfies $E[T_n]=5$ units and $\operatorname{Var}(T_n)=9/n$ squared units. Use Chebyshev’s inequality to show convergence in probability to $5$ units.

The finite means equal the target, and the variances tend to zero. For any fixed tolerance $\varepsilon>0$ measured in units, $$ P(|T_n-5|>\varepsilon) \leq P(|T_n-5|\geq\varepsilon) \leq \frac{\operatorname{Var}(T_n)}{\varepsilon^2} =\frac{9}{n\varepsilon^2} \longrightarrow 0. $$

Thus $T_n\xrightarrow{p}5$ units. For example, with tolerance $\varepsilon=2$ units and $n=100$, the bound is $9/(100\cdot 4)=0.0225$. The probability of an error exceeding 2 units is therefore at most 0.0225 at that sample size, under the stated mean and variance conditions.

Using the Bounds Carefully

These inequalities are valuable because they require limited distributional information. Markov needs a finite expectation of a nonnegative variable; Chebyshev needs a finite variance. They can establish that large deviations become unlikely without specifying a full probability distribution. In asymptotic arguments, the main task is often to show that the relevant expectation or variance shrinks at a sufficient rate.

A common mistake is to report an inequality as an exact probability. For instance, from $P(|X-\mu|\geq t)\leq \sigma^2/t^2$, it is incorrect to conclude that the tail probability equals $\sigma^2/t^2$. The correction is to state that the event’s probability is no larger than the calculated bound. Another mistake is to use Chebyshev’s inequality without checking that the variance is finite, or to infer convergence to $c$ from shrinking variance while ignoring whether $E[X_n]$ approaches $c$.

Finally, a bound larger than one is not a contradiction; it is simply uninformative, since probabilities are already at most one. These inequalities are general tools, not guarantees of tight bounds. Their practical value depends on whether the available moment information makes the resulting upper bound small.

Check Your Understanding

Use the assumptions stated in each question, and distinguish an upper bound from an exact probability.

If $X\geq 0$ and $E[X]=8$, what upper bound does Markov’s inequality give for $P(X\geq 20)$? What does the bound mean?
A random variable has mean $12$ and variance $16$. Use Chebyshev’s inequality to bound the probability that it differs from its mean by at least $8$.
Why can Markov’s inequality be applied to $(X-\mu)^2$ when deriving Chebyshev’s inequality?
If $E[|X_n-c|]\to 0$, explain how Markov’s inequality establishes convergence in probability to $c$.
Why does $\operatorname{Var}(X_n)\to 0$ by itself not establish convergence in probability to a specified constant $c$?