Proving the Weak Law with Chebyshev’s Inequality

Why a Chebyshev Proof Is Useful

A sample mean typically varies less as the sample size increases. The weak law makes that idea precise: for any fixed positive tolerance, the probability that the sample mean differs from the population mean by at least that tolerance approaches zero. A proof using Chebyshev’s inequality requires only independence, identical distribution, and finite variance. It does not require a normal distribution or a central limit approximation.

The resulting inequality is a guarantee, not usually an exact probability. It is useful when a distribution is known only through its mean and variance, or when a conservative bound is sufficient. It can also be loose, so a large bound should not be mistaken for a claim that the deviation is likely.

Definition: Let $X_1,X_2,\ldots$ be independent and identically distributed random variables with $E[X_i]=\mu$ and $\operatorname{Var}(X_i)=\sigma^2<\infty$. The sample mean of the first $n$ observations is $\bar X_n=(X_1+\cdots+X_n)/n$. The weak law of large numbers states that, for every fixed $\varepsilon>0$, $P(|\bar X_n-\mu|\geq\varepsilon)\to 0$ as $n\to\infty$. This is convergence in probability of $\bar X_n$ to $\mu$.

The Chebyshev Bound

Chebyshev’s inequality applies to any random variable $Z$ with finite mean and variance. It bounds the probability of being at least a specified distance from the mean using the variance alone. The iid assumptions enter when we apply the inequality to the sample mean: independence and identical distribution give $\operatorname{Var}(\bar X_n)=\sigma^2/n$.

Theorem: If $Z$ has finite mean $E[Z]$ and finite variance $\operatorname{Var}(Z)$, then for every $\varepsilon>0$, $$ P\bigl(|Z-E[Z]|\geq\varepsilon\bigr) \leq \frac{\operatorname{Var}(Z)}{\varepsilon^2}. $$ Consequently, for iid observations with mean $\mu$ and finite variance $\sigma^2$, $$ P\bigl(|\bar X_n-\mu|\geq\varepsilon\bigr) \leq \frac{\sigma^2}{n\varepsilon^2}. $$

The right-hand side is a valid upper bound, but it can exceed 1. Since a probability cannot exceed 1, the sharper probability statement is $P(|\bar X_n-\mu|\geq\varepsilon)\leq\min\{1,\sigma^2/(n\varepsilon^2)\}$. When reporting the Chebyshev calculation, it is useful to distinguish the raw bound from this trivial probability cap.

Why the Inequality Holds

The proof starts with a pointwise comparison. For every outcome, the indicator of the event $|Z-E[Z]|\geq\varepsilon$ is at most the squared deviation divided by $\varepsilon^2$. On the event, that squared deviation is at least $\varepsilon^2$; outside the event, the indicator is zero. Taking expectations turns this comparison into the probability bound.

\mathbf{1}_{\{|Z-E[Z]|\geq\varepsilon\}} \leq \frac{(Z-E[Z])^2}{\varepsilon^2} \quad\Longrightarrow\quad P(|Z-E[Z]|\geq\varepsilon) \leq \frac{E[(Z-E[Z])^2]}{\varepsilon^2} =\frac{\operatorname{Var}(Z)}{\varepsilon^2}.

To use this result for $\bar X_n$, first calculate its variance. Identical distribution gives $\operatorname{Var}(X_i)=\sigma^2$, and independence makes every covariance between distinct observations zero. Thus,

\operatorname{Var}(\bar X_n) =\operatorname{Var}\left(\frac{1}{n}\sum_{i=1}^{n}X_i\right) =\frac{1}{n^2}\left(\sum_{i=1}^{n}\operatorname{Var}(X_i) +2\sum_{i<j}\operatorname{Cov}(X_i,X_j)\right) =\frac{n\sigma^2}{n^2} =\frac{\sigma^2}{n}.

Applying Chebyshev’s inequality to $Z=\bar X_n$, whose expectation is $\mu$, now gives the sample-mean bound. For fixed $\varepsilon>0$ and fixed finite $\sigma^2$, its right-hand side tends to zero:

0\leq P(|\bar X_n-\mu|\geq\varepsilon) \leq\frac{\sigma^2}{n\varepsilon^2} \longrightarrow 0.

The probability is nonnegative and bounded above by a quantity tending to zero, so it too tends to zero. This proves the weak law under the stated assumptions. If $\sigma^2=0$, each observation equals $\mu$ with probability one, and the conclusion holds directly.

Using the Result

A Chebyshev calculation has three inputs: the variance of one observation, the sample size, and the deviation threshold. The threshold is expressed in the same units as the observations; the variance is in squared units. The inequality concerns a deviation of the sample mean from the population mean, not a deviation of an individual observation.

Check the model.
Confirm that the observations are independent and identically distributed and that their common variance is finite.

Identify the target event.
Write the requested tolerance $\varepsilon$ in the observation’s units, and confirm that the event concerns $\bar X_n$.

Substitute into the bound.
Calculate $\sigma^2/(n\varepsilon^2)$, keeping the sample size in the denominator.

Interpret the result.
Report the raw bound and, if it exceeds 1, note that the probability is trivially at most 1. A bound is not an exact probability.

Worked Example: A Bound That Exceeds One

A hypothetical measurement has distribution $P(X=0)=0.25$, $P(X=2)=0.50$, and $P(X=4)=0.25$. Let $X_1,\ldots,X_{25}$ be iid measurements from this distribution. Find the Chebyshev bound for the sample mean’s deviation of at least $0.2$ from its expectation. State both the raw bound and the probability cap.

Method and model. Use the sample-mean form of Chebyshev’s inequality. The observations are independent and identically distributed with the stated three-point distribution; there is no interest or mortality assumption in this measurement model.

First calculate the mean: $E[X]=0(0.25)+2(0.50)+4(0.25)=0+1+1=2$. Then calculate the variance: $\operatorname{Var}(X)=0.25(0-2)^2+0.50(2-2)^2+0.25(4-2)^2=0.25(4)+0+0.25(4)=2$.

Here $n=25$ and $\varepsilon=0.2$, so $\varepsilon^2=0.04$. The raw Chebyshev bound is $2/(25(0.04))=2/1=2$. Therefore the requested probability is at most 2 by the raw inequality, but since probabilities cannot exceed 1, the useful capped bound is 1. To four decimal places, the raw bound is $2.0000$ and the capped bound is $1.0000$. This does not say that the deviation is certain; it says Chebyshev’s inequality provides no nontrivial upper bound here.

Common mistake avoided. Using $0.25$ as the population variance would produce a bound of $0.2500$, but that is incorrect: the variance is $2$, as the squared deviations above show. The raw answer is $2.0000$, not $0.2500$.

Worked Example: Bounding a Mean Deviation in Thousands of Dollars

In a hypothetical study, an individual’s annual service cost $X$, measured in thousands of dollars, is 10, 20, or 30 with probabilities $0.25$, $0.50$, and $0.25$, respectively. Suppose 200 independently selected individuals follow this same distribution. Bound the probability that the sample mean differs from the population mean by at least 1 thousand dollars.

Method and model. Apply Chebyshev’s inequality to the sample mean. The common distribution is the stated three-point distribution, with iid observations. The costs are already expressed in thousands of dollars; no discounting or mortality model is being applied.

The mean cost is $E[X]=10(0.25)+20(0.50)+30(0.25)=2.5+10+7.5=20$ thousand dollars. The variance is $\operatorname{Var}(X)=0.25(10-20)^2+0.50(20-20)^2+0.25(30-20)^2=25+0+25=50$ in squared thousands of dollars.

The deviation threshold is $\varepsilon=1$ thousand dollars, so $\varepsilon^2=1$, and $n=200$. Thus $P(|\bar X_{200}-20|\geq1)\leq50/(200(1)^2)=50/200=0.25$. The probability is at most $0.2500$, or 25%.

Check. The variance of the sample mean is $50/200=0.25$ squared thousands of dollars. Dividing this by the squared threshold $1$ gives the same bound, $0.25$.

Common mistake avoided. The deviation is 1 thousand dollars, not 1 dollar or 1,000 thousands of dollars. Keeping both the observation and threshold in thousands makes the squared units cancel correctly.

Worked Example: Choosing a Sufficient Sample Size

A hypothetical annual payment $X$ to a randomly selected member is either 0 or 40 dollars, each with probability $0.50$. Assume different members’ payments are independent and identically distributed. Find the smallest sample size $n$ for which Chebyshev’s inequality guarantees that the sample mean is within 5 dollars of its population mean with probability at least $0.95$.

Method and model. Calculate the individual-payment variance, then require the Chebyshev upper bound on the probability of a deviation of at least 5 dollars to be no more than $0.05$. The payment distribution and its probabilities are specified; no interest or mortality assumption is required for this one-period hypothetical model.

The mean payment is $E[X]=0(0.50)+40(0.50)=20$ dollars. The variance is $\operatorname{Var}(X)=0.50(0-20)^2+0.50(40-20)^2=0.50(400)+0.50(400)=400$ dollars squared.

For $\varepsilon=5$, the bound is $400/(n(5)^2)=400/(25n)=16/n$. To guarantee the requested probability, require $P(|\bar X_n-20|<5)\geq0.95$, which follows if $P(|\bar X_n-20|\geq5)\leq0.05$. It is sufficient to impose $16/n\leq0.05$. Solving gives $16\leq0.05n$, hence $n\geq320$.

At $n=320$, the bound equals $16/320=0.05$, so the guaranteed probability of a deviation strictly less than 5 dollars is at least $1-0.05=0.95$. The smallest integer sample size certified by this bound is 320.

Common mistake avoided. The sample-size condition uses the squared tolerance, $5^2=25$, and the desired failure probability, $0.05$. Replacing either with an unsquared tolerance or with $0.95$ would not express the required guarantee.

Common Exam Traps and Limitations

The proof is short, but its assumptions and interpretation matter. Independence is what removes covariance terms when finding the variance of the sample mean. Identical distribution ensures a common mean and variance for the observations. If observations are dependent, the variance may contain nonzero covariance terms, and the displayed $\sigma^2/n$ formula need not hold. If the variance is infinite, this finite-variance proof does not apply.

Dropping the factor $n$. The variance used in Chebyshev’s inequality is the variance of the sample mean, $\sigma^2/n$, not the variance $\sigma^2$ of one observation.
Using a standard deviation where a variance is required. The numerator is $\sigma^2$, and the denominator contains $\varepsilon^2$.
Treating the bound as an exact probability. Chebyshev gives an upper bound that may be conservative. A bound of $0.25$ means the probability is no more than $0.25$, not that it equals $0.25$.
Ignoring a bound above one. A raw bound greater than one is still the result of the inequality, but it is uninformative as a probability bound; the probability is also at most one.
Changing the tolerance as $n$ changes. The weak law fixes $\varepsilon>0$ and lets $n$ increase. A tolerance that shrinks with $n$ is a different question and is not established by this statement alone.

Key takeaway: For iid observations with finite variance, Chebyshev’s inequality gives $P(|\bar X_n-\mu|\geq\varepsilon)\leq\sigma^2/(n\varepsilon^2)$. For every fixed positive tolerance, this upper bound tends to zero as the sample size grows, which proves the weak law.

Check Your Understanding

For each problem, use the stated iid model and report a Chebyshev bound or sufficient sample size. Treat each scenario as hypothetical.

A device’s operating time $X$, in hours, is 1 or 3 with probability $0.50$ each. For $n=10$ independent devices, find the Chebyshev bound for the sample mean to differ from its expectation by at least 0.5 hour. (Answer: $0.4000$)
A measurement $X$ is 2, 4, or 8 with probabilities $0.50$, $0.25$, and $0.25$, respectively. For $n=30$ iid measurements, find the bound for a sample-mean deviation of at least 1 unit. (Answer: $0.2000$)
A hypothetical per-account processing cost $X$, in dollars, is 0, 10, or 30 with probabilities $0.50$, $0.25$, and $0.25$, respectively. For 50 iid accounts, find the bound for a deviation of at least 2 dollars from the population mean. (Answer: $0.7500$)
An iid population has variance $81$ square units. Find the smallest integer $n$ for which Chebyshev’s inequality guarantees probability at least $0.96$ that the sample mean is within 3 units of the population mean. (Answer: $225$)
An iid population has variance $12$ square units, and the sample size is 40. What is the smallest positive tolerance $\varepsilon$ for which the Chebyshev bound on $P(|\bar X_{40}-\mu|\geq\varepsilon)$ is at most $0.10$? (Answer: $\sqrt{3}$ units, approximately $1.7321$ units)

What You'll Learn