Tutorials › Statistical Theory I - Probability and Distribution Theory › Chebyshev’s Inequality from Centered Moments

Home  /  Tutorials  /  Moment Inequalities and Jensen

Moment Inequalities and Jensen · Tutorial 376 of 994

Chebyshev's Inequality from Centered Moments

Learn how a mean and variance give a distribution-free bound on the probability of a large deviation, and how to calculate and interpret that bound.

Intermediate 9 min read

What You'll Learn

  • State the moment conditions needed for Chebyshev’s inequality.
  • Derive the two-sided deviation bound from Markov’s inequality.
  • Convert the result into a bound measured in standard deviations.
  • Calculate and interpret bounds for specified distributions and sample proportions.
  • Recognize when the bound is loose or uninformative.

Why Centered Moments Bound Deviations

A mean describes a distribution’s center, and a variance describes the average squared distance from that center. Together, they can control the probability of a large deviation even when the full distribution is unknown. Chebyshev’s inequality makes this connection precise: among random variables with a fixed finite variance, the probability of lying far from the mean cannot be arbitrarily large.

The key move is to square the centered random variable. Squaring makes deviations in either direction nonnegative, while a deviation of at least \(t\) becomes a squared deviation of at least \(t^2\). Markov’s inequality can then bound the probability of that event using the expectation of the square, which is the variance.

Definition: Let \(X\) be a random variable with finite mean \(\mu=E[X]\) and finite variance \(\sigma^2=\operatorname{Var}(X)=E[(X-\mu)^2]\). For a threshold \(t>0\), a deviation of at least \(t\) means the event \(\{|X-\mu|\ge t\}\). The absolute value includes deviations both above and below the mean.

Chebyshev’s Inequality

Markov’s inequality states that if \(Y\) is nonnegative and \(E[Y]<\infty\), then, for any \(a>0\), \(P(Y\ge a)\le E[Y]/a\). Apply it to \(Y=(X-\mu)^2\) and \(a=t^2\). The following theorem records the result and its standard form in units of standard deviations.

Theorem (Chebyshev’s inequality): If \(X\) has finite mean \(\mu\) and finite variance \(\sigma^2\), then for every \(t>0\),
$$ P(|X-\mu|\ge t)\le \frac{\sigma^2}{t^2}. $$
If \(\sigma>0\), then for every \(k>0\),
$$ P(|X-\mu|\ge k\sigma)\le \frac{1}{k^2}. $$

The second statement follows by setting \(t=k\sigma\) in the first and simplifying \(\sigma^2/(k^2\sigma^2)=1/k^2\). When this upper bound exceeds 1, it is valid but uninformative; since every probability is also at most 1, the usable bound is the smaller of 1 and the displayed bound.

1
Choose a nonnegative random variable.
Set \(Y=(X-\mu)^2\). It is nonnegative, and its expectation exists and equals \(\sigma^2\) because the variance is finite.
2
Translate the deviation event.
Since \(t>0\), \(|X-\mu|\ge t\) holds exactly when \((X-\mu)^2\ge t^2\). Thus the two events have the same probability.
3
Apply Markov’s inequality.
Use \(Y\) and the positive threshold \(t^2\) to bound the event probability by \(E[Y]/t^2=\sigma^2/t^2\).

In full, the proof is the chain

$$ P(|X-\mu|\ge t) =P\big((X-\mu)^2\ge t^2\big) \le \frac{E[(X-\mu)^2]}{t^2} =\frac{\sigma^2}{t^2}. $$

The only moment used in this argument is the finite second centered moment. No density, symmetry, or particular distributional shape is required. This makes the inequality useful when a distribution is known only through its mean and variance.

If \(\sigma^2=0\), the same inequality gives \(P(|X-\mu|\ge t)=0\) for every \(t>0\). Consequently \(X=\mu\) almost surely: the variable has no positive-probability deviations from its mean. The standardized form using \(k\sigma\) is stated only for \(\sigma>0\), so it never divides by zero.

Worked Applications

Worked Example: A Deviation Bound for a Normal Variable

Let \(X\sim N(50,16)\), so \(E[X]=50\) and \(\operatorname{Var}(X)=16\). Bound the probability that \(X\) is at least 6 units from its mean.

Here \(t=6\), and the variance is \(\sigma^2=16\). Chebyshev’s inequality gives

$$ P(|X-50|\ge 6)\le \frac{16}{6^2} =\frac{16}{36} =\frac{4}{9}\approx 0.4444. $$

Equivalently, the event \(|X-50|<6\) has probability at least \(1-4/9=5/9\approx0.5556\). This is a distribution-free bound, so it does not use the additional information that \(X\) is normal. In this case the normal model can give a more precise probability, but the inequality remains valid without relying on that model.

Worked Example: Bounding a Tail From a Discrete Distribution

Suppose \(X\) takes the values \(-2,0,2\) with probabilities \(1/4,1/2,1/4\), respectively. Find its mean and variance, then use Chebyshev’s inequality to bound \(P(|X-E[X]|\ge1.5)\).

First calculate the mean:

$$ E[X]=(-2)\left(\frac14\right)+0\left(\frac12\right)+2\left(\frac14\right)=0. $$

The second moment is

$$ E[X^2]=(-2)^2\left(\frac14\right)+0^2\left(\frac12\right)+2^2\left(\frac14\right)=1+1=2. $$

Since \(E[X]=0\), the variance is \(\operatorname{Var}(X)=E[X^2]-(E[X])^2=2\). Therefore,

$$ P(|X-E[X]|\ge1.5) \le \frac{2}{(1.5)^2} =\frac{2}{2.25} =\frac{8}{9}\approx0.8889. $$

For this particular distribution, the event occurs exactly when \(X=-2\) or \(X=2\), so its actual probability is \(1/4+1/4=1/2\). The bound \(8/9\) is correct but loose. The example illustrates that Chebyshev’s inequality guarantees an upper limit; it does not claim that the limit equals the actual probability.

Worked Example: A Bound for a Sample Proportion

Assume \(X_1,\ldots,X_{400}\) are independent Bernoulli random variables with success probability \(p=0.20\), and let \(\bar X=400^{-1}\sum_{i=1}^{400}X_i\) be the sample proportion of successes. Bound the probability that \(\bar X\) differs from \(0.20\) by at least \(0.05\).

Each \(X_i\) has mean \(p=0.20\) and variance \(p(1-p)=0.20(0.80)=0.16\). Independence gives the variance of the sum as the sum of the variances, so

$$ E[\bar X]=0.20,\qquad \operatorname{Var}(\bar X) =\frac{1}{400^2}\operatorname{Var}\left(\sum_{i=1}^{400}X_i\right) =\frac{400(0.16)}{400^2} =\frac{0.16}{400} =0.0004. $$

Now take \(t=0.05\). Chebyshev’s inequality gives

$$ P(|\bar X-0.20|\ge0.05) \le\frac{0.0004}{0.05^2} =\frac{0.0004}{0.0025} =0.16. $$

Thus the probability of a deviation of at least five percentage points is no more than \(0.16\), or 16%. The calculation depends on the stated independence and common success probability through the variance of \(\bar X\); Chebyshev’s inequality itself only needs that \(\bar X\) has the calculated finite mean and variance.

Sharpness, Interpretation, and a Common Pitfall

The bound cannot generally be improved using only the mean and variance. To see why, fix \(t>0\) and \(q\) with \(0\le q\le1\). Define a variable that equals \(t\) with probability \(q/2\), equals \(-t\) with probability \(q/2\), and equals 0 with probability \(1-q\). Its mean is 0, and its variance is

$$ E[X^2]=t^2\left(\frac q2\right)+t^2\left(\frac q2\right)=qt^2. $$

Also, \(P(|X|\ge t)=q\), so \(\operatorname{Var}(X)/t^2=qt^2/t^2=q\). In this example Chebyshev’s inequality is exact. The distribution places just enough probability at the threshold and the remaining probability at the mean.

A common mistake is to treat the result as a one-sided bound, such as \(P(X-\mu\ge t)\le\sigma^2/t^2\), or to read it as an estimate of the exact tail probability. The theorem bounds the two-sided event \(|X-\mu|\ge t\), and its upper bound may be much larger than the actual probability. A second frequent error is to use \(t\) as a number of standard deviations without converting units: the bound for \(k\) standard deviations is \(1/k^2\), while the bound for a distance \(t\) in the units of \(X\) is \(\sigma^2/t^2\).

Chebyshev’s inequality is especially useful when only a mean and variance are available, or when a probability bound must hold across many possible distributional shapes. It also shows directly how increasing the threshold tightens the bound: replacing \(t\) by \(ct\), for \(c>1\), reduces the right-hand side by a factor of \(c^2\). For an application, first verify that the variance is finite, then express the event as a deviation from the mean, and finally keep the threshold in the same units as the variable.

Key takeaway: For any random variable with finite variance, the chance of being at least \(t\) away from its mean is at most \(\sigma^2/t^2\). This is a distribution-free upper bound, not an exact tail probability.

Check Your Understanding

Use the definitions and results in this tutorial to answer the following questions.

  1. If \(E[X]=12\) and \(\operatorname{Var}(X)=9\), what upper bound does Chebyshev’s inequality give for \(P(|X-12|\ge6)\)?
  2. State the moment condition needed to apply Chebyshev’s inequality. Is a density or symmetry assumption required?
  3. A random variable has standard deviation 4. What bound does Chebyshev’s inequality give for a deviation of at least 12 from its mean?
  4. Why does a bound greater than 1 provide no useful information, even though it is still a valid inequality?
  5. Give a three-point distribution centered at zero for which Chebyshev’s inequality is exact at a positive threshold \(t\).

Previous

Next

Back to Tutorials