Chebyshev's Inequality from Centered Second Moments

Why Centered Second Moments Control Tails

Knowing only the mean and variance of a random variable does not determine its distribution. It does, however, constrain how much probability can lie far from the mean. The key is to compare a large deviation with its square: whenever \(X\) is at least \(t\) units from its mean, its squared deviation is at least \(t^2\). Since the expected squared deviation is the variance, this pointwise comparison gives a bound on the probability of a large deviation.

The resulting inequality is useful precisely when more detailed distributional information is unavailable or unwarranted. It requires neither a density nor symmetry nor a particular family of distributions. Its cost is that a bound based on variance alone can be conservative.

The threshold \(t\) is a distance from the mean, and must be positive. The variance has units equal to the square of the units of \(X\), so the ratio \(\sigma^2/t^2\) is dimensionless, as a probability bound must be. If \(\sigma^2/t^2\) exceeds 1, the inequality remains true but may say no more than the elementary fact that a probability is at most 1.

Deriving the Inequality

The proof is an application of Markov’s inequality to the nonnegative random variable \((X-\mu)^2\). For completeness, Markov’s inequality says that if \(Z\ge 0\) almost surely, \(E[Z]<\infty\), and \(a>0\), then \(P(Z\ge a)\le E[Z]/a\). It follows by taking expectations in \(a\,\mathbf{1}_{\{Z\ge a\}}\le Z\).

This proof also shows why the mean must be the center used in the bound: the expectation of the squared centered variable is exactly the variance. More generally, applying Markov’s inequality to \((X-c)^2\) for another constant \(c\) gives a bound involving \(E[(X-c)^2]\), not necessarily the variance.

Standardized Form and Its Consequences

When \(\sigma>0\), the result can be expressed in units of standard deviations. For \(k>0\), set \(t=k\sigma\). Then

No assumption that the standardized variable is normally distributed is involved. The bound says, for example, that at most one quarter of the probability can lie at least two standard deviations from the mean, regardless of the distribution, provided its variance is finite.

Taking complements gives an equivalent way to describe concentration around the mean:

If a desired upper bound on the probability outside a tolerance \(t\) is \(\alpha\), where \(0<\alpha\le 1\), choose \(t=\sigma/\sqrt{\alpha}\). Then \(P(|X-\mu|\ge t)\le\alpha\). This is a guarantee based on variance alone; it need not be the smallest tolerance that would work if the distribution were known.

If \(\sigma^2=0\), then \(E[(X-\mu)^2]=0\). A nonnegative random variable with expectation zero is zero almost surely, so \(X=\mu\) almost surely. Thus the tail probability is zero for every \(t>0\), consistent with the inequality.

When the Bound Is Sharp

The example above is not an accident. For any prescribed \(t>0\) and any variance \(v\) satisfying \(0<v\le t^2\), one can construct a mean-zero random variable for which Chebyshev’s inequality is an equality. This establishes that no uniformly smaller upper bound depending only on \(v\) and \(t\) can replace \(v/t^2\) throughout this range.

Sharpness does not mean that every distribution attains the bound. A distribution may put very little probability far from its mean even when its variance is fixed. Chebyshev’s inequality is a worst-case guarantee over distributions with the specified second moment.

Applying the Bound to a Sample Mean

Chebyshev’s inequality can also control the error of an estimator when its variance is known. Suppose \(X_1,\ldots,X_n\) are independent and identically distributed, with \(E[X_i]=\mu\) and \(\operatorname{Var}(X_i)=\sigma^2<\infty\). Their sample mean is \(\bar X=n^{-1}\sum_{i=1}^n X_i\). Independence gives \(\operatorname{Var}(\bar X)=\sigma^2/n\), so the inequality applies directly to \(\bar X\).

The factor \(1/n\) makes the bound shrink as the sample size grows. To verify the variance calculation, independence makes every covariance between distinct observations zero:

The independence assumption matters for this displayed variance formula. Without it, the variance of the sum includes covariance terms, and the same \(1/n\) conclusion does not follow from identical marginal variances alone.

Common Mistakes and Practical Interpretation

• Forgetting the finite-variance condition. The inequality requires a finite mean and finite variance. If the variance is infinite, the stated right-hand side is not a finite variance bound.
• Using the variance instead of its square root as a distance. The threshold \(t\) has the same units as \(X\); the numerator \(\operatorname{Var}(X)\) has squared units. The ratio is \(\sigma^2/t^2\), not \(\sigma^2/t\).
• Assuming the bound is an exact probability. It is an upper bound. As the Bernoulli example shows, the actual tail probability can be much smaller.
• Assuming a particular distributional shape. Chebyshev’s inequality does not require normality, symmetry, or a continuous distribution. Its strength is that it applies without those assumptions.
• Ignoring a bound above one. If \(\sigma^2/t^2>1\), the inequality is true but uninformative. A probability cannot exceed one, so the useful combined bound is \(\min(1,\sigma^2/t^2)\).