Variance from Centered Squares

Why Center at the Mean?

A distribution describes how probability is spread across possible values. To measure that spread, we need a reference point and a way to keep deviations from cancelling. The mean supplies the reference point; squaring each deviation makes values above and below it contribute positively. Variance is the expected squared distance from the mean.

The centering matters: variance is not the expected square of \(X\), but the expected square of \(X\) after subtracting its mean. For an explicit calculation, first find the mean, then subtract it from each possible value, square those deviations, and average using the distribution’s probabilities or density.

Definition: Let \(X\) be a random variable with finite second moment, \(E[X^2]<\infty\), and let \(\mu=E[X]\). The variance of \(X\) is

\operatorname{Var}(X)=E[(X-\mu)^2]=E[(X-E[X])^2].

The quantity \(X-E[X]\) is the deviation from the mean. Variance is the expected square of that deviation.

The finite-second-moment condition ensures both that the mean exists and that the squared deviations have a finite expectation. The standard deviation is \(\sqrt{\operatorname{Var}(X)}\); unlike variance, it is expressed in the same units as \(X\).

Calculating Variance from a Distribution

The definition applies to every distribution, but its calculation takes a different form depending on how probability is represented. For a discrete variable, weight each centered square by its probability. For a continuous variable, integrate the centered square against the density. When a distribution has both point masses and a density, include both contributions.

Discrete formula: If \(X\) takes values \(x_i\) with probabilities \(p_i=P(X=x_i)\), then, provided the second moment is finite,

E[X]=\sum_i x_i p_i,\qquad \operatorname{Var}(X)=\sum_i (x_i-E[X])^2p_i.

For a countable set of possible values, the sums are over all its values.

Continuous formula: If \(X\) has density \(f\), then

E[X]=\int_{-\infty}^{\infty}x f(x)\,dx,\qquad \operatorname{Var}(X)=\int_{-\infty}^{\infty}(x-E[X])^2 f(x)\,dx.

The integrals are taken where the density is nonzero.

Mixed formula: Suppose the distribution consists of point masses \(p_i\) at values \(a_i\), together with an absolutely continuous part having density \(g\). If these parts account for all probability, then

E[X]=\sum_i a_i p_i+\int_{-\infty}^{\infty}xg(x)\,dx,\qquad \operatorname{Var}(X)=\sum_i(a_i-E[X])^2p_i+ \int_{-\infty}^{\infty}(x-E[X])^2g(x)\,dx.

Here \(\sum_i p_i+\int g(x)\,dx=1\). The density \(g\) represents the probability in the continuous part; it need not integrate to one by itself.

Worked Example: A Discrete Distribution

Let \(X\) take the values \(-1,0,2\) with probabilities \(0.2,0.5,0.3\), respectively. Find its mean and variance using centered squares.

First check that the probabilities sum to one: \(0.2+0.5+0.3=1\). The mean is

$$ E[X]=(-1)(0.2)+(0)(0.5)+(2)(0.3)=-0.2+0+0.6=0.4. $$

Center each possible value at \(0.4\), square the deviation, and weight by its probability:

\begin{aligned} \operatorname{Var}(X) &=(-1-0.4)^2(0.2)+(0-0.4)^2(0.5)+(2-0.4)^2(0.3)\\ &=(-1.4)^2(0.2)+(-0.4)^2(0.5)+(1.6)^2(0.3)\\ &=1.96(0.2)+0.16(0.5)+2.56(0.3)\\ &=0.392+0.080+0.768=1.24. \end{aligned}

Thus \(\operatorname{Var}(X)=1.24\). Each contribution is nonnegative, including the contribution from the value below the mean.

Worked Example: A Continuous Uniform Distribution

Suppose \(X\) is uniform on \([0,4]\), with density \(f(x)=1/4\) for \(0\leq x\leq4\) and zero elsewhere. Find its variance from the centered integral.

The density integrates to one, since \(\int_0^4(1/4)\,dx=1\). Its mean is

E[X]=\int_0^4 x\frac14\,dx =\frac14\left[\frac{x^2}{2}\right]_0^4 =2.

Now center at \(2\) and integrate the squared deviation:

\begin{aligned} \operatorname{Var}(X) &=\int_0^4(x-2)^2\frac14\,dx\\ &=\frac14\int_0^4(x^2-4x+4)\,dx\\ &=\frac14\left[\frac{x^3}{3}-2x^2+4x\right]_0^4\\ &=\frac14\left(\frac{64}{3}-32+16\right) =\frac14\cdot\frac{16}{3} =\frac43. \end{aligned}

The variance is \(4/3\), and the standard deviation is \(\sqrt{4/3}=2/\sqrt{3}\). The calculation uses the density’s value \(1/4\) throughout the interval; omitting that factor would not average with respect to the distribution.

Point Masses and Density Together

A mixed distribution can assign positive probability to an exact value while also spreading probability continuously over an interval. A density alone does not describe the point mass: the probability at that exact value must be added separately. The following calculation uses the same mean for both parts of the centered-square expectation.

Worked Example: A Mixed Distribution

Let \(X\) have probability \(1/3\) at \(0\), and have density \(g(x)=1/3\) on \(1\leq x\leq3\), with density zero elsewhere. The density part has total probability \(\int_1^3(1/3)\,dx=2/3\); together with the atom, the total probability is one. Find the mean and variance.

The atom contributes zero to the mean. The continuous part gives

E[X]=(0)\frac13+\int_1^3x\frac13\,dx =\frac13\left[\frac{x^2}{2}\right]_1^3 =\frac13\cdot4=\frac43.

For the variance, the atom contributes its probability times its squared deviation from \(4/3\), and the density contributes an integral:

\begin{aligned} \operatorname{Var}(X) &=\left(0-\frac43\right)^2\frac13 +\int_1^3\left(x-\frac43\right)^2\frac13\,dx\\ &=\frac{16}{27} +\frac13\int_1^3\left(x^2-\frac83x+\frac{16}{9}\right)\,dx\\ &=\frac{16}{27} +\frac13\left[\frac{x^3}{3}-\frac43x^2+\frac{16}{9}x\right]_1^3\\ &=\frac{16}{27}+\frac13\left(\frac{14}{9}\right) =\frac{16}{27}+\frac{14}{27} =\frac{10}{9}. \end{aligned}

The atom’s contribution is \(16/27\), and the continuous part’s contribution is \(14/27\). Adding them gives \(\operatorname{Var}(X)=10/9\). Treating the density as though it had total probability one, or leaving out the atom, would describe a different distribution.

Why the Definition Is Finite and Nonnegative

The following facts justify the usual variance definition and explain two of its basic properties. First, a finite second moment guarantees a finite centered second moment. Second, the variance can be zero only when the random variable equals its mean with probability one.

Theorem: If \(E[X^2]<\infty\), then \(E[X]\) exists as a finite number and \(E[(X-E[X])^2]<\infty\). Moreover, \(\operatorname{Var}(X)\geq0\), and

\operatorname{Var}(X)=0 \quad\Longleftrightarrow\quad P(X=E[X])=1.

Proof. By the Cauchy–Schwarz inequality, \(E[|X|]\leq\sqrt{E[X^2]}<\infty\), so \(\mu=E[X]\) is finite. For every real \(x\), \((x-\mu)^2\leq2x^2+2\mu^2\). Taking expectations shows \(E[(X-\mu)^2]\leq2E[X^2]+2\mu^2<\infty\). A square is nonnegative, so its expectation is nonnegative.

For the zero-variance claim, set \(Y=(X-\mu)^2\), which is nonnegative. If \(E[Y]=0\), then for each positive integer \(k\),

0=E[Y]\geq \frac{1}{k}P(Y\geq 1/k),

so \(P(Y\geq1/k)=0\). Since \(\{Y>0\}=\bigcup_{k=1}^{\infty}\{Y\geq1/k\}\), countable subadditivity gives \(P(Y>0)=0\). Thus \(X=\mu\) with probability one. Conversely, if \(X=\mu\) with probability one, then \((X-\mu)^2=0\) with probability one, and its expectation is zero. \(\square\)

There is also a useful converse about finiteness. If \(E[|X|]<\infty\) and \(E[(X-E[X])^2]<\infty\), then \(E[X^2]<\infty\), because \(X^2\leq2(X-E[X])^2+2(E[X])^2\). Thus, when the mean exists, finiteness of the centered second moment is equivalent to finiteness of the second moment.

Common Pitfalls

Squaring before centering. Calculating \(E[X^2]\) does not by itself give the definition of variance. The centered quantity is \((X-E[X])^2\); use it directly in the discrete sum, integral, or mixed calculation.
Forgetting to find the mean first. The deviations must be measured from \(E[X]\), not from zero or an arbitrary value. A wrong center changes every squared deviation.
Leaving out probability weights. A discrete squared deviation must be multiplied by its probability. A continuous squared deviation must be integrated against the density.
Dropping a point mass in a mixed law. The density describes only its continuous component. Add each atom’s probability times its centered square as a separate term.
Assuming zero variance means zero values. It means \(X\) equals its own mean with probability one. That constant need not be zero.

Key takeaway: Variance is the expectation of \((X-E[X])^2\). Find the mean first, then average the centered squares according to the full distribution, including any point masses.

Check Your Understanding

For each question, identify the centering value before calculating or interpreting variance.

A discrete variable takes values \(1\) and \(3\) with probabilities \(1/4\) and \(3/4\). What is its mean, and how would you write its variance as a centered sum?
A density is constant on \([2,6]\). What probability weight must appear in the variance integral, and why?
A distribution has an atom at \(0\) and a density on \([1,2]\). Which two contributions must appear in its centered-square expectation?
Explain why a random variable with zero variance must equal its mean with probability one.
If the mean exists but the second moment is infinite, can the variance be finite? Use the finiteness result to justify your answer.

What You'll Learn