Why Variances Add Under Independence
For independent random variables, the mean of a sum is the sum of the means, and the variance of a sum is the sum of the variances. The second statement has a specific hypothesis: independence makes the cross terms vanish when the square of a centered sum is expanded. Without that hypothesis, covariance terms remain and can change the answer substantially.
Throughout, \((\Omega,\mathcal{F},\mathbb{P})\) is a probability space, and all random variables in the results below are real-valued and measurable. A random variable \(X\) is square-integrable if \(\mathbb{E}[X^2]<\infty\). This assumption ensures that its mean and variance are finite. Random-variable identities in the derivations hold almost surely; expectations, variances and covariances are numerical quantities.
The products in these definitions are integrable: for example, Cauchy–Schwarz gives \(\mathbb{E}[|XY|]\le (\mathbb{E}[X^2])^{1/2}(\mathbb{E}[Y^2])^{1/2}<\infty\). The equivalent covariance formula follows by expanding the product and using linearity of expectation. We will also use the expectation-factorization result for independent random variables: if \(X\) and \(Y\) are independent and integrable, and \(XY\) is integrable, then \(\mathbb{E}[XY]=\mathbb{E}[X]\mathbb{E}[Y]\). In particular, it applies to square-integrable independent variables.
The Finite-Sum Formula
Proof. Write \(Z_i=X_i-\mathbb{E}[X_i]\). Each \(Z_i\) is square-integrable, has mean zero, and \(\sum_i Z_i=\sum_i X_i-\mathbb{E}[\sum_iX_i]\), by linearity of expectation. The finite sum is square-integrable, since \((\sum_i Z_i)^2\le n\sum_i Z_i^2\) pointwise. Expanding its square and taking expectations gives $$ \operatorname{Var}\left(\sum_{i=1}^{n}X_i\right) =\mathbb{E}\left[\left(\sum_{i=1}^{n}Z_i\right)^2\right] =\sum_{i=1}^{n}\mathbb{E}[Z_i^2]+2\sum_{1\le i<j\le n}\mathbb{E}[Z_iZ_j]. $$ Here the expansion is a finite sum, so linearity applies directly; all terms are integrable by Cauchy–Schwarz. By definition, \(\mathbb{E}[Z_i^2]=\operatorname{Var}(X_i)\) and \(\mathbb{E}[Z_iZ_j]=\operatorname{Cov}(X_i,X_j)\), proving the first identity. If \(X_i\) and \(X_j\) are independent, then \(Z_i\) and \(Z_j\) are independent, and expectation factorization gives \(\mathbb{E}[Z_iZ_j]=\mathbb{E}[Z_i]\mathbb{E}[Z_j]=0\). Under pairwise independence this holds for every pair \(i<j\); thus every covariance term vanishes. Mutual independence implies pairwise independence, so it gives the same conclusion. When \(n=1\), the pair sum is empty and the identity reduces to \(\operatorname{Var}(X_1)=\operatorname{Var}(X_1)\). \(\square\)
The mean formula needs no independence: for any finite family of integrable random variables, \(\mathbb{E}[\sum_iX_i]=\sum_i\mathbb{E}[X_i]\). Independence enters the variance calculation only through the cross terms. More generally, for real constants \(a_1,\ldots,a_n\), applying the same expansion to \(\sum_i a_iX_i\) gives $$ \operatorname{Var}\left(\sum_{i=1}^{n}a_iX_i\right) =\sum_{i=1}^{n}a_i^2\operatorname{Var}(X_i) +2\sum_{1\le i<j\le n}a_i a_j\operatorname{Cov}(X_i,X_j). $$ For an independent family, the second sum is zero. Constants do not affect variance: the formula with one variable gives \(\operatorname{Var}(aX+b)=a^2\operatorname{Var}(X)\), including \(a=0\).
Worked Example: A Sum of Three Independent Bernoulli Variables
Let \(X_1,X_2,X_3\) be independent Bernoulli random variables with success probabilities \(1/4,1/2,3/4\), respectively. Thus \(X_i\) takes values \(1\) and \(0\), with probabilities \(p_i\) and \(1-p_i\). Find the mean and variance of \(S=X_1+X_2+X_3\), and check the variance by computing the second moment.
For a Bernoulli variable with success probability \(p\), \(\mathbb{E}[X]=p\), since \(1\cdot p+0\cdot(1-p)=p\). Also \(X^2=X\), so \(\operatorname{Var}(X)=\mathbb{E}[X^2]-\mathbb{E}[X]^2=p-p^2=p(1-p)\). Therefore, $$ \begin{aligned} \mathbb{E}[S]&=\frac14+\frac12+\frac34=\frac{2+4+6}{8}=\frac32,\\ \operatorname{Var}(X_1)&=\frac14\cdot\frac34=\frac{3}{16},\\ \operatorname{Var}(X_2)&=\frac12\cdot\frac12=\frac14=\frac{4}{16},\\ \operatorname{Var}(X_3)&=\frac34\cdot\frac14=\frac{3}{16}. \end{aligned} $$ Independence makes each pairwise covariance zero. The variance formula gives \(\operatorname{Var}(S)=3/16+4/16+3/16=10/16=5/8\).
Check: Since \(X_i^2=X_i\), expansion gives \(\mathbb{E}[S^2]=\sum_i\mathbb{E}[X_i]+2\sum_{i<j}\mathbb{E}[X_iX_j]\). Independence factorizes the three pair expectations: $$ \mathbb{E}[X_1X_2]=\frac18,\qquad \mathbb{E}[X_1X_3]=\frac{3}{16},\qquad \mathbb{E}[X_2X_3]=\frac38. $$ Thus \(\mathbb{E}[S^2]=3/2+2(1/8+3/16+3/8)=3/2+22/16=23/8\). Subtracting \((\mathbb{E}[S])^2=(3/2)^2=9/4=18/8\) gives \(\operatorname{Var}(S)=23/8-18/8=5/8\), as before.
Worked Example: Computing Covariance from Independent Finite Laws
Let \(U\) and \(V\) be independent. Suppose \(U=-1\) with probability \(2/3\) and \(U=2\) with probability \(1/3\); suppose \(V=0\) with probability \(3/4\) and \(V=3\) with probability \(1/4\). Compute their means, variances and covariance, then find the mean and variance of \(T=U+V\).
Using the given probability laws, $$ \begin{aligned} \mathbb{E}[U]&=(-1)\frac23+2\frac13=0,& \mathbb{E}[U^2]&=1\frac23+4\frac13=2,\\ \mathbb{E}[V]&=0\frac34+3\frac14=\frac34,& \mathbb{E}[V^2]&=0\frac34+9\frac14=\frac94. \end{aligned} $$ It follows that \(\operatorname{Var}(U)=2-0^2=2\), while \(\operatorname{Var}(V)=9/4-(3/4)^2=9/4-9/16=27/16\). Independence and factorization give \(\mathbb{E}[UV]=\mathbb{E}[U]\mathbb{E}[V]=0\), so $$ \operatorname{Cov}(U,V)=\mathbb{E}[UV]-\mathbb{E}[U]\mathbb{E}[V]=0-0\cdot\frac34=0. $$ Consequently, \(\mathbb{E}[T]=0+3/4=3/4\), and \(\operatorname{Var}(T)=2+27/16=32/16+27/16=59/16\).
Check: Expanding the square gives \(\mathbb{E}[T^2]=\mathbb{E}[U^2]+2\mathbb{E}[UV]+\mathbb{E}[V^2]=2+0+9/4=17/4\). Hence \(\operatorname{Var}(T)=17/4-(3/4)^2=68/16-9/16=59/16\). This agrees with the variance formula, and the covariance calculation also shows explicitly where independence was used.
Worked Example: An Affine Combination of Independent Variables
Let \(X\) take values \(-1\) and \(1\), each with probability \(1/2\), and let \(Y\) be independent of \(X\), with \(\mathbb{P}(Y=1)=1/3\) and \(\mathbb{P}(Y=0)=2/3\). Set \(A=2X+3\) and \(B=4Y-1\). Find the means, variances and covariance of \(A,B\), and then find the mean and variance of \(A+B\).
First, \(\mathbb{E}[X]=(-1)/2+1/2=0\) and \(\mathbb{E}[X^2]=1\), so \(\operatorname{Var}(X)=1\). Also, \(\mathbb{E}[Y]=1/3\) and \(Y^2=Y\), so \(\operatorname{Var}(Y)=1/3-(1/3)^2=2/9\). Therefore, $$ \mathbb{E}[A]=2(0)+3=3,\qquad \operatorname{Var}(A)=2^2(1)=4, $$ and $$ \mathbb{E}[B]=4(1/3)-1=1/3,\qquad \operatorname{Var}(B)=4^2(2/9)=32/9. $$ Because \(A\) is a function of \(X\), \(B\) is a function of \(Y\), and \(X,Y\) are independent, \(A,B\) are independent. Thus \(\operatorname{Cov}(A,B)=0\), \(\mathbb{E}[A+B]=3+1/3=10/3\), and \(\operatorname{Var}(A+B)=4+32/9=68/9\).
Check: \(A\) takes values \(1,5\) with probability \(1/2\) each, so \(\mathbb{E}[A^2]=(1+25)/2=13\). The variable \(B\) takes values \(-1,3\) with probabilities \(2/3,1/3\), so \(\mathbb{E}[B^2]=1(2/3)+9(1/3)=11/3\). Independence gives \(\mathbb{E}[AB]=\mathbb{E}[A]\mathbb{E}[B]=3(1/3)=1\). Hence \(\mathbb{E}[(A+B)^2]=13+2(1)+11/3=56/3\), and $$ \operatorname{Var}(A+B)=\frac{56}{3}-\left(\frac{10}{3}\right)^2 =\frac{168}{9}-\frac{100}{9} =\frac{68}{9}. $$
When the Covariance Term Cannot Be Dropped
Independence is a sufficient condition for zero covariance, but it is not the definition of zero covariance; conversely, dependent random variables can have nonzero covariance. The general formula remains valid when variables are dependent, provided they are square-integrable. In that case, one must compute or otherwise account for the covariance terms rather than silently omit them.
Worked Example: The Error in Treating Dependent Variables as Independent
Let \(X\) be Bernoulli with success probability \(1/2\), and set \(Y=X\). Both variables are square-integrable, but they are not independent: for example, \(\mathbb{P}(X=1,Y=1)=1/2\), whereas \(\mathbb{P}(X=1)\mathbb{P}(Y=1)=1/4\). Find the variance of \(X+Y\).
Each variable has mean \(1/2\) and variance \(1/2-(1/2)^2=1/4\). Their product satisfies \(XY=X^2=X\), so \(\mathbb{E}[XY]=1/2\) and $$ \operatorname{Cov}(X,Y)=\frac12-\frac12\cdot\frac12=\frac14. $$ The correct formula gives \(\operatorname{Var}(X+Y)=1/4+1/4+2(1/4)=1\). A wrong calculation that treats them as independent would discard the covariance term and report \(1/2\). Directly, \(X+Y=2X\), which is \(2\) with probability \(1/2\) and \(0\) with probability \(1/2\); its mean is \(1\), its second moment is \(2\), and its variance is \(2-1^2=1\). The direct calculation confirms that the omitted covariance term matters.
For a finite independent family, the practical procedure is therefore straightforward: check square-integrability, compute each mean and variance from its law, and add the variances. For a weighted sum, multiply each variance by the square of its coefficient before adding. If independence is not available, use the covariance version of the formula. The special case of a constant random variable is included: its variance is zero, and adding it changes the mean but not the variance.
Check Your Understanding
In each question, state which assumptions justify the calculation, and distinguish independence from zero covariance.
- Let \(X_1,\ldots,X_4\) be independent Bernoulli variables with success probability \(1/3\). Find the mean and variance of their sum.
- If \(X,Y\) are independent and square-integrable, derive \(\operatorname{Var}(2X-3Y)\) in terms of \(\operatorname{Var}(X)\) and \(\operatorname{Var}(Y)\).
- For square-integrable \(X,Y\), expand \(\operatorname{Var}(X+Y)\) and state the condition on their covariance that makes it equal to the sum of their variances.
- Let \(Y=1-X\), where \(X\) is Bernoulli with success probability \(1/2\). Compute \(\operatorname{Cov}(X,Y)\) and \(\operatorname{Var}(X+Y)\).
- Explain why pairwise independence suffices for the variance formula for a finite sum, even though mutual independence is a stronger assumption.