Dependent Variance-of-a-Sum Problems

Why Dependence Changes the Variance of a Sum

If several random quantities are added, their individual variances do not usually tell the whole story. The quantities may tend to move together, or one may tend to be high when another is low. That dependence affects how much the total varies. For an exam problem, the key is to identify the variance of each term and then account for every pairwise covariance with the correct sign and coefficient.

The formula works for discrete or continuous random variables, provided the variables have finite variances. Independence is not required to use it. Independence, when it is actually justified, is one way to conclude that a covariance is zero; it is not a default assumption about variables merely because they are listed separately.

Definition: For random variables $X_1,\ldots,X_n$ with finite variances, the variance of their sum is the sum of their individual variances plus twice the sum of the covariances over all distinct unordered pairs: $$ \operatorname{Var}\!\left(\sum_{i=1}^{n}X_i\right) = \sum_{i=1}^{n}\operatorname{Var}(X_i) + 2\sum_{1\le i<j\le n}\operatorname{Cov}(X_i,X_j). $$

The restriction $i<j$ means that each pair appears once in the covariance sum. The factor of 2 accounts for the fact that, when the variance is expanded, both $\operatorname{Cov}(X_i,X_j)$ and $\operatorname{Cov}(X_j,X_i)$ occur. Since covariance is symmetric, these two terms are equal.

Why the Covariance Terms Appear

The formula follows directly from the definition of variance. Let $S=\sum_{i=1}^{n}X_i$, and suppose each variable has a finite second moment. Center each variable by subtracting its mean, expand the square, and take expectations:

\begin{aligned} \operatorname{Var}(S) &=E\!\left[\left(S-E[S]\right)^2\right]\\ &=E\!\left[\left(\sum_{i=1}^{n}(X_i-E[X_i])\right)^2\right]\\ &=\sum_{i=1}^{n}E[(X_i-E[X_i])^2] +2\sum_{1\le i<j\le n}E[(X_i-E[X_i])(X_j-E[X_j])]\\ &=\sum_{i=1}^{n}\operatorname{Var}(X_i) +2\sum_{1\le i<j\le n}\operatorname{Cov}(X_i,X_j). \end{aligned}

The second line uses linearity of expectation, so $E[S]=\sum_i E[X_i]$. In the expansion, each squared term gives an individual variance. Each product involving two different centered variables gives a covariance; it appears twice in the square, which produces the factor of 2.

For a weighted sum, the same expansion gives a useful version that should be used whenever coefficients are present. For constants $a_1,\ldots,a_n$,

\operatorname{Var}\!\left(\sum_{i=1}^{n}a_iX_i\right) = \sum_{i=1}^{n}a_i^2\operatorname{Var}(X_i) + 2\sum_{1\le i<j\le n}a_i a_j\operatorname{Cov}(X_i,X_j).

The individual variance terms contain squared coefficients. Each covariance term contains the product of the two coefficients, so a negative coefficient can change the sign of a contribution. Do not treat every covariance contribution as positive just because the covariance itself is positive.

A Reliable Calculation Process

For three or more variables, writing terms in a fixed order reduces omissions and duplicate counting. A covariance ledger lists the distinct pairs, their covariances, and the contribution they make to the total variance. For an unweighted sum, that contribution is twice the covariance; for a weighted sum, it is twice the coefficient product times the covariance.

Write the variables and coefficients.
For a plain sum, every coefficient is 1. For a weighted sum, record the sign and value of each coefficient before calculating.

List individual variances.
Add each variance once, multiplying by the square of its coefficient if the sum is weighted.

List every distinct pair.
For $n$ variables there are $n(n-1)/2$ unordered pairs. Include a covariance only if it is known or can be established from the information given.

Apply the factor of 2 and check.
Double the sum of the distinct-pair contributions, combine signs carefully, and confirm the final variance is nonnegative.

A zero covariance term can be omitted from the arithmetic, but the reason it is zero matters. Earlier material on independence establishes that independent variables with finite variances have zero covariance. The converse is not generally true: zero covariance alone does not establish independence. For calculating a variance of a sum, however, zero covariance is sufficient to make that pair's covariance contribution vanish.

Worked Example: A Dependent Binary Pair and an Independent Add-On

Let $X$ and $Y$ be binary random variables with joint probabilities $P(X=0,Y=0)=0.2$, $P(X=1,Y=0)=0.3$, $P(X=0,Y=1)=0.3$, and $P(X=1,Y=1)=0.2$. Let $Z$ be independent of the pair $(X,Y)$, with $P(Z=1)=0.5$ and $P(Z=0)=0.5$. Find $\operatorname{Var}(X+Y+Z)$. The method is to find the marginal variances and the covariance of the dependent pair, then use the variance-of-a-sum formula. The common mistake is to treat $X$ and $Y$ as independent because $Z$ is independent of them.

Step 1: Find the marginal distributions of $X$ and $Y$. From the joint probabilities, $P(X=1)=0.3+0.2=0.5$, and $P(Y=1)=0.3+0.2=0.5$. A binary variable with probability $p$ of being 1 has variance $p(1-p)$, so

\operatorname{Var}(X)=0.5(0.5)=0.25,\qquad \operatorname{Var}(Y)=0.5(0.5)=0.25,\qquad \operatorname{Var}(Z)=0.5(0.5)=0.25.

Step 2: Find the covariance of $X$ and $Y$. Their product is 1 only when both variables equal 1, so $E[XY]=P(X=1,Y=1)=0.2$. Also $E[X]=E[Y]=0.5$. Therefore,

\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y] =0.2-(0.5)(0.5)=-0.05.

Step 3: Use the independence information carefully. Since $Z$ is independent of the pair $(X,Y)$, it is independent of $X$ and of $Y$, so $\operatorname{Cov}(X,Z)=\operatorname{Cov}(Y,Z)=0$. No such independence is stated for $X$ and $Y$; their covariance is $-0.05$.

\begin{aligned} \operatorname{Var}(X+Y+Z) &=0.25+0.25+0.25+2(-0.05+0+0)\\ &=0.75-0.10\\ &=0.65. \end{aligned}

Check. The possible totals have probabilities $P(X+Y+Z=0)=0.1$, $P(X+Y+Z=1)=0.4$, $P(X+Y+Z=2)=0.4$, and $P(X+Y+Z=3)=0.1$. Thus $E[X+Y+Z]=1.5$, and $E[(X+Y+Z)^2]=0.4+1.6+0.9=2.9$. The variance is $2.9-(1.5)^2=0.65$, matching the covariance calculation.

Worked Example: Keeping Track of All Pairs in a Four-Variable Sum

Four variables have variances $2,3,4,5$, respectively. Their distinct pairwise covariances are $\operatorname{Cov}(X_1,X_2)=0.5$, $\operatorname{Cov}(X_1,X_3)=-0.25$, $\operatorname{Cov}(X_1,X_4)=0$, $\operatorname{Cov}(X_2,X_3)=0.75$, $\operatorname{Cov}(X_2,X_4)=0$, and $\operatorname{Cov}(X_3,X_4)=0.25$. Find the variance of $X_1+X_2+X_3+X_4$. Treat the stated variances and covariances as the inputs; no independence assumption is needed. The method is to list all six distinct pairs, sum their covariances, and double that sum. A common mistake is to add the distinct covariances only once.

Step 1: Add the individual variances.

$$ 2+3+4+5=14. $$

Step 2: Add the six distinct covariances. Keeping the negative and zero values in the list gives

$$ 0.5-0.25+0+0.75+0+0.25=1.25. $$

Step 3: Double the covariance total.

\operatorname{Var}(X_1+X_2+X_3+X_4) =14+2(1.25)=14+2.5=16.5.

Check. There are $4(4-1)/2=6$ distinct pairs, and all six were included. Adding the covariance total only once would give $14+1.25=15.25$, which is not the variance formula. The result $16.5$ is nonnegative, as a variance must be.

Coefficients and the Sign of a Contribution

A negative coefficient does not make a variance term negative, because the coefficient is squared in that term. It can, however, reverse the sign of a covariance contribution. The next example uses a covariance matrix whose entries are supplied directly; the calculation does not require inferring a dependence model from a scenario.

Worked Example: A Weighted Sum With Mixed Signs

Suppose $X,Y,Z$ have finite variances $\operatorname{Var}(X)=4$, $\operatorname{Var}(Y)=9$, and $\operatorname{Var}(Z)=1$, with covariances $\operatorname{Cov}(X,Y)=2$, $\operatorname{Cov}(X,Z)=-1$, and $\operatorname{Cov}(Y,Z)=0.5$. Find $\operatorname{Var}(2X-Y+3Z)$. The method is to use the weighted formula with coefficients $2,-1,3$. The main trap is to forget that the coefficient product for the $X,Y$ pair is negative.

Step 1: Calculate the individual variance contributions. The squared coefficients are $2^2=4$, $(-1)^2=1$, and $3^2=9$, so the sum of these contributions is

$$ 4(4)+1(9)+9(1)=16+9+9=34. $$

Step 2: Calculate each distinct-pair contribution. For $X,Y$, the contribution before doubling is $(2)(-1)(2)=-4$. For $X,Z$, it is $(2)(3)(-1)=-6$. For $Y,Z$, it is $(-1)(3)(0.5)=-1.5$. Their total is $-4-6-1.5=-11.5$.

Step 3: Double and combine.

\operatorname{Var}(2X-Y+3Z) =34+2(-11.5) =34-23 =11.

Check. The pair terms are all negative because their coefficient products and covariances have opposite signs. They reduce the sum of individual variance contributions from $34$ to $11$; they do not make the total variance negative.

Common Mistakes and Exam Checks

Assuming independence without evidence. A list of random variables is not a statement that they are independent. If a covariance is not given, infer that it is zero only when the problem provides a valid reason, such as independence or an explicit zero-covariance statement.
Omitting the factor of 2. For an unweighted sum, add each distinct covariance once, then double their total. Equivalently, write every ordered pair, but do not do both.
Counting a pair twice. In the compact formula, use $i<j$ so that $(X_i,X_j)$ and $(X_j,X_i)$ are not both included before doubling.
Losing a minus sign. In a weighted sum, the covariance contribution depends on $a_i a_j\operatorname{Cov}(X_i,X_j)$. Check the coefficient product as well as the sign of the covariance.
Using zero covariance to claim independence. Zero covariance removes a term from the variance calculation but, in general, does not establish independence.

Key takeaway: For a dependent sum, use the individual variances and every distinct pairwise covariance. Independence is one justification for a zero covariance, not a default assumption; for weighted sums, include the product of the pair's coefficients before doubling.

Check Your Understanding

For each problem, identify the covariance terms that belong in the calculation before simplifying.

$\operatorname{Var}(U)=4$, $\operatorname{Var}(V)=9$, and $\operatorname{Cov}(U,V)=-2$. Find $\operatorname{Var}(U+V)$. (Answer: $9$)
Three variables have variances $1,2,3$, and distinct covariances $\operatorname{Cov}(A,B)=0.5$, $\operatorname{Cov}(A,C)=-0.25$, and $\operatorname{Cov}(B,C)=0.75$. Find $\operatorname{Var}(A+B+C)$. (Answer: $8$)
Suppose $\operatorname{Var}(X)=1$, $\operatorname{Var}(Y)=4$, $\operatorname{Var}(Z)=9$, $\operatorname{Cov}(X,Y)=1$, $\operatorname{Cov}(X,Z)=0$, and $\operatorname{Cov}(Y,Z)=-1$. Find $\operatorname{Var}(X+2Y-Z)$. (Answer: $34$)
$\operatorname{Var}(P)=2$, $\operatorname{Var}(Q)=3$, and $\operatorname{Var}(R)=4$. Suppose $P$ and $Q$ have covariance $-1$, and $R$ is independent of the pair $(P,Q)$. Find $\operatorname{Var}(P+Q+R)$. (Answer: $7$)
$X$ and $Y$ are binary, with $P(X=1,Y=1)=0.3$, $P(X=1,Y=0)=0.2$, $P(X=0,Y=1)=0.1$, and $P(X=0,Y=0)=0.4$. Find $\operatorname{Var}(X+Y)$. (Answer: $0.69$)
$\operatorname{Var}(X)=4$, $\operatorname{Var}(Y)=9$, and $\operatorname{Cov}(X,Y)=-2$. Let $Z$ be independent of the pair $(X,Y)$, with $\operatorname{Var}(Z)=5$. Find $\operatorname{Var}(2X-Y+Z)$. (Answer: $38$)

What You'll Learn