Why the Dependence Between Summands Matters
A sum can be easy to describe even when its terms are not independent: it may represent a combined score, total demand, or total number of events from several sources. Its expected value is straightforward to calculate, but its variance also depends on how the terms move together. Positive covariance increases the variance of the sum; negative covariance decreases it.
The key distinction is that expectations add without an independence assumption, whereas variances generally require covariance terms. This tutorial develops both formulas and uses them to calculate moments from distribution parameters and joint information. The results apply when the relevant expectations and variances are finite.
Here \(\operatorname{Cov}(X_i,X_j)=E[(X_i-\mu_i)(X_j-\mu_j)]\) measures how the two variables vary together. The sum over \(i<j\) includes each distinct pair once. For two variables, the general variance formula is
The covariance term is not optional unless it is known to be zero. When the summands are pairwise independent and have finite variances, their pairwise covariances are zero, so the formula simplifies to
For this variance calculation, pairwise independence is enough: it makes every covariance in the formula zero. Mutual independence is a sufficient, commonly encountered condition. A zero covariance also makes the covariance term vanish, but it does not in general imply independence.
Why the Formulas Hold
Linearity of expectation says that the expectation of a finite sum is the sum of the expectations. It does not require independence. For variance, center each variable by its mean, expand the square, and take expectations. This reveals exactly where dependence enters.
Since \(E[T]=\sum_i\mu_i\), we have \(T-E[T]=\sum_i(X_i-\mu_i)\).
The square of the centered sum contains each squared term and two copies of each distinct cross-product: \(\left(\sum_i(X_i-\mu_i)\right)^2=\sum_i(X_i-\mu_i)^2+2\sum_{i<j}(X_i-\mu_i)(X_j-\mu_j)\).
The squared terms have expectations \(\operatorname{Var}(X_i)\), and each cross-product has expectation \(\operatorname{Cov}(X_i,X_j)\). Thus \(\operatorname{Var}(T)=\sum_i\operatorname{Var}(X_i)+2\sum_{i<j}\operatorname{Cov}(X_i,X_j)\).
In particular, for two variables, the cross-product appears twice in the expansion: once as \((X-E[X])(Y-E[Y])\) and once in the reverse order. They are equal, which explains the factor of \(2\). If \(X\) and \(Y\) are independent, then \(E[XY]=E[X]E[Y]\), so \(\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y]=0\). This gives the independent-variable formula.
Worked Calculations
Worked Example: A Sum of Correlated Normal Measurements
Method: Use linearity for the mean and include the covariance term for the variance. In a hypothetical laboratory study, two measurements \(X\) and \(Y\) are jointly normal with mean vector \((12,8)\) and covariance matrix \(\begin{pmatrix}9&3\\3&4\end{pmatrix}\). Let \(T=X+Y\). Find its mean and variance. The stated matrix is a valid covariance matrix: its diagonal entries are positive and its determinant is \(9(4)-3^2=27>0\).
Mean. The individual means are \(E[X]=12\) and \(E[Y]=8\). Therefore,
Variance. The matrix gives \(\operatorname{Var}(X)=9\), \(\operatorname{Var}(Y)=4\), and \(\operatorname{Cov}(X,Y)=3\). Hence,
The standard deviation is \(\sqrt{19}\approx4.3589\), rounded to four decimal places. The result is checked by noting that the positive covariance makes the sum more variable than the variance \(9+4=13\) that would result if the measurements were independent. Common mistake avoided: adding only the two variances would incorrectly give \(13\); the covariance contributes \(6\).
Worked Example: Independent Counts From Two Sources
Method: Find each distribution’s mean and variance, then add the moments because the counts are independent. In a hypothetical monitoring period, \(X\), the number of alerts from one system, has a binomial distribution with parameters \(n=5\) and \(p=0.2\). Independently, \(Y\), the number of alerts from a second system, has a Poisson distribution with parameter \(\lambda=2\). For \(T=X+Y\), find the mean and variance.
For \(X\sim\operatorname{Binomial}(5,0.2)\), \(E[X]=np\) and \(\operatorname{Var}(X)=np(1-p)\). Substitution gives
For \(Y\sim\operatorname{Poisson}(2)\), \(E[Y]=\lambda=2\) and \(\operatorname{Var}(Y)=\lambda=2\). Independence means \(\operatorname{Cov}(X,Y)=0\), so
Thus the total has mean \(3\) alerts and variance \(2.8\) alerts-squared. The calculation checks against the general formula: the only possible covariance term is \(2\operatorname{Cov}(X,Y)=0\). Common mistake avoided: a binomial variance is \(np(1-p)\), not \(np\); using \(np\) here would give the wrong variance for \(X\).
Worked Example: Three Summands With Pairwise Covariances
Method: Add the three variances and twice the sum of the three distinct pairwise covariances. In a hypothetical joint-normal measurement model, \(X,Y,W\) have mean vector \((10,20,5)\) and covariance matrix
Let \(T=X+Y+W\). The diagonal entries give variances \(3,4,6\); the off-diagonal entries give \(\operatorname{Cov}(X,Y)=1\), \(\operatorname{Cov}(X,W)=2\), and \(\operatorname{Cov}(Y,W)=2\). The matrix is a valid covariance matrix; its leading principal minors are \(3\), \(11\), and \(46\), all positive.
First, add the means:
Next, include each covariance once inside the pairwise sum, then multiply that sum by \(2\):
The standard deviation is \(\sqrt{23}\approx4.7958\), rounded to four decimal places. The calculation includes three distinct pairs, not six: the outer factor of \(2\) already accounts for the two cross-products for each pair in the expanded square. Common mistake avoided: adding each covariance twice inside the parentheses and then multiplying by \(2\) would count every cross-product twice too many.
Using the Formulas Carefully
A reliable calculation starts by listing the information needed for the sum. For the mean, record each summand’s mean. For the variance, record each summand’s variance and the covariance for every distinct pair, unless a condition such as pairwise independence establishes that the covariances are zero. A table can help prevent missed terms when there are several summands.
| Information available | Mean of the sum | Variance of the sum |
|---|---|---|
| Individual means and variances only | Add the means | Not generally determined; pairwise covariances are also needed |
| Individual means, variances, and pairwise covariances | Add the means | Add variances and twice all distinct pairwise covariances |
| Pairwise independent summands with finite variances | Add the means | Add the variances |
For \(n\) summands there are \(n(n-1)/2\) distinct pairs. This count is useful as a check: for three variables there are three pairs, and for four variables there are six. Covariances can be negative, so the covariance contribution need not increase the variance. The final variance must nevertheless be nonnegative, as required by its definition. A negative result signals an arithmetic error or inconsistent input moments.
A frequent exam trap is to assume independence just because the problem gives each variable’s distribution separately. Marginal distributions alone do not specify the dependence between variables. If the question asks for the variance of a sum and neither independence nor covariance information is supplied, the variance may not be determined from the stated information. State that limitation rather than silently discarding covariance terms.
Check Your Understanding
For each problem, identify the moment formula that applies and keep the covariance terms whenever independence is not stated.
- Independent counts \(A\sim\operatorname{Binomial}(8,0.25)\) and \(B\sim\operatorname{Binomial}(4,0.5)\) are combined as \(S=A+B\). Find \(E[S]\) and \(\operatorname{Var}(S)\). (Answer: \(E[S]=4,\ \operatorname{Var}(S)=2.5\))
- Variables \(X,Y\) have means \(6,9\), variances \(4,16\), and covariance \(-3\). Find the mean and variance of \(X+Y\). (Answer: mean \(15\), variance \(14\))
- Three independent Poisson counts have parameters \(1.2,2.3,0.5\). Find the mean and variance of their total. (Answer: mean \(4\), variance \(4\))
- Variables \(A,B,C\) have means \(4,10,16\), variances \(5,7,8\), and covariances \(\operatorname{Cov}(A,B)=1\), \(\operatorname{Cov}(A,C)=-2\), and \(\operatorname{Cov}(B,C)=3\). Find the mean and variance of \(A+B+C\). (Answer: mean \(30\), variance \(24\))
- Indicators \(I,J\) record whether two events occur. Given \(P(I=1)=0.4\), \(P(J=1)=0.5\), and \(P(I=1,J=1)=0.25\), find the mean and variance of \(I+J\). (Answer: mean \(0.9\), variance \(0.59\))