Adding Means And Variances Across Independent Responses

Why Adding Responses Requires Two Separate Questions

When several responses are combined into a total, two questions matter: what are the total’s mean and variance, and what is its distribution? The first can often be answered from the component moments. The second requires more information. In particular, knowing that a sum has a certain mean and variance does not by itself establish that it is binomial or Poisson.

Throughout, let $X_1,\ldots,X_m$ be responses with finite second moments, and define their sum by $S=\sum_{j=1}^m X_j$. The responses may have different distributions and different means. Independence is the key condition for adding their variances without extra terms; it is not needed to add their means.

Definition: The sum of $m$ responses is $S=\sum_{j=1}^m X_j$. The responses are pairwise independent if every pair $X_j,X_k$, $j\ne k$, is independent. For the variance formula below, pairwise independence is sufficient; full mutual independence is a stronger condition that will be used in the distributional results.

Adding Means and Variances

The expectation of a finite sum is the sum of the expectations. This follows from linearity of expectation and does not require independence. Variance, in contrast, includes a covariance contribution for each pair of responses.

Theorem: Suppose $X_1,\ldots,X_m$ have finite second moments. Then $$ E[S]=\sum_{j=1}^m E[X_j] $$ and $$ \operatorname{Var}(S) =\sum_{j=1}^m\operatorname{Var}(X_j) +2\sum_{1\le j<k\le m}\operatorname{Cov}(X_j,X_k). $$ In particular, if the responses are pairwise independent, then $$ \operatorname{Var}(S)=\sum_{j=1}^m\operatorname{Var}(X_j). $$

Proof. By linearity of expectation, $E[S]=E[\sum_j X_j]=\sum_j E[X_j]$. For the variance, subtract the mean of the sum and expand the square:

\begin{aligned} \operatorname{Var}(S) &=E\left[\left\{\sum_{j=1}^m(X_j-E[X_j])\right\}^2\right]\\ &=\sum_{j=1}^m E[(X_j-E[X_j])^2] +2\sum_{1\le j<k\le m}E[(X_j-E[X_j])(X_k-E[X_k])]\\ &=\sum_{j=1}^m\operatorname{Var}(X_j) +2\sum_{1\le j<k\le m}\operatorname{Cov}(X_j,X_k). \end{aligned}

All terms are well-defined because finite second moments imply finite variances and covariances. If $X_j$ and $X_k$ are independent, their product expectation factors, so $E[X_jX_k]=E[X_j]E[X_k]$, and hence their covariance is zero. Pairwise independence therefore makes every covariance term vanish. $\square$

This result accommodates non-identical responses: each response contributes its own mean and variance. Identical distributions are not required. If responses are dependent, the covariance terms must be retained; some may be positive, negative, or zero.

Worked Example: Adding Independent Binomial Counts

Suppose $X_1\sim\operatorname{Binomial}(4,0.3)$ and $X_2\sim\operatorname{Binomial}(6,0.3)$ are independent. Let $S=X_1+X_2$. First calculate its mean and variance, then identify its distribution.

For a binomial response with size $n$ and success probability $p$, the mean is $np$ and the variance is $np(1-p)$. Thus $E[X_1]=4(0.3)=1.2$, $\operatorname{Var}(X_1)=4(0.3)(0.7)=0.84$, $E[X_2]=6(0.3)=1.8$, and $\operatorname{Var}(X_2)=6(0.3)(0.7)=1.26$. Independence gives $E[S]=1.2+1.8=3$ and $\operatorname{Var}(S)=0.84+1.26=2.10$.

To identify the distribution, use probability generating functions. For a nonnegative-integer-valued response $X$, its probability generating function is $G_X(z)=E[z^X]$, for $0\le z\le1$. The binomial theorem gives $G_{X_1}(z)=(0.7+0.3z)^4$ and $G_{X_2}(z)=(0.7+0.3z)^6$. Since the counts are independent, the generating function of their sum is the product:

G_S(z)=G_{X_1}(z)G_{X_2}(z) =(0.7+0.3z)^{10}.

This is the generating function of $\operatorname{Binomial}(10,0.3)$, so $S\sim\operatorname{Binomial}(10,0.3)$. Its mean $10(0.3)=3$ and variance $10(0.3)(0.7)=2.10$ agree with the separate moment calculation.

When Binomial Counts Add to a Binomial Count

The preceding calculation uses an important condition: the independent binomial counts share the same success probability. Their trial counts can differ. The result follows directly from the generating functions of the counts themselves; it does not require making any claim about how trial-level outcomes might be arranged across groups.

Theorem: If $X_1,\ldots,X_m$ are mutually independent and $X_j\sim\operatorname{Binomial}(n_j,p)$ for a common $p$, then $S=\sum_{j=1}^m X_j\sim\operatorname{Binomial}(\sum_{j=1}^m n_j,p)$.

Proof. The probability generating function of $X_j$ is $G_{X_j}(z)=(1-p+pz)^{n_j}$, by the binomial theorem. Independence implies that the generating function of the sum is the product of the individual generating functions. Therefore

G_S(z)=\prod_{j=1}^m(1-p+pz)^{n_j} =(1-p+pz)^{\sum_{j=1}^m n_j}.

The final expression is the generating function of a binomial response with size $\sum_j n_j$ and success probability $p$. A probability generating function determines the probabilities of a nonnegative-integer-valued response: its power-series coefficient of $z^r$ is $P(S=r)$. Thus $S$ has the stated binomial distribution. $\square$

Independent Poisson Counts

The corresponding result for Poisson responses allows different rate parameters. For $X\sim\operatorname{Poisson}(\lambda)$, the mean and variance are both $\lambda$, and its probability generating function follows by summing the Poisson probabilities.

Theorem: If $X_1,\ldots,X_m$ are mutually independent with $X_j\sim\operatorname{Poisson}(\lambda_j)$, where $\lambda_j\ge0$, then $S=\sum_{j=1}^m X_j\sim\operatorname{Poisson}(\sum_{j=1}^m\lambda_j)$.

Proof. For $0\le z\le1$, the probability generating function of $X_j$ is

G_{X_j}(z) =\sum_{r=0}^{\infty}z^r e^{-\lambda_j}\frac{\lambda_j^r}{r!} =e^{-\lambda_j}\exp(\lambda_j z) =\exp\{\lambda_j(z-1)\}.

By mutual independence, multiply these generating functions:

G_S(z)=\prod_{j=1}^m\exp\{\lambda_j(z-1)\} =\exp\left\{\left(\sum_{j=1}^m\lambda_j\right)(z-1)\right\}.

This is the generating function of a Poisson response with rate $\sum_j\lambda_j$. Uniqueness of the probability generating function gives the claimed distribution. Its mean and variance are both $\sum_j\lambda_j$, which also follows by adding the component means and variances. $\square$

Worked Example: Combining Independent Poisson Counts

Two independent clinics record daily event counts $X_1\sim\operatorname{Poisson}(2.4)$ and $X_2\sim\operatorname{Poisson}(1.6)$. Find the distribution, mean, and variance of their combined count $S=X_1+X_2$.

The Poisson-sum result applies because the counts are independent. The combined rate is $2.4+1.6=4$, so $S\sim\operatorname{Poisson}(4)$. Directly adding moments gives $E[S]=E[X_1]+E[X_2]=2.4+1.6=4$, and $\operatorname{Var}(S)=\operatorname{Var}(X_1)+\operatorname{Var}(X_2)=2.4+1.6=4$.

For a numerical check, the probability of no events at either clinic is $P(S=0)=e^{-4}\approx0.0183$. Independence gives the same calculation as $P(X_1=0)P(X_2=0)=e^{-2.4}e^{-1.6}=e^{-4}$.

When Moments Add but the Distribution Does Not

Independence guarantees the moment formulas, but it does not make every sum belong to the same family as its components. For example, binomial counts with different success probabilities generally do not sum to a binomial count. Their means and variances still add, but distributional closure cannot be concluded from those moments.

Worked Example: Different Success Probabilities

Let $X_1\sim\operatorname{Binomial}(1,0.2)$ and $X_2\sim\operatorname{Binomial}(1,0.7)$ be independent. Compute the moments of $S=X_1+X_2$, and check whether it is binomial.

The component means are $0.2$ and $0.7$, so $E[S]=0.9$. The component variances are $0.2(0.8)=0.16$ and $0.7(0.3)=0.21$, so independence gives $\operatorname{Var}(S)=0.16+0.21=0.37$.

The exact probabilities are $P(S=0)=(0.8)(0.3)=0.24$, $P(S=2)=(0.2)(0.7)=0.14$, and $P(S=1)=(0.2)(0.3)+(0.8)(0.7)=0.06+0.56=0.62$. They sum to $0.24+0.62+0.14=1$. If $S$ were $\operatorname{Binomial}(2,q)$, its mean would require $2q=0.9$, or $q=0.45$. That binomial distribution would have probability $(1-0.45)^2=0.3025$ at zero, not $0.24$. Thus $S$ is not binomial, despite being a sum of independent binomial responses.

Common Mistake and Why the Distinction Matters

A frequent error is to infer a distribution from its first two moments: for instance, to see that a total has mean $4$ and variance $4$ and declare it Poisson. Those moments are consistent with a Poisson distribution, but they do not prove that the total is Poisson. The Poisson-sum conclusion above follows from the component distributions, their independence, and the product of their generating functions—not from matching two moments alone.

Another error is to add variances while overlooking dependence. If two responses have positive covariance, then the variance of their sum exceeds the sum of their separate variances by twice that covariance. For a total of clinical counts, the independence assumption therefore has substantive meaning: if responses tend to rise and fall together, omitting covariance understates the uncertainty in the total.

Key takeaway: Means add for any finite collection of responses with finite expectations. Variances add when the responses are pairwise independent (or, more generally, when all covariance terms sum to zero). Distributional claims require additional conditions: independent binomial counts share a success probability to sum to a binomial response, while independent Poisson counts may have different rates.

Check Your Understanding

For each question, distinguish a moment calculation from a claim about the full distribution.

Suppose $X_1,\ldots,X_4$ are pairwise independent, with means $2,3,1,4$ and variances $1,2,1,3$. Find the mean and variance of their sum.
Does linearity of expectation require independence? State the condition needed for the sum’s expectation to be defined.
Two independent binomial counts have sizes 5 and 8 and share success probability $0.4$. State the distribution, mean, and variance of their sum.
Two independent Poisson counts have rates $1.2$ and $3.8$. State the distribution and calculate the probability their total is zero.
Why do the independent Bernoulli responses with success probabilities $0.2$ and $0.7$ in the example not sum to a binomial response?