Weighted Sums of Independent Normal Variables

Why Weighted Normal Sums Stay Normal

A weighted sum combines two features that matter in distribution theory: the shape of the component distributions and the dependence between the variables. When the variables are independent and normal, their weighted sum is again normal. The weights need not be positive, equal, or sum to one. A negative weight changes the direction in which a variable contributes to the sum, but its contribution to the variance is still nonnegative.

Characteristic functions make the distributional argument direct: independence turns the characteristic function of a sum into a product, and the product of the resulting normal characteristic functions has the form of a normal characteristic function. We will also derive the mean and variance and examine the boundary case in which the sum has zero variance.

Definition: Let \(X_1,\ldots,X_m\) be independent random variables with \(X_i\sim N(\mu_i,\sigma_i^2)\), where \(\mu_i\in\mathbb{R}\) and \(\sigma_i^2\geq 0\). For fixed real weights \(a_1,\ldots,a_m\), the weighted sum is \(S=\sum_{i=1}^m a_iX_i\). The convention \(\sigma_i^2=0\) denotes the degenerate normal distribution concentrated at \(\mu_i\): \(P(X_i=\mu_i)=1\).

The characteristic function of a real-valued random variable \(X\) is \(\varphi_X(t)=E[e^{itX}]\), for real \(t\). We use two standard facts: independent variables have a joint expectation of a product equal to the product of their expectations when the factors are integrable; and two probability distributions with the same characteristic function are equal. The second fact is the uniqueness theorem for characteristic functions.

The Normal Characteristic Function

For clarity, first establish the characteristic function used in the main result. If \(Z\sim N(0,1)\), its density is \(\phi(z)=(2\pi)^{-1/2}e^{-z^2/2}\), and its characteristic function is

\varphi_Z(t)=e^{-t^2/2}.

Here is a derivation. Since \(E[|Z|]\) is finite, differentiation under the expectation is justified by dominated convergence: the absolute value of the difference quotient of \(e^{itZ}\) is at most \(|Z|\). Thus \(\varphi_Z'(t)=iE[Ze^{itZ}]\). Because \(\phi'(z)=-z\phi(z)\), integration by parts gives

E[Ze^{itZ}] =-\int_{-\infty}^{\infty} e^{itz}\phi'(z)\,dz =\int_{-\infty}^{\infty} it e^{itz}\phi(z)\,dz =it\varphi_Z(t).

The boundary term vanishes because \(\phi(z)\) tends to zero as \(z\) tends to either infinity and \(|e^{itz}|=1\). Consequently \(\varphi_Z'(t)=-t\varphi_Z(t)\), with \(\varphi_Z(0)=1\). Solving this differential equation yields \(\varphi_Z(t)=e^{-t^2/2}\). For \(\sigma^2>0\), write \(X=\mu+\sigma Z\), where \(\sigma\) is the positive square root of \(\sigma^2\). It follows that

\varphi_X(t)=E[e^{it(\mu+\sigma Z)}] =e^{it\mu}\varphi_Z(\sigma t) =\exp\!\left(it\mu-\frac{\sigma^2t^2}{2}\right).

When \(\sigma^2=0\), \(X=\mu\) almost surely, so its characteristic function is \(e^{it\mu}\), which is the same formula with \(\sigma^2=0\).

Main Result: Distribution, Mean, and Variance

Theorem: Let \(X_1,\ldots,X_m\) be independent, with \(X_i\sim N(\mu_i,\sigma_i^2)\), and let \(a_1,\ldots,a_m\) be real constants. Then \(S=\sum_{i=1}^m a_iX_i\) has distribution \(N\!\left(\sum_{i=1}^m a_i\mu_i,\ \sum_{i=1}^m a_i^2\sigma_i^2\right)\). In particular, \(E[S]=\sum_{i=1}^m a_i\mu_i\) and \(\operatorname{Var}(S)=\sum_{i=1}^m a_i^2\sigma_i^2\). If \(\sum_{i=1}^m a_i^2\sigma_i^2=0\), then \(S=\sum_{i=1}^m a_i\mu_i\) almost surely.

Proof. For any real \(t\), the characteristic function of \(a_iX_i\) is \(\varphi_{a_iX_i}(t)=\varphi_{X_i}(a_it)\). Independence implies that the characteristic function of \(S\) is the product of these functions. Substituting the normal characteristic function derived above gives

\begin{aligned} \varphi_S(t) &=\prod_{i=1}^m \varphi_{X_i}(a_it)\\ &=\prod_{i=1}^m \exp\!\left(ia_it\mu_i-\frac{a_i^2t^2\sigma_i^2}{2}\right)\\ &=\exp\!\left( it\sum_{i=1}^m a_i\mu_i -\frac{t^2}{2}\sum_{i=1}^m a_i^2\sigma_i^2 \right). \end{aligned}

This is the characteristic function of the stated normal distribution, including its degenerate case. The uniqueness theorem for characteristic functions therefore identifies the distribution of \(S\).

Linearity of expectation gives \(E[S]=\sum_i a_i\mu_i\). For the variance, expand the centered sum. Independence and finite second moments imply that for \(i\ne j\), \(E[(X_i-\mu_i)(X_j-\mu_j)]=E[X_i-\mu_i]E[X_j-\mu_j]=0\). Hence

\begin{aligned} \operatorname{Var}(S) &=E\!\left[\left(\sum_{i=1}^m a_i(X_i-\mu_i)\right)^2\right]\\ &=\sum_{i=1}^m a_i^2E[(X_i-\mu_i)^2] +2\sum_{i<j}a_ia_jE[(X_i-\mu_i)(X_j-\mu_j)]\\ &=\sum_{i=1}^m a_i^2\sigma_i^2. \end{aligned}

Each term \(a_i^2\sigma_i^2\) is nonnegative. Their sum is zero exactly when every term is zero. In that case, each \(a_i(X_i-\mu_i)\) has variance zero and mean zero, so it is zero almost surely. Summing these almost-sure equalities gives \(S=\sum_i a_i\mu_i\) almost surely. \(\square\)

Worked Applications

Worked Example: A Difference With Unequal Variances

Suppose \(X\sim N(2,4)\) and \(Y\sim N(-1,9)\) are independent. Find the distribution of \(T=3X-2Y\).

The weights are \(3\) and \(-2\). The mean is \(3E[X]-2E[Y]=3(2)-2(-1)=6+2=8\). The variance is the sum of the squared weights times the component variances: \(3^2(4)+(-2)^2(9)=36+36=72\). Therefore \(T\sim N(8,72)\). The negative weight affects the mean through its sign, but contributes \(4(9)=36\), not a negative amount, to the variance.

Worked Example: A Weighted Average

Independent measurements satisfy \(X_1\sim N(10,4)\) and \(X_2\sim N(13,9)\). Let \(A=\frac13X_1+\frac23X_2\). Determine its distribution.

The weights sum to one, so the mean is \(\frac13(10)+\frac23(13)=\frac{10}{3}+\frac{26}{3}=12\). The variance is \(\left(\frac13\right)^2(4)+\left(\frac23\right)^2(9)=\frac49+\frac{36}{9}=\frac{40}{9}\). Thus \(A\sim N(12,40/9)\). The fact that the weights form an average does not make the variance an ordinary weighted average of the variances; each variance is multiplied by the square of its weight.

Worked Example: A Degenerate Weighted Sum

Let \(X_1\sim N(5,0)\) and \(X_2\sim N(-2,0)\) be independent, and set \(T=2X_1+3X_2\). Identify its distribution, including its variance.

The mean is \(2(5)+3(-2)=10-6=4\). The variance is \(2^2(0)+3^2(0)=0\). By the definition of the degenerate normal distribution, \(X_1=5\) and \(X_2=-2\) almost surely, so \(T=2(5)+3(-2)=4\) almost surely. Its distribution is \(N(4,0)\), meaning a point mass at \(4\), not a normal density spread over the real line.

The characteristic function confirms the result: \(\varphi_T(t)=\exp(it\cdot4-\frac12t^2\cdot0)=e^{4it}\), which is the characteristic function of the constant \(4\).

Zero Variance and a Common Pitfall

The zero-variance condition is exact: because the variance is a sum of nonnegative terms, it vanishes if and only if \(a_i=0\) or \(\sigma_i^2=0\) for every \(i\). A nondegenerate independent normal component cannot be canceled by another independent component merely by assigning it a negative weight. For example, if \(X\) and \(Y\) are independent with positive variances, then \(\operatorname{Var}(X-Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)>0\).

Common mistake: “The weights in \(X-Y\) have opposite signs, so the variance contributions may cancel.” This confuses the signed mean calculation with the variance calculation. The correct expansion squares each weight; for independent variables, the cross-covariance term is zero, so the variance contributions add. Negative weights can cancel parts of the mean, but they do not create negative variance.

The independence condition is essential to the stated variance formula. For variables that are not independent, a covariance term can enter the variance of a sum. The characteristic-function product used in the distribution proof also relies on independence. Thus the theorem should not be applied just because the individual variables are normal.

Why the Result Matters

Weighted sums arise in averages, contrasts, measurement combinations, and linear transformations of normal observations. The theorem gives a complete distributional answer from the component means, variances, weights, and independence assumption; it does more than supply a mean and variance. Because the resulting distribution is known exactly, probabilities about the sum can be evaluated using its normal distribution, with the degenerate case handled separately as a constant.

A useful final check is to inspect the weights and units. The mean uses \(a_i\mu_i\), while the variance uses \(a_i^2\sigma_i^2\). If a weight is dimensionless, the corresponding variance retains squared measurement units. If one weight is zero, that variable has no effect on the sum, its mean, or its variance.

Check Your Understanding

Use the theorem and show the relevant mean and variance calculations.

If \(X\sim N(4,1)\) and \(Y\sim N(2,4)\) are independent, what is the distribution of \(X+2Y\)?
For independent \(X_1\sim N(0,9)\) and \(X_2\sim N(1,1)\), find the distribution of \(2X_1-X_2\).
Can a weighted sum of independent, nondegenerate normal variables have zero variance? Explain using the variance formula.
If all component variances are zero, what is the distribution of \(\sum_i a_iX_i\)? State its location.
Explain why a negative coefficient can reduce the mean but cannot subtract from the variance when the variables are independent.