Normal Sums Through MGFs

Why Normal Sums Are Useful

A total often combines several sources of variation: measurement errors from different instruments, independent process deviations, or separate components of a hypothetical portfolio model. When those components are independent and normally distributed, their total is also normal. The moment generating function (MGF) makes this result direct and shows exactly how the component means and variances combine.

The component normal distributions need not have the same mean or variance. For an ordinary sum, add the means and add the variances. The variance rule is especially easy to misapply: standard deviations do not add, and the independence assumption is essential to the MGF product rule.

Model and MGF of a Normal Variable

Definition: A random variable \(X\) has a normal distribution with mean \(\mu\) and variance \(\sigma^2\), written \(X\sim N(\mu,\sigma^2)\), when its density is \[ f_X(x)=\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right),\qquad -\infty<x<\infty, \] where \(\sigma^2>0\). Its MGF is \(M_X(t)=E[e^{tX}]\), for values of \(t\) where this expectation is finite.

For a normal variable, the MGF exists for every real \(t\) and is

M_X(t)=\exp\left(\mu t+\frac{\sigma^2t^2}{2}\right),\qquad -\infty<t<\infty.

The location parameter \(\mu\) appears in the term linear in \(t\), while the variance \(\sigma^2\) appears in the quadratic term. This distinction will make the parameters of a sum visible in its MGF.

Deriving the Distribution of an Independent Normal Sum

Let \(X_1,\ldots,X_n\) be mutually independent random variables, with \(X_i\sim N(\mu_i,\sigma_i^2)\) and \(\sigma_i^2>0\). Define their sum by \(S=X_1+\cdots+X_n\). Mutual independence means the joint distribution factors into the product of the marginal distributions; it allows the MGF of the sum to factor into a product.

Theorem: If \(X_1,\ldots,X_n\) are mutually independent and \(X_i\sim N(\mu_i,\sigma_i^2)\), then \[ S=\sum_{i=1}^{n}X_i\sim N\left(\sum_{i=1}^{n}\mu_i,\ \sum_{i=1}^{n}\sigma_i^2\right). \] The variance parameters, not the standard deviations, are added.

Here is the MGF justification. By the product rule for independent MGFs, \(M_S(t)=\prod_{i=1}^{n}M_{X_i}(t)\). Substitute the normal MGF for each component and collect the terms in the exponent:

\begin{aligned} M_S(t) &=\prod_{i=1}^{n}\exp\left(\mu_i t+\frac{\sigma_i^2t^2}{2}\right)\\ &=\exp\left(\sum_{i=1}^{n}\left(\mu_i t+\frac{\sigma_i^2t^2}{2}\right)\right)\\ &=\exp\left(\left(\sum_{i=1}^{n}\mu_i\right)t+ \frac{\left(\sum_{i=1}^{n}\sigma_i^2\right)t^2}{2}\right). \end{aligned}

This is the MGF of a normal variable whose mean is \(\sum_i\mu_i\) and whose variance is \(\sum_i\sigma_i^2\). Since the MGF exists in an interval containing zero and identifies the distribution uniquely, \(S\) has that normal distribution. This argument covers any finite number of independent normal components; identical means or variances are not required.

The assumptions matter. Each component must be normal, and the variables must be mutually independent for this product-rule derivation. If the components are dependent, this argument does not establish the result or justify adding their variances. Likewise, knowing only that each component is normal is not by itself enough to apply the independent-sum theorem.

Worked Example: Combining Two Independent Normal Deviations

In a hypothetical calibration model, two independent instrument deviations, measured in units, are \(X\sim N(2,9)\) and \(Y\sim N(-1,4)\). Let \(S=X+Y\). Find the distribution of \(S\), and then find \(P(S>5)\).

Method and model check. Both variables are normal, and the problem states that they are independent. The sum theorem therefore applies. The component means are \(2\) and \(-1\); the component variances are \(9\) and \(4\).

Find the parameters. The mean of the sum is \(2+(-1)=1\). Its variance is \(9+4=13\), so its standard deviation is \(\sqrt{13}\). Thus \(S\sim N(1,13)\).

The MGF confirms the parameter calculation:

\begin{aligned} M_S(t) &=e^{2t+9t^2/2}e^{-t+4t^2/2}\\ &=e^{t+13t^2/2}. \end{aligned}

This is the MGF of \(N(1,13)\). To calculate the requested probability, standardize using the mean \(1\) and standard deviation \(\sqrt{13}\):

\begin{aligned} P(S>5) &=P\left(\frac{S-1}{\sqrt{13}}>\frac{5-1}{\sqrt{13}}\right)\\ &=P\left(Z>\frac{4}{\sqrt{13}}\right)\\ &=P(Z>1.1094)\approx 0.1336, \end{aligned}

where \(Z\sim N(0,1)\), and the probability is rounded to four decimals. As a check, the resulting distribution has mean \(1\), as expected from adding the component means, and variance \(13\), as expected from adding the independent component variances. The common mistake avoided here is adding standard deviations \(3+2\) instead of variances \(9+4\).

Independent Normal Variables With Different Parameters

The theorem does not require the components to be identically distributed. Each component contributes its own mean and variance. This is useful when a total includes sources with different expected levels or different amounts of variability. Keep the parameter pairs in the form \(N(\mu_i,\sigma_i^2)\): the second entry is a variance, not a standard deviation.

Check the model.
Confirm that every summand is normal and that the summands are mutually independent.

Identify each parameter.
For every summand, record its mean and variance separately. Do not treat the stated variance as a standard deviation.

Add the parameters.
Add the means to obtain the total mean, and add the variances to obtain the total variance.

State the distribution.
Write the sum as normal with the calculated mean and variance. If a probability is requested, standardize using the square root of that variance.

Worked Example: A Total From Three Independent Sources

A hypothetical processing model represents three independent adjustments, all in dollars, by \(X_1\sim N(120,36)\), \(X_2\sim N(90,16)\), and \(X_3\sim N(40,48)\). Let \(T=X_1+X_2+X_3\). Find the distribution of \(T\) and the probability that the total exceeds \(260\).

Method and assumption check. The three adjustments are each normal and are specified to be independent, so their MGFs multiply. The means are \(120,90,40\), and the variances are \(36,16,48\).

The total mean is \(120+90+40=250\). The total variance is \(36+16+48=100\), giving standard deviation \(\sqrt{100}=10\). Therefore \(T\sim N(250,100)\).

For the upper-tail probability, standardize the threshold \(260\):

\begin{aligned} P(T>260) &=P\left(\frac{T-250}{10}>\frac{260-250}{10}\right)\\ &=P(Z>1)\\ &=1-\Phi(1)\\ &\approx 1-0.8413=0.1587. \end{aligned}

The probability is \(0.1587\) to four decimals. The parameters check: the expected total is \(250\), and the independent variances produce a standard deviation of \(10\), not \(36+16+48=100\). The value \(100\) is the variance; taking its square root is necessary when standardizing.

Coefficients in a Normal Sum

Sums may include multiples of independent normal variables. If \(X_i\sim N(\mu_i,\sigma_i^2)\), then a scaled and shifted term \(a_iX_i+b_i\) is normal with mean \(a_i\mu_i+b_i\) and variance \(a_i^2\sigma_i^2\). The squared coefficient in the variance follows because scaling by \(a_i\) multiplies deviations from the mean by \(a_i\), and variance is the expected squared deviation. For independent \(X_i\), the terms \(a_iX_i+b_i\) remain independent, so their means and variances can be added.

Thus, for \(L=\sum_{i=1}^{n}(a_iX_i+b_i)\),

L\sim N\left(\sum_{i=1}^{n}(a_i\mu_i+b_i),\ \sum_{i=1}^{n}a_i^2\sigma_i^2\right).

Worked Example: Coefficients and a Negative Contribution

A hypothetical score is defined as \(L=2X-Y+3W\), where \(X\), \(Y\), and \(W\) are independent and \(X\sim N(5,4)\), \(Y\sim N(1,9)\), and \(W\sim N(2,1)\). Find the distribution of \(L\).

Method and assumption check. The variables are independent and normal, so their scaled terms can be combined by adding their means and variances. The coefficients are \(2\), \(-1\), and \(3\); the negative sign changes the mean contribution from \(Y\), but the coefficient is squared for its variance contribution.

The mean is \(2(5)-1(1)+3(2)=10-1+6=15\). The variance is \(2^2(4)+(-1)^2(9)+3^2(1)=16+9+9=34\). Therefore \(L\sim N(15,34)\).

Equivalently, the MGF is

\begin{aligned} M_L(t) &=M_X(2t)M_Y(-t)M_W(3t)\\ &=e^{10t+8t^2}e^{-t+9t^2/2}e^{6t+9t^2/2}\\ &=e^{15t+17t^2}, \end{aligned}

which has the normal MGF form with mean \(15\) and variance \(34\). The result checks against the direct calculation. A common error would give \(Y\) a mean contribution of \(+1\), ignoring the minus sign, or a variance contribution of \(-9\); the correct contributions are \(-1\) to the mean and \(9\) to the variance.

Common Exam Traps and Checks

Adding standard deviations. For independent \(X_i\), add the variances \(\sigma_i^2\), then take the square root if a standard deviation is needed. For example, variances \(9\) and \(4\) give total variance \(13\) and total standard deviation \(\sqrt{13}\), not \(3+2=5\).
Forgetting independence. The product rule used in the derivation requires independent variables. If independence is not given, do not claim that this proof applies.
Misreading \(N(\mu,\sigma^2)\). Under this convention the second parameter is the variance. If a problem instead gives a standard deviation, square it before adding variances.
Dropping a coefficient’s square. In a linear combination, \(aX\) contributes \(a\mu\) to the mean but \(a^2\sigma^2\) to the variance.
Using a normal-sum result for nonnormal components. The theorem above is specifically for independent normal summands. A total of nonnormal variables is not covered by this result just because it has a mean and variance.

Key takeaway: For a finite sum of independent normal variables, the MGF product has the normal form again. Add component means to get the total mean, and add component variances to get the total variance.

Check Your Understanding

For each problem, state the distribution of the requested total before giving any probability.

Independent \(A\sim N(4,9)\) and \(B\sim N(3,16)\) represent hypothetical measurement errors. Find the distribution of \(A+B\). (Answer: \(A+B\sim N(7,25)\).)
Independent \(C\sim N(50,25)\) and \(D\sim N(70,75)\) are hypothetical dollar adjustments. Find the distribution of \(C+D\), then calculate \(P(C+D>140)\). (Answer: \(C+D\sim N(120,100)\), and \(P(C+D>140)=0.0228\).)
Independent \(U\sim N(2,4)\), \(V\sim N(-3,9)\), and \(W\sim N(5,1)\). Find the distribution of \(U+V+W\). (Answer: \(U+V+W\sim N(4,14)\).)
Independent \(R\sim N(10,4)\) and \(Q\sim N(6,9)\) contribute to the hypothetical score \(H=R-2Q\). Find the distribution of \(H\). (Answer: \(H\sim N(-2,40)\).)
Independent \(G\sim N(30,16)\), \(J\sim N(20,9)\), and \(K\sim N(10,25)\) are hypothetical components of a total \(M=G+J+K\). Find \(P(M<45)\). (Answer: \(M\sim N(60,50)\), and \(P(M<45)\approx 0.0169\).)

What You'll Learn