MGFs Of Independent Sums And Normal Variables

Why MGFs Help with Sums

A sum of random variables can be difficult to study directly: even when each summand has a familiar distribution, its sum may not have an obvious density or probability mass function. A moment generating function (MGF) can simplify the calculation. Independence turns the MGF of a sum into a product, and for normal summands that product has the form of a normal MGF.

Work on a probability space $(\Omega,\mathcal{F},\mathbb{P})$. For a real-valued random variable $X$, its MGF is $M_X(t)=\mathbb{E}[e^{tX}]$ at each real $t$ for which this expectation is finite. Its domain need not be all of $\mathbb{R}$. In particular, multiplying MGFs at a value of $t$ requires that each factor be finite there.

Theorem (Product rule for independent sums): Let $X_1,\ldots,X_n$ be independent real-valued random variables. For any real $t$ such that $M_{X_r}(t)<\infty$ for every $r=1,\ldots,n$, the MGF of $S=X_1+\cdots+X_n$ is finite and $$ M_S(t)=\prod_{r=1}^n M_{X_r}(t). $$ Thus the identity holds throughout the intersection of the individual MGF domains.

Proof. Independence means that the joint distribution of $(X_1,\ldots,X_n)$ is the product of their individual distributions. Therefore, for the nonnegative function $(x_1,\ldots,x_n)\mapsto \prod_{r=1}^n e^{tx_r}$, integration with respect to the joint distribution factors into iterated integrals with respect to the individual distributions. The integrals are finite by the hypothesis on $t$, so

\begin{aligned} M_S(t) &=\mathbb{E}\!\left[e^{t(X_1+\cdots+X_n)}\right]\\ &=\mathbb{E}\!\left[\prod_{r=1}^n e^{tX_r}\right]\\ &=\prod_{r=1}^n\mathbb{E}[e^{tX_r}] =\prod_{r=1}^n M_{X_r}(t). \end{aligned}

This also proves finiteness of $M_S(t)$, since the product on the right is finite. The factorization of the integral follows from the product form of the joint distribution and the theorem for iterated integrals of nonnegative functions. $\square$

The domain condition is part of the rule, not a technical afterthought. If each MGF is finite on an open interval containing zero, then their product is also finite on the intersection of those intervals, which is itself an open interval containing zero. Such an interval is useful when comparing an MGF with a candidate distribution.

Normal MGFs and the Sum Theorem

For $\sigma^2>0$, a normal random variable $X\sim N(\mu,\sigma^2)$ has MGF $$ M_X(t)=\exp\!\left(\mu t+\frac{\sigma^2t^2}{2}\right),\qquad t\in\mathbb{R}. $$

One way to obtain this formula is to insert the normal density into the expectation and complete the square in the exponent. The resulting integral is the integral of a normal density shifted by $\sigma^2t$, hence equals one, leaving the displayed exponential factor. If $\sigma^2=0$, the convention $N(\mu,0)$ means the constant random variable $\mu$; its MGF is $e^{\mu t}$, which is also given by the same formula with $\sigma^2=0$.

Theorem (Sum of independent normal variables): Suppose $X_1,\ldots,X_n$ are independent and $X_r\sim N(\mu_r,\sigma_r^2)$, where $\sigma_r^2\ge0$. Then $$ \sum_{r=1}^n X_r\sim N\!\left(\sum_{r=1}^n\mu_r,\ \sum_{r=1}^n\sigma_r^2\right). $$ The variances may be zero; the conclusion then includes the possibility that the sum is constant.

Proof. First consider two independent normal variables $X\sim N(\mu_1,a)$ and $Y\sim N(\mu_2,b)$, where $a>0$ and $b>0$. Independence gives the density of $Z=X+Y$ by convolution. At a real value $z$, $$ f_Z(z)=\int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi a}}\exp\!\left(-\frac{(x-\mu_1)^2}{2a}\right) \frac{1}{\sqrt{2\pi b}}\exp\!\left(-\frac{(z-x-\mu_2)^2}{2b}\right)\,dx. $$

Set $c=(b\mu_1+a(z-\mu_2))/(a+b)$. Expanding both sides verifies the identity $$ \frac{(x-\mu_1)^2}{2a}+\frac{(z-x-\mu_2)^2}{2b} = \frac{(z-\mu_1-\mu_2)^2}{2(a+b)} +\frac{a+b}{2ab}(x-c)^2. $$

Use this identity in the convolution integral. The factor involving $z$ can be taken outside the integral, and the remaining integral is a Gaussian integral: $$ \begin{aligned} f_Z(z) &=\frac{\exp\!\left(-\frac{(z-\mu_1-\mu_2)^2}{2(a+b)}\right)} {2\pi\sqrt{ab}} \int_{-\infty}^{\infty} \exp\!\left(-\frac{a+b}{2ab}(x-c)^2\right)\,dx\\ &=\frac{1}{\sqrt{2\pi(a+b)}} \exp\!\left(-\frac{(z-\mu_1-\mu_2)^2}{2(a+b)}\right). \end{aligned} $$

This is the density of $N(\mu_1+\mu_2,a+b)$, proving the result for two positive variances. If either variance is zero, the corresponding variable is constant, so adding it shifts the mean and leaves the other variable’s variance unchanged. If both are zero, their sum is the constant $\mu_1+\mu_2$, which is $N(\mu_1+\mu_2,0)$ under the stated convention. This establishes the two-variable result in all cases. Applying that result repeatedly proves the claim for any finite $n$: at each step, the partial sum is independent of the next variable, because the original variables are independent, and its mean and variance are the respective partial sums. $\square$

The product rule gives a second route to the same conclusion. Multiplying the individual normal MGFs gives $$ \prod_{r=1}^n \exp\!\left(\mu_r t+\frac{\sigma_r^2t^2}{2}\right) = \exp\!\left(t\sum_{r=1}^n\mu_r+ \frac{t^2}{2}\sum_{r=1}^n\sigma_r^2\right), $$ the MGF of the normal distribution in the theorem. To conclude that equality of MGFs implies equality of distributions in a general identification problem, one uses the standard uniqueness theorem for MGFs: if two probability distributions have MGFs finite on an open interval containing zero and those MGFs agree throughout an open interval containing zero, then the distributions are equal. Its proof is beyond this tutorial. The density argument above proves the normal-sum result directly, so it does not depend on that uniqueness theorem.

Worked Example: Adding Two Normal Variables

Let $X\sim N(1,4)$ and $Y\sim N(-2,1)$ be independent. Find the distribution of $X+Y$ using MGFs, and check the parameters.

The individual MGFs, each finite for all real $t$, are $$ M_X(t)=\exp\!\left(t+\frac{4t^2}{2}\right)=e^{t+2t^2}, \qquad M_Y(t)=\exp\!\left(-2t+\frac{t^2}{2}\right). $$ By the product rule, $$ M_{X+Y}(t)=e^{t+2t^2}e^{-2t+t^2/2} =e^{-t+5t^2/2}. $$

This is the MGF of $N(-1,5)$, so $X+Y\sim N(-1,5)$. The normal-sum theorem independently gives mean $1+(-2)=-1$ and variance $4+1=5$. As a derivative check, $M_{X+Y}'(t)=(-1+5t)e^{-t+5t^2/2}$, so $M_{X+Y}'(0)=-1$. Differentiating once more gives $M_{X+Y}''(0)=6$, and hence the variance is $6-(-1)^2=5$, as claimed.

Worked Example: A Linear Combination of Standard Normals

Let $X$ and $Y$ be independent $N(0,1)$ variables. Find the distribution of $2X-3Y$.

For any real $a$, $M_{aX}(t)=\mathbb{E}[e^{taX}]=M_X(at)$. Thus $$ M_{2X}(t)=e^{(2t)^2/2}=e^{2t^2}, \qquad M_{-3Y}(t)=e^{(-3t)^2/2}=e^{9t^2/2}. $$ The scaled variables remain independent, so the product rule gives $$ M_{2X-3Y}(t)=e^{2t^2}e^{9t^2/2}=e^{13t^2/2}. $$

This is the MGF of $N(0,13)$. The normal-sum theorem also applies to the independent normal variables $2X\sim N(0,4)$ and $-3Y\sim N(0,9)$, giving mean $0+0=0$ and variance $4+9=13$. As a check, the product MGF has first derivative zero at $t=0$ and second derivative $13$ there, giving mean zero and variance $13$.

Worked Example: A Degenerate Normal Summand

Let $C\sim N(2,0)$, so $C=2$ with probability one, and let $Y\sim N(-1,9)$ be independent of $C$. Find the distribution of $C+Y$.

The MGFs are $$ M_C(t)=e^{2t}, \qquad M_Y(t)=\exp\!\left(-t+\frac{9t^2}{2}\right). $$ Independence gives $$ M_{C+Y}(t)=e^{2t}\exp\!\left(-t+\frac{9t^2}{2}\right) =\exp\!\left(t+\frac{9t^2}{2}\right). $$

Therefore $C+Y\sim N(1,9)$. Directly, $C+Y=2+Y$, which shifts the mean from $-1$ to $-1+2=1$ and leaves the variance at $9$. Substitution into the normal MGF formula gives $\exp(1\cdot t+9t^2/2)$, matching the product exactly.

When Identification Works—and a Common Mistake

Matching an MGF to a known formula identifies a distribution when both MGFs are finite on an open interval containing zero and agree throughout such an interval. The uniqueness theorem then guarantees equality in distribution. Matching only a few values of $t$, or matching only the mean and variance, does not in general identify a distribution. The special conclusion for a sum of independent normal variables is justified by the sum theorem; it is not a general rule that sums are normal.

Independence is essential to the product rule. For dependent variables, the expectation of a product need not equal the product of expectations. A common incorrect calculation is to multiply marginal MGFs without checking independence.

Worked Example: The Dependence Pitfall

Let $X\sim N(0,1)$ and set $Y=X$. Both variables have MGF $e^{t^2/2}$, but they are not independent. Find the MGF of $X+Y$ and compare it with the incorrect product calculation.

The actual sum is $X+Y=2X$, so $$ M_{X+Y}(t)=M_X(2t)=e^{(2t)^2/2}=e^{2t^2}. $$ This is the MGF of $N(0,4)$. If one incorrectly applied the independent-sum product rule, one would get $$ M_X(t)M_Y(t)=e^{t^2/2}e^{t^2/2}=e^{t^2}, $$ which is the MGF of $N(0,2)$, not $N(0,4)$. The check is decisive: the actual variance is $\operatorname{Var}(2X)=4\operatorname{Var}(X)=4$, whereas the incorrect result has variance $2$. The product rule cannot be used here because $Y=X$ is determined by $X$, rather than independent of it.

For independent normal summands, both the MGF calculation and the direct sum theorem are useful checks. The MGF calculation is compact and extends immediately to any finite number of summands. The theorem also clarifies why variances add: independence is needed, and a zero-variance normal variable contributes a constant shift but no variance. For dependent normal variables, adding marginal means and variances alone is not valid; dependence can change the variance of the sum.

Check Your Understanding

For each question, state any independence or MGF-domain condition you use.

If independent variables have MGFs $M_1(t)$ and $M_2(t)$, on what set of $t$-values does the product rule apply?
Suppose independent $X\sim N(3,2)$ and $Y\sim N(-1,5)$. What is the distribution of $X+Y$, and what is its MGF?
Let $X_1,\ldots,X_n$ be independent $N(0,1)$ variables. What is the MGF and distribution of $X_1+\cdots+X_n$?
Why does the MGF product calculation fail for $Y=X$, even when $X$ has a finite MGF for every real $t$?
What hypotheses are needed to infer equality in distribution from equality of two MGFs on an interval?

MGFs Of Independent Sums And Normal Variables

What You'll Learn