Products of MGFs for Independent Sums

Why Products Appear for Independent Sums

When a statistic is a sum of independent contributions, its distribution may be harder to work with directly than the distributions of the individual contributions. An MGF converts the sum into a product: the exponential of a sum factors into a product of exponentials, and independence factors the expectation. This identity can make moment calculations systematic, even when the distribution of the sum has not been identified.

The qualification “where the MGFs are finite” matters. A moment-generating function need not be finite at every real argument, so the product identity should be stated on the domain where the individual expectations exist. Moment calculations near zero require a little more: each MGF must be finite on a neighborhood of zero so its derivatives there represent moments.

Definition: For a random variable $X$, its moment-generating function is $M_X(t)=E[e^{tX}]$ at every real $t$ for which this expectation is finite. Its finite domain is $D_X=\{t\in\mathbb{R}:M_X(t)<\infty\}$. For a finite collection $X_1,\ldots,X_n$, write $S=\sum_{i=1}^n X_i$.

The Product Identity and Its Domain

Independence is the condition that lets us factor the expectation. The result applies to any finite number of independent real-valued random variables; they need not have the same distribution. In fact, for independent summands the sum’s finite MGF domain is exactly the intersection of their finite domains.

Theorem: Let $X_1,\ldots,X_n$ be independent random variables, where $n$ is finite, and let $S=\sum_{i=1}^n X_i$. For every $t\in\mathbb{R}$, $$ M_S(t)=\prod_{i=1}^n M_{X_i}(t), $$ where the equality is interpreted for nonnegative expectations, possibly taking the value $+\infty$. In particular, $$ D_S=\bigcap_{i=1}^n D_{X_i}, $$ and on this domain the product is finite.

Proof. Independence means that the joint distribution of $(X_1,\ldots,X_n)$ is the product of the marginal distributions. Thus the expectation defining $M_S(t)$ is the product-measure integral of $\prod_{i=1}^n e^{tX_i}$. Each integrand factor is nonnegative. Tonelli’s theorem therefore permits the iterated integral to be evaluated as a product, even if one or more integrals are infinite:

M_S(t) =E\left[\prod_{i=1}^n e^{tX_i}\right] =\prod_{i=1}^n E[e^{tX_i}] =\prod_{i=1}^n M_{X_i}(t).

Each factor $E[e^{tX_i}]$ is strictly positive: $e^{tX_i}>0$ almost surely, so its expectation is positive (or infinite). Consequently the product is finite exactly when every factor is finite. This proves both the identity and the domain statement. $\square$

Because the number of summands is finite, if each $M_{X_i}$ is finite on some open interval containing zero, their domains have a common open interval containing zero. The sum MGF is then finite there as well. A standard MGF differentiation result says that if $M_X$ is finite on an open interval containing zero, then its derivatives at zero give the raw moments: $M_X^{(k)}(0)=E[X^k]$ for every nonnegative integer $k$. In particular, $M_X'(0)=E[X]$ and $M_X''(0)=E[X^2]$. These conditions will allow us to differentiate the product at zero.

First and Second Moments of an Independent Sum

When all the summand MGFs are finite on a common open interval around zero, differentiate their product. Evaluating at zero uses $M_{X_i}(0)=E[1]=1$, so the product derivatives simplify.

Theorem: Suppose $X_1,\ldots,X_n$ are independent and each MGF is finite on an open interval containing zero. Then $$ E[S]=\sum_{i=1}^n E[X_i], \qquad \operatorname{Var}(S)=\sum_{i=1}^n \operatorname{Var}(X_i). $$ The raw second moment is $$ E[S^2]=\sum_{i=1}^n E[X_i^2] +2\sum_{1\leq i<j\leq n} E[X_i]E[X_j]. $$

Proof. On the common interval, the theorem above gives $M_S(t)=\prod_i M_{X_i}(t)$. Differentiate once using the product rule, then set $t=0$:

M_S'(0) =\sum_{i=1}^n M_{X_i}'(0)\prod_{j\ne i}M_{X_j}(0) =\sum_{i=1}^n E[X_i].

Differentiate a second time. Terms in which the same factor is differentiated twice contribute $M_{X_i}''(t)$; terms in which two distinct factors are differentiated contribute twice, once for each order of differentiation. Therefore,

M_S''(0) =\sum_{i=1}^n E[X_i^2] +2\sum_{1\leq i<j\leq n}E[X_i]E[X_j].

Since $M_S'(0)=E[S]$ and $M_S''(0)=E[S^2]$, this gives the stated raw second moment. Finally, $\operatorname{Var}(S)=E[S^2]-(E[S])^2$. Expanding the square of $\sum_i E[X_i]$ cancels the cross terms in the raw second moment, leaving

\operatorname{Var}(S) =\sum_{i=1}^n\left(E[X_i^2]-(E[X_i])^2\right) =\sum_{i=1}^n\operatorname{Var}(X_i).

All the moments in these calculations are finite because the individual MGFs are finite on neighborhoods of zero. $\square$

Worked Example: Two Independent Bernoulli Summands

Let $X_1$ and $X_2$ be independent, with $X_1\sim\operatorname{Bernoulli}(0.3)$ and $X_2\sim\operatorname{Bernoulli}(0.6)$. Find the MGF of $S=X_1+X_2$, its mean, and its variance.

A Bernoulli variable with success probability $p$ is 1 with probability $p$ and 0 with probability $1-p$. Its MGF is

M_X(t)=(1-p)e^{0t}+pe^{1t}=1-p+pe^t.

Thus $M_{X_1}(t)=0.7+0.3e^t$ and $M_{X_2}(t)=0.4+0.6e^t$. Independence gives

M_S(t)=(0.7+0.3e^t)(0.4+0.6e^t) =0.28+0.54e^t+0.18e^{2t}.

For a Bernoulli variable, $E[X]=p$ and $\operatorname{Var}(X)=p(1-p)$. The moment formulas for independent sums therefore give

E[S]=0.3+0.6=0.9, \qquad \operatorname{Var}(S)=0.3(0.7)+0.6(0.4)=0.21+0.24=0.45.

As a check using the displayed MGF, $M_S'(t)=0.54e^t+0.36e^{2t}$, so $M_S'(0)=0.90$. Also $M_S''(t)=0.54e^t+0.72e^{2t}$, giving $M_S''(0)=1.26$. Hence $M_S''(0)-(M_S'(0))^2=1.26-0.81=0.45$, as required.

Worked Example: Summands That Can Be Negative

Let $X$ and $Y$ be independent. Suppose $X=-1$ with probability $2/3$ and $X=2$ with probability $1/3$; suppose $Y=0$ with probability $3/4$ and $Y=3$ with probability $1/4$. Find the MGF, mean, and variance of $S=X+Y$.

The MGFs are finite for every real $t$, because each variable has finite support. Directly from their probability masses,

M_X(t)=\frac{2}{3}e^{-t}+\frac{1}{3}e^{2t}, \qquad M_Y(t)=\frac{3}{4}+\frac{1}{4}e^{3t}.

By independence, $M_S(t)=M_X(t)M_Y(t)$. The individual means are

E[X]=(-1)\frac{2}{3}+2\frac{1}{3}=0, \qquad E[Y]=0\frac{3}{4}+3\frac{1}{4}=\frac{3}{4}.

Their second moments and variances are

E[X^2]=1\frac{2}{3}+4\frac{1}{3}=2, \qquad \operatorname{Var}(X)=2-0^2=2, $$ $$ E[Y^2]=0^2\frac{3}{4}+9\frac{1}{4}=\frac{9}{4}, \qquad \operatorname{Var}(Y)=\frac{9}{4}-\left(\frac{3}{4}\right)^2 =\frac{27}{16}.

It follows that

E[S]=0+\frac{3}{4}=\frac{3}{4}, \qquad \operatorname{Var}(S)=2+\frac{27}{16}=\frac{59}{16}.

The raw second moment provides a further check: $E[S^2]=\operatorname{Var}(S)+(E[S])^2=\frac{59}{16}+\frac{9}{16}=\frac{17}{4}$. Equivalently, the raw-moment formula gives $2+\frac{9}{4}+2(0)(\frac{3}{4})=\frac{17}{4}$. Negative values cause no problem: the MGF product identity applies to real-valued summands, not only nonnegative ones.

Worked Example: Using the Common Finite Domain

Let $X$ and $Y$ be independent exponential variables with rates 2 and 5, respectively. Their densities are $f_X(x)=2e^{-2x}$ and $f_Y(y)=5e^{-5y}$ for $x\geq0$. Find the MGF domain for $S=X+Y$, and calculate its mean and variance.

For $X$, integrate the defining expectation:

M_X(t)=\int_0^\infty e^{tx}2e^{-2x}\,dx =2\int_0^\infty e^{-(2-t)x}\,dx =\frac{2}{2-t},\qquad t<2.

The integral diverges for $t\geq2$, so $D_X=(-\infty,2)$. In the same way, $M_Y(t)=5/(5-t)$ for $t<5$, and $D_Y=(-\infty,5)$. Their intersection is $(-\infty,2)$; the product theorem gives

M_S(t)=\frac{2}{2-t}\frac{5}{5-t}, \qquad t<2.

Differentiate the two individual MGFs at zero, or use their derivatives:

M_X'(t)=\frac{2}{(2-t)^2}, \quad M_X'(0)=\frac12; \qquad M_Y'(t)=\frac{5}{(5-t)^2}, \quad M_Y'(0)=\frac15.

Thus $E[S]=\frac12+\frac15=\frac{7}{10}$. Differentiating again gives $M_X''(t)=4/(2-t)^3$, so $M_X''(0)=1/2$; and $M_Y''(t)=10/(5-t)^3$, so $M_Y''(0)=2/25$. Therefore $\operatorname{Var}(X)=1/2-(1/2)^2=1/4$ and $\operatorname{Var}(Y)=2/25-(1/5)^2=1/25$. Independence yields

\operatorname{Var}(S)=\frac14+\frac1{25}=\frac{29}{100}.

The domain is determined by the stricter restriction, $t<2$, not by adding the two upper endpoints or requiring only that the product formula be algebraically defined.

Common Mistake and Practical Use

A common mistake is to write $M_{X+Y}(t)=M_X(t)M_Y(t)$ without checking independence. The algebraic equality $e^{t(X+Y)}=e^{tX}e^{tY}$ always holds, but it does not by itself imply that the expectation of the product equals the product of the expectations. That factoring step requires independence. For dependent variables, the general second-moment identity includes their covariance:

\operatorname{Var}(X+Y) =\operatorname{Var}(X)+\operatorname{Var}(Y)+2\operatorname{Cov}(X,Y).

Another error is to differentiate an MGF at zero without establishing that the needed moments exist. A common open interval around zero on which the MGF is finite supplies the required condition. Once that condition holds, the product identity turns individual MGF calculations into formulas for aggregate moments. It is useful when summands have different distributions, when their MGFs are simpler than the sum’s distribution, or when an explicit sum distribution is not needed.

Key takeaway: For finitely many independent summands, the sum MGF is the product of the individual MGFs precisely on their common finite domain. If that domain contains a neighborhood of zero, differentiating the product gives the sum’s moments; in particular, means and variances add.

Check Your Understanding

Answer these questions using the conditions and identities developed above.

If $X_1,\ldots,X_n$ are independent, what is the finite domain of the MGF of their sum?
Why is independence needed to replace $E[e^{tX}e^{tY}]$ by $E[e^{tX}]E[e^{tY}]$?
Suppose two independent variables have means 2 and $-1$, and variances 3 and 4. Find the mean and variance of their sum.
If each summand MGF is finite on an open interval containing zero, what derivative gives the sum’s raw second moment?
For independent $X$ and $Y$, their MGFs are finite for $t<1$ and $t<4$, respectively. On what interval is the product MGF finite?

Products of MGFs for Independent Sums

What You'll Learn