Transform Products for Independent Sums

Why Transform Products Identify Sums

A sum of independent random variables can be difficult to study from its distribution directly. A transform often makes the sum simpler: the transform of the sum factors into a product of transforms of the individual variables. If that product matches a familiar transform, uniqueness can identify the distribution of the sum.

The three transforms considered here have different domains and uses. A probability-generating function (PGF) applies to nonnegative integer-valued variables; a moment-generating function (MGF) requires exponential moments at the arguments being used; and a characteristic function (CF) exists for every real-valued random variable. The product principle is shared, but the conditions and identification tools are not identical.

Definitions and Product Rules

Definition: If \(X\) is nonnegative and integer-valued, its probability-generating function is \(G_X(s)=E[s^X]\), for \(0\leq s\leq 1\). If \(X\) is real-valued, its moment-generating function is \(M_X(t)=E[e^{tX}]\) wherever this expectation is finite, and its characteristic function is \(\varphi_X(t)=E[e^{itX}]\), for \(t\in\mathbb{R}\).

The PGF is finite on \([0,1]\), since \(0\leq s^X\leq 1\) there. The CF is also always finite because \(|e^{itX}|=1\). An MGF, by contrast, may be infinite at some values of \(t\); saying that it exists at \(t\) means \(E[e^{tX}]<\infty\).

Theorem (product rules for independent sums): Let \(X_1,\ldots,X_n\) be mutually independent random variables and let \(S=\sum_{j=1}^n X_j\).

If each \(X_j\) is nonnegative and integer-valued, then for \(0\leq s\leq 1\), \(G_S(s)=\prod_{j=1}^n G_{X_j}(s)\).
For any real \(t\) at which the MGFs \(M_{X_j}(t)\) are finite, \(M_S(t)=\prod_{j=1}^n M_{X_j}(t)\).
For every real \(t\), \(\varphi_S(t)=\prod_{j=1}^n\varphi_{X_j}(t)\).

For the MGF statement, finiteness of each factor is a convenient explicit condition ensuring a finite product. More generally, the expectation identity holds with extended nonnegative expectations, so if an individual MGF is infinite at \(t\), the MGF of the sum is infinite there as well. The CF statement needs no moment assumptions.

Why Independence Produces a Product

Here is the argument for two variables; the \(n\)-variable result follows by applying the same reasoning to the joint distribution of the independent variables. Independence means that the joint distribution is the product of the marginal distributions. Thus expectations of products of functions of separate variables factor, provided the expectations are defined. For nonnegative functions this follows from Tonelli's theorem; for integrable complex functions it follows from Fubini's theorem.

Write the sum inside the transform.
For an MGF, \(e^{t(X+Y)}=e^{tX}e^{tY}\). For a CF, \(e^{it(X+Y)}=e^{itX}e^{itY}\). For a PGF, \(s^{X+Y}=s^Xs^Y\) when \(s\geq 0\).

Use independence to factor the expectation.
The joint distribution is the product of the marginal distributions, so the expectation of the product factors into the two marginal expectations. For the MGF, Tonelli applies to the nonnegative factors; when the expectations are finite, their product is finite. For the CF, the factors are bounded and Fubini applies.

Recognize the individual transforms.
The resulting factors are the transforms of \(X\) and \(Y\), giving the corresponding product rule. Repeating the argument gives the formula for any finite number of mutually independent variables.

For example, when the MGFs are finite at \(t\), the derivation is

M_{X+Y}(t) =E[e^{t(X+Y)}] =E[e^{tX}e^{tY}] =E[e^{tX}]E[e^{tY}] =M_X(t)M_Y(t).

The factorization is not a general rule for dependent variables. Without independence, the expectation of a product need not equal the product of expectations.

Worked Examples

Worked Example: A PGF Gives the Sum’s Probabilities

Let \(X\) and \(Y\) be independent. Suppose \(P(X=0)=2/3\), \(P(X=1)=1/3\), while \(P(Y=0)=3/4\), \(P(Y=2)=1/4\). Find the distribution of \(S=X+Y\) using PGFs.

The individual PGFs are

G_X(s)=\frac{2}{3}+\frac{s}{3}, \qquad G_Y(s)=\frac{3}{4}+\frac{s^2}{4}.

By independence, \(G_S(s)=G_X(s)G_Y(s)\). Multiplying out gives

G_S(s) =\left(\frac{2}{3}+\frac{s}{3}\right) \left(\frac{3}{4}+\frac{s^2}{4}\right) =\frac{1}{2}+\frac{s}{4}+\frac{s^2}{6}+\frac{s^3}{12}.

For a nonnegative integer-valued variable, the coefficient of \(s^k\) in its PGF is \(P(S=k)\). Therefore \(P(S=0)=1/2\), \(P(S=1)=1/4\), \(P(S=2)=1/6\), and \(P(S=3)=1/12\). These probabilities are nonnegative and sum to \(1/2+1/4+1/6+1/12=6/12+3/12+2/12+1/12=1\), as required.

Worked Example: An MGF Identifies a Three-Point Sum

Let \(X\) and \(Y\) be independent, each taking the values \(-1\) and \(1\) with probability \(1/2\). Find the distribution of \(S=X+Y\) from its MGF.

For either variable,

M_X(t)=M_Y(t)=\frac{e^{-t}+e^t}{2}=\cosh(t),

which is finite for every real \(t\). Hence

M_S(t)=\cosh^2(t) =\left(\frac{e^{-t}+e^t}{2}\right)^2 =\frac{e^{-2t}+2+e^{2t}}{4}.

The last expression is the MGF of a variable taking \(-2,0,2\) with probabilities \(1/4,1/2,1/4\), respectively: its MGF is \((1/4)e^{-2t}+(1/2)+(1/4)e^{2t}\). MGFs that are finite on an open interval containing zero uniquely determine distributions. The computed MGF therefore identifies the law of \(S\) as this three-point distribution.

Worked Example: A Cauchy Sum Despite the Lack of an MGF

Let \(X\) and \(Y\) be independent Cauchy variables with location \(0\) and scales \(1\) and \(2\), respectively. Identify the distribution of \(S=X+Y\).

A Cauchy variable with location \(0\) and scale \(a>0\) has characteristic function \(\varphi(t)=e^{-a|t|}\). Thus

\varphi_S(t) =\varphi_X(t)\varphi_Y(t) =e^{-|t|}e^{-2|t|} =e^{-3|t|}.

This is the characteristic function of a Cauchy variable with location \(0\) and scale \(3\). Characteristic functions uniquely determine distributions, so \(S\) has that Cauchy distribution. Its MGF does not exist on any open interval containing zero, but that causes no difficulty: the CF product rule is valid for every real \(t\).

Using a Product to Identify a Distribution

A transform product is useful when the resulting expression can be recognized as the transform of a candidate distribution. The identification step depends on the transform:

Transform	Identification principle	Important condition
PGF	Equal PGFs give equal probabilities at each nonnegative integer.	The variables must be nonnegative and integer-valued.
MGF	Equal MGFs identify equal distributions.	The common MGF must be finite on an open interval containing zero.
Characteristic function	Equal characteristic functions identify equal distributions.	No additional moment or integrability condition is needed.

For PGFs, the identification fact follows from uniqueness of power-series coefficients: \(G_X(s)=\sum_{k=0}^{\infty}P(X=k)s^k\) for \(0\leq s<1\), and a power series has unique coefficients within its interval of convergence. The MGF condition above is essential to its standard uniqueness theorem. The CF uniqueness theorem applies without such an interval-of-finiteness requirement.

Common Mistakes and Practical Checks

Factoring without independence. The equation \(E[g(X)h(Y)]=E[g(X)]E[h(Y)]\) is justified here by independence. It is not true in general for dependent variables.
Assuming an MGF always exists. Unlike a CF, an MGF can be infinite even for a familiar distribution. Check finiteness at the argument being used.
Confusing pairwise and mutual independence. For a sum of several variables, the product of all their transforms is guaranteed by mutual independence. Pairwise independence alone does not generally give the joint factorization required.
Recognizing a transform but skipping uniqueness. A matching expression identifies a distribution only when the relevant uniqueness result applies. State the candidate transform and the applicable uniqueness condition.
Forgetting to check a PGF’s coefficients. In a finite polynomial, coefficients are probabilities; they must be nonnegative and sum to one.

The practical strategy is to verify independence, select a transform that exists, apply its product rule, and then compare the result with a known transform under the appropriate uniqueness condition. This works even when the sum's density or mass function would be inconvenient to calculate directly.

Check Your Understanding

Use the definitions and conditions in this tutorial to answer the following.

What independence assumption justifies factoring the transform of a sum of three random variables into three individual transforms?
For which values of \(s\) is a PGF necessarily finite when the variable is nonnegative and integer-valued? Explain why.
Two independent variables have MGFs finite at \(t=2\). What is the MGF of their sum at \(2\), and what fact justifies the factorization?
Why can a characteristic function identify the Cauchy sum in the worked example even though an MGF cannot be used?
If a PGF product expands as \(1/3+s/2+s^2/6\), what probabilities does it assign to the sum, and what check should be made?