Mixture Transforms Versus Sum Transforms

Two Different Ways to Combine Distributions

A distribution can be assembled by choosing one component or by adding independent random variables. These constructions can involve the same component distributions, but their transforms obey different rules. A mixture transform is a weighted average of the component transforms; the transform of an independent sum is a product.

The distinction matters because averaging and multiplying generally give different functions, even when the component distributions are familiar. We will use characteristic functions, which exist for every probability distribution, and also use Laplace transforms for nonnegative random variables.

Definition: Let $F_1,\ldots,F_m$ be probability distributions and let $w_1,\ldots,w_m$ satisfy $w_j\geq 0$ and $\sum_{j=1}^m w_j=1$. A random variable $X$ has the finite mixture distribution $\sum_{j=1}^m w_j F_j$ if, for every measurable set $A$, $$ P(X\in A)=\sum_{j=1}^m w_j F_j(A). $$ Equivalently, first choose a component label $J$ with $P(J=j)=w_j$, then draw $X$ from $F_j$ conditional on $J=j$.

For a random variable $X$, its characteristic function is $\phi_X(t)=E[e^{itX}]$, for real $t$. If $X\geq 0$, its Laplace transform is $L_X(s)=E[e^{-sX}]$, for $s\geq 0$. The characteristic function always exists because $|e^{itX}|=1$. The Laplace transform of a nonnegative random variable also exists for $s\geq 0$, since $0\leq e^{-sX}\leq 1$.

The Mixture Rule and the Independent-Sum Rule

Theorem: If $X$ has the mixture distribution $\sum_{j=1}^m w_jF_j$, with component characteristic functions $\phi_j(t)$, then $$ \phi_X(t)=\sum_{j=1}^m w_j\phi_j(t). $$ If the components are distributions on the nonnegative real line, their Laplace transforms satisfy $L_X(s)=\sum_{j=1}^m w_jL_j(s)$ for $s\geq 0$. By contrast, if $X_1,\ldots,X_n$ are independent and $S=\sum_{k=1}^n X_k$, then $$ \phi_S(t)=\prod_{k=1}^n\phi_{X_k}(t). $$ For nonnegative independent variables, $L_S(s)=\prod_{k=1}^n L_{X_k}(s)$ for $s\geq 0$.

The mixture identities follow by conditioning on the selected component. In the characteristic-function case, $e^{itX}$ is bounded, so the conditional expectations exist.

\phi_X(t)=E[e^{itX}] =\sum_{j=1}^m P(J=j)E[e^{itX}\mid J=j] =\sum_{j=1}^m w_j\phi_j(t).

For nonnegative $X$, the same conditioning argument applies to the bounded quantity $e^{-sX}$, giving $L_X(s)=\sum_jw_jL_j(s)$.

For independent variables, the product rule follows from independence: the joint distribution factors into the marginal distributions, so the expectation of the product of the functions $e^{itX_k}$ factors. These functions are bounded, and thus integrable.

\phi_S(t) =E\left[\prod_{k=1}^n e^{itX_k}\right] =\prod_{k=1}^n E[e^{itX_k}] =\prod_{k=1}^n\phi_{X_k}(t).

For nonnegative variables, $e^{-sX_k}$ is bounded by one, so the same independence argument gives the product rule for Laplace transforms. For moment-generating functions, the corresponding product identity holds when the expectations involved are finite; it is not necessary to use moment-generating functions to obtain the characteristic-function identities above.

Worked Example: A Discrete Mixture Versus a Bernoulli Sum

Let $X$ equal 0 with probability $1/4$ and 2 with probability $3/4$. This is a mixture of a point mass at 0 and a point mass at 2. Find its characteristic function, mean, and variance. Then compare it with the sum $S=B_1+B_2$ of two independent Bernoulli random variables with success probability $3/4$.

The mixture transform is the weighted average of the two point-mass transforms. A point mass at $a$ has characteristic function $e^{ita}$, so $$ \phi_X(t)=\frac14+\frac34e^{2it}. $$ The first two moments are $$ E[X]=0\left(\frac14\right)+2\left(\frac34\right)=\frac32, \qquad E[X^2]=0^2\left(\frac14\right)+2^2\left(\frac34\right)=3. $$ Therefore $\operatorname{Var}(X)=E[X^2]-(E[X])^2=3-\frac94=\frac34$.

For a Bernoulli variable $B_k$ with success probability $3/4$, $\phi_{B_k}(t)=\frac14+\frac34e^{it}$. Independence gives a product: $$ \phi_S(t)=\left(\frac14+\frac34e^{it}\right)^2. $$ In fact, $S$ has probabilities $P(S=0)=1/16$, $P(S=1)=6/16$, and $P(S=2)=9/16$. Its mean is $3/2$, while its variance is $2(3/4)(1/4)=3/8$. The mean matches that of $X$, but the variance and transform do not. For example, at $t=\pi$, $\phi_X(\pi)=-1/2$, whereas $\phi_S(\pi)=1/4$. The mixture selects one value, 0 or 2; the sum adds two Bernoulli outcomes.

Worked Example: A Symmetric Normal Mixture and a Normal Sum

Let $X$ be an equal mixture of $N(-1,1)$ and $N(1,1)$. Compare it with $S=Z_1+Z_2$, where $Z_1$ and $Z_2$ are independent $N(0,1)$ variables. Compute both characteristic functions, means, and variances.

The characteristic function of $N(\mu,\sigma^2)$ is $\exp(i\mu t-\sigma^2t^2/2)$. Thus the mixture transform is $$ \phi_X(t) =\frac12e^{-it-t^2/2}+\frac12e^{it-t^2/2} =e^{-t^2/2}\cos(t). $$ The mixture is symmetric about zero, so $E[X]=0$. Its component means are $-1$ and 1, and both component variances are 1. Directly, $$ E[X^2]=\frac12(1+(-1)^2)+\frac12(1+1^2)=2, \qquad \operatorname{Var}(X)=2. $$

For the independent sum, multiply the two standard normal characteristic functions: $$ \phi_S(t)=e^{-t^2/2}e^{-t^2/2}=e^{-t^2}. $$ Also, $E[S]=0+0=0$ and, by independence, $\operatorname{Var}(S)=1+1=2$. The means and variances agree, but the transforms do not: at $t=\pi/2$, $\phi_X(t)=0$, whereas $\phi_S(t)=e^{-\pi^2/4}>0$. Matching a few moments does not make the averaging and multiplication rules interchangeable.

Mixture Means and Variances

The same distinction appears in moments. Suppose component $j$ has finite mean $\mu_j$ and finite variance $\sigma_j^2$, and $X$ has mixture weights $w_j$. Let $\mu=\sum_jw_j\mu_j$. Conditioning on the component gives

E[X]=\sum_{j=1}^m w_j\mu_j, \qquad \operatorname{Var}(X) =\sum_{j=1}^m w_j\sigma_j^2 +\sum_{j=1}^m w_j(\mu_j-\mu)^2.

To verify the variance formula, first use $E[X^2\mid J=j]=\sigma_j^2+\mu_j^2$. Then $$ \operatorname{Var}(X) =\sum_jw_j(\sigma_j^2+\mu_j^2)-\mu^2 =\sum_jw_j\sigma_j^2+\sum_jw_j(\mu_j-\mu)^2, $$ where the last equality follows by expanding the squared terms and using $\sum_jw_j\mu_j=\mu$ and $\sum_jw_j=1$. The second term is the variation among component means. For an independent sum, instead, means add and variances add, provided the variables have finite variances. That variance rule follows by expanding the squared centered sum and using that independence makes each cross-covariance zero.

Worked Example: A Finite Mixture of Exponential Distributions

Let $X$ be an equal mixture of exponential distributions with rates 1 and 3. For an exponential variable with rate $\lambda$, $L(s)=\lambda/(\lambda+s)$, $E[X]=1/\lambda$, and $\operatorname{Var}(X)=1/\lambda^2$. Find the mixture Laplace transform, mean, and variance, and compare the transform with the sum of independent exponentials having those two rates.

The mixture transform is the weighted average: $$ L_X(s) =\frac12\frac{1}{1+s}+\frac12\frac{3}{3+s} =\frac{3+2s}{(1+s)(3+s)},\qquad s\geq 0. $$ The component means are 1 and $1/3$, so $E[X]=\frac12(1)+\frac12(1/3)=2/3$. The second moments of the components are $2$ and $2/9$, since $E[X^2]=\operatorname{Var}(X)+(E[X])^2=2/\lambda^2$ for an exponential variable. Hence $$ E[X^2]=\frac12(2)+\frac12\left(\frac29\right)=\frac{10}{9}, \qquad \operatorname{Var}(X)=\frac{10}{9}-\left(\frac23\right)^2=\frac23. $$

Now let $S=U+V$, where $U$ and $V$ are independent exponentials with rates 1 and 3. Their Laplace transforms multiply: $$ L_S(s)=\frac{1}{1+s}\frac{3}{3+s} =\frac{3}{(1+s)(3+s)}. $$ At $s=1$, $L_X(1)=5/8$, while $L_S(1)=3/8$. The mixture mean is $2/3$, whereas $E[S]=1+1/3=4/3$. In one construction a single rate is selected for the draw; in the other, two independent waiting times are added.

Common Mistake and Practical Use

Common mistake: “There are two component distributions, so the transform of the result must be the product of their transforms.” This applies the independent-sum rule to the wrong construction. A mixture does not add one draw from each component: it selects one component according to the weights. Its transform is therefore the weighted average. A product is appropriate when the random variables being added are independent.

When working with a transform, first describe how the random variable is constructed. If a component label is selected and one value is drawn from that component, average the component transforms. If independent random variables are added, multiply their transforms. This distinction is useful in distribution calculations and in checking proposed formulas. The identities establish how the transforms are formed; they do not, by themselves, settle every question about whether a transform uniquely identifies a distribution.

Check Your Understanding

For each question, decide whether averaging or multiplication is the relevant transform rule, and justify your choice.

A random variable is selected from $F_1$ with probability $0.2$ and from $F_2$ with probability $0.8$. Write its characteristic function using the component characteristic functions.
If $A$ and $B$ are independent, why does the characteristic function of $A+B$ use a product rather than a weighted average?
A mixture has two component means, 0 and 4, with equal weights. What is its mean? What contribution to its variance comes from variation between component means?
Two independent nonnegative variables have Laplace transforms $L_1(s)$ and $L_2(s)$. Write the Laplace transform of their sum and state the condition that justifies the product rule.
Can two distributions have the same mean and variance but different characteristic functions? Identify an example from this tutorial and a value of $t$ that displays the difference.

Mixture Transforms Versus Sum Transforms

What You'll Learn