Convolution of Mixture Distributions

Why Mixtures Change the Convolution

A mixture distribution describes a random variable whose law is selected from several component laws, with specified probabilities. When two independent mixture variables are added, the sum is not generally one of the original component distributions. Instead, each possible pair of component choices produces its own sum law. The weight of that law is the probability of that pair of choices.

This is a direct application of conditioning: first identify which component generated each variable, then use the convolution for that pair of component laws, and finally average over all pairs. The labels matter even when different pairs happen to produce the same sum distribution.

Definition: A random variable \(X\) has a finite mixture distribution with weights \(\alpha_1,\ldots,\alpha_K\) and component laws \(F_1,\ldots,F_K\) if \(\alpha_i\geq 0\), \(\sum_{i=1}^K\alpha_i=1\), and \(P(X\in A)=\sum_{i=1}^K\alpha_i P_i(A)\) for every measurable set \(A\), where \(P_i\) is the probability law corresponding to \(F_i\). Equivalently, one can generate \(X\) by selecting a component label \(C\) with \(P(C=i)=\alpha_i\), then drawing from component law \(F_i\).

Suppose \(X\) and \(Y\) are independent, with mixture weights \(\alpha_i\) and \(\beta_j\) and component laws \(F_i\) and \(G_j\), respectively. Their component labels can be taken independently, so the probability of the pair \((i,j)\) is \(\alpha_i\beta_j\). Write \(H_{ij}\) for the distribution of the sum of independent variables having laws \(F_i\) and \(G_j\). The sum distribution is the mixture of the \(H_{ij}\) laws with those product weights.

The General Conditioning Formula

Theorem: Let \(X\) and \(Y\) be independent finite-mixture random variables with weights \(\alpha_i\) and \(\beta_j\), and component laws \(F_i\) and \(G_j\). For \(S=X+Y\), at every real \(s\),

P(S\leq s)=\sum_{i=1}^{K}\sum_{j=1}^{J}\alpha_i\beta_j \int_{\mathbb R}G_j(s-x)\,dF_i(x).

Here \(G_j(t)=P(Y_j\leq t)\) is the distribution function of component \(j\) for \(Y\), and the integral is with respect to the probability law \(F_i\).

Proof. Let \(C\) and \(D\) be the component labels for \(X\) and \(Y\). Independence of \(X\) and \(Y\), together with the independent component selection, gives \(P(C=i,D=j)=\alpha_i\beta_j\). Conditional on \(C=i,D=j\), the variables have laws \(F_i,G_j\) and are independent. Thus, conditional on \(X=x\) and these labels, the probability that \(X+Y\leq s\) is \(G_j(s-x)\). Integrating over \(X\) under \(F_i\), then summing over all label pairs, gives

P(S\leq s)=\sum_{i=1}^{K}\sum_{j=1}^{J} P(C=i,D=j)\int_{\mathbb R}G_j(s-x)\,dF_i(x) =\sum_{i=1}^{K}\sum_{j=1}^{J}\alpha_i\beta_j \int_{\mathbb R}G_j(s-x)\,dF_i(x).

The label pairs partition the sample space, so this conditioning accounts for every possible way the sum can be generated. \(\square\)

When the component laws have densities \(f_i\) and \(g_j\), the component-wise convolution has density \(\int_{\mathbb R} f_i(x)g_j(s-x)\,dx\). Consequently, the sum has density

f_S(s)=\sum_{i=1}^{K}\sum_{j=1}^{J}\alpha_i\beta_j \int_{\mathbb R}f_i(x)g_j(s-x)\,dx.

This formula applies when every component law is absolutely continuous. If a component has atoms, the sum law may itself have atoms and a density alone will not describe it. The distribution-function formula above remains valid without a continuity assumption. For discrete components, the corresponding probability mass function is

P(S=s)=\sum_{i=1}^{K}\sum_{j=1}^{J}\alpha_i\beta_j \sum_x P_i(X_i=x)P_j(Y_j=s-x),

where the inner sum is over the possible values \(x\) of the first component. These are the usual component-wise convolution formulas, combined with the mixture weights.

Worked Example: A Sum of Two Normal Mixtures

Suppose \(X\) has law \(0.3N(0,1)+0.7N(2,1)\), and \(Y\), independently, has law \(0.4N(1,4)+0.6N(-1,1)\). Find the distribution of \(S=X+Y\).

For independent normal variables, the means add and the variances add. Pair each component of \(X\) with each component of \(Y\); each pair receives the product of its two weights:

The first components have weight \(0.3(0.4)=0.12\), mean \(0+1=1\), and variance \(1+4=5\), giving \(N(1,5)\).
The first component of \(X\) and second of \(Y\) have weight \(0.3(0.6)=0.18\), mean \(0+(-1)=-1\), and variance \(1+1=2\), giving \(N(-1,2)\).
The second component of \(X\) and first of \(Y\) have weight \(0.7(0.4)=0.28\), mean \(2+1=3\), and variance \(1+4=5\), giving \(N(3,5)\).
The second components have weight \(0.7(0.6)=0.42\), mean \(2+(-1)=1\), and variance \(1+1=2\), giving \(N(1,2)\).

Therefore,

S\sim 0.12N(1,5)+0.18N(-1,2)+0.28N(3,5)+0.42N(1,2).

The weights sum to \(0.12+0.18+0.28+0.42=1\). The two components with mean \(1\) have different variances, so they are not the same distribution and should remain separate.

Combining Coincident Component Laws

Different label pairs can produce identical sum laws. Their weights may then be added to express the result more compactly. The following discrete example shows why it is useful to retain the pairwise weights before combining them.

Worked Example: A Sum of Poisson Mixtures

Let \(X\) have law \(0.25\,\operatorname{Poisson}(1)+0.75\,\operatorname{Poisson}(4)\). Independently, let \(Y\) have law \(0.6\,\operatorname{Poisson}(2)+0.4\,\operatorname{Poisson}(5)\). Find the law of \(S=X+Y\) and calculate \(P(S=0)\).

For each pair, independent Poisson variables add to a Poisson variable whose parameter is the sum of the parameters. The four pairs give:

Parameters \(1\) and \(2\): weight \(0.25(0.6)=0.15\), sum law \(\operatorname{Poisson}(3)\).
Parameters \(1\) and \(5\): weight \(0.25(0.4)=0.10\), sum law \(\operatorname{Poisson}(6)\).
Parameters \(4\) and \(2\): weight \(0.75(0.6)=0.45\), sum law \(\operatorname{Poisson}(6)\).
Parameters \(4\) and \(5\): weight \(0.75(0.4)=0.30\), sum law \(\operatorname{Poisson}(9)\).

The two \(\operatorname{Poisson}(6)\) components have combined weight \(0.10+0.45=0.55\). Hence

S\sim 0.15\,\operatorname{Poisson}(3)+0.55\,\operatorname{Poisson}(6)+0.30\,\operatorname{Poisson}(9).

These weights sum to \(0.15+0.55+0.30=1\). Since a \(\operatorname{Poisson}(\lambda)\) variable is zero with probability \(e^{-\lambda}\),

P(S=0)=0.15e^{-3}+0.55e^{-6}+0.30e^{-9} \approx 0.007468+0.001363+0.000037 =0.008868.

The probability is a weighted sum of the component probabilities, not the zero probability of a Poisson law with an averaged parameter.

Mixtures With Both Atoms and a Density

A component law need not be purely discrete or purely continuous. The distribution-function formula handles such cases without separating the derivation into special cases. The next example calculates a mixed sum explicitly; in particular, it checks the continuous mass by integrating each interval separately.

Worked Example: Adding a Bernoulli Variable to a Mixed Variable

Let \(X\) have an atom \(P(X=0)=1/8\) and density

f_X(t)= \begin{cases} (1+3t)/8, & 0<t<1,\\ (9-3t)/8, & 1<t<2,\\ 0, & \text{otherwise}. \end{cases}

Let \(Y\), independently, equal \(0\) or \(1\) with probability \(1/2\) each. First check the law of \(X\). The density mass on the first interval is

\int_0^1\frac{1+3t}{8}\,dt =\frac{1}{8}\left[t+\frac{3t^2}{2}\right]_0^1 =\frac{5}{16}.

On the second interval it is

\int_1^2\frac{9-3t}{8}\,dt =\frac{1}{8}\left[9t-\frac{3t^2}{2}\right]_1^2 =\frac{9}{16}.

Thus the continuous mass is \(5/16+9/16=7/8\), and the atom contributes the remaining \(1/8\); the total probability is one.

Write \(S=X+Y\). The atom of \(X\) produces \(P(S=0)=P(X=0,Y=0)=1/16\) and \(P(S=1)=P(X=0,Y=1)=1/16\). The continuous part has density \(g(t)=\tfrac12 f_X(t)+\tfrac12 f_X(t-1)\), because \(Y\) shifts \(X\) by zero or one with equal probability. On \(0<t<1\), only the unshifted density contributes; on \(1<t<2\), both terms contribute; on \(2<t<3\), only the shifted density contributes. Substitution gives

g(t)= \begin{cases} (1+3t)/16, & 0<t<1,\\ 7/16, & 1<t<2,\\ (12-3t)/16, & 2<t<3,\\ 0, & \text{otherwise}. \end{cases}

For the middle interval, for example,

g(t)=\frac12\left(\frac{9-3t}{8}+\frac{1+3(t-1)}{8}\right) =\frac12\left(\frac{7}{8}\right)=\frac{7}{16}.

The density integrates to \(5/32\) on \((0,1)\), \(7/16=14/32\) on \((1,2)\), and \(9/32\) on \((2,3)\). Its total mass is therefore \(28/32=7/8\). Together with the two atoms, whose total mass is \(1/16+1/16=1/8\), the sum law has total probability one.

A Proof of the Density Formula

The density formula is the continuous version of the general conditioning result. For completeness, its derivation also shows why the mixture weights can be placed outside the component convolution integrals.

Result: If every component law \(F_i\) and \(G_j\) has a density \(f_i\) and \(g_j\), then \(S=X+Y\) has density \(f_S(s)=\sum_{i,j}\alpha_i\beta_j\int_{\mathbb R}f_i(x)g_j(s-x)\,dx\) for almost every \(s\).

Proof. For any Borel set \(A\), independence and the mixture laws give

P(S\in A)=\sum_{i=1}^{K}\sum_{j=1}^{J}\alpha_i\beta_j \int_{\mathbb R}\int_{\mathbb R} \mathbf{1}_{A}(x+y)f_i(x)g_j(y)\,dy\,dx.

All integrands are nonnegative, so Tonelli's theorem permits the iterated integration and the change of variable \(s=x+y\) in the inner integral. Thus

P(S\in A)=\int_A\left\{ \sum_{i=1}^{K}\sum_{j=1}^{J}\alpha_i\beta_j \int_{\mathbb R}f_i(x)g_j(s-x)\,dx \right\}ds.

The expression in braces is nonnegative and integrates to one, since it represents \(P(S\in A)\) for all Borel sets \(A\). It is therefore a density of \(S\), proving the result. \(\square\)

Common Mistakes and a Practical Check

A common error is to average component parameters first and then treat the sum as if it came from the averaged component laws. For example, a mixture of Poisson distributions is generally not Poisson with the mixture-weighted average parameter. The labels select different component laws; averaging parameters discards that variation. The correct procedure is to form every pair of component laws, convolve each pair, and weight it by the product of the corresponding probabilities.

Another error is to use only the density formula when some component has atoms. In that situation the convolution may have both a continuous part and point masses, as in the mixed example. The distribution-function or probability-law formula remains valid, while a density describes only the absolutely continuous part.

Key takeaway: For independent finite mixtures, the sum law is a mixture over component-label pairs. Each pair contributes its component-wise convolution, weighted by the product of the two component probabilities; identical resulting laws may be combined by adding their weights.

Check Your Understanding

Use conditioning on the component-label pair in each question.

If \(X\) has two mixture components with weights \(0.2,0.8\), and \(Y\) has three with weights \(0.1,0.3,0.6\), how many pairwise sum components arise before any identical laws are combined?
Why is the weight for the pair \((i,j)\) equal to \(\alpha_i\beta_j\) under the independence assumption?
Two different pairs produce the same component sum law, with weights \(0.12\) and \(0.28\). What weight should that law have after combining?
When all component laws have densities, state the convolution formula for the density of the sum.
Why can a density formula alone fail to describe a sum when one component law has an atom?