Mixtures Are Not Sums of Random Variables

One Component Is Selected—or Two Values Are Added

A mixture and a sum can both involve multiple distributions, but they describe different experiments. In a mixture, a selector chooses one component, and the observed value follows that component's law. In a sum, multiple random variables are generated and their values are added. Confusing these operations changes the distribution, often its support and its point probabilities as well as its density.

The distinction is easiest to see by asking what happens in one trial. Does the experiment choose one of several possible mechanisms and record that mechanism's outcome? Or does it generate two or more outcomes and combine them? A mixture describes the first procedure; addition describes the second.

Definition: Let $J$ be a random selector taking values in $\{1,\ldots,m\}$, with $P(J=j)=q_j$, where $q_j\geq 0$ and $\sum_{j=1}^m q_j=1$. A random variable $X$ has a mixture distribution with component CDFs $F_1,\ldots,F_m$ if, conditional on $J=j$, $X$ has CDF $F_j$. The selector chooses one component law for the observation. By contrast, a sum such as $S=Y_1+Y_2$ combines the values of two random variables; its law depends on their joint distribution.

The selector representation is a way to describe the experiment and its distribution. It does not require that every component outcome be generated and then discarded. For a finite mixture, the distribution function is an average of the component distribution functions, with the selection probabilities as weights. When the components have probability mass functions or densities, the same averaging applies to those functions (for densities, equality is understood almost everywhere).

Why the Mixture Formula Is an Average

Theorem: If $X$, conditional on $J=j$, has CDF $F_j$, then $$ F_X(x)=P(X\leq x)=\sum_{j=1}^m q_j F_j(x). $$ If each component is discrete with mass function $f_j$, then $P(X=x)=\sum_{j=1}^m q_j f_j(x)$. If each component has density $f_j$, then $X$ has density $f_X(x)=\sum_{j=1}^m q_j f_j(x)$ almost everywhere.

Proof. The events $\{J=j\}$ are disjoint and together exhaust all possible selector values. Partition the event $\{X\leq x\}$ according to which component was selected. For each $j$, the definition of conditional distribution gives $P(X\leq x\mid J=j)=F_j(x)$ when $q_j>0$. Therefore,

\begin{aligned} P(X\leq x) &=\sum_{j:q_j>0}P(X\leq x,\ J=j)\\ &=\sum_{j:q_j>0}P(X\leq x\mid J=j)P(J=j)\\ &=\sum_{j=1}^m q_jF_j(x). \end{aligned}

Terms with $q_j=0$ contribute zero, regardless of the conditional law assigned to that unused component. For a discrete variable, apply the same partition to the event $\{X=x\}$ to obtain the mass-function formula. For component densities, the probability of every Borel set $A$ is $\sum_j q_j\int_A f_j(x)\,dx=\int_A\sum_jq_jf_j(x)\,dx$; the finite sum can be moved inside the integral. Thus the displayed sum of densities is a density of $X$. $\square$

A sum follows a different calculation. In general, the law of a sum is not an average of the input laws: it depends on how the input values can combine. For independent variables with densities, the relevant formula is convolution.

Theorem: If $U$ and $V$ are independent real-valued random variables with densities $f_U$ and $f_V$, then $S=U+V$ has density $$ f_S(s)=\int_{-\infty}^{\infty} f_U(u)f_V(s-u)\,du $$ for almost every $s$.

Proof. Independence gives the joint density $f_U(u)f_V(v)$. For any Borel set $A$, integrate this joint density over the region where $u+v\in A$. By Tonelli's theorem, the integrand is nonnegative, so the order and form of integration can be changed:

\begin{aligned} P(U+V\in A) &=\int_{\mathbb R}\int_{\mathbb R} \mathbf{1}_{A}(u+v)f_U(u)f_V(v)\,dv\,du\\ &=\int_{\mathbb R}\int_{\mathbb R} \mathbf{1}_{A}(u+v)f_U(u)f_V(v)\,du\,dv. \end{aligned}

For each fixed $u$, substitute $s=u+v$, so $v=s-u$ and $dv=ds$. The resulting integral is

P(U+V\in A) =\int_A\left[\int_{\mathbb R}f_U(u)f_V(s-u)\,du\right]ds.

The bracketed function therefore gives a density of $S$, proving the formula. $\square$

Worked Examples: Mixtures and Sums Side by Side

Worked Example: Selecting a Bernoulli Law Versus Adding Outcomes

A fair selector chooses one of two Bernoulli distributions. Under the first component, the value is 1 with probability $0.2$; under the second, it is 1 with probability $0.8$. Compare the resulting mixture with the sum of independent variables $A\sim\operatorname{Bernoulli}(0.2)$ and $B\sim\operatorname{Bernoulli}(0.8)$.

Mixture. The selector chooses one Bernoulli outcome, not both. The probability of recording 1 is the average of the two component probabilities:

P(X=1)=\tfrac12(0.2)+\tfrac12(0.8)=0.5.

Consequently, $P(X=0)=0.5$. The mixture has support $\{0,1\}$.

Sum. For $S=A+B$, both Bernoulli variables are generated. By independence,

\begin{aligned} P(S=0)&=P(A=0,B=0)=(0.8)(0.2)=0.16,\\ P(S=1)&=P(A=1,B=0)+P(A=0,B=1)\\ &=(0.2)(0.2)+(0.8)(0.8)=0.04+0.64=0.68,\\ P(S=2)&=P(A=1,B=1)=(0.2)(0.8)=0.16. \end{aligned}

The probabilities sum to $0.16+0.68+0.16=1$, as required. The sum can be 2 because both variables can equal 1; the mixture cannot be 2 because only one Bernoulli outcome is recorded. Even at 1, the probabilities differ: $0.5$ for the mixture and $0.68$ for the sum.

Worked Example: Uniform Mixture Versus a Sum of Uniform Variables

A selector chooses with equal probability between a Uniform$(0,1)$ component and a Uniform$(2,3)$ component. Compare this mixture with $S=U+V$, where $U\sim\operatorname{Uniform}(0,1)$ and $V\sim\operatorname{Uniform}(2,3)$ are independent. Calculate the probability of being at most $2.5$ in each experiment.

Mixture. The mixture has density $1/2$ on each selected interval and zero elsewhere. Thus

P(X\leq 2.5) =\tfrac12 P(\operatorname{Uniform}(0,1)\leq 2.5) +\tfrac12 P(\operatorname{Uniform}(2,3)\leq 2.5) =\tfrac12(1)+\tfrac12(0.5)=0.75.

The first component is always at most $2.5$. Within the second component, the interval from 2 to 2.5 is half its length, so its conditional probability is $0.5$.

Sum. Write $V=2+W$, where $W\sim\operatorname{Uniform}(0,1)$ is independent of $U$. Then $S\leq2.5$ exactly when $U+W\leq0.5$. The pair $(U,W)$ is uniform over the unit square, whose area is 1. The event is the right triangle below the line $u+w=0.5$, with legs of length $0.5$. Its area, and hence its probability, is

P(S\leq2.5)=\frac12(0.5)(0.5)=0.125.

The result is not $0.75$: the mixture records one value from one interval, while the sum adds a value from each interval and is supported between 2 and 4.

Worked Example: A Mixed Component and the Presence of an Atom

Suppose $X$ is zero with probability $0.3$, and with probability $0.7$ it is drawn from an exponential distribution with rate 1. Compare $X$ with $S=B+E$, where $B$ is 0 with probability $0.3$ and 1 with probability $0.7$, and $E\sim\operatorname{Exponential}(1)$ is independent of $B$.

Mixture. The exponential component has CDF $1-e^{-x}$ for $x\geq0$ and has no probability at a single point. The mixture therefore has an atom of size $0.3$ at zero. For $x\geq0$,

F_X(x)=0.3+0.7(1-e^{-x}), \qquad P(X=0)=0.3.

At $x=0.5$, $F_X(0.5)=0.3+0.7(1-e^{-0.5})\approx0.5754$.

Sum. If $B=0$, then $S=E$; if $B=1$, then $S=1+E$. For $0\leq x<1$, only the first case can contribute to $P(S\leq x)$, giving

F_S(x)=0.3(1-e^{-x}),\qquad 0\leq x<1.

In particular, $F_S(0)=P(S=0)=0$, because $E$ has no atom at zero. At $x=0.5$, $F_S(0.5)=0.3(1-e^{-0.5})\approx0.1180$, far below the mixture probability. The atom in $X$ comes from directly selecting the point-mass component. Adding the continuous $E$ removes that point mass in $S$.

How to Tell Which Calculation the Experiment Requires

The key question is not whether a description mentions several distributions or random variables. It is whether the recorded outcome comes from one selected component or from combining multiple values. In a mixture, component probabilities are averaged because exactly one component is selected. For an independent sum, values from both variables contribute, so the distribution accounts for all ways their values can add to the observed total.

Feature	Mixture	Sum
What happens in one trial?	A selector chooses one component law	Each input variable is generated; their values are added
Typical probability calculation	Weighted average of component probabilities	Combine input values that produce the target; for independent densities, use convolution
What determines the law?	Component laws and selector probabilities	The joint law of the variables being added
Can an atom remain?	Yes, if a selected component assigns positive probability to that point	It depends on the input laws and their dependence

A mixture does not require independent component variables to be generated: only the selected outcome is observed. A sum, on the other hand, cannot be determined from the two marginal distributions alone unless additional information, such as independence, is supplied. The convolution formula used above requires independence; with dependence, the joint distribution must be used instead.

Common Mistake: Averaging When the Model Adds

A frequent error is to see two distributions and average their probabilities without checking what the random experiment does. For instance, in the Bernoulli example, one might calculate $\tfrac12(0.2)+\tfrac12(0.8)=0.5$ and report this as $P(A+B=1)$. That average answers a different question: it is the probability of 1 when one of the two Bernoulli laws is selected with equal chance. For the sum, the event $A+B=1$ occurs in two ways, and the correct calculation is $P(A=1,B=0)+P(A=0,B=1)=0.68$.

The reverse mistake is also possible: treating a selected component as if outcomes from all components were generated and combined. A useful check is to identify what a single observed value represents. If it is one component's outcome, partition by the selector and average. If it is the total of several outcomes, enumerate or integrate over combinations that produce that total.

Key takeaway: A mixture selects one component law; a sum combines random-variable values. Averaging component probabilities gives mixture probabilities, while a sum requires the joint behavior of its inputs.

Check Your Understanding

For each question, first identify what the experiment records, then choose the corresponding probability calculation.

A fair selector chooses between a constant-zero law and a constant-four law. What values can the mixture take? How does that support differ from the sum of independent variables with those two constant laws?
A selector chooses equally between Bernoulli$(0.1)$ and Bernoulli$(0.7)$. What is the mixture probability of 1?
Why does the convolution formula for the density of $U+V$ require independence as stated?
In the exponential example, why does the mixture have positive probability at zero but $B+E$ does not?
A machine randomly selects one of two production lines and records one part's weight. Is the observed weight a mixture or a sum? What would need to happen for a sum model to be appropriate?