Mixture Weights and Component Probabilities

Why Mixture Weights Matter

A mixture distribution describes a random variable generated by first selecting one of several component distributions and then drawing from the selected component. The weights describe how often each component is selected; they do not, by themselves, tell us the probability of an event involving the final observation. To find that probability, we combine each component's event probability with its selection weight.

This distinction is especially important when the components have different forms. One component may put all its probability at a point, while another spreads probability continuously over an interval. An event can receive contributions from both, and two different components can contribute probability to the same point. The calculation is systematic: condition on which component was selected, then add the weighted component probabilities.

Finite Mixtures and Component Weights

Definition: Let \(J\) take values in \(\{1,\ldots,m\}\), with \(P(J=j)=w_j\), where \(w_j\geq 0\) and \(\sum_{j=1}^{m}w_j=1\). Given \(J=j\), let the real-valued random variable \(X\) have component distribution \(F_j\). The distribution of \(X\) is the finite mixture with weights \(w_1,\ldots,w_m\) and component distributions \(F_1,\ldots,F_m\). For a Borel set \(A\), write \(P_j(X\in A)\) for the probability assigned to \(A\) by component \(j\).

The index \(J\) records the selected component. If a weight is zero, that component is never selected and contributes nothing to any probability. Conditional probabilities given a zero-probability component are not needed; all formulas below can be read as sums over the components with positive weights.

For a Borel event \(A\), the component probability \(P_j(X\in A)\) is calculated under component \(j\)'s law. The weight \(w_j\) is then used to account for the chance that this component was selected. The result is a weighted average of component probabilities, not generally equal to any one of them.

Theorem: For every Borel set \(A\), the probability under the mixture is

P(X\in A)=\sum_{j=1}^{m}w_jP_j(X\in A).

The distribution function therefore satisfies

F(x)=\sum_{j=1}^{m}w_jF_j(x),\qquad x\in\mathbb{R}.

Proof. The events \(\{J=j\}\), for \(j=1,\ldots,m\), are disjoint and their union is the whole sample space. Consequently, the events \(\{X\in A,J=j\}\) partition \(\{X\in A\}\), so finite additivity gives

P(X\in A)=\sum_{j=1}^{m}P(X\in A,J=j).

For each component with \(w_j>0\), the definition of conditional probability gives

P(X\in A,J=j)=P(J=j)P(X\in A\mid J=j) =w_jP_j(X\in A).

A component with \(w_j=0\) contributes zero. Adding the terms proves the event formula. Taking \(A=(-\infty,x]\) proves the formula for \(F(x)\), since \(P_j(X\leq x)=F_j(x)\). \(\square\)

Point Masses and Interval Probabilities

A point mass, or atom, at \(a\) is the probability \(P(X=a)\). Since the singleton \(\{a\}\) is an event, the mixture formula immediately gives the atom size.

Theorem: At any \(a\in\mathbb{R}\), the mixture's point mass is

P(X=a)=\sum_{j=1}^{m}w_jP_j(X=a).

Each component's contribution is its weight multiplied by its own probability at \(a\). If only one component has an atom at \(a\), only that component contributes. If several components have an atom there, their contributions add. A component with a density contributes zero to a singleton, because a singleton has length zero.

For an interval, the same event formula applies without modification. If component \(j\) has density \(f_j\) on the interval of interest, its probability over endpoints \(a\) and \(b\), where \(a<b\), is the integral of \(f_j\) over that interval. Whether an endpoint is included does not affect this probability for a component with a density. For a component with atoms, endpoint inclusion can matter, so use the event exactly as stated.

Worked Example: A Point Mass Mixed With a Uniform Component

Suppose \(J=1\) with probability \(0.3\) and \(J=2\) with probability \(0.7\). Given \(J=1\), let \(X=0\) with probability one. Given \(J=2\), let \(X\) have the uniform distribution on \((0,2)\). Find \(P(X=0)\), \(P(0<X\leq 1)\), and \(P(X\leq 1)\).

Under component 1, \(P_1(X=0)=1\), while under component 2 the distribution is continuous and \(P_2(X=0)=0\). Thus

$$ P(X=0)=0.3(1)+0.7(0)=0.3. $$

For the event \(0<X\leq 1\), component 1 contributes zero. Under the uniform distribution on an interval of length \(2\), the subinterval \((0,1]\) has length \(1\), so its probability is \(1/2\). Therefore

P(0<X\leq 1)=0.3(0)+0.7\left(\frac{1}{2}\right)=0.35.

The event \(X\leq 1\) includes the point mass at zero, as well as the uniform component's probability on \((0,1]\). Hence

P(X\leq 1)=0.3(1)+0.7\left(\frac{1}{2}\right)=0.65.

Worked Example: Adding Overlapping Component Atoms

A mixture selects component 1 with weight \(0.4\) and component 2 with weight \(0.6\). Component 1 assigns probability \(0.5\) to \(X=0\) and \(0.5\) to \(X=1\). Component 2 assigns probability \(0.25\) to \(X=0\) and \(0.75\) to \(X=2\). Find each mixture atom mass and \(P(X\in\{0,2\})\).

At zero, both components contribute. Their weighted contributions are \(0.4(0.5)=0.20\) and \(0.6(0.25)=0.15\), giving

$$ P(X=0)=0.20+0.15=0.35. $$

At one, only component 1 contributes, so \(P(X=1)=0.4(0.5)=0.20\). At two, only component 2 contributes, so \(P(X=2)=0.6(0.75)=0.45\). The masses sum to \(0.35+0.20+0.45=1\), which checks the calculation. For the event \(\{0,2\}\), add its atom masses:

P(X\in\{0,2\})=P(X=0)+P(X=2)=0.35+0.45=0.80.

Equivalently, component 1 assigns probability \(0.5\) to \(\{0,2\}\), and component 2 assigns probability \(1\), so the weighted calculation is \(0.4(0.5)+0.6(1)=0.80\). The two methods agree.

Combining Atoms and Continuous Probabilities

An event can include both a component's atom and a portion of one or more continuous components. The safest calculation is to write the component probability for the exact event first, and then weight it. This prevents an endpoint atom from being omitted or counted twice.

Worked Example: A Point Event and an Interval Event

Let a mixture have three components: with weight \(0.2\), \(X=1\) with probability one; with weight \(0.5\), \(X\) is uniform on \((0,2)\); and with weight \(0.3\), \(X\) has an exponential distribution with rate \(1\), with density \(f(x)=e^{-x}\) for \(x>0\). Calculate \(P(X=1)\), \(P(0.5<X\leq1.5)\), and \(P(X>1)\).

Only the point-mass component has positive probability at exactly one. Both continuous components assign probability zero to a singleton, so

$$ P(X=1)=0.2(1)+0.5(0)+0.3(0)=0.2. $$

For \(0.5<X\leq1.5\), the point-mass component contributes probability one. The uniform component gives interval length \(1\) divided by its total length \(2\), or \(1/2\). For the exponential component, integrate its density:

P_3(0.5<X\leq1.5)=\int_{0.5}^{1.5}e^{-x}\,dx =e^{-0.5}-e^{-1.5}.

Thus the mixture probability is

\begin{aligned} P(0.5<X\leq1.5) &=0.2(1)+0.5\left(\frac{1}{2}\right) +0.3(e^{-0.5}-e^{-1.5})\\ &\approx 0.2+0.25+0.3(0.60653-0.22313)\\ &\approx 0.56502. \end{aligned}

For \(X>1\), the point-mass component contributes zero because its value equals one, not a value greater than one. The uniform component contributes \(1/2\), and the exponential component contributes

P_3(X>1)=\int_{1}^{\infty}e^{-x}\,dx=e^{-1}.

Therefore

P(X>1)=0.2(0)+0.5\left(\frac{1}{2}\right)+0.3e^{-1} \approx 0.25+0.11036=0.36036.

This last calculation highlights why the exact inequality matters: the component concentrated at one contributes to \(P(X\geq1)\), but not to \(P(X>1)\).

A Density Formula When Every Component Has a Density

If all the components have densities \(f_1,\ldots,f_m\), then the mixture also has a density, given by the weighted sum of those densities. This statement requires that each component be absolutely continuous with respect to length; it does not apply to a component with a point mass.

Result: If component \(j\) has density \(f_j\) for every \(j\), then a density for the mixture is

f(x)=\sum_{j=1}^{m}w_jf_j(x).

Derivation. For any Borel set \(A\), the event formula and the density definition give

P(X\in A)=\sum_{j=1}^{m}w_j\int_A f_j(x)\,dx =\int_A\left(\sum_{j=1}^{m}w_jf_j(x)\right)\,dx.

The equality between the finite sum of integrals and the integral of the sum follows from linearity of integration. The function in parentheses is nonnegative, and its total integral is

\int_{\mathbb{R}}\sum_{j=1}^{m}w_jf_j(x)\,dx =\sum_{j=1}^{m}w_j\int_{\mathbb{R}}f_j(x)\,dx =\sum_{j=1}^{m}w_j=1.

It therefore is a density for the mixture. In the presence of atoms, this density formula alone does not describe the entire law: atom probabilities must still be calculated using the singleton event formula. \(\square\)

Common Mistakes and a Reliable Calculation

The most common error is to treat the component weight as if it were the probability of an event. A weight answers “How likely was this component to be selected?” It does not answer “How likely is the event under that component?” These are separate quantities, and both are needed.

Using a weight alone. The contribution from component \(j\) to event \(A\) is \(w_jP_j(X\in A)\), not just \(w_j\).
Ignoring overlap at a point. If several components assign positive probability to \(X=a\), their weighted contributions all belong in \(P(X=a)\).
Assuming every component contributes to an atom. A component with a density assigns probability zero to any single point, even when its density is positive there.
Dropping endpoint atoms. For events such as \(X>1\) and \(X\geq1\), a component concentrated at one contributes differently. Apply the formula to the stated event rather than treating the endpoints as interchangeable.

A dependable procedure is to list the components, write down each component's probability for the event, multiply by its weight, and add. For a point event, use the component's point mass; for an interval under a continuous component, integrate its density. Check that the weights sum to one and, where possible, compare the result with a complementary event or with the total mass.

Key takeaway: A finite mixture's probability for any event is the sum of the component probabilities, each multiplied by its selection weight. The same rule handles continuous probabilities, atoms, and events receiving contributions from both.

Check Your Understanding

Use the component event probabilities and mixture weights separately in each calculation.

A mixture selects a point mass at zero with weight \(0.25\) or a uniform distribution on \((0,4)\) with weight \(0.75\). Find \(P(X=0)\) and \(P(1<X\leq3)\).
Two components have weights \(0.3\) and \(0.7\). Their respective probabilities at \(X=2\) are \(0.4\) and \(0.1\). What is the mixture's atom mass at two?
Explain why a component with a density contributes zero to \(P(X=a)\), even if its density is positive at \(a\).
A mixture includes a point mass at one and a continuous component. Which event, \(X>1\) or \(X\geq1\), receives the point-mass component's contribution?
State the mixture formula for \(P(X\in A)\) and explain in words what each factor in a summand represents.