Why Condition on a Category?
A random outcome may come from one of several different mechanisms, while the mechanism used on a particular trial is not observed. For example, a hypothetical service claim might come from one of two product types, or a device might have been manufactured by one of several processes. If the category is known, the outcome may follow a simple distribution; if it is hidden, the unconditional outcome combines those distributions.
The category is a random variable that helps describe how \(X\) is generated. It is not itself the outcome \(X\). The mixture distribution is obtained by conditioning on the category and then weighting each conditional distribution by the probability of that category. The weights must be nonnegative and sum to one.
The model says to select a category according to the weights \(w_j\), then generate \(X\) from that category's conditional distribution. Thus, the distribution of \(X\) alone must account for every possible category. This is different from adding independent component random variables: a single outcome comes from one selected component, rather than from a sum of all components.
Combining Component Distributions
For any real number \(x\), the event \(\{X\leq x\}\) can occur together with exactly one of the category events \(\{Z=j\}\). Those category events are disjoint and cover all possibilities. The law of total probability therefore gives the mixture cumulative distribution function.
This formula applies to discrete, continuous, or mixed component distributions. It automatically retains jumps: if one or more components have probability at a point, the mixture can also have probability at that point. When all components have probability mass functions, the mixture pmf is the corresponding weighted sum of conditional pmfs. When all components have densities, the mixture density is the weighted sum of conditional densities. These pmf and density statements require the stated type of component distribution; the cdf formula does not.
The same conditioning principle gives moments, provided the moments exist. If each component has a finite mean \(\mu_j=E[X\mid Z=j]\), then
For a finite second moment, write \(m_{2,j}=E[X^2\mid Z=j]\). Then
A useful equivalent form uses the component variances \(\sigma_j^2=\operatorname{Var}(X\mid Z=j)\):
The first term averages the variation within each component. The second records the variation in component means around the overall mean. It is nonnegative, and it is zero only when every positive-weight component has the same mean.
Why the Variance Formula Holds
The derivation follows from the second-moment identity \(\operatorname{Var}(X)=E[X^2]-(E[X])^2\). Within component \(j\), the definition of variance gives \(E[X^2\mid Z=j]=\sigma_j^2+\mu_j^2\). Averaging this conditional second moment over categories yields
Subtract \(\mu^2\), where \(\mu=\sum_jw_j\mu_j\). The difference between the mean-square terms can be rewritten as
because expanding the right-hand side gives \(\sum_jw_j\mu_j^2-2\mu\sum_jw_j\mu_j+\mu^2\sum_jw_j\), and the last two terms combine to \(-2\mu^2+\mu^2=-\mu^2\). This proves the stated variance formula. The derivation requires finite component second moments; without them, the variance need not be finite or defined.
Worked Example: A Claim Amount from One of Two Exponential Components
A hypothetical portfolio has two unobserved claim categories. With probability \(0.25\), a claim amount \(X\), measured in thousands of dollars, has an exponential distribution with rate \(1\) per thousand dollars. With probability \(0.75\), it has an exponential distribution with rate \(2\) per thousand dollars. Conditional on its category, the claim follows the stated exponential model. Find \(P(X\leq 1)\), \(E[X]\), and \(\operatorname{Var}(X)\), with amounts in thousands of dollars.
Method. Condition on the category for the cdf and moments. For an exponential distribution with rate \(\lambda\), \(F(x)=1-e^{-\lambda x}\) for \(x\geq0\), \(E[X]=1/\lambda\), and \(\operatorname{Var}(X)=1/\lambda^2\).
At \(x=1\), the two conditional cdf values are \(1-e^{-1}\) and \(1-e^{-2}\). Therefore
Rounded to four decimal places, \(P(X\leq1)=0.8065\). The calculation uses the exact expressions before rounding, rather than first rounding each component cdf.
The component means are \(1/1=1\) and \(1/2=0.5\). Thus
The component variances are \(1\) and \(1/4=0.25\). The conditional second moments are variance plus squared mean: \(1+1^2=2\) and \(0.25+0.5^2=0.5\). Hence
As a check, the within-component contribution is \(0.25(1)+0.75(0.25)=0.4375\), and the between-component contribution is \(0.25(0.75)(1-0.5)^2=0.046875\). Their sum is \(0.484375\), as above. The common mistake avoided here is rounding component cdf values too early: \(0.25(0.6321)+0.75(0.8647)\) gives \(0.80655\), which can misleadingly round differently. The exact probability rounds to \(0.8065\) to four decimal places.
Worked Example: A Discrete Mixture with a Binomial and a Poisson Component
A hypothetical count \(X\) is generated by first selecting category A with probability \(0.40\) or category B with probability \(0.60\). Conditional on A, \(X\sim\operatorname{Binomial}(2,0.5)\); conditional on B, \(X\sim\operatorname{Poisson}(1)\). Find \(P(X\leq1)\), \(E[X]\), and \(\operatorname{Var}(X)\).
Method. Compute each requested quantity conditionally, then weight by the category probabilities. For the binomial component, the mean is \(np\) and the variance is \(np(1-p)\). For the Poisson component, both the mean and variance equal its parameter.
For category A, \(P(X\leq1\mid A)=P(X=0)+P(X=1)=0.25+0.50=0.75\). For category B, \(P(X\leq1\mid B)=e^{-1}+e^{-1}=2e^{-1}=0.73575888\ldots\). Thus
Rounded to four decimal places, the probability is \(0.7415\). The conditional means are both \(1\), so \(E[X]=0.40(1)+0.60(1)=1\).
For category A, the second moment is variance plus squared mean: \(0.5+1^2=1.5\). For category B it is \(1+1^2=2\). Therefore
The answer is consistent with the variance formula: the component means are equal, so there is no between-component contribution, and the variance is \(0.40(0.5)+0.60(1)=0.80\). Notice that equal component means do not imply equal component distributions or equal variances.
Worked Example: A Mixture of Normal Distributions
A hypothetical measurement \(X\) is drawn from one of two normal populations. The first is selected with probability \(0.30\) and has distribution \(N(1,1)\); the second is selected with probability \(0.70\) and has distribution \(N(3,4)\). The second argument of \(N(\mu,\sigma^2)\) is the variance. Find \(P(X\leq2)\), the mixture mean, and the mixture variance.
Method. Standardize the threshold within each conditional normal distribution for the cdf, then combine the conditional means and variances using the mixture moment formulas.
For the first component, the standard deviation is \(1\), so the standardized value at \(2\) is \((2-1)/1=1\). For the second, the standard deviation is \(2\), so it is \((2-3)/2=-0.5\). Using \(\Phi(1)=0.841344746\ldots\) and \(\Phi(-0.5)=0.308537539\ldots\),
Thus \(P(X\leq2)=0.4684\) to four decimal places. The mixture mean is
The weighted within-component variance is \(0.30(1)+0.70(4)=3.10\). The between-component contribution is \(0.30(1-2.40)^2+0.70(3-2.40)^2=0.30(1.96)+0.70(0.36)=0.588+0.252=0.840\). Consequently,
A check using second moments gives \(E[X^2]=0.30(1+1^2)+0.70(4+3^2)=0.60+9.10=9.70\), and \(9.70-(2.40)^2=9.70-5.76=3.94\). The two methods agree.
Common Exam Trap: Averaging Variances Alone
The weighted average of the component variances is generally not the variance of the mixture. It accounts only for variation within the selected component. If component means differ, outcomes can also vary because the selected category changes the center of the distribution.
For cdf calculations, another frequent error is to use a component distribution without conditioning on the category or to weight it by the wrong category probability. A reliable sequence is: identify the categories, write their probabilities, calculate the requested conditional quantity in each category, and only then take the weighted sum. Check that all weights are nonnegative and add to one.
Mixture models are useful whenever a population is heterogeneous but each subpopulation has a tractable model. In exam problems, the wording may describe a random selection of types, routes, sources, or operating conditions rather than use the word “mixture.” Representing that selection as a category variable makes the conditioning explicit and prevents accidentally treating the components as independent quantities that are all added together.
Check Your Understanding
For each problem, condition on the stated category and combine the results. Probabilities are reported to four decimal places where needed.
- A device has category A with probability \(0.20\) and category B with probability \(0.80\). Conditional on A, its failure indicator \(X\) is Bernoulli with parameter \(0.30\); conditional on B, it is Bernoulli with parameter \(0.70\). Find \(P(X=1)\) and \(E[X]\). (Answer: \(P(X=1)=0.6200,\ E[X]=0.62\))
- A waiting time \(X\) is exponential with rate \(2\) per hour with probability \(0.40\), and exponential with rate \(0.5\) per hour with probability \(0.60\). Find \(P(X>2)\) and \(E[X]\). (Answer: \(P(X>2)=0.2281,\ E[X]=1.4\) hours)
- With equal probabilities, \(X\) has either distribution \(N(0,1)\) or \(N(2,1)\). Find \(E[X]\), \(\operatorname{Var}(X)\), and \(P(X\leq1)\). (Answer: \(E[X]=1,\ \operatorname{Var}(X)=2,\ P(X\leq1)=0.5000\))
- A count \(X\) has a binomial distribution with parameters \(3\) and \(0.5\) with probability \(0.25\), and a Poisson distribution with parameter \(2\) with probability \(0.75\). Find \(E[X]\) and \(P(X\leq1)\). (Answer: \(E[X]=1.875,\ P(X\leq1)=0.4295\))