Mixture Distribution Sampling

Why Mixture Sampling Requires Two Steps

A model may describe a population as a blend of distinct groups, even when the group membership is not observed for each simulated observation. For example, a hypothetical portfolio might contain one group of relatively small claims and another group of larger claims. A finite mixture represents this structure by assigning a probability to each component distribution.

To generate one observation, first select a component according to its weight. Then generate an observation from that selected component. The component-selection step is not a choice of the most likely group: it is a random choice with the specified probabilities. The second step must use the distribution of the component actually selected.

Definition: A finite mixture distribution has component distributions $F_1,\ldots,F_k$ and weights $\pi_1,\ldots,\pi_k$, where $\pi_j\geq 0$ and $\sum_{j=1}^{k}\pi_j=1$. Its cumulative distribution function is $F(x)=\sum_{j=1}^{k}\pi_jF_j(x)$. Equivalently, choose a component label $J$ with $P(J=j)=\pi_j$, then draw $X$ from $F_j$ conditional on $J=j$.

For simulation, use one uniform variate to select the component and then use the random input required by that component’s sampling method. If $U_1\sim\operatorname{Uniform}(0,1)$, the cumulative selection intervals are formed from the weights. Define $c_j=\sum_{r=1}^{j}\pi_r$, with $c_0=0$; select component $j$ when $c_{j-1}\leq U_1<c_j$. A boundary can be assigned to either adjacent interval because a continuous uniform variate equals any particular boundary with probability zero.

\begin{array}{c|c} \text{Component} & \text{Selection interval for }U_1\\ 1 &[0,\pi_1)\\ 2 &[\pi_1,\pi_1+\pi_2)\\ \vdots &\vdots\\ j &[c_{j-1},c_j) \end{array}

After choosing $J=j$, generate $X$ from $F_j$. For an inverse-transform component sampler, use a fresh uniform variate $U_2$ and set $X=F_j^{-1}(U_2)$, where $F_j^{-1}(u)=\inf\{x:F_j(x)\geq u\}$. Other component-specific generators may be used instead. The random input for component generation should be independent of the input used to select the component.

Why the Two-Step Algorithm Has the Mixture Distribution

The component-selection probabilities ensure that each component contributes to the output distribution in proportion to its weight. Conditional on selecting component $j$, the chance that the generated value is at most $x$ is $F_j(x)$. Therefore the unconditional chance is the weighted sum of these conditional chances.

Justification: By conditioning on the selected component,

\begin{aligned} P(X\leq x) &=\sum_{j=1}^{k}P(J=j)P(X\leq x\mid J=j)\\ &=\sum_{j=1}^{k}\pi_jF_j(x)=F(x). \end{aligned}

Thus the two-step procedure produces the specified mixture CDF, provided the selection probabilities are correct and the conditional generator for every component is correct. This argument also applies when a component is discrete or has a point mass: $F_j$ need not have a density.

Prepare the weights.
Check that all weights are nonnegative and sum to one. Compute their cumulative sums.

Select a component.
Compare a uniform variate with the cumulative sums and identify the interval containing it.

Generate conditionally.
Use the sampling method and parameters for the selected component, with independent random input where required.

Check the result.
Confirm that the selected interval, component-specific calculation, and units all agree with the model.

Worked Examples

Worked Example: A Two-Component Exponential Mixture

A hypothetical claim amount $X$ is modeled as a mixture: with probability $0.30$, it is exponential with mean $\theta_1=\$400$; with probability $0.70$, it is exponential with mean $\theta_2=\$1{,}200$. Use independent uniform variates $U_1=0.24$ and $U_2=0.50$ to generate one claim amount. Round the final result to the nearest cent.

Method and component selection. The cumulative weights are $c_1=0.30$ and $c_2=1.00$. Since $0\leq 0.24<0.30$, select component 1, the exponential distribution with mean $\$400$.

Generate from the selected component. An exponential distribution with mean $\theta$ has CDF $F(x)=1-e^{-x/\theta}$ for $x\geq 0$. Setting $U_2=F(x)$ and solving gives $x=-\theta\ln(1-U_2)$. Therefore,

X=-400\ln(1-0.50)=-400\ln(0.50) =400\ln(2)=277.2589\ldots

The simulated claim amount is $\$277.26$. As a check, $U_2=0.50$ is the median input for the selected exponential distribution, whose median is $400\ln(2)=\$277.26$.

Common mistake avoided. Reusing $U_1=0.24$ for the exponential step would give $-400\ln(0.76)=\$109.77$, not the result from the specified independent inputs. It also links the component choice to the within-component draw. Use a separate, independent variate for component generation.

Worked Example: Sampling a Mixture of Normal Distributions

A hypothetical measurement $X$, in dollars, has two possible component distributions: $N(1000,200^2)$ with weight $0.25$, and $N(1600,300^2)$ with weight $0.75$. Here the second argument of $N(\mu,\sigma^2)$ is the variance. Given an independent component-selection variate $U_1=0.40$ and a standard normal variate $Z=0.80$, generate $X$. Round to the nearest cent.

Method and component selection. The cumulative weights are $0.25$ and $1.00$. Since $0.25\leq0.40<1.00$, select component 2. Its mean is $\$1{,}600$ and its standard deviation is $\$300$, since $\sqrt{300^2}=300$.

Apply the normal location-scale transformation. For a selected component $N(\mu,\sigma^2)$, use $X=\mu+\sigma Z$. Thus,

$$ X=1600+300(0.80)=1600+240=1840. $$

The generated measurement is $\$1{,}840.00$. The value is above the selected component’s mean because the supplied standard normal variate is positive.

Common mistake avoided. The first component is not selected just because it is listed first. The threshold is $0.25$, not $0.50$: $U_1=0.40$ lies in the second component’s interval. Also, the normal transformation uses the selected component’s standard deviation of $300$, not its variance of $90{,}000$.

Worked Example: A Mixture With a Point Mass at Zero

A hypothetical coverage payment is zero with probability $0.20$. With probability $0.80$, it is exponential with mean $\$500$. Use independent uniforms $U_1=0.65$ and $U_2=0.40$ to generate one payment. Give the result to the nearest cent and check it against the mixture CDF.

Method and component selection. The zero-payment component occupies $[0,0.20)$, and the exponential component occupies $[0.20,1)$. Since $0.65\geq0.20$, select the exponential component.

Generate the payment. Applying the exponential inverse transform with mean $\$500$,

X=-500\ln(1-0.40)=-500\ln(0.60) =255.4128\ldots

The payment is $\$255.41$. For a CDF check, the continuous-component CDF at the unrounded simulated value is $F_2(X)=U_2=0.40$. The mixture CDF there is $0.20+0.80(0.40)=0.52$, which equals the mixture probability accumulated from the zero mass and the exponential component up to this value. The generated value is positive, as required after selecting the exponential component.

Common mistake avoided. The weight $0.20$ is a probability of selecting the zero component; it is not an amount to subtract from the exponential draw. Once the exponential component is selected, the conditional draw uses its own full exponential distribution.

Useful Checks for a Mixture Model

A sample path check confirms that one generated value follows the correct branch. For a larger simulation, mixture moments give additional checks on the distribution as a whole. If component $j$ has finite mean $\mu_j$ and variance $\sigma_j^2$, then the mixture mean $\mu$ and variance are

\mu=E[X]=\sum_{j=1}^{k}\pi_j\mu_j, \qquad \operatorname{Var}(X)=\sum_{j=1}^{k}\pi_j\left(\sigma_j^2+(\mu_j-\mu)^2\right).

The variance expression follows by conditioning on the component. Within component $j$, the squared deviation from the overall mean has expectation $\sigma_j^2+(\mu_j-\mu)^2$: the first term is variation around that component’s own mean, while the second is the squared distance between its mean and the mixture mean. Weighting these conditional expectations by $\pi_j$ gives the formula. It applies when the component means and variances are finite.

For the normal mixture in Worked Example 2, the overall mean is $0.25(1000)+0.75(1600)=1450$. Its variance is

\begin{aligned} \operatorname{Var}(X) &=0.25\left(200^2+(1000-1450)^2\right) +0.75\left(300^2+(1600-1450)^2\right)\\ &=0.25(40{,}000+202{,}500)+0.75(90{,}000+22{,}500)\\ &=60{,}625+84{,}375=145{,}000. \end{aligned}

The standard deviation is $\sqrt{145{,}000}=\$380.79$, rounded to the nearest cent. The between-component terms matter: averaging only the component variances would omit the variability caused by the different component means.

Common Exam Traps

Using weights as if they were cutoffs individually. With three weights $0.20,0.30,0.50$, the intervals are $[0,0.20)$, $[0.20,0.50)$, and $[0.50,1)$. The second cutoff is the cumulative weight $0.50$, not $0.30$.
Reversing weights or component parameters. Keep each weight attached to its own distribution. Selecting the correct interval but then using another component’s parameters generates from the wrong model.
Using one random input for both steps. The selection variate and component-specific random input must be independent for the standard two-step construction.
Ignoring point masses. A component may generate a fixed value, such as zero. The mixture can therefore have a discrete part even if another component is continuous.
Confusing a mixture with a sum. One mixture observation comes from one selected component. It is not the sum of one draw from every component.

Key takeaway: Convert the weights to cumulative intervals, select exactly one component, and then generate from that component using its specified distribution. Conditioning on the selected component proves that the resulting CDF is the weighted mixture CDF.

Check Your Understanding

For each problem, keep component selection separate from component generation.

A mixture has weights $0.20,0.50,0.30$, with exponential component means $\$100,\$300,\$900$, respectively. Given independent $U_1=0.68$ and $U_2=0.50$, identify the selected component and generate $X$, to the nearest cent. (Answer: Component 2; $X=300\ln 2=\$207.94$.)
A measurement has distribution $N(500,100^2)$ with weight $0.40$ and $N(900,200^2)$ with weight $0.60$. Given $U_1=0.35$ and $Z=1.20$, which component is selected and what is the generated measurement? (Answer: Component 1; $X=500+100(1.20)=\$620.00$.)
A payment is zero with probability $0.15$; otherwise it is exponential with mean $\$250$. Given independent $U_1=0.70$ and $U_2=0.20$, generate a payment and give the mixture CDF at that unrounded value. (Answer: The exponential component is selected; $X=-250\ln(0.80)=\$55.79$ to the nearest cent, and the mixture CDF at the unrounded value is $0.15+0.85(0.20)=0.3200$.)
A mixture has weights $0.25$ and $0.75$, with component means $\$700$ and $\$1{,}300$, and component variances $40{,}000$ and $90{,}000$ dollars squared. Calculate the mixture mean and variance. (Answer: $E[X]=\$1{,}150$; $\operatorname{Var}(X)=0.25(40{,}000+202{,}500)+0.75(90{,}000+22{,}500)=145{,}000$ dollars squared.)
A three-component mixture has weights $0.10,0.35,0.55$, and exponential means $\$400,\$1{,}000,\$2{,}000$, respectively. Given $U_1=0.46$ and an independent $U_2=0.50$, identify the selected component, generate a value to the nearest cent, and calculate the mixture mean. (Answer: Component 3; $X=-2{,}000\ln(0.50)=\$1{,}386.29$; $E[X]=0.10(400)+0.35(1{,}000)+0.55(2{,}000)=\$1{,}490$.)

What You'll Learn