Deriving Total Variance For Mixture Problems

Why Mixture Variance Has Two Sources

When a random outcome depends on a hidden category, knowing the category would reduce uncertainty about the outcome. But uncertainty remains: outcomes still vary within each category, and the category itself is not known in advance. Total variance accounts for both sources. This distinction matters in mixture problems and in changing-risk scenarios, where a shared environment or latent type changes the conditional distribution.

Let $X$ be a random variable with finite second moment, and let $Z$ represent a category or other information on which we condition. The conditional mean $E[X\mid Z]$ is itself random before $Z$ is observed: its value depends on which category occurs. Likewise, $\operatorname{Var}(X\mid Z)$ is a random quantity whose value depends on the category. The law of total variance combines these conditional quantities.

Theorem: If $E[X^2]$ is finite, then $$ \operatorname{Var}(X)=E[\operatorname{Var}(X\mid Z)]+\operatorname{Var}(E[X\mid Z]). $$ The first term is the expected variance within the categories. The second term is the variance of the category-specific means. The formula applies whether $Z$ is discrete or continuous, provided the required expectations exist.

For a finite mixture with categories $z_1,\ldots,z_k$, write $p_j=P(Z=z_j)$, $\mu_j=E[X\mid Z=z_j]$, and $\sigma_j^2=\operatorname{Var}(X\mid Z=z_j)$. Then

E[\operatorname{Var}(X\mid Z)]=\sum_{j=1}^k p_j\sigma_j^2, \qquad \operatorname{Var}(E[X\mid Z])=\sum_{j=1}^k p_j(\mu_j-\mu)^2, \qquad \mu=\sum_{j=1}^k p_j\mu_j.

Thus, the first component averages the conditional variances using the category probabilities. The second measures how far the conditional means lie from the overall mean. Even if each category has little internal variation, the overall variance can be large when category means differ substantially.

Why the Decomposition Holds

The key is to split the deviation of $X$ from its overall mean into a within-category deviation and a difference between means. The within-category part has conditional mean zero, which makes the cross term vanish.

Split the deviation.
Let $m(Z)=E[X\mid Z]$ and $\mu=E[X]$. Then $X-\mu=(X-m(Z))+(m(Z)-\mu)$.

Square and condition.
Expanding the square gives the two squared terms and the cross term $2(X-m(Z))(m(Z)-\mu)$. Conditional on $Z$, the factor $m(Z)-\mu$ is fixed and $E[X-m(Z)\mid Z]=0$, so the conditional expectation of the cross term is zero.

Take expectations.
The expected first squared term is $E[\operatorname{Var}(X\mid Z)]$. The second is $E[(m(Z)-\mu)^2]=\operatorname{Var}(m(Z))$, because $E[m(Z)]=E[X]=\mu$.

Combining these terms yields the theorem. This argument also explains why the between-category term is a variance, not an additional conditional variance: it measures the spread of the conditional means across possible values of $Z$.

Applying Total Variance to Mixtures

For a mixture, first identify the latent category, then calculate the mean and variance within each category. The mixture mean is a probability-weighted average of conditional means. The total variance is not generally a probability-weighted average of conditional variances alone.

Worked Example: A Two-Class Claim Severity Mixture

A hypothetical portfolio has claim severity $X$, measured in dollars. A randomly selected claim is from class $L$ with probability $0.7000$ and class $H$ with probability $0.3000$. Given class $L$, $X$ is exponential with mean $\$1{,}000$; given class $H$, it is exponential with mean $\$3{,}000$. For an exponential distribution with mean $\theta$, the variance is $\theta^2$. Find $E[X]$ and $\operatorname{Var}(X)$.

Method and assumptions. Condition on class $Z$. The stated exponential models have finite second moments, so total variance applies. The class probabilities and the conditional distributions are the complete model information.

The conditional means and variances are

\mu_L=1{,}000,\quad \sigma_L^2=1{,}000^2=1{,}000{,}000, \qquad \mu_H=3{,}000,\quad \sigma_H^2=3{,}000^2=9{,}000{,}000.

First find the overall mean:

E[X]=0.7000(1{,}000)+0.3000(3{,}000) =700+900 =\$1{,}600.

The expected conditional variance is

E[\operatorname{Var}(X\mid Z)] =0.7000(1{,}000{,}000)+0.3000(9{,}000{,}000) =700{,}000+2{,}700{,}000 =3{,}400{,}000.

The conditional means are $\$600$ below and $\$1{,}400$ above the overall mean. Therefore,

\operatorname{Var}(E[X\mid Z]) =0.7000(1{,}000-1{,}600)^2+0.3000(3{,}000-1{,}600)^2 $$ $$ =0.7000(360{,}000)+0.3000(1{,}960{,}000) =252{,}000+588{,}000 =840{,}000.

Adding the two components gives

\operatorname{Var}(X)=3{,}400{,}000+840{,}000 =4{,}240{,}000\text{ dollars}^2.

Check. Within class, $E[X^2\mid L]=1{,}000{,}000+1{,}000^2=2{,}000{,}000$, and $E[X^2\mid H]=9{,}000{,}000+3{,}000^2=18{,}000{,}000$. Thus $E[X^2]=0.7000(2{,}000{,}000)+0.3000(18{,}000{,}000)=6{,}800{,}000$. Subtracting $E[X]^2=1{,}600^2=2{,}560{,}000$ gives $4{,}240{,}000$, as above. The common mistake this avoids is reporting only $3{,}400{,}000$, which ignores variation between the class means.

Changing Risk Across an Environment

A mixture need not be described as a portfolio of permanent types. It can also arise when a common, unobserved environment changes the probability of each outcome. Conditional on that environment, a familiar distribution may apply; unconditionally, the outcomes can be more variable because the environment itself is uncertain.

Worked Example: Failures Under a Changing Operating Regime

A hypothetical system has 10 components. During a randomly selected operating period, the system is in regime $A$ with probability $0.7500$, where each component fails independently with probability $0.1000$. It is in regime $B$ with probability $0.2500$, where each component fails independently with probability $0.3000$. Let $N$ be the number of failures. Find $E[N]$ and $\operatorname{Var}(N)$.

Method and assumptions. Conditional on the regime, $N$ is binomial: $N\mid A\sim\operatorname{Binomial}(10,0.1000)$ and $N\mid B\sim\operatorname{Binomial}(10,0.3000)$. The component failures are independent within a regime, as specified. The regime is selected once for the period, so its failure probability applies to all 10 components in that period.

For a binomial random variable with parameters $n$ and $p$, the conditional mean is $np$ and the conditional variance is $np(1-p)$. Hence,

E[N\mid A]=1,\quad \operatorname{Var}(N\mid A)=10(0.1000)(0.9000)=0.9000, $$ $$ E[N\mid B]=3,\quad \operatorname{Var}(N\mid B)=10(0.3000)(0.7000)=2.1000.

The overall mean is

$$ E[N]=0.7500(1)+0.2500(3)=0.7500+0.7500=1.5000. $$

The expected conditional variance is

E[\operatorname{Var}(N\mid Z)] =0.7500(0.9000)+0.2500(2.1000) =0.6750+0.5250 =1.2000.

The variance of the conditional means is

\operatorname{Var}(E[N\mid Z]) =0.7500(1-1.5000)^2+0.2500(3-1.5000)^2 $$ $$ =0.7500(0.2500)+0.2500(2.2500) =0.1875+0.5625 =0.7500.

Therefore, $\operatorname{Var}(N)=1.2000+0.7500=1.9500$ failures squared. As a check, the conditional second moments are $E[N^2\mid A]=0.9000+1^2=1.9000$ and $E[N^2\mid B]=2.1000+3^2=11.1000$. Thus $E[N^2]=0.7500(1.9000)+0.2500(11.1000)=4.2000$, and $4.2000-(1.5000)^2=1.9500$. The extra $0.7500$ is the variation associated with the uncertain regime; using only the variance for one fixed failure probability would not describe this model.

Discrete Mixtures and a Reliable Check

The same calculation works when conditional distributions are discrete and not members of a named family. Compute each conditional mean and variance from its probability table, then use the category weights. A second-moment calculation provides a useful independent check:

\operatorname{Var}(X)=E[X^2]-\{E[X]\}^2, \qquad E[X^2]=E[\operatorname{Var}(X\mid Z)]+\ E[(E[X\mid Z])^2].

The second identity follows by applying $E[X^2\mid Z]=\operatorname{Var}(X\mid Z)+(E[X\mid Z])^2$ and then taking expectations. It is especially useful for detecting arithmetic errors in either component.

Worked Example: A Warranty Payment With Two Repair Types

A hypothetical warranty payment $X$ is measured in thousands of dollars. A repair is type $A$ with probability $0.8000$, and the payment is 10 or 30 with equal conditional probabilities. A repair is type $B$ with probability $0.2000$, and the payment is 200 with probability $0.2500$ or 600 with probability $0.7500$. Find the mean payment and its variance.

Method and assumptions. The repair type is the conditioning category. All values and probabilities are conditional on the stated type; the two type probabilities sum to $1.0000$. The payment has finite support, so its second moment is finite.

For type $A$,

E[X\mid A]=0.5000(10)+0.5000(30)=20, $$ $$ E[X^2\mid A]=0.5000(10^2)+0.5000(30^2)=50+450=500, $$ $$ \operatorname{Var}(X\mid A)=500-20^2=100.

For type $B$,

E[X\mid B]=0.2500(200)+0.7500(600)=50+450=500, $$ $$ E[X^2\mid B]=0.2500(200^2)+0.7500(600^2) =10{,}000+270{,}000=280{,}000, $$ $$ \operatorname{Var}(X\mid B)=280{,}000-500^2=30{,}000.

The overall mean is $0.8000(20)+0.2000(500)=16+100=116$ thousand dollars. The expected conditional variance is

E[\operatorname{Var}(X\mid Z)] =0.8000(100)+0.2000(30{,}000) =80+6{,}000 =6{,}080.

The variance of the conditional means is

\operatorname{Var}(E[X\mid Z]) =0.8000(20-116)^2+0.2000(500-116)^2 $$ $$ =0.8000(9{,}216)+0.2000(147{,}456) =7{,}372.8+29{,}491.2 =36{,}864.

Therefore, the total variance is $6{,}080+36{,}864=42{,}944$ in squared thousands of dollars. To check, $E[X^2]=0.8000(500)+0.2000(280{,}000)=400+56{,}000=56{,}400$; subtracting $116^2=13{,}456$ gives $42{,}944$. The large between-type component is plausible because the type-specific means, 20 and 500, are far apart.

Common Trap: Omitting Between-Category Variation

A frequent error is to calculate $\sum_j p_j\sigma_j^2$ and call it the total variance. That quantity is only $E[\operatorname{Var}(X\mid Z)]$, the average within-category variance. It omits the variation in conditional means. In the claim-severity example, this incorrect calculation gives $3{,}400{,}000$ dollars squared, while the correct total is $4{,}240{,}000$ dollars squared. The omitted $840{,}000$ is exactly the variance of the class-specific means.

The reverse error is also possible: calculating the variance of the conditional means but forgetting the within-category variances. A sound exam method is to label both terms before adding them:

Within categories: average the conditional variances using the category probabilities.
Between categories: calculate the variance of the conditional means around the overall mean.
Combine: add the two nonnegative components.

Do not confuse a shared changing environment with independent changes in the risk probability for each observation. If one regime is selected for an entire period, condition on that regime to determine the count distribution, then use total variance. The conditional model must match the problem’s timing and dependence assumptions.

Key takeaway: Total variance equals expected within-category variance plus between-category variance. Whenever conditional distributions differ across hidden types or environments, calculate both terms; the conditional variances alone do not capture all uncertainty.

Check Your Understanding

For each problem, identify the conditioning category, calculate the two variance components, and combine them.

A category $A$ has probability $0.4000$, conditional mean 5, and conditional variance 4. Category $B$ has probability $0.6000$, conditional mean 9, and conditional variance 16. Find the overall mean and variance. (Answer: mean $7.4000$, variance $15.0400$)
A system has 8 components. With probability $0.8000$, each fails independently with probability $0.0500$; otherwise, each fails independently with probability $0.2000$. The regime is shared across the 8 components. Find the mean and variance of the failure count. (Answer: mean $0.6400$, variance $0.7904$)
A loss is exponential with mean 400 when $Z=A$, and exponential with mean 800 when $Z=B$. The category probabilities are $0.2500$ and $0.7500$, respectively. Find the overall variance. (Answer: $550{,}000$)
A payment is 10 or 30 with equal probabilities in category $A$, and 20 or 60 with equal probabilities in category $B$. Category $A$ has probability $0.7000$. Find the overall variance. (Answer: $274$)
A variable has total variance 50. Its category probabilities are $0.6000$ and $0.4000$, and its conditional means are 4 and 10. If the conditional variance in the first category is 9, find the conditional variance in the second category. (Answer: $89.9000$)

What You'll Learn