Why Mixture Quantiles and Modes Are Not Averaged

Why Mixture Summaries Need Their Own Calculation

A mixture distribution describes a population whose members come from different component distributions, with specified probabilities of belonging to each component. Its mean is the weighted average of the component means, but that convenient identity does not generally extend to other summaries. In particular, a mixture quantile is determined by accumulated probability, and a mixture mode is determined by the largest probability mass or density. Neither calculation averages component summaries.

This distinction matters when a reported median or most likely value is meant to describe the combined population. A weighted average of component medians can fall in a region where the mixture places little or no probability; an average of component modes may not even be a mode of the combined law.

Mixture Laws and Quantiles

Definition: Let $X$ have a finite mixture distribution with component distribution functions $F_1,\ldots,F_k$ and weights $w_1,\ldots,w_k$, where $w_j\geq 0$ and $\sum_{j=1}^k w_j=1$. Its distribution function is $$ F(x)=\sum_{j=1}^k w_j F_j(x). $$ For $0<\alpha<1$, the $\alpha$-quantile is the generalized inverse $$ Q(\alpha)=\inf\{x:F(x)\geq\alpha\}. $$ A median is a $0.5$-quantile. If the distribution function is continuous and strictly increasing at the crossing of $\alpha$, then $Q(\alpha)$ is the unique solution of $F(x)=\alpha$.

The mixture distribution function follows directly by conditioning on the component label: the probability that $X\leq x$ is the sum, over components, of the probability of selecting that component times its conditional probability of falling at or below $x$. For two components with weights $w$ and $1-w$, a continuous mixture quantile therefore satisfies

wF_1(Q(\alpha))+(1-w)F_2(Q(\alpha))=\alpha,

when the mixture distribution function is continuous and strictly increasing at that quantile. This equation, rather than a weighted average of component quantiles, determines the answer.

Worked Example: The Median of Two Separated Uniform Components

Let $X$ have a mixture distribution with probability $0.6$ from a Uniform$(0,1)$ component and probability $0.4$ from a Uniform$(3,4)$ component. Find the mixture median and compare it with the weighted average of the component medians.

The component distribution functions are $F_1(x)=x$ for $0\leq x\leq1$, and $F_2(x)=0$ for $x<3$. For $0\leq x\leq1$, the mixture distribution function is consequently

$$ F(x)=0.6x+0.4(0)=0.6x. $$

Since $F(1)=0.6$, the mixture median lies in $[0,1]$. Solving $F(x)=0.5$ gives

0.6x=0.5 \qquad\Longrightarrow\qquad Q(0.5)=x=\frac{0.5}{0.6}=\frac{5}{6}\approx0.8333.

The component medians are $0.5$ and $3.5$. Their weighted average is

$$ 0.6(0.5)+0.4(3.5)=0.3+1.4=1.7. $$

Thus the mixture median is about $0.8333$, not $1.7$. The mixture puts $0.6$ of its probability in the first interval, so its halfway point is reached before the first component's support ends. Averaging the component medians ignores how the component probabilities accumulate across the real line.

Theorem: For a finite mixture, $Q(\alpha)$ is determined by the weighted distribution-function equation $F(Q(\alpha))=\alpha$ whenever $F$ is continuous and strictly increasing at the unique crossing. In general, if $F$ has a jump, the generalized-inverse definition applies and $F(Q(\alpha))$ need not equal $\alpha$.

Proof. By the mixture-law identity, $F(x)=\sum_{j=1}^k w_jF_j(x)$. If $F$ is continuous and strictly increasing at the unique point where it crosses level $\alpha$, the set $\{x:F(x)\geq\alpha\}$ begins at the unique solution of $F(x)=\alpha$. Its infimum is that solution, which is $Q(\alpha)$. If $F$ has a jump, the infimum can occur at a point where $F$ has already jumped past $\alpha$; this is why the generalized inverse, rather than the equality alone, is the general definition. $\square$

Modes Must Be Found in the Mixture Law

Definition: For a discrete random variable, a mode is a value at which its probability mass function $f(x)$ is largest. For a continuous random variable with density $f$, a global mode is a point at which the density attains its global maximum. A distribution may have more than one mode. For a finite mixture with component densities $f_j$, its density is $f(x)=\sum_{j=1}^k w_jf_j(x)$; for a discrete mixture, the same weighted-sum formula holds for the probability mass function.

A mixture's mode is a maximizer of its combined mass function or density. For a differentiable two-component density, any interior mode $m$ must satisfy

$$ w f_1'(m)+(1-w)f_2'(m)=0. $$

This is a condition on the component densities' derivatives at the same point. It does not say that $m$ is a weighted average of the component modes. Also, a stationary point need not be a mode: the density must be checked for a maximum, not merely for a zero derivative.

Worked Example: A Discrete Mixture with a Different Mode

With probability $0.6$, let $X$ come from a Bernoulli distribution with success probability $0.2$. With probability $0.4$, let it equal $10+B$, where $B$ is Bernoulli with success probability $0.2$. Find the mixture mode and compare it with the weighted average of the component modes.

The first component assigns probability $0.8$ to 0 and $0.2$ to 1. The second assigns probability $0.8$ to 10 and $0.2$ to 11. Weighting each component's masses gives

P(X=0)=0.6(0.8)=0.48,\quad P(X=1)=0.6(0.2)=0.12,

P(X=10)=0.4(0.8)=0.32,\quad P(X=11)=0.4(0.2)=0.08.

These probabilities sum to $0.48+0.12+0.32+0.08=1$, and the largest is $0.48$, at $X=0$. The first component's unique mode is 0 and the second's is 10, so the weighted average of those modes is $0.6(0)+0.4(10)=4$. But the mixture puts no probability at 4; its mode is 0.

Worked Example: Two Modes in a Symmetric Normal Mixture

Let $X$ be equally likely to come from $N(-3,1)$ or $N(3,1)$. Find the mixture's global modes and compare them with the weighted average of the component modes.

The component modes are $-3$ and $3$, and the mixture density is

f(x)=\frac{1}{2\sqrt{2\pi}} \left\{\exp\left(-\frac{(x+3)^2}{2}\right) +\exp\left(-\frac{(x-3)^2}{2}\right)\right\}.

Using $(x+3)^2=x^2+6x+9$ and $(x-3)^2=x^2-6x+9$, this can be written as

f(x)=\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{x^2+9}{2}\right)\cosh(3x).

The density is positive, so its stationary points are the zeros of its log-density derivative:

\frac{d}{dx}\log f(x)=-x+3\tanh(3x).

Write $g(x)=3\tanh(3x)-x$. For $x>0$, $g(0)=0$, $g'(0)=8>0$, and $g(3)=3\tanh(9)-3<0$. Moreover,

g''(x)=-18\operatorname{sech}^2(3x)\tanh(3x)<0 \qquad (x>0).

Thus $g$ is strictly concave on the positive half-line: it initially rises above zero and, since $g(3)<0$, crosses zero exactly once at some $r\in(0,3)$. The log-density derivative is positive between 0 and $r$ and negative beyond $r$, so $r$ is a local maximum. By symmetry, $-r$ is also a local maximum; $x=0$ is a local minimum because the derivative changes from negative to positive there. The two global modes are therefore $-r$ and $r$, where

r=3\tanh(3r),\qquad 0<r<3.

Numerically, $r$ is approximately $2.9999999$. The weighted average of the component modes is $\tfrac12(-3)+\tfrac12(3)=0$, which is the local minimum, not a mode of the mixture.

Common Mistake and Practical Check

Incorrect reasoning: “The mixture uses each component with probability $w_j$, so its $\alpha$-quantile or mode must be $\sum_j w_j$ times the corresponding component summary.” This reasoning incorrectly transfers the mean's linearity to summaries defined by a threshold crossing or a maximum.

Correction: For a quantile, first form the mixture distribution function and find where it reaches probability $\alpha$. For a mode, first form the mixture mass function or density and find where that combined function is largest. The component summaries may help describe the components, but they do not replace either calculation.

For a quantile.
Write $F(x)=\sum_jw_jF_j(x)$, then solve $F(x)=\alpha$ when continuity and strict increase make that equation applicable. Otherwise use $Q(\alpha)=\inf\{x:F(x)\geq\alpha\}$.

For a mode.
Write the mixture mass function or density, then compare its values over the support. For a differentiable density, stationary points are candidates; check which are global maxima.

The distinction is useful whenever a mixture summarizes heterogeneous groups. A combined population can have a median in one component's range, a mode belonging to the more heavily weighted component, or multiple modes even when each component has only one. Calculating from the mixture law preserves the actual probability structure instead of substituting an average of summaries.

Check Your Understanding

Use the mixture law in each question rather than assuming component summaries average.

A mixture assigns weights $0.25$ and $0.75$ to two continuous components. State the equation that determines its unique $\alpha$-quantile when the mixture distribution function is continuous and strictly increasing at that quantile.
For the Uniform$(0,1)$ and Uniform$(3,4)$ mixture in the first example, what is the mixture distribution function for $1<x<3$? What does this imply for quantiles in that interval?
In the discrete example, what would change in the mixture mode if the component weights were $0.4$ and $0.6$ instead?
Why does a zero derivative of a mixture density not by itself establish that a point is a mode?
In the symmetric normal example, why is the weighted average of the component modes not a global mode?

Why Mixture Quantiles and Modes Are Not Averaged

What You'll Learn