Expectation Under a Mixed Distribution

Why a Mixed Distribution Needs Two Contributions

Some random variables have both a positive probability of taking particular values and a continuous range of other possible values. For example, a medical cost may be exactly zero for some patients, while positive costs vary continuously. Neither a probability mass function alone nor a density alone describes the entire distribution.

The expectation must account for both parts of the law: each atom contributes its value multiplied by its probability, and the density component contributes an integral. The main bookkeeping challenge is to represent the density component with its actual probability mass, rather than accidentally treating it as a full probability distribution.

Definition and Expectation Formula

Definition: A random variable $X$ has a mixed distribution of the discrete-plus-absolutely-continuous form if its probability law can be written as $$ P(X\in A)=\sum_{j}p_j\,\mathbf{1}\{a_j\in A\}+\int_A f(x)\,dx $$ for every measurable set $A$. Here $a_j$ are distinct atom locations, $p_j=P(X=a_j)>0$, and $f$ is a nonnegative density component. The masses satisfy $$ \sum_j p_j+\int_{-\infty}^{\infty} f(x)\,dx=1. $$ The density component need not integrate to one: its integral is the probability assigned to the continuous part.

The atoms may be finite or countably infinite in number. The density $f$ can be zero outside one or more regions and can be piecewise defined. At any single point, the integral of a density contributes zero probability, so an atom and the continuous component can be distinguished even when an atom lies within the density’s support.

Theorem: If $E[|X|]<\infty$, then the expectation under this mixed distribution is $$ E[X]=\sum_j a_jp_j+\int_{-\infty}^{\infty}x f(x)\,dx. $$ Both contributions are absolutely integrable: $\sum_j |a_j|p_j<\infty$ and $\int |x|f(x)\,dx<\infty$.

The integrability condition ensures the expectation is a finite, well-defined number. In particular, it allows the atom contributions to be added without relying on an order-dependent cancellation of large positive and negative terms.

Why the Formula Holds

The expectation is the integral of $x$ with respect to the probability law of $X$. The law itself is the sum of an atomic measure, placing mass $p_j$ at each $a_j$, and a measure with density $f$. Integration against those two parts gives the sum and the integral in the formula.

Here is a precise justification. For any nonnegative simple measurable function $h$, the definition of integration and the displayed expression for $P(X\in A)$ give $$ \int h(x)\,dP_X(x)=\sum_j p_j h(a_j)+\int h(x)f(x)\,dx. $$ Every nonnegative measurable $h$ is the increasing limit of nonnegative simple functions. The monotone convergence theorem therefore extends this identity to all such $h$, allowing either side to be infinite.

For a signed integrable function, first apply the identity to its bounded truncations and use dominated convergence, with the absolute value of the function as an integrable bound. Taking the function to be $h(x)=x$ gives the expectation formula. Taking $h(x)=|x|$ shows that finiteness of $E[|X|]$ makes the atomic absolute sum and the density absolute integral finite. Thus both pieces of the formula are justified.

Worked Calculations

Worked Example: An Atom at Zero and a Uniform Component

Suppose $X$ equals 0 with probability $0.3$. With probability $0.7$, $X$ is uniformly distributed on $(0,2)$. Find $E[X]$ using the mixed-distribution formula.

The atom contributes mass $0.3$ at zero. The continuous component has total probability $0.7$, so its density as part of the full law is $0.7(1/2)=0.35$ on $(0,2)$, and zero elsewhere. The factor $1/2$ is the conditional uniform density; the additional factor $0.7$ gives the continuous component its actual mass.

0.3+\int_0^2 0.35\,dx=0.3+0.70=1.

This checks that the stated atom and density make a probability distribution. The atom’s contribution to the mean is $0(0.3)=0$. The continuous contribution is

\int_0^2 x(0.35)\,dx =0.35\left[\frac{x^2}{2}\right]_0^2 =0.35(2)=0.70.

Therefore $E[X]=0+0.70=0.70$. Equivalently, the continuous component has probability $0.7$ and conditional mean $1$, giving $0.7(1)=0.7$.

Worked Example: Two Atoms and a Piecewise Density

Suppose $P(X=-1)=1/4$, $P(X=2)=1/4$, and the density component is

f(x)= \begin{cases} 1/4, & 0<x<2,\\ 0, & \text{otherwise}. \end{cases}

First check the total probability. The atoms have combined mass $1/4+1/4=1/2$, and the density component has mass

\int_0^2 \frac14\,dx=\frac12.

Thus the total mass is $1/2+1/2=1$. The two atoms contribute

(-1)\left(\frac14\right)+2\left(\frac14\right) =-\frac14+\frac12=\frac14.

The density contribution is

\int_0^2 x\left(\frac14\right)\,dx =\frac14\left[\frac{x^2}{2}\right]_0^2 =\frac14(2)=\frac12.

Adding the contributions gives $E[X]=1/4+1/2=3/4$. The negative atom reduces the mean, while the atom at 2 and the density region contribute positive amounts.

Worked Example: A Point Mass and an Exponential Component

Let $P(X=0)=0.4$, with the remaining probability $0.6$ having an exponential distribution with rate 1 on $(0,\infty)$. Find the expectation.

The exponential component has conditional density $e^{-x}$ for $x>0$. As a part of the full distribution its density is $0.6e^{-x}$, whose total mass is $0.6\int_0^\infty e^{-x}\,dx=0.6$. Together with the atom this gives total probability $0.4+0.6=1$.

The atom contributes $0(0.4)=0$. For the continuous component,

\int_0^\infty x(0.6e^{-x})\,dx =0.6\int_0^\infty xe^{-x}\,dx.

Integration by parts on $[0,b]$ gives

\int_0^b xe^{-x}\,dx =\left[-xe^{-x}\right]_0^b+\int_0^b e^{-x}\,dx =-be^{-b}+1-e^{-b}.

As $b\to\infty$, both $be^{-b}$ and $e^{-b}$ tend to zero, so the integral is 1. Hence $E[X]=0+0.6(1)=0.6$.

Finite Mixtures: A Useful Equivalent Calculation

A mixed law is often described by first choosing a component and then drawing from that component’s distribution. The component distributions might themselves be discrete, continuous, or mixed. The following result expresses the expectation in terms of their probabilities and means.

Theorem: Suppose a probability law is a finite mixture $P_X=\sum_{k=1}^K w_kP_k$, where $w_k\geq 0$, $\sum_{k=1}^K w_k=1$, and each component law $P_k$ has a finite expectation $\mu_k$. Then $$ E[X]=\sum_{k=1}^K w_k\mu_k. $$

To prove this, let $h$ be any nonnegative simple measurable function. Integration against the mixture law gives

\int h\,dP_X =\sum_{k=1}^K w_k\int h\,dP_k,

by the definition of a simple-function integral and the identity $P_X(A)=\sum_{k=1}^K w_kP_k(A)$. Increasing simple approximations and monotone convergence extend the identity to nonnegative measurable functions. Applying it to bounded truncations of $x$, and then using dominated convergence under the assumed finite absolute moments, yields $E[X]=\sum_{k=1}^K w_kE_k[X]=\sum_{k=1}^K w_k\mu_k$. This proves the result. $\square$

For example, the first worked example has two components: a point mass at zero with weight $0.3$ and a uniform distribution on $(0,2)$ with weight $0.7$. Their means are 0 and 1, respectively, so the mixture calculation gives $0.3(0)+0.7(1)=0.7$, matching the atom-plus-integral calculation.

Common Mistakes and a Reliable Check

The most common error is to count a mixture weight twice—or not at all. If a component has weight $w$ and conditional density $g$, then its density in the full distribution is $wg$. Use either the full density in the integral, $\int xwg(x)\,dx$, or the component weight times its conditional mean, $wE_g[X]$. Do not multiply by $w$ again after integrating the full density.

A second mistake is to assume that a density component must integrate to one. In a mixed distribution its integral is only the probability assigned to that component. A quick normalization check is

\text{sum of all atom probabilities}+\text{integral of the density component}=1.

A practical calculation can follow these steps:

List the atoms.
Record each location $a_j$ and its probability $p_j$.

Write the full density component.
If given a conditional density for a mixture component, multiply it by that component’s probability weight.

Check total probability.
Verify that atom masses plus integrated density mass equal one.

Calculate and add the mean contributions.
Evaluate $\sum_j a_jp_j$ and $\int x f(x)\,dx$, then combine them.

The method matters whenever a probability law contains a point mass alongside a continuous range of values. Keeping the components separate until the final addition makes the calculation transparent and helps catch both normalization errors and double-counting.

Check Your Understanding

For each question, identify the contribution from each part of the distribution before combining them.

A variable equals 1 with probability $0.2$, and otherwise is uniform on $(0,3)$. What is the full density component, and what is the expectation?
A law has an atom of probability $0.4$ at $-2$ and density $0.3$ on $(0,2)$. Check whether the stated pieces form a probability distribution.
Explain why a conditional density with integral one must be multiplied by its mixture weight when writing the full density component.
A variable has atoms at 0 and 4 with probabilities $0.1$ and $0.2$, and density $0.35$ on $(0,2)$. Calculate its expectation.
In the finite-mixture formula, what does each $\mu_k$ represent, and where does its component weight enter the expectation?