Discrete, Continuous, and Mixed Vector Laws

Three Ways Probability Can Be Distributed

For a random vector $X$ in $\mathbb{R}^d$, probabilities can be concentrated on countably many points, spread across regions according to a density, or divided between point masses and a density component. The distinction matters whenever an event includes both particular vector values and a region of positive volume: the point probabilities and the density integral must both be counted.

Here, a density means a density with respect to $d$-dimensional Lebesgue measure. This qualification is important: a vector can have continuous-looking coordinates without having a joint density in $\mathbb{R}^d$. The tutorial on singular laws treats distributions concentrated on lower-dimensional sets in more detail. Our focus is the distinction among countable-support laws, absolutely continuous laws, and mixtures of those two types.

Definition: A random vector $X$ in $\mathbb{R}^d$ has a discrete law if there is a countable set $S\subseteq\mathbb{R}^d$ such that $P(X\in S)=1$. Its probability mass function is $p(x)=P(X=x)$, and $\sum_{x\in S}p(x)=1$. The law is absolutely continuous if there is a nonnegative measurable function $f$ such that, for every Borel set $A$, $P(X\in A)=\int_A f(x)\,dx$, where $\int_{\mathbb{R}^d}f(x)\,dx=1$. The function $f$ is a density. A discrete–continuous mixture has both a nonzero discrete component and a nonzero absolutely continuous component, and no other component.

For an absolutely continuous law, every single point has probability zero because a point has Lebesgue measure zero. For a discrete law, the probabilities at its support points add to one. A discrete–continuous mixture combines these contributions. In particular, the density of a mixture describes only its absolutely continuous component; it does not record the probability masses at atoms.

Worked Example: A Discrete Vector and a Joint Event

Let $X=(X_1,X_2)$ have the following probability mass function. The table gives $P(X_1=i,X_2=j)$.

$X_1\backslash X_2$	0	1
0	0.10	0.20
1	0.30	0.40

The four probabilities sum to $0.10+0.20+0.30+0.40=1$, so this specifies a discrete vector law. For the event $A=\{X_1+X_2\geq 1\}$, include every table entry whose coordinates satisfy the inequality. These are $(0,1)$, $(1,0)$, and $(1,1)$, so

P(X\in A)=0.20+0.30+0.40=0.90.

The event is a set of vector values, not a region to integrate over. Summing the masses at those values is the appropriate calculation.

Recognizing a Density and Integrating Over a Region

For an absolutely continuous vector, probabilities are computed by integrating the joint density over the event region. A density value is not itself a probability at a point: probabilities come from integrals over sets. In two dimensions, the integral accounts for area; in $d$ dimensions, it accounts for $d$-dimensional volume.

Worked Example: A Continuous Vector on a Triangle

Suppose $X=(X_1,X_2)$ has density $f(x_1,x_2)=2$ on the triangle $0<x_1<x_2<1$, and density zero elsewhere. First check normalization. The triangle has area $1/2$, so the integral of the density is $2(1/2)=1$.

Calculate $P(X_1+X_2<1)$. Within the support, the event requires $0<x_1<x_2$ and $x_1<1-x_2$. The two upper bounds on $x_1$ switch at $x_2=1/2$. Thus the probability is

\begin{aligned} P(X_1+X_2<1) &=\int_0^{1/2}\int_0^{x_2}2\,dx_1\,dx_2 +\int_{1/2}^1\int_0^{1-x_2}2\,dx_1\,dx_2\\ &=\int_0^{1/2}2x_2\,dx_2 +\int_{1/2}^1 2(1-x_2)\,dx_2\\ &=\frac14+\frac14=\frac12. \end{aligned}

The support restriction $x_1<x_2$ is part of the calculation. Integrating over the whole unit square would count points where the density is zero.

Adding Atoms and a Density Component

A common mixed model first chooses a component and then draws a vector from that component’s law. Suppose component 0 is chosen with probability $w$ and has a discrete law with masses $p_x$, while component 1 is chosen with probability $1-w$ and has density $f$. For any Borel event $A$, the total probability is the weighted sum of the probabilities within the two components.

Theorem: Let $0<w<1$. If the discrete component has masses $p_x$ on a countable set $S$, and the continuous component has density $f$ on $\mathbb{R}^d$, the mixture law satisfies $$ P(X\in A)=w\sum_{x\in S}p_x\,\mathbf{1}_{A}(x)+(1-w)\int_A f(x)\,dx $$ for every Borel set $A$. Its atom at $x\in S$ has probability $P(X=x)=wp_x$, and its absolutely continuous component has density $(1-w)f$.

Proof. Let $C$ indicate which component was selected, with $P(C=0)=w$ and $P(C=1)=1-w$. By the law of total probability,

\begin{aligned} P(X\in A) &=P(X\in A\mid C=0)P(C=0) +P(X\in A\mid C=1)P(C=1)\\ &=w\sum_{x\in S}p_x\mathbf{1}_{A}(x) +(1-w)\int_A f(x)\,dx. \end{aligned}

For a singleton $A=\{x\}$, the integral term is zero, so $P(X=x)=wp_x$ when $x\in S$; points outside $S$ have zero mass as well. The second term is an integral against the density $(1-w)f$. Since the weights sum to one and both component laws are normalized, the mixture law is normalized. $\square$

Worked Example: An Atom Plus a Uniform Vector

Let $X\in\mathbb{R}^2$ equal $(0,0)$ with probability $0.3$. With probability $0.7$, let $X$ be uniform on the unit square $[0,1]^2$. The uniform component has density $1$ on that square and zero outside it, so the mixture’s density component is $0.7$ on the square. The atom contributes a separate mass of $0.3$ at the origin.

Find the probability that $X_1+X_2<1$. The atom at the origin belongs to the event. In the square, the event is a triangle of area $1/2$. Therefore

P(X_1+X_2<1)=0.3+0.7\left(\frac12\right)=0.65.

The same result follows by integrating the weighted density over the triangle and then adding the atom:

P(X_1+X_2<1) =0.3+\int_0^1\int_0^{1-x_1}0.7\,dx_2\,dx_1 =0.3+0.7\int_0^1(1-x_1)\,dx_1 =0.65.

If the event were instead $\{X=(0,0)\}$, its probability would be $0.3$, not the value of the density at the origin. The continuous component assigns probability zero to that single point.

Calculating Expectations From Both Components

The same separation applies to expectations. The discrete part contributes a sum of function values weighted by masses; the continuous part contributes an integral. The formula below is useful for probabilities by taking the function to be an event indicator, and for moments by taking it to be a coordinate or other measurable function.

Theorem: Suppose a vector law has the discrete–continuous representation $$ P(X\in A)=\sum_{x\in S}p_x\mathbf{1}_A(x)+\int_A g(x)\,dx, $$ where $S$ is countable, $p_x\geq 0$, $g\geq 0$ is measurable, and $\sum_{x\in S}p_x+\int_{\mathbb{R}^d}g(x)\,dx=1$. For any measurable $\varphi$ with $E[|\varphi(X)|]<\infty$, $$ E[\varphi(X)]=\sum_{x\in S}p_x\varphi(x)+\int_{\mathbb{R}^d}\varphi(x)g(x)\,dx. $$

Proof. First take $\varphi$ to be a nonnegative simple function, written as $\varphi=\sum_{j=1}^m a_j\mathbf{1}_{A_j}$ for disjoint Borel sets $A_j$ and $a_j\geq 0$. By the definition of expectation and the assumed law,

\begin{aligned} E[\varphi(X)] &=\sum_{j=1}^m a_jP(X\in A_j)\\ &=\sum_{j=1}^m a_j\left(\sum_{x\in S}p_x\mathbf{1}_{A_j}(x)+\int_{A_j}g(x)\,dx\right)\\ &=\sum_{x\in S}p_x\varphi(x)+\int_{\mathbb{R}^d}\varphi(x)g(x)\,dx. \end{aligned}

For a general nonnegative measurable $\varphi$, choose nonnegative simple functions increasing pointwise to $\varphi$. The monotone convergence theorem applies to the expectation, the countable sum, and the integral, giving the same identity. For integrable signed $\varphi$, apply this result separately to $\varphi^+=\max(\varphi,0)$ and $\varphi^-=\max(-\varphi,0)$, then subtract. The integrability assumption ensures both expectations are finite. $\square$

Worked Example: The Mean of a Mixed Scalar Law

Let $X=0$ with probability $0.2$, and with probability $0.8$ draw $X$ uniformly from $[1,3]$. The uniform density is $1/2$ on $[1,3]$, so the continuous component of the mixture has density $0.8(1/2)=0.4$ there. Using the expectation formula,

\begin{aligned} E[X] &=0.2(0)+\int_1^3 x(0.4)\,dx\\ &=0.4\left[\frac{x^2}{2}\right]_1^3\\ &=0.4\left(\frac92-\frac12\right) =0.4(4)=1.6. \end{aligned}

The atom contributes zero to the mean in this example, but it still contributes $0.2$ to the total probability. A zero contribution to a particular expectation does not mean the atom can be omitted from the law.

Common Pitfalls and Why Classification Matters

Using only a density for a mixed law. A density integrates to the probability of the absolutely continuous component over a region; it does not include point masses. Add the atomic probabilities separately.
Treating a density value as a point probability. For an absolutely continuous law, $P(X=x)=0$ at every fixed point, regardless of whether the density is large there.
Assuming that no atoms means there is a density. A law can have no point masses and still fail to be absolutely continuous. The relevant definition of absolute continuity is that every Lebesgue-null set has probability zero.
Classifying a joint law from its marginals alone. Continuous marginals do not guarantee a joint density in the full vector space. Classification requires the joint law.

The classification determines how probabilities should be computed and what a density can represent. In applications, a point mass might represent a structural zero or a distinct outcome, while the continuous component describes variation within another group. Keeping the components separate prevents an apparently convenient integral from silently dropping real probability.

Check Your Understanding

For each question, identify which probability masses or integrals are needed and explain your reasoning.

A vector has probability $0.25$ at $(1,1)$ and otherwise has a density on the unit square. How would you calculate the probability of a region that contains $(1,1)$?
Why does an absolutely continuous vector assign probability zero to each individual point?
A discrete vector has masses $0.1,0.2,0.3,0.4$ at four points. What check verifies that these values define a probability mass function?
For a mixture that selects a discrete component with probability $w$, how does a component mass $p_x$ contribute to $P(X=x)$ when the other component has a density?
Why is observing that every coordinate has a continuous marginal distribution not sufficient to conclude that the joint vector has a density?