Representing Laws of Random Vectors

One Joint Law, Different Representations

A random vector records several random quantities together. Its central object is not a particular formula but its joint law: the probability assigned to each measurable set of possible values. A joint cumulative distribution function, a probability mass function, and a density are different ways of recording that law. Which representation is available depends on the structure of the distribution.

The distinction matters in applications. Counts often have a PMF, measurements with a continuous distribution may have a density, and a dataset can contain both kinds of coordinates. Every random vector has a joint law and a joint CDF, but a PMF or density need not exist in the usual form.

Definition: Let $X=(X_1,\ldots,X_d)$ be a random vector on a probability space $(\Omega,\mathcal{A},P)$, with values in $\mathbb{R}^d$. Its joint law, also called its distribution, is the probability measure $P_X$ on the Borel sets of $\mathbb{R}^d$ defined by $$ P_X(B)=P(X\in B), \qquad B\subseteq\mathbb{R}^d \text{ Borel}. $$ Thus, the law describes probabilities for the vector as a whole, not just for its separate coordinates.

The law is often written $F$, or $F_X$ when identifying the vector is helpful. A representation of the law gives a way to recover $P(X\in B)$ for relevant sets $B$. The main representations used here are summarized below.

Representation	What it records	Typical setting
Joint CDF	Probabilities that every coordinate is at most a specified threshold	Every random vector
Joint PMF	Probability at each possible vector value	Countable support
Joint density	Probability per unit of volume, relative to a specified measure	Absolutely continuous laws

The Joint CDF

For $x=(x_1,\ldots,x_d)$, the joint CDF is the probability that all coordinates lie below their corresponding thresholds. This gives a cumulative representation of the joint law. It is defined whether the coordinates are discrete, continuous, or a mixture of the two.

Definition: The joint CDF of $X=(X_1,\ldots,X_d)$ is $$ F_X(x_1,\ldots,x_d) =P(X_1\le x_1,\ldots,X_d\le x_d), \qquad (x_1,\ldots,x_d)\in\mathbb{R}^d. $$

A CDF is cumulative: its value at a threshold vector concerns an entire lower region of the state space, rather than the probability at one point. The joint CDF is always defined from the law, even if the law has no PMF or density. It is not generally a formula that can be differentiated or summed to obtain a convenient representation.

Worked Example: CDF of a Discrete Random Vector

Let $X$ and $Y$ take values in $\{0,1\}$, with probabilities $P(X=0,Y=0)=0.20$, $P(X=0,Y=1)=0.30$, $P(X=1,Y=0)=0.10$, and $P(X=1,Y=1)=0.40$. The four probabilities are nonnegative and sum to $1$, so they define a joint law.

At thresholds $(x,y)=(0,1)$, the event in the joint CDF is $\{X\le 0,Y\le 1\}$. The constraint on $X$ requires $X=0$, while both values of $Y$ satisfy $Y\le 1$. Therefore,

F_{(X,Y)}(0,1) =P(X=0,Y=0)+P(X=0,Y=1) =0.20+0.30=0.50.

The CDF value is a sum of point probabilities in this example because the law is discrete. In contrast, the CDF is defined by the same threshold-event formula for laws that are not discrete.

The Joint PMF

When a random vector takes values in a countable set, its point probabilities can be listed as a joint PMF. For a discrete vector, these probabilities determine the law: probabilities of sets are obtained by adding the masses of their points. A PMF is not a probability density; its value at a point is itself a probability.

Definition: If $X$ has countable support $S\subseteq\mathbb{R}^d$, its joint PMF is $$ p_X(x)=P(X=x), \qquad x\in S. $$ It satisfies $p_X(x)\ge 0$ for every $x\in S$ and $\sum_{x\in S}p_X(x)=1$. For any set $B$, $P(X\in B)=\sum_{x\in B\cap S}p_X(x)$.

Result: For a countable-support random vector, the PMF gives probabilities of all events by summation.

Proof. The events $\{X=x\}$, for $x\in B\cap S$, are pairwise disjoint. Their union is exactly $\{X\in B\}$, since $X$ takes values in $S$. Countable additivity of probability therefore gives $$ P(X\in B) =P\left(\bigcup_{x\in B\cap S}\{X=x\}\right) =\sum_{x\in B\cap S}P(X=x) =\sum_{x\in B\cap S}p_X(x). $$ Taking $B=\mathbb{R}^d$ also gives $\sum_{x\in S}p_X(x)=1$. $\square$

Worked Example: Probability From a Joint PMF

Use the four-point law from the previous example. Find $P(X+Y=1)$ and $P(X\le Y)$.

The event $X+Y=1$ contains exactly the outcomes $(0,1)$ and $(1,0)$. Thus,

$$ P(X+Y=1)=p_X(0,1)+p_X(1,0)=0.30+0.10=0.40. $$

The event $X\le Y$ contains $(0,0)$, $(0,1)$, and $(1,1)$. Hence $$ P(X\le Y)=0.20+0.30+0.40=0.90. $$ Both calculations add point masses because the outcomes form a countable support.

The Joint Density

A density represents probabilities by integration rather than by assigning a probability to each point. In the usual continuous case, the reference measure is $d$-dimensional Lebesgue measure, corresponding to volume in $\mathbb{R}^d$. A density’s value is not generally the probability of the vector taking that exact value.

Definition: A joint density of $X$ with respect to Lebesgue measure is a nonnegative measurable function $f_X$ such that, for every Borel set $B\subseteq\mathbb{R}^d$, $$ P(X\in B)=\int_B f_X(x)\,dx. $$ In particular, $f_X$ must integrate to $1$ over $\mathbb{R}^d$. A law with such a density is called absolutely continuous with respect to Lebesgue measure.

The defining equation immediately gives the event-probability calculation: integrate the density over the set in question. This formula applies only when the law has the stated density. In particular, a distribution that puts positive probability on a point cannot have a density with respect to ordinary Lebesgue measure, since an individual point has Lebesgue measure zero.

Worked Example: A Density on a Triangular Region

Let $(X,Y)$ have joint density $$ f_{(X,Y)}(x,y)= \begin{cases} 2, & 0<x<y<1,\\ 0, & \text{otherwise}. \end{cases} $$ First verify that it is normalized. For each $y$ between $0$ and $1$, $x$ ranges from $0$ to $y$, so $$ \int_0^1\int_0^y 2\,dx\,dy =\int_0^1 2y\,dy =\left[y^2\right]_0^1 =1. $$

Now calculate $P(X+Y<1)$. Inside the support, this event requires $0<x<y<1$ and $x<1-y$. For $0<y<\tfrac12$, the full interval $0<x<y$ satisfies the extra constraint. For $\tfrac12<y<1$, the upper limit is $1-y$. Therefore,

\begin{aligned} P(X+Y<1) &=\int_0^{1/2}\int_0^y 2\,dx\,dy +\int_{1/2}^1\int_0^{1-y}2\,dx\,dy\\ &=\int_0^{1/2}2y\,dy+\int_{1/2}^1 2(1-y)\,dy\\ &=\frac14+\frac14=\frac12. \end{aligned}

The boundary lines do not change the integral: they have two-dimensional Lebesgue measure zero. Here the density, not a point probability, is the representation used to calculate the event probability.

Mixed Coordinates and the Choice of Measure

A vector can have a discrete coordinate and a continuous coordinate. Its joint law then need not have a density with respect to ordinary two-dimensional Lebesgue measure, because positive probability may lie on vertical lines corresponding to the discrete values. A useful representation instead uses counting measure for the discrete coordinate and Lebesgue measure for the continuous one.

For $X\in\{0,1\}$ and $Y\in\mathbb{R}$, a mixed density $g(x,y)$ with respect to counting measure in $x$ and Lebesgue measure in $y$ means $$ P((X,Y)\in B)=\sum_{x\in\{0,1\}}\int_{\{y:(x,y)\in B\}} g(x,y)\,dy. $$ This representation keeps the discrete masses while integrating over the continuous coordinate.

Worked Example: A Discrete-Continuous Joint Law

Suppose $P(X=0)=0.6$, and conditional on $X=0$, $Y$ is uniform on $[0,1]$. Suppose $P(X=1)=0.4$, and conditional on $X=1$, $Y$ is uniform on $[1,2]$. A mixed density relative to counting measure in $x$ and Lebesgue measure in $y$ is $$ g(0,y)=0.6\,\mathbf{1}_{[0,1]}(y), \qquad g(1,y)=0.4\,\mathbf{1}_{[1,2]}(y), $$ where $\mathbf{1}_A(y)$ equals $1$ when $y\in A$, and $0$ otherwise.

Its total mass is $$ \int_0^1 0.6\,dy+\int_1^2 0.4\,dy=0.6+0.4=1. $$ For example, $P(X=0,Y\le 0.5)=\int_0^{0.5}0.6\,dy=0.3$. Also, $$ P(X\le1,Y\le1.5) =\int_0^1 0.6\,dy+\int_1^{1.5}0.4\,dy =0.6+0.2=0.8. $$ The second calculation includes all values possible when $X=0$, and the portion from $1$ to $1.5$ when $X=1$.

This law cannot have an ordinary density with respect to two-dimensional Lebesgue measure. Indeed, $P(X=0)=0.6$, so the vertical line $\{(0,y):y\in\mathbb{R}\}$ has probability at least $0.6$, while that line has two-dimensional Lebesgue measure zero. The mixed representation handles this positive line mass directly.

How the Representations Fit Together

The joint law is the common object behind all three representations. The joint CDF records probabilities of lower-threshold events. A PMF lists probabilities at countably many points, and a density converts probability into an integral relative to a chosen measure. For a discrete law, the CDF can be calculated by adding the masses below the thresholds; for a law with a density, it can be calculated by integrating over the corresponding lower region. Those are consequences of the event-probability formulas, not definitions of the CDF.

The representation must match the law. A continuous-looking formula is not automatically a density: it must be nonnegative, measurable, and integrate to one relative to the stated measure. A PMF must likewise have nonnegative entries summing to one. Mixed laws call for a representation that preserves their discrete and continuous parts; forcing them into an ordinary Lebesgue density loses the point masses.

Common Mistake

Wrong reasoning: “The joint density at $(x,y)$ is the probability that $(X,Y)=(x,y)$.” This treats density values as point masses.
Correction: A density gives probability only after integration over a region. For an absolutely continuous vector, a single point has probability zero; a PMF, by contrast, assigns point probabilities directly.
Wrong reasoning: “Every joint law has an ordinary density because every law has a joint CDF.”
Correction: A CDF exists for every law, but a Lebesgue density exists only for laws absolutely continuous with respect to Lebesgue measure. Discrete and mixed laws generally require other representations.

Key takeaway: Start with the joint law. Use its CDF for cumulative threshold probabilities, its PMF for a countable support, and a density only relative to a suitable measure. The CDF is universal; the PMF and density are structure-dependent representations.

Check Your Understanding

Answer each question by identifying the representation that fits the stated law.

What event does the joint CDF $F_X(x_1,x_2)$ assign a probability to?
A vector takes values in a finite set. What does its joint PMF record, and how would you find the probability of a set of outcomes?
Why is a density value not generally the probability that a continuous random vector equals one specified point?
A vector has $P(X=0)>0$ for one coordinate $X$. Explain why it cannot have an ordinary two-dimensional Lebesgue density when paired with a continuous coordinate.
For the mixed example, what measure is used for the discrete coordinate and what measure is used for the continuous coordinate?