Joint Laws, Supports, and Probability

Joint Laws and Joint Cumulative Distribution Functions

A multivariate observation records several coordinates together. Their joint law describes not only which values each coordinate can take, but also which combinations can occur and how probability is allocated among those combinations. This is why a collection of marginal distributions alone is not enough: it does not generally determine the relationship between coordinates.

Definition: A random vector $X=(X_1,\ldots,X_d)$ is a measurable map from a probability space into $\mathbb{R}^d$. Its joint distribution is the probability measure $P_X$ on the Borel sets of $\mathbb{R}^d$ defined by $P_X(A)=P(X\in A)$. Its joint cumulative distribution function (joint CDF) is $$ F(x_1,\ldots,x_d)=P(X_1\le x_1,\ldots,X_d\le x_d),\qquad (x_1,\ldots,x_d)\in\mathbb{R}^d. $$

The joint CDF assigns probability to a lower-left rectangle whose upper corner is $x$. In two dimensions, for instance, $F(x,y)=P(X_1\le x,\ X_2\le y)$. A single value of $F$ gives the probability of that particular lower-left rectangle, not the probability of every rectangle or every other set. But the full function $F$ determines the joint law.

Rectangle Probabilities and Boundary Conventions

For real numbers $a_i<b_i$, the rectangle $\prod_{i=1}^d(a_i,b_i]$ includes each upper face and excludes each lower face. Its probability is an alternating sum of joint CDF values. Write $c_S$ for the vector whose $i$th coordinate is $a_i$ when $i$ belongs to $S$, and $b_i$ otherwise. Then

Theorem: For any random vector and any $a_i<b_i$, $$ P(a_i<X_i\le b_i\text{ for every }i) =\sum_{S\subseteq\{1,\ldots,d\}}(-1)^{|S|}F(c_S). $$

In two dimensions this becomes

P(a_1<X_1\le b_1,\ a_2<X_2\le b_2) =F(b_1,b_2)-F(a_1,b_2)-F(b_1,a_2)+F(a_1,a_2).

To see why, let $I_i(t)$ be the indicator of the event $X_i\le t$. The indicator of $a_i<X_i\le b_i$ is $I_i(b_i)-I_i(a_i)$. Multiplying these differences across coordinates gives the indicator of the rectangle. Expanding the product produces one term for each subset $S$, with sign $(-1)^{|S|}$; taking expectations turns each product of indicators into $F(c_S)$. This proves the formula, including when some coordinates have atoms on the rectangle’s faces.

The endpoint convention matters. For example, the formula subtracts the event $X_i\le a_i$, so a value exactly at a lower endpoint is excluded. To include a lower face, define $F$ with selected coordinates just below their endpoints: $F(a_i^-,b_{-i})$ means $P(X_i<a_i,\ X_j\le b_j\text{ for }j\ne i)$. More generally, $F$ with coordinates in a set $S$ replaced by $a_i^-$ means the probability with $X_i<a_i$ for $i\in S$ and the remaining coordinates bounded above as specified. Thus

P(a_i\le X_i\le b_i\text{ for every }i) =\sum_{S\subseteq\{1,\ldots,d\}}(-1)^{|S|} F(c_S^-),

where $c_S^-$ uses $a_i^-$ for $i\in S$ and $b_i$ otherwise. The notation $a_i^-$ is a limit from below: the corresponding event has the strict inequality $X_i<a_i$. This limit is valid because the events with $X_i\le t$ increase to $X_i<a_i$ as $t$ increases to $a_i$, and probability is continuous for increasing sequences of events. When there are no atoms at the lower endpoints, the distinction between strict and inclusive lower boundaries does not change the probability; with atoms, it can.

Worked Example: A Rectangle With Boundary Atoms

Suppose $X=(X_1,X_2)$ takes the four values in the table. Find $P(0<X_1\le1,\ 0<X_2\le1)$, and compare it with $P(0\le X_1\le1,\ 0\le X_2\le1)$.

$(X_1,X_2)$	Probability
$(0,0)$	0.2
$(0,1)$	0.3
$(1,0)$	0.1
$(1,1)$	0.4

For the half-open rectangle, $F(1,1)=1$, $F(0,1)=0.5$, $F(1,0)=0.3$, and $F(0,0)=0.2$. Therefore

P(0<X_1\le1,\ 0<X_2\le1) =1-0.5-0.3+0.2=0.4.

Only the point $(1,1)$ is inside that rectangle. If both lower endpoints are included, all four points are inside, so the probability is $1$. The increase, $0.6$, is exactly the total mass on the other three points, which lie on the lower faces or their intersection. The endpoint convention is therefore substantive, not just notation.

What the Joint CDF Determines

The rectangle formula is useful for direct calculations. More broadly, the joint CDF determines probabilities of all Borel sets, even when a set is not itself a rectangle. The reason is that probabilities of half-open rectangles determine a probability measure uniquely.

Theorem: If two probability measures on $\mathbb{R}^d$ have the same joint CDF at every point, then they assign the same probability to every Borel set.

For a proof, apply the rectangle formula to both measures: equality of their CDFs gives equality of their probabilities on every rectangle $\prod_i(a_i,b_i]$. Such rectangles form a generating class for the Borel sets of $\mathbb{R}^d$, and the two measures are finite and agree on the whole space. The uniqueness theorem for measures on a generating class (equivalently, the $\pi$-$\lambda$ theorem applied to these rectangles) then gives equality on every Borel set. Thus $F$ contains the full joint law, not just probabilities of lower-left rectangles. To calculate a particular nonrectangular event, one can sometimes partition it into rectangles or recover its constituent point probabilities; the CDF does not generally give a one-term formula for an arbitrary shape. $\square$

Support: Where the Law Is Concentrated

Definition: The support of the distribution of $X$ is the set $$ \operatorname{supp}(X)=\{x\in\mathbb{R}^d:P(X\in B(x,\varepsilon))>0\text{ for every }\varepsilon>0\}, $$ where $B(x,\varepsilon)$ is the open Euclidean ball of radius $\varepsilon$ around $x$.

A point is in the support when every neighborhood of it has positive probability. The point itself need not have positive probability: a continuous distribution can have support made up entirely of points that individually have probability zero. Conversely, an isolated point with positive probability is in the support.

The support is closed and has probability one. To see the first fact, if $x$ is outside the support, some open ball around $x$ has probability zero. Every sufficiently nearby point also has a smaller zero-probability neighborhood, so the complement of the support is open. For the second fact, use the countable base of open balls with rational centers and rational radii in $\mathbb{R}^d$. Every point outside the support lies in one such zero-probability ball: choose a zero-probability neighborhood around the point, then a rational ball containing the point and lying inside that neighborhood. The complement of the support is therefore covered by countably many zero-probability balls and has probability zero. Hence $P(X\in\operatorname{supp}(X))=1$.

Worked Example: A Continuous Law Supported on a Diagonal

Let $U$ be uniform on $[0,1]$ and set $X_1=U,\ X_2=U$. Find the joint CDF, the support, and the probability of $0.2<X_1\le0.7,\ 0.5<X_2\le0.9$.

Both coordinates must be at most their respective thresholds, which is equivalent to $U\le\min(x_1,x_2)$. Consequently,

F(x_1,x_2)= \begin{cases} 0,&\min(x_1,x_2)<0,\\ \min(x_1,x_2),&0\le\min(x_1,x_2)\le1,\\ 1,&\min(x_1,x_2)>1. \end{cases}

The support is $\{(u,u):0\le u\le1\}$. Every neighborhood of a point on this diagonal contains values $(u,u)$ for an interval of $u$-values with positive length, and thus has positive probability. For a point off the diagonal, a sufficiently small neighborhood does not meet the diagonal at all, so its probability is zero.

The rectangle event requires $U\in(0.2,0.7]$ and $U\in(0.5,0.9]$ at the same time. Their intersection is $(0.5,0.7]$, whose uniform probability is $0.7-0.5=0.2$. The joint law is concentrated on a line, despite the CDF being defined over the whole plane.

Worked Example: A Nonrectangular Event From Point Masses

Suppose $X$ has the following distribution. Find $P(X_1+X_2\le1)$ and show how the joint CDF can recover the mass at $(1,1)$.

Value of $X$	Probability
$(0,0)$	0.15
$(0,2)$	0.25
$(1,1)$	0.35
$(2,0)$	0.25

The event $X_1+X_2\le1$ contains $(0,0)$ and none of the other three possible values. Its probability is therefore $0.15$. This is not one lower-left rectangle, but the point masses can be identified by CDF differences. In particular, the mass at $(1,1)$ is

$$ F(1,1)-F(0,1)-F(1,0)+F(0,0) =0.50-0.15-0.15+0.15=0.35. $$

Here $F(1,1)=0.50$ includes $(0,0)$ and $(1,1)$; $F(0,1)=F(1,0)=F(0,0)=0.15$. The CDF differences isolate the atom at the upper corner of $(0,1]\times(0,1]$, while the listed support identifies which atoms belong to the nonrectangular event.

Common Pitfalls and Why the Distinctions Matter

The support describes where probability can be found locally, while the joint CDF records how probability accumulates across coordinatewise bounds. Together they help clarify the geometry and the probability content of multivariate events. These distinctions are especially important when a model places probability on boundaries, on a lower-dimensional set, or at isolated points.

Using the wrong endpoint convention. The rectangle $(a,b]$ excludes an atom at $a$, whereas $[a,b]$ includes it. Check the inequalities before applying a CDF formula.
Assuming support points have positive point probability. A point can belong to the support because every neighborhood has positive probability, even if the probability of that point is zero.
Assuming support must be a full-dimensional region. The diagonal example has support on a line. A random vector need not spread probability throughout the ambient space.
Treating marginals as a joint law. Knowing each coordinate’s distribution does not, by itself, specify how the coordinates occur together.

Key takeaway: The joint CDF determines the full joint law. Its finite differences give rectangle probabilities, provided boundary conventions are handled correctly; the support consists of points every neighborhood of which has positive probability.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

For a bivariate CDF, write the formula for $P(a_1<X_1\le b_1,\ a_2<X_2\le b_2)$.
Why can $P(X=x)=0$ even when $x$ belongs to the support of $X$?
What probability does $F(x_1,x_2)$ represent in two dimensions?
When calculating a rectangle that includes its lower faces, why may replacing $a_i^-$ by $a_i$ change the result?
In the diagonal example, why does every point off the diagonal have a neighborhood with probability zero?

\((X_1,X_2)\)	Probability
\((0,0)\)	0.2
\((0,1)\)	0.3
\((1,0)\)	0.1
\((1,1)\)	0.4