Constructing Finite Product Probability Measures

From Coordinate Laws to a Joint Probability Measure

Suppose each coordinate has its own probability space. To describe several coordinates jointly, we need a probability measure on the product space that assigns the expected probability to every event specified separately in each coordinate. The central prescription is simple for rectangles: the probability that every coordinate belongs to its specified set is the product of the individual probabilities. The construction must then show that these assignments extend consistently to the sigma-algebra generated by the rectangles.

Let $(X_i,\mathcal{M}_i,P_i)$, $1\le i\le n$, be probability spaces, where $n$ is a positive integer. Thus $P_i(X_i)=1$. Set $X=\prod_{i=1}^n X_i$, and let $\mathcal{M}=\bigotimes_{i=1}^n\mathcal{M}_i$ denote the sigma-algebra generated by measurable rectangles $\prod_{i=1}^n A_i$, with $A_i\in\mathcal{M}_i$. The coordinate map $\pi_i:X\to X_i$ is given by $\pi_i(x_1,\ldots,x_n)=x_i$.

Definition: A finite product probability measure is a probability measure $P$ on $(X,\mathcal{M})$ satisfying $$ P\left(\prod_{i=1}^n A_i\right)=\prod_{i=1}^n P_i(A_i) $$ for all $A_i\in\mathcal{M}_i$. Under this measure the coordinate maps are independent: for every choice of $A_i\in\mathcal{M}_i$, $P(\bigcap_{i=1}^n\pi_i^{-1}(A_i))=\prod_{i=1}^nP(\pi_i^{-1}(A_i))$.

We construct the measure first for two factors and then apply that construction repeatedly. For two probability spaces $(X_1,\mathcal{M}_1,P_1)$ and $(X_2,\mathcal{M}_2,P_2)$, consider the algebra $\mathcal{A}$ of finite unions of measurable rectangles $A\times B$. An algebra is closed under finite unions and complements; it need not be closed under countable unions. The intended rectangle content is $q(A\times B)=P_1(A)P_2(B)$. A finite grid, formed from the finitely many coordinate sets in a rectangle representation and their complements, partitions each coordinate space into finitely many measurable pieces. Its product cells are disjoint rectangles. Consequently, the sum of the rectangle contents of the cells making up a set $E\in\mathcal{A}$ gives a well-defined value $q(E)$, independent of the representation. This definition gives finite additivity and $q(X_1\times X_2)=1$.

Why the Rectangle Content Extends

To apply the extension theorem, the key point is countable additivity whenever a disjoint sequence of algebra sets has its union in the algebra. The covering estimate below supplies the needed inequality. It uses only countable subadditivity of the coordinate probability $P_2$ and the Monotone Convergence Theorem on the first coordinate space.

Lemma (rectangle covering estimate): If $A\times B\subseteq\bigcup_{k=1}^\infty(A_k\times B_k)$, with all coordinate sets measurable, then $$ P_1(A)P_2(B)\le \sum_{k=1}^\infty P_1(A_k)P_2(B_k). $$

Proof. For each $x\in A$, the covering assumption implies $B\subseteq\bigcup_{\{k:x\in A_k\}}B_k$. Countable subadditivity of $P_2$ therefore gives $P_2(B)\le\sum_{k=1}^\infty P_2(B_k)\chi_{A_k}(x)$. Integrate this inequality over $A$ with respect to $P_1$. The terms on the right are nonnegative measurable functions, and their partial sums increase to the displayed series. By the Monotone Convergence Theorem, their integral is the sum of their integrals. Thus $$ P_1(A)P_2(B)\le\sum_{k=1}^\infty P_2(B_k)P_1(A\cap A_k)\le\sum_{k=1}^\infty P_1(A_k)P_2(B_k). $$ This also covers zero-probability sets and infinite values of the series. $\square$

Proof that $q$ is a premeasure. Let $E_1,E_2,\ldots$ be pairwise disjoint members of $\mathcal{A}$, and suppose $E=\bigcup_{k=1}^\infty E_k$ also belongs to $\mathcal{A}$. Each algebra set is a finite disjoint union of rectangles, by refining a finite rectangle representation into a finite grid. Finite additivity then gives $q(E)\ge\sum_{k=1}^Nq(E_k)$ for every $N$, since the first $N$ sets are disjoint subsets of $E$. Letting $N$ increase gives $q(E)\ge\sum_{k=1}^\infty q(E_k)$.

For the reverse inequality, write $E$ as a finite disjoint union of rectangles. Each of those rectangles is covered by the countable collection of rectangle pieces used to represent all the $E_k$. Apply the rectangle covering estimate to each rectangle in $E$ and add the resulting finitely many inequalities. The sum of the contents of the pieces representing $E_k$ is $q(E_k)$, so $q(E)\le\sum_{k=1}^\infty q(E_k)$. Hence equality holds, which is precisely countable additivity on the algebra when the union is in the algebra. $\square$

Theorem (finite product construction): For finitely many probability spaces $(X_i,\mathcal{M}_i,P_i)$, there exists a probability measure $P$ on $(\prod_{i=1}^nX_i,\bigotimes_{i=1}^n\mathcal{M}_i)$ such that $P(\prod_iA_i)=\prod_iP_i(A_i)$. It is unique.

Proof. For two factors, the preceding argument shows that $q$ is a premeasure on $\mathcal{A}$, and its total mass is one. The Caratheodory extension theorem states that a finite premeasure on an algebra extends to a measure on the sigma-algebra generated by that algebra, and that the extension is unique when the premeasure is finite. Applying it gives the required probability measure on $\mathcal{M}_1\otimes\mathcal{M}_2$. For $n>2$, apply the two-factor construction inductively, treating the product of the first $n-1$ spaces, with its constructed probability measure, as the first factor. For $n=1$, the required measure is $P_1$ itself. This proves existence for every positive finite $n$.

For uniqueness, rectangles form a pi-system: they include the whole space and are closed under finite intersections. They generate $\mathcal{M}$. If $P$ and $Q$ are two probability measures agreeing on rectangles, the collection $\mathcal{D}=\{E\in\mathcal{M}:P(E)=Q(E)\}$ contains the whole space, is closed under complements, and is closed under countable disjoint unions. Thus it is a lambda-system. The pi-lambda theorem implies that $\mathcal{D}$ contains the sigma-algebra generated by the rectangles, namely $\mathcal{M}$. Hence $P=Q$. $\square$

The coordinate marginal law follows directly: taking $A_i$ arbitrary and every other coordinate set equal to its whole space gives $P(\pi_i^{-1}(A_i))=P_i(A_i)$. The product formula then says that coordinate events specified in separate coordinates are independent. The measure is defined on the product sigma-algebra; it does not assert that every subset of the product is measurable, or that the product sigma-algebra is complete.

Calculating Probabilities Under the Product Measure

Worked Example: Two Independent Bernoulli Coordinates

Let $X_1=X_2=\{0,1\}$, with each coordinate assigning probability $1/3$ to $1$ and $2/3$ to $0$. Under the product probability measure, calculate the probability that the two coordinates differ.

The event is the disjoint union of the two rectangles $\{(1,0)\}$ and $\{(0,1)\}$. Their probabilities are $$ P(\{1\}\times\{0\})=\frac13\cdot\frac23=\frac29,\qquad P(\{0\}\times\{1\})=\frac23\cdot\frac13=\frac29. $$ Therefore $P(X_1\ne X_2)=\frac29+\frac29=\frac49$. Check: the complementary event consists of $(0,0)$ and $(1,1)$, with probability $\frac23\cdot\frac23+\frac13\cdot\frac13=\frac49+\frac19=\frac59$. The two event probabilities sum to $1$.

Worked Example: A Continuous and a Discrete Coordinate

Let $X_1=[0,1]$ with Lebesgue probability measure and $X_2=\{a,b,c\}$, with probabilities $P_2(\{a\})=\frac12$, $P_2(\{b\})=\frac13$, and $P_2(\{c\})=\frac16$. Calculate the probability of $$ E=([0,\tfrac12]\times\{a,b\})\cup([\tfrac12,1]\times\{c\}). $$

The two rectangles are disjoint: their second-coordinate sets are disjoint. The shared endpoint $1/2$ in their first-coordinate intervals causes no overlap because $a,b$ are distinct from $c$. The interval $[0,\frac12]$ has Lebesgue measure $\frac12$, and $[\frac12,1]$ also has measure $\frac12$. Therefore $$ P(E)=\frac12\left(\frac12+\frac13\right)+\frac12\left(\frac16\right) =\frac12\cdot\frac56+\frac1{12} =\frac5{12}+\frac1{12} =\frac12. $$ Check: the complement consists of $[0,\frac12]\times\{c\}$ and $[\frac12,1]\times\{a,b\}$, whose probability is $\frac12\cdot\frac16+\frac12\cdot\frac56=\frac1{12}+\frac5{12}=\frac12$. The probabilities add to one.

Worked Example: At Least One Coordinate Falls in an Interval

Take three independent coordinates, each uniform on $[0,1]$, and let $A=[0,\frac14]$. Calculate the probability that at least one coordinate belongs to $A$.

The event is a union of three coordinate events, so calculate its probability by taking the complement: all three coordinates lie in $A^c=(\frac14,1]$. The interval has measure $1-\frac14=\frac34$. The product formula gives $$ P(\text{at least one coordinate is in }A) =1-P(A^c\times A^c\times A^c) =1-\left(\frac34\right)^3 =1-\frac{27}{64} =\frac{37}{64}. $$ Check: inclusion-exclusion gives $3(\frac14)-3(\frac1{16})+\frac1{64}=\frac{48-12+1}{64}=\frac{37}{64}$. Each intersection probability used here is a rectangle probability from the product formula.

A Common Error and a Useful Check

A common mistake is to add coordinate probabilities when calculating the probability of a rectangle. For independent events $A$ in the first coordinate and $B$ in the second, the rectangle event is their intersection, not their union. For example, if $P_1(A)=\frac13$ and $P_2(B)=\frac12$, the incorrect calculation $\frac13+\frac12=\frac56$ does not give the rectangle probability. The correct calculation is $P(A\times B)=\frac13\cdot\frac12=\frac16$. It also passes the basic bound $P(A\times B)\le P(A)$ and $P(A\times B)\le P(B)$; the incorrect value $\frac56$ does not.

For a more general event, first express it as a finite disjoint union of measurable rectangles whenever possible, then sum their product probabilities. If the event is instead a union with overlaps, do not add the rectangle probabilities without correcting for the overlaps; a complement or a disjoint partition may simplify the calculation. The construction guarantees the product rule for coordinate rectangles, while the extension theorem guarantees a well-defined probability for every event in the generated product sigma-algebra.

Check Your Understanding

For each question, identify the product space and use the rectangle rule only for measurable coordinate sets.

If two independent coordinates satisfy $P_1(A)=\frac25$ and $P_2(B)=\frac34$, what is the probability of $A\times B$?
For the two Bernoulli coordinates above, calculate the probability that exactly one coordinate equals $0$.
Explain why the rectangle content must be countably additive when a disjoint sequence of algebra sets has its union in the algebra.
For three independent uniform coordinates on $[0,1]$, calculate the probability that all coordinates lie in $(\frac12,1]$.
State which theorem gives existence of the extension and which theorem gives uniqueness from agreement on rectangles.

Constructing Finite Product Probability Measures

What You'll Learn