Building IID Sequences from Product Measures

From One Distribution to an Infinite Sequence

To construct an independent, identically distributed sequence, it is not enough to prescribe the distribution of each coordinate separately: those one-coordinate distributions do not determine how the coordinates depend on one another. The product measure supplies the required joint laws. Its coordinate maps then form an IID sequence, with every finite collection of coordinates having the corresponding finite product distribution.

Let \((S,\mathcal{B})\) be a standard Borel space, meaning a measurable space measurably isomorphic to a Borel subset of a complete separable metric space. Let \(\nu\) be a probability measure on \((S,\mathcal{B})\). The countable product \(S^{\mathbb{N}}\) is equipped with the product sigma-algebra \(\mathcal{B}^{\otimes\mathbb{N}}=\sigma(\pi_n^{-1}(A): n\in\mathbb{N},\,A\in\mathcal{B})\), where \(\pi_n(\omega)=\omega_n\) is the \(n\)th coordinate map. The probability space constructed below is finite, hence sigma-finite; it need not be complete.

The events in this definition are cylinder events: they restrict only finitely many coordinates. The construction uses the Kolmogorov Extension Theorem, established earlier in the course. In the form needed here, it says that a consistent family of finite-dimensional probability distributions on standard Borel coordinate spaces determines a unique probability measure on the product sigma-algebra. We check consistency before applying it.

Constructing the Product Probability Space

For each \(k\ge1\), give the first \(k\) coordinates the finite product probability measure \(\nu^{\otimes k}\) on \((S^k,\mathcal{B}^{\otimes k})\). Its value on a measurable rectangle is

These distributions are consistent under projection. Indeed, for \(m<k\), the projection of \(\nu^{\otimes k}\) onto the first \(m\) coordinates assigns the rectangle \(A_1\times\cdots\times A_m\) the value \(\prod_{j=1}^m\nu(A_j)\): the omitted coordinates contribute \(\nu(S)=1\). This is exactly \(\nu^{\otimes m}(A_1\times\cdots\times A_m)\). Since probability measures agreeing on measurable rectangles agree on the product sigma-algebra, the projected measure is \(\nu^{\otimes m}\).

Proof. The preceding consistency check verifies the hypothesis of the Kolmogorov Extension Theorem for the finite-dimensional distributions \(\nu^{\otimes k}\). The coordinate spaces are standard Borel, so that theorem gives a unique probability measure \(\mathbb{P}\) on \((S^{\mathbb{N}},\mathcal{B}^{\otimes\mathbb{N}})\) with the specified finite-dimensional distributions. The existence and uniqueness assertion is precisely the application of that standard theorem; its proof is not repeated here.

For each \(n\), \(X_n=\pi_n\) is measurable: for \(A\in\mathcal{B}\), \(X_n^{-1}(A)=\pi_n^{-1}(A)\) belongs to the product sigma-algebra by its definition. The one-coordinate distribution is \(\mathbb{P}(X_n\in A)=\nu(A)\), since the one-coordinate marginal is \(\nu^{\otimes1}=\nu\).

Now take distinct indices \(i_1,\dots,i_k\). The joint map \(\omega\mapsto(X_{i_1}(\omega),\dots,X_{i_k}(\omega))\) is measurable. Its distribution on a rectangle \(A_1\times\cdots\times A_k\) is \(\prod_{j=1}^k\nu(A_j)\): this follows from the specified finite-dimensional distribution on the coordinates up to \(\max_j i_j\), by projecting away the unused coordinates. Rectangles determine the probability measure on \(\mathcal{B}^{\otimes k}\), so this joint distribution is \(\nu^{\otimes k}\). In particular, the probability of the cylinder event is \(\prod_{j=1}^k\nu(A_j)\), proving independence for every finite selection of distinct coordinates. Together with the common marginal \(\nu\), this proves that the sequence is IID. \(\square\)

Worked Constructions and Probability Checks

What the Product Construction Gives—and What It Does Not

The construction gives more than a formula for a single pattern. For any finite set of distinct indices and any Borel restrictions on those coordinates, the joint probability is determined by the product of their individual probabilities. This verifies the IID property directly at the level of cylinder events, which generate the product sigma-algebra. It also makes the common law flexible: the coordinate space may be finite or countably infinite, a Borel subset of Euclidean space, or any other standard Borel space.

A common error is to infer independence from identical distributions. The preceding example shows why that inference is invalid: the marginal probabilities alone do not specify joint probabilities. In the product construction, independence is guaranteed by the finite-dimensional product laws, not merely by the fact that every coordinate has law \(\nu\).

Another useful distinction is between a cylinder probability and the probability of an event involving infinitely many coordinates. The defining calculation above applies to finitely many restrictions. Events depending on infinitely many coordinates may also be measurable in the product sigma-algebra, but their probabilities generally require an additional argument; they are not computed by multiplying infinitely many marginal probabilities without justification.