The Kolmogorov Extension Theorem

From Finite-Dimensional Laws to a Product Measure

A family of probability laws for every finite set of coordinates describes all finite-dimensional observations, but it does not by itself constitute a probability measure on the space of infinite sequences. The Kolmogorov extension theorem supplies that measure when the laws are consistent under marginalization and the coordinate spaces are standard Borel. The key point in the existence proof is to show that the induced set function on finite-coordinate events is countably additive whenever the union remains in its domain.

Let \(S_1,S_2,\ldots\) be nonempty standard Borel spaces, meaning measurable spaces isomorphic to Borel subsets of Polish spaces. Give each \(S_j\) a Polish topology that generates its specified Borel sigma-algebra. Write \(S=\prod_{j\ge1}S_j\), with product sigma-algebra \(\mathcal{S}=\bigotimes_{j\ge1}\mathcal{B}_{S_j}\). For a finite index set \(F\subset\mathbb{N}\), let \(\pi_F:S\to S_F:=\prod_{j\in F}S_j\) be the coordinate projection. The empty index set has a one-point product.

Here is the theorem, including the sigma-algebra on which uniqueness is asserted. The resulting probability measure need not be complete; the theorem concerns the product sigma-algebra itself.

The Cylinder Premeasure

A cylinder set is an event determined by finitely many coordinates: it has the form \(\pi_F^{-1}(A)\), with \(F\) finite and \(A\) Borel in \(S_F\). The cylinder sets form an algebra \(\mathcal{A}\): they contain \(S\), and complements and finite unions can be represented using a common finite set of coordinates. They also generate \(\mathcal{S}\), by the definition of the product sigma-algebra.

Consistency defines a set function \(p\) on \(\mathcal{A}\) by \(p(\pi_F^{-1}(A))=\mu_F(A)\). To check that this does not depend on the representation, suppose \(\pi_F^{-1}(A)=\pi_G^{-1}(B)\). Put \(H=F\cup G\). Pulling both descriptions back to \(S_H\) gives the same subset of \(S_H\), so consistency says their \(\mu_H\)-measures, and hence their assigned values, agree. The same reduction to a common finite coordinate set shows that \(p\) is finitely additive and \(p(S)=1\).

Finite additivity is not yet enough for the Caratheodory extension theorem. Since \(p\) is finite, it suffices to prove continuity from above at the empty set: if \(C_1\supseteq C_2\supseteq\cdots\) are in \(\mathcal{A}\) and \(\bigcap_k C_k=\varnothing\), then \(p(C_k)\to0\). We use compact approximation. The standard regularity theorem for finite Borel measures on Polish spaces says that a Borel set can be approximated from inside by compact sets. This regularity result is used here as a standard theorem; its proof is not part of this argument.

Proof of continuity from above. Fix \(\eta>0\). The one-coordinate marginals \(\mu_{\{j\}}\) are probabilities on Polish spaces. By regularity, choose compact \(L_j\subseteq S_j\) such that \(\mu_{\{j\}}(L_j^c)<\eta 2^{-j-2}\). These compact sets can be chosen nonempty, since their measures can be made greater than \(1-\eta 2^{-j-2}>0\) after decreasing \(\eta\) if necessary. The product \(L=\prod_{j\ge1}L_j\) is compact by compactness of countable products.

For each \(k\), represent \(C_k=\pi_{F_k}^{-1}(A_k)\). Regularity of \(\mu_{F_k}\) on the finite product \(S_{F_k}\) gives a compact \(K_k\subseteq A_k\) with \(\mu_{F_k}(A_k\setminus K_k)<\eta 2^{-k-2}\). Put \(D_k=\pi_{F_k}^{-1}(K_k)\). The sets \(D_k\) are closed in \(S\), because \(K_k\) is compact in the Hausdorff space \(S_{F_k}\); also \(D_k\subseteq C_k\).

The compact sets \(L\cap\bigcap_{k=1}^n D_k\) decrease as \(n\) increases. Their intersection is empty: a point in it would belong to every \(C_k\), contrary to the assumed empty intersection. Compactness therefore implies that for some \(n_0\), \(L\cap\bigcap_{k=1}^{n_0}D_k=\varnothing\). Let \(G=\bigcup_{k=1}^{n_0}F_k\), a finite set of coordinates. Any point of \(C_{n_0}\) must either lie in \(C_k\setminus D_k\) for some \(k\le n_0\), or have a coordinate \(j\in G\) outside \(L_j\). Indeed, otherwise its coordinates in \(G\) satisfy every \(D_k\) and lie in their corresponding \(L_j\); choosing arbitrary points of the nonempty \(L_j\) for coordinates outside \(G\) would produce a point of \(L\cap\bigcap_{k\le n_0}D_k\), a contradiction.

All events in that finite cover are cylinders, so finite subadditivity of \(p\) applies. For each \(k\), \(p(C_k\setminus D_k)=\mu_{F_k}(A_k\setminus K_k)<\eta 2^{-k-2}\), and for \(j\in G\), \(p(\{x:x_j\notin L_j\})=\mu_{\{j\}}(L_j^c)<\eta 2^{-j-2}\). Consequently, using \(\sum_{k\ge1}2^{-k-2}=\sum_{j\ge1}2^{-j-2}=1/4\), we obtain \(p(C_{n_0})<\eta/4+\eta/4=\eta/2\). Since \(C_n\subseteq C_{n_0}\) for \(n\ge n_0\), monotonicity gives \(0\le p(C_n)<\eta/2\) for all such \(n\). As \(\eta>0\) was arbitrary, \(p(C_n)\to0\). \(\square\)

For completeness, this continuity property implies countable additivity on the algebra whenever the union belongs to the algebra. If pairwise disjoint \(A_k\in\mathcal{A}\) have union \(A\in\mathcal{A}\), then \(R_n=A\setminus\bigcup_{k=1}^n A_k\) decreases to the empty set. Finite additivity gives \(p(A)=\sum_{k=1}^n p(A_k)+p(R_n)\); taking \(n\to\infty\) and using continuity from above yields \(p(A)=\sum_{k\ge1}p(A_k)\). Thus \(p\) is a premeasure on \(\mathcal{A}\).

Existence and Uniqueness

The Caratheodory extension theorem states that a premeasure on an algebra extends to a measure on the sigma-algebra it generates; if the premeasure is finite, that extension is unique there. It applies to \((\mathcal{A},p)\), since \(p(S)=1\), and \(\sigma(\mathcal{A})=\mathcal{S}\). The resulting measure \(P\) has total mass \(P(S)=p(S)=1\), so it is a probability measure. For every finite \(F\) and Borel \(A\subseteq S_F\), the defining cylinder identity gives \(P(\pi_F^{-1}(A))=\mu_F(A)\). The uniqueness clause applies because \(p\) is finite and \(\mathcal{A}\) generates \(\mathcal{S}\). This proves both existence and uniqueness.

A Pitfall: Consistency Is Not Countable Additivity

Consistency directly guarantees finite additivity on cylinder sets, but it does not itself justify assigning a measure to every countable union of cylinders. Nor is it valid to claim that a decreasing sequence of nonempty cylinders with empty intersection must have an empty member. For example, in \(\mathbb{R}^{\mathbb{N}}\), the cylinders \(C_n=\{x:x_1\ge n\}\) are nonempty and decrease, while their intersection is empty. For the standard normal first-coordinate marginal, \(p(C_n)=\int_n^\infty (2\pi)^{-1/2}e^{-x^2/2}\,dx\) tends to zero, but no \(C_n\) is empty.

The proof must establish the needed continuity from above rather than infer it from finite-dimensional consistency alone. Standard Borel spaces supply regular finite-dimensional laws and compact coordinate approximations; the compactness argument then controls the cylinder probabilities. Once continuity from above is established, the premeasure and extension theorem complete the construction.