Countable Products of Discrete Probability Spaces

From Finite Laws to an Infinite Product

Finite-dimensional laws specify the probabilities of events involving finitely many coordinates. For an infinite sequence, that information is useful only if it comes from a genuine probability measure: in particular, its probabilities must obey countable additivity. When each coordinate takes values in a countable discrete space, an interval construction gives a direct way to produce such a measure and verify the required additivity.

Let \(S_i\) be a nonempty countable set for each \(i\ge1\), equipped with the sigma-algebra of all its subsets, and set \(S=\prod_{i\ge1}S_i\). For a finite set of indices \(F\subset\mathbb{N}\), write \(S_F=\prod_{i\in F}S_i\), and let \(\pi_F:S\to S_F\) be the coordinate projection. A finite-dimensional law \(\lambda_F\) is a probability measure on \(S_F\). Assume the laws are consistent: whenever \(F\subseteq G\) are finite, the distribution on \(S_F\) obtained by projecting \(\lambda_G\) is \(\lambda_F\). The law on the empty coordinate set gives mass one to its sole element.

Because each \(S_F\) is countable and has all subsets measurable, there is no restriction on the set \(B\). The first step is to assign a candidate probability to a cylinder by setting \(q(\pi_F^{-1}(B))=\lambda_F(B)\). This must be independent of the particular representation of the cylinder; then we must still prove more than finite additivity.

Well-Defined Cylinder Probabilities

Proof. Suppose a cylinder \(A\) has two representations, \(A=\pi_F^{-1}(B)=\pi_G^{-1}(C)\). Put \(H=F\cup G\). Lift \(B\) and \(C\) to subsets of \(S_H\) by taking their inverse images under the finite-coordinate projections. These lifted sets are equal: if they differed at some point of \(S_H\), that point could be extended to an element of \(S\), since every coordinate space is nonempty, and the two cylinder sets would differ there. Consistency says that the \(\lambda_H\)-probability of each lifted set equals the probability assigned by its original finite-dimensional law. Thus \(\lambda_F(B)=\lambda_G(C)\), proving that \(q\) is well-defined. If \(A_1,\ldots,A_r\) are disjoint cylinders, represent them on one common finite coordinate set \(H\), with corresponding disjoint subsets \(B_1,\ldots,B_r\subseteq S_H\). Finite additivity of the probability measure \(\lambda_H\) gives \(q(\bigcup_{j=1}^r A_j)=\sum_{j=1}^r q(A_j)\). The whole space and empty set have probabilities one and zero by the laws on the empty coordinate set. \(\square\)

Finite additivity alone does not establish countable additivity. Rather than assuming that consistency settles this point, we construct a probability measure on \(S\) that agrees with \(q\) on every cylinder. Its countable additivity will then imply countable additivity on the cylinder algebra.

A Nested-Interval Construction

For \(n\ge1\), let \(p_n(s_1,\ldots,s_n)=\lambda_{\{1,\ldots,n\}}(\{(s_1,\ldots,s_n)\})\), the probability of a particular prefix, and put \(p_0=1\). Consistency and countable additivity of the finite-dimensional laws give, for every prefix \((s_1,\ldots,s_{n-1})\),

Choose an enumeration of each countable \(S_n\), using a finite list when \(S_n\) is finite. Begin with \(I_{\varnothing}=[0,1)\). Recursively subdivide each prefix interval \(I_{(s_1,\ldots,s_{n-1})}\) into consecutive half-open intervals \(I_{(s_1,\ldots,s_{n-1},t)}\), in the chosen order, of lengths \(p_n(s_1,\ldots,s_{n-1},t)\). The displayed sum ensures that the child intervals partition the parent interval; intervals of length zero are empty. Each \(x\in[0,1)\) therefore belongs to exactly one interval at each level, and the corresponding prefixes are nested. Define \(T(x)\in S\) to be the resulting sequence.

Proof. First, \(T\) is measurable. For any coordinate \(i\) and state \(t\in S_i\), the set \(\{x:T(x)_i=t\}\) is the union of the level-\(i\) prefix intervals whose last coordinate is \(t\). This is a countable union of intervals, hence Lebesgue measurable in \([0,1)\). More generally, the preimage under \(T\) of a cylinder on coordinates \(F\) is a countable union of level-\(N\) intervals, where \(N=\max F\); it is measurable. Since cylinders generate \(\sigma(\mathcal{A})\), their measurable preimages imply that \(T\) is measurable.

For a prefix \(s=(s_1,\ldots,s_N)\), the construction gives \(T^{-1}(\{x:x_1=s_1,\ldots,x_N=s_N\})=I_s\), whose Lebesgue measure is \(p_N(s)\). Now fix a finite nonempty \(F\), put \(N=\max F\), and take \(B\subseteq S_F\). The inverse image \(T^{-1}(\pi_F^{-1}(B))\) is the disjoint union of the intervals \(I_s\) over those \(s\in S_{\{1,\ldots,N\}}\) whose restriction to \(F\) lies in \(B\). Countable additivity of Lebesgue measure and marginal consistency give

For \(F=\varnothing\), the cylinder is either \(S\) or \(\varnothing\), and its inverse image has measure one or zero, respectively. Define \(P(A)=m(T^{-1}(A))\) for \(A\in\sigma(\mathcal{A})\). This is a probability measure: \(P(S)=m([0,1))=1\), and for pairwise disjoint measurable sets \(A_j\), their preimages are pairwise disjoint, so countable additivity of \(m\) gives \(P(\bigcup_jA_j)=\sum_jP(A_j)\). The cylinder calculation shows \(P=q\) on \(\mathcal{A}\). In particular, if disjoint \(A_j\in\mathcal{A}\) have union \(A\in\mathcal{A}\), then \(q(A)=P(A)=\sum_jP(A_j)=\sum_jq(A_j)\). Thus \(q\) is a premeasure on the cylinder algebra. \(\square\)

This construction proves countable additivity on the algebra by realizing its probabilities inside an already countably additive Lebesgue measure. It does not claim that finite additivity by itself implies countable additivity. The probability \(P\) is defined on the sigma-algebra generated by the cylinders; no completeness assertion is made.

Worked Examples

A Common Pitfall: Zero-Probability Prefixes

A tempting way to make the interval subdivision is to define the next-state probability after a prefix \(s\) as \(p_n(s,t)/p_{n-1}(s)\). This is valid only when \(p_{n-1}(s)>0\). For example, let \(S_1=S_2=\{0,1\}\) and prescribe \(X_1=0\) and \(X_2=1\) with probability one. The prefix \(X_1=1\) has probability zero, as does each of its extensions. The proposed conditional ratio at that prefix is \(0/0\), not a defined probability.

The correction is to assign any probability distribution on \(S_2\) after a zero-probability prefix, for example probability one to state \(0\). The parent interval for the prefix \(X_1=1\) has length zero and is empty, so its subdivision contributes no points of \([0,1)\) and cannot change any cylinder probability. Positive-probability prefixes are subdivided using the ratios; zero-probability prefixes require no ratio. In the present example, the only nonempty level-two interval is the interval for \((0,1)\), namely \([0,1)\), with length one.

The broader lesson is that a construction using conditional probabilities must state what happens on zero-probability conditioning events. Consistency supplies the needed interval lengths, while arbitrary choices at null prefixes do not affect the resulting law.