Product Sigma-Algebras for Observed Datasets

Why Observed Data Need a Measurable Space

A statistical dataset is an ordered collection of observations, and each observation may itself contain several components. To treat the dataset as a random object, we need to specify which subsets of its possible values count as measurable events. Product sigma-algebras provide this structure: they let us describe events using measurable conditions on individual coordinates, while also accommodating events generated by combining those conditions.

Suppose the feature takes values in a measurable space \((\mathcal X,\mathcal A)\), and the response takes values in a measurable space \((\mathcal Y,\mathcal B)\). The product construction depends on these spaces, not on a probability law or an independence assumption. Once the measurable space for one observation has been defined, the same construction can be applied to an entire dataset.

One Feature-Response Pair

Definition: The product sigma-algebra \(\mathcal A\otimes\mathcal B\) on \(\mathcal X\times\mathcal Y\) is the smallest sigma-algebra containing every measurable rectangle \(A\times B\), where \(A\in\mathcal A\) and \(B\in\mathcal B\). Equivalently, it is the sigma-algebra generated by the collection of these rectangles. The measurable space for one feature-response pair is \((\mathcal X\times\mathcal Y,\mathcal A\otimes\mathcal B)\).

A rectangle \(A\times B\) describes a joint condition: the feature lies in \(A\) and the response lies in \(B\). The product sigma-algebra also contains complements and countable unions of such rectangles, as required of a sigma-algebra. It is therefore generally larger than the collection of rectangles itself.

The coordinate projections extract the feature or response from a pair:

\pi_{\mathcal X}(x,y)=x, \qquad \pi_{\mathcal Y}(x,y)=y.

Their measurability follows directly from the construction. For any \(A\in\mathcal A\) and \(B\in\mathcal B\),

\pi_{\mathcal X}^{-1}(A)=A\times\mathcal Y\in\mathcal A\otimes\mathcal B, \qquad \pi_{\mathcal Y}^{-1}(B)=\mathcal X\times B\in\mathcal A\otimes\mathcal B.

A map into a measurable space is measurable when the preimage of every measurable set is measurable. Thus these preimage calculations verify that both projections are measurable; they do not require a probability distribution on the pair.

Worked Example: A Pair of Finite-Valued Variables

Let \(\mathcal X=\{a,b\}\) and \(\mathcal Y=\{0,1\}\), each with its power-set sigma-algebra. The power set of a set is the collection of all its subsets. The four singleton rectangles are

\{a\}\times\{0\},\quad \{a\}\times\{1\},\quad \{b\}\times\{0\},\quad \{b\}\times\{1\}.

These are all the singleton subsets of \(\mathcal X\times\mathcal Y\). Every subset of this four-element product is a union of singleton subsets, so \(\mathcal A\otimes\mathcal B\) is the power set of \(\mathcal X\times\mathcal Y\).

For the feature projection, the preimage of \(\{a\}\) is \(\{a\}\times\{0,1\}\), a union of two measurable rectangles. For the response projection, the preimage of \(\{1\}\) is \(\{a,b\}\times\{1\}\). Both preimages belong to the product sigma-algebra, as required.

The Measurable Space for an Ordered Dataset

Fix a positive integer \(n\). An ordered dataset of \(n\) feature-response pairs is an element of \((\mathcal X\times\mathcal Y)^n\). Its coordinates are ordered: the first observed pair, second observed pair, and so on have designated positions. Write \(\mathcal Z=\mathcal X\times\mathcal Y\) and \(\mathcal C=\mathcal A\otimes\mathcal B\). The measurable space for the dataset is

\bigl(\mathcal Z^n,\mathcal C^{\otimes n}\bigr) = \bigl((\mathcal X\times\mathcal Y)^n, (\mathcal A\otimes\mathcal B)^{\otimes n}\bigr).

Here \(\mathcal C^{\otimes n}\) is the sigma-algebra generated by sets of the form \(C_1\times\cdots\times C_n\), with each \(C_i\in\mathcal C\). Such a set imposes a measurable condition on each observed pair. A useful equivalent description uses feature and response coordinates separately: the dataset can be written as \((x_1,y_1,\ldots,x_n,y_n)\), and its sigma-algebra is generated by measurable rectangles in these \(2n\) coordinates.

For each position \(i\), define the pair projection \(\pi_i\) by \(\pi_i(z_1,\ldots,z_n)=z_i\). Within that pair, define \(\pi_{i,\mathcal X}(z_1,\ldots,z_n)\) to be its feature and \(\pi_{i,\mathcal Y}(z_1,\ldots,z_n)\) to be its response. These maps are measurable because their preimages of measurable sets are coordinate cylinders:

\pi_i^{-1}(C) = \mathcal Z^{i-1}\times C\times\mathcal Z^{n-i} \in\mathcal C^{\otimes n}, \qquad C\in\mathcal C.

For example, the event that the first feature belongs to \(A\in\mathcal A\) and the last response belongs to \(B\in\mathcal B\) is a measurable subset of the dataset space. It is the intersection of the preimage of \(A\times\mathcal Y\) under \(\pi_{1}\) and the preimage of \(\mathcal X\times B\) under \(\pi_n\). This description applies whether or not the observations are independent or identically distributed.

Theorem: Let \((\Omega,\mathcal F)\) be a measurable space, and let \(T_i:\Omega\to E_i\) be maps into measurable spaces \((E_i,\mathcal E_i)\), for \(i=1,\ldots,n\). The tuple map \(T=(T_1,\ldots,T_n)\) into \(\prod_{i=1}^n E_i\), equipped with \(\bigotimes_{i=1}^n\mathcal E_i\), is measurable if and only if every coordinate map \(T_i\) is measurable.

Proof. First suppose \(T\) is measurable. The projection \(p_i:\prod_{j=1}^n E_j\to E_i\) is measurable: for any \(D\in\mathcal E_i\), its preimage is the measurable cylinder \(E_1\times\cdots\times D\times\cdots\times E_n\). Since \(T_i=p_i\circ T\), each \(T_i\) is measurable.

Conversely, suppose every \(T_i\) is measurable. The product sigma-algebra is generated by measurable rectangles \(D_1\times\cdots\times D_n\), with \(D_i\in\mathcal E_i\). For any such rectangle,

T^{-1}(D_1\times\cdots\times D_n) = \bigcap_{i=1}^n T_i^{-1}(D_i).

Each set on the right belongs to \(\mathcal F\), and a finite intersection of members of a sigma-algebra belongs to that sigma-algebra. Thus the preimage of every generating rectangle is in \(\mathcal F\). The collection of subsets of the product whose preimages belong to \(\mathcal F\) is itself a sigma-algebra. Since it contains the generators, it contains the whole product sigma-algebra. Hence \(T\) is measurable. \(\square\)

In particular, a random dataset is a measurable map into the dataset space precisely when each of its feature and response coordinates is a measurable random variable. This criterion is about measurability alone; it makes no claim about the joint distribution of those coordinates.

Worked Example: Two Observations with Real Features and Responses

Take \(\mathcal X=\mathcal Y=\mathbb R\), with each space equipped with its Borel sigma-algebra \(\mathcal B(\mathbb R)\). A two-observation dataset has space \((\mathbb R^2)^2\), equipped with \((\mathcal B(\mathbb R)\otimes\mathcal B(\mathbb R))^{\otimes 2}\). Consider the event that the first feature is in \([0,1]\) and the second response is positive.

Writing a dataset as \((x_1,y_1,x_2,y_2)\), the event is

\bigl([0,1]\times\mathbb R\bigr) \times \bigl(\mathbb R\times(0,\infty)\bigr).

The set \([0,1]\) is Borel, as is \((0,\infty)\), and \(\mathbb R\) is measurable in its own Borel sigma-algebra. Each pair-coordinate condition is therefore measurable, and their product is a generator of the dataset sigma-algebra. The event belongs to that sigma-algebra.

If a random dataset is given by measurable real-valued maps \(X_1,Y_1,X_2,Y_2\) on some \((\Omega,\mathcal F)\), the preimage of this event is \(X_1^{-1}([0,1])\cap Y_2^{-1}((0,\infty))\), which is in \(\mathcal F\). This is the tuple-map criterion applied to the four scalar coordinates.

A Familiar Continuous Example

For real-valued features and responses, the product sigma-algebra \(\mathcal B(\mathbb R)\otimes\mathcal B(\mathbb R)\) agrees with the Borel sigma-algebra \(\mathcal B(\mathbb R^2)\). This equality is useful because it connects the abstract product construction with the familiar measurable sets in the plane.

Result: \(\mathcal B(\mathbb R)\otimes\mathcal B(\mathbb R)=\mathcal B(\mathbb R^2)\).

Proof. For any \(A,B\in\mathcal B(\mathbb R)\), the rectangle \(A\times B\) is the intersection of the preimages of \(A\) and \(B\) under the two continuous coordinate projections from \(\mathbb R^2\). Continuous maps are Borel measurable, so these preimages, and hence their intersection, are Borel in \(\mathbb R^2\). Therefore every generator of \(\mathcal B(\mathbb R)\otimes\mathcal B(\mathbb R)\) is Borel, which gives \(\mathcal B(\mathbb R)\otimes\mathcal B(\mathbb R)\subseteq\mathcal B(\mathbb R^2)\).

For the reverse inclusion, every open subset of \(\mathbb R^2\) is a union of open rectangles with rational endpoints that are contained in that set. There are only countably many such rational rectangles, so this union is countable. Each rectangle is a product of Borel subsets of \(\mathbb R\), and thus belongs to \(\mathcal B(\mathbb R)\otimes\mathcal B(\mathbb R)\). The product sigma-algebra contains every open subset of \(\mathbb R^2\), and therefore contains the Borel sigma-algebra generated by those open sets. The two sigma-algebras are equal. \(\square\)

Worked Example: A Measurable Event in the Plane

Let \(X\) and \(Y\) be the feature and response coordinates in \(\mathbb R^2\). Consider the event \(X+Y>1\). The set of pairs satisfying this condition is open in \(\mathbb R^2\), because the map \((x,y)\mapsto x+y\) is continuous and \((1,\infty)\) is open. By the equality just proved, this set belongs to \(\mathcal B(\mathbb R)\otimes\mathcal B(\mathbb R)\).

This event need not itself be a single rectangle: whether \(x+y>1\) holds depends jointly on both coordinates. Its measurability follows because the product sigma-algebra contains countable operations on rectangles, not because every measurable event must be a rectangle.

Common Pitfall: Product Does Not Mean Independent

A common mistake is to read “product sigma-algebra” as a statement that the observations, or the feature and response, are independent. The product sigma-algebra specifies which subsets of the product space are measurable. Independence is a property of a probability law on that space. The measurable space can be used with many different probability laws, including laws with dependent coordinates.

A second mistake is to assume that the product sigma-algebra always consists of every subset of the product. This holds in the finite example because every subset is a finite union of singleton rectangles. For uncountable spaces, a product sigma-algebra is generated using only countable sigma-algebra operations; it need not be the power set. The definition, rather than an assumption that all subsets are measurable, determines the event space.

Key takeaway: Build the pair space as \((\mathcal X\times\mathcal Y,\mathcal A\otimes\mathcal B)\), then build a fixed-size dataset by taking the finite product of pair spaces. Coordinate projections are measurable, and a tuple-valued data map is measurable exactly when each coordinate map is measurable.

Check Your Understanding

Use the product-space definitions and the coordinate criterion to answer the following.

Write the preimage of \(A\in\mathcal A\) under the feature projection from \(\mathcal X\times\mathcal Y\).
For a dataset of size \(n\), describe the measurable cylinder expressing that observation \(i\) belongs to \(C\in\mathcal A\otimes\mathcal B\).
Why does measurability of each coordinate imply measurability of a tuple-valued map?
Does equipping a dataset space with a product sigma-algebra assert that its observations are independent? Explain.
In the finite worked example, why is the product sigma-algebra the power set of the pair space?