Measurability and Finite Risk

Why Measurability and Finiteness Are Separate Questions

A population risk is an expectation of a loss evaluated at a prediction. To treat that quantity rigorously, two questions must be answered. First, is the evaluated loss a measurable random variable, so its expectation is defined? Second, is that expectation finite? A measurable nonnegative loss has an extended expectation, which may equal infinity; measurability alone does not guarantee a finite risk.

The distinction matters even when a predictor is simple. Squared error is measurable for ordinary real-valued data and predictions, but its expectation can be infinite if the response has a sufficiently heavy tail. Conversely, if the relevant functions are not measurable, writing an expectation does not by itself make it mathematically well-defined.

Measurability of an Evaluated Loss

Let \((\mathcal X,\mathcal A)\) be the feature space, \((\mathcal Y,\mathcal B)\) the response space, and \((\mathcal U,\mathcal C)\) the prediction space. Assume the observed pair \((X,Y)\) is a random element taking values in \(\mathcal X\times\mathcal Y\), equipped with the product sigma-algebra \(\mathcal A\otimes\mathcal B\). A predictor is a measurable function \(h:\mathcal X\to\mathcal U\). Take a nonnegative loss \(\ell:\mathcal Y\times\mathcal U\to[0,\infty]\) measurable with respect to \(\mathcal B\otimes\mathcal C\) and the Borel sigma-algebra on \([0,\infty]\).

Under these conditions, \(Z_h\) is measurable. To see why, the map \((x,y)\mapsto(y,h(x))\) is measurable from \((\mathcal X\times\mathcal Y,\mathcal A\otimes\mathcal B)\) to \((\mathcal Y\times\mathcal U,\mathcal B\otimes\mathcal C)\). For a measurable rectangle \(B\times C\), its preimage is \((\mathcal X\times B)\cap(h^{-1}(C)\times\mathcal Y)\), which belongs to \(\mathcal A\otimes\mathcal B\). Since measurable rectangles generate \(\mathcal B\otimes\mathcal C\), the map is measurable. Composing it with the measurable loss, and then with the random element \((X,Y)\), gives a measurable random variable \(Z_h\).

The nonnegativity assumption is convenient because expectations of nonnegative measurable variables are defined even when they are infinite. If a loss is allowed to take both positive and negative values, its expectation requires additional care: in particular, it is defined as a finite real number when the loss is integrable, meaning the expectation of its absolute value is finite. The losses considered below are nonnegative.

Integrating a Measurable Function Over One Coordinate

A related measurability fact is useful when studying a family of predictors or decisions. A pointwise expectation over data can itself vary with a parameter. The integral-measurability result below ensures that this parameter-dependent risk is a measurable function, provided the loss is jointly measurable.

Proof. For \(A\in\mathcal T\otimes\mathcal G\), write \(A_t=\{z:(t,z)\in A\}\). First consider the class \(\mathcal D\) of sets \(A\) for which \(t\mapsto\mu(A_t)\) is measurable. Every measurable rectangle \(U\times V\) belongs to \(\mathcal D\), since its section is \(V\) for \(t\in U\) and is empty otherwise. The rectangles form a pi-system that generates \(\mathcal T\otimes\mathcal G\).

The class \(\mathcal D\) is a lambda system. It contains \(T\times\mathcal Z\), whose section measure is the constant \(\mu(\mathcal Z)\). If \(A\in\mathcal D\), then \((A^c)_t=\mathcal Z\setminus A_t\), and finiteness of \(\mu\) gives \(\mu((A^c)_t)=\mu(\mathcal Z)-\mu(A_t)\), a measurable function of \(t\). For pairwise disjoint \(A_1,A_2,\ldots\) in \(\mathcal D\), countable additivity gives \(\mu((\bigcup_j A_j)_t)=\sum_j\mu((A_j)_t)\), also measurable. By the pi-lambda theorem, \(\mathcal D\) contains the entire product sigma-algebra. This establishes the claim for indicators of all product-measurable sets, not just rectangles.

For a nonnegative simple function \(q=\sum_{j=1}^m c_j\mathbf 1_{A_j}\), with \(c_j\ge 0\) and \(A_j\in\mathcal T\otimes\mathcal G\), its integral is \(\sum_{j=1}^m c_j\mu((A_j)_t)\), hence is measurable. For an arbitrary nonnegative measurable \(q\), choose nonnegative simple measurable \(q_n\) increasing pointwise to \(q\). By the monotone convergence theorem, \(\int q_n(t,z)\,\mu(dz)\) increases to \(\int q(t,z)\,\mu(dz)\) for every \(t\). A pointwise limit of measurable functions is measurable. This proves the theorem. \(\square\)

For example, let \(h_t:\mathcal X\to\mathcal U\) be a family of predictors indexed by \(t\in T\), and suppose \((t,x)\mapsto h_t(x)\) is jointly measurable. If \(\ell\) is measurable, then \(q(t,(x,y))=\ell(y,h_t(x))\) is jointly measurable. Applying the theorem with \(\mu\) equal to the probability law of \((X,Y)\) shows that \(t\mapsto R(h_t)\) is measurable. Joint measurability is important here: measurability of each individual \(h_t\) does not, by itself, assert that the whole family varies measurably with \(t\).

When Is Risk Finite?

A direct sufficient condition is boundedness. If \(0\le \ell(y,u)\le M\) for every response and prediction, where \(M<\infty\), then \(0\le Z_h\le M\) and therefore \(0\le R(h)\le M\). This applies to every measurable predictor. The bound must hold on the values that can occur; an almost-sure bound \(Z_h\le M\) is also sufficient.

For unbounded losses, moment conditions provide useful sufficient bounds. With real-valued responses and predictions, the triangle inequality and the inequality \((a-b)^2\le 2a^2+2b^2\) give \(|Y-h(X)|\le |Y|+|h(X)|\) and \((Y-h(X))^2\le 2Y^2+2h(X)^2\). Consequently, \(E|Y|<\infty\) and \(E|h(X)|<\infty\) suffice for finite absolute-error risk, while \(E[Y^2]<\infty\) and \(E[h(X)^2]<\infty\) suffice for finite squared-error risk.

Common Pitfall: Treating Measurability as a Finiteness Guarantee

A common mistake is: “The loss is measurable, so its expected value is finite.” The conclusion does not follow. Measurability ensures that a nonnegative loss has a well-defined extended expectation; it does not rule out an infinite value. In the Pareto example, the squared loss is measurable but has infinite expectation.

The correction is to check the two requirements separately. Establish measurability from the predictor, the data random element, and the loss. Then establish finiteness using a uniform bound or an appropriate moment condition. For unbounded losses, a finite moment of too low an order may not be enough, as the contrast between absolute and squared error shows.