Almost-Sure Versions of Conditional Prediction Targets

Why Conditional Prediction Targets Have Versions

A conditional prediction target is often written as a function of the covariates: at feature value $x$, it gives a conditional mean or another response summary. But the data distribution may never assign positive probability to some feature values. Changing the target at such values need not change any probability statement about the experiment. To understand what a target says pointwise, we must distinguish the conditional expectation as a random variable from a function representing it.

Let $X$ be a random feature taking values in a measurable space $(\mathcal X,\mathcal A)$, and let $Y$ be an integrable real-valued response, so $E[|Y|]<\infty$. The notation $P_X$ denotes the distribution of $X$: for $B\in\mathcal A$, $P_X(B)=P(X\in B)$. The information contained in observing $X$ is the sigma-algebra $\sigma(X)=\{X^{-1}(B):B\in\mathcal A\}$.

Definition: A conditional expectation of $Y$ given $X$, written $E[Y\mid\sigma(X)]$, is an integrable, $\sigma(X)$-measurable random variable $U$ satisfying $$ \int_A U\,dP=\int_A Y\,dP $$ for every $A\in\sigma(X)$. Two such random variables that agree with probability one are called versions of the same conditional expectation.

The definition specifies averages over events that can be determined from $X$, not a required value at every possible feature value. In particular, “$U$ is a conditional expectation” does not mean that its value at every outcome is uniquely prescribed. The fundamental uniqueness statement is almost-sure uniqueness.

Almost-Sure Uniqueness

Theorem: If $U$ and $V$ are both conditional expectations of the same integrable $Y$ given $\sigma(X)$, then $P(U=V)=1$.

Proof. Set $D=U-V$. It is integrable and $\sigma(X)$-measurable. By the defining integral identity, $\int_A D\,dP=0$ for every $A\in\sigma(X)$. In particular, this holds for $A=\{D>0\}$. If $P(D>0)>0$, then at least one set $\{D\geq 1/k\}$, for a positive integer $k$, has positive probability, because $\{D>0\}=\bigcup_{k=1}^\infty\{D\geq 1/k\}$. Consequently, $$ \int_{\{D>0\}}D\,dP \geq \frac{1}{k}P(D\geq 1/k)>0, $$ contradicting the integral identity. Thus $P(D>0)=0$. Applying the same argument to $-D$ gives $P(D<0)=0$. Therefore $P(U=V)=1$. $\square$

Almost-sure uniqueness is exactly what the definition supports: changes on an event of probability zero leave every integral in the defining identity unchanged. It does not imply that two versions agree at every outcome. The distinction matters especially when we express the conditional expectation as a function of $x$.

From a Conditional Expectation to a Function of Features

A $\sigma(X)$-measurable real random variable can be represented as a measurable function of $X$. Thus a version $U$ of $E[Y\mid\sigma(X)]$ can be written $U=m(X)$ for a measurable $m:\mathcal X\to\mathbb R$. Such a function is often called a regression function when the target is the conditional mean. The function $m$ is not unique at every feature value.

Theorem: Suppose $m_1$ and $m_2$ are measurable real-valued functions on $\mathcal X$ such that $m_1(X)$ and $m_2(X)$ are versions of $E[Y\mid\sigma(X)]$. Then $$ P_X\bigl(\{x:m_1(x)=m_2(x)\}\bigr)=1. $$ Conversely, if two measurable functions agree $P_X$-almost everywhere, their compositions with $X$ agree almost surely.

Proof. Almost-sure uniqueness gives $P(m_1(X)\ne m_2(X))=0$. By the definition of the distribution $P_X$, $$ P_X\bigl(\{x:m_1(x)\ne m_2(x)\}\bigr) =P\bigl(m_1(X)\ne m_2(X)\bigr)=0. $$ This proves the first claim. For the converse, if $N=\{x:m_1(x)\ne m_2(x)\}$ has $P_X(N)=0$, then $P(m_1(X)\ne m_2(X))=P_X(N)=0$. $\square$

This theorem says that a conditional mean function is identified by the data law only $P_X$-almost everywhere. Its values on feature sets of probability zero can be changed without changing the conditional expectation. More generally, the conditional expectation is a random variable defined up to almost-sure equality; its function representation inherits the corresponding ambiguity under the feature distribution.

Worked Example: A Feature Value With Zero Probability

Let $X$ take values $a,b,c$, with $P(X=a)=1/2$, $P(X=b)=1/2$, and $P(X=c)=0$. At $a$, let $Y=0$ or $Y=2$, each with probability $1/4$. At $b$, let $Y=1$ or $Y=5$, each with probability $1/4$. These four outcomes account for the full probability.

For the event $A=\{X=a\}$, the total response integral is $$ \int_A Y\,dP=0\cdot\frac14+2\cdot\frac14=\frac12. $$ Since $P(A)=1/2$, a conditional mean at $a$ must be $1$. Likewise, $$ \int_{\{X=b\}}Y\,dP=1\cdot\frac14+5\cdot\frac14=\frac32, $$ so the conditional mean at $b$ is $3$. The possible events in $\sigma(X)$ are unions of $\{X=a\}$, $\{X=b\}$, and the null event $\{X=c\}$. These calculations show that $m(a)=1$ and $m(b)=3$ satisfy the conditional-expectation identity; for the whole space, both sides equal $2$.

The value $m(c)$ is unrestricted. For example, both $m_1(c)=0$ and $m_2(c)=100$, together with $m_1(a)=m_2(a)=1$ and $m_1(b)=m_2(b)=3$, give the same conditional expectation. Indeed, $P(X=c)=0$, so $m_1(X)=m_2(X)$ almost surely. The function value at $c$ is not determined by this experiment.

Worked Example: Changing a Regression Function at One Point

Let $X$ be uniform on $[0,1]$ and set $Y=X$. Then $X$ itself is a version of $E[Y\mid\sigma(X)]$: it is measurable with respect to $\sigma(X)$, and for every Borel set $B\subseteq[0,1]$, $$ \int_{\{X\in B\}}Y\,dP=\int_{\{X\in B\}}X\,dP. $$ The corresponding function is $m_0(x)=x$.

Now define $m_1(x)=x$ for $x\ne 1/2$, and $m_1(1/2)=7$. This is a measurable function. Since a uniform random variable has $P(X=1/2)=0$, $m_1(X)=X$ almost surely. Equivalently, for every Borel set $B$, $$ \int_B m_1(x)\,dP_X(x) =\int_B x\,dP_X(x), $$ because changing the integrand at the singleton $\{1/2\}$ does not change the integral. Thus $m_1(X)$ is also a version of the conditional expectation.

The two functions make different pointwise claims at $x=1/2$, but the experiment cannot distinguish those claims through its conditional-expectation identity. The fact that $1/2$ lies in the support of the uniform distribution does not change this: its individual probability is still zero.

When Additional Regularity Selects a Version

In applications, one may want a pointwise description of a regression target. Extra assumptions, such as continuity, can select a particular version on part or all of the feature space. Such a selection uses more than the conditional-expectation identity: it uses the additional regularity requirement.

Worked Example: Continuity Determines Values on the Support

Again let $X$ be uniform on $[0,1]$ and $Y=X$. Suppose we require the representing function $m:[0,1]\to\mathbb R$ to be continuous. The function $m_0(x)=x$ is a continuous version. We show that every continuous version must equal it everywhere on $[0,1]$.

Any version satisfies $m(X)=X$ almost surely, so $m(x)=x$ for $P_X$-almost every $x$. Suppose instead that a continuous version differed at some $x_0\in[0,1]$. Then $|m(x_0)-x_0|>0$. By continuity, there is a relative neighborhood of $x_0$ in $[0,1]$ on which $|m(x)-x|$ remains greater than half this positive difference. Every such relative neighborhood has positive uniform probability, contradicting equality almost everywhere. Therefore $m(x)=x$ at every point of $[0,1]$.

Here continuity removes the ambiguity on the support. It is not part of the conditional-expectation definition itself. If the feature space were larger than $[0,1]$, the distribution would still impose no direct constraint at feature values outside its support; any further pointwise claims there would require additional structure or assumptions.

A Common Pitfall and Why the Distinction Matters

A common mistake is to reason, “The conditional mean at $x$ is uniquely defined, so any two valid regression functions must have the same value at every $x$.” The correction is that conditional expectations are unique only almost surely, and representing functions are consequently unique only $P_X$-almost everywhere. At points, or on sets, having zero feature probability, the integral identity cannot distinguish versions.

This distinction matters when a prediction rule is described at a fixed covariate value. A pointwise statement about the target at that value requires a chosen version; a conditional-expectation identity alone may not supply one uniquely. A continuity or other regularity assumption can make a pointwise description more specific, but that assumption should be stated rather than treated as a consequence of almost-sure equality.

Key takeaway: The conditional expectation $E[Y\mid\sigma(X)]$ is an almost-sure equivalence class. A measurable function $m$ representing it is determined only $P_X$-almost everywhere. Pointwise values at probability-zero feature sets require a version choice or additional assumptions.

Check Your Understanding

Use the distinction between almost-sure equality and pointwise equality to answer these questions.

What two properties must a random variable satisfy to be a conditional expectation of $Y$ given $\sigma(X)$?
If two measurable functions agree $P_X$-almost everywhere, what can you conclude about their compositions with $X$?
Why can the value of a representing function at a feature value with probability zero be changed without changing the conditional expectation?
For a uniform $X$ on $[0,1]$, does almost-sure equality of two functions alone imply equality at $x=1/2$? Why or why not?
In the uniform example with $Y=X$, what additional condition made the function value at every point of $[0,1]$ unique?