Conditional Risk From Population Risk

From Average Loss to Conditional Risk

A prediction rule is judged by its loss across the population it will encounter. Population risk averages that loss over both the predictors and the outcome. But an average alone can hide an important distinction: the outcome distribution, and therefore the expected loss, may vary with the predictor value. Conditional risk makes that variation explicit by asking for the expected loss at a given predictor value.

This distinction is useful even before choosing a prediction rule. It gives a way to describe how costly an action is in different parts of the predictor space, while population risk summarizes those costs using the population’s distribution of predictors. The central identity is an application of conditional expectation, but stating it precisely requires care about the loss, measurability, and conditional distributions.

Definition: Let $X$ be a predictor taking values in a measurable space $\mathcal X$, let $Y$ be an outcome in a measurable space $\mathcal Y$, and let actions lie in a measurable space $\mathcal A$. Let $\ell:\mathcal A\times\mathcal Y\to[0,\infty]$ be a measurable loss. A conditional risk for action $a$ at predictor value $x$ is $$ \mathcal R(a\mid x)=E[\ell(a,Y)\mid X=x], $$ understood through a regular conditional distribution of $Y$ given $X=x$. For a measurable prediction rule $g:\mathcal X\to\mathcal A$, its population risk is $R(g)=E[\ell(g(X),Y)]$.

Conditional Distributions and the Risk Identity

Suppose $\mathcal X$ and $\mathcal Y$ are standard Borel spaces, as are the usual Euclidean spaces and finite sets used in statistical learning. Then there is a regular conditional distribution $K(x,C)$ of $Y$ given $X=x$. For each fixed $x$, $K(x,\cdot)$ is a probability distribution on $\mathcal Y$; for each measurable set $C$, $K(\cdot,C)$ is measurable. It satisfies $K(X,C)=P(Y\in C\mid \sigma(X))$ almost surely.

The notation “given $X=x$” can be delicate when $P(X=x)=0$. A regular conditional distribution supplies a measurable version of the conditional law, but its value at a particular $x$ can be changed on a set of predictor values having probability zero without changing any population expectation. Thus conditional risks are fundamentally determined up to such null sets.

Theorem: Let $K$ be a regular conditional distribution of $Y$ given $X$, let $\ell$ be nonnegative and jointly measurable, and let $g$ be a measurable prediction rule. Then $$ \mathcal R(a\mid x)=\int_{\mathcal Y}\ell(a,y)\,K(x,dy) $$ is a measurable conditional-risk function, and $$ R(g)=E[\mathcal R(g(X)\mid X)] =\int_{\mathcal X}\mathcal R(g(x)\mid x)\,P_X(dx). $$ The identities hold as equalities of nonnegative extended real numbers, so they allow infinite risk. If the loss can take both signs, the same identities hold whenever $E[|\ell(g(X),Y)|]<\infty$; more generally, they hold when the loss is bounded below by an integrable function.

Here is a proof of the integration step underlying the theorem. For a nonnegative measurable function $h(x,y)$, define $q(x)=\int h(x,y)K(x,dy)$. The defining conditional-law property gives $$ E[\mathbf 1_{\{X\in B\}}\mathbf 1_{\{Y\in C\}}] =E[\mathbf 1_{\{X\in B\}}K(X,C)] $$ for measurable $B\subseteq\mathcal X$ and $C\subseteq\mathcal Y$. By linearity, the corresponding identity holds for nonnegative simple functions made from indicators of measurable rectangles. The monotone class theorem extends it to nonnegative product-measurable functions, using monotone convergence to pass from simple functions. Consequently, for every measurable $B$, $$ E[\mathbf 1_{\{X\in B\}}h(X,Y)] =E[\mathbf 1_{\{X\in B\}}q(X)]. $$ This is the defining property of $q(X)$ as a version of $E[h(X,Y)\mid\sigma(X)]$.

For fixed $a$, take $h(x,y)=\ell(a,y)$. This identifies the integral defining $\mathcal R(a\mid X)$ with the conditional expectation of the loss given $X$. For a prediction rule, take $h(x,y)=\ell(g(x),y)$. Joint measurability of $g$ and $\ell$ makes $h$ measurable, and the same argument gives the risk identity. Finally, expectation with respect to $X$ is integration against $P_X$, its distribution. If the loss is signed and integrable, apply the nonnegative argument to its positive and negative parts and subtract. $\square$

Worked Example: Binary Classification With Two Predictor Values

Suppose $X$ takes values $A$ and $B$, with $P(X=A)=0.6$ and $P(X=B)=0.4$. The outcome $Y$ is binary, and $P(Y=1\mid X=A)=0.2$, while $P(Y=1\mid X=B)=0.75$. Use zero-one loss, $\ell(a,y)=\mathbf 1_{\{a\ne y\}}$, and consider the fixed rule $g(A)=0,\ g(B)=1$.

At $A$, the rule predicts 0, so it incurs loss 1 exactly when $Y=1$. Its conditional risk is $$ \mathcal R(g(A)\mid A)=P(Y=1\mid X=A)=0.2. $$ At $B$, the rule predicts 1, so it incurs loss 1 exactly when $Y=0$. Therefore $$ \mathcal R(g(B)\mid B)=P(Y=0\mid X=B)=1-0.75=0.25. $$

Average these conditional risks using the probabilities of the predictor values: $$ R(g)=0.6(0.2)+0.4(0.25)=0.12+0.10=0.22. $$ The same value follows by adding the probabilities of the two kinds of error: $P(X=A,Y=1)=0.6(0.2)=0.12$, and $P(X=B,Y=0)=0.4(0.25)=0.10$. Their sum is $0.22$, as required.

Worked Example: Conditional Risk With a Cost Matrix

Let the outcome be $Y\in\{0,1,2\}$, and let the available actions be $A$ and $B$. Their losses are

Action	Outcome 0	Outcome 1	Outcome 2
A	0	2	5
B	3	1	0

Suppose $P(X=0)=0.25$, $P(X=1)=0.75$, and the conditional outcome probabilities are $P(Y=0,1,2\mid X=0)=(0.5,0.3,0.2)$ and $P(Y=0,1,2\mid X=1)=(0.1,0.4,0.5)$. Consider the rule $g(0)=A,\ g(1)=B$.

At $X=0$, multiply each loss for action $A$ by its conditional outcome probability and add: $$ \mathcal R(A\mid 0)=0.5(0)+0.3(2)+0.2(5)=0+0.6+1=1.6. $$ At $X=1$, do the same for action $B$: $$ \mathcal R(B\mid 1)=0.1(3)+0.4(1)+0.5(0)=0.3+0.4+0=0.7. $$ Thus the population risk of this rule is $$ R(g)=0.25(1.6)+0.75(0.7)=0.4+0.525=0.925. $$ This computation uses the conditional outcome distribution at each predictor value and then weights by how often that predictor value occurs.

Worked Example: A Continuous Outcome and a Threshold Loss

Suppose $P(X=0)=P(X=1)=0.5$. Conditional on $X=0$, let $Y$ be uniform on $[0,2]$; conditional on $X=1$, let $Y$ be uniform on $[1,3]$. The target label is $\mathbf 1_{\{Y>1.5\}}$, and the loss is one if a binary action disagrees with that label and zero otherwise. Use the rule $g(0)=0,\ g(1)=1$.

For $X=0$, predicting 0 is wrong when $Y>1.5$. The conditional density is $1/2$ on an interval of length 2, so $$ \mathcal R(0\mid 0)=P(Y>1.5\mid X=0) =\int_{1.5}^{2}\frac12\,dy =\frac{0.5}{2}=0.25. $$ For $X=1$, predicting 1 is wrong when $Y\le 1.5$. The conditional density is again $1/2$, so $$ \mathcal R(1\mid 1)=P(Y\le 1.5\mid X=1) =\int_{1}^{1.5}\frac12\,dy =\frac{0.5}{2}=0.25. $$ Averaging over the two predictor values gives $R(g)=0.5(0.25)+0.5(0.25)=0.25$. This example uses an integral rather than a finite sum, but the conditional-risk identity is unchanged.

Interpreting the Identity Carefully

The equality between population risk and average conditional risk is a decomposition of an expectation, not a claim that all predictor values contribute equally. The distribution $P_X$ determines their weights. A region of the predictor space with high conditional risk may contribute little to the total if that region is rare; a modest conditional risk across a common region may contribute more.

It is also important to distinguish conditioning on $X$ from conditioning on an event $X=x$ in the elementary ratio formula for conditional probabilities. If $P(X=x)=0$, that ratio is undefined. Conditional distributions provide a principled version of the conditional law, but the value assigned at an individual zero-probability point does not affect the integral for population risk. In continuous-predictor settings, statements about conditional risk are therefore understood almost surely unless a particular version is specified.

Common Mistakes

Taking an unweighted average over predictor values. Averaging the risks at $A$ and $B$ equally is wrong when $P(X=A)\ne P(X=B)$. Weight each conditional risk by the probability of its predictor value, or integrate with respect to $P_X$.
Treating conditional risk as one number for the whole population. It is a function of the action and predictor value. Its value can differ across predictor values because the conditional outcome distribution can differ.
Using $P(Y\mid X=x)$ as a ratio when $P(X=x)=0$. The ratio definition does not apply there. Use a regular conditional distribution, and remember that its values on predictor-null sets do not change population risk.
Confusing a conditional risk with a new objective. Conditional risk describes expected loss locally; population risk is recovered by averaging it over the predictor distribution. Neither identity by itself selects an action or prediction rule.

Key takeaway: For a measurable rule and a nonnegative measurable loss, population risk is the expectation of the loss conditional on the predictors, and hence the integral of conditional risk over the predictor distribution. The averaging weights are part of the identity.

Check Your Understanding

Use the definitions and identity above to answer the following.

Why must conditional risks be weighted by the distribution of $X$ when computing population risk?
In the binary example, what would the population risk be if the rule instead predicted 0 at both predictor values? Show the conditional risks and their weighted average.
What property of the loss allows the population-risk identity to hold even when the risk is infinite?
Why can a conditional risk at one continuous predictor value depend on the chosen version of the conditional distribution without changing population risk?
For a finite outcome space, how is conditional risk computed from the conditional outcome probabilities and the losses?

Conditional Risk From Population Risk

What You'll Learn