Population Risk as Expected Loss

Why Expected Loss Is a Population Quantity

A prediction function can make different errors on different observations. To summarize its performance, we need to specify not only a penalty for each prediction error but also how frequently the corresponding predictor-response pairs occur in the population of interest. Population risk combines these two ingredients: it is the expected loss when a pair is drawn from the specified data distribution.

This definition makes risk depend on both the prediction rule and the distribution being studied. The same rule can have different risks in populations with different covariate or response distributions. Risk is therefore not an intrinsic score attached to a formula alone; it is performance relative to a particular data law and loss.

Definition: Let $(X,Y)$ have a specified joint distribution $P$, let $h$ be a measurable prediction function, and let $\ell(y,a)$ be a measurable loss for response $y$ and prediction $a$. The population risk of $h$, under $P$ and $\ell$, is $$ R_P(h)=E_P[\ell(Y,h(X))]. $$ For a nonnegative loss, this expectation is defined as a number in $[0,\infty]$. It is a finite risk when the expectation is finite. If the loss can be negative, its expectation is well-defined as a finite value when it is integrable.

The expectation is taken over a fresh pair $(X,Y)$ governed by $P$, not over a particular observed dataset. The subscript $P$ highlights that the population law matters; when the law and loss are fixed throughout a discussion, we may write $R(h)$. For a fixed $h$, the random quantity inside the expectation is $\ell(Y,h(X))$. The risk is its population average.

The definition applies to many prediction tasks. In classification, a loss might be zero-one loss, which is $0$ for a correct class prediction and $1$ otherwise. In regression, a common choice is squared-error loss, $\ell(y,a)=(y-a)^2$. The loss sets the penalty scale: a risk of $0.2$ under zero-one loss is an error probability of $0.2$, whereas a risk of $0.2$ under squared error is an average squared discrepancy, in squared response units.

Risk on a Finite Distribution

When the joint distribution has finitely many possible predictor-response pairs, the expectation is a weighted sum. Each pair contributes its loss multiplied by its probability. This is a direct calculation from the definition, rather than an approximation based on a sample.

Finite-support formula: Suppose $(X,Y)$ takes values $(x_j,y_j)$, $j=1,\ldots,k$, with probabilities $p_j=P(X=x_j,Y=y_j)$, where $p_j\geq 0$ and $\sum_{j=1}^k p_j=1$. Then $$ R_P(h)=\sum_{j=1}^k p_j\,\ell(y_j,h(x_j)). $$

To verify the formula, define $W=\ell(Y,h(X))$. On the event $\{(X,Y)=(x_j,y_j)\}$, $W$ equals the fixed value $\ell(y_j,h(x_j))$. The events for the possible pairs are disjoint and have total probability one. Thus the expectation of $W$ is the sum of each value times the probability of its event, giving the stated expression.

Worked Example: Zero-One Classification Risk

Suppose the population has four possible predictor-response pairs, with probabilities shown below. A classifier predicts class A at $x=0$ and class B at $x=1$. Find its population risk under zero-one loss.

Predictor value	Response	Probability	Prediction	Loss
0	A	0.30	A	0
0	B	0.20	A	1
1	A	0.10	B	1
1	B	0.40	B	0

The first and fourth pairs are classified correctly, so they contribute zero. The second pair contributes $0.20\times1=0.20$, and the third contributes $0.10\times1=0.10$. Therefore, $$ R_P(h)=0.30(0)+0.20(1)+0.10(1)+0.40(0)=0.30. $$ Under zero-one loss, this risk is the probability of an incorrect classification. Here that probability is $0.20+0.10=0.30$.

Worked Example: Squared-Error Risk on Finite Support

Let $P(X=0,Y=1)=0.25$, $P(X=0,Y=3)=0.25$, and $P(X=1,Y=2)=0.50$. Consider $h(x)=1+x$ and squared-error loss. Find the population risk.

At $(X,Y)=(0,1)$, the prediction is $h(0)=1$, giving loss $(1-1)^2=0$. At $(0,3)$, the prediction is still $1$, giving loss $(3-1)^2=4$. At $(1,2)$, the prediction is $h(1)=2$, giving loss $(2-2)^2=0$. Weighting the losses by their probabilities gives $$ R_P(h)=0.25(0)+0.25(4)+0.50(0)=1. $$ The risk is one squared response unit. It is not the square root of this value; taking a square root would define a different summary.

A Squared-Error Decomposition

For regression, population risk can be understood more deeply by separating variation in the response that remains even when $X$ is known from the discrepancy between the predictor and the conditional mean. The following result requires finite second moments; these conditions ensure that the squared-error risks and terms in the identity are finite.

Theorem: Suppose $E[Y^2]<\infty$, $E[h(X)^2]<\infty$, and let $m(X)=E[Y\mid X]$. Then $$ E[(Y-h(X))^2] = E[\operatorname{Var}(Y\mid X)] + E[(m(X)-h(X))^2]. $$ Here $\operatorname{Var}(Y\mid X)$ denotes the conditional variance of $Y$ given $X$.

Proof. Write $Y-h(X)=(Y-m(X))+(m(X)-h(X))$, and square: $$ (Y-h(X))^2=(Y-m(X))^2+2(Y-m(X))(m(X)-h(X))+(m(X)-h(X))^2. $$ All terms are integrable under the stated second-moment assumptions. In particular, conditional expectation gives $E[m(X)^2]\leq E[Y^2]$, and products of square-integrable terms are integrable by the Cauchy–Schwarz inequality. The middle term has expectation zero: $m(X)-h(X)$ is a function of $X$, and $$ E[(Y-m(X))(m(X)-h(X))] =E\!\left[(m(X)-h(X))E[Y-m(X)\mid X]\right]=0, $$ because $E[Y-m(X)\mid X]=0$. Finally, by the definition of conditional variance and the law of iterated expectation, $$ E[(Y-m(X))^2]=E[\operatorname{Var}(Y\mid X)]. $$ Taking expectations in the expansion proves the identity. $\square$

The first term in the decomposition depends on the data distribution, not on $h$. The second term measures the squared discrepancy between the prediction function and the conditional mean. Thus, under squared-error loss, the risk of a predictor is the conditional variation plus its average squared distance from that mean. This identity is about the value of risk; it does not assert that every candidate function has finite risk without the moment assumptions.

Worked Example: Risk in a Simple Regression Model

Let $X$ equal $-1$ or $1$, each with probability $1/2$. Suppose $Y=2X+\varepsilon$, where $\varepsilon$ is independent of $X$, has mean $0$, and has variance $1$. Consider the linear predictor $h(x)=b_0+b_1x$, with $b_0=1$ and $b_1=1$. Calculate its squared-error population risk.

The conditional mean is $m(X)=2X$, and the conditional variance is $1$. For this predictor, $h(X)=1+X$, so $m(X)-h(X)=X-1$. When $X=-1$, the discrepancy is $-2$, whose square is $4$; when $X=1$, it is $0$, whose square is $0$. Hence $$ E[(m(X)-h(X))^2]=\tfrac12(4)+\tfrac12(0)=2. $$ The decomposition gives $$ R_P(h)=E[\operatorname{Var}(Y\mid X)]+E[(m(X)-h(X))^2]=1+2=3. $$ A direct check reaches the same answer. At $X=-1$, the prediction is $0$, and the squared error is $(-2+\varepsilon)^2$, with expectation $4+1=5$. At $X=1$, the prediction is $2$, and the squared error is $\varepsilon^2$, with expectation $1$. Averaging over the two equally likely values of $X$ gives $\tfrac12(5)+\tfrac12(1)=3$.

What Risk Does—and Does Not—Say

A risk is meaningful only after its ingredients are fixed: a population distribution, a prediction function, and a loss. For example, two predictors can be ranked differently under squared error and absolute error because the losses penalize discrepancies differently. Likewise, a predictor’s risk can change if the population distribution changes, even if the prediction function itself stays exactly the same.

The finite-support formula also clarifies a common conceptual distinction. Population risk weights possible outcomes by their probabilities under the specified law. It is not, by definition, a sum over the observations in a particular dataset. Replacing population probabilities with observed frequencies leads to a different quantity, which is the subject of empirical risk.

Common mistake: A student sees that a predictor makes two errors in a dataset of ten cases and concludes that its population risk is $0.2$. That number is an observed proportion of errors, not automatically the population risk. It equals the population risk under zero-one loss only if the relevant population probabilities are represented by those observed proportions—for example, as an exact finite population with those frequencies. In general, population risk is defined by expectation under $P$; an observed dataset alone does not make the population probabilities equal to its frequencies.

For squared error, another mistake is to report only the average discrepancy $E[Y-h(X)]$ as the risk. Positive and negative errors can cancel in that expectation, although both may be large. Squared-error risk instead averages the nonnegative quantities $(Y-h(X))^2$, so cancellation does not occur.

Key takeaway: Population risk is the expected loss of a prediction function under a specified data distribution. On finite support, calculate it by summing probability times loss; in squared-error regression, it can be decomposed into conditional response variation and squared discrepancy from the conditional mean.

Check Your Understanding

Use the definitions and calculations above to answer the following questions.

For a finite distribution, why must the probabilities in the weighted-risk sum add to one?
A classifier is wrong on outcomes with total population probability $0.12$. What is its population risk under zero-one loss?
Suppose $P(X=0,Y=0)=0.5$ and $P(X=1,Y=2)=0.5$. Calculate the squared-error risk of $h(x)=x$.
In the squared-error decomposition, which term depends on the chosen prediction function?
Why does observing a fraction of errors in one dataset not, by itself, determine population risk?

Population Risk as Expected Loss

What You'll Learn