Conditional Expectation as a Random Variable

From a Conditional Mean to a Random Variable

Conditioning on a random variable does not usually produce one fixed number. The conditional mean can change with the value observed, so it is naturally a function of that value. Once the conditioning variable is random, the function evaluated at it is random too. This distinction is useful in multivariate probability: it lets us describe how the expected value of one quantity varies across the possible values of another.

For example, a hypothetical equipment study might record a machine's operating category \(Y\) and its repair time \(X\). The mean repair time within each category can differ. The list of category-specific means is a function of the category; selecting a machine at random makes the corresponding mean a random variable.

Definition: If \(X\) is integrable and \(Y\) is a random variable, the conditional expectation of \(X\) given \(Y\), written \(E[X\mid Y]\), is a random variable that is a function of \(Y\). We can write it as \(g(Y)\), where \(g(y)\) is the conditional mean of \(X\) at \(Y=y\). For discrete \(Y\), \(g(y)=E[X\mid Y=y]\) whenever \(P(Y=y)>0\). For a continuous \(Y\) with conditional density \(f_{X\mid Y}(x\mid y)\), \(g(y)=\int_{-\infty}^{\infty}x f_{X\mid Y}(x\mid y)\,dx\) wherever that conditional distribution is defined.

The notation \(E[X\mid Y=y]\) refers to a number, for a specified value \(y\). The notation \(E[X\mid Y]\) refers to the random variable \(g(Y)\). For instance, if \(g(0)=2\), \(g(1)=4\), and \(g(2)=7\), then \(E[X\mid Y]\) takes the values 2, 4, and 7 according to which value \(Y\) takes.

When \(Y\) is continuous, conditional distributions and their means are generally specified for almost every \(y\) under the distribution of \(Y\). A value assigned to \(g(y)\) at a single point with probability zero does not change the random variable in the probabilistic sense. Thus conditional expectations that differ only on a set of probability zero are treated as the same random variable.

Calculating and Checking the Random Variable

The calculation has two stages. First find the conditional mean as a function of a fixed value \(y\). Then replace that fixed value by the random variable \(Y\). Afterward, use the distribution of \(Y\) to calculate probabilities or expectations involving \(E[X\mid Y]\).

Find the conditional distribution.
For each possible value \(y\), identify the distribution of \(X\) conditional on \(Y=y\).

Calculate the conditional mean.
Use a probability-weighted sum for a discrete conditional distribution or an integral for a continuous one. Keep \(y\) as a variable.

Substitute the random conditioning variable.
If the conditional mean is \(g(y)\), then the conditional-expectation random variable is \(g(Y)\).

Check its expectation.
Use the distribution of \(Y\) to calculate \(E[g(Y)]\), and compare with a direct calculation of \(E[X]\) when that calculation is available.

Worked Example: A Discrete Conditioning Variable

In a hypothetical service study, \(Y\) is a workload category and \(X\) is the number of hours needed to resolve a case. Suppose \(P(Y=0)=0.2000\), \(P(Y=1)=0.5000\), and \(P(Y=2)=0.3000\). Conditional on \(Y=0\), \(X\) is 1 or 3 hours with probability 0.5000 each. Conditional on \(Y=1\), \(X\) is 2 or 6 hours with probability 0.5000 each. Conditional on \(Y=2\), \(X\) is 4 or 10 hours with probability 0.5000 each. Find \(E[X\mid Y]\) and its expectation, then check against \(E[X]\).

Method. Calculate each conditional mean first, then treat the resulting list of means as a function of \(Y\). No interest or mortality assumption is involved; the model is the stated discrete probability distribution.

For \(Y=0\), the conditional mean is \(E[X\mid Y=0]=(1)(0.5000)+(3)(0.5000)=0.5000+1.5000=2\) hours.

For \(Y=1\), \(E[X\mid Y=1]=(2)(0.5000)+(6)(0.5000)=1.0000+3.0000=4\) hours. For \(Y=2\), \(E[X\mid Y=2]=(4)(0.5000)+(10)(0.5000)=2.0000+5.0000=7\) hours.

Therefore \(g(0)=2\), \(g(1)=4\), and \(g(2)=7\), so the conditional-expectation random variable is \(E[X\mid Y]=g(Y)\). Its values are 2, 4, and 7 hours with probabilities 0.2000, 0.5000, and 0.3000, respectively.

Its expectation is \(E[g(Y)]=(2)(0.2000)+(4)(0.5000)+(7)(0.3000)=0.4000+2.0000+2.1000=4.5000\) hours.

To check directly, the six joint outcomes for \(X\) have probabilities 0.1000, 0.1000, 0.2500, 0.2500, 0.1500, and 0.1500, corresponding respectively to \(X=1,3,2,6,4,10\). Thus \(E[X]=(1)(0.1000)+(3)(0.1000)+(2)(0.2500)+(6)(0.2500)+(4)(0.1500)+(10)(0.1500)=0.1000+0.3000+0.5000+1.5000+0.6000+1.5000=4.5000\) hours. The two calculations agree. The common mistake to avoid is averaging the three conditional means equally: \((2+4+7)/3=4.3333\) hours is wrong because the categories do not have equal probabilities.

Why the Expectation Checks Out

For a discrete conditioning variable, the expectation of the conditional-mean random variable weights each conditional mean by the probability of its conditioning value. In a finite model, the same weights also combine the conditional distributions of \(X\) into its marginal distribution. This is why the two calculations of the mean agree.

E[E[X\mid Y]] =\sum_y P(Y=y)E[X\mid Y=y] =E[X].

Here the sum is over values of \(Y\) with positive probability, and \(X\) must have a finite expectation. In the discrete example, the first sum is \(0.2000(2)+0.5000(4)+0.3000(7)=4.5000\). Expanding each conditional mean into its probability-weighted outcomes gives the marginal weighted sum for \(X\); the direct calculation above confirms the same result. For continuous conditioning, expectations of \(g(Y)\) are calculated by integrating \(g(y)\) against the density of \(Y\), when that density exists. A direct check can also be made from the joint distribution.

Worked Example: Continuous Conditioning

In a hypothetical model, \(Y\), a measured operating level, has a uniform distribution on \((1,3)\). Given \(Y=y\), a positive response \(X\) has a uniform distribution on \((0,y)\). Find \(E[X\mid Y]\) and check its expectation by calculating \(E[X]\) directly from the joint density.

Method. Use the conditional density to find a function of \(y\), substitute \(Y\), and then compare the integral of that function with the integral from the joint density. No interest or mortality assumption is involved.

Since \(Y\) is uniform on \((1,3)\), its density is \(f_Y(y)=1/2\) for \(1<y<3\). Given \(Y=y\), the density of \(X\) is \(f_{X\mid Y}(x\mid y)=1/y\) for \(0<x<y\). Therefore,

E[X\mid Y=y] =\int_0^y x\frac{1}{y}\,dx =\frac{1}{y}\left[\frac{x^2}{2}\right]_0^y =\frac{y}{2}.

Substituting the random variable gives \(E[X\mid Y]=Y/2\). Its expectation is

E[Y/2] =\int_1^3 \frac{y}{2}\frac{1}{2}\,dy =\frac{1}{4}\left[\frac{y^2}{2}\right]_1^3 =\frac{1}{8}(9-1) =1.

For a direct check, the joint density is \(f_{X,Y}(x,y)=f_{X\mid Y}(x\mid y)f_Y(y)=1/(2y)\) when \(1<y<3\) and \(0<x<y\). Hence

E[X] =\int_1^3\int_0^y x\frac{1}{2y}\,dx\,dy =\int_1^3 \frac{y}{4}\,dy =\frac{1}{8}(9-1) =1.

Both checks give 1. The conditional mean is not the constant 1: it is \(Y/2\), which varies with the operating level. Its average happens to be 1 under this particular distribution of \(Y\).

Scaling and Additivity

Conditional expectation preserves linear combinations. If \(X\) and \(W\) are integrable, and \(a\), \(b\), and \(c\) are constants, then

E[aX+bW+c\mid Y] =aE[X\mid Y]+bE[W\mid Y]+c \quad\text{almost surely}.

The reason is that, within each fixed conditioning group, expectation is linear: the mean of \(aX+bW+c\) is \(a\) times the mean of \(X\), plus \(b\) times the mean of \(W\), plus \(c\). Applying this at each value of \(Y\) gives the stated function of \(Y\). This does not require \(X\) and \(W\) to be independent. It does require the expectations involved to exist.

Worked Example: A Conditional Linear Combination

In a hypothetical audit, \(Y\) is a case type with \(P(Y=0)=0.4000\) and \(P(Y=1)=0.6000\). Given \(Y=0\), the pair \((X,W)\) is \((1,0)\) or \((5,2)\), each with probability 0.5000. Given \(Y=1\), the pair is \((4,1)\) or \((10,3)\), each with probability 0.5000. For \(Z=2X-3W+5\), calculate \(E[Z\mid Y]\) both from the conditional outcomes and from scaling and additivity.

Method. Compute the conditional means of \(X\) and \(W\), apply linearity, and verify by evaluating \(Z\) at the outcomes. The common mistake is to assume independence is needed; linearity holds whether or not the variables are independent.

When \(Y=0\), \(E[X\mid Y=0]=(1+5)/2=3\), and \(E[W\mid Y=0]=(0+2)/2=1\). Linearity gives \(E[Z\mid Y=0]=2(3)-3(1)+5=6-3+5=8\). Directly, the two \(Z\) outcomes are \(2(1)-3(0)+5=7\) and \(2(5)-3(2)+5=9\). Their mean is \((7+9)/2=8\).

When \(Y=1\), \(E[X\mid Y=1]=(4+10)/2=7\), and \(E[W\mid Y=1]=(1+3)/2=2\). Thus \(E[Z\mid Y=1]=2(7)-3(2)+5=14-6+5=13\). Direct evaluation gives \(2(4)-3(1)+5=10\) and \(2(10)-3(3)+5=16\), whose mean is \((10+16)/2=13\).

The random variable \(E[Z\mid Y]\) therefore takes the value 8 with probability 0.4000 and 13 with probability 0.6000. Its expectation is \(0.4000(8)+0.6000(13)=3.2000+7.8000=11.0000\). As a further check, \(E[X]=0.4000(3)+0.6000(7)=5.4000\) and \(E[W]=0.4000(1)+0.6000(2)=1.6000\). Therefore \(2E[X]-3E[W]+5=2(5.4000)-3(1.6000)+5=10.8000-4.8000+5=11.0000\), as required.

Common Interpretations and Traps

The central interpretation is that \(E[X\mid Y]\) is a new random variable summarizing the conditional mean of \(X\) at the observed value of \(Y\). Its distribution is determined by two ingredients: the function \(g\), and the distribution of \(Y\). A list of conditional means alone does not tell you the probabilities with which the random variable takes those values.

Do not treat the conditional mean as a single overall mean. \(E[X\mid Y]\) can vary with \(Y\); \(E[X]\) is one number.
Do not weight categories equally unless they are equally likely. The expectation of \(g(Y)\) uses the probabilities of the corresponding values of \(Y\).
Do not confuse conditioning on a value with conditioning on a random variable. \(E[X\mid Y=y]\) is a number; \(E[X\mid Y]=g(Y)\) is random.
Do not require independence for linearity. Scaling and additivity of conditional expectation apply to dependent variables as well, provided the required expectations exist.

Key takeaway: First calculate \(g(y)=E[X\mid Y=y]\); then write \(E[X\mid Y]=g(Y)\). The probabilities or density of \(Y\) determine how this random variable behaves, and its expectation provides a useful check against \(E[X]\).

Check Your Understanding

For each problem, identify the conditional-expectation random variable before calculating its expectation.

A hypothetical rating \(Y\) equals 0 with probability 0.2500 and 1 with probability 0.7500. The conditional means of a cost \(X\) are \(E[X\mid Y=0]=6\) and \(E[X\mid Y=1]=2\). Find \(E[X\mid Y]\) and its expectation. (Answer: \(E[X\mid Y]=6\) when \(Y=0\), 2 when \(Y=1\); \(E[E[X\mid Y]]=3.0000\).)
Let \(Y\) be uniform on \((0,4)\), and suppose \(X\mid Y=y\) is uniform on \((y,y+2)\). Find \(E[X\mid Y]\) and its expectation. (Answer: \(E[X\mid Y]=Y+1\); \(E[E[X\mid Y]]=3\).)
Let \(Y\) take values 0, 1, and 2 with probabilities 0.2000, 0.3000, and 0.5000. Suppose \(E[A\mid Y=y]\) is respectively 2, 5, and 4, while \(E[B\mid Y=y]\) is respectively 1, 2, and 3. Find \(E[3A-2B+1\mid Y]\) at each value of \(Y\), then its expectation. (Answer: the conditional values are 5, 12, and 7; the expectation is 8.1000.)
Let \(Y\) be uniform on \((2,4)\), and let \(X\mid Y=y\) be uniform on \((0,2y)\). Find \(E[X\mid Y]\) and its expectation. (Answer: \(E[X\mid Y]=Y\); \(E[E[X\mid Y]]=3\).)
A discrete \(Y\) has values 0 and 1 with probabilities 0.3000 and 0.7000. Given \(Y=0\), \(X\) is 2 or 8 with equal probabilities; given \(Y=1\), \(X\) is 1 or 5 with equal probabilities. Find the values and distribution of \(E[X\mid Y]\), then compare its expectation with \(E[X]\). (Answer: \(E[X\mid Y]\) is 5 with probability 0.3000 and 3 with probability 0.7000; both expectations are 3.6000.)

What You'll Learn