Linearity and Bounds for Conditional Expectation

Why Conditional Expectation Has Useful Rules

A conditional expectation is an average calculated with information about a conditioning variable available. Once that information is fixed, quantities determined by it behave like known values, while other quantities remain uncertain. This distinction lets us simplify expressions without needing to calculate a complete conditional distribution every time.

The rules in this tutorial apply to conditioning on a random variable \(X\). They are statements about random variables and hold almost surely: they may fail on a set of conditioning values having probability zero without affecting the result. Unless a rule states otherwise, assume the expectations involved are finite. In particular, a conditional expectation \(E[Y\mid X]\) is defined for an integrable random variable \(Y\), meaning \(E[|Y|]<\infty\).

Definition: \(E[Y\mid X]\) is a random variable determined by \(X\) that represents the conditional mean of \(Y\) given \(X\). It is measurable with respect to the information in \(X\), and for every event \(A\) determined by \(X\), it satisfies \(E[\mathbf{1}_A E[Y\mid X]]=E[\mathbf{1}_A Y]\). The definition determines it up to equality almost surely.

For a discrete \(X\), \(E[Y\mid X]\) can be written as a function of \(X\), with its value at \(X=x\) calculated from the conditional distribution of \(Y\) given \(X=x\), when \(P(X=x)>0\). For continuous \(X\), the same idea applies through the conditional distribution given \(X=x\), for almost every \(x\). Thus notation such as \(E[Y\mid X=x]\) describes a value of the conditional-mean function, whereas \(E[Y\mid X]\) is the resulting random variable.

Linearity and Taking Out What Is Known

Conditional expectation is linear. If \(Y\) and \(Z\) are integrable and \(a,b\) are constants, then

E[aY+bZ\mid X]=aE[Y\mid X]+bE[Z\mid X].

A constant can be treated as a known factor. More generally, if \(g(X)\) is determined by \(X\) and \(g(X)Y\) is integrable, then

E[g(X)Y\mid X]=g(X)E[Y\mid X].

This is called the taking-out-what-is-known rule. The factor must be determined by the conditioning information; an arbitrary random variable cannot be taken outside.

Derivation: To verify linearity, let \(A\) be any event determined by \(X\). By the defining property of conditional expectation,

\begin{aligned} E[\mathbf{1}_A\{aE[Y\mid X]+bE[Z\mid X]\}] &=aE[\mathbf{1}_A Y]+bE[\mathbf{1}_A Z]\\ &=E[\mathbf{1}_A(aY+bZ)]. \end{aligned}

The expression \(aE[Y\mid X]+bE[Z\mid X]\) is determined by \(X\), and its integral over every event determined by \(X\) agrees with that of \(aY+bZ\). The defining property therefore identifies it as \(E[aY+bZ\mid X]\), up to almost-sure equality. For the taking-out rule, first consider a factor \(g(X)\) that takes finitely many values. On each event where \(g(X)\) is constant, that constant can be taken outside the conditional expectation. The general rule follows by extending this argument to measurable factors under the stated integrability condition.

Worked Example: A Linear Combination From Conditional Tables

Let \(X\) be a risk category with \(P(X=A)=0.4\) and \(P(X=B)=0.6\). The conditional outcomes for two measurements \(Y\) and \(Z\) are given below. Find \(E[3Y-2Z+5\mid X]\) in each category.

Given \(X=A\), the outcomes \((Y,Z)\) are \((0,2)\), \((2,1)\), and \((4,0)\), with probabilities \(0.25\), \(0.50\), and \(0.25\). Thus

E[Y\mid X=A]=0(0.25)+2(0.50)+4(0.25)=2, \qquad E[Z\mid X=A]=2(0.25)+1(0.50)+0(0.25)=1.

By linearity and the constant rule,

E[3Y-2Z+5\mid X=A]=3(2)-2(1)+5=9.

Given \(X=B\), the outcomes are \((1,3)\) and \((3,2)\), each with probability \(0.50\). Therefore

E[Y\mid X=B]=1(0.50)+3(0.50)=2, \qquad E[Z\mid X=B]=3(0.50)+2(0.50)=2.5,

E[3Y-2Z+5\mid X=B]=3(2)-2(2.5)+5=6.

The answer is \(9\) in category \(A\) and \(6\) in category \(B\). As a check, direct calculation in category \(A\) gives transformed values \(1,9,17\), with mean \(1(0.25)+9(0.50)+17(0.25)=9\). In category \(B\), the transformed values are \(2\) and \(10\), with mean \(6\). The category probabilities are not used to calculate these conditional means; they would matter if an unconditional mean were requested. Method: calculate the component conditional means, then apply linearity. Mistake avoided: averaging across categories before conditioning.

Nonnegativity, Monotonicity, and Bounds

Conditional expectation also preserves order. These results are useful when a conditional distribution is complicated, but the possible values of a random variable are restricted.

Theorem: If \(Y\geq 0\) almost surely, then \(E[Y\mid X]\geq 0\) almost surely. If \(Y\leq Z\) almost surely and both variables are integrable, then \(E[Y\mid X]\leq E[Z\mid X]\) almost surely. Consequently, if \(L\leq Y\leq U\) almost surely, where \(L\) and \(U\) are integrable and determined by \(X\), then

L\leq E[Y\mid X]\leq U \quad\text{almost surely}.

The last result combines monotonicity with the taking-out rule: \(E[L\mid X]=L\) and \(E[U\mid X]=U\), because both bounds are already determined by \(X\). A useful special case is \(|Y|\leq c\) for a constant \(c\geq0\), which implies \(-c\leq E[Y\mid X]\leq c\). More generally, \(|E[Y\mid X]|\leq E[|Y|\mid X]\), since \(-|Y|\leq Y\leq |Y|\).

Worked Example: Bounding a Conditional Mean From Its Support

Suppose \(X\) has a continuous uniform distribution on \([0,1]\). Given \(X=x\), let \(Y\) have a uniform distribution on \([x,2x+1]\). Find its conditional mean and verify the bounds implied by its support, including at \(x=0.4\).

For a uniform distribution on an interval \([a,b]\), the mean is \((a+b)/2\). Here the conditional lower and upper endpoints are \(x\) and \(2x+1\), respectively, so

E[Y\mid X=x]=\frac{x+(2x+1)}{2}=\frac{3x+1}{2}.

Because every conditional value of \(Y\) lies between \(x\) and \(2x+1\), monotonicity gives

x\leq E[Y\mid X=x]\leq 2x+1.

At \(x=0.4\), the support is \([0.4,1.8]\), and the conditional mean is

E[Y\mid X=0.4]=\frac{3(0.4)+1}{2}=\frac{2.2}{2}=1.1.

The check is \(0.4\leq1.1\leq1.8\), which holds. The formula also satisfies the bounds for every \(x\in[0,1]\): its distance above the lower endpoint is \((x+1)/2\geq0\), and its distance below the upper endpoint is \((x+1)/2\geq0\). Method: use the conditional support to bound the mean, then calculate it from the conditional uniform law. Mistake avoided: using the endpoints of the unconditional range instead of the endpoints conditional on the given \(x\).

Using a Factor Determined by the Conditioning Variable

In expressions involving both \(X\) and another random variable, first identify which quantities are known once \(X\) is given. Any function of \(X\) can then be taken outside, provided the relevant products are integrable. A useful procedure is to expand with linearity, take out each \(X\)-determined factor, and only then insert the conditional means.

Separate the terms.
Use linearity to write the conditional expectation of a sum as a sum of conditional expectations.

Identify what is known given \(X\).
For example, \(X+1\) and \(X^2\) are determined by \(X\), but a separate random variable \(W\) is not necessarily determined by \(X\).

Take out only the known factors.
Replace \(E[g(X)W\mid X]\) by \(g(X)E[W\mid X]\), not by \(g(X)W\).

Substitute the conditional mean and check.
Evaluate at each conditioning value, or compare with a direct calculation from the conditional distribution.

Worked Example: A Category-Dependent Multiplier

Let \(X\) take values \(1\) and \(2\), with probabilities \(0.6\) and \(0.4\). Given \(X=1\), let \(W=2\) with probability \(0.25\) and \(W=6\) with probability \(0.75\). Given \(X=2\), let \(W=1\) and \(W=5\), each with probability \(0.50\). Find \(E[(X+1)W-2X\mid X]\).

First calculate the conditional means of \(W\):

E[W\mid X=1]=2(0.25)+6(0.75)=0.5+4.5=5,

E[W\mid X=2]=1(0.50)+5(0.50)=0.5+2.5=3.

The factor \(X+1\) and the term \(-2X\) are determined by \(X\). Linearity and the taking-out rule therefore give

E[(X+1)W-2X\mid X]=(X+1)E[W\mid X]-2X.

At \(X=1\), this is \(2(5)-2=8\). At \(X=2\), it is \(3(3)-4=5\). A direct check at \(X=1\) gives transformed outcomes \(2(2)-2=2\) and \(2(6)-2=10\), with mean \(2(0.25)+10(0.75)=8\). At \(X=2\), the transformed outcomes are \(3(1)-4=-1\) and \(3(5)-4=11\), with mean \((-1)(0.50)+11(0.50)=5\). Method: take out the multiplier after conditioning on \(X\). Mistake avoided: replacing \(E[W\mid X]\) by \(W\); \(W\) remains random even after \(X\) is known.

Common Exam Traps

Taking out a factor that is not known. In general, \(E[ZW\mid X]\) is not \(Z E[W\mid X]\) if \(Z\) is not determined by \(X\). The rule applies directly to \(g(X)\), not to every random factor.
Using linearity without checking existence. The conditional-expectation rules require the relevant random variables and products to be integrable. A formal algebraic manipulation does not establish that an expectation exists.
Confusing conditional and unconditional bounds. If the support of \(Y\) changes with \(X\), use the bounds that apply at the given conditioning value. A global bound may be valid but unnecessarily weak.
Ignoring almost-sure qualifications. Conditional expectation identities need hold only almost surely. A conditional distribution at a conditioning value with probability zero does not determine a unique value there.

Key takeaway: Conditional expectation is linear, preserves inequalities, and treats functions of the conditioning variable as known. Before simplifying, check integrability and distinguish factors determined by \(X\) from random quantities that remain uncertain given \(X\).

Check Your Understanding

Apply the rules while keeping track of the information specified by the conditioning variable.

Let \(X\) take values \(0\) and \(1\). Given \(X=0\), \(Y\) is \(1\) or \(5\), each with probability \(0.50\). Given \(X=1\), \(Y=2\) with probability \(0.25\) and \(Y=6\) with probability \(0.75\). Find \(E[4Y-3X+2\mid X]\) in each category. (Answer: \(14\) when \(X=0\); \(19\) when \(X=1\))
Given \(X=x\), suppose \(Y\) is uniform on \([2x,4x+2]\), where \(0\leq x\leq2\). Find \(E[Y\mid X=x]\), and evaluate the lower bound, conditional mean, and upper bound at \(x=1.5\). (Answer: \(E[Y\mid X=x]=3x+1\); at \(x=1.5\), \(3\leq5.5\leq8\))
Suppose \(E[A\mid X]=2X+1\) and \(E[B\mid X]=5-X\), with the variables integrable. Find \(E[3A-2B+X\mid X]\) as a function of \(X\), then evaluate it at \(X=2\). (Answer: \(9X-7\); at \(X=2\), \(11\))
Given \(X=a\), suppose \(U\leq V\) almost surely, \(E[U\mid X=a]=3\), and \(E[V\mid X=a]=7\). Given \(X=b\), suppose \(U\leq V\) almost surely, \(E[U\mid X=b]=5\), and \(E[V\mid X=b]=6\). Find \(E[2U+V\mid X]\) in each category and check that monotonicity is consistent with the stated means. (Answer: \(13\) when \(X=a\), and \(16\) when \(X=b\); in both categories \(E[U\mid X]\leq E[V\mid X]\))
Suppose \(X\) takes values \(1\) and \(3\), and \(E[Q\mid X]=X+2\). Find \(E[(X^2-1)Q+4\mid X]\) as a function of \(X\), then give its values at both possible conditioning values. (Answer: \((X^2-1)(X+2)+4\); the values are \(4\) at \(X=1\) and \(44\) at \(X=3\))

What You'll Learn