Linearity of Conditional Expectation

Linearity from the Defining Identity

Conditional expectation is linear for the same structural reason as ordinary expectation: its defining integrals are linear. There is one important qualification. A conditional expectation is determined only up to almost-sure equality, so linearity means equality almost surely, not necessarily equality at every point for arbitrary choices of versions. No independence assumption is needed.

Throughout, $(\Omega,\mathcal{F},\mathbb{P})$ is a probability space, with $\mathbb{P}(\Omega)=1$, and $\mathcal{G}$ is a sub-sigma-algebra of $\mathcal{F}$. The probability space need not be complete. Random variables in the theorem below are real-valued and integrable, meaning $\mathbb{E}[|X|]<\infty$. Conditional expectations are real-valued, integrable, and $\mathcal{G}$-measurable.

Definition: For an integrable real-valued random variable $X$, a conditional expectation of $X$ given $\mathcal{G}$ is an integrable, $\mathcal{G}$-measurable random variable $U$ such that $$ \int_A U\,d\mathbb{P}=\int_A X\,d\mathbb{P}\qquad\text{for every }A\in\mathcal{G}. $$ It is denoted by $\mathbb{E}[X\mid\mathcal{G}]$. Such a variable exists, and any two variables satisfying these conditions are equal $\mathbb{P}$-almost surely. Existence and almost-sure uniqueness are the standard results established in the earlier material on constructing and characterizing conditional expectation.

The integral condition is imposed on every event in $\mathcal{G}$, not just on $\Omega$. Testing $A=\Omega$ alone only checks that the unconditional expectations agree; it does not check that the conditional expectation captures the information represented by $\mathcal{G}$.

The Linearity Theorem

Theorem: Let $X,Y$ be integrable, real-valued random variables on $(\Omega,\mathcal{F},\mathbb{P})$, and let $a,b\in\mathbb{R}$ be fixed constants. For any sub-sigma-algebra $\mathcal{G}\subseteq\mathcal{F}$, $$ \mathbb{E}[aX+bY\mid\mathcal{G}] =a\,\mathbb{E}[X\mid\mathcal{G}] +b\,\mathbb{E}[Y\mid\mathcal{G}] \qquad \mathbb{P}\text{-almost surely}. $$

Proof. Choose versions $U=\mathbb{E}[X\mid\mathcal{G}]$ and $V=\mathbb{E}[Y\mid\mathcal{G}]$. Since $X,Y$ are integrable, $|aX+bY|\le |a||X|+|b||Y|$, so $aX+bY$ is integrable. The random variable $aU+bV$ is $\mathcal{G}$-measurable because $U,V$ are, and it is integrable because $|aU+bV|\le |a||U|+|b||V|$. For every $A\in\mathcal{G}$, linearity of the integral for integrable real-valued functions and the defining identities for $U,V$ give $$ \int_A(aU+bV)\,d\mathbb{P} =a\int_AU\,d\mathbb{P}+b\int_AV\,d\mathbb{P} =a\int_AX\,d\mathbb{P}+b\int_AY\,d\mathbb{P} =\int_A(aX+bY)\,d\mathbb{P}. $$ Thus $aU+bV$ satisfies all the defining conditions for a conditional expectation of $aX+bY$ given $\mathcal{G}$. Almost-sure uniqueness implies the asserted equality $\mathbb{P}$-almost surely. This argument includes $a=0$, $b=0$, and negative coefficients. $\square$

Corollary: If $X_1,\ldots,X_n$ are integrable, real-valued random variables on $(\Omega,\mathcal{F},\mathbb{P})$, and $a_1,\ldots,a_n\in\mathbb{R}$ are fixed constants, then $$ \mathbb{E}\left[\sum_{j=1}^n a_jX_j\,\middle|\,\mathcal{G}\right] =\sum_{j=1}^n a_j\mathbb{E}[X_j\mid\mathcal{G}] \qquad \mathbb{P}\text{-almost surely}. $$

Proof. The sum $\sum_{j=1}^n a_jX_j$ is integrable, since its absolute value is bounded by $\sum_{j=1}^n|a_j||X_j|$, whose expectation is finite. For each $j$, choose a version $U_j=\mathbb{E}[X_j\mid\mathcal{G}]$. The finite sum $\sum_{j=1}^n a_jU_j$ is integrable and $\mathcal{G}$-measurable. For each $A\in\mathcal{G}$, finite linearity of the integral and the defining identity for each $U_j$ give $$ \int_A\sum_{j=1}^n a_jU_j\,d\mathbb{P} =\sum_{j=1}^n a_j\int_AU_j\,d\mathbb{P} =\sum_{j=1}^n a_j\int_AX_j\,d\mathbb{P} =\int_A\sum_{j=1}^n a_jX_j\,d\mathbb{P}. $$ The sum of the chosen versions therefore satisfies the definition, and almost-sure uniqueness proves the claim. $\square$

Computations on a Finite Probability Space

Worked Example: Adding and Subtracting on a Partition

Let $\Omega=\{\omega_1,\omega_2,\omega_3,\omega_4\}$, with all subsets measurable and $\mathbb{P}(\{\omega_i\})=1/4$ for each $i$. This is a finite, complete probability space. Let $\mathcal{G}$ be generated by the partition $A=\{\omega_1,\omega_2\}$, $B=\{\omega_3,\omega_4\}$. Define $X=(1,3,2,6)$ and $Y=(5,-1,4,0)$, listing values in the order $\omega_1,\ldots,\omega_4$. Compute the conditional expectation of $X+Y$ and of $2X-3Y$ given $\mathcal{G}$.

On each partition set, a $\mathcal{G}$-measurable conditional expectation is constant. On $A$, which has probability $2/4=1/2$, the averages are $$ \mathbb{E}[X\mid\mathcal{G}]\big|_A=\frac{1+3}{2}=2, \qquad \mathbb{E}[Y\mid\mathcal{G}]\big|_A=\frac{5+(-1)}{2}=2. $$ On $B$, also of probability $1/2$, they are $$ \mathbb{E}[X\mid\mathcal{G}]\big|_B=\frac{2+6}{2}=4, \qquad \mathbb{E}[Y\mid\mathcal{G}]\big|_B=\frac{4+0}{2}=2. $$ Thus linearity gives $4$ on $A$ and $6$ on $B$ for $X+Y$. Directly, $X+Y$ has values $6,2,6,6$, whose averages on $A,B$ are $(6+2)/2=4$ and $(6+6)/2=6$.

For $2X-3Y$, linearity gives $2(2)-3(2)=-2$ on $A$, and $2(4)-3(2)=2$ on $B$. Direct substitution gives the four values $-13,9,-8,12$. Their averages are $(-13+9)/2=-2$ on $A$ and $(-8+12)/2=2$ on $B$. As a check of the defining identities, the integral of $2X-3Y$ over $A$ is $(-13+9)/4=-1$, and the integral of the proposed conditional expectation there is $(-2)(1/2)=-1$. Over $B$, both integrals are $1$. These are all the nontrivial atom checks for $\mathcal{G}$; the identities for $\varnothing$ and $\Omega$ follow by additivity.

Worked Example: Unequal Atom Probabilities

Take $\Omega=\{\omega_1,\omega_2,\omega_3,\omega_4\}$ with all subsets measurable and point probabilities $1/10,2/10,3/10,4/10$, respectively. This finite probability space is complete. Let $\mathcal{G}$ be generated by $A=\{\omega_1,\omega_2\}$ and $B=\{\omega_3,\omega_4\}$. Let $X=(4,-1,2,8)$ and $Y=(3,5,-2,1)$. Find the conditional expectations of $X,Y$, and $X+Y$.

The probabilities of the two atoms are $\mathbb{P}(A)=1/10+2/10=3/10$ and $\mathbb{P}(B)=3/10+4/10=7/10$. The weighted integral of $X$ over $A$ is $4(1/10)+(-1)(2/10)=2/10$, so its conditional average there is $(2/10)/(3/10)=2/3$. Over $B$, its integral is $2(3/10)+8(4/10)=38/10$, giving $(38/10)/(7/10)=38/7$. For $Y$, the corresponding integrals are $3(1/10)+5(2/10)=13/10$ on $A$, and $(-2)(3/10)+1(4/10)=-2/10$ on $B$. Hence its conditional averages are $13/3$ and $-2/7$.

Adding these conditional expectations gives $2/3+13/3=5$ on $A$ and $38/7-2/7=36/7$ on $B$. Directly, $X+Y=(7,4,0,9)$. Its integral over $A$ is $7(1/10)+4(2/10)=15/10$, and division by $3/10$ gives $5$. Its integral over $B$ is $0(3/10)+9(4/10)=36/10$, and division by $7/10$ gives $36/7$. As an additional check, the total integrals are $\mathbb{E}[X]=2/10+38/10=4$ and $\mathbb{E}[Y]=13/10-2/10=11/10$. The proposed conditional expectations have total integrals $5(3/10)+(36/7)(7/10)=15/10+36/10=51/10$, which equals $4+11/10=51/10=\mathbb{E}[X+Y]$.

A Trivial Conditioning Sigma-Algebra

Worked Example: Conditioning on No Nontrivial Events

Let $\Omega=[0,1]$, $\mathcal{F}=\mathcal{L}$, and $\mathbb{P}=m$ restricted to $[0,1]$. This is a complete probability space. Let $\mathcal{G}=\{\varnothing,\Omega\}$, and set $X(x)=x$, $Y(x)=x^2$. Both are real-valued and integrable. Every $\mathcal{G}$-measurable real-valued random variable is constant, so the defining identity for $A=\Omega$ says that the conditional expectation must equal the ordinary expectation. (The identity for $A=\varnothing$ is automatic.)

Using the antiderivatives, $\mathbb{E}[X\mid\mathcal{G}]=\int_0^1x\,dx=1/2$ and $\mathbb{E}[Y\mid\mathcal{G}]=\int_0^1x^2\,dx=1/3$, as constant random variables. Therefore $3\mathbb{E}[X\mid\mathcal{G}]-2\mathbb{E}[Y\mid\mathcal{G}]=3/2-2/3=5/6$. Directly, $\int_0^1(3x-2x^2)\,dx=3/2-2/3=5/6$. The direct computation verifies the defining identity on $\Omega$, and on $\varnothing$ both integrals are zero.

Versions, Hypotheses, and a Common Mistake

The theorem’s equality is almost sure because conditional expectations are specified only up to almost-sure equality. If $U$ is a version of $\mathbb{E}[X\mid\mathcal{G}]$, changing $U$ on a $\mathbb{P}$-null set may give another representative of the same almost-sure class. Thus the formula asserts that the two sides agree outside a null set; it need not hold at every point for every independently chosen pair of versions. In a probability space that is not complete, an arbitrary modification on a null set might not even be $\mathcal{G}$-measurable. The theorem concerns versions that satisfy the defining measurability condition.

Common mistake: keeping the original random variables instead of conditioning them. In the first worked example, at $\omega_1$ the value of $2X-3Y$ is $-13$. But the conditional expectation given $\mathcal{G}$ is $-2$ on all of $A$. Writing $\mathbb{E}[2X-3Y\mid\mathcal{G}]=2X-3Y$ is wrong because $2X-3Y$ is not constant on the atoms of $\mathcal{G}$, and therefore is not $\mathcal{G}$-measurable. The correct computation combines the conditional expectations: $2\mathbb{E}[X\mid\mathcal{G}]-3\mathbb{E}[Y\mid\mathcal{G}]$, which equals $-2$ on $A$ and $2$ on $B$.

Integrability is also essential to this statement: the definition used here applies to integrable random variables. For example, on $[0,1]$ with Lebesgue probability measure, define $X(x)=1/x$ for $x>0$ and $X(0)=0$. This is a real-valued measurable function, but $\int_0^1|X|\,dm=\infty$. For the trivial sigma-algebra, a real-valued integrable conditional expectation $U$ would have to satisfy $\int_\Omega U\,d\mathbb{P}=\int_\Omega X\,d\mathbb{P}=\infty$, which is impossible for integrable $U$. Thus the integrable conditional expectation in this theorem is not defined for this $X$.

The constants in the theorem are fixed real numbers. The result does not, by itself, assert a formula for multiplying by a random variable, even if that variable is measurable with respect to $\mathcal{G}$; that requires a separate result with its own hypotheses. For sums and fixed scalar multiples, however, there is no need to assume independence, and negative coefficients cause no difficulty.

Check Your Understanding

Use the defining integral identities, and keep track of whether each conclusion is pointwise or almost sure.

If $X,Y$ are integrable and $a,b\in\mathbb{R}$, which conditions must $a\mathbb{E}[X\mid\mathcal{G}]+b\mathbb{E}[Y\mid\mathcal{G}]$ satisfy to be a conditional expectation of $aX+bY$?
On a finite probability space, let $\mathcal{G}$ be generated by a partition with atoms $A$ and $B$. Explain how to check the defining identity for a proposed conditional expectation by checking the atoms.
For the unequal-probability example, verify the defining integral identity on $A$ for $X+Y$ using the proposed conditional expectation $5$ on $A$.
Why does linearity not require $X$ and $Y$ to be independent?
In a probability space that is not complete, why should one not assume that every arbitrary modification on a null set remains $\mathcal{G}$-measurable?