Tutorials › Probability Theory (Graduate) › Independence and Conditional Expectation

Home  /  Tutorials  /  Conditional Expectation

Conditional Expectation · Tutorial 234 of 946

Independence and Conditional Expectation

See why independence from the information being conditioned on makes conditional expectation equal to ordinary expectation, and where pairwise independence falls short.

Intermediate 9 min read

What You'll Learn

  • Define independence between a random variable and a sigma-algebra.
  • Prove the integral factorization that independence provides.
  • Establish when conditional expectation is almost surely constant.
  • Compute conditional expectations in finite and continuous examples.
  • Recognize why pairwise independence does not guarantee independence from a generated sigma-algebra.

When Conditioning Does Not Change an Expectation

A conditional expectation records what can be predicted about an integrable random variable using the information in a sigma-algebra. If that information is independent of the variable, it should not improve the prediction: the conditional expectation is simply the ordinary expectation.

The key point is to state independence at the right level. Saying that a random variable is independent of each of several other random variables separately may not mean it is independent of all the information they jointly contain. The theorem below uses independence from the entire sigma-algebra on which we condition.

Independence of a Random Variable and a Sigma-Algebra

Definition: Let \(X\) be a real-valued random variable and let \(\mathcal G\) be a sigma-algebra. We say that \(X\) is independent of \(\mathcal G\) if the sigma-algebra \(\sigma(X)\) generated by \(X\) is independent of \(\mathcal G\). This means that for every \(C\in\sigma(X)\) and every \(A\in\mathcal G\), $$ \mathbb P(C\cap A)=\mathbb P(C)\mathbb P(A). $$

In particular, each event of the form \(\{X\in B\}\), for a Borel set \(B\subseteq\mathbb R\), is independent of every event in \(\mathcal G\). The full definition asks for independence of the two sigma-algebras, not just a selected collection of events.

Recall the defining criterion for conditional expectation. If \(X\) is integrable, then \(\mathbb E[X\mid\mathcal G]\) is a \(\mathcal G\)-measurable integrable random variable \(Z\) such that, for every \(A\in\mathcal G\), $$ \mathbb E[\mathbf 1_A Z]=\mathbb E[\mathbf 1_A X]. $$ Here \(\mathbf 1_A\) is the indicator of \(A\). Conditional expectations satisfying this criterion are unique almost surely.

The Integral Factorization Behind the Result

Independence first gives a useful identity: multiplying an integrable independent variable by the indicator of an event in \(\mathcal G\) factors the expectation into two ordinary expectations. We prove the identity before applying the conditional-expectation criterion.

Lemma: Suppose \(X\) is integrable and independent of \(\mathcal G\). Then, for every \(A\in\mathcal G\), $$ \mathbb E[\mathbf 1_A X]=\mathbb P(A)\mathbb E[X]. $$
Proof. First suppose \(X\geq 0\). For an indicator \(\mathbf 1_C\) with \(C\in\sigma(X)\), independence gives $$ \mathbb E[\mathbf 1_A\mathbf 1_C] =\mathbb P(A\cap C) =\mathbb P(A)\mathbb P(C) =\mathbb P(A)\mathbb E[\mathbf 1_C]. $$ The same identity follows for every nonnegative simple \(\sigma(X)\)-measurable random variable by writing it as a finite sum of nonnegative multiples of indicators of sets in \(\sigma(X)\). Choose an increasing sequence of such simple random variables converging pointwise to \(X\). The monotone convergence theorem, applied to both sides, gives $$ \mathbb E[\mathbf 1_A X]=\mathbb P(A)\mathbb E[X]. $$ For a general integrable \(X\), write \(X=X^+-X^-\), where \(X^+=\max(X,0)\) and \(X^-=\max(-X,0)\). Both are nonnegative and integrable, and both are \(\sigma(X)\)-measurable. Applying the nonnegative case to each and subtracting the resulting finite equalities proves the identity for \(X\). \(\square\)

Conditional Expectation Is Constant Under Independence

Theorem: If \(X\) is an integrable real-valued random variable independent of a sigma-algebra \(\mathcal G\), then $$ \mathbb E[X\mid\mathcal G]=\mathbb E[X]\quad\text{almost surely}. $$
Proof. Let \(c=\mathbb E[X]\), viewed as a constant random variable. It is integrable and \(\mathcal G\)-measurable. For every \(A\in\mathcal G\), the lemma gives $$ \mathbb E[\mathbf 1_A X] =\mathbb P(A)\mathbb E[X] =\mathbb E[\mathbf 1_A c]. $$ Thus \(c\) satisfies the defining criterion for \(\mathbb E[X\mid\mathcal G]\). By almost-sure uniqueness of conditional expectation, \(\mathbb E[X\mid\mathcal G]=c\) almost surely. \(\square\)

The conclusion is an almost-sure equality, as is usual for conditional expectations. A particular version may differ from the constant on a null set. The assertion does not require \(X\) itself to be constant: its values can vary, while information in \(\mathcal G\) gives no way to predict those variations.

Worked Example: A Bernoulli Variable and a Finite Conditioning Sigma-Algebra

Worked Example: A Bernoulli Variable and a Finite Conditioning Sigma-Algebra

Let \(X\) and \(Y\) take values in \(\{0,1\}\) and \(\{r,s\}\), respectively, with joint probabilities $$ \begin{aligned} \mathbb P(X=1,Y=r)&=\frac{3}{20},& \mathbb P(X=0,Y=r)&=\frac{9}{20},\\ \mathbb P(X=1,Y=s)&=\frac{1}{10},& \mathbb P(X=0,Y=s)&=\frac{3}{10}. \end{aligned} $$ These four probabilities sum to \(1\). The marginals are \(\mathbb P(X=1)=1/4\), \(\mathbb P(X=0)=3/4\), \(\mathbb P(Y=r)=3/5\), and \(\mathbb P(Y=s)=2/5\). Each joint probability is the product of its corresponding marginals; for example, $$ \mathbb P(X=0,Y=s)=\frac{3}{10}=\frac{3}{4}\cdot\frac{2}{5}. $$ Thus \(X\) and \(Y\) are independent. Set \(\mathcal G=\sigma(Y)\). Since \(\mathbb E[X]=1/4\), consider the constant candidate \(1/4\).

The two nonempty atoms of \(\mathcal G\) satisfy $$ \mathbb E[X\mathbf 1_{\{Y=r\}}]=\frac{3}{20} =\frac{3}{5}\cdot\frac{1}{4}, \qquad \mathbb E[X\mathbf 1_{\{Y=s\}}]=\frac{1}{10} =\frac{2}{5}\cdot\frac{1}{4}. $$ Every event in \(\mathcal G\) is a union of these atoms, so the defining integral identity holds for every event in \(\mathcal G\). Therefore $$ \mathbb E[X\mid\sigma(Y)]=\frac{1}{4}\quad\text{almost surely}. $$

Worked Example: Conditioning on an Independent Coordinate

Worked Example: Conditioning on an Independent Coordinate

Take the unit square \([0,1]^2\) with two-dimensional Lebesgue probability measure, and let \(U\) and \(V\) be its first and second coordinate functions. These coordinates are independent and each is uniform on \([0,1]\). Let \(X=U^2\) and \(\mathcal G=\sigma(V)\). Because \(X\) is a measurable function of \(U\), it is independent of \(V\), and hence independent of \(\sigma(V)\).

The ordinary expectation is $$ \mathbb E[X]=\int_0^1 u^2\,du=\frac{1}{3}. $$ To see the conditional-expectation identity directly, any event \(A\in\sigma(V)\) has the form \(\{V\in B\}\) for a Borel set \(B\subseteq[0,1]\). Writing \(\lambda\) for one-dimensional Lebesgue measure, integration over the square gives $$ \mathbb E[X\mathbf 1_A] =\int_0^1\int_0^1 u^2\mathbf 1_B(v)\,du\,dv =\left(\int_0^1u^2\,du\right)\lambda(B) =\frac{1}{3}\mathbb P(A). $$ Consequently, \(\mathbb E[U^2\mid\sigma(V)]=1/3\) almost surely. Knowing the second coordinate does not change the expectation of the square of the first.

Worked Example: Pairwise Independence Is Not Enough

Worked Example: Pairwise Independence Is Not Enough

Let \(A\) and \(B\) be independent fair \(\{0,1\}\)-valued random variables. Define $$ X=\mathbf 1_{\{A\ne B\}},\qquad Y=A,\qquad Z=B. $$ All four pairs \((A,B)\) have probability \(1/4\), so \(\mathbb E[X]=\mathbb E[Y]=\mathbb E[Z]=1/2\). For any \(x,y\in\{0,1\}\), there is exactly one value of \(B\) that makes \(X=x\) when \(A=y\). Hence \(\mathbb P(X=x,Y=y)=1/4\), which is the product of the two marginal probabilities. The same argument with \(A\) and \(B\) interchanged shows that \(X\) and \(Z\) are independent. Also, \(Y\) and \(Z\) are independent by construction.

Despite these pairwise independence statements, \(X\) is determined by the pair \((Y,Z)\). In particular, it is measurable with respect to \(\sigma(Y,Z)\). Therefore $$ \mathbb E[X\mid\sigma(Y,Z)]=X\quad\text{almost surely}, $$ since \(X\) itself satisfies the defining criterion when it is already measurable with respect to the conditioning sigma-algebra. This conditional expectation is not equal almost surely to \(\mathbb E[X]=1/2\): \(X\) takes only the values \(0\) and \(1\). Thus pairwise independence does not imply that \(X\) is independent of the joint information \(\sigma(Y,Z)\).

Why the Sigma-Algebra Matters

The theorem is useful whenever a conditioning sigma-algebra represents information known to be independent of the quantity being averaged. It immediately identifies the conditional expectation without requiring a calculation on each event in that sigma-algebra.

A common pitfall is to check independence only against individual variables that generate the information. The last example shows why this can fail: \(X\) is independent of \(Y\) and independent of \(Z\), but the pair \((Y,Z)\) reveals \(X\) exactly. To apply the theorem, verify independence of \(\sigma(X)\) and the entire conditioning sigma-algebra \(\mathcal G\).

There is also a distinction between independence and measurability. If \(X\) is \(\mathcal G\)-measurable, conditioning on \(\mathcal G\) retains \(X\), rather than generally replacing it by its ordinary expectation. The constant conclusion applies when \(X\) is independent of \(\mathcal G\); it is not a general rule for conditional expectations.

Check Your Understanding

Use the definition of conditional expectation and the independence criterion to answer these questions.

  1. What does it mean for a random variable \(X\) to be independent of a sigma-algebra \(\mathcal G\)?
  2. Why does independence imply \(\mathbb E[\mathbf 1_A X]=\mathbb P(A)\mathbb E[X]\) for every \(A\in\mathcal G\)?
  3. Which properties of the constant \(\mathbb E[X]\) allow it to serve as a version of \(\mathbb E[X\mid\mathcal G]\)?
  4. In the finite example, what are the two atoms of \(\sigma(Y)\), and how do their integrals verify the conditional-expectation criterion?
  5. Why do pairwise independence of \(X\) with \(Y\) and \(Z\) not suffice to conclude that \(\mathbb E[X\mid\sigma(Y,Z)]=\mathbb E[X]\)?

Previous

Next

Back to Tutorials
Clinical Biostats