Conditional Expectation Given a Discrete Random Variable

From Conditional Probabilities to Conditional Means

When a response $Y$ is observed alongside a discrete variable $X$, its average may depend on the value of $X$. For example, a treatment group, a risk category, or a state of a stochastic system may have its own conditional distribution of outcomes. The conditional expectation records the mean response at each value of the conditioning variable. Its calculation is a weighted average, but the weights must come from the conditional distribution, not the marginal distribution of $Y$.

Work on a probability space $(\Omega,\mathcal{F},\mathbb{P})$. Suppose $X$ takes values in a countable set $D$, and $Y$ is a countably valued real random variable with $\mathbb{E}|Y|<\infty$. Write $p_x=\mathbb{P}(X=x)$. For $x\in D$ with $p_x>0$, the conditional probability mass function of $Y$ given $X=x$ is

q(y\mid x)=\mathbb{P}(Y=y\mid X=x) =\frac{\mathbb{P}(Y=y,\ X=x)}{p_x}.

Its conditional mean is the probability-weighted sum of the response values. The integrability assumption ensures that this sum is finite for every $x$ with positive probability: indeed, $p_x\mathbb{E}[|Y|\mid X=x]\leq \mathbb{E}|Y|<\infty$, so $\mathbb{E}[|Y|\mid X=x]<\infty$.

Definition: For each $x\in D$ with $p_x>0$, define $g(x)=\sum_y yq(y\mid x)$. A version of the conditional expectation of $Y$ given $X$ is $g(X)$, denoted $\mathbb{E}[Y\mid X]$. Equivalently, it is the conditional expectation given the sigma-algebra $\sigma(X)$. At a value $x$ with $p_x=0$, the ratio defining $q(y\mid x)$ is undefined. The conditional distribution there can be chosen arbitrarily; throughout this tutorial, use the convention $q(\,\cdot\mid x)=\delta_0$, so $g(x)=0$. Changing this convention does not change $\mathbb{E}[Y\mid X]$ except on an event of probability zero.

The notation $\mathbb{E}[Y\mid X]$ refers to the random variable $g(X)$, not just to a single number. Once $X$ is observed to equal $x$, its value is $g(x)$. The defining property of conditional expectation says that $g(X)$ is measurable with respect to $\sigma(X)$ and has the same integral as $Y$ over every event in $\sigma(X)$. Such events are unions of the fibers $\{X=x\}$. The next result verifies that averaging over the conditional mass functions gives exactly this object.

Theorem: Under the assumptions above, $g(X)$ is integrable and, for every $A\in\sigma(X)$, $$ \mathbb{E}\big[\mathbf{1}_A g(X)\big] =\mathbb{E}\big[\mathbf{1}_A Y\big]. $$ It is the unique $\sigma(X)$-measurable integrable random variable with this property, up to equality almost surely.

Proof. The countable-valued random variable $g(X)$ is $\sigma(X)$-measurable. Also, for $p_x>0$, the triangle inequality for the conditional sum gives $|g(x)|\leq\mathbb{E}[|Y|\mid X=x]$. The convention $g(x)=0$ on zero-probability fibers therefore yields $\mathbb{E}|g(X)|=\sum_{x:p_x>0}p_x|g(x)|\leq\mathbb{E}|Y|<\infty$. For a fiber with $p_x>0$, the definition of $q$ gives \[ \mathbb{E}[\mathbf{1}_{\{X=x\}}Y] =\sum_y y\,\mathbb{P}(Y=y,X=x) =p_x\sum_y yq(y\mid x) =p_xg(x) =\mathbb{E}[\mathbf{1}_{\{X=x\}}g(X)]. \] All sums are absolutely convergent, since $\mathbb{E}|Y|<\infty$ and $g(X)$ is integrable. If $p_x=0$, both expectations over that fiber are zero. Any $A\in\sigma(X)$ is a union of such fibers, so summing the fiber equalities proves the required integral identity for $A$. For uniqueness, let $H$ be the difference between two $\sigma(X)$-measurable integrable variables satisfying the identity. Then $\mathbb{E}[\mathbf{1}_A H]=0$ for every $A\in\sigma(X)$. Taking $A=\{H>0\}$ gives $\mathbb{E}[\mathbf{1}_{\{H>0\}}H]=0$; a nonnegative random variable with expectation zero is zero almost surely, so $H=0$ on $\{H>0\}$ almost surely. Taking $A=\{H<0\}$ and applying the same reasoning to $-H$ shows $H=0$ on $\{H<0\}$ almost surely. Thus $H=0$ almost surely. $\square$

Computing the Conditional Mean

For each positive-probability value of $X$, the calculation has two stages: first obtain the conditional mass function, then take its mean. If the joint probabilities are given, divide each joint probability in a fiber by the probability of that fiber. The resulting conditional probabilities must sum to one. The examples make both calculations and their checks explicit.

Worked Example: A Joint Probability Table and a Null Value

Suppose $X$ takes values in $D=\{0,1,2\}$, $Y$ takes values in $\{-2,0,1,4\}$, and their joint probabilities are shown below. Find $\mathbb{E}[Y\mid X]$.

$X\backslash Y$	$-2$	$0$	$1$	$4$	$\mathbb{P}(X=x)$
$0$	$\tfrac18$	$0$	$\tfrac38$	$0$	$\tfrac12$
$1$	$0$	$\tfrac16$	$0$	$\tfrac16$	$\tfrac13$
$2$	$0$	$0$	$0$	$0$	$0$

The joint probabilities sum to $\tfrac18+\tfrac38+\tfrac16+\tfrac16=\tfrac12+\tfrac13=\tfrac56$, so as written they do not form a probability table. Correct the first row's entry for $Y=1$ to $\tfrac12$; its row total is then $\tfrac18+\tfrac12=\tfrac58$, which still does not equal the stated marginal. Instead use the following normalized joint probabilities: in the row $X=0$, let the entries be $\tfrac18,0,\tfrac38,0$, total $\tfrac12$; in the row $X=1$, let them be $0,\tfrac16,0,\tfrac16$, total $\tfrac13$; and assign the remaining $\tfrac16$ to $X=2,Y=0$. Thus the final row is $0,\tfrac16,0,0$, and $\mathbb{P}(X=2)=\tfrac16$. The corrected table now has total probability $\tfrac12+\tfrac13+\tfrac16=1$.

For $X=0$, divide the joint entries by $\tfrac12$. The conditional masses at $Y=-2$ and $Y=1$ are $\tfrac14$ and $\tfrac34$, respectively, and they sum to $1$. Hence $g(0)=(-2)(\tfrac14)+(1)(\tfrac34)=-\tfrac12+\tfrac34=\tfrac14$. For $X=1$, the conditional masses at $Y=0$ and $Y=4$ are each $(\tfrac16)/(\tfrac13)=\tfrac12$, so $g(1)=0(\tfrac12)+4(\tfrac12)=2$. For $X=2$, the conditional masses are $q(0\mid2)=1$ and zero elsewhere, so $g(2)=0$.

Check: On the $X=0$ fiber, $\mathbb{E}[\mathbf{1}_{\{X=0\}}Y]=(-2)(\tfrac18)+(1)(\tfrac38)=-\tfrac14+\tfrac38=\tfrac18$, while $\mathbb{E}[\mathbf{1}_{\{X=0\}}g(X)]=(\tfrac12)(\tfrac14)=\tfrac18$. On $X=1$, the corresponding values are $4(\tfrac16)=\tfrac23$ and $(\tfrac13)(2)=\tfrac23$. On $X=2$, both are zero. Also the final joint table has row sums $\tfrac12,\tfrac13,\tfrac16$, which sum to $1$.

Worked Example: A Signed Response

Let $X$ take values $a$ and $b$, with $\mathbb{P}(X=a)=\tfrac25$ and $\mathbb{P}(X=b)=\tfrac35$. Given $X=a$, suppose $Y$ has masses $\tfrac14,\tfrac12,\tfrac14$ at $-2,0,2$. Given $X=b$, suppose its masses at those same values are $\tfrac12,\tfrac14,\tfrac14$. Compute the conditional mean at each value.

For $X=a$, the masses sum to $\tfrac14+\tfrac12+\tfrac14=1$, and \[ g(a)=(-2)(\tfrac14)+0(\tfrac12)+2(\tfrac14)=-\tfrac12+0+\tfrac12=0. \] For $X=b$, its masses also sum to $\tfrac12+\tfrac14+\tfrac14=1$, and \[ g(b)=(-2)(\tfrac12)+0(\tfrac14)+2(\tfrac14)=-1+0+\tfrac12=-\tfrac12. \] Therefore $\mathbb{E}[Y\mid X]$ equals $0$ on $\{X=a\}$ and $-\tfrac12$ on $\{X=b\}$.

Check: The joint probabilities on $X=a$ are $\tfrac25\cdot\tfrac14=\tfrac1{10}$, $\tfrac25\cdot\tfrac12=\tfrac15$, and $\tfrac25\cdot\tfrac14=\tfrac1{10}$; their row sum is $\tfrac1{10}+\tfrac15+\tfrac1{10}=\tfrac25$. The $Y$-weighted joint sum on this fiber is $(-2)(\tfrac1{10})+2(\tfrac1{10})=0=(\tfrac25)g(a)$. On $X=b$, the joint probabilities are $\tfrac3{10},\tfrac3{20},\tfrac3{20}$, with row sum $\tfrac3{10}+\tfrac3{20}+\tfrac3{20}=\tfrac35$. The weighted sum is $(-2)(\tfrac3{10})+2(\tfrac3{20})=-\tfrac35+\tfrac3{10}=-\tfrac3{10}=(\tfrac35)g(b)$.

Worked Example: A Conditional Event Probability as a Mean

Let $Y=\mathbf{1}_B$ for an event $B$, and suppose $X$ takes values $0$ and $1$, each with probability $\tfrac12$. Assume $\mathbb{P}(B\mid X=0)=\tfrac15$ and $\mathbb{P}(B\mid X=1)=\tfrac45$. Find $\mathbb{E}[Y\mid X]$.

Since $Y$ is $1$ on $B$ and $0$ off $B$, its conditional mass at $1$ is $\mathbb{P}(B\mid X=x)$, and its mass at $0$ is $1-\mathbb{P}(B\mid X=x)$. At $X=0$, these masses are $\tfrac15$ and $\tfrac45$, so $g(0)=1(\tfrac15)+0(\tfrac45)=\tfrac15$. At $X=1$, the masses are $\tfrac45$ and $\tfrac15$, so $g(1)=1(\tfrac45)+0(\tfrac15)=\tfrac45$. Thus $\mathbb{E}[\mathbf{1}_B\mid X]$ is the conditional probability of $B$ at the observed value of $X$.

Check: The joint probabilities of $B$ with the two fibers are $\mathbb{P}(B,X=0)=(\tfrac12)(\tfrac15)=\tfrac1{10}$ and $\mathbb{P}(B,X=1)=(\tfrac12)(\tfrac45)=\tfrac25$. Their sum is $\tfrac1{10}+\tfrac25=\tfrac12$. The weighted conditional means give $(\tfrac12)(\tfrac15)+(\tfrac12)(\tfrac45)=\tfrac1{10}+\tfrac25=\tfrac12$, the same probability.

Zero-Probability Values and Common Errors

A conditional probability given $X=x$ is defined by a ratio only when $\mathbb{P}(X=x)>0$. If $x$ has probability zero, the ratio has denominator zero, so it does not specify a conditional mass function. A value can still belong to the declared range $D$ of $X$; that does not make its fiber have positive probability. The conditional distribution on such a fiber may be assigned freely. The convention $q(\,\cdot\mid x)=\delta_0$ used here is convenient because it assigns conditional mean zero, but another convention gives the same random variable almost surely: the event $\{X=x\}$ itself has probability zero.

Common mistake: using marginal masses as conditional weights. In the first example, the marginal probabilities of $Y=-2$ and $Y=1$ are not the weights for computing the mean given $X=0$. The correct weights are obtained by dividing the joint entries in the $X=0$ row by $\mathbb{P}(X=0)=\tfrac12$, giving $\tfrac14$ and $\tfrac34$. Using the unnormalized row entries $\tfrac18$ and $\tfrac38$ would produce $(-2)(\tfrac18)+(1)(\tfrac38)=\tfrac18$, not the conditional mean $\tfrac14$. The result fails the required row check: $\tfrac18+\tfrac38=\tfrac12$, not $1$. Dividing by the row probability corrects the weights and gives $(-2)(\tfrac14)+(1)(\tfrac34)=\tfrac14$.

A second error is to report $\mathbb{E}[Y\mid X]$ as one constant. It is a random variable, typically a function of $X$; in the signed-response example it takes the values $0$ and $-\tfrac12$ on different fibers. Finally, a conditional mean is not necessarily one of the possible response values. It is an average, so it may lie between possible values, as $\tfrac14$ does in the first example.

Check Your Understanding

For each question, state the conditional weights you use and check that they sum to one on every positive-probability fiber.

If $X$ has positive-probability values $r,s$, and given $X=r$, $Y$ is $-1$ with probability $\tfrac13$ and $2$ with probability $\tfrac23$, what is $g(r)$?
If a joint table gives $\mathbb{P}(Y=y,X=x)$ and $\mathbb{P}(X=x)>0$, what division produces the conditional mass at $y$?
Suppose $X$ has a declared value $u$ with $\mathbb{P}(X=u)=0$. Why is the ratio formula for $q(y\mid u)$ unavailable, and what convention for $g(u)$ was used here?
For $Y=\mathbf{1}_B$, express $\mathbb{E}[Y\mid X=x]$ using a conditional probability.
Why must two versions of $\mathbb{E}[Y\mid X]$ that differ only at a zero-probability value of $X$ still represent the same conditional expectation almost surely?

\(X\backslash Y\)	\(-2\)	\(0\)	\(1\)	\(4\)	\(\mathbb{P}(X=x)\)
\(0\)	\(\tfrac18\)	\(0\)	\(\tfrac38\)	\(0\)	\(\tfrac12\)
\(1\)	\(0\)	\(\tfrac16\)	\(0\)	\(\tfrac16\)	\(\tfrac13\)
\(2\)	\(0\)	\(0\)	\(0\)	\(0\)	\(0\)

Conditional Expectation Given a Discrete Random Variable

What You'll Learn