Expectation of Simple Random Variables

Expectation as a Finite Weighted Sum

For a random variable with finitely many possible values, expectation is a finite sum: each value is weighted by the probability that the variable takes that value. This is the probability-space version of integrating a simple function. The weights matter: outcomes need not be equally likely, and distinct outcomes can give the same value of the random variable.

Throughout, $(\Omega,\mathcal{F},\mathbb{P})$ is a probability space, so $\mathbb{P}(\Omega)=1$. A random variable here is a real-valued measurable function $X:\Omega\to\mathbb{R}$. We call $X$ finite-valued if its range is a finite set. This condition concerns the values of $X$, not the size of $\Omega$: the sample space may be infinite. Since the expectation below is a finite sum of finite real numbers, it is always finite for a finite-valued $X$.

Definition: If a finite-valued random variable $X$ takes the distinct values $x_1,\ldots,x_k$, its expectation is $$ \mathbb{E}[X]=\sum_{i=1}^{k}x_i\,\mathbb{P}(X=x_i). $$ Here $\{X=x_i\}=\{\omega\in\Omega:X(\omega)=x_i\}$ is measurable because $X$ is measurable. The probabilities of these events sum to $1$, since the events are disjoint and their union is $\Omega$.

Using distinct values in the definition ensures that each value is counted once, with its full probability. Equivalently, if $A_1,\ldots,A_k$ are a measurable partition of $\Omega$ and $X=x_i$ on $A_i$, then $\mathbb{E}[X]=\sum_{i=1}^k x_i\mathbb{P}(A_i)$. The next result verifies that this computation does not depend on which partition representation is used.

The Definition Does Not Depend on a Representation

Proposition: Suppose $(\Omega,\mathcal{F},\mathbb{P})$ is a probability space and a finite-valued real random variable $X$ is represented in two ways as $X=\sum_{i=1}^r a_i\chi_{A_i}=\sum_{j=1}^s b_j\chi_{B_j}$, where each family is a finite measurable partition of $\Omega$. Then $$ \sum_{i=1}^r a_i\mathbb{P}(A_i)=\sum_{j=1}^s b_j\mathbb{P}(B_j). $$

Proof. Form the common refinement $C_{ij}=A_i\cap B_j$. For each fixed $i$, the sets $C_{ij}$, $1\le j\le s$, are disjoint and have union $A_i$. Finite additivity of the probability measure therefore gives $\mathbb{P}(A_i)=\sum_{j=1}^s\mathbb{P}(C_{ij})$. Multiplying by $a_i$ and summing over $i$ yields $\sum_i a_i\mathbb{P}(A_i)=\sum_{i,j}a_i\mathbb{P}(C_{ij})$. Similarly, $\sum_j b_j\mathbb{P}(B_j)=\sum_{i,j}b_j\mathbb{P}(C_{ij})$. On every nonempty $C_{ij}$, both representations give the value of $X$, so $a_i=b_j$. If $C_{ij}$ is empty, its probability is zero, so its contribution to either sum is zero. Thus the two refined sums agree, proving the claim. $\square$

In particular, the partition can be taken to be the level sets $\{X=x_i\}$ for the distinct values $x_i$. If a different partition splits one of these level sets into smaller pieces, finite additivity recombines their probabilities to the probability of the whole level set.

Indicators and Basic Computations

Proposition: For any event $A\in\mathcal{F}$, the indicator random variable $\chi_A$, equal to $1$ on $A$ and $0$ on $A^c$, satisfies $\mathbb{E}[\chi_A]=\mathbb{P}(A)$.

Proof. If $A$ and $A^c$ are both nonempty, the distinct values of $\chi_A$ are $1$ and $0$. The definition gives $\mathbb{E}[\chi_A]=1\cdot\mathbb{P}(A)+0\cdot\mathbb{P}(A^c)=\mathbb{P}(A)$. If $A=\varnothing$, then $\chi_A$ is identically $0$, and both sides are zero. If $A=\Omega$, then $\chi_A$ is identically $1$, and both sides equal $\mathbb{P}(\Omega)=1$. These cases establish the identity for every event. $\square$

Worked Example: An Indicator on a Fair Die

Let $\Omega=\{1,2,3,4,5,6\}$, with $\mathcal{F}$ the set of all subsets of $\Omega$ and $\mathbb{P}(\{\omega\})=1/6$ for each outcome $\omega$. Let $A=\{2,4,6\}$, the event that the result is even. Each of its three outcomes has probability $1/6$, so $\mathbb{P}(A)=3(1/6)=1/2$. The variable $\chi_A$ takes value $1$ on $A$ and value $0$ on $A^c=\{1,3,5\}$. Therefore

\mathbb{E}[\chi_A]=1\cdot\frac{3}{6}+0\cdot\frac{3}{6}=\frac12.

Check: the indicator proposition gives $\mathbb{E}[\chi_A]=\mathbb{P}(A)=3/6=1/2$, agreeing with the direct weighted sum. The membership test here is evenness: $2,4,6\in A$, while $1,3,5\notin A$.

Worked Example: A Finite Distribution With Negative Values

Suppose a real random variable $X$ takes the distinct values $-2,0,3$ with probabilities $1/4,1/2,1/4$, respectively. These probabilities are nonnegative and sum to $1/4+1/2+1/4=1$. Applying the definition gives

\mathbb{E}[X]=(-2)\frac14+0\frac12+3\frac14=-\frac12+0+\frac34=\frac14.

Check: separating the negative and positive contributions gives $-2(1/4)=-1/2$ and $3(1/4)=3/4$; adding them gives $-1/2+3/4=1/4$. The value $0$ contributes $0(1/2)=0$. Negative values require no special convention here: the expectation is defined by the same finite sum.

Worked Example: A Nonuniform Finite Probability Space

Let $\Omega=\{a,b,c,d,e\}$, let $\mathcal{F}$ contain all subsets, and assign point probabilities $\mathbb{P}(\{a\})=1/10$, $\mathbb{P}(\{b\})=1/5$, $\mathbb{P}(\{c\})=3/10$, $\mathbb{P}(\{d\})=1/4$, and $\mathbb{P}(\{e\})=1/4$. They sum to $1/10+1/5+3/10+1/4+1/4=6/10+1/2=1$. Define $X(a)=X(b)=2$, $X(c)=-1$, and $X(d)=X(e)=4$. The level sets and their probabilities are $\{X=2\}=\{a,b\}$, with probability $1/10+1/5=3/10$; $\{X=-1\}=\{c\}$, with probability $3/10$; and $\{X=4\}=\{d,e\}$, with probability $1/4+1/4=1/2$. Thus

\mathbb{E}[X]=2\frac{3}{10}+(-1)\frac{3}{10}+4\frac12 =\frac{6}{10}-\frac{3}{10}+2 =\frac{23}{10}.

Check: sum over the individual outcomes instead. The five contributions are $2(1/10)=1/5$, $2(1/5)=2/5$, $(-1)(3/10)=-3/10$, $4(1/4)=1$, and $4(1/4)=1$. Their sum is $1/5+2/5-3/10+1+1=6/10-3/10+2=23/10$, the same result.

Almost-Sure Equality Does Not Change the Expectation

Two random variables are equal $\mathbb{P}$-almost surely if the measurable event on which they differ has probability zero. This is weaker than equality at every point of $\Omega$, but finite-valued expectations cannot distinguish variables that differ only on such an event.

Proposition: Let $X$ and $Y$ be finite-valued real random variables on the probability space $(\Omega,\mathcal{F},\mathbb{P})$. If $X=Y$ $\mathbb{P}$-almost surely, then $\mathbb{E}[X]=\mathbb{E}[Y]$.

Proof. Let $N=\{\omega:X(\omega)\ne Y(\omega)\}$, which is measurable and has probability zero by the assumption. Let $S$ be the finite union of the ranges of $X$ and $Y$. For each $t\in S$, the events $\{X=t\}$ and $\{Y=t\}$ agree outside $N$, so their symmetric difference is contained in $N$. Consequently their probabilities are equal: $\mathbb{P}(X=t)=\mathbb{P}(Y=t)$. Use the expectation definition for each variable, summing over $S$; values in $S$ that are not in a variable's range have probability zero for that variable. It follows that $\mathbb{E}[X]=\sum_{t\in S}t\mathbb{P}(X=t)=\sum_{t\in S}t\mathbb{P}(Y=t)=\mathbb{E}[Y]$. $\square$

A Common Weighting Error

On the five-point space in the preceding example, the outcomes do not have equal probabilities. A common incorrect calculation is to take the ordinary average of the five values of $X$, as if each point had probability $1/5$: $(2+2-1+4+4)/5=11/5$. This is not the expectation for the probability measure specified there. The correction is to weight each outcome by its own probability, which gives $2(1/10)+2(1/5)-1(3/10)+4(1/4)+4(1/4)=23/10$. The error is not in averaging finitely many numbers; it is using weights that do not match $\mathbb{P}$.

The finite-valued definition is useful because it makes expectation concrete while retaining the measure-theoretic structure: probabilities of measurable level sets are the weights, and finite additivity allows a partition to be refined or regrouped without changing the answer. It also makes clear why a finite sample space is not required. Only the number of values of $X$ must be finite.

Check Your Understanding

For each question, begin by identifying the distinct values of the random variable and the probabilities of their level sets.

For an event $A$ with probability $p$, compute the expectation of $2\chi_A-1$ directly from its distinct values and their probabilities.
A finite-valued random variable takes values $1,4$ with probabilities $2/5,3/5$. Compute its expectation.
Explain why splitting one level set into two disjoint measurable pieces does not change the weighted-sum definition.
Give an example of a finite-valued random variable on an infinite sample space.
If two finite-valued random variables are equal outside a measurable null set, which part of the expectation formula shows that their expectations agree?