Expectation from Probability Mass Functions

Expectation as a Weighted Sum

A probability mass function assigns a probability to each possible value of a discrete random variable. Expectation combines those values with their probabilities: values that are more likely contribute more to the average. For a finite set of possible values, this is a finite weighted sum. For a countably infinite set, one must also check whether the sum converges. A valid probability mass function does not by itself guarantee a finite expectation.

Let \(X:\Omega\to\mathbb{R}\) be a discrete random variable on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), with countable support \(D=\{x:\mathbb{P}(X=x)>0\}\). Write \(p_X(x)=\mathbb{P}(X=x)\). The masses satisfy \(p_X(x)\ge 0\) and \(\sum_{x\in D}p_X(x)=1\). To define the expectation even when the sum may not be finite, separate positive and negative values.

Definition: Define \(X^+=\max(X,0)\) and \(X^-=\max(-X,0)\). Their expectations are the nonnegative sums \(\mathbb{E}[X^+]=\sum_{x\in D}\max(x,0)p_X(x)\) and \(\mathbb{E}[X^-]=\sum_{x\in D}\max(-x,0)p_X(x)\), each of which may be infinite. If these two sums are not both infinite, define \(\mathbb{E}[X]=\mathbb{E}[X^+]-\mathbb{E}[X^-]\), allowing an infinite value. If both are infinite, \(\mathbb{E}[X]\) is undefined. The expectation is finite precisely when \(\sum_{x\in D}|x|p_X(x)<\infty\).

For a nonnegative series, the sum means the limit of its increasing finite partial sums, so it is well-defined as a finite number or \(+\infty\). In the finite-expectation case, the positive and negative contributions are both finite, and the expectation can be written as the absolutely convergent sum \(\sum_{x\in D}x p_X(x)\). Absolute convergence matters: it ensures that grouping or enumerating the terms does not change the answer.

Proposition (Finite-expectation criterion): For a countably supported \(X\), the expectation is a finite real number if and only if \(\sum_{x\in D}|x|p_X(x)<\infty\). In that case, \(\mathbb{E}[X]=\sum_{x\in D}x p_X(x)\), with absolute convergence.

Proof. For every real \(x\), \(|x|=\max(x,0)+\max(-x,0)\). Multiplying by \(p_X(x)\ge0\) and summing over \(D\) gives \(\sum_{x\in D}|x|p_X(x)=\mathbb{E}[X^+]+\mathbb{E}[X^-]\), including the possibility that a side is infinite. The sum on the left is finite exactly when both nonnegative sums on the right are finite. By the definition of expectation, that is exactly when \(\mathbb{E}[X]\) is a finite real number. In this case, \(x=\max(x,0)-\max(-x,0)\) term by term; subtracting the two finite sums gives \(\mathbb{E}[X]=\sum_{x\in D}x p_X(x)\). Since the sum of the absolute values of those terms is finite, this series is absolutely convergent. \(\square\)

Finite-Support and Truncated Calculations

When \(D\) is finite, the expectation is an ordinary finite sum, so no convergence question arises. The result is a probability-weighted average of the support values. For infinite support, calculations often approximate the expectation using only an initial segment of the mass function. Absolute convergence gives a precise condition and error bound for that procedure.

Proposition (Truncation bound): Suppose \(D\) is countable and \(\sum_{x\in D}|x|p_X(x)<\infty\). Enumerate its support as \(x_1,x_2,\ldots\), and set \(s_m=\sum_{r=1}^{m}x_rp_X(x_r)\). Then \(s_m\to\mathbb{E}[X]\), and \(\left|\mathbb{E}[X]-s_m\right|\le\sum_{r>m}|x_r|p_X(x_r)\). The bound tends to zero as \(m\to\infty\).

Proof. Absolute convergence says that \(\sum_{r\ge1}|x_r|p_X(x_r)\) is finite. Therefore its tails \(\sum_{r>m}|x_r|p_X(x_r)\) tend to zero. The difference between the full expectation and \(s_m\) is the convergent tail \(\sum_{r>m}x_rp_X(x_r)\). The triangle inequality bounds its absolute value by \(\sum_{r>m}|x_r|p_X(x_r)\), which tends to zero. Hence \(s_m\to\mathbb{E}[X]\), with the stated error bound. \(\square\)

Worked Example: A Finite-Support Random Variable

Suppose \(X\) takes the values \(-2\), \(1\), and \(4\) with probabilities \(\tfrac14\), \(\tfrac12\), and \(\tfrac14\), respectively. The masses are nonnegative and sum to \(\tfrac14+\tfrac12+\tfrac14=1\). The support is finite, so the expectation is a finite sum:

\mathbb{E}[X]=(-2)\tfrac14+1\cdot\tfrac12+4\cdot\tfrac14 =-\tfrac12+\tfrac12+1=1.

Check: The result lies between the smallest and largest possible values, \(-2\) and \(4\). Separating positive and negative contributions gives \(\mathbb{E}[X^+]=1\cdot\tfrac12+4\cdot\tfrac14=\tfrac32\) and \(\mathbb{E}[X^-]=2\cdot\tfrac14=\tfrac12\). Their difference is \(\tfrac32-\tfrac12=1\), agreeing with the direct sum.

This weighted-average interpretation also gives a useful bound when the support is finite. Every support value lies between the smallest and largest support values, and the probabilities are nonnegative weights summing to one. Thus their weighted sum lies in the same interval. If all mass is at one value, the expectation equals that value; otherwise the expectation need not be an attainable value of \(X\).

Bernoulli and Geometric Examples

Worked Example: A Bernoulli Random Variable

Let \(X\) be Bernoulli with success probability \(p=\tfrac38\): it equals \(1\) with probability \(\tfrac38\) and \(0\) with probability \(\tfrac58\). The probabilities are nonnegative and sum to \(\tfrac38+\tfrac58=1\). Directly from the mass function,

\mathbb{E}[X]=0\cdot\tfrac58+1\cdot\tfrac38=\tfrac38.

Check: The zero outcome contributes \(0\), while the one outcome contributes its probability, \(\tfrac38\). Thus the sum is \(\tfrac38\), as calculated. More generally, for a Bernoulli variable with \(0\le p\le1\), the same two terms give \(\mathbb{E}[X]=0(1-p)+1p=p\), including the degenerate cases \(p=0\) and \(p=1\).

Worked Example: Geometric Waiting Time

Let \(N\) be the number of independent trials up to and including the first success, where each trial succeeds with probability \(p=\tfrac13\). Its support is \(\{1,2,\ldots\}\), and its mass function is \(p_N(k)=p(1-p)^{k-1}\). Put \(q=1-p=\tfrac23\). The probabilities sum to \(\sum_{k=1}^{\infty}p q^{k-1}=p/(1-q)=1\), since \(1-q=p\). We now compute the expectation.

For \(0<q<1\), differentiating the finite geometric identity \(\sum_{k=0}^{m}q^k=(1-q^{m+1})/(1-q)\) gives

\sum_{k=1}^{m}kq^{k-1} =\frac{1-(m+1)q^m+mq^{m+1}}{(1-q)^2}.

As \(m\to\infty\), both \(mq^m\) and \((m+1)q^m\) tend to zero: for example, the ratio of consecutive terms of \(mq^m\) is \(q(1+1/m)\), which is eventually bounded above by a constant strictly less than \(1\). Therefore \(\sum_{k=1}^{\infty}kq^{k-1}=1/(1-q)^2\). It follows that

\mathbb{E}[N]=\sum_{k=1}^{\infty}kpq^{k-1} =\frac{p}{(1-q)^2} =\frac{1/3}{(1/3)^2}=3.

Check: The general expression \(p/(1-q)^2\) equals \(p/p^2=1/p\), so \(p=\tfrac13\) gives \(3\). The expectation is finite because the derived nonnegative series sums to \(3\). If \(p=1\), the separate degenerate case has \(N=1\) with probability one and expectation \(1\).

Expectation of a Count-Valued Variable

Worked Example: A Poisson Count

Let \(N\) have a Poisson distribution with parameter \(2\), so \(N\) takes values \(k=0,1,2,\ldots\), with \(p_N(k)=e^{-2}2^k/k!\). The masses sum to \(e^{-2}\sum_{k=0}^{\infty}2^k/k!=e^{-2}e^2=1\). The expectation is a nonnegative sum, and its \(k=0\) term is zero. For the remaining terms, reindex with \(j=k-1\):

\begin{aligned} \mathbb{E}[N] &=\sum_{k=1}^{\infty} k e^{-2}\frac{2^k}{k!}\\ &=e^{-2}\sum_{k=1}^{\infty}\frac{2^k}{(k-1)!}\\ &=2e^{-2}\sum_{j=0}^{\infty}\frac{2^j}{j!}\\ &=2e^{-2}e^2=2. \end{aligned}

Check: The reindexed exponential series is \(\sum_{j=0}^{\infty}2^j/j!=e^2\); multiplying by \(2e^{-2}\) gives \(2\). This also verifies that the nonnegative expectation sum is finite.

Worked Example: A Valid Count Law with Infinite Expectation

Define a count-valued \(X\) by \(\mathbb{P}(X=n)=1/[n(n+1)]\) for \(n=1,2,\ldots\). These probabilities are positive. Since \(1/[n(n+1)]=1/n-1/(n+1)\), the partial sum through \(m\) is \(1-1/(m+1)\), which tends to \(1\). Thus this is a probability mass function.

Its expectation is

\mathbb{E}[X]=\sum_{n=1}^{\infty}n\frac{1}{n(n+1)} =\sum_{n=1}^{\infty}\frac{1}{n+1}.

This series diverges. To verify divergence, for each integer \(r\ge1\), consider the \(2^r\) terms with \(2^r\le n\le2^{r+1}-1\). Each denominator \(n+1\) is at most \(2^{r+1}\), so each term is at least \(1/2^{r+1}\). That block therefore contributes at least \(2^r/2^{r+1}=\tfrac12\). Infinitely many such disjoint blocks force the partial sums to grow without bound, so \(\mathbb{E}[X]=+\infty\).

Check: The telescoping probability sum is \(1\), but the expectation sum diverges by the block bound. Hence normalization of a mass function does not imply a finite mean.

Common Mistakes and a Practical Check

A frequent error with a geometric waiting time is shifting the exponent in its mass function. For the number of trials through the first success, the probability of \(k\) trials is \(p(1-p)^{k-1}\), not \(p(1-p)^k\). Using the wrong exponent with \(p=\tfrac13\) would give \(\sum_{k\ge1} k(\tfrac13)(\tfrac23)^k\). The geometric-series calculation gives \(\sum_{k\ge1}kq^k=q/(1-q)^2\), so that incorrect sum is \((\tfrac13)(\tfrac23)/(\tfrac13)^2=2\), not \(3\). The correction is to use \(q^{k-1}\), yielding \((\tfrac13)/(1-\tfrac23)^2=3\).

Before accepting an answer from a discrete mass function, check that the probabilities are nonnegative and sum to one, include every support value, and examine convergence if there are infinitely many terms. For a signed random variable, do not rely on cancellation between two divergent sums: if both \(\mathbb{E}[X^+]\) and \(\mathbb{E}[X^-]\) are infinite, the expectation is undefined, not zero. When absolute convergence holds, the truncation bound gives a principled way to assess any finite calculation.

Check Your Understanding

For each question, state any convergence condition you use and verify your result from the mass function.

A Bernoulli variable equals \(1\) with probability \(\tfrac25\). Compute its expectation directly.
A geometric waiting time counts trials through the first success with success probability \(\tfrac12\). Write its mass function and compute its expectation.
Why does a probability mass function summing to one not guarantee a finite expectation? Use the count-valued law in this tutorial to explain.
For a countable support with \(\sum_x |x|p_X(x)<\infty\), what bound controls the error after retaining only the first \(m\) terms?
What is the expectation of a finite-support random variable whose only possible values are \(2\) and \(5\), with probabilities \(\tfrac34\) and \(\tfrac14\)?