When a Discrete Expectation Exists

Why Convergence Matters

For a discrete random variable, the expected value is built from a countable weighted sum: each possible value is multiplied by its probability. A finite support makes the calculation a finite sum. With infinitely many possible values, however, the sum may converge to a finite number, or it may diverge. The probabilities alone do not settle the question: large values can outweigh small probabilities.

To decide whether the expectation is a finite real number, the key test is absolute convergence. It ensures that the weighted values have a finite total magnitude, so the resulting expectation does not depend on how the support is listed.

Definition and Convergence Criterion

Definition: Let $X$ have a countable support consisting of distinct values $x_1,x_2,\ldots$, with $p_j=P(X=x_j)$ and $\sum_{j=1}^{\infty}p_j=1$. The probability-weighted series for its expectation is $\sum_{j=1}^{\infty}x_jp_j$. The expectation is finite when this series is absolutely convergent, meaning $$ \sum_{j=1}^{\infty}|x_j|p_j<\infty. $$ In that case, $E[X]=\sum_{j=1}^{\infty}x_jp_j$.

The absolute-value series asks a direct question: is the total probability-weighted magnitude of the possible values finite? If it is, then the signed expectation is finite too, because $$ \left|\sum_{j=1}^{\infty}x_jp_j\right| \leq \sum_{j=1}^{\infty}|x_j|p_j. $$ The inequality applies first to finite partial sums; absolute convergence then permits passage to the infinite sum.

Theorem: If $\sum_{j=1}^{\infty}|x_j|p_j<\infty$, then the expectation series converges absolutely, its value is unchanged by any enumeration of the support, and its tails satisfy $$ \left|\sum_{j=m}^{\infty}x_jp_j\right| \leq \sum_{j=m}^{\infty}|x_j|p_j \longrightarrow 0 \quad\text{as }m\longrightarrow\infty. $$

Proof. Set $a_j=x_jp_j$. For any integers $r\geq q$, $$ \left|\sum_{j=q}^{r}a_j\right| \leq \sum_{j=q}^{r}|a_j|. $$ Because $\sum_j|a_j|$ converges, its tails can be made arbitrarily small. The displayed inequality therefore shows that the partial sums of $\sum_j a_j$ satisfy the Cauchy criterion and converge to a finite value. Taking limits in the same bound gives the stated tail inequality.

To see why the value is independent of the enumeration, fix $\varepsilon>0$. There is a finite set of support indices whose complement has $\sum |a_j|<\varepsilon$. Any finite partial collection containing that set differs from its sum over the full support by at most $\varepsilon$. Thus any ordering has partial sums approaching the same limit. $\square$

This criterion concerns a finite expectation. For a nonnegative random variable, its weighted sum may instead diverge to $+\infty$; that is not a finite mean. A finite signed sum should not be inferred from a particular ordering or from cancellation in selected partial sums. Absolute convergence is the dependable test for a finite expectation.

Worked Examples

Worked Example: A Geometric Distribution Has a Finite Mean

Let $X$ take values $k=0,1,2,\ldots$ with $P(X=k)=\frac{2}{3}(\frac{1}{3})^k$. First check that these probabilities sum to one: $$ \sum_{k=0}^{\infty}\frac{2}{3}\left(\frac{1}{3}\right)^k = \frac{2}{3}\frac{1}{1-\frac{1}{3}} =1. $$

All values are nonnegative, so the absolute-expectation series is the expectation series. The geometric power series $\sum_{k=0}^{\infty}r^k=1/(1-r)$ can be differentiated for $|r|<1$, giving $\sum_{k=0}^{\infty}kr^k=r/(1-r)^2$. With $r=1/3$, $$ E[X] = \frac{2}{3}\sum_{k=0}^{\infty}k\left(\frac{1}{3}\right)^k = \frac{2}{3}\frac{\frac{1}{3}}{(1-\frac{1}{3})^2} = \frac{2}{3}\cdot\frac{3}{4} = \frac{1}{2}. $$ The series converges absolutely, so the mean is finite and equals $1/2$.

Worked Example: A Power-Law Distribution With a Finite Mean

For $\alpha>1$, define $\zeta(\alpha)=\sum_{k=1}^{\infty}k^{-\alpha}$, which is finite. Let $X$ take positive-integer values with $$ P(X=k)=\frac{k^{-5/2}}{\zeta(5/2)},\qquad k=1,2,\ldots. $$ These probabilities sum to one by the definition of $\zeta(5/2)$. The absolute-expectation series is $$ \sum_{k=1}^{\infty}kP(X=k) = \frac{1}{\zeta(5/2)} \sum_{k=1}^{\infty}k^{-3/2} = \frac{\zeta(3/2)}{\zeta(5/2)}. $$

Both zeta series in this ratio converge because their exponents exceed one. Therefore $E[X]$ exists as a finite number, equal to $\zeta(3/2)/\zeta(5/2)$. The probabilities decrease at a power-law rate, but the weighted sum still converges because the exponent in the mean series is $3/2$.

Worked Example: A Power-Law Distribution With an Infinite Mean

Now let $P(X=k)=k^{-2}/\zeta(2)$ for $k=1,2,\ldots$. Again, the probabilities sum to one. But weighting each probability by $k$ gives $$ \sum_{k=1}^{\infty}kP(X=k) = \frac{1}{\zeta(2)} \sum_{k=1}^{\infty}\frac{1}{k}. $$ The harmonic series diverges. One way to verify this is to group its terms in blocks: for every integer $m\geq 0$, $$ \sum_{k=2^m+1}^{2^{m+1}}\frac{1}{k} \geq 2^m\frac{1}{2^{m+1}} = \frac{1}{2}. $$ Each block contributes at least $1/2$, so the partial sums are unbounded. Thus the absolute-expectation series diverges, and $X$ does not have a finite expectation. Although the probabilities themselves are normalized, their decay is not fast enough to make the mean finite.

Worked Example: Sparse Support Can Still Produce an Infinite Mean

Let $X$ take values $2^j$, for $j=1,2,\ldots$, with probabilities $P(X=2^j)=1/(j^2\zeta(2))$. These probabilities are valid because their sum is $\zeta(2)/\zeta(2)=1$. The mean series is $$ \sum_{j=1}^{\infty}2^jP(X=2^j) = \frac{1}{\zeta(2)} \sum_{j=1}^{\infty}\frac{2^j}{j^2}. $$

Write $a_j=2^j/j^2$. Then $$ \frac{a_{j+1}}{a_j} = \frac{2j^2}{(j+1)^2} \longrightarrow 2. $$ In particular, this ratio exceeds $3/2$ for all sufficiently large $j$. From that point onward the terms grow by at least a factor of $3/2$ at each step, so $a_j$ does not tend to zero. A convergent series must have terms tending to zero; therefore the mean series diverges. This example shows why checking the probabilities’ decay alone is insufficient: the sizes of the support values matter as well.

A Useful Power-Series Test

For distributions on the positive integers with probabilities proportional to $k^{-\alpha}$, the expectation test reduces to a familiar series. The convergence boundary can be proved directly.

Result: For real $q$, the series $\sum_{k=1}^{\infty}k^{-q}$ converges if and only if $q>1$.

Proof. If $q\leq 0$, the terms $k^{-q}$ do not tend to zero, so the series diverges. Suppose $q>0$. The function $t^{-q}$ is positive and decreasing on $[1,\infty)$, so comparison with its integral gives, for integers $N\geq 2$, $$ \int_1^N t^{-q}\,dt \leq \sum_{k=1}^{N-1}k^{-q} \leq 1+\int_1^N t^{-q}\,dt. $$ If $q\neq 1$, the integral is $(N^{1-q}-1)/(1-q)$. It has a finite limit as $N\to\infty$ when $q>1$, and grows without bound when $0<q<1$. If $q=1$, the integral is $\log N$, which also grows without bound. These comparisons prove the claim. $\square$

Consequently, for a positive-integer power-law mass function proportional to $k^{-\alpha}$, where $\alpha>1$ ensures normalization, the mean series is proportional to $\sum k^{-(\alpha-1)}$. It converges exactly when $\alpha-1>1$, or $\alpha>2$. The cases $\alpha=5/2$ and $\alpha=2$ above illustrate the two sides of this boundary.

Common Mistake: Looking Only at the Probability Masses

A common but incorrect argument is: “The probabilities $p_j$ get small, so the expectation must be finite.” Small probabilities ensure that their total can equal one; they do not ensure that $|x_j|p_j$ has a finite sum. In the sparse-support example, probabilities decrease like $1/j^2$, but the corresponding values grow like $2^j$, making the weighted terms fail even to approach zero.

Another tempting shortcut is to pair terms or choose an ordering in which positive and negative contributions appear to cancel. That can produce a misleading finite-looking sum. For a finite expectation, test $\sum |x_j|p_j$ directly. If it is finite, the signed sum converges independently of ordering; if not, such a rearrangement does not establish a finite expectation.

Key takeaway: For a countable discrete distribution, check $\sum_j|x_j|P(X=x_j)$. If it is finite, the expectation is finite and equals the absolutely convergent weighted sum. If it diverges, the distribution does not have a finite expectation.

Check Your Understanding

Use the absolute-expectation series in each question.

For $P(X=k)=k^{-3}/\zeta(3)$, $k=1,2,\ldots$, write the series that tests whether $E[X]$ is finite.
Why does normalization of a probability mass function not by itself guarantee a finite expectation?
Suppose $X$ takes values $j$ with probabilities proportional to $j^{-4}$, for $j=1,2,\ldots$. Does its mean converge? Identify the exponent in the weighted series.
For a countable distribution, what does absolute convergence guarantee about the value of the expectation series under a different enumeration of the support?
A series of weighted terms appears to sum to a finite number when its positive and negative terms are paired. What additional convergence test is needed before calling that number a finite expectation?