Infinite-Support Expectation Calculations

Expectations as Infinite Series

For a discrete random variable with infinitely many possible values, an expectation is a series, not a finite sum. The calculation therefore has two parts: identify the terms using the probability mass function, then check whether the series converges. A familiar algebraic simplification does not make a divergent series finite.

We use the discrete form of the law of the unconscious statistician (LOTUS). If \(X\) has probability mass function \(f\) on a countable support \(S\), then for a function \(g\), the expectation is given by the weighted sum over that support. For nonnegative \(g(X)\), this sum is allowed to equal \(+\infty\); for a real-valued \(g(X)\), a finite expectation requires absolute convergence.

The examples below focus on common infinite-support distributions. In each case, the support and parameter range matter: they determine the sum to calculate and whether it converges.

Geometric Expectations from Power Series

Let \(X\) count the number of failures before the first success in independent Bernoulli trials, each with success probability \(p\), where \(0<p\leq 1\). Write \(q=1-p\). Then \(X\) has support \(\{0,1,2,\ldots\}\) and mass function \(P(X=k)=p q^k\). For \(0<p<1\), its moments can be computed from geometric-series identities.

To derive these formulas, begin with \(\sum_{k=0}^{\infty}t^k=1/(1-t)\) for \(|t|<1\). A power series can be differentiated term by term at any point strictly inside its radius of convergence, which here is 1. Differentiating once and multiplying by \(t\) gives \(\sum_{k=0}^{\infty} k t^k=t/(1-t)^2\). Differentiating this identity and multiplying by \(t\) gives \(\sum_{k=0}^{\infty} k^2t^k=t(1+t)/(1-t)^3\). These sums converge because \(0\leq q<1\); substituting \(t=q\) and multiplying by \(p\) yields the first two moments. Subtracting the square of the mean gives the variance: $$ E[X^2]-\{E[X]\}^2 =\frac{q(1+q)}{p^2}-\frac{q^2}{p^2} =\frac{q}{p^2}. $$

When \(p=1\), \(X=0\) with probability 1, so both moments are zero. The displayed formulas also give zero in that case, although the power-series derivation above used \(q<1\).

A convention matters here. Some texts define a geometric variable as the trial number of the first success, so its support begins at 1 and it is one larger than the failure count used here. The support and mass function should be checked before applying a moment formula.

Poisson Expectations by Shifting Indices

A Poisson random variable with parameter \(\lambda>0\) has support \(\{0,1,2,\ldots\}\) and mass function \(P(X=k)=e^{-\lambda}\lambda^k/k!\). Many expectations can be evaluated by rewriting a factor involving \(k\) to match a factorial, then shifting the summation index. Since the summands in the following moment calculations are nonnegative, the index manipulation is valid; the resulting exponential series is finite.

For the mean, the \(k=0\) term vanishes. For \(k\geq1\), \(k/k!=1/(k-1)!\). Thus $$ E[X] =e^{-\lambda}\sum_{k=1}^{\infty}\frac{k\lambda^k}{k!} =e^{-\lambda}\lambda\sum_{j=0}^{\infty}\frac{\lambda^j}{j!} =e^{-\lambda}\lambda e^\lambda =\lambda, $$

where \(j=k-1\). For the second moment, use \(k^2=k(k-1)+k\). The first part has no contribution at \(k=0\) or \(k=1\), and for \(k\geq2\), \(k(k-1)/k!=1/(k-2)!\). Hence $$ \begin{aligned} E[X^2] &=e^{-\lambda}\sum_{k=2}^{\infty}\frac{k(k-1)\lambda^k}{k!} +E[X]\\ &=e^{-\lambda}\lambda^2\sum_{j=0}^{\infty}\frac{\lambda^j}{j!} +\lambda\\ &=\lambda^2+\lambda. \end{aligned} $$

This also gives \(\operatorname{Var}(X)=\lambda\), since \(E[X^2]-\{E[X]\}^2=\lambda^2+\lambda-\lambda^2\).

The same series method handles some nonlinear functions. For \(a>0\), all summands in \(E[a^X]\) are nonnegative, and $$ E[a^X] =e^{-\lambda}\sum_{k=0}^{\infty}\frac{(a\lambda)^k}{k!} =e^{-\lambda}e^{a\lambda} =e^{\lambda(a-1)}. $$

The exponential series converges for every finite \(a\lambda\), so this expectation is finite for every \(a>0\).

Power-Law Mass Functions and Moment Convergence

A power-law-type mass function assigns probability proportional to a negative power of the support value. Let \(\alpha>1\), and define a variable \(X\) on the positive integers by \(P(X=k)=Ck^{-\alpha}\). The normalization constant \(C\) is determined by requiring the probabilities to sum to 1.

The integral test says that \(\sum_{k=1}^{\infty}k^{-s}\) converges exactly when \(s>1\). In particular, because \(\alpha>1\), the proposed normalizing series is finite and positive, so \(C=1/\zeta(\alpha)\) makes the probabilities sum to 1. For \(r\geq0\), LOTUS gives $$ E[X^r] =\frac{1}{\zeta(\alpha)}\sum_{k=1}^{\infty} k^{r-\alpha} =\frac{1}{\zeta(\alpha)}\sum_{k=1}^{\infty}\frac{1}{k^{\alpha-r}}. $$

Applying the same convergence criterion with \(s=\alpha-r\) proves the result: the series converges when \(\alpha-r>1\), and diverges otherwise. Since \(X^r\) is nonnegative, divergence means its expectation is infinite.

Common Pitfalls and Why Convergence Matters

Infinite-support calculations often look like ordinary algebra after the first few terms, but the infinite sum imposes an additional requirement. For nonnegative terms, the order of summation and valid index shifts can be handled through the series itself; an algebraic rearrangement cannot turn a divergent positive series into a finite expectation.

• Ignoring the support convention. The geometric variable counting failures starts at 0, while the trial number of the first success starts at 1. Their means differ by 1.
• Using a moment formula without checking convergence. For a power-law distribution, \(E[X]\) may be finite while \(E[X^2]\) is infinite. Test the relevant series exponent.
• Confusing a second moment with a variance. Use \(\operatorname{Var}(X)=E[X^2]-\{E[X]\}^2\) only when the second moment is finite.
• Applying linearity to divergent pieces. The expectation of a signed variable requires absolute integrability; separate infinite sums cannot be subtracted as though both were finite.