Why Tonelli Is Useful for Expectations
Fubini’s Theorem allows us to change the order of integration for absolutely integrable functions. Tonelli’s Theorem serves a different purpose: for nonnegative measurable functions, it permits iterated integration without first knowing that the integral is finite. The result may be \(+\infty\), but the iterated integrals still agree. This is particularly useful in probability, where an expectation of a nonnegative random variable can be infinite and where a quantity may be expressed as a sum over infinitely many events.
Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space. A nonnegative random variable here means a measurable function \(X:\Omega\to[0,\infty]\), and its expectation is its nonnegative Lebesgue integral. We write \(\mathbb{E}X=\int_\Omega X\,d\mathbb{P}\). We will use Tonelli’s Theorem for nonnegative functions and cite the Integral of a Nonnegative Series from earlier in the course when an expectation is written as an infinite sum.
Products of Independent Nonnegative Random Variables
Independence often turns a two-variable expectation into a product of one-variable expectations. For bounded random variables, this is a familiar consequence of integrating a product function over a product distribution. Tonelli lets us extend the identity to nonnegative random variables even when one or both expectations are infinite. In the extended nonnegative reals, we use the convention \(0\cdot\infty=0\).
Proof. For each positive integer \(n\), define \(X_n=X\wedge n\) and \(Y_n=Y\wedge n\). These are bounded nonnegative measurable random variables, and \(X_n\uparrow X\) and \(Y_n\uparrow Y\) pointwise. Independence implies that the joint distribution of \((X,Y)\) is the product of its marginal distributions. Consequently, the joint distribution of \((X_n,Y_n)\) is also the product of its marginal distributions. The factorization result for nonnegative product functions established in the tutorial “Iterated Integration” therefore gives $$ \mathbb{E}(X_nY_n)=(\mathbb{E}X_n)(\mathbb{E}Y_n). $$ The sequence \(X_nY_n\) increases pointwise to \(XY\). This includes the case where one variable is zero and the other is infinite: both the limiting product and every truncated product are zero there. By the Monotone Convergence Theorem, $$ \mathbb{E}(XY)=\lim_{n\to\infty}\mathbb{E}(X_nY_n). $$ Applying the same theorem to each truncation sequence gives \(\mathbb{E}X_n\uparrow\mathbb{E}X\) and \(\mathbb{E}Y_n\uparrow\mathbb{E}Y\). The products of these two increasing nonnegative sequences converge to \((\mathbb{E}X)(\mathbb{E}Y)\), with the stated extended-value convention. Taking limits in the truncation identity proves the result.
Worked Example: A Product of Independent Discrete Variables
Suppose \(X\) and \(Y\) are independent, \(X\) takes the values \(0\) and \(4\) with probabilities \(1/3\) and \(2/3\), and \(Y\) takes the values \(1\) and \(3\) with probabilities \(1/2\) and \(1/2\). Their expectations are $$ \mathbb{E}X=0\cdot\frac13+4\cdot\frac23=\frac83, \qquad \mathbb{E}Y=1\cdot\frac12+3\cdot\frac12=2. $$ Independence and the theorem give \(\mathbb{E}(XY)=16/3\). We can check this directly: the four possible products are \(0,0,4,12\), with respective probabilities \(1/6,1/6,1/3,1/3\). Thus $$ \mathbb{E}(XY) =0\cdot\frac16+0\cdot\frac16+4\cdot\frac13+12\cdot\frac13 =\frac43+4 =\frac{16}{3}. $$ The computation verifies both the independence-based formula and the joint probabilities used in the direct calculation.
The theorem does not require \(\mathbb{E}X\) and \(\mathbb{E}Y\) to be finite. If both are positive and one is infinite, the expectation of the product is infinite. If, for example, \(X=0\) almost surely, then \(\mathbb{E}(XY)=0\), even if \(\mathbb{E}Y=\infty\). This is one reason to state the result in the extended nonnegative reals and specify the convention for \(0\cdot\infty\).
Expected Counts and Finitely Many Occurrences
Let \(A_1,A_2,\ldots\) be measurable events, and let \(N(\omega)\) count how many of them occur: $$ N(\omega)=\sum_{n=1}^{\infty}\mathbf{1}_{A_n}(\omega). $$ The count can be infinite. The Integral of a Nonnegative Series gives an exact formula for its expectation: \(\mathbb{E}N=\sum_{n=1}^{\infty}\mathbb{P}(A_n)\). No independence assumption is needed. The consequence below turns a finite sum of event probabilities into an almost-sure statement about the count.
Proof. The function \(N=\sum_{n=1}^{\infty}\mathbf{1}_{A_n}\) is measurable as the pointwise limit of its measurable partial sums. By the Integral of a Nonnegative Series, $$ \mathbb{E}N=\sum_{n=1}^{\infty}\mathbb{E}\mathbf{1}_{A_n} =\sum_{n=1}^{\infty}\mathbb{P}(A_n)<\infty. $$ Let \(D=\{\omega:N(\omega)=\infty\}\). For every positive integer \(m\), \(N\geq m\mathbf{1}_D\) pointwise, so monotonicity of the integral gives $$ \mathbb{E}N\geq m\mathbb{P}(D). $$ If \(\mathbb{P}(D)>0\), the right-hand side is unbounded as \(m\) increases, contradicting \(\mathbb{E}N<\infty\). Thus \(\mathbb{P}(D)=0\), which means that only finitely many of the events occur almost surely.
Worked Example: A Summable Sequence of Event Probabilities
Take \(\Omega=[0,1]\) with Lebesgue probability measure, and define \(A_n=[0,2^{-n}]\). Then \(\mathbb{P}(A_n)=2^{-n}\) and $$ \sum_{n=1}^{\infty}\mathbb{P}(A_n) =\sum_{n=1}^{\infty}2^{-n} =1. $$ For \(0<\omega\leq1\), the condition \(\omega\in A_n\) is equivalent to \(n\leq\log_2(1/\omega)\), so only finitely many \(A_n\) contain \(\omega\). The point \(\omega=0\) lies in every \(A_n\), but it has probability zero. The theorem explains this almost-sure conclusion from the sum of the probabilities alone; it does not require the events to be independent.
The implication is one-way. A count can be finite almost surely and still have infinite expectation. For example, let \(N\) be a positive-integer-valued random variable with $$ \mathbb{P}(N=n)=\frac{1}{n(n+1)},\qquad n\geq1. $$ These probabilities sum to one because \(1/[n(n+1)]=1/n-1/(n+1)\). The variable \(N\) is finite at every outcome in this space, but $$ \mathbb{E}N =\sum_{n=1}^{\infty}n\frac{1}{n(n+1)} =\sum_{n=1}^{\infty}\frac{1}{n+1} =\infty. $$ Thus almost-sure finiteness does not by itself give a finite expected count. A finite expectation is a stronger condition.
The Minimum of Independent Random Variables
Tonelli also helps calculate expectations by describing the event that a random variable exceeds a threshold. For nonnegative \(X\) and \(Y\), their minimum exceeds \(t\) exactly when both variables exceed \(t\). If they are independent, the probability of that intersection factors. Combining this observation with the Tail Formula for Positive Moments from earlier in the course gives a useful formula for the expected minimum.
Proof. Set \(Z=\min(X,Y)\), which is a nonnegative measurable random variable. For each \(t\geq0\), $$ \{Z>t\}=\{X>t\}\cap\{Y>t\}. $$ Independence gives $$ \mathbb{P}(Z>t)=\mathbb{P}(X>t)\mathbb{P}(Y>t). $$ Apply the Tail Formula for Positive Moments with exponent \(1\) to \(Z\): $$ \mathbb{E}Z=\int_0^\infty\mathbb{P}(Z>t)\,dt. $$ Substituting the probability identity proves the formula. The integral is permitted to be infinite.
Worked Example: The Minimum of Two Exponential Variables
Suppose \(X\) and \(Y\) are independent exponential random variables with rates \(2\) and \(3\), respectively. Their survival probabilities are \(\mathbb{P}(X>t)=e^{-2t}\) and \(\mathbb{P}(Y>t)=e^{-3t}\) for \(t\geq0\). Hence $$ \mathbb{E}(\min(X,Y)) =\int_0^\infty e^{-2t}e^{-3t}\,dt =\int_0^\infty e^{-5t}\,dt =\left[-\frac15e^{-5t}\right]_{0}^{\infty} =\frac15. $$ The product of the two survival probabilities is the survival probability of the minimum because independence makes the two threshold events independent.
Choosing Tonelli Carefully
The examples use Tonelli in different ways. For an independent product, truncation connects the nonnegative case to a product-integral factorization while allowing the final expectation to be infinite. For an event count, the Integral of a Nonnegative Series converts the sum of indicators into a sum of probabilities. For a minimum, the tail formula turns a distributional identity into an expectation calculation. In each case, nonnegativity is essential to the unrestricted use of Tonelli; a signed function requires the absolute-integrability condition in Fubini’s Theorem before its integrals can be interchanged.
A common pitfall is to infer more from a finite expectation than it actually says, or to assume independence where none has been given. The finite-count theorem needs only summability of the probabilities, but its converse is false. The product and minimum formulas, by contrast, use independence explicitly. Also, extended-valued random variables need not be finite at every outcome. Truncation handles this cleanly, and statements involving their values should be understood in the extended nonnegative reals or almost surely as appropriate.
Check Your Understanding
Use the theorems and examples in this tutorial to answer the following questions.
- Why can the product-expectation identity be proved by truncating independent nonnegative random variables?
- Does the finite-expected-count theorem require the events to be independent? Explain.
- Why does a finite expected count imply that the probability of infinitely many occurrences is zero?
- Give an example of why almost-sure finiteness of a count does not imply finite expectation.
- How does independence enter the formula for the expected minimum?