Point Masses Determine a Countable Probability Model
On a finite sample space, one can assign a weight to each outcome and add the weights of the outcomes in an event. The same description works on a countably infinite sample space, but the weights must form a convergent nonnegative series with total one. This gives a concrete way to build probability measures and to compute probabilities even when events contain infinitely many outcomes.
Throughout, let \(\Omega\) be a nonempty countable set, finite or countably infinite, and equip it with the sigma-algebra \(\mathcal{P}(\Omega)\) of all subsets. A probability measure \(\mathbb{P}\) on \((\Omega,\mathcal{P}(\Omega))\) is a nonnegative, countably additive set function with \(\mathbb{P}(\Omega)=1\). Thus this is a finite measure space, and it is complete: every subset of a measurable null set is measurable because every subset of \(\Omega\) is measurable.
A sum of nonnegative numbers indexed by a countable set is understood as the supremum of its finite subsums. If the set is enumerated as \(\Omega=\{\omega_1,\omega_2,\ldots\}\), this agrees with the limit of the increasing partial sums. In the finite case, the sum is just the finite sum; the sum over an empty set is zero.
Proof. First suppose numbers \(p_\omega\ge0\) with total sum one are given, and define \(\mathbb{P}(A)=\sum_{\omega\in A}p_\omega\). Then \(\mathbb{P}(\varnothing)=0\), and \(\mathbb{P}(\Omega)=1\). Also \(0\le\mathbb{P}(A)\le1\), since every finite subsum over \(A\) is bounded by the sum over \(\Omega\).
We verify countable additivity. If \(A,B\subseteq\Omega\) are disjoint, every finite subset of \(A\cup B\) splits into a finite subset of \(A\) and a finite subset of \(B\). This gives \(\mathbb{P}(A\cup B)\le\mathbb{P}(A)+\mathbb{P}(B)\) on taking suprema of finite subsums. Conversely, finite subsums chosen from \(A\) and \(B\) combine to a finite subsum from \(A\cup B\). Taking suprema gives the reverse inequality, so \(\mathbb{P}(A\cup B)=\mathbb{P}(A)+\mathbb{P}(B)\).
Now let \(A_1,A_2,\ldots\) be pairwise disjoint subsets of \(\Omega\), and put \(A=\bigcup_{j=1}^\infty A_j\). For any finite \(F\subseteq A\), each point of \(F\) belongs to exactly one \(A_j\), so \(F\) splits into finitely many disjoint pieces \(F\cap A_j\). Therefore \(\sum_{\omega\in F}p_\omega\le\sum_{j=1}^\infty\mathbb{P}(A_j)\). Taking the supremum over finite \(F\) gives \(\mathbb{P}(A)\le\sum_j\mathbb{P}(A_j)\). For every positive integer \(J\), finite additivity gives \(\sum_{j=1}^J\mathbb{P}(A_j)=\mathbb{P}(\bigcup_{j=1}^J A_j)\le\mathbb{P}(A)\). Taking the supremum over \(J\) proves the opposite inequality. Thus \(\mathbb{P}\) is countably additive.
For the converse, suppose \(\mathbb{P}\) is a probability measure and set \(p_\omega=\mathbb{P}(\{\omega\})\). These numbers are nonnegative. The singleton sets partition \(\Omega\), so countable additivity gives \(\sum_{\omega\in\Omega}p_\omega=\mathbb{P}(\Omega)=1\). For any \(A\subseteq\Omega\), its singleton sets partition \(A\), and countable additivity gives \(\mathbb{P}(A)=\sum_{\omega\in A}p_\omega\). This also proves uniqueness, because the mass at \(\omega\) must equal \(\mathbb{P}(\{\omega\})\). The argument includes finite \(\Omega\), where the relevant sums are finite. \(\square\)
Computing Events by Summing Their Outcomes
The theorem reduces probability calculations to identifying which outcomes belong to the event and adding their point masses. It also gives immediate checks: every mass is nonnegative, the total mass is one, and an event and its complement have probabilities adding to one. A point may have mass zero; countable additivity requires the total sum to be one, not that every point mass be positive.
Worked Example: A Geometric Waiting Time
Let \(\Omega=\{1,2,3,\ldots\}\), where the outcome \(n\) represents a waiting time of \(n\) steps. Fix \(q\) with \(0<q<1\), and assign \(p_n=(1-q)q^{n-1}\). Verify that these masses define a probability measure, then calculate the probability that the waiting time exceeds two steps when \(q=\tfrac13\).
For \(N\ge1\), the finite geometric sum is $$ \sum_{n=1}^{N}(1-q)q^{n-1} =(1-q)\frac{1-q^N}{1-q} =1-q^N. $$ Since \(0<q<1\), \(q^N\to0\), so the infinite sum of the masses is one. Each mass is nonnegative, and the theorem therefore defines a probability measure on \((\Omega,\mathcal{P}(\Omega))\).
The event that the waiting time exceeds two is \(\{3,4,5,\ldots\}\), with strict inequality \(n>2\). Its probability is $$ \mathbb{P}(n>2)=\sum_{n=3}^{\infty}(1-q)q^{n-1}=q^2. $$ For \(q=\tfrac13\), substitution gives \(\mathbb{P}(n>2)=(\tfrac13)^2=\tfrac19\). The complementary event is \(n\le2\), so its probability is \(p_1+p_2=\tfrac23+\tfrac29=\tfrac89\). Check: \(\tfrac19+\tfrac89=1\), as required for an event and its complement.
Worked Example: An Infinite Distribution on the Nonnegative Integers
Let \(\Omega=\{0,1,2,\ldots\}\) and assign \(p_n=2^{-(n+1)}\). Find the probability of an even outcome and of an odd outcome. In testing parity, even outcomes include \(0\).
The total mass is a geometric series: $$ \sum_{n=0}^{\infty}2^{-(n+1)} =\frac{\tfrac12}{1-\tfrac12} =1. $$ Thus these masses define a probability measure. The even outcomes are exactly \(n=2j\) for integers \(j\ge0\), so $$ \mathbb{P}(n\text{ even}) =\sum_{j=0}^{\infty}2^{-(2j+1)} =\frac{\tfrac12}{1-\tfrac14} =\frac23. $$ The odd outcomes are exactly \(n=2j+1\) for \(j\ge0\), giving $$ \mathbb{P}(n\text{ odd}) =\sum_{j=0}^{\infty}2^{-(2j+2)} =\frac{\tfrac14}{1-\tfrac14} =\frac13. $$ Check: the even and odd outcomes partition \(\Omega\), and \(\tfrac23+\tfrac13=1\). Also the first even outcome has mass \(p_0=2^{-1}=\tfrac12\), which is included in the even sum.
Worked Example: A Telescoping Discrete Distribution
On \(\Omega=\{1,2,3,\ldots\}\), assign \(p_n=\frac{1}{n(n+1)}\). Verify normalization and calculate the probability of the event \(n\ge4\).
For each \(n\ge1\), $$ \frac{1}{n(n+1)}=\frac1n-\frac1{n+1}. $$ Hence the first \(N\) masses sum to $$ \sum_{n=1}^{N}\frac{1}{n(n+1)} =1-\frac{1}{N+1}, $$ which tends to one. The masses are positive, so they define a probability measure.
The event \(n\ge4\) consists of the outcomes \(4,5,\ldots\). Its probability is the tail sum $$ \mathbb{P}(n\ge4) =\sum_{n=4}^{\infty}\left(\frac1n-\frac1{n+1}\right) =\frac14. $$ Check: the complementary event is \(n<4\), namely \(n=1,2,3\). Its probability is \(\frac12+\frac16+\frac1{12}=\frac{6+2+1}{12}=\frac34\), and \(\frac14+\frac34=1\).
What the Conditions Rule Out
The normalization condition is essential: nonnegative weights alone do not define a probability measure. For example, on \(\{1,2,\ldots\}\), the assignment \(p_n=2^{-n-1}\) has total mass \(\frac12\), since \(\sum_{n=1}^\infty 2^{-n-1}=(\frac14)/(1-\frac12)=\frac12\). It defines a finite measure by summing over subsets, but it is not a probability measure because its value on the whole space is not one.
A countably infinite space also cannot have the same point mass \(c\) at every outcome and still have total mass one. If \(c=0\), every finite subsum is zero, so the total is zero. If \(c>0\), the sum over the first \(N\) distinct outcomes is \(Nc\), which exceeds one once \(N>1/c\); therefore the total cannot equal one. Thus a probability measure on a countably infinite space must assign unequal masses or assign zero mass to some outcomes. Pitfall: saying “give each of infinitely many outcomes probability \(1/N\)” has no meaning unless a finite \(N\) is specified; there is no positive constant point mass whose countable sum is one.
For any event \(A\subseteq\Omega\), the calculation is always \(\mathbb{P}(A)=\sum_{\omega\in A}p_\omega\). If \(A\) contains infinitely many outcomes, the sum is still well-defined because its terms are nonnegative and its finite subsums are bounded above by the total mass one. No ordering-dependent cancellation is involved.
Check Your Understanding
In each question, use the point masses and specify the outcomes included in the event before summing.
- On \(\Omega=\{1,2,\ldots\}\), let \(p_n=3/4^n\). Verify that these masses sum to one, then find \(\mathbb{P}(n\ge3)\).
- For the geometric waiting-time masses \(p_n=(1-q)q^{n-1}\), where \(0<q<1\), calculate \(\mathbb{P}(n\le k)\) for a positive integer \(k\).
- On \(\Omega=\{0,1,2,\ldots\}\), with \(p_n=2^{-(n+1)}\), find the probability of an outcome divisible by three, including zero.
- Explain why equal positive point masses cannot define a probability measure on a countably infinite sample space.
- If \(\Omega\) is countable and \(\mathbb{P}\) is a probability measure on \(\mathcal{P}(\Omega)\), show that the masses \(p_\omega=\mathbb{P}(\{\omega\})\) determine \(\mathbb{P}(A)\) for every subset \(A\).