Countable Additivity as a Series Rule

From Finite Sums to Infinite Unions

For a finite collection of disjoint events, the probability of their union is the sum of their probabilities. With infinitely many events, the corresponding sum must be understood as a limit of finite partial sums. Countable additivity is the rule that makes this passage precise: it connects the probability of an infinite union to probabilities that can be calculated at finite stages.

The disjointness condition matters. If events overlap, adding their probabilities generally counts some outcomes more than once. An increasing sequence of events may overlap, but its new portion at each stage can be separated off to produce disjoint events. This gives a useful limit rule for probabilities of increasing unions.

Definition: A sequence of events $A_1,A_2,\ldots$ is pairwise disjoint if $A_i\cap A_j=\varnothing$ whenever $i\ne j$. A probability model is countably additive if, for every pairwise disjoint sequence of events, $$ P\left(\bigcup_{k=1}^{\infty} A_k\right)=\sum_{k=1}^{\infty}P(A_k), $$ where the infinite sum means the limit of its partial sums: $$ \sum_{k=1}^{\infty}P(A_k)=\lim_{n\to\infty}\sum_{k=1}^{n}P(A_k). $$

Because each probability is nonnegative, the partial sums on the right cannot decrease. They are also at most $1$: finite additivity gives $\sum_{k=1}^{n}P(A_k)=P(\bigcup_{k=1}^{n}A_k)\le 1$. Thus these partial sums have a finite limit. Countable additivity says that this limit is exactly the probability of the full union, not merely a convenient approximation to it.

Worked Examples with Disjoint Events

Worked Example: The First Head in a Sequence of Tosses

Consider an infinite sequence of independent fair coin tosses. For each positive integer $k$, let $E_k$ be the event that the first head occurs on toss $k$. Find the probability that a head occurs eventually, and the probability that the first head occurs on toss 4 or later.

Disjointness. The first head cannot occur on two different tosses, so the events $E_1,E_2,\ldots$ are pairwise disjoint. For the first head to occur on toss $k$, the first $k-1$ tosses must be tails and toss $k$ must be a head. Independence and fairness give $$ P(E_k)=\left(\frac12\right)^{k-1}\frac12=2^{-k}. $$

Eventually a head. For a finite number $n$ of possible first-head positions, the geometric sum is $$ P\left(\bigcup_{k=1}^{n}E_k\right) =\sum_{k=1}^{n}2^{-k} =\frac12\sum_{j=0}^{n-1}\left(\frac12\right)^j =1-2^{-n}. $$ As $n$ increases, the finite union increases to $\bigcup_{k=1}^{\infty}E_k$, the event that a head eventually occurs. Countable additivity therefore gives $$ P\left(\bigcup_{k=1}^{\infty}E_k\right) =\lim_{n\to\infty}(1-2^{-n})=1. $$

First head on toss 4 or later. These positions correspond to the disjoint events $E_4,E_5,\ldots$, so $$ P\left(\bigcup_{k=4}^{\infty}E_k\right) =\sum_{k=4}^{\infty}2^{-k} =\frac{1}{16}\sum_{j=0}^{\infty}\left(\frac12\right)^j =\frac{1}{8}. $$ The starting term is $2^{-4}=1/16$; the geometric series multiplying it sums to $2$, giving $1/8$.

Worked Example: A Tail Event in a Countable Model

Let $X$ take values in the positive integers with $P(X=k)=2^{-k}$ for $k=1,2,\ldots$. This is a probability assignment because its total mass is $\sum_{k=1}^{\infty}2^{-k}=1$. Find $P(X\ge 4)$ by taking limits of finite unions.

For each $k\ge 4$, let $A_k=\{X=k\}$. These singleton events are pairwise disjoint. For $n\ge4$, define $B_n=\bigcup_{k=4}^{n}A_k$. Then $B_n$ increases with $n$, and its union is $\{X\ge4\}$. The finite-stage probability is $$ P(B_n)=\sum_{k=4}^{n}2^{-k} =\frac{1}{16}\sum_{j=0}^{n-4}\left(\frac12\right)^j =\frac{1}{8}\left(1-2^{-(n-3)}\right). $$

For example, when $n=4$, this expression is $\frac18(1-\frac12)=\frac1{16}$, which agrees with $P(B_4)=P(X=4)=2^{-4}$. Taking the limit gives $$ P(X\ge4)=\lim_{n\to\infty}P(B_n) =\frac18\left(1-\lim_{n\to\infty}2^{-(n-3)}\right) =\frac18. $$

Increasing Unions and Continuity from Below

A sequence $A_1,A_2,\ldots$ is increasing if $A_n\subseteq A_{n+1}$ for every $n$. Its events need not be disjoint. Nevertheless, the probability of the union is the limit of the probabilities of the finite stages. This result is called continuity from below.

Theorem (Continuity from Below): If $A_1,A_2,\ldots$ is an increasing sequence of events in a probability model, then $$ P\left(\bigcup_{n=1}^{\infty}A_n\right)=\lim_{n\to\infty}P(A_n). $$

Proof. Define $C_1=A_1$, and for $n\ge2$, define $C_n=A_n\setminus A_{n-1}$. These are the new outcomes added at each stage. They are pairwise disjoint: if $i<j$, then $C_i\subseteq A_i\subseteq A_{j-1}$, while $C_j$ is disjoint from $A_{j-1}$. For each $n$, the new portions through stage $n$ make up exactly $A_n$, so $A_n=\bigcup_{k=1}^{n}C_k$. Also, $\bigcup_{n=1}^{\infty}A_n=\bigcup_{k=1}^{\infty}C_k$, since every outcome that enters any $A_n$ belongs to the new portion $C_k$ at its first stage of entry. By finite additivity, $P(A_n)=\sum_{k=1}^{n}P(C_k)$. By countable additivity, $$ P\left(\bigcup_{n=1}^{\infty}A_n\right) =\sum_{k=1}^{\infty}P(C_k) =\lim_{n\to\infty}\sum_{k=1}^{n}P(C_k) =\lim_{n\to\infty}P(A_n). $$ This proves the result. $\square$

Worked Example: An Increasing Sequence of Integer Events

Use the countable model $P(X=k)=2^{-k}$ for positive integers. Let $A_n=\{X\le n\}$. These events overlap and increase as $n$ grows. Find the probability of their union by calculating the finite-stage probabilities.

For each $n\ge1$, the event $A_n$ consists of the outcomes $1,\ldots,n$, so $$ P(A_n)=\sum_{k=1}^{n}2^{-k}=1-2^{-n}. $$ The union over all positive integers $n$ is the entire sample space of possible values of $X$. Continuity from below yields $$ P\left(\bigcup_{n=1}^{\infty}A_n\right) =\lim_{n\to\infty}(1-2^{-n})=1. $$ Here the events $A_n$ are not disjoint; it is the increasing-union theorem, rather than direct addition of $P(A_n)$, that justifies the limit.

Worked Example: The Probability of an Even Integer

Again let $P(X=k)=2^{-k}$ for $k=1,2,\ldots$. For each $j\ge1$, let $D_j=\{X=2j\}$. These singleton events are pairwise disjoint, and their union is the event that $X$ is even. Therefore $$ P(X\text{ is even}) =\sum_{j=1}^{\infty}P(D_j) =\sum_{j=1}^{\infty}2^{-2j} =\sum_{j=1}^{\infty}\left(\frac14\right)^j =\frac{\frac14}{1-\frac14} =\frac13. $$ The first term is $1/4$, and each later term is one quarter of the previous term. The infinite sum is the limit of its finite partial sums, as required by countable additivity.

Why the Rule Matters—and a Common Pitfall

Countable additivity lets a probability model handle events described by infinitely many possibilities. In applications, one often calculates a sequence of finite-stage probabilities and then takes a limit. Continuity from below explains when that procedure gives the probability of the limiting event: the events must be increasing, and the limit is taken for their probabilities, not by adding those probabilities together.

Pitfall: adding probabilities of overlapping events. A mistaken argument might say that for increasing events $A_n$, $P(\bigcup_n A_n)=\sum_n P(A_n)$, because the union contains every $A_n$. This generally counts outcomes repeatedly. For $A_n=\{X\le n\}$ above, the union has probability $1$, whereas the sum $\sum_n P(A_n)=\sum_n(1-2^{-n})$ diverges.
Correction: Direct countable addition applies to pairwise disjoint events. For increasing events, use continuity from below: $P(\bigcup_n A_n)=\lim_nP(A_n)$. Alternatively, separate out the disjoint increments $C_n=A_n\setminus A_{n-1}$ and add their probabilities.
Pitfall: stopping at a finite partial sum. A partial sum gives the probability of a finite union, not automatically the probability of the infinite union. To obtain the latter, establish the limit and use countable additivity or continuity from below.

Key takeaway: For pairwise disjoint events, the probability of a countable union is the sum of the event probabilities, with the sum defined as a limit of partial sums. For increasing events, the probability of the union is the limit of the finite-stage probabilities.

Check Your Understanding

For each question, identify whether the events are disjoint or increasing and state the probability rule that applies.

Events $E_1,E_2,\ldots$ are pairwise disjoint, and $P(E_k)=3^{-k}$. Write $P(\bigcup_{k=1}^{\infty}E_k)$ as a series, and state what the infinite sum means.
For increasing events $A_n$, explain why $P(\bigcup_n A_n)=\lim_nP(A_n)$ does not say that $P(\bigcup_n A_n)=\sum_nP(A_n)$.
In the model $P(X=k)=2^{-k}$, calculate the finite-stage probability $P(4\le X\le n)$ for $n\ge4$, then give its limit as $n$ tends to infinity.
Describe how to turn an increasing sequence $A_n$ into a sequence of pairwise disjoint events whose union is unchanged.
A sequence of events has partial-union probabilities that converge to $0.6$. What additional condition lets countable additivity identify $0.6$ as the probability of the full union?