Random Sums by Conditioning on the Count

Conditioning Reveals the Distribution

A random sum has two sources of randomness: the number of terms and the values of the terms. If the count is independent of an identically distributed sequence, then fixing the count turns the random sum into an ordinary sum of a fixed number of independent variables. Its law is the corresponding convolution law. Averaging those conditional laws over the possible counts gives the unconditional distribution.

This representation is useful whether the summands are discrete, continuous, or a mixture of both. It also makes the role of a zero count explicit: if there are no terms, the sum is zero.

Definition: Let $N$ be a random variable taking values in $\{0,1,2,\ldots\}$, and let $X_1,X_2,\ldots$ be independent and identically distributed real-valued random variables, with $N$ independent of the entire sequence. The random sum is $$ S_N=\sum_{i=1}^{N}X_i, $$ where the empty sum is defined to be $S_0=0$. Write $p_n=P(N=n)$. For $n\geq 1$, let $F^{*n}$ denote the distribution of a sum of $n$ independent variables, each with the same distribution $F$ as $X_1$. Let $F^{*0}$ denote the point mass at zero.

The notation $F^{*n}$ refers to repeated convolution of probability distributions. For example, if the summands have densities, the density of a sum of two independent summands is the convolution of their densities. The key fact is that, conditional on $N=n$, the random sum has distribution $F^{*n}$.

The Mixture-of-Convolutions Formula

Theorem: Under the assumptions in the definition, for every Borel set $A\subseteq\mathbb{R}$, $$ P(S_N\in A)=\sum_{n=0}^{\infty}p_n F^{*n}(A). $$ Equivalently, the distribution of $S_N$ is the countable mixture $\sum_{n=0}^{\infty}p_n F^{*n}$. In particular, for every real $x$, $$ P(S_N\leq x)=\sum_{n=0}^{\infty}p_n F^{*n}(x). $$

Each component describes the sum when the count is fixed at a particular value, and its mixture weight is the probability of that count. The term for $n=0$ is important: $F^{*0}(A)$ equals $1$ if $0\in A$ and $0$ otherwise.

Proof. The events $\{N=n\}$, for $n=0,1,2,\ldots$, are disjoint and have union equal to the whole sample space. Therefore, $$ P(S_N\in A)=\sum_{n=0}^{\infty}P(S_N\in A,\ N=n). $$ On the event $\{N=n\}$, the random sum equals $S_n=\sum_{i=1}^{n}X_i$, with $S_0=0$. Independence of $N$ from the sequence gives $$ P(S_N\in A,\ N=n)=P(S_n\in A,\ N=n) =P(S_n\in A)P(N=n) =F^{*n}(A)p_n. $$ Substituting this equality into the countable sum proves the formula. Taking $A=(-\infty,x]$ gives the CDF formula. $\square$

For discrete summands, the theorem immediately gives a probability-mass formula. If each $F^{*n}$ has mass function $f^{*n}$, then at any point $s$, $$ P(S_N=s)=\sum_{n=0}^{\infty}p_n f^{*n}(s), $$ where $f^{*0}(s)=1$ at $s=0$ and is zero elsewhere. The sum over $n$ is legitimate because it adds nonnegative probabilities, and its total mass is $\sum_n p_n=1$.

Density Formula and the Zero-Count Atom

There is an analogous density formula when the summands have a density $f$. For $n\geq 1$, write $f^{*n}$ for the density of the sum of $n$ independent summands. It can be obtained recursively from $f^{*1}=f$ and $$ f^{*(n+1)}(s)=\int_{-\infty}^{\infty} f^{*n}(s-y)f(y)\,dy. $$ This convolution formula follows by integrating the joint density of the independent sum and its last summand.

Theorem: Suppose $X_1,X_2,\ldots$ are independent and identically distributed with a density $f$, and are independent of the nonnegative integer-valued $N$. For every Borel set $A$, $$ P(S_N\in A)=p_0\,\delta_0(A)+\int_A\left\{\sum_{n=1}^{\infty}p_n f^{*n}(s)\right\}\,ds, $$ where $\delta_0$ is the point mass at zero. Thus the continuous part has density $\sum_{n\geq1}p_n f^{*n}$; if $p_0>0$, the full distribution also has an atom of size $p_0$ at zero.

Proof. Apply the mixture formula and separate the $n=0$ term. For each $n\geq1$, $F^{*n}$ has density $f^{*n}$, so $$ P(S_N\in A)=p_0\delta_0(A)+\sum_{n=1}^{\infty}p_n\int_A f^{*n}(s)\,ds. $$ The integrands and weights are nonnegative. Tonelli's theorem therefore permits exchanging the sum and integral: $$ \sum_{n=1}^{\infty}p_n\int_A f^{*n}(s)\,ds =\int_A\sum_{n=1}^{\infty}p_n f^{*n}(s)\,ds. $$ This proves the stated decomposition. $\square$

The atom statement does not say that all probability at zero must come from $N=0$. If a summand can equal zero, or if different summands can cancel, some laws $F^{*n}$ for $n\geq1$ may also assign probability to zero. The displayed atom $p_0\delta_0$ is the contribution guaranteed by the zero-count case.

Worked Distributions

Worked Example: A Bernoulli Count and Bernoulli Summands

Let $N$ have probabilities $P(N=0)=0.2$, $P(N=1)=0.5$, and $P(N=2)=0.3$. Independently, let each $X_i$ be Bernoulli with success probability $0.4$. Find the distribution of $S_N$.

For $N=0$, the sum is zero with probability one. For $N=1$, its probabilities at $0$ and $1$ are $0.6$ and $0.4$. For $N=2$, the sum has a binomial distribution with two trials: its probabilities at $0,1,2$ are $$ (0.6)^2=0.36,\qquad 2(0.4)(0.6)=0.48,\qquad (0.4)^2=0.16. $$

Mixing these distributions over the count gives $$ P(S_N=0)=0.2+0.5(0.6)+0.3(0.36)=0.608, $$ $$ P(S_N=1)=0.5(0.4)+0.3(0.48)=0.344, \qquad P(S_N=2)=0.3(0.16)=0.048. $$ The probabilities sum to $0.608+0.344+0.048=1$, as required. In particular, the probability at zero includes both the zero-count case and cases with a positive count but no successes.

Worked Example: An Exponential Random Sum With an Atom

Suppose $P(N=0)=0.25$, $P(N=1)=0.5$, and $P(N=2)=0.25$. Let the independent summands have an exponential distribution with rate $2$, whose density is $f(x)=2e^{-2x}$ for $x>0$ and zero otherwise. Find the atom and continuous density of $S_N$.

When $N=1$, the density is $2e^{-2x}$. When $N=2$, convolution gives, for $x>0$, $$ f^{*2}(x)=\int_0^x 2e^{-2y}\,2e^{-2(x-y)}\,dy =4xe^{-2x}. $$ The count-zero component contributes an atom of size $0.25$ at zero. For $x>0$, the density is the weighted sum $$ 0.5(2e^{-2x})+0.25(4xe^{-2x}) =e^{-2x}(1+x). $$

As a check, the continuous component has total probability $0.5+0.25=0.75$: the weighted exponential density integrates to $0.5$, and the weighted two-summand density integrates to $0.25$. Adding the atom gives total probability $0.75+0.25=1$. For $x\geq0$, the CDF can also be written $$ P(S_N\leq x)=1-e^{-2x}\left(0.75+0.5x\right). $$ At $x=0$, this equals $0.25$, the mass at zero.

Worked Example: A Mixture of Uniform and Triangular Laws

Let $P(N=1)=0.25$ and $P(N=2)=0.75$. Let each $X_i$ be independent and uniform on $(0,1)$, independently of $N$. Find the density of $S_N$.

For $N=1$, the density is $1$ on $(0,1)$. For $N=2$, the convolution density at $s$ is the length of the set of $u$ satisfying both $0<u<1$ and $0<s-u<1$. When $0<s<1$, this set has length $s$; when $1<s<2$, it has length $2-s$. Thus $$ f^{*2}(s)= \begin{cases} s,&0<s<1,\\ 2-s,&1<s<2,\\ 0,&\text{otherwise}. \end{cases} $$

The mixture density is therefore $$ f_{S_N}(s)= \begin{cases} 0.25+0.75s,&0<s<1,\\ 0.75(2-s),&1<s<2,\\ 0,&\text{otherwise}. \end{cases} $$ There is no atom at zero because $N\geq1$ and each uniform summand is positive with probability one. As a check, the first interval contributes $0.25+0.375=0.625$ to the integral, and the second contributes $0.375$; the total is $1$.

Common Pitfall: Averaging the Conditional Laws Incorrectly

The mixture formula averages distributions using the probabilities of the count. It does not say that the random sum has the law of one convolution selected by an “average count.” For example, if $N$ can equal zero or two, the conditional laws are a point mass at zero and a two-fold convolution; an intermediate noninteger count does not define the required law. The correct calculation retains each possible integer count and its own weight.

Another error is to omit $n=0$ because a sum is imagined to contain at least one term. Under the convention $S_0=0$, omitting this component loses probability $p_0$ and can remove an atom from the distribution. Always check whether the count's support includes zero before writing a density for the entire law.

Key takeaway: For an independent count $N$ and an iid sequence of summands, condition on $N=n$, use the $n$-fold convolution law, and mix with weights $P(N=n)$. Include the point mass at zero when $N=0$ has positive probability.

Check Your Understanding

Use conditioning on the count to answer each question.

State the distribution of $S_N$ when $N=0$ with probability one.
If $P(N=0)=0.1$, $P(N=1)=0.9$, and the summands have density $f$, write the atom and continuous density components of the law of $S_N$.
For a count taking values $0,1,2$, explain which convolution law corresponds to each possible count.
Why does a density formula that includes only $n\geq1$ fail to describe the full distribution when $P(N=0)>0$?
Suppose the summands can be zero with positive probability. Can the probability $P(S_N=0)$ exceed $P(N=0)$? Explain using the mixture formula.