When Random Counts and Summands Are Dependent

Why Conditioning on the Count Matters

For a random sum $S=\sum_{i=1}^{N}X_i$, the count $N$ determines how many terms are included. Familiar random-sum formulas often assume that $N$ is independent of a sequence of identically distributed summands. Without that assumption, the law of a summand may change with the count, and summands in the same count stratum may also be dependent on one another. The reliable starting point is to condition on $N$, then use the distribution and moments that actually hold at each count.

Throughout, $N$ takes values in $\{0,1,2,\ldots\}$, $P(N=n)=p_n$, and the empty sum is defined by $S=0$ when $N=0$. For any $n$ with $p_n>0$, probabilities and expectations conditional on $N=n$ describe the count-$n$ stratum. No independence between $N$ and the $X_i$, or among the $X_i$, is assumed unless stated.

Definition: The count-conditional law of the summands is the joint distribution of $(X_1,\ldots,X_n)$ given $N=n$, for each $n$ with $p_n>0$. The count and summands are independent only if these conditional laws agree with the corresponding unconditional laws in the required sense; a common way this occurs is when the sequence of summands is independent of $N$. Conditional independence of the summands given $N=n$ is a separate property and does not follow from either the definition of a random sum or dependence between $N$ and the summands.

The Distribution as a Mixture Over Counts

The distribution of $S$ is obtained by mixing its count-conditional distributions using the probabilities $p_n$. This statement does not require the summands to be identically distributed or conditionally independent.

Theorem: For every measurable set $A$, $$ P(S\in A)=\sum_{n=0}^{\infty}p_n P\!\left(\sum_{i=1}^{n}X_i\in A\mid N=n\right), $$ where the $n=0$ sum is zero. In particular, for every real $s$, $$ F_S(s)=\sum_{n=0}^{\infty}p_n F_{S\mid N=n}(s). $$

To prove the result, partition the event $\{S\in A\}$ according to the mutually exclusive count events $\{N=n\}$. Countable additivity gives

P(S\in A) =\sum_{n=0}^{\infty}P(S\in A,N=n) =\sum_{n=0}^{\infty}P(N=n)P(S\in A\mid N=n).

On $\{N=n\}$, $S=\sum_{i=1}^{n}X_i$, so this is the claimed formula. The terms are nonnegative and sum to at most one, so the countable mixture is well-defined. Taking $A=(-\infty,s]$ gives the distribution-function formula. $\square$

If the summands are conditionally independent given $N=n$, their conditional convolution determines $F_{S\mid N=n}$. For example, when the relevant characteristic functions exist, conditional independence yields

\varphi_{S\mid N=n}(t)=\prod_{i=1}^{n}\varphi_{X_i\mid N=n}(t).

Without conditional independence, the characteristic function is instead the joint conditional expectation $E[\exp\{it\sum_{i=1}^{n}X_i\}\mid N=n]$; it need not factor into marginal characteristic functions.

Worked Example: A Count-Dependent Exponential Model

Let $P(N=1)=P(N=2)=1/2$. Given $N=1$, let $X_1$ have an exponential distribution with rate $2$. Given $N=2$, let $X_1$ and $X_2$ be conditionally independent exponential variables, each with rate $1$. Find the distribution function of $S$ and its mean.

For $s\geq 0$, the conditional distribution given $N=1$ is exponential with rate $2$, so $F_{S\mid N=1}(s)=1-e^{-2s}$. Given $N=2$, $S=X_1+X_2$ has a gamma distribution with shape $2$ and rate $1$. Its density is $se^{-s}$ for $s\geq0$, and integrating gives $F_{S\mid N=2}(s)=1-e^{-s}(1+s)$.

Mixing these conditional distributions,

F_S(s)= \begin{cases} 0, & s<0,\\[2mm] \frac12(1-e^{-2s})+\frac12\{1-e^{-s}(1+s)\}, & s\geq 0. \end{cases}

The count-conditional means are $E[S\mid N=1]=1/2$ and $E[S\mid N=2]=2$. Therefore,

E[S]=\frac12\left(\frac12\right)+\frac12(2)=\frac54.

The mean is not obtained by multiplying $E[N]=3/2$ by a single unconditional summand mean: the summand distribution changes with $N$. Conditioning keeps those different laws distinct.

Conditional Means and Their Conditions

For each $n$ with $p_n>0$, write $m_n=E[S\mid N=n]$ when this conditional expectation is defined. If $E|S|<\infty$, each $m_n$ is finite and the unconditional mean is the probability-weighted sum of these conditional means. This assertion concerns the sum $S$, not necessarily the individual summands.

Theorem: If $E|S|<\infty$, then $$ E[S]=\sum_{n:p_n>0}p_n m_n, \qquad m_n=E[S\mid N=n]. $$ If, in addition, each $X_i\mid N=n$ is integrable for every $i\leq n$ and every $n$ with $p_n>0$, then $$ m_n=\sum_{i=1}^{n}\mu_{i,n}, \qquad \mu_{i,n}=E[X_i\mid N=n]. $$

For the first equality, the events $\{N=n\}$ partition the sample space. Absolute integrability ensures that the expectation can be split over this countable partition:

E[S]=\sum_{n=0}^{\infty}E[S\,\mathbf{1}_{\{N=n\}}] =\sum_{n:p_n>0}p_n E[S\mid N=n].

The series is absolutely convergent because $\sum_n E[|S|\mathbf{1}_{\{N=n\}}]=E|S|<\infty$. For the second equality, the finite sum $\sum_{i=1}^{n}X_i$ has integrable terms under the stated assumptions. Linearity of conditional expectation therefore gives

E[S\mid N=n] =E\!\left[\sum_{i=1}^{n}X_i\mid N=n\right] =\sum_{i=1}^{n}E[X_i\mid N=n].

If the summands are nonnegative, the corresponding expectation identity also holds with extended nonnegative expectations by additivity for finite sums and Tonelli's theorem for the countable mixture. In particular, $E[S]=\sum_n p_n m_n$ is valid as an extended expectation for nonnegative $S$, whether or not it is finite. $\square$

When each count stratum has conditionally identically distributed summands with conditional mean $\mu_n$, the additional condition gives $m_n=n\mu_n$, and hence $E[S]=\sum_n p_n n\mu_n$ when the mean is finite. The mean $\mu_n$ may depend on $n$; replacing it by a common mean requires a justification such as independence of the summand sequence and $N$.

Worked Example: Count-Dependent Bernoulli Summands

Suppose $P(N=0)=0.2$, $P(N=1)=0.5$, and $P(N=2)=0.3$. Given $N=1$, let $X_1$ be Bernoulli with success probability $0.2$. Given $N=2$, let $X_1,X_2$ be conditionally independent Bernoulli variables with success probability $0.8$. Find the distribution and mean of $S$.

When $N=0$, $S=0$. When $N=1$, the conditional probabilities of $S=0,1$ are $0.8,0.2$. When $N=2$, $S$ is binomial with two trials and success probability $0.8$, so its conditional probabilities at $0,1,2$ are $0.2^2=0.04$, $2(0.8)(0.2)=0.32$, and $0.8^2=0.64$.

Weighting by the count probabilities gives

\begin{aligned} P(S=0)&=0.2+0.5(0.8)+0.3(0.04)=0.612,\\ P(S=1)&=0.5(0.2)+0.3(0.32)=0.196,\\ P(S=2)&=0.3(0.64)=0.192. \end{aligned}

These probabilities sum to $0.612+0.196+0.192=1$. The conditional means are $m_0=0$, $m_1=0.2$, and $m_2=2(0.8)=1.6$. Thus

$$ E[S]=0.2(0)+0.5(0.2)+0.3(1.6)=0.58. $$

The conditional Bernoulli success probability is different at counts one and two. The mixture formula accommodates that dependence directly.

Conditional Variance and Dependence Within a Stratum

If $E[S^2]<\infty$, the law of total variance applies to $S$ itself:

Theorem: If $E[S^2]<\infty$, then $$ \operatorname{Var}(S) =E[\operatorname{Var}(S\mid N)] +\operatorname{Var}(E[S\mid N]). $$ For a fixed $n$, if each $X_i\mid N=n$ has a finite second moment, then $$ \operatorname{Var}(S\mid N=n) =\sum_{i=1}^{n}\operatorname{Var}(X_i\mid N=n) +2\sum_{1\leq i<j\leq n}\operatorname{Cov}(X_i,X_j\mid N=n). $$

For the first identity, write $m(N)=E[S\mid N]$. The defining property of conditional expectation gives $E[m(N)]=E[S]$. Expanding the squared deviation and conditioning on $N$, the cross term vanishes because $E[S-m(N)\mid N]=0$:

\begin{aligned} E[(S-E[S])^2] &=E[(S-m(N))^2]+E[(m(N)-E[S])^2]\\ &\quad+2E[(S-m(N))(m(N)-E[S])]\\ &=E[\operatorname{Var}(S\mid N)] +\operatorname{Var}(m(N)). \end{aligned}

The cross term is zero since $E[(S-m(N))(m(N)-E[S])]=E[(m(N)-E[S])E[S-m(N)\mid N]]=0$. The finite second moment ensures these quantities are defined. For the second identity, expand the square of $\sum_{i=1}^{n}(X_i-E[X_i\mid N=n])$; its diagonal terms are the conditional variances, and each off-diagonal product occurs twice and has conditional expectation equal to the corresponding covariance. Finite conditional second moments make every displayed term well-defined. $\square$

In particular, $E[S^2]<\infty$ is enough for the law of total variance for $S$, but does not by itself guarantee that each summand has a finite conditional second moment. Use the expanded covariance formula only when its individual terms exist. Conditional independence is one sufficient condition for the covariances to vanish, but dependence within a count stratum can make them substantial.

Worked Example: Perfectly Dependent Summands Given the Count

Let $P(N=1)=0.4$ and $P(N=2)=0.6$. Given $N=1$, let $X_1$ be Bernoulli with success probability $1/2$. Given $N=2$, let $X_1=X_2=Z$, where $Z$ is Bernoulli with success probability $1/2$. Find the mean and variance of $S$.

For $N=1$, $S=X_1$ has mean $0.5$ and variance $0.25$. For $N=2$, $S=2Z$, so its mean is $1$ and its variance is $4\operatorname{Var}(Z)=4(0.25)=1$. Therefore $E[S]=0.4(0.5)+0.6(1)=0.8$. The expected conditional variance is $0.4(0.25)+0.6(1)=0.7$.

The conditional means are $0.5$ with probability $0.4$ and $1$ with probability $0.6$. Their variance is

$$ 0.4(0.5-0.8)^2+0.6(1-0.8)^2 =0.4(0.09)+0.6(0.04)=0.06. $$

Thus $\operatorname{Var}(S)=0.7+0.06=0.76$. Check directly: $P(S=0)=0.4(0.5)+0.6(0.5)=0.5$, $P(S=1)=0.4(0.5)=0.2$, and $P(S=2)=0.6(0.5)=0.3$. Hence $E[S^2]=0.2+4(0.3)=1.4$, and $1.4-(0.8)^2=0.76$, as required. Given $N=2$, the two summands are perfectly dependent; treating them as conditionally independent would give the wrong variance.

Common Mistakes and a Careful Checklist

Using a common summand law without checking it. A formula such as $E[S]=E[N]E[X_1]$ needs assumptions that justify treating the summand mean as unchanged by the count. In the dependent setting, use $E[S\mid N=n]$ and mix over $n$.
Dropping conditional covariances. The conditional variance of a sum includes covariance terms unless conditional independence (or another zero-covariance condition) has been established.
Inferring summand integrability from integrability of the sum. For example, let $N=2$ surely, let $Y$ have a Cauchy distribution, and set $X_1=Y$, $X_2=-Y$. Then $S=0$ is integrable, but neither summand has a finite mean. Consequently, $E[S\mid N=2]$ exists while $E[X_1\mid N=2]$ and $E[X_2\mid N=2]$ do not.
Inferring summand second moments from the sum's second moment. In the same example, $E[S^2]=0$, but the individual summands have no finite second moments. The variance identity for $S$ remains meaningful; the expansion into individual variances and covariances does not.

A safe calculation proceeds in order: determine the count probabilities, specify the conditional joint law of the summands for each possible count, calculate the conditional distribution or moments of the sum, and then average over the count. State integrability or second-moment assumptions before replacing a conditional moment of the sum by a sum of individual moments.

Check Your Understanding

Use conditioning on the count and check which moment assumptions are needed.

Why does the mixture formula for $F_S$ not require conditional independence of the summands?
If $E|S|<\infty$, which equality is always justified: $E[S]=\sum_n p_nE[S\mid N=n]$, or also $E[S\mid N=n]=\sum_{i=1}^{n}E[X_i\mid N=n]$? State the extra condition for the second equality.
For a fixed $n$, what additional terms appear in $\operatorname{Var}(S\mid N=n)$ when the summands are dependent?
Can $E[S^2]<\infty$ hold while a summand has no finite conditional second moment? Explain using a cancellation construction.
In the Bernoulli worked example, verify $E[S]$ from the final probability mass function and compare it with the count-conditional calculation.