Negative-Binomial Moments from Independent Waiting Times

Why Break a Negative-Binomial Count into Waiting Times?

A negative-binomial count records how many failures occur before a specified number of successes. Its moments are especially transparent when viewed as a sequence of waiting periods: count the failures before the first success, then the failures between the first and second successes, and continue until the \(r\)th success. Each waiting period has a geometric distribution, and the periods are independent under independent Bernoulli trials.

This decomposition turns a potentially less familiar count into a sum of simpler random variables. It also makes clear why the mean and variance grow in proportion to the target number of successes.

Definition and Waiting-Count Decomposition

Definition: Suppose an infinite sequence of independent Bernoulli trials has success probability \(p\), where \(0<p\leq 1\), and let \(q=1-p\). For a positive integer \(r\), let \(X\) be the number of failures observed before the \(r\)th success. Its support is \(\{0,1,2,\ldots\}\). For \(j=1,\ldots,r\), let \(G_j\) be the number of failures after the \((j-1)\)st success and before the \(j\)th success, with the start of the sequence used when \(j=1\). Then

X=\sum_{j=1}^{r}G_j.

The count \(G_j\) is a geometric waiting count in the failures-before-success convention. In particular, for \(k=0,1,2,\ldots\),

$$ P(G_j=k)=q^k p. $$

When \(p>0\), every waiting period ends with probability one: the probability of at least \(m\) consecutive failures before a success is \(q^m\), which tends to zero as \(m\) tends to infinity. Thus the \(r\) waiting counts are finite with probability one.

Worked Example: Identifying the Components

A sequence of independent trials has success probability \(p=0.4\). The goal is to stop at the third success. Suppose the observed trial sequence up to that point is \(F,F,S,S,F,S\), where \(F\) denotes failure and \(S\) denotes success. Find \(G_1,G_2,G_3\) and \(X\).

Before the first success, the sequence contains two failures, so \(G_1=2\). Between the first and second successes there are no failures, so \(G_2=0\). Between the second and third successes there is one failure, so \(G_3=1\). Therefore

$$ X=G_1+G_2+G_3=2+0+1=3. $$

The three components count only failures. The three successes determine the stopping point but are not included in \(X\).

Why the Waiting Counts Are Independent

Each \(G_j\) has the same geometric distribution, but the moment calculation also needs independence. That fact follows directly from the probabilities of the trial sequences that define the waiting periods.

Result: Under independent Bernoulli trials with success probability \(p>0\), \(G_1,\ldots,G_r\) are independent and identically distributed, with \(P(G_j=k)=q^k p\) for every nonnegative integer \(k\).

To verify the result, choose any nonnegative integers \(k_1,\ldots,k_r\). The event \(G_1=k_1,\ldots,G_r=k_r\) requires exactly \(k_1\) failures followed by a success, then \(k_2\) failures followed by a success, and so on. It therefore specifies a finite sequence containing \(k_1+\cdots+k_r\) failures and \(r\) successes. By independence of the Bernoulli trials,

P(G_1=k_1,\ldots,G_r=k_r) =q^{k_1+\cdots+k_r}p^r =\prod_{j=1}^{r}(q^{k_j}p).

Each factor \(q^{k_j}p\) is \(P(G_j=k_j)\). The joint probability thus factors into the marginal probabilities for every possible tuple \((k_1,\ldots,k_r)\), which proves both the claimed marginal distributions and independence.

Moments of One Geometric Waiting Count

For completeness, the required moments follow from the geometric probability mass function. Let \(G\) have \(P(G=k)=q^k p\) for \(k=0,1,2,\ldots\), where \(0<p\leq1\) and \(q=1-p\). Since \(0\leq q<1\), the geometric series and its derivatives converge, giving

\sum_{k=0}^{\infty}q^k=\frac{1}{1-q},\qquad \sum_{k=0}^{\infty}kq^k=\frac{q}{(1-q)^2},\qquad \sum_{k=0}^{\infty}k^2q^k=\frac{q(1+q)}{(1-q)^3}.

The last two identities follow by differentiating the geometric-series identity for \(|q|<1\): first multiply its derivative by \(q\) to obtain the sum involving \(k\), then differentiate that sum and multiply by \(q\) to obtain the sum involving \(k^2\). Using \(1-q=p\), we obtain the first and second raw moments.

E[G]=\sum_{k=0}^{\infty}kq^k p =p\frac{q}{(1-q)^2} =\frac{q}{p},

E[G^2]=\sum_{k=0}^{\infty}k^2q^k p =p\frac{q(1+q)}{(1-q)^3} =\frac{q(1+q)}{p^2}.

Consequently, subtracting the squared mean from the second raw moment gives the variance:

\operatorname{Var}(G) =E[G^2]-(E[G])^2 =\frac{q(1+q)}{p^2}-\frac{q^2}{p^2} =\frac{q}{p^2}.

These identities also cover \(p=1\): then \(q=0\), \(G=0\) with probability one, and both its mean and variance are zero.

Mean and Variance of the Negative-Binomial Count

Theorem: If \(X\) counts the failures before the \(r\)th success in independent Bernoulli trials with success probability \(0<p\leq1\), for a positive integer \(r\), then

E[X]=\frac{rq}{p}, \qquad \operatorname{Var}(X)=\frac{rq}{p^2}, \qquad q=1-p.

Proof. The waiting-time decomposition gives \(X=\sum_{j=1}^{r}G_j\). Each \(G_j\) has mean \(q/p\), so linearity of expectation for a finite sum yields

E[X]=\sum_{j=1}^{r}E[G_j] =\sum_{j=1}^{r}\frac{q}{p} =\frac{rq}{p}.

For the variance, independence implies \(\operatorname{Cov}(G_i,G_j)=0\) for \(i\ne j\). Applying the variance formula for a finite sum therefore gives

\operatorname{Var}(X) =\sum_{j=1}^{r}\operatorname{Var}(G_j) =\sum_{j=1}^{r}\frac{q}{p^2} =\frac{rq}{p^2}.

When \(p=1\), all waiting counts are zero, and the same formulas give zero for both moments. This proves the result for the full stated range. \(\square\)

Worked Example: Mean and Variance for Three Successes

Let \(X\) count the failures before the third success when each independent trial succeeds with probability \(p=0.4\). Here \(r=3\) and \(q=1-0.4=0.6\). The mean is

E[X]=\frac{rq}{p} =\frac{3(0.6)}{0.4} =\frac{1.8}{0.4} =4.5.

The variance is

\operatorname{Var}(X)=\frac{rq}{p^2} =\frac{3(0.6)}{(0.4)^2} =\frac{1.8}{0.16} =11.25.

Equivalently, there are three independent waiting counts, each with mean \(0.6/0.4=1.5\) and variance \(0.6/(0.4)^2=3.75\). Their means sum to \(3(1.5)=4.5\), and their variances sum to \(3(3.75)=11.25\), as required.

Worked Example: A High Success Probability

Suppose the target is \(r=4\) successes and each trial succeeds with probability \(p=0.8\). Then \(q=0.2\). Each waiting count has mean

E[G_j]=\frac{0.2}{0.8}=0.25,

so the total failure count has mean \(4(0.25)=1\). Each waiting count has variance

\operatorname{Var}(G_j)=\frac{0.2}{(0.8)^2} =\frac{0.2}{0.64} =0.3125,

and the total variance is \(4(0.3125)=1.25\). Direct substitution confirms both calculations:

E[X]=\frac{4(0.2)}{0.8}=1, \qquad \operatorname{Var}(X)=\frac{4(0.2)}{0.64}=1.25.

The variance exceeds the mean here because \(1/p^2\) in the variance formula is larger than \(1/p\) in the mean formula when \(p<1\).

Worked Example: A Low Success Probability

For \(r=2\) and success probability \(p=0.2\), \(q=0.8\). Each success requires a geometric waiting count with mean \(0.8/0.2=4\) failures and variance \(0.8/(0.2)^2=20\). Adding two independent waiting counts gives

E[X]=4+4=8,\qquad \operatorname{Var}(X)=20+20=40.

Using the formulas for the full count gives the same results:

E[X]=\frac{2(0.8)}{0.2}=8,\qquad \operatorname{Var}(X)=\frac{2(0.8)}{0.04}=40.

The larger variance reflects the long and variable waits that can occur when success is relatively unlikely.

Interpreting the Formulas and Avoiding a Common Error

The factor \(r\) appears in both moments because the total is the sum of \(r\) identically distributed independent waiting counts. The factor \(q/p\) is the average number of failures in one waiting period; the factor \(q/p^2\) is its variance. The variance calculation specifically depends on independence: without independence, a variance of a sum would also include covariance terms.

A frequent mistake is to confuse the failure count \(X\) with the total number of trials required to obtain \(r\) successes. If \(T\) denotes that total number of trials, then every completed sequence contains exactly \(r\) successes, so \(T=X+r\). Thus

E[T]=E[X]+r=\frac{r}{p}, \qquad \operatorname{Var}(T)=\operatorname{Var}(X)=\frac{rq}{p^2}.

The correction is to identify what the random variable counts before choosing a formula: adding the fixed number \(r\) changes the mean by \(r\), but adding a constant does not change the variance. The \(r/p\) mean belongs to the trials count, not the failures count.

Key takeaway: For failures before the \(r\)th success, split the count into \(r\) independent geometric waiting counts. Their means and variances add, yielding \(E[X]=rq/p\) and \(\operatorname{Var}(X)=rq/p^2\).

Check Your Understanding

Use the waiting-count decomposition and the failures-before-success convention in your answers.

For \(r=5\) and \(p=0.5\), calculate \(E[X]\) and \(\operatorname{Var}(X)\).
Explain why the waiting counts \(G_1,\ldots,G_r\) are independent under independent Bernoulli trials.
For \(r=2\) and \(p=0.75\), find the mean and variance of the failures count. Show the value of \(q\) you use.
If \(T\) counts all trials through the \(r\)th success, how are \(T\) and \(X\) related, and how do their means and variances differ?
What assumption allows the variance of \(X=\sum_{j=1}^r G_j\) to be computed by simply adding the component variances?