Generating Functions for Binomial Counts

Why Generate a Function for a Count?

A binomial random variable records the number of successes in a fixed number of independent trials with a common success probability. Its probability mass function gives each possible count separately. A generating function packages all those probabilities into one expression. For a binomial count, the expression is especially simple because the count is a sum of independent Bernoulli variables: the generating function of the sum is obtained by multiplying the functions for the trials.

Two functions are useful for different reasons. The probability-generating function (PGF) uses powers of a variable to keep track of the possible integer counts. The moment-generating function (MGF) uses an exponential argument and yields ordinary moments through differentiation. We will derive both from the trial-by-trial representation.

Definition: If $X$ takes values in the nonnegative integers, its probability-generating function is $$ G_X(s)=E[s^X]=\sum_{k=0}^{\infty}P(X=k)s^k $$ whenever the sum is defined. For a random variable whose MGF is finite for $t$ in a neighborhood of zero, its moment-generating function is $$ M_X(t)=E[e^{tX}]. $$ A finite-valued random variable has an MGF finite for every real $t$.

From Independent Trials to the Binomial PGF

Let $X_1,\ldots,X_n$ be independent Bernoulli random variables with common success probability $p$, where $0\leq p\leq1$, and let $X=\sum_{i=1}^n X_i$. Then $X\sim\operatorname{Bin}(n,p)$. For one trial, $X_i$ is either zero or one, so its PGF is $G_{X_i}(s)=(1-p)+ps$: the term $1-p$ corresponds to failure, and $ps$ to success.

Because $X$ is a sum, $s^X=\prod_{i=1}^n s^{X_i}$. Independence lets the expectation of this product factor into the product of the expectations. This gives the binomial PGF.

Theorem: If $X\sim\operatorname{Bin}(n,p)$, then for every real $s$, $$ G_X(s)=(1-p+ps)^n. $$ For $n=0$, the count is identically zero and the formula gives $G_X(s)=1$.

Proof. For each trial, $E[s^{X_i}]=(1-p)+ps$. Since $X=\sum_{i=1}^n X_i$, we have $s^X=\prod_{i=1}^n s^{X_i}$. Independence of the $X_i$ implies $$ G_X(s)=E\left[\prod_{i=1}^n s^{X_i}\right] =\prod_{i=1}^n E[s^{X_i}] =\prod_{i=1}^n(1-p+ps) =(1-p+ps)^n. $$ The variables have finite support, so these expectations and products are well-defined for every real $s$. For $n=0$, the empty sum is zero and $s^0=1$, as required. $\square$

The formula is a polynomial of degree at most $n$. Expanding by the binomial theorem gives $$ G_X(s)=\sum_{k=0}^n \binom{n}{k}(1-p)^{n-k}p^k s^k. $$ The coefficient of $s^k$ is $P(X=k)$. Thus the PGF stores the full distribution, not just a summary such as its mean.

Worked Example: Expanding a Binomial PGF

Let $X\sim\operatorname{Bin}(4,0.3)$. Find its PGF, expand it, and read off the probability of exactly two successes.

Here $1-p=0.7$, so the trial PGF is $0.7+0.3s$. Multiplying the four identical trial functions gives $$ G_X(s)=(0.7+0.3s)^4. $$

Expanding by powers of $s$, $$ G_X(s)=0.7^4+4(0.7)^3(0.3)s+6(0.7)^2(0.3)^2s^2+4(0.7)(0.3)^3s^3+(0.3)^4s^4. $$ The coefficients are $0.2401$, $0.4116$, $0.2646$, $0.0756$, and $0.0081$. In particular, the coefficient of $s^2$ is $$ P(X=2)=6(0.7)^2(0.3)^2=6(0.49)(0.09)=0.2646. $$ As a check, the coefficients sum to $0.2401+0.4116+0.2646+0.0756+0.0081=1$, which agrees with $G_X(1)=1$.

The Binomial MGF

For one Bernoulli trial, the MGF is $E[e^{tX_i}]=(1-p)+pe^t$, since $e^{tX_i}$ equals $1$ on failure and $e^t$ on success. The same independence argument applies to the sum. It is also possible to obtain the MGF directly from the PGF by substituting $s=e^t$.

Theorem: If $X\sim\operatorname{Bin}(n,p)$, then for every real $t$, $$ M_X(t)=E[e^{tX}]=(1-p+pe^t)^n=G_X(e^t). $$

Proof. Since $X=\sum_{i=1}^nX_i$, we have $e^{tX}=\prod_{i=1}^n e^{tX_i}$. Independence gives $$ M_X(t)=E\left[\prod_{i=1}^n e^{tX_i}\right] =\prod_{i=1}^nE[e^{tX_i}] =\prod_{i=1}^n(1-p+pe^t) =(1-p+pe^t)^n. $$ The random variable $X$ takes only the values $0,\ldots,n$, so $e^{tX}$ is bounded over its support for any fixed real $t$; consequently the expectation is finite. Finally, substituting $s=e^t$ in the PGF definition gives $G_X(e^t)=E[(e^t)^X]=E[e^{tX}]$. $\square$

Worked Example: Using MGF Derivatives

Let $X\sim\operatorname{Bin}(3,0.4)$. Use its MGF to find $E[X]$ and $E[X^2]$, then verify the variance.

The MGF is $$ M_X(t)=(0.6+0.4e^t)^3. $$ Because $X$ has finite support, we may differentiate its finite expectation term by term: $M_X'(0)=E[X]$ and $M_X''(0)=E[X^2]$. Differentiating once gives $$ M_X'(t)=3(0.6+0.4e^t)^2(0.4e^t). $$ At zero this is $M_X'(0)=3(1)^2(0.4)=1.2$, so $E[X]=1.2$.

Differentiating a second time using the product and chain rules gives $$ M_X''(t)=3\left[2(0.6+0.4e^t)(0.4e^t)^2+(0.6+0.4e^t)^2(0.4e^t)\right]. $$ Thus $$ E[X^2]=M_X''(0)=3\left[2(1)(0.16)+(1)(0.4)\right]=3(0.72)=2.16. $$ Since $\operatorname{Var}(X)=E[X^2]-(E[X])^2$, the variance is $2.16-(1.2)^2=2.16-1.44=0.72$. This agrees with $np(1-p)=3(0.4)(0.6)=0.72$.

What Derivatives of the PGF Tell Us

The PGF and MGF carry related but distinct derivative information. If $X$ has finite support on the nonnegative integers, then differentiating its finite polynomial gives $$ G_X'(1)=E[X],\qquad G_X''(1)=E[X(X-1)]. $$ The second derivative gives the second factorial moment, rather than $E[X^2]$ directly. Since $X^2=X(X-1)+X$, it follows that $E[X^2]=G_X''(1)+G_X'(1)$.

For a binomial count with $n\geq2$, differentiating $G_X(s)=(1-p+ps)^n$ yields $$ G_X'(s)=np(1-p+ps)^{n-1}, \qquad G_X''(s)=n(n-1)p^2(1-p+ps)^{n-2}. $$ For $n=1$, $G_X(s)=1-p+ps$, so $G_X'(s)=p$ and $G_X''(s)=0$; for $n=0$, $G_X(s)=1$, so both derivatives are zero. Evaluating at $s=1$ gives $G_X'(1)=np$ and $G_X''(1)=n(n-1)p^2$ in all cases $n\geq0$. Therefore $E[X^2]=n(n-1)p^2+np$. These identities illustrate how derivatives extract moment information from a compact function.

More generally, the coefficient of $s^k$ in a PGF is $P(X=k)$. This coefficient interpretation follows directly from the defining sum, and for the binomial polynomial it matches the coefficients in the binomial expansion. A PGF also satisfies $G_X(1)=\sum_kP(X=k)=1$, providing a quick normalization check. At $s=0$, $G_X(0)=P(X=0)$, because all terms with positive powers vanish.

Worked Example: Evaluating a PGF at a Useful Argument

Let $X\sim\operatorname{Bin}(5,0.2)$. Find $G_X(0)$ and calculate $E[(1/2)^X]$ using the PGF.

The PGF is $$ G_X(s)=(0.8+0.2s)^5. $$ At zero, $$ G_X(0)=(0.8)^5=0.32768. $$ Because $G_X(0)=P(X=0)$, this is the probability of no successes.

For the second quantity, substitute $s=1/2$: $$ E[(1/2)^X]=G_X(1/2)=(0.8+0.2(1/2))^5=(0.9)^5=0.59049. $$ This is a weighted average over the possible counts: a count $k$ contributes the weight $(1/2)^k$, multiplied by its probability. It is not itself the probability of a particular count.

Boundary Cases and a Common Pitfall

The formula accommodates the endpoint success probabilities. If $p=0$, then $X=0$ with probability one and $G_X(s)=1$, $M_X(t)=1$. If $p=1$, then $X=n$ with probability one and $G_X(s)=s^n$, $M_X(t)=e^{nt}$. These follow by substituting the values into the general formulas, and agree with the deterministic counts.

A common mistake is to write $G_X(s)=n(1-p+ps)$, as if one should add the $n$ single-trial PGFs. That expression corresponds to adding expectations of the individual powers, not to the power of $s$ indexed by the total count. The count is a sum of trials, and $s^X=s^{X_1+\cdots+X_n}=\prod_i s^{X_i}$; independence therefore produces a product of trial PGFs, namely $(1-p+ps)^n$. Independence is essential to this factorization. If trials are dependent, the marginal Bernoulli PGFs alone do not determine the PGF of their sum.

Key takeaway: For a binomial count, $G_X(s)=(1-p+ps)^n$ and $M_X(t)=(1-p+pe^t)^n$. The PGF encodes count probabilities in its coefficients; its derivatives yield factorial moments, while MGF derivatives at zero yield ordinary moments.

Check Your Understanding

Try these questions using the definitions and derivations above.

For $X\sim\operatorname{Bin}(6,0.25)$, write its PGF and MGF without expanding either expression.
Explain why the PGF of a sum of independent Bernoulli trials is a product rather than a sum of their individual PGFs.
For $X\sim\operatorname{Bin}(2,0.5)$, expand the PGF and identify $P(X=1)$ from its coefficient.
What quantity is $G_X''(1)$, and how can it be combined with $G_X'(1)$ to obtain $E[X^2]$?
What do the PGF and MGF become when $p=1$, and what deterministic value does that describe?