Probability-Generating Functions and Coefficient Extraction

Why Encode a Count Distribution With a Function?

A nonnegative integer-valued random variable is described by a sequence of probabilities: $P(X=0), P(X=1), P(X=2),\ldots$. A probability-generating function (PGF) stores this entire sequence as the coefficients of a power series. That representation is useful because algebra on the function can encode features of the distribution; here, the central task is to define the function carefully, locate its domain, and show that no point probability is lost in the encoding.

This tutorial concerns the function and its coefficients. In particular, knowing that a PGF exists does not by itself assert that moments are finite or that derivatives can be taken at the boundary of its domain. Those questions require additional conditions.

Definition and Basic Interpretation

Definition: Let $X$ be a random variable with $P(X\in\{0,1,2,\ldots\})=1$, and write $p_k=P(X=k)$ for $k=0,1,2,\ldots$. Its probability-generating function is the power series $$ G_X(s)=E[s^X]=\sum_{k=0}^{\infty}p_k s^k $$ at every value of $s$ for which the expectation is defined. We use $s^0=1$, including when $s=0$. Thus $G_X(0)=p_0$.

The expectation notation is literal whenever $E[|s|^X]<\infty$. For complex $s$, this condition means absolute integrability; for $|s|\leq 1$, it holds automatically because $|s|^X\leq 1$. Grouping the expectation according to the possible values of $X$ gives the displayed series. For a nonnegative real $s$, the same equality follows by summing over the countable values of $X$, with the series allowed to diverge to infinity if the expectation is infinite.

The coefficient of $s^k$ is the probability $p_k$, not a normalized or rescaled version of it. At $s=1$, all powers are one, so the total probability gives $G_X(1)=1$. At other values, the powers weight different outcomes differently.

Where the PGF Converges

The defining probabilities satisfy $p_k\geq 0$ and $\sum_{k=0}^{\infty}p_k=1$. These two facts guarantee convergence on the entire closed unit disk, not just for $s$ strictly between zero and one.

Theorem: For every nonnegative integer-valued $X$, the series $\sum_{k=0}^{\infty}p_k s^k$ converges absolutely for every complex $s$ with $|s|\leq 1$, and $$ |G_X(s)|\leq 1. $$ Its radius of convergence $R$ is at least $1$. Consequently, the power series defines an analytic function for $|s|<R$, in particular for $|s|<1$.

Proof. For $|s|\leq 1$, the sum of the absolute values is bounded by $$ \sum_{k=0}^{\infty}|p_k s^k| \leq \sum_{k=0}^{\infty}p_k =1. $$ Thus the series is absolutely convergent, and the triangle inequality gives $|G_X(s)|\leq 1$. To see that its radius of convergence is at least one, note that $p_k\leq 1$ for every $k$. Hence $\limsup_{k\to\infty}p_k^{1/k}\leq 1$. The power-series radius formula gives $R=1/\limsup_{k\to\infty}p_k^{1/k}$, with $R=\infty$ if the limsup is zero. Therefore $R\geq 1$. The analyticity assertion follows from the power-series theorem on the open disk of convergence. $\square$

The unit disk is a guaranteed region, not necessarily the exact domain of convergence. A distribution whose probabilities decay quickly can have $R>1$; a heavier-tailed count distribution can have $R=1$. Even when $R=1$, absolute convergence at $|s|=1$ follows from the probability sum, including at points where the general theory of power series alone would not guarantee endpoint convergence.

For real $s$ with $0\leq s\leq 1$, each term $p_k s^k$ is nonnegative, so $0\leq G_X(s)\leq 1$. Moreover, $G_X$ is nondecreasing on this interval: if $s_1\leq s_2$, then $s_1^k\leq s_2^k$ for every integer $k\geq 0$, and summing with weights $p_k$ preserves the inequality. The series also converges uniformly on $[0,1]$, since its tail is bounded uniformly by $\sum_{k>N}p_k$, which tends to zero. As a uniform limit of continuous partial sums, $G_X$ is continuous on $[0,1]$.

Outside the closed unit disk, finiteness depends on the distribution. If $E[|s|^X]<\infty$, then $E[s^X]$ is defined and equals its absolutely convergent power series. The possibility of such values beyond one is illustrated by the geometric example below.

Recovering the Probabilities From the Coefficients

The PGF does not merely summarize the probabilities; its power-series coefficients identify them uniquely. Coefficient extraction can be stated without assuming that $G_X$ has any particular derivatives at $s=1$.

Theorem: Let $G_X(s)=\sum_{j=0}^{\infty}p_j s^j$ be the PGF of a nonnegative integer-valued random variable. Then $$ p_0=G_X(0), \qquad p_k=\lim_{s\downarrow 0} \frac{G_X(s)-\sum_{j=0}^{k-1}p_j s^j}{s^k}, \quad k\geq 1. $$ Thus the PGF on $0\leq s<1$ determines every point probability.

Proof. The first identity follows by setting $s=0$: every term with $j\geq 1$ vanishes, while the $j=0$ term is $p_0$. Now fix $k\geq 1$ and $0<s<1$. Subtracting the first $k$ terms and dividing by $s^k$ gives $$ \frac{G_X(s)-\sum_{j=0}^{k-1}p_j s^j}{s^k} =p_k+\sum_{j=k+1}^{\infty}p_j s^{j-k}. $$ The final sum is nonnegative and bounded by $$ 0\leq \sum_{j=k+1}^{\infty}p_j s^{j-k} \leq s\sum_{j=k+1}^{\infty}p_j \leq s, $$ because $j-k\geq 1$ and the probabilities sum to one. The bound tends to zero as $s\downarrow 0$, proving the formula. Applying it successively for $k=0,1,2,\ldots$ recovers the full probability sequence. $\square$

This argument also explains the idea of coefficient extraction: first identify the constant term, then remove it; the lowest remaining power reveals the next probability, and the process continues. It is a limit at zero, not a claim about derivatives at one.

Worked Examples

Worked Example: A Bernoulli Random Variable

Let $X\sim\operatorname{Bernoulli}(0.3)$, so $P(X=1)=0.3$ and $P(X=0)=0.7$. The only nonzero probabilities are $p_0=0.7$ and $p_1=0.3$. Therefore $$ G_X(s)=0.7+0.3s. $$

The coefficient of $s^0$ is $0.7=P(X=0)$, and the coefficient of $s^1$ is $0.3=P(X=1)$; all higher coefficients are zero. Direct checks give $G_X(0)=0.7$ and $G_X(1)=0.7+0.3=1$. Since this is a polynomial, its radius of convergence is infinite.

Worked Example: A Geometric Count and a Larger Convergence Disk

Suppose $P(X=k)=0.4(0.6)^k$ for $k=0,1,2,\ldots$. These probabilities sum to one because $$ \sum_{k=0}^{\infty}0.4(0.6)^k =0.4\frac{1}{1-0.6} =1. $$ For $|0.6s|<1$, the geometric-series identity gives $$ G_X(s)=\sum_{k=0}^{\infty}0.4(0.6s)^k =\frac{0.4}{1-0.6s}. $$

The power series has radius $R=1/0.6=5/3$, which exceeds one. For example, $s=1.2$ lies inside this disk, and $$ G_X(1.2)=\frac{0.4}{1-0.72}=\frac{0.4}{0.28}=\frac{10}{7}. $$ This value exceeds one because $s>1$; it is a finite expectation of $1.2^X$, not a probability. The coefficient of $s^3$ is $0.4(0.6)^3=0.0864$, so $P(X=3)=0.0864$.

Worked Example: A Poisson Distribution

Let $X\sim\operatorname{Poisson}(2)$, so $P(X=k)=e^{-2}2^k/k!$ for $k=0,1,2,\ldots$. Substitution into the PGF definition yields $$ G_X(s)=\sum_{k=0}^{\infty}e^{-2}\frac{2^k}{k!}s^k =e^{-2}\sum_{k=0}^{\infty}\frac{(2s)^k}{k!} =e^{-2}e^{2s} =e^{2(s-1)}. $$

The exponential series converges for every complex $s$, so the PGF has infinite radius of convergence. As checks, $G_X(0)=e^{-2}=P(X=0)$, and $G_X(1)=e^0=1$. The coefficient of $s^2$ is $e^{-2}2^2/2!=2e^{-2}$, which agrees with $P(X=2)$.

Common Pitfalls and Uses

A common mistake is to treat $G_X(1)=1$ as though it gave the mean of $X$. It gives the sum of the probabilities, which is one. The value of a function at one and its behavior near one are distinct pieces of information; connecting derivatives to moments requires separate results and suitable conditions.

Another mistake is to assume that every PGF has radius exactly one. The geometric and Poisson examples show otherwise. The general guarantee is $R\geq 1$, with possible values strictly larger than one or infinite. Conversely, convergence for $|s|\leq 1$ does not mean that the expectation is finite for every $|s|>1$.

At zero, remember the convention $s^0=1$. It ensures that the contribution from $X=0$ is $p_0$, including at $s=0$. Finally, coefficients refer to the ordinary powers $s^k$: the coefficient at power $k$ is exactly $P(X=k)$.

Key takeaway: For a nonnegative integer-valued $X$, $G_X(s)=\sum_{k\geq 0}P(X=k)s^k$ converges absolutely on $|s|\leq 1$, and its power-series coefficients recover the entire distribution.

Check Your Understanding

Use the definition and coefficient-extraction argument to answer these questions.

If $P(X=0)=0.2$, what is $G_X(0)$, and why?
What bound on $|G_X(s)|$ follows when $|s|\leq 1$? State the probability fact that makes the bound work.
For a distribution with $G_X(s)=0.25+0.5s+0.25s^2$, identify $P(X=0)$, $P(X=1)$, and $P(X=2)$, and check $G_X(1)$.
Why does absolute convergence on $|s|\leq 1$ not imply that the PGF is finite at every $s$ with $|s|>1$?
In the coefficient-extraction formula for $p_k$, why does the contribution from powers above $k$ vanish as $s\downarrow 0$?