Probability Generating Functions And Probability Encoding

From a Probability Law to a Generating Function

A probability law on the nonnegative integers is a sequence of numbers \(p_0,p_1,\ldots\), where \(p_k\) is the probability of the value \(k\), each \(p_k\ge0\), and \(\sum_{k=0}^{\infty}p_k=1\). A probability generating function packages this whole sequence into one function. This is useful because a function can be easier to manipulate than an infinite list of probabilities, while still retaining all the information in that list.

Let \(X\) be a nonnegative integer-valued random variable on a probability space \((\Omega,\mathcal{F},\mathbb{P})\), and write \(p_k=\mathbb{P}(X=k)\). For a complex number \(z\), use the convention \(z^0=1\), including when \(z=0\). A complex expectation \(\mathbb{E}[z^X]\) is defined when \(\mathbb{E}|z^X|<\infty\). The probability generating function, or PGF, is

at every \(z\) for which the series converges absolutely. In particular, this definition applies throughout the closed unit disk \(|z|\le1\). The coefficients are probabilities, not moments: the coefficient of \(z^k\) is exactly \(\mathbb{P}(X=k)\).

Convergence on the Unit Disk

For \(|z|\le1\), each term satisfies \(|p_kz^k|\le p_k\). Since the probabilities sum to \(1\), the series converges absolutely there. In fact, the convergence is uniform on the entire closed unit disk: after truncating at \(N\), the absolute size of the omitted terms is bounded by \(\sum_{k>N}p_k\), a quantity that tends to zero and does not depend on \(z\).

Proof. For every integer \(N\ge0\) and every \(|z|\le1\), the tail satisfies \(\left|\sum_{k>N}p_kz^k\right|\le\sum_{k>N}p_k\). The right-hand side tends to zero because \(\sum_{k=0}^{\infty}p_k=1\). This proves uniform convergence, and the same bound with \(N\) tending to infinity proves absolute convergence at each point. Every partial sum is a polynomial and hence continuous. A uniform limit of continuous functions is continuous, so \(G_X\) is continuous on the closed disk. On the open disk the series is a power series; the standard power-series theorem gives analyticity inside its radius of convergence. The radius is at least \(1\), as also follows from the coefficient bound \(\sum p_k=1\). \(\square\)

The radius may be larger than \(1\), or even infinite. In the usual power-series formula it is \(R=1/\limsup_{k\to\infty}p_k^{1/k}\), with reciprocal conventions when the limit superior is zero. This formula specifies absolute convergence for \(|z|<R\) and divergence for \(|z|>R\); it does not decide convergence at \(|z|=R\). The important guaranteed region for every PGF is the closed unit disk.

How PGF Values Encode the Distribution

One immediate check is \(G_X(1)=1\), since substituting \(1\) sums all probabilities. Also \(G_X(0)=p_0\), because all positive powers of zero vanish. More generally, the whole function on the interval \(0\le z<1\) determines every probability, not just these two facts. The next result gives a direct proof that does not require differentiating the PGF.

Proof. Set \(p_k=\mathbb{P}(X=k)\) and \(q_k=\mathbb{P}(Y=k)\). At \(x=0\), equality of the PGFs gives \(p_0=q_0\). Suppose for some \(m\ge1\) that \(p_k=q_k\) for every \(k<m\). For \(0<x<1\), subtract the equal lower-order terms from \(G_X(x)=G_Y(x)\) and divide by \(x^m\). This gives

The absolute value of the remaining sum is at most \(x\sum_{k>m}|p_k-q_k|\le2x\), because \(x^{k-m}\le x\) for \(k>m\), and \(\sum_{k>m}|p_k-q_k|\le\sum_{k>m}p_k+\sum_{k>m}q_k\le2\). As \(x\) decreases to zero this bound tends to zero. The displayed equality therefore implies \(p_m-q_m=0\). Starting from \(p_0=q_0\), induction proves \(p_k=q_k\) for every \(k\ge0\). Equal probabilities at every possible value are exactly equality of the distributions. \(\square\)

This is an encoding statement: a PGF is not merely a convenient summary that might discard information. If two PGFs agree throughout \(0\le x<1\), their probability laws agree. It does not say that one function value, such as \(G_X(\tfrac12)\), identifies a law; the theorem uses agreement over the whole interval.

Where the Expectation Definition Stops

For \(|z|\le1\), the expectation and the absolutely convergent probability series always agree. Outside that disk, it is useful to distinguish two claims: the power series may or may not converge, and the complex random variable \(z^X\) has an expectation only if it is integrable. If the series converges absolutely, then \(\mathbb{E}|z^X|=\sum_{k\ge0}p_k|z|^k<\infty\), so the expectation is defined and equals the series. If a series converges only conditionally, it can be discussed as a series extension, but it is not the expectation \(\mathbb{E}[z^X]\).

Common mistake. It is incorrect to treat conditional convergence of \(\sum p_kz^k\) as sufficient for the expectation \(\mathbb{E}[z^X]\). For a concrete case, let \(p_k=C\,2^{-k}/(k+1)\) for \(k\ge0\), where \(C=(2\ln2)^{-1}\). The identity \(\sum_{k\ge0}2^{-k}/(k+1)=2\ln2\) shows that these probabilities sum to \(1\). At \(z=-2\), the formal series becomes \(C\sum_{k\ge0}(-1)^k/(k+1)\), which converges conditionally. But

Thus \((-2)^X\) is not integrable and its expectation is not defined, even though the formal series has a conditional sum. The correction is to define \(G_X(z)=\mathbb{E}[z^X]\) where the series converges absolutely; if one separately studies a conditionally convergent extension, define it by the series and do not identify it with an expectation.

Summary and Interpretation

A PGF begins with the probabilities of a nonnegative integer-valued random variable and forms the power series whose coefficients are those probabilities. The probability coefficients ensure absolute and uniform convergence on \(|z|\le1\), continuity on the closed disk, and analyticity in its interior. Its values across \(0\le z<1\) determine the entire distribution, as the coefficient-by-coefficient proof shows.

Keep three checks in view when using a PGF: the probabilities must sum to \(1\); \(G_X(1)=1\); and an expectation representation outside the unit disk requires absolute integrability, not merely convergence of a formal series. These conditions prevent confusing a valid encoding of a probability law with an unrelated or undefined complex expectation.