Recovering A Distribution From Its PGF

From a Function Back to Probabilities

A probability generating function (PGF) encodes the entire distribution of a nonnegative integer-valued random variable in one function. To use that encoding in reverse, examine the function near zero: the coefficient of $s^k$ is the probability of the value $k$. This gives a direct way to identify a distribution from a proposed PGF and to decide whether two PGFs represent the same law.

Let $(\Omega,\mathcal{F},\mathbb{P})$ be a probability space, and let $X:\Omega\to\mathbb{Z}_{\ge0}$ be a random variable. Write $p_k=\mathbb{P}(X=k)$ for $k\ge0$. Its PGF is

G_X(s)=\mathbb{E}[s^X]=\sum_{k=0}^{\infty}p_k s^k,\qquad 0\le s\le1.

At $s=0$, use the convention $0^0=1$, so the term for $X=0$ is included and $G_X(0)=p_0$. The series also defines a power series for $|s|<1$: it converges absolutely there because $\sum_{k\ge0}p_k|s|^k\le\sum_{k\ge0}p_k=1$. In fact, it converges at $s=1$, where $G_X(1)=1$. The probabilities are its Taylor coefficients at zero.

Theorem (Coefficient recovery): For every integer $k\ge0$, the PGF is $k$ times differentiable for $|s|<1$, and $$ \mathbb{P}(X=k)=p_k=\frac{G_X^{(k)}(0)}{k!}, $$ where $G_X^{(0)}=G_X$ and $0!=1$.

Proof. Fix any $r$ with $0<r<1$. For each fixed nonnegative integer $m$, the series obtained by differentiating the power series $m$ times converges uniformly on $|s|\le r$: its terms are bounded in absolute value by $p_n n^m r^{n-m}$ for $n\ge m$, and their sum is finite because $p_n\le1$ and $\sum_{n\ge m}n^m r^{n-m}$ converges. The term-by-term differentiation theorem for power series therefore applies inside the unit disk. After $k$ differentiations, the term $p_n s^n$ becomes zero if $n<k$, becomes $k!p_k$ if $n=k$, and still contains a positive power of $s$ if $n>k$. Evaluating at zero leaves exactly $G_X^{(k)}(0)=k!p_k$. Dividing by $k!$ proves the formula. $\square$

The factorial matters: a derivative gives a factorial multiple of the coefficient, not the coefficient itself. For $k=0$, the formula simply reads $p_0=G_X(0)$. For $k\ge1$, derivatives at zero recover individual probabilities even when the distribution has infinitely many possible values.

Extracting Coefficients in Practice

Worked Example: Recovering a Poisson Distribution

Suppose $X$ has a Poisson distribution with parameter $2$, so $\mathbb{P}(X=k)=e^{-2}2^k/k!$ for $k\ge0$. Recover its probabilities from its PGF.

Substituting the probabilities into the definition gives

G_X(s)=\sum_{k=0}^{\infty}e^{-2}\frac{2^k}{k!}s^k =e^{-2}\sum_{k=0}^{\infty}\frac{(2s)^k}{k!} =e^{-2}e^{2s} =e^{2(s-1)}.

For every $k\ge0$, differentiating $k$ times gives $G_X^{(k)}(s)=2^k e^{2(s-1)}$. At zero this is $G_X^{(k)}(0)=2^k e^{-2}$, so coefficient recovery yields $p_k=2^k e^{-2}/k!$. In particular, $p_0=e^{-2}$, $p_1=2e^{-2}$, and $p_2=4e^{-2}/2=2e^{-2}$.

Check: Summing the recovered probabilities gives $\sum_{k\ge0}2^ke^{-2}/k!=e^{-2}e^2=1$. For $k=2$, the derivative is $G_X''(0)=4e^{-2}$, and dividing by $2!=2$ gives $p_2=2e^{-2}$, as required.

For a rational PGF, it can be simpler to expand the function as a power series than to differentiate repeatedly. The geometric-series identity $1/(1-u)=\sum_{k\ge0}u^k$, valid when $|u|<1$, provides a common route. Each coefficient still needs to be interpreted as a probability and checked against total mass one.

Worked Example: Reading a Geometric Law from a Rational PGF

Consider $G(s)=1/(3-2s)$. Find the distribution encoded by this function and verify that it is a PGF.

Factor out $3$ from the denominator and apply the geometric-series identity:

G(s)=\frac{1}{3}\frac{1}{1-\frac23s} =\frac13\sum_{k=0}^{\infty}\left(\frac23\right)^k s^k,\qquad |s|<\frac32.

Thus the coefficient of $s^k$ is $p_k=\frac13(\frac23)^k$. The first three probabilities are $p_0=\frac13$, $p_1=\frac13\cdot\frac23=\frac29$, and $p_2=\frac13\cdot\frac49=\frac4{27}$. In general, all coefficients are nonnegative. Their sum is $\frac13\sum_{k\ge0}(\frac23)^k=\frac13\cdot\frac{1}{1-\frac23}=1$, so they define a probability distribution. It is the geometric distribution on $\{0,1,2,\ldots\}$ with success probability $1/3$.

Check by differentiation: $G(0)=1/3=p_0$. Also $G'(s)=2/(3-2s)^2$, so $G'(0)=2/9=p_1$. Differentiating again gives $G''(s)=8/(3-2s)^3$, hence $G''(0)/2!=\frac{8/27}{2}=4/27=p_2$, agreeing with the expansion.

Worked Example: A Rational PGF with a Separate Zero Mass

Let $G(s)=(1+s)/(3-s)$. Expand it and identify every probability.

Rewrite the denominator and expand:

G(s)=\frac{1+s}{3}\frac{1}{1-\frac{s}{3}} =\frac{1+s}{3}\sum_{k=0}^{\infty}\left(\frac{s}{3}\right)^k,\qquad |s|<3.

The coefficient of $s^0$ comes only from the first factor $1$, so $p_0=1/3$. For $k\ge1$, there are two contributions to the coefficient of $s^k$: $1/3^{k+1}$ from the term $s^k$ in the series multiplied by $1$, and $1/3^k$ from the term $s^{k-1}$ in the series multiplied by $s$. Therefore $p_k=1/3^{k+1}+1/3^k=4/3^{k+1}$ for $k\ge1$. For example, $p_1=1/9+1/3=4/9$, and $p_2=1/27+1/9=4/27$.

Check: The mass at zero is $1/3$, while the remaining mass is $\sum_{k\ge1}4/3^{k+1}=(4/9)\sum_{j\ge0}(1/3)^j=(4/9)(3/2)=2/3$. The total is $1/3+2/3=1$. Direct substitution also gives $G(1)=(1+1)/(3-1)=1$, consistent with the total probability.

When Do Two PGFs Describe the Same Distribution?

Coefficient recovery gives a precise uniqueness principle: a PGF determines its distribution. Equality of the functions on the whole interval $[0,1]$ is more than enough, and equality on $[0,1)$ is also enough. The next result proves the latter without requiring a separate complex-analytic uniqueness theorem.

Theorem (Uniqueness of a PGF): If $X$ and $Y$ take values in $\mathbb{Z}_{\ge0}$ and $G_X(s)=G_Y(s)$ for every $s\in[0,1)$, then $\mathbb{P}(X=k)=\mathbb{P}(Y=k)$ for every $k\ge0$. Hence $X$ and $Y$ have the same distribution.

Proof. Write $p_k=\mathbb{P}(X=k)$ and $q_k=\mathbb{P}(Y=k)$. For $0<s<1$, \[ 0\le G_X(s)-p_0=\sum_{k\ge1}p_ks^k\le s\sum_{k\ge1}p_k\le s. \] The same bound holds for $G_Y(s)-q_0$. Letting $s$ decrease to zero and using $G_X(s)=G_Y(s)$ gives $p_0=q_0$.

Now fix $n\ge1$ and suppose $p_j=q_j$ for all $j<n$. Subtract the common terms of degree less than $n$ from the equal PGFs and divide by $s^n$. This gives \[ 0=(p_n-q_n)+\sum_{k>n}(p_k-q_k)s^{k-n}. \] The absolute value of the remaining sum is at most \[ \sum_{k>n}p_k s^{k-n}+\sum_{k>n}q_k s^{k-n} \le s\sum_{k>n}p_k+s\sum_{k>n}q_k\le2s, \] because $k-n\ge1$ and $0<s<1$. Letting $s$ decrease to zero gives $p_n=q_n$. Induction proves equality of every pair of probabilities, so the distributions are identical. $\square$

This result supports two useful ways to compare distributions. One can extract coefficients and find a value of $k$ at which the probabilities differ, or compare the functions and use uniqueness. Matching a few probabilities—or even matching a mean—does not establish that two distributions are equal.

Worked Example: Equal Means, Different Distributions

Compare a Bernoulli random variable $X$ with success probability $1/2$ and a random variable $Y$ with probabilities $\mathbb{P}(Y=0)=3/5$, $\mathbb{P}(Y=1)=3/10$, and $\mathbb{P}(Y=2)=1/10$. Show that their means agree but their distributions do not.

The PGFs are $G_X(s)=1/2+s/2$ and $G_Y(s)=3/5+(3/10)s+(1/10)s^2$. The probabilities for $Y$ sum to $3/5+3/10+1/10=6/10+3/10+1/10=1$, so this is a valid distribution. Its mean is $\mathbb{E}[Y]=0(3/5)+1(3/10)+2(1/10)=3/10+2/10=1/2$. For $X$, $\mathbb{E}[X]=0(1/2)+1(1/2)=1/2$.

Despite the equal means, the zero probabilities differ: $\mathbb{P}(X=0)=1/2$, whereas $\mathbb{P}(Y=0)=3/5$. Correspondingly, $G_X(0)=1/2$ and $G_Y(0)=3/5$, so their PGFs are not equal and the distributions are not the same. The different laws are visible even though their means agree.

Check: Both PGFs take the value $1$ at $s=1$: $G_X(1)=1/2+1/2=1$ and $G_Y(1)=3/5+3/10+1/10=1$. Their derivatives at $1$ also agree: $G_X'(1)=1/2$, while $G_Y'(s)=3/10+s/5$, so $G_Y'(1)=3/10+1/5=1/2$. Equal normalization and equal means do not override the different values at zero.

A Common Coefficient Error

A frequent mistake is to identify $G^{(k)}(0)$ itself with $p_k$. The derivative multiplies the coefficient by $k!$, so the correct formula divides by $k!$. For example, take $G(s)=1/4+(1/2)s+(1/4)s^2$. Then $G''(s)=1/2$, so $G''(0)=1/2$. The incorrect calculation would claim $p_2=1/2$, but the coefficient of $s^2$ is $1/4$. The correction is $p_2=G''(0)/2!= (1/2)/2=1/4$. As a check, the coefficients $1/4+1/2+1/4$ sum to $1$.

When extracting a distribution from a candidate function, check both the algebra and the probability conditions: the expansion must have nonnegative coefficients, and those coefficients must sum to one. A function that looks like a PGF but fails either condition does not encode a probability distribution on the nonnegative integers. Once it is a valid PGF, its coefficients identify the law uniquely.

Check Your Understanding

Use coefficient extraction or the uniqueness result to answer each question.

If $G(s)=2/5+(3/5)s$, what are the probabilities at zero and one, and what is the probability at every $k\ge2$?
For $G(s)=1/(4-3s)$, expand the function as a power series at zero and state the coefficient of $s^k$.
Suppose $G_X(s)=G_Y(s)$ for every $s\in[0,1)$. What can be concluded about $\mathbb{P}(X=k)$ and $\mathbb{P}(Y=k)$ for each $k$?
If $G''(0)=3/5$, what is $\mathbb{P}(X=2)$? Explain the role of the factorial.
Can two nonnegative integer-valued random variables have equal means but different distributions? Give a reason using their PGFs or their probabilities.