Bernoulli, Binomial, and Poisson Moment Workshop

Why Work With Generating Functions?

For a count variable, direct summation can make even a variance calculation cumbersome. A probability-generating function (PGF) packages the probabilities into a single function whose derivatives give factorial moments. A moment-generating function (MGF) provides ordinary moments directly. Bernoulli, binomial, and Poisson variables are especially convenient: their functions have simple forms, and their derivatives expose useful patterns in the moments.

The two approaches answer related but different questions. PGF derivatives naturally produce products such as \(X(X-1)\), while MGF derivatives produce powers such as \(X^2\). Recognizing which type of moment a derivative gives is essential when moving from a generating function to a variance or a central moment.

PGF and MGF Formulas for the Three Distributions

For the distributions considered here, the PGFs are polynomials or an exponential function, and the MGFs are finite for every real argument. Thus the derivatives used below exist. Write \(q=1-p\) for a Bernoulli or binomial parameter \(p\).

For example, the Poisson PGF follows by summing its probability mass function: \(G_X(s)=\sum_{k=0}^{\infty}s^ke^{-\lambda}\lambda^k/k! =e^{-\lambda}e^{\lambda s}\). Replacing \(s\) with \(e^t\) gives the MGF. The Bernoulli formulas follow by summing over \(X=0,1\). The binomial formulas follow from its probability mass function and the binomial theorem.

Factorial Moments From PGF Derivatives

The usefulness of a PGF is especially clear for these three models. The derivative pattern can be obtained explicitly. For Bernoulli\((p)\), \(G_X'(s)=p\), and all derivatives of order at least two are zero. For \(X\sim\operatorname{Binomial}(n,p)\), repeated differentiation gives, for \(1\le r\le n\),

Here \((n)_r=n(n-1)\cdots(n-r+1)\). If \(r>n\), the derivative and factorial moment are zero. For a Poisson variable, differentiating \(G_X(s)=e^{\lambda(s-1)}\) repeatedly yields \(G_X^{(r)}(s)=\lambda^r e^{\lambda(s-1)}\), so \(E[(X)_r]=\lambda^r\).

The result for the Poisson distribution can also be checked from the defining series: its \(r\)th factorial moment is \(e^{-\lambda}\sum_{k=r}^{\infty}(k)_r\lambda^k/k!\). Using \((k)_r/k!=1/(k-r)!\), this becomes \(e^{-\lambda}\lambda^r\sum_{j=0}^{\infty}\lambda^j/j!=\lambda^r\). This verifies the derivative calculation directly from the probability mass function.

From Factorial Moments to Means and Variances

The first two factorial moments are \(E[X]\) and \(E[X(X-1)]\). Since \(X^2=X(X-1)+X\), the raw second moment and variance satisfy

For the binomial distribution, this gives \(E[X]=np\) and \(E[X^2]=n(n-1)p^2+np\). Subtracting \(n^2p^2\) gives \(\operatorname{Var}(X)=np(1-p)\). For the Poisson distribution, \(E[X]=\lambda\) and \(E[X^2]=\lambda^2+\lambda\), so \(\operatorname{Var}(X)=\lambda\). A Bernoulli variable is the binomial case \(n=1\), giving mean \(p\) and variance \(pq\).

These results can also be read from the MGFs. Differentiating an MGF at zero gives raw moments: \(M_X'(0)=E[X]\) and \(M_X''(0)=E[X^2]\). For example, \(M_X(t)=(q+pe^t)^n\) has \(M_X'(0)=np\); the second derivative at zero is \(n(n-1)p^2+np\), agreeing with the PGF calculation. This is a useful independent check.

Selected Higher Moments

The identities \(X^3=(X)_3+3(X)_2+X\) and \(X^4=(X)_4+6(X)_3+7(X)_2+X\) translate factorial moments into raw moments. They follow by expanding the products on the right. These conversions make higher-order calculations manageable without summing each power against the probability mass function.

For a binomial variable, the factorial-moment result gives \(E[X^3]=(n)_3p^3+3(n)_2p^2+np\). For a Poisson variable, it gives \(E[X^3]=\lambda^3+3\lambda^2+\lambda\) and \(E[X^4]=\lambda^4+6\lambda^3+7\lambda^2+\lambda\). Central moments then follow by expanding powers of \(X-E[X]\). In particular, the third central moment of a binomial variable is \(npq(1-2p)\), and for a Poisson variable the third central moment is \(\lambda\). For Poisson, the fourth central moment is \(\lambda+3\lambda^2\).

MGF Derivatives as a Direct Check

The explicit MGFs provide a compact route to the first two moments. For \(X\sim\operatorname{Poisson}(\lambda)\), \(M_X(t)=e^{\lambda(e^t-1)}\). Differentiating gives \(M_X'(t)=\lambda e^tM_X(t)\), and \(M_X''(t)=(\lambda e^t+\lambda^2e^{2t})M_X(t)\). Since \(M_X(0)=1\), \(E[X]=M_X'(0)=\lambda\) and \(E[X^2]=M_X''(0)=\lambda+\lambda^2\). The variance is therefore \(\lambda\).

For \(X\sim\operatorname{Bernoulli}(p)\), differentiating \(M_X(t)=q+pe^t\) any positive number of times gives \(pe^t\), so \(E[X^r]=p\) for every integer \(r\ge1\). This is also immediate from \(X^r=X\), since \(X\) takes only the values zero and one. That observation is a useful check on higher raw moments in the Bernoulli case.

Common Mistake: Confusing the Two Types of Moments

A frequent error is to treat \(G_X''(1)\) as \(E[X^2]\). It is instead \(E[X(X-1)]\). The distinction matters even for the simplest count models: for \(X\sim\operatorname{Poisson}(\lambda)\), \(G_X''(1)=\lambda^2\), whereas \(E[X^2]=\lambda^2+\lambda\). The correction is \(X^2=(X)_2+X\), so the first factorial moment must be added.

Another error is to evaluate a PGF derivative at zero when seeking a moment. The coefficient interpretation at zero is useful for extracting probabilities, but factorial moments come from derivatives at one. For example, \(G_X'(1)=E[X]\); in general \(G_X'(0)\) is not the mean. Keeping the evaluation point and the moment type visible in each calculation prevents both mistakes.