Generating Functions for a Bernoulli Variable

Why Generate Functions for a Two-Point Distribution?

A Bernoulli variable has only two possible outcomes, so its distribution can be written down directly. Generating functions give another way to record those probabilities: the probability-generating function uses powers of an argument to encode integer outcomes, while the moment-generating function uses exponentials. For a Bernoulli variable, both functions are simple, but their derivatives make a useful distinction between ordinary moments and factorial moments.

The main calculation is compact: the probability-generating function has one derivative that yields the mean and a second derivative that yields a factorial moment; the moment-generating function yields ordinary moments from derivatives at zero. Keeping track of which function is differentiated and where is essential.

Definitions and the Bernoulli Generating Functions

For this two-point distribution the expectations are finite sums, so both functions exist for every real \(s\) and \(t\). In particular, the convention \(s^0=1\) applies, including when \(s=0\). Evaluating each expectation by summing over the two outcomes gives the following formulas.

The functions have different input variables and different useful evaluation points. For the probability-generating function, evaluate derivatives at \(s=1\). For the moment-generating function, evaluate derivatives at \(t=0\). The values at those points are both \(1\), as expected: \(G_X(1)=E[1^X]=1\) and \(M_X(0)=E[e^0]=1\).

Derivatives of the Probability-Generating Function

For a nonnegative integer-valued variable, differentiating a probability-generating function brings down powers of the outcome. At \(s=1\), the first derivative gives the mean. The second derivative gives the expectation of \(X(X-1)\), not \(X^2\), because two differentiations of \(s^x\) produce \(x(x-1)s^{x-2}\).

Here the derivatives can be computed directly from the polynomial \(G_X(s)=1-p+ps\): \(G_X'(s)=p\) and \(G_X''(s)=0\). To see why these values correspond to the stated expectations, differentiate the defining finite sum \(G_X(s)=\sum_{x=0}^1P(X=x)s^x\). Because it is a finite sum, differentiation term by term is valid. The first derivative is \(\sum_{x=0}^1P(X=x)x s^{x-1}\); at \(s=1\), this is \(E[X]\). The second derivative at \(1\) is \(\sum_{x=0}^1P(X=x)x(x-1)=E[X(X-1)]\). For this Bernoulli distribution the \(x=0\) term and the \(x=1\) term in the second derivative are both zero.

Finally, \(X^2=X(X-1)+X\), so the ordinary second moment is \(E[X^2]=G_X''(1)+G_X'(1)=p\). Subtracting the square of the mean gives the variance:

This calculation is also a direct verification of the moment formulas for the Bernoulli distribution. The probability-generating function returns the mean from its first derivative at one, and its second derivative contributes the factorial moment needed to construct the ordinary second moment.

Derivatives of the Moment-Generating Function

The moment-generating function encodes ordinary moments directly. For the Bernoulli variable, differentiating \(M_X(t)=1-p+pe^t\) gives \(M_X'(t)=pe^t\) and \(M_X''(t)=pe^t\). At \(t=0\), these derivatives are both \(p\). The second derivative therefore gives \(E[X^2]\), not \(E[X(X-1)]\).

The identification follows from the defining expectation. Since \(X\) takes only the values \(0\) and \(1\), \(M_X(t)=\sum_{x=0}^1P(X=x)e^{tx}\). Differentiating this finite sum once gives \(\sum_xP(X=x)xe^{tx}\), and twice gives \(\sum_xP(X=x)x^2e^{tx}\). Setting \(t=0\) makes each exponential factor \(1\), leaving the first and second ordinary moments, respectively. The finite sum requires no interchange of a limit with an infinite series or integral.

Why the Two Derivative Rules Differ

The probability-generating function differentiates powers \(s^x\), while the moment-generating function differentiates exponentials \(e^{tx}\). Their derivative rules explain why the PGF’s second derivative produces \(x(x-1)\), whereas the MGF’s second derivative produces \(x^2\). For a Bernoulli variable, both \(E[X]\) and \(E[X^2]\) equal \(p\), but that equality comes from the support \(\{0,1\}\); it does not mean the two derivative rules are the same.

For the PGF, the conversion from a factorial moment to an ordinary moment uses the identity \(X^2=X(X-1)+X\). The direct finite-sum calculation for this particular variable is:

For the MGF, the corresponding calculation uses \(x^2\):

Boundary Cases and Practical Use

The formulas remain valid at the boundary values of the parameter. If \(p=0\), then \(X=0\) with probability one, \(G_X(s)=1\), \(M_X(t)=1\), and both the mean and variance are zero. If \(p=1\), then \(X=1\) with probability one, \(G_X(s)=s\), \(M_X(t)=e^t\), the mean is one, and the variance is zero. The variance formula \(p(1-p)\) gives zero in both cases.

In practice, these calculations provide a concise way to check a Bernoulli distribution’s moments and to learn how generating functions encode distributions. The distinction to retain is simple: \(G_X'(1)\) gives the mean and \(G_X''(1)\) gives a factorial moment; \(M_X'(0)\) gives the mean and \(M_X''(0)\) gives the second ordinary moment. For this variable both routes lead to the same variance once the appropriate moment identity is used.