Characteristic Functions of Binomial and Poisson Laws

From Probability Mass Functions to Characteristic Functions

A characteristic function converts a probability mass function into a complex-valued function of a real argument. For the Bernoulli, binomial, and Poisson laws, the calculation can be done directly: weight each possible value \(k\) by \(e^{itk}\) and sum. The resulting formulas are compact, but their details matter—especially the probability attached to each term in a finite sum.

For a real-valued random variable \(X\), its characteristic function is \(\varphi_X(t)=E[e^{itX}]\), where \(t\in\mathbb{R}\) and \(i^2=-1\). If \(X\) is integer-valued with probability mass function \(f(k)=P(X=k)\), this expectation is the absolutely convergent sum \(\sum_k e^{itk}f(k)\). Absolute convergence follows from \(|e^{itk}f(k)|=f(k)\) and \(\sum_k f(k)=1\).

Definition: A Bernoulli random variable with success probability \(p\), where \(0\leq p\leq 1\), takes value \(1\) with probability \(p\) and value \(0\) with probability \(1-p\). A binomial random variable with parameters \(n\) and \(p\), where \(n\) is a nonnegative integer, counts successes in \(n\) independent Bernoulli trials. A Poisson random variable with rate \(\lambda\), where \(\lambda\geq 0\), has mass function \(P(X=k)=e^{-\lambda}\lambda^k/k!\) for \(k=0,1,2,\ldots\); when \(\lambda=0\), it is identically zero.

The Three Characteristic-Function Formulas

The Bernoulli formula comes from its two possible values. For the binomial law, the finite sum of its probability masses is a binomial expansion. For the Poisson law, the corresponding infinite sum is the power series for the exponential function.

Theorem: If \(B\sim\operatorname{Bernoulli}(p)\), \(X\sim\operatorname{Binomial}(n,p)\), and \(N\sim\operatorname{Poisson}(\lambda)\), then for every real \(t\), \(\varphi_B(t)=1-p+pe^{it}\), \(\varphi_X(t)=(1-p+pe^{it})^n\), and \(\varphi_N(t)=\exp\{\lambda(e^{it}-1)\}\).

Proof. For a Bernoulli variable, the two terms in the expectation correspond to \(B=0\) and \(B=1\):

\varphi_B(t)=E[e^{itB}] =(1-p)e^{it\cdot 0}+pe^{it\cdot 1} =1-p+pe^{it}.

For \(X\sim\operatorname{Binomial}(n,p)\), its mass at \(k\) is \(\binom{n}{k}p^k(1-p)^{n-k}\) for \(0\leq k\leq n\). Thus

\begin{aligned} \varphi_X(t) &=\sum_{k=0}^{n} e^{itk}\binom{n}{k}p^k(1-p)^{n-k}\\ &=\sum_{k=0}^{n}\binom{n}{k}(pe^{it})^k(1-p)^{n-k}\\ &=(1-p+pe^{it})^n, \end{aligned}

by the binomial theorem. Finally, for \(\lambda>0\), the Poisson mass function gives

\begin{aligned} \varphi_N(t) &=\sum_{k=0}^{\infty}e^{itk}e^{-\lambda}\frac{\lambda^k}{k!}\\ &=e^{-\lambda}\sum_{k=0}^{\infty}\frac{(\lambda e^{it})^k}{k!}\\ &=e^{-\lambda}\exp(\lambda e^{it}) =\exp\{\lambda(e^{it}-1)\}. \end{aligned}

The series may be evaluated at \(\lambda e^{it}\) because the exponential power series converges absolutely for every complex argument; indeed, the sum of the absolute values of its terms is \(\sum_{k=0}^\infty\lambda^k/k!=e^\lambda<\infty\). If \(\lambda=0\), \(N=0\) with probability one, so \(\varphi_N(t)=1\), which also equals \(\exp\{0(e^{it}-1)\}\). The formulas include the endpoint cases \(p=0\), \(p=1\), and \(n=0\). \(\square\)

Worked Example: A Bernoulli Characteristic Function at \(t=\pi\)

Let \(B\sim\operatorname{Bernoulli}(0.3)\). Since \(e^{i\pi}=-1\), the formula gives

\varphi_B(\pi)=1-0.3+0.3e^{i\pi}=0.7-0.3=0.4.

A direct calculation checks the result. The contribution from \(B=0\) is \(0.7e^0=0.7\), while the contribution from \(B=1\) is \(0.3e^{i\pi}=-0.3\). Their sum is \(0.4\). In particular, the answer is real here because the two possible phases are \(1\) and \(-1\).

Worked Example: Checking a Binomial Formula Term by Term

Let \(X\sim\operatorname{Binomial}(3,1/2)\). Its probabilities at \(k=0,1,2,3\) are respectively \(1/8,3/8,3/8,1/8\). At \(t=\pi/2\), the four factors \(e^{itk}\) are \(1,i,-1,-i\). Therefore

\begin{aligned} \varphi_X(\pi/2) &=\frac18+\frac{3i}{8}-\frac38-\frac{i}{8}\\ &=-\frac14+\frac14i. \end{aligned}

The binomial formula gives the same value:

(1-\tfrac12+\tfrac12 i)^3 =\left(\frac{1+i}{2}\right)^3 =\frac{(1+i)^3}{8} =\frac{-2+2i}{8} =-\frac14+\frac14i.

In the direct sum, the \(k=3\) term is \(-i/8\), because its probability is \(1/8\). The phase \(-i\) does not change that probability to \(3/8\).

Worked Example: A Poisson Characteristic Function at \(t=\pi/3\)

Suppose \(N\sim\operatorname{Poisson}(2)\). Since \(e^{i\pi/3}=1/2+i\sqrt{3}/2\), substitution into the formula gives

\begin{aligned} \varphi_N(\pi/3) &=\exp\{2(e^{i\pi/3}-1)\}\\ &=\exp\{2(-1/2+i\sqrt{3}/2)\}\\ &=e^{-1+i\sqrt{3}} =e^{-1}\bigl(\cos(\sqrt{3})+i\sin(\sqrt{3})\bigr). \end{aligned}

The real part of the exponent is \(-1\), so the modulus of this value is \(e^{-1}\). This also follows from the general expression \(|\varphi_N(t)|=\exp\{\lambda(\cos t-1)\}\).

Independent Sums: Products of the Formulas

For independent random variables \(X_1,\ldots,X_m\), the characteristic function of their sum is the product of their characteristic functions. In this finite-sum setting, the identity follows from independence and the factorization of the expectation of a product of bounded functions.

Theorem: If \(X_1,\ldots,X_m\) are independent real-valued random variables, then \(\varphi_{X_1+\cdots+X_m}(t)=\prod_{j=1}^{m}\varphi_{X_j}(t)\) for every \(t\in\mathbb{R}\).

Proof. Fix \(t\in\mathbb{R}\), and set \(Z_j=e^{itX_j}\). Each \(Z_j\) is measurable and bounded in modulus by \(1\), so all the expectations below exist. Independence of \(X_1,\ldots,X_m\) implies independence of the \(Z_j\), and hence the expectation of their product factors. Since \(e^{it(X_1+\cdots+X_m)}=\prod_{j=1}^m Z_j\), it follows that

\varphi_{X_1+\cdots+X_m}(t) =E\left[\prod_{j=1}^{m}Z_j\right] =\prod_{j=1}^{m}E[Z_j] =\prod_{j=1}^{m}\varphi_{X_j}(t).

\(\square\)

This identity checks the binomial formula particularly directly. If \(B_1,\ldots,B_n\) are independent Bernoulli variables, each with success probability \(p\), their sum \(S\) is binomial: for any \(k\), exactly \(\binom{n}{k}\) possible success patterns have \(k\) successes, and each pattern has probability \(p^k(1-p)^{n-k}\). Consequently, the direct Bernoulli characteristic functions give

\varphi_S(t)=\prod_{j=1}^n(1-p+pe^{it})=(1-p+pe^{it})^n,

in agreement with the calculation from the binomial mass function. This verifies the formula both by summing over the binomial masses and by multiplying the factors for independent trials.

Worked Example: Multiplying Bernoulli Factors

Let \(B_1,B_2,B_3\) be independent Bernoulli variables, each with success probability \(1/2\), and let \(S=B_1+B_2+B_3\). The probability that \(S=k\) is \(\binom{3}{k}/8\), so \(S\sim\operatorname{Binomial}(3,1/2)\). At \(t=\pi/2\), each factor is

\varphi_{B_j}(\pi/2)=\frac12+\frac12i=\frac{1+i}{2}.

Independence therefore yields

\varphi_S(\pi/2) =\left(\frac{1+i}{2}\right)^3 =-\frac14+\frac14i.

For a direct check, the four possible values of \(S\) have probabilities \(1/8,3/8,3/8,1/8\). Weighting these by \(1,i,-1,-i\), respectively, gives \(1/8+3i/8-3/8-i/8=-1/4+i/4\), as required.

A related product calculation applies to independent Poisson variables, without needing any distribution-identification result. If \(N_1,\ldots,N_m\) are independent with rates \(\lambda_1,\ldots,\lambda_m\), then

\varphi_{N_1+\cdots+N_m}(t) =\prod_{j=1}^{m}\exp\{\lambda_j(e^{it}-1)\} =\exp\left\{\left(\sum_{j=1}^{m}\lambda_j\right)(e^{it}-1)\right\}.

This establishes the transform of the sum by the independent-sum identity. The calculation alone is not being used here to infer the sum's distribution from its characteristic function.

Worked Example: Two Independent Poisson Variables

Take independent \(N_1\sim\operatorname{Poisson}(1.2)\) and \(N_2\sim\operatorname{Poisson}(0.8)\). At \(t=\pi/3\), their characteristic functions are

\varphi_{N_1}(\pi/3)=e^{1.2(-1/2+i\sqrt{3}/2)} =e^{-0.6+i0.6\sqrt{3}},

and

\varphi_{N_2}(\pi/3)=e^{0.8(-1/2+i\sqrt{3}/2)} =e^{-0.4+i0.4\sqrt{3}}.

By independence, the characteristic function of \(N_1+N_2\) at this argument is their product:

e^{-0.6+i0.6\sqrt{3}}e^{-0.4+i0.4\sqrt{3}} =e^{-1+i\sqrt{3}}.

Equivalently, substituting the summed rate \(1.2+0.8=2\) into the Poisson expression gives \(e^{2(e^{i\pi/3}-1)}=e^{-1+i\sqrt{3}}\). The two routes agree.

Common Pitfall and Why the Formulas Matter

A frequent error in a finite characteristic-function calculation is to attach the wrong mass to a phase. The factor \(e^{itk}\) determines the phase for the value \(k\); it does not determine the probability of that value. In the binomial example above, \(k=3\) contributes \(P(X=3)e^{3it}\), which at \(t=\pi/2\) is \((1/8)(-i)=-i/8\), not \((3/8)(-i)\). The correction is to write each probability and its corresponding phase together before simplifying.

The closed forms are useful because multiplying characteristic functions turns sums of independent variables into algebra with a small number of factors. They also provide a quick internal check: a characteristic function must equal \(1\) at \(t=0\). Here, the Bernoulli formula gives \(1-p+p=1\); the binomial formula gives \(1^n=1\); and the Poisson formula gives \(e^0=1\). These checks can catch errors, although passing them does not by itself verify a calculation.

Check Your Understanding

Work from the mass functions or the stated formulas, and keep the probability attached to each phase.

For \(B\sim\operatorname{Bernoulli}(0.4)\), calculate \(\varphi_B(\pi)\) directly from its two possible values.
Derive the characteristic function of \(X\sim\operatorname{Binomial}(4,p)\) from its mass function.
For \(N\sim\operatorname{Poisson}(3)\), simplify \(\varphi_N(\pi)\).
If \(B_1\) and \(B_2\) are independent Bernoulli variables with success probabilities \(0.2\) and \(0.7\), respectively, write the characteristic function of \(B_1+B_2\) as a product.
In the \(\operatorname{Binomial}(3,1/2)\) calculation at \(t=\pi/2\), what are the probability and contribution of the \(k=3\) term?