Characteristic Functions Of Standard Distributions

From a Distribution to Its Characteristic Function

A characteristic function packages a probability distribution into a complex-valued function of a real argument. For the standard distributions in this tutorial, its calculation is direct: sum over the possible values of a discrete random variable, or integrate its density for a continuous one. The resulting formulas are useful only when their parameters and signs are handled correctly, so each calculation below begins with the distribution itself and is checked against a concrete case.

The argument \(t\) is real, but the value of \(\varphi_X(t)\) is generally complex. The factor \(i\) in the exponent is essential. In particular, these calculations use \(e^{itx}\), not \(e^{tx}\), which is the exponential appearing in a moment generating function. No moment assumptions are needed for a characteristic function to exist.

Discrete Distributions: Finite Sums and a Probability Series

Proof. For a Bernoulli random variable, the value is \(0\) with probability \(1-p\) and \(1\) with probability \(p\). The finite-sum definition therefore gives \((1-p)e^{it\cdot0}+pe^{it\cdot1}=1-p+pe^{it}\). This calculation includes \(p=0\) and \(p=1\).

For a binomial random variable, the probability of \(k\) is \(\binom{n}{k}p^k(1-p)^{n-k}\), for \(k=0,\ldots,n\). Substituting those probabilities into the definition and applying the binomial theorem gives $$ \begin{aligned} \varphi(t) &=\sum_{k=0}^{n}\binom{n}{k}p^k(1-p)^{n-k}e^{itk}\\ &=\sum_{k=0}^{n}\binom{n}{k}(pe^{it})^k(1-p)^{n-k}\\ &=(1-p+pe^{it})^n. \end{aligned} $$ The formula also covers \(n=0\), when the variable is identically zero, and \(p=0\) or \(p=1\).

For a Poisson random variable, the probability of \(k\) is \(e^{-\lambda}\lambda^k/k!\), for \(k=0,1,\ldots\). Thus $$ \begin{aligned} \varphi(t) &=\sum_{k=0}^{\infty}e^{-\lambda}\frac{\lambda^k}{k!}e^{itk}\\ &=e^{-\lambda}\sum_{k=0}^{\infty}\frac{(\lambda e^{it})^k}{k!}\\ &=e^{-\lambda}\exp(\lambda e^{it}) =\exp\!\bigl(\lambda(e^{it}-1)\bigr). \end{aligned} $$ The series evaluation uses the exponential power series, which converges for every complex argument. When \(\lambda=0\), the variable is zero with probability one and the displayed formula equals \(1\). In all three cases, setting \(t=0\) gives \(1\), as required by \(\mathbb{E}[e^{i0X}]=1\). \(\square\)

Continuous Distributions: Density Integrals

Proof. For the exponential density, the definition gives $$ \varphi_X(t)=\int_0^\infty e^{itx}\lambda e^{-\lambda x}\,dx =\lambda\int_0^\infty e^{-(\lambda-it)x}\,dx. $$ Since \(\lambda>0\), the real part of \(\lambda-it\) is positive, and \(e^{-(\lambda-it)x}\) tends to zero as \(x\) tends to infinity. Evaluating the antiderivative, $$ \lambda\left[\frac{-e^{-(\lambda-it)x}}{\lambda-it}\right]_{0}^{\infty} =\frac{\lambda}{\lambda-it}. $$ This establishes the exponential formula, including its convergence condition.

For the normal formula, first take a standard normal random variable \(Z\), whose density is \(g(x)=(2\pi)^{-1/2}e^{-x^2/2}\), and write $$ I(t)=\int_{-\infty}^{\infty}e^{itx}g(x)\,dx. $$ Differentiation under the integral is valid because the absolute value of the derivative of the integrand is \(|x|g(x)\), which is integrable. It follows that \(I'(t)=i\int_{-\infty}^{\infty}xe^{itx}g(x)\,dx\). Since \(g'(x)=-xg(x)\), this becomes \(I'(t)=-i\int_{-\infty}^{\infty}e^{itx}g'(x)\,dx\). Integration by parts has zero boundary term because \(g(x)\) tends to zero at both ends and \(|e^{itx}|=1\). Therefore $$ \int_{-\infty}^{\infty}e^{itx}g'(x)\,dx =-it\int_{-\infty}^{\infty}e^{itx}g(x)\,dx=-itI(t), $$ so \(I'(t)=-tI(t)\). Also \(I(0)=1\), since \(g\) is a probability density. Multiplying the differential equation by \(e^{t^2/2}\) gives \(\frac{d}{dt}(e^{t^2/2}I(t))=0\); its constant value is \(1\) at \(t=0\). Hence \(I(t)=e^{-t^2/2}\).

For a normal random variable \(X\) with mean \(\mu\) and variance \(\sigma^2>0\), substitute \(x=\mu+\sigma z\) in its density integral. The density times \(dx\) becomes \(g(z)\,dz\), and the exponential factor becomes \(e^{it(\mu+\sigma z)}\). Thus $$ \varphi_X(t)=e^{i\mu t}\int_{-\infty}^{\infty}e^{i(\sigma t)z}g(z)\,dz =e^{i\mu t}I(\sigma t) =\exp\!\left(i\mu t-\frac{\sigma^2t^2}{2}\right). $$ This proves both continuous-distribution formulas. \(\square\)

Reading and Checking the Formulas

The finite-sum and integral calculations share one check: at \(t=0\), every characteristic function must equal \(1\). This follows directly from its definition, but it is also a useful way to detect missing probability mass or a misplaced parameter. For instance, the exponential expression \(\lambda/(\lambda-it)\) gives \(1\) at zero only when its numerator and rate in the denominator agree.

A second useful check is to keep the complex argument intact until the calculation is finished. For the exponential density, the integral involves \(\lambda-it\), not \(\lambda-t\). For the normal density, the location contributes \(i\mu t\), while the variance contributes the real, nonpositive term \(-\sigma^2t^2/2\). These signs are consistent with the fact that a characteristic function has absolute value at most \(1\): for the normal formula its absolute value is \(e^{-\sigma^2t^2/2}\le1\), and for the exponential formula it is \(\lambda/\sqrt{\lambda^2+t^2}\le1\).

Common mistake: substituting a real argument into an MGF formula. For an exponential variable with rate \(2\), an incorrect calculation might take the MGF expression \(2/(2-s)\) and put \(s=1\), obtaining \(2\). That cannot be the characteristic function at \(t=1\), since its absolute value would exceed \(1\). The correct substitution is \(s=it\), yielding \(2/(2-i)=(4+2i)/5\), whose absolute value is \(2/\sqrt{5}\le1\). The same distinction matters for a normal variable: the location term is \(i\mu t\), not \(\mu t\).