Exponential Moments and Moment-Generating Functions

Why Moments and the Moment-Generating Function Matter

The exponential distribution models a nonnegative waiting time with a constant rate parameter. Its simple density makes it possible to calculate several useful summaries exactly: the mean, the variance, and every nonnegative integer raw moment. The same calculation also gives its moment-generating function (MGF), but that function is finite only for a restricted range of real arguments. Identifying that range is part of the result, not a technical afterthought.

Throughout, $\lambda>0$ denotes the rate. The corresponding scale is $1/\lambda$. In particular, a larger rate means a smaller typical waiting time. The exponential distribution has density $\lambda e^{-\lambda x}$ for $x\geq 0$, and zero density for $x<0$.

Definition: If $X$ has an exponential distribution with rate $\lambda>0$, write $X\sim\operatorname{Exponential}(\lambda)$ and use the density $$ f(x)= \begin{cases} \lambda e^{-\lambda x}, & x\geq 0,\\ 0, & x<0. \end{cases} $$ For real $t$, the moment-generating function is $M_X(t)=E[e^{tX}]$ when this expectation is finite. Its domain of existence here means the set of real $t$ for which $E[e^{tX}]<\infty$.

Deriving the Raw Moments

For a nonnegative integer $k$, the $k$th raw moment is $E[X^k]$. For $k=0$, this is $E[1]=1$. For positive $k$, integrating the density gives

E[X^k]=\lambda\int_0^\infty x^k e^{-\lambda x}\,dx.

Integration by parts reduces this integral to the one for the preceding power. The boundary terms vanish: at $x=0$, $x^k=0$ for $k\geq 1$; as $x\to\infty$, $x^k e^{-\lambda x}\to 0$ because exponential decay dominates any fixed power of $x$.

Theorem: If $X\sim\operatorname{Exponential}(\lambda)$ with $\lambda>0$, then for every nonnegative integer $k$, $$ E[X^k]=\frac{k!}{\lambda^k}, $$ where $0!=1$.

Proof. The formula holds at $k=0$, since $E[X^0]=1=0!/\lambda^0$. For an integer $k\geq 1$, integration by parts with $u=x^k$ and $dv=e^{-\lambda x}\,dx$ gives

\begin{aligned} E[X^k] &=\lambda\int_0^\infty x^k e^{-\lambda x}\,dx\\ &=\lambda\left(\frac{k}{\lambda}\int_0^\infty x^{k-1}e^{-\lambda x}\,dx\right)\\ &=\frac{k}{\lambda}E[X^{k-1}]. \end{aligned}

The boundary term is zero by the limits just stated. Applying this recurrence repeatedly, starting from $E[X^0]=1$, yields

E[X^k]=\frac{k}{\lambda}\frac{k-1}{\lambda}\cdots\frac{1}{\lambda} =\frac{k!}{\lambda^k}.

This also establishes that each of these moments is finite. $\square$

The mean is the first raw moment. The variance is the second raw moment minus the square of the mean:

E[X]=\frac{1}{\lambda},\qquad E[X^2]=\frac{2}{\lambda^2},\qquad \operatorname{Var}(X)=E[X^2]-(E[X])^2=\frac{1}{\lambda^2}.

Thus the standard deviation is also $1/\lambda$. The equality of the mean and standard deviation is a feature of this distribution, and both quantities have the same measurement units as $X$. The variance, as usual, has squared units.

Worked Example: Moments at a Rate of Two per Hour

Suppose $X$ is a waiting time in hours and $X\sim\operatorname{Exponential}(2)$, so the event rate is $2$ per hour. Find the mean, variance, and third raw moment.

Substitute $\lambda=2$ into the moment formulas:

E[X]=\frac{1}{2}\text{ hour},\qquad \operatorname{Var}(X)=\frac{1}{2^2}=\frac14\text{ hour}^2.

For the third raw moment, use $3!=6$:

E[X^3]=\frac{3!}{2^3}=\frac{6}{8}=\frac34\text{ hour}^3.

The units provide a useful check: $E[X]$ has units of hours, while $E[X^3]$ has units of hours cubed.

The Moment-Generating Function and Its Domain

To find the MGF, multiply the density by $e^{tx}$ and integrate over the support. For real $t$, the defining integral is

M_X(t)=E[e^{tX}] =\lambda\int_0^\infty e^{-(\lambda-t)x}\,dx,

provided the integral is finite. If $t<\lambda$, then $\lambda-t>0$, and evaluating the integral gives $\lambda/(\lambda-t)$. If $t=\lambda$, the integrand in the expectation is $\lambda$ over the whole nonnegative half-line, so the integral diverges. If $t>\lambda$, the integrand grows exponentially and the integral also diverges.

Theorem: If $X\sim\operatorname{Exponential}(\lambda)$ with $\lambda>0$, its MGF is $$ M_X(t)=\frac{\lambda}{\lambda-t}\quad\text{for }t<\lambda. $$ For $t\geq\lambda$, $E[e^{tX}]=\infty$. Therefore the real domain of existence is $(-\infty,\lambda)$, not all of $\mathbb{R}$.

Proof. For $t<\lambda$, set $a=\lambda-t>0$. Then

\lambda\int_0^\infty e^{-ax}\,dx =\lambda\left[-\frac{1}{a}e^{-ax}\right]_{0}^{\infty} =\frac{\lambda}{a} =\frac{\lambda}{\lambda-t}.

For $t=\lambda$, the integrand $\lambda e^{-(\lambda-t)x}$ is the constant $\lambda>0$, whose integral over $[0,\infty)$ is infinite. For $t>\lambda$, let $b=t-\lambda>0$. The integrand is $\lambda e^{bx}$, and its integral over $[0,\infty)$ diverges. These cases establish the stated domain. $\square$

The MGF is finite on an open interval around zero: since $\lambda>0$, every $t$ with $-\lambda<t<\lambda$ is in its domain. In that interval, its formula can be differentiated. Direct differentiation gives

M_X'(t)=\frac{\lambda}{(\lambda-t)^2}, \qquad M_X''(t)=\frac{2\lambda}{(\lambda-t)^3}.

Evaluating at zero gives $M_X'(0)=1/\lambda$ and $M_X''(0)=2/\lambda^2$, agreeing with the first two raw moments obtained by integration. More generally, differentiating $\lambda/(\lambda-t)$ $k$ times at $t=0$ gives $k!/\lambda^k$, matching the moment formula. The integral derivation above establishes those moments independently of differentiation.

Worked Example: Evaluating an MGF Inside Its Domain

A waiting time $X$ has rate $\lambda=0.25$ per week. Evaluate $M_X(0.10)$, and determine whether $M_X(0.30)$ is finite.

The MGF exists only when $t<0.25$. Since $0.10<0.25$, substitution into the formula is valid:

M_X(0.10)=\frac{0.25}{0.25-0.10} =\frac{0.25}{0.15} =\frac{5}{3}.

For $t=0.30$, the inequality $0.30<0.25$ fails. In fact, the expectation contains the integral

0.25\int_0^\infty e^{-(0.25-0.30)x}\,dx =0.25\int_0^\infty e^{0.05x}\,dx,

which diverges. Thus $M_X(0.30)=\infty$, not a negative or otherwise finite value obtained by substituting into the rational expression.

Using the Scale Parameter Carefully

Some applications specify the exponential distribution by its scale $\beta>0$, where $\beta=1/\lambda$. In that notation the density is $\beta^{-1}e^{-x/\beta}$ for $x\geq0$, and the moment results become

E[X^k]=k!\,\beta^k,\qquad E[X]=\beta,\qquad \operatorname{Var}(X)=\beta^2,\qquad M_X(t)=\frac{1}{1-\beta t}\quad\text{for }t<\frac{1}{\beta}.

These are the same formulas expressed with a different parameter. In particular, the MGF’s boundary is the rate $1/\beta$, not the scale $\beta$.

Worked Example: Converting a Mean Waiting Time to a Rate

Suppose a waiting time has mean $4$ weeks and is exponential. Find its rate, variance, fourth raw moment, and the domain of its MGF.

For an exponential distribution, the mean is the reciprocal of the rate. Thus

\frac{1}{\lambda}=4 \quad\Longrightarrow\quad \lambda=\frac14\text{ per week}.

The variance and fourth raw moment are therefore

\operatorname{Var}(X)=\frac{1}{\lambda^2}=16\text{ weeks}^2, \qquad E[X^4]=\frac{4!}{\lambda^4} =\frac{24}{(1/4)^4} =24\cdot256 =6144\text{ weeks}^4.

The MGF is $M_X(t)=\frac{1/4}{1/4-t}$, and it is finite exactly for $t<1/4$, where $t$ is measured in inverse weeks. In scale notation, $\beta=4$ weeks, so the same boundary is $t<1/\beta=1/4$ per week.

Common Mistakes and a Practical Check

A frequent error is to substitute the rate into a formula that expects the scale, or vice versa. If $\lambda$ is the rate, the mean is $1/\lambda$ and the MGF denominator is $\lambda-t$. If $\beta$ is the scale, the mean is $\beta$ and the MGF denominator is $1-\beta t$. The two parameterizations agree only after using $\beta=1/\lambda$.

Another mistake is to report the rational expression $\lambda/(\lambda-t)$ for every real $t$. That expression is an MGF formula only when $t<\lambda$. At $t=\lambda$, its denominator is zero and the defining expectation diverges; for $t>\lambda$, the defining integral diverges as well. A formula obtained by algebra cannot override the domain of the integral from which it came.

A quick consistency check is to compare the first two derivatives at zero with the directly calculated moments. They must give $E[X]=1/\lambda$ and $E[X^2]=2/\lambda^2$, so the resulting variance is $1/\lambda^2$. Also check units: $t$ has inverse-time units, making $\lambda-t$ dimensionally valid.

Key takeaway: For rate $\lambda>0$, the exponential raw moments are $E[X^k]=k!/\lambda^k$ for nonnegative integers $k$; consequently $E[X]=1/\lambda$ and $\operatorname{Var}(X)=1/\lambda^2$. Its MGF is $\lambda/(\lambda-t)$ exactly when $t<\lambda$, and is infinite for $t\geq\lambda$.

Check Your Understanding

Try these questions using the rate parameterization unless a scale is specified.

If $X\sim\operatorname{Exponential}(5)$, find $E[X]$, $\operatorname{Var}(X)$, and $E[X^3]$.
For a rate $\lambda$, use the MGF to calculate $M_X'(0)$ and $M_X''(0)$. Which raw moments do these equal?
For $X\sim\operatorname{Exponential}(3)$, is $M_X(3)$ finite? Explain using the defining integral, not only the rational formula.
An exponential waiting time has scale $6$ minutes. State its rate and the real domain of its MGF.
Explain why substituting $t>\lambda$ into $\lambda/(\lambda-t)$ does not give a valid finite MGF value.