Exponential and Gamma Moments from MGFs

From Distribution Form to Moments

The exponential and gamma distributions are useful models for positive quantities such as waiting times and accumulated event times. Their densities look different, but their moment-generating functions have a compact form that makes repeated differentiation systematic. The main care required is to keep the rate and scale parameterizations distinct: they are reciprocals, and confusing them changes both the MGF and the resulting moments.

We use \(M_X(t)=E[e^{tX}]\) for the moment-generating function (MGF), wherever this expectation is finite on an open interval containing zero. When such an interval exists, its derivatives at zero give raw moments: \(M_X^{(r)}(0)=E[X^r]\) for nonnegative integers \(r\). The earlier material on MGF domains and differentiation establishes the conditions for this rule; here we apply it to these two distribution families.

MGFs Under Rate and Scale Conventions

In the scale convention, the gamma density instead uses a scale \(\theta>0\): \(f(x)=x^{\alpha-1}e^{-x/\theta}/(\Gamma(\alpha)\theta^\alpha)\). These describe the same family when \(\theta=1/\lambda\). Some texts write \(\operatorname{Gamma}(\alpha,\theta)\) for this scale form; always check which second parameter is intended.

To obtain the rate-form MGF, combine the density and the factor \(e^{tx}\). If \(t<\lambda\), the resulting integral converges, and the change of variable \(u=(\lambda-t)x\) gives

The last equality follows from the defining gamma integral after the change of variable. If \(t\geq\lambda\), the integrand does not have an exponentially decreasing tail, and the expectation is infinite. Substituting \(\lambda=1/\theta\) gives the scale form and its domain. Setting \(\alpha=1\), with \(\Gamma(1)=1\), gives the exponential result.

Raw Moments by Differentiation

For compact notation, define the rising factorial \((\alpha)^{\overline r}=\alpha(\alpha+1)\cdots(\alpha+r-1)\) for integer \(r\geq1\), and \((\alpha)^{\overline 0}=1\). The MGF formula yields every raw moment in one expression.

Proof. Write \(M_X(t)=\lambda^\alpha(\lambda-t)^{-\alpha}\). Repeated differentiation gives

This formula holds for \(r=0\). If it holds for \(r\), differentiating the power \((\lambda-t)^{-(\alpha+r)}\) multiplies it by \(\alpha+r\) and changes its exponent to \(-(\alpha+r+1)\), giving the formula for \(r+1\). Thus it holds for every nonnegative integer \(r\). Since \(0\) lies inside the MGF's open domain \(t<\lambda\), the MGF differentiation rule applies. Evaluating at zero gives \(E[X^r]=M_X^{(r)}(0)=(\alpha)^{\overline r}/\lambda^r\). Replacing \(\lambda\) by \(1/\theta\) gives the scale expression. For \(\alpha=1\), \((1)^{\overline r}=r!\), giving the exponential formula. \(\square\)

Central Moments from the Centered MGF

A central moment is a moment of the centered variable \(X-E[X]\). The centered MGF is \(E[e^{t(X-E[X])}]=e^{-tE[X]}M_X(t)\). For a gamma variable, the mean from the raw-moment formula is \(\alpha/\lambda\). Using the scale-free variable \(z=t/\lambda\), its centered MGF is

The logarithmic expansion \(-\log(1-z)=\sum_{j=1}^{\infty}z^j/j\) is valid for \(|z|<1\). Hence the logarithm of this centered MGF is \(\alpha(z^2/2+z^3/3+z^4/4+\cdots)\). Expanding its exponential through the fourth power gives the central moments below.

Derivation. Put \(Z=X-E[X]\). Through order four, the centered MGF, expressed as a power series in \(z=t/\lambda\), is

There is no linear term because \(E[Z]=0\). The coefficient of \(t^r\) in an MGF is \(E[Z^r]/r!\). The coefficient of \(t^2\) is \(\alpha/(2\lambda^2)\), so \(E[Z^2]=2!\alpha/(2\lambda^2)=\alpha/\lambda^2\). The coefficient of \(t^3\) is \(\alpha/(3\lambda^3)\), so \(E[Z^3]=3!\alpha/(3\lambda^3)=2\alpha/\lambda^3\). Finally, the coefficient of \(t^4\) is \((\alpha/4+\alpha^2/8)/\lambda^4\), so

All these derivatives are valid because the centered MGF is finite on an open interval containing zero. The scale versions follow from \(\lambda=1/\theta\). \(\square\)

Parameterization and Moment Pitfalls

The formulas are easy to misapply when the second gamma parameter is not identified as a rate or a scale. For shape \(\alpha\), rate \(\lambda\), the mean is \(\alpha/\lambda\); for shape \(\alpha\), scale \(\theta\), it is \(\alpha\theta\). The MGF domains differ in the same reciprocal way: \(t<\lambda\) versus \(t<1/\theta\). Before differentiating, write down the density or explicitly state the convention.

A second common mistake is to call \(E[X^2]\) the variance. It is the second raw moment; variance requires subtracting the squared mean: \(\operatorname{Var}(X)=E[X^2]-(E[X])^2\). For the rate-\(2\), shape-\(3\) example, the raw second moment is \(3\), whereas the variance is \(3-(3/2)^2=3/4\). These quantities are not interchangeable.