Exponential and Gamma Cumulants

Why These Cumulants Have a Simple Pattern

The exponential and gamma distributions are central continuous models with cumulants that can be calculated at every order. Their formulas also make parameterization matter: a gamma distribution may be described by a rate or by a scale, and confusing those parameters changes the answer. We will derive the moment-generating functions, take their logarithms, and obtain the cumulants under both conventions.

The gamma family includes the exponential family as the special case with shape parameter one. Checking that relationship is more than a notational convenience: it is a useful way to verify that the formulas agree across two common descriptions of the same distribution.

Definitions and Parameterizations

Definition: A random variable $X$ has a gamma distribution with shape $\alpha>0$ and rate $\beta>0$, written here as $X\sim\operatorname{Gamma}(\alpha,\beta)$, if its density is $$ f(x)=\frac{\beta^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\beta x},\qquad x>0. $$ The gamma function is $\Gamma(\alpha)=\int_0^\infty u^{\alpha-1}e^{-u}\,du$ for $\alpha>0$. The scale parameterization uses $\theta=1/\beta$, with density $f(x)=x^{\alpha-1}e^{-x/\theta}/(\Gamma(\alpha)\theta^\alpha)$ for $x>0$. An exponential distribution with rate $\lambda>0$, written $X\sim\operatorname{Exp}(\lambda)$, has density $f(x)=\lambda e^{-\lambda x}$ for $x>0$.

The moment-generating function is $M_X(t)=E[e^{tX}]$ wherever this expectation is finite. When it is finite on an open interval containing zero, the cumulant-generating function is $K_X(t)=\log M_X(t)$ on that interval, and the cumulant of order $r$ is $\kappa_r=K_X^{(r)}(0)$, for $r=1,2,\ldots$. In particular, $\kappa_1=E[X]$ and $\kappa_2=\operatorname{Var}(X)$.

Deriving the Gamma Cumulant-Generating Function

For a gamma random variable with shape $\alpha$ and rate $\beta$, integrate against its density. For $t<\beta$, the exponential factor in the integral decays at infinity, and the gamma integral gives

\begin{aligned} M_X(t) &=\int_0^\infty e^{tx}\frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha-1}e^{-\beta x}\,dx\\ &=\frac{\beta^\alpha}{\Gamma(\alpha)} \int_0^\infty x^{\alpha-1}e^{-(\beta-t)x}\,dx\\ &=\frac{\beta^\alpha}{(\beta-t)^\alpha} =\left(1-\frac{t}{\beta}\right)^{-\alpha}. \end{aligned}

The integral in the second line is finite precisely when $\beta-t>0$. If $t\geq\beta$, its integrand does not decay sufficiently at infinity, so the expectation diverges. Thus the domain of finiteness is $t<\beta$, which includes an open interval around zero. Taking the logarithm on this domain yields the cumulant-generating function.

Theorem: If $X\sim\operatorname{Gamma}(\alpha,\beta)$ in the rate parameterization, then for $t<\beta$, $$ K_X(t)=-\alpha\log\left(1-\frac{t}{\beta}\right), \qquad \kappa_r=\frac{\alpha(r-1)!}{\beta^r},\quad r=1,2,\ldots. $$ In the scale parameterization $\theta=1/\beta$, these become $K_X(t)=-\alpha\log(1-\theta t)$, for $t<1/\theta$, and $\kappa_r=\alpha(r-1)!\theta^r$.

To prove the cumulant formula, differentiate the displayed expression for $K_X$. For every integer $r\geq1$, repeated differentiation gives

K_X^{(r)}(t)= \frac{\alpha(r-1)!}{\beta^r} \left(1-\frac{t}{\beta}\right)^{-r}.

This identity follows by induction: the first derivative is $\alpha\beta^{-1}(1-t/\beta)^{-1}$; differentiating the order-$r$ expression multiplies its coefficient by $r/\beta$ and raises the exponent of the final factor from $-r$ to $-(r+1)$. Evaluating at $t=0$ gives $\kappa_r=\alpha(r-1)!/\beta^r$. Since $\theta=1/\beta$, substitution gives the scale formulas. All derivatives exist at zero because zero lies inside the domain $t<\beta$. $\square$

The Exponential Case

The exponential distribution is the gamma distribution with shape $\alpha=1$ and rate $\beta=\lambda$. Direct integration gives the same result, and also makes its domain especially transparent.

Theorem: If $X\sim\operatorname{Exp}(\lambda)$, then $M_X(t)=\lambda/(\lambda-t)$ and $K_X(t)=-\log(1-t/\lambda)$ for $t<\lambda$. Its cumulants are $$ \kappa_r=\frac{(r-1)!}{\lambda^r},\quad r=1,2,\ldots. $$

Indeed, for $t<\lambda$,

M_X(t)=\int_0^\infty e^{tx}\lambda e^{-\lambda x}\,dx =\lambda\int_0^\infty e^{-(\lambda-t)x}\,dx =\frac{\lambda}{\lambda-t}.

For $t\geq\lambda$, the integral diverges, so the domain is exactly $t<\lambda$. Taking a logarithm gives $K_X(t)=-\log(1-t/\lambda)$. Differentiating $r$ times gives $K_X^{(r)}(t)=(r-1)!\lambda^{-r}(1-t/\lambda)^{-r}$, and setting $t=0$ proves the stated formula. This is also the gamma formula with $\alpha=1$ and $\beta=\lambda$. $\square$

Worked Applications

Worked Example: Exponential Cumulants at Rate One-Half

Let $X\sim\operatorname{Exp}(1/2)$. Find its cumulant-generating function and its first four cumulants.

Here $\lambda=1/2$, so the domain is $t<1/2$. Substituting the rate into the formula gives $$ K_X(t)=-\log\left(1-\frac{t}{1/2}\right)=-\log(1-2t). $$

For each order $r$, $\kappa_r=(r-1)!/(1/2)^r=(r-1)!2^r$. In particular, $$ \kappa_1=1\cdot2=2,\quad \kappa_2=1\cdot4=4,\quad \kappa_3=2\cdot8=16,\quad \kappa_4=6\cdot16=96. $$

Thus the mean is $2$ and the variance is $4$. The third and fourth cumulants are $16$ and $96$, respectively. The domain check is consistent with the MGF $M_X(t)=1/(1-2t)$, which is finite for $t<1/2$.

Worked Example: Gamma Cumulants With a Rate Parameter

Let $X\sim\operatorname{Gamma}(3,2)$, where the second argument is the rate. Find $K_X(t)$, its domain, and the first four cumulants.

The rate is $\beta=2$, so the domain is $t<2$. The cumulant-generating function is $$ K_X(t)=-3\log\left(1-\frac{t}{2}\right). $$

Using $\kappa_r=3(r-1)!/2^r$, we obtain $$ \kappa_1=\frac{3}{2},\quad \kappa_2=\frac{3}{4},\quad \kappa_3=\frac{6}{8}=\frac{3}{4},\quad \kappa_4=\frac{18}{16}=\frac{9}{8}. $$

In particular, $E[X]=3/2$ and $\operatorname{Var}(X)=3/4$. As a direct consistency check, the rate-parameter density has mean shape divided by rate, $3/2$, and variance shape divided by rate squared, $3/4$, matching the first two cumulants.

Worked Example: Gamma Cumulants With a Scale Parameter

Suppose $Y$ has a gamma distribution with shape $\alpha=5/2$ and scale $\theta=2$. Find its cumulant-generating function and first four cumulants, then identify its equivalent rate.

The scale formula gives the domain $t<1/\theta=1/2$ and $$ K_Y(t)=-\frac{5}{2}\log(1-2t). $$

For each $r$, $\kappa_r=(5/2)(r-1)!2^r$. Therefore $$ \kappa_1=5,\quad \kappa_2=10,\quad \kappa_3=\frac{5}{2}\cdot2\cdot8=40,\quad \kappa_4=\frac{5}{2}\cdot6\cdot16=240. $$

The equivalent rate is $\beta=1/\theta=1/2$. Substituting that rate into the rate formula gives, for example, $\kappa_2=(5/2)/(1/2)^2=10$ and $\kappa_4=(5/2)6/(1/2)^4=240$, agreeing with the scale calculation.

How the Exponential Fits the Gamma Formulas

Set $\alpha=1$ in the gamma rate density. Since $\Gamma(1)=1$, the density becomes $\beta e^{-\beta x}$ for $x>0$, exactly the exponential density with rate $\lambda=\beta$. The gamma cumulant-generating function becomes $$ K_X(t)=-\log\left(1-\frac{t}{\beta}\right), $$ and its cumulants reduce to $(r-1)!/\beta^r$, the exponential formulas. In the scale convention, shape one and scale $\theta$ correspond to exponential rate $\lambda=1/\theta$, not $\lambda=\theta$.

This special case provides a practical check on any calculation: substituting shape one into a gamma result must recover the exponential result under matching rate and scale conventions. It also shows that the factor $\alpha$ in every gamma cumulant is essential; it represents the shape dependence rather than a rate or scale adjustment.

Common Parameterization Pitfall

A frequent error is to use the scale in a formula written for the rate. For example, with scale $\theta=2$, the rate is $1/2$. The mean is $\alpha\theta$, not $\alpha/\theta$; the second cumulant is $\alpha\theta^2$, not $\alpha/\theta^2$. The cure is to write down the density or explicitly convert $\beta=1/\theta$ before substituting.

A second error is to report a formula without its domain. For a gamma distribution the MGF and CGF are finite for $t<\beta$ under the rate convention, or $t<1/\theta$ under the scale convention. The cumulants are derivatives at zero, and the open neighborhood of zero contained in that domain is what makes these derivatives available.

Key takeaway: For shape $\alpha$, rate $\beta$, and scale $\theta=1/\beta$, the gamma cumulants are $\alpha(r-1)!/\beta^r=\alpha(r-1)!\theta^r$. Exponential cumulants follow by setting $\alpha=1$ and matching the exponential rate to the gamma rate.

Check Your Understanding

Use the parameterization stated in each question, and show the rate-scale conversion when needed.

For $X\sim\operatorname{Gamma}(\alpha,\beta)$ in the rate convention, state the domain of its MGF and write its cumulant-generating function.
A gamma random variable has shape $4$ and scale $3$. What is its equivalent rate, and what are its first two cumulants?
Show from the general gamma cumulant formula that an exponential random variable with rate $\lambda$ has fourth cumulant $6/\lambda^4$.
For shape $1$ and scale $\theta$, identify the equivalent exponential rate and state the domain of the CGF.
Explain why an MGF formula valid for $t<\beta$ must not be used as a finite expectation when $t\geq\beta$.