Moments and Variance of Gamma Distributions

Why Gamma Moments Depend on the Parameterization

The gamma distribution is described using a positive shape parameter and either a rate or a scale parameter. Those two conventions describe the same family of distributions, but the numerical parameter has a different meaning: a larger rate concentrates the distribution closer to zero, whereas a larger scale stretches it out. Deriving the moments from the density makes this distinction precise and gives a reliable way to check calculations.

The key step is simple: when a power of $x$ is included in a gamma density, the exponent of $x$ increases. The gamma-function recurrence then turns the resulting integral into a product of factors. This yields not only the mean and variance but a formula for every raw moment that exists.

Gamma Densities and the Gamma-Function Recurrence

Let the shape parameter be $\alpha>0$. In the rate parameterization, the rate is $\lambda>0$; in the scale parameterization, the scale is $\theta>0$. For $x>0$, their densities are

f(x)=\frac{\lambda^\alpha}{\Gamma(\alpha)}x^{\alpha-1}e^{-\lambda x} \quad\text{(rate \(\lambda\))}, \qquad f(x)=\frac{1}{\Gamma(\alpha)\theta^\alpha}x^{\alpha-1}e^{-x/\theta} \quad\text{(scale \(\theta\))}.

Both densities are zero for $x\leq 0$. The rate and scale describe the same distribution when $\theta=1/\lambda$. The gamma function is defined, for $z>0$, by

\Gamma(z)=\int_0^\infty t^{z-1}e^{-t}\,dt.

For completeness, its recurrence follows by integration by parts. For $z>0$, take $u=t^z$ and $dv=e^{-t}\,dt$. The boundary term $ -t^ze^{-t}$ vanishes at zero because $z>0$, and at infinity because exponential decay dominates a power. Thus

\Gamma(z+1)=\int_0^\infty t^ze^{-t}\,dt =z\int_0^\infty t^{z-1}e^{-t}\,dt =z\Gamma(z).

This recurrence is what reduces the moment integrals to finite products. In particular, repeated application gives $\Gamma(\alpha+k)/\Gamma(\alpha)=\alpha(\alpha+1)\cdots(\alpha+k-1)$ for a positive integer $k$; for $k=0$, the ratio is 1.

Definition: The $k$th raw moment of $X$ is $E[X^k]$, when the expectation exists. The mean is the first raw moment, $E[X]$, and the variance is $\operatorname{Var}(X)=E[(X-E[X])^2]$.

Deriving the Raw Moments

The same calculation works for any real exponent $r$ for which the integral is finite. For a rate-parameterized gamma variable, near zero the integrand for $E[X^r]$ is proportional to $x^{\alpha+r-1}$. Its integral is finite at zero exactly when $\alpha+r>0$. At infinity, the factor $e^{-\lambda x}$, with $\lambda>0$, makes the integral finite for every fixed real $r$. Consequently, the moment exists exactly when $r>-\alpha$.

Theorem: If $X$ has a gamma distribution with shape $\alpha>0$ and rate $\lambda>0$, then, for real $r>-\alpha$, $$ E[X^r]=\frac{\Gamma(\alpha+r)}{\Gamma(\alpha)\lambda^r}. $$ In the scale parameterization with scale $\theta>0$, the corresponding formula is $$ E[X^r]=\theta^r\frac{\Gamma(\alpha+r)}{\Gamma(\alpha)}. $$

Proof for the rate parameterization. By the density and the definition of expectation,

E[X^r] =\int_0^\infty x^r\frac{\lambda^\alpha}{\Gamma(\alpha)} x^{\alpha-1}e^{-\lambda x}\,dx =\frac{\lambda^\alpha}{\Gamma(\alpha)} \int_0^\infty x^{\alpha+r-1}e^{-\lambda x}\,dx.

Set $t=\lambda x$, so $x=t/\lambda$ and $dx=dt/\lambda$. The integral becomes

\int_0^\infty x^{\alpha+r-1}e^{-\lambda x}\,dx =\lambda^{-(\alpha+r)} \int_0^\infty t^{\alpha+r-1}e^{-t}\,dt =\lambda^{-(\alpha+r)}\Gamma(\alpha+r).

The final gamma integral is finite because $\alpha+r>0$. Multiplying by the density's factor $\lambda^\alpha/\Gamma(\alpha)$ gives the stated formula. For the scale parameterization, the substitution $t=x/\theta$ gives a factor $\theta^{\alpha+r}$ from the integral and a factor $\theta^{-\alpha}$ from the density, leaving $\theta^r$. This proves both formulas. $\square$

For integer $k\geq 0$, the recurrence makes the rate formula especially convenient:

E[X^k] =\frac{\alpha(\alpha+1)\cdots(\alpha+k-1)}{\lambda^k}, \qquad k\geq 1,

and $E[X^0]=1$. In the scale parameterization, replace $1/\lambda^k$ with $\theta^k$. The first two moments immediately give the mean and variance.

Theorem: If $X$ has shape $\alpha>0$, then $$ E[X]=\frac{\alpha}{\lambda},\qquad \operatorname{Var}(X)=\frac{\alpha}{\lambda^2} $$ in the rate parameterization, and $$ E[X]=\alpha\theta,\qquad \operatorname{Var}(X)=\alpha\theta^2 $$ in the scale parameterization.

Derivation. Substituting $k=1$ and $k=2$ into the raw-moment formula gives, under the rate convention,

E[X]=\frac{\alpha}{\lambda}, \qquad E[X^2]=\frac{\alpha(\alpha+1)}{\lambda^2}.

By expanding the definition of variance and using $E[X]=\alpha/\lambda$,

\begin{aligned} \operatorname{Var}(X) &=E[(X-E[X])^2]\\ &=E[X^2]-2E[X]E[X]+(E[X])^2\\ &=E[X^2]-(E[X])^2\\ &=\frac{\alpha(\alpha+1)}{\lambda^2} -\frac{\alpha^2}{\lambda^2} =\frac{\alpha}{\lambda^2}. \end{aligned}

Under the scale convention, the same raw-moment calculation gives $E[X]=\alpha\theta$ and $E[X^2]=\alpha(\alpha+1)\theta^2$. Subtracting the square of the mean gives $\alpha(\alpha+1)\theta^2-\alpha^2\theta^2=\alpha\theta^2$. Thus the displayed mean and variance formulas follow in both parameterizations. $\square$

Worked Calculations

Worked Example: First Two Moments With a Rate

Suppose $X$ has a gamma distribution with shape $\alpha=3$ and rate $\lambda=2$. Find its first two raw moments, mean, and variance.

For the first raw moment, the recurrence gives $\Gamma(4)=3\Gamma(3)$, so

E[X]=\frac{\Gamma(4)}{\Gamma(3)2} =\frac{3}{2}=1.5.

For the second raw moment, $\Gamma(5)=4\Gamma(4)=4\cdot3\Gamma(3)$. Therefore

E[X^2]=\frac{\Gamma(5)}{\Gamma(3)2^2} =\frac{4\cdot3}{4}=3.

The variance is the second raw moment minus the squared mean:

\operatorname{Var}(X)=3-(1.5)^2=3-2.25=0.75.

This agrees with the direct formula $\alpha/\lambda^2=3/4=0.75$.

Worked Example: Calculating Moments With a Scale

Let $X$ have shape $\alpha=2.5$ and scale $\theta=4$. Find $E[X]$, $E[X^2]$, and $\operatorname{Var}(X)$.

The first raw moment is

E[X]=\alpha\theta=2.5(4)=10.

The second raw moment uses both recurrence factors:

E[X^2]=\theta^2\alpha(\alpha+1) =4^2(2.5)(3.5) =16(8.75)=140.

Thus

\operatorname{Var}(X)=E[X^2]-(E[X])^2 =140-100=40.

The direct scale formula checks the result: $\alpha\theta^2=2.5(16)=40$.

Worked Example: Checking a Rate-Scale Conversion

A gamma distribution has shape $\alpha=4$ and rate $\lambda=0.5$. Express it using a scale and calculate its mean and variance under both descriptions.

The equivalent scale is the reciprocal rate:

\theta=\frac{1}{\lambda}=\frac{1}{0.5}=2.

Using the rate formulas gives

E[X]=\frac{4}{0.5}=8, \qquad \operatorname{Var}(X)=\frac{4}{(0.5)^2} =\frac{4}{0.25}=16.

Using the scale formulas gives the same quantities:

E[X]=4(2)=8, \qquad \operatorname{Var}(X)=4(2^2)=16.

The agreement is a useful check: changing from rate to scale changes the parameter's numerical value to its reciprocal, not the distribution's moments.

Common Mistakes and Why the Formulas Matter

The first common mistake is to use the rate in the position of the scale. A rate has units of inverse $x$, while a scale has the same units as $x$. Accordingly, the mean is $\alpha/\lambda$ under a rate and $\alpha\theta$ under a scale. If $\lambda=0.5$, dividing by $0.5$ increases the mean; multiplying by $0.5$ would incorrectly shrink it.

A second mistake is to call $E[X^2]$ the variance. It is the second raw moment, not the variance. The required correction is to subtract $(E[X])^2$. For example, in the first worked example, $E[X^2]=3$, but the variance is $3-(1.5)^2=0.75$, not 3.

These formulas are useful when a model uses a gamma distribution to represent a positive quantity: they give its average and spread directly from its parameters. They also provide quick consistency checks when a problem switches parameterization. Keep the shape fixed, convert rate to reciprocal scale, and verify that both descriptions produce the same mean and variance.

Key takeaway: The gamma raw moment is obtained by multiplying the density by a power of $x$, changing variables in the resulting integral, and applying the gamma-function recurrence. For shape $\alpha$, the mean is $\alpha/\lambda$ and the variance is $\alpha/\lambda^2$ with rate $\lambda$; with scale $\theta$, they are $\alpha\theta$ and $\alpha\theta^2$.

Check Your Understanding

Use the raw-moment formula and distinguish carefully between rate and scale.

For shape $\alpha=5$ and rate $\lambda=2$, calculate $E[X]$, $E[X^2]$, and $\operatorname{Var}(X)$.
For shape $\alpha=3$ and scale $\theta=0.5$, calculate the mean and variance.
State the condition on a real exponent $r$ for $E[X^r]$ to exist when $X$ has shape $\alpha$.
A gamma variable has shape 2 and rate 4. What is its equivalent scale, and what is its mean under either parameterization?
Explain why the second raw moment alone is not the variance.