Gamma Sampling With Integer Shape

Why Summing Exponentials Generates a Gamma Variate

A gamma random variable with an integer shape parameter can be generated without directly inverting a gamma cumulative distribution function. Instead, generate several independent exponential variates with a common rate and add them. The number of terms determines the gamma shape, while the exponential rate determines the gamma scale.

This construction is useful in simulation because exponential variates can be generated from uniform random numbers using the inverse transform. It also gives a direct way to check the distribution's mean and variance before using simulated values in a model. The method described here is aligned with the SOA Exam ASTAM (Advanced Short-Term Actuarial Mathematics) syllabus.

Definition: Let \(E_1,\ldots,E_k\) be independent exponential random variables, each with rate \(\lambda>0\), so that \(f_E(x)=\lambda e^{-\lambda x}\) for \(x>0\). Their sum \(G=\sum_{j=1}^k E_j\), for a positive integer \(k\), has a gamma distribution with shape \(k\) and scale \(\theta=1/\lambda\). We write \(G\sim\operatorname{Gamma}(k,\theta)\). In this shape-scale parameterization, its density is \(f_G(x)=x^{k-1}e^{-x/\theta}/(\Gamma(k)\theta^k)\) for \(x>0\), where \(\Gamma(k)=(k-1)!\) for integer \(k\).

The words rate and scale describe reciprocal parameters, not interchangeable labels. If the exponential rate is \(\lambda\), then the gamma scale is \(\theta=1/\lambda\). The gamma shape \(k\) is a positive integer in this construction because it counts the exponential terms being added.

Turning Uniform Draws into a Gamma Draw

For a uniform random variable \(U\) on \((0,1)\), the inverse transform for an exponential variable with rate \(\lambda\) is \(E=-\ln(U)/\lambda\). Since \(1-U\) is also uniform on \((0,1)\), \(E=-\ln(1-U)/\lambda\) is an equally valid alternative. Use independent uniform draws for the different exponential terms; independence is what makes the sum have the stated gamma distribution.

Set the target parameters.
Choose the positive integer shape \(k\) and the positive scale \(\theta\). Convert to the common exponential rate \(\lambda=1/\theta\).

Generate independent uniforms.
Draw \(U_1,\ldots,U_k\) independently from the uniform distribution on \((0,1)\).

Transform and add.
For each draw, compute \(E_j=-\ln(U_j)/\lambda\), then return \(G=\sum_{j=1}^k E_j\).

Check the distribution and moments.
The result has shape \(k\), scale \(1/\lambda\), mean \(k/\lambda\), and variance \(k/\lambda^2\).

Why the Sum Has a Gamma Distribution

The distribution can be established from the density of a sum of independent random variables. For \(k=1\), the sum is just one exponential variate, with density \(\lambda e^{-\lambda x}\) for \(x>0\). Suppose that the sum of \(k\) such variates has density \(f_k(x)=\lambda^k x^{k-1}e^{-\lambda x}/(k-1)!\). Adding one more independent exponential variate gives the convolution

\begin{aligned} f_{k+1}(x) &=\int_0^x f_k(t)\lambda e^{-\lambda(x-t)}\,dt\\ &=\int_0^x \frac{\lambda^k t^{k-1}e^{-\lambda t}}{(k-1)!} \lambda e^{-\lambda(x-t)}\,dt\\ &=\frac{\lambda^{k+1}e^{-\lambda x}}{(k-1)!} \int_0^x t^{k-1}\,dt\\ &=\frac{\lambda^{k+1}x^k e^{-\lambda x}}{k!},\qquad x>0. \end{aligned}

The second equality uses independence: the density of the sum is the convolution of the two densities. The exponential terms combine because \(e^{-\lambda t}e^{-\lambda(x-t)}=e^{-\lambda x}\). The final expression is the gamma density with shape \(k+1\) and rate \(\lambda\), or equivalently scale \(1/\lambda\). Starting from \(k=1\), this proves the stated density for each positive integer shape.

The mean and variance follow directly from adding independent exponential variates. One exponential variate with rate \(\lambda\) has mean \(1/\lambda\) and variance \(1/\lambda^2\). Expectations add for any sum, and variances add when the terms are independent. Thus

E[G]=\frac{k}{\lambda}=k\theta, \qquad \operatorname{Var}(G)=\frac{k}{\lambda^2}=k\theta^2.

These formulas are also useful checks on a proposed parameter conversion. They do not say that a particular simulated value must equal the mean; the mean describes the distribution over repeated draws, not a required value for each draw.

Worked Example: Generating a Gamma Variate From Three Uniforms

A hypothetical simulation requires a gamma variate with shape \(k=3\) and scale \(\theta=0.5\). The independent uniform draws are \(U_1=e^{-0.2}\), \(U_2=e^{-0.5}\), and \(U_3=e^{-1}\). Generate the variate, state its distribution, and calculate its mean and variance.

Method and parameter conversion. Use the sum-of-exponentials method. The common exponential rate is \(\lambda=1/\theta=1/0.5=2\).

First exponential draw. \(E_1=-\ln(e^{-0.2})/2=0.2/2=0.10\).

Second exponential draw. \(E_2=-\ln(e^{-0.5})/2=0.5/2=0.25\).

Third exponential draw. \(E_3=-\ln(e^{-1})/2=1/2=0.50\).

Sum. \(G=E_1+E_2+E_3=0.10+0.25+0.50=0.85\). The generated value is \(0.85\).

Distribution and moment check. The sum has distribution \(\operatorname{Gamma}(3,0.5)\), in shape-scale notation. Its mean is \(3(0.5)=1.5\), and its variance is \(3(0.5)^2=3(0.25)=0.75\). The sampled value \(0.85\) need not equal the mean; it is one realization from that distribution.

Common mistake avoided. The exponential rate is \(2\), but the gamma scale is \(0.5\). Treating the rate as the scale would incorrectly give mean \(3(2)=6\). The correct mean is \(3/2=1.5\).

Applying the Method and Checking the Parameters

A problem may give a rate rather than a scale, or may specify a desired mean and variance. Translate all such information into one consistent parameterization before transforming any uniforms. If the rate is given, divide each negative log uniform by that rate. If the scale is given, multiply each negative log uniform by the scale, because \(1/\lambda=\theta\).

Worked Example: Using a Given Exponential Rate

A hypothetical model calls for a gamma variate formed from \(k=4\) independent exponential variates, each with rate \(\lambda=0.4\). The uniform draws are \(e^{-0.4}\), \(e^{-1.2}\), \(e^{-0.8}\), and \(e^{-0.6}\). Calculate the gamma variate and its distributional mean and variance.

Method and scale. Use the sum-of-exponentials method. The gamma scale is \(\theta=1/0.4=2.5\).

Transform each uniform. Dividing each negative log by \(0.4\) gives \(E_1=0.4/0.4=1\), \(E_2=1.2/0.4=3\), \(E_3=0.8/0.4=2\), and \(E_4=0.6/0.4=1.5\).

Sum the draws. \(G=1+3+2+1.5=7.5\). Thus the generated variate is \(7.5\).

Check the gamma parameters and moments. Four independent exponential terms give shape \(4\), and the common scale is \(2.5\). Therefore \(G\sim\operatorname{Gamma}(4,2.5)\), \(E[G]=4(2.5)=10\), and \(\operatorname{Var}(G)=4(2.5)^2=4(6.25)=25\).

Common mistake avoided. A given rate of \(0.4\) does not mean that each exponential draw is \(0.4\) times the negative log uniform. The correct operation is \(-\ln(U)/0.4\), which here is equivalent to multiplying \(-\ln(U)\) by the scale \(2.5\).

Worked Example: Recovering the Scale From a Target Mean

A hypothetical simulation specification requires an integer-shape gamma variate with shape \(5\) and mean \(12.5\). Use the independent uniforms \(e^{-0.1}\), \(e^{-0.3}\), \(e^{-0.5}\), \(e^{-0.8}\), and \(e^{-0.2}\) to generate one variate. Determine the scale and variance, and check that the parameters agree with the target mean.

Method and scale calculation. For shape \(k=5\), the mean is \(k\theta\). Therefore \(\theta=12.5/5=2.5\), so the exponential rate is \(\lambda=1/2.5=0.4\).

Transform the five uniforms. Their negative logarithms are \(0.1,0.3,0.5,0.8,0.2\). Dividing each by \(0.4\) gives \(E_1=0.1/0.4=0.25\), \(E_2=0.3/0.4=0.75\), \(E_3=0.5/0.4=1.25\), \(E_4=0.8/0.4=2\), and \(E_5=0.2/0.4=0.5\).

Sum and check. \(G=0.25+0.75+1.25+2+0.5=4.75\). The result has distribution \(\operatorname{Gamma}(5,2.5)\). Its mean is \(5(2.5)=12.5\), as specified, and its variance is \(5(2.5)^2=5(6.25)=31.25\).

Common mistake avoided. The target mean alone does not give the exponential rate until it is divided by the shape: the scale is \(12.5/5=2.5\), not \(12.5\). The corresponding rate is \(0.4\).

Assumptions, Limits, and Exam Traps

The construction applies when the shape is a positive integer and all exponential terms have the same rate. It also requires independent exponential terms. If the terms have different rates, the sum is not generally a gamma variate with the parameters above. If the terms are dependent, knowing their individual exponential distributions is not enough to conclude that their sum has the stated gamma distribution.

The integer restriction is specific to this sum construction: the number of exponential draws must be a whole number. A gamma distribution can have a noninteger shape, but generating such a variate by this particular sum-of-exponentials method is not justified. Similarly, the formulas \(E[G]=k\theta\) and \(\operatorname{Var}(G)=k\theta^2\) are based on a common scale; do not apply them by simply adding terms with different rates.

In an exam-style calculation, keep three checks visible: the number of independent terms equals the shape; the rate and scale are reciprocals; and the variance uses the square of the scale. For instance, with shape \(k\) and rate \(\lambda\), the variance is \(k/\lambda^2\), not \(k/\lambda\). The latter expression is the mean.

Key takeaway: To generate \(\operatorname{Gamma}(k,\theta)\) for positive integer \(k\), generate \(k\) independent uniforms and calculate \(G=-\theta\sum_{j=1}^k\ln(U_j)\). The resulting distribution has mean \(k\theta\) and variance \(k\theta^2\).

Check Your Understanding

For each problem, identify the scale or rate carefully before calculating. Answers are provided for checking.

A hypothetical simulation uses two independent exponential draws with rate \(4\), generated from uniforms \(e^{-1}\) and \(e^{-3}\). Calculate the sum and state the gamma mean and variance. (Answer: The exponential draws are \(0.25\) and \(0.75\), so the sum is \(1\); the distribution is \(\operatorname{Gamma}(2,0.25)\), with mean \(0.5\) and variance \(0.125\).)
A gamma variate is formed by summing three independent exponential draws, each with scale \(1.5\). State its shape-scale distribution, mean, variance, and common exponential rate. (Answer: Shape \(3\), scale \(1.5\), rate \(2/3\), mean \(4.5\), and variance \(6.75\).)
Four independent uniforms are \(e^{-0.25}\), \(e^{-0.5}\), \(e^{-1}\), and \(e^{-1.5}\). Transform and sum them using exponential rate \(0.5\), then state the gamma mean and variance. (Answer: The exponential draws are \(0.5,1,2,3\), their sum is \(6.5\), and the distribution is \(\operatorname{Gamma}(4,2)\), with mean \(8\) and variance \(16\).)
A hypothetical gamma target has integer shape \(6\) and mean \(9\). Find the scale, exponential rate, and variance required for the sum construction. (Answer: Scale \(1.5\), rate \(2/3\), and variance \(6(1.5)^2=13.5\).)
A model sums three exponential variates, each with rate \(2\), but the variates are dependent. Can the gamma density and moment formulas from this construction be asserted solely from the stated marginal distributions? Explain the condition that is missing. (Answer: No. Independence is required for the convolution argument and for adding variances as stated; marginal exponential distributions alone do not establish the gamma sum distribution.)

What You'll Learn