Why the Gamma Distribution Has a GLM Form
The Gamma distribution is used for positive, continuous responses whose variability often grows as their mean grows. To use it as a response family, it helps to express its density in exponential-family form: this makes the mean and variance explicit and shows how a single dispersion parameter controls their relationship. Here the focus is the case in which the Gamma shape is fixed, while the mean may vary across observations.
Parameterizations differ across textbooks and software. We will use shape \(k>0\) and scale \(s>0\), so that a Gamma random variable \(Y\) has density proportional to \(y^{k-1}\exp(-y/s)\) on positive values. In the model below, \(k\) is common and the scale changes with the mean. Stating the parameterization is essential: the same Gamma distribution can also be described using a rate, which is the reciprocal of the scale.
The density uses scale \(\mu/k\), not mean \(\mu\). For example, if \(k=4\) and \(\mu=10\), then the scale is \(10/4=2.5\). The mean is the product of shape and scale, \(4(2.5)=10\). The variance is shape times scale squared, \(4(2.5)^2=25\). These relationships are the starting point for the exponential-family representation.
Writing the Density in Exponential-Family Form
Use the dispersion-form convention $$ f(y\mid \theta,\phi)=\exp\left\{\frac{y\theta-b(\theta)}{a(\phi)}+c(y,\phi)\right\}. $$ For this Gamma model, set the dispersion to \(\phi=1/k\), so \(a(\phi)=\phi\). The natural parameter and log-partition function are $$ \theta=-\frac{1}{\mu}<0,\qquad b(\theta)=-\log(-\theta). $$ The remaining density term is $$ c(y,\phi)=\left(\frac{1}{\phi}-1\right)\log y-\log\Gamma\left(\frac{1}{\phi}\right)-\frac{1}{\phi}\log\phi. $$
To verify the expression, note that \(-\theta=1/\mu\), so \(b(\theta)=\log\mu\). The part involving the natural parameter becomes \( (y\theta-b(\theta))/\phi=-y/(\phi\mu)-\log(\mu)/\phi\). Since \(k=1/\phi\), adding \(c(y,\phi)\) gives $$ -\frac{y}{\phi\mu}-\frac{\log\mu}{\phi} +\left(\frac{1}{\phi}-1\right)\log y -\log\Gamma\left(\frac{1}{\phi}\right)-\frac{1}{\phi}\log\phi. $$ This is the log of the stated Gamma density: its exponential term is \(\exp[-y/(\phi\mu)]\), and its normalizing factor includes \((\phi\mu)^{-1/\phi}\). Thus the representation has the correct density, not just the correct mean and variance.
Worked Example: Converting Shape to Dispersion
Suppose a hypothetical positive response is modeled as Gamma with fixed shape \(k=4\), and consider an observation with mean \(\mu=10\). Find its scale, dispersion, and variance.
Scale. Since the mean is shape times scale, \(s=\mu/k=10/4=2.5\) response units.
Dispersion. The exponential-family dispersion is the reciprocal of shape: $$ \phi=\frac{1}{k}=\frac{1}{4}=0.25. $$
Variance. Gamma variance is \(ks^2\), so $$ \operatorname{Var}(Y)=4(2.5)^2=4(6.25)=25 $$ squared response units. Equivalently, the variance function gives \(\phi\mu^2=0.25(10)^2=25\). The two calculations agree. In this parameterization, the rate is \(1/s=0.4\) per response unit; it is not the dispersion.
Mean and Variance From the Log-Partition Function
For an exponential-family density with the form above, the standard log-partition identities give \(E(Y)=b'(\theta)\) and \(\operatorname{Var}(Y)=a(\phi)b''(\theta)\). They apply here because the natural parameter is in its valid domain \(\theta<0\), the support \(y>0\) does not depend on \(\theta\), and the density is normalized for the stated parameter values.
Differentiate the Gamma log-partition function: $$ b(\theta)=-\log(-\theta),\qquad b'(\theta)=-\frac{1}{\theta},\qquad b''(\theta)=\frac{1}{\theta^2}. $$ Substitute \(\theta=-1/\mu\). Then \(b'(\theta)=\mu\), and \(b''(\theta)=\mu^2\). Since \(a(\phi)=\phi\), the mean and variance are $$ E(Y)=\mu,\qquad \operatorname{Var}(Y)=\phi\mu^2. $$
This establishes the Gamma variance function \(V(\mu)=\mu^2\), with dispersion multiplier \(\phi\). In particular, holding shape fixed means holding \(\phi\) fixed; it does not mean holding the variance fixed. As the mean changes, the variance changes quadratically. The standard deviation is \(\sqrt{\phi}\mu\), so the coefficient of variation (standard deviation divided by mean) is \(\sqrt{\phi}=1/\sqrt{k}\), constant across means under the common-shape model.
For shape \(k\) and scale \(s\), \(E(Y)=ks\) and \(\operatorname{Var}(Y)=ks^2\).
Set \(s=\mu/k\). The variance becomes \(k(\mu/k)^2=\mu^2/k\).
Writing \(1/k=\phi\) gives \(\operatorname{Var}(Y)=\phi\mu^2\), and the exponential-family form uses the same \(\phi\).
Worked Example: Checking a Density Numerically
Take a hypothetical response \(Y\) with shape \(k=3\) and mean \(\mu=6\). Find the density at \(y=4\), then verify it using the exponential-family expression.
The scale is \(s=6/3=2\), so the shape-scale density gives $$ f(4\mid 6,3)=\frac{4^{3-1}\exp(-4/2)}{\Gamma(3)2^3} =\frac{16\exp(-2)}{2(8)} =\exp(-2)\approx 0.1353. $$
For the exponential-family calculation, \(\phi=1/3\), \(\theta=-1/6\), and \(b(\theta)=\log 6\). The exponent is $$ \frac{4(-1/6)-\log 6}{1/3} +\left(3-1\right)\log 4-\log\Gamma(3)-3\log(1/3). $$
The first term is \(-2-3\log 6\). The remaining terms are \(2\log4-\log2+3\log3\). Together these simplify to \(-2\), since \(2\log4-\log2+3\log3-3\log6=0\). Thus the exponential-family density is \(\exp(-2)\), matching the shape-scale calculation. The density value has units of inverse response units, as a continuous density should.
What Fixed Shape Implies
A common shape is a substantive restriction on the response distribution. It implies that the variance-to-squared-mean ratio is constant: $$ \frac{\operatorname{Var}(Y)}{\mu^2}=\phi=\frac{1}{k}. $$ Equivalently, the squared coefficient of variation is \(\phi\). Two groups can have different means and different variances under this model, but their variance-to-mean-squared ratios should agree if they share the same shape.
The fixed-shape assumption is a statement about the conditional response distribution, not a claim that all observed values are equal or that each observation has the same variance when its mean differs. For a set of responses with different conditional means, each response can have a different variance \(\phi\mu_i^2\), while sharing the same \(\phi\).
Worked Example: Assessing a Common Shape
Suppose hypothetical summaries for two groups are: group A has conditional mean \(4\) response units and variance \(3.2\) squared response units; group B has conditional mean \(8\) response units and variance \(12.8\) squared response units. Check whether these summaries are compatible with one common Gamma shape.
For group A, the variance-to-squared-mean ratio is $$ \frac{3.2}{4^2}=\frac{3.2}{16}=0.20. $$ For group B, it is $$ \frac{12.8}{8^2}=\frac{12.8}{64}=0.20. $$ The ratios agree, so the summaries are compatible with \(\phi=0.20\), corresponding to \(k=1/\phi=5\). Their variances differ because their means differ: the model predicts \(0.20(4^2)=3.2\) and \(0.20(8^2)=12.8\), respectively.
This agreement does not prove that the Gamma distribution or common shape is correct. It only shows that these two mean-variance summaries do not contradict the fixed-shape variance relationship. More complete assessment also considers whether observations are positive and whether the observed distributional shape and tails are plausible.
Parameterizations, Alternatives, and Diagnostics
| Description | Parameterization | Relationship in this tutorial |
|---|---|---|
| Shape and scale | \(k,s\) | \(\mu=ks\), \(\phi=1/k\), \(s=\phi\mu\) |
| Shape and rate | \(k,r\), where \(r=1/s\) | \(\mu=k/r\), \(r=k/\mu\) |
| Mean and dispersion | \(\mu,\phi\) | \(k=1/\phi\), \(\operatorname{Var}(Y)=\phi\mu^2\) |
The shape-scale and shape-rate forms describe the same family; the rate is the reciprocal of the scale. The mean-dispersion form is convenient for GLM notation because it displays the variance as \(\phi V(\mu)\). Do not read \(k\), \(s\), \(r\), and \(\phi\) as interchangeable parameters.
A basic diagnostic check is whether the response values are strictly positive and whether the spread grows roughly in proportion to the squared conditional mean. Exact agreement is not expected in finite samples, but marked differences in variance-to-mean-squared ratios across comparable groups may indicate that one common shape is restrictive. A substantial number of zeros cannot arise from this continuous Gamma density. Strongly incompatible skewness or tail behavior can also signal that the family is a poor description.
A model with a different shape for different observations would no longer have one common dispersion \(\phi\); it requires a different specification. A lognormal model is another possible distribution for positive, right-skewed responses, but it has a different distributional and mean-variance structure. The choice depends on the response scale and scientific assumptions, not simply on which density has a convenient formula. The Normal model with known dispersion is another exponential-family case, but it allows real-valued responses and has a different variance relationship.
Common Mistakes and Limits
- Using the mean as the Gamma scale. With shape \(k\), the scale is \(s=\mu/k\); the mean is \(ks\).
- Calling the shape the dispersion. In this convention, \(\phi=1/k\). Larger shape means smaller relative dispersion.
- Assuming fixed shape means fixed variance. The variance is \(\phi\mu^2\), so it changes with the mean.
- Applying the density at zero or below. Its support is \(y>0\). A standard Gamma response model does not assign positive probability to zero.
- Treating matching variance ratios as proof. The ratio check concerns the mean-variance implication only; it does not establish the full Gamma distribution or other modeling assumptions.
The representation identifies a response distribution and its variance structure. It does not, by itself, specify how the mean depends on predictors or guarantee that the model is appropriate for any particular data set. Those questions require further modeling choices and diagnostic evidence.
Check Your Understanding
Answer each question, then compare with the concise solutions below.
- A Gamma response has fixed shape \(k=2\) and mean \(\mu=5\). What are its scale, dispersion, and variance?
- For a Gamma model with \(\phi=0.1\), what shape and variance are implied when the mean is \(3\)?
- Why does fixed shape not imply a common variance across observations with different means?
- Two groups have mean-variance pairs \((2,1)\) and \((6,9)\). Are they compatible with one common dispersion according to the variance function?
- Can the standard Gamma density used here describe an observed response equal to zero? Explain.
Solutions. (1) \(s=\mu/k=5/2=2.5\), \(\phi=1/k=0.5\), and \(\operatorname{Var}(Y)=\phi\mu^2=0.5(25)=12.5\) squared response units. (2) The shape is \(k=1/\phi=10\), and the variance is \(0.1(3^2)=0.9\) squared response units. (3) A common shape fixes \(\phi\), but variance is \(\phi\mu^2\), which changes when \(\mu\) changes. (4) Yes: the ratios are \(1/2^2=0.25\) and \(9/6^2=0.25\), so both imply \(\phi=0.25\). (5) No. The density has support only on \(y>0\), so a response equal to zero is outside its support.