The Gamma Variance Function

Why the Gamma Variance Depends on the Mean

A Gamma response is positive and often right-skewed. In a generalized linear model, its variance is not constant as the mean changes: a larger mean comes with a larger variance. The exact relationship is quadratic. Deriving it from the exponential-dispersion representation makes clear which part is due to the mean and which part is due to dispersion.

Throughout, take the mean \(\mu\) and dispersion \(\phi\) to satisfy \(\mu>0\) and \(\phi>0\). We use a Gamma parameterization whose shape is \(1/\phi\) and whose scale is \(\phi\mu\). This choice makes \(\mu\) the response mean and \(\phi\) the dispersion in the exponential-dispersion form.

Writing the Density in Exponential-Dispersion Form

An exponential-dispersion density has the form $$ f(y;\theta,\phi) = \exp\left\{\frac{y\theta-b(\theta)}{\phi}+c(y,\phi)\right\}, $$ where \(\theta\) is the natural parameter and \(b\) is the log-partition function. For the Gamma density, take $$ \theta=-\frac{1}{\mu},\qquad b(\theta)=-\log(-\theta),\qquad \theta<0. $$ Since \(-\theta=1/\mu\), this gives \(b(\theta)=\log\mu\). The natural-parameter domain is \(\theta<0\), corresponding exactly to \(\mu>0\).

To verify the representation, take the logarithm of the Gamma density: $$ \log f(y;\mu,\phi) = \left(\frac{1}{\phi}-1\right)\log y -\frac{y}{\phi\mu} -\log\Gamma(1/\phi) -\frac{1}{\phi}\log(\phi\mu). $$ Collect the terms involving \(\mu\) with the terms involving \(y\): $$ \log f(y;\mu,\phi) = \frac{-y/\mu-\log\mu}{\phi} + \left(\left(\frac{1}{\phi}-1\right)\log y -\log\Gamma(1/\phi)-\frac{1}{\phi}\log\phi\right). $$ The first group is \(\{y\theta-b(\theta)\}/\phi\), and the second group depends only on \(y\) and \(\phi\). Thus $$ c(y,\phi)= \left(\frac{1}{\phi}-1\right)\log y -\log\Gamma(1/\phi)-\frac{1}{\phi}\log\phi. $$ This verifies the exponential-dispersion form directly from the density.

Deriving the Variance from the Log-Partition Function

For an exponential-dispersion family with density proportional to \(\exp\{(y\theta-b(\theta))/\phi\}\), the mean and variance are obtained from derivatives of \(b\): $$ E[Y]=b'(\theta),\qquad \operatorname{Var}(Y)=\phi b''(\theta). $$ Here \(\phi\) is held fixed when differentiating with respect to \(\theta\). The differentiations are justified for this Gamma family: on any compact interval of natural parameters contained in \((-\infty,0)\), the density and its first two \(\theta\)-derivatives are bounded by an integrable function of the form \(C(1+y^2)y^{1/\phi-1}\exp(-cy)\), for constants \(C,c>0\). Thus differentiation under the integral is valid.

Proof. The Gamma log-partition function is \(b(\theta)=-\log(-\theta)\), defined for \(\theta<0\). Differentiating gives $$ b'(\theta)=-\frac{1}{\theta},\qquad b''(\theta)=\frac{1}{\theta^2}. $$ Since \(\theta=-1/\mu\), it follows that \(b'(\theta)=\mu\) and \(b''(\theta)=\mu^2\). Applying the mean and variance derivative identities yields $$ E[Y]=b'(\theta)=\mu,\qquad \operatorname{Var}(Y)=\phi b''(\theta)=\phi\mu^2. $$ Therefore \(V(\mu)=\mu^2\), with the multiplier \(\phi\) giving the response variance. \(\square\)

What Quadratic Growth Means

At fixed dispersion, multiplying the mean by a positive factor \(k\) multiplies the variance by \(k^2\). This is a direct consequence of the Gamma variance function, not a separate assumption: $$ \operatorname{Var}(Y\text{ at mean }k\mu) =\phi(k\mu)^2 =k^2\phi\mu^2. $$ For example, doubling the mean multiplies the variance by four. The standard deviation, in contrast, is \(\sqrt{\phi}\mu\), so it grows in direct proportion to the mean.

This also explains why the coefficient of variation is constant across means when \(\phi\) is fixed. The coefficient of variation is the standard deviation divided by the mean. Because both are positive, $$ \frac{\sqrt{\operatorname{Var}(Y)}}{E[Y]} = \frac{\sqrt{\phi\mu^2}}{\mu} = \frac{\sqrt{\phi}\mu}{\mu} = \sqrt{\phi}. $$ Thus the Gamma model keeps relative spread fixed while absolute spread increases with the mean.

Checking the Parameterization Directly

A second route to the same variance is to use the standard Gamma moments. If a Gamma distribution has shape \(\alpha>0\) and scale \(s>0\), then its mean is \(\alpha s\) and its variance is \(\alpha s^2\). In the parameterization used here, \(\alpha=1/\phi\) and \(s=\phi\mu\). Substitution gives $$ E[Y]=\alpha s=\frac{1}{\phi}(\phi\mu)=\mu, \qquad \operatorname{Var}(Y)=\alpha s^2 =\frac{1}{\phi}(\phi\mu)^2 =\phi\mu^2. $$ This check confirms that the exponential-dispersion derivation agrees with the usual Gamma moment formulas.

The mean-variance law has a practical interpretation: the model’s variance increases with the square of the fitted mean. This makes it important to use a distributional assumption compatible with the response’s positive support and increasing spread. The variance function describes how spread changes with the mean; it does not, by itself, determine every feature of a response distribution.

Common Mistake

A frequent error is to report \(V(\mu)=\phi\mu^2\) and then multiply by \(\phi\) again when calculating the variance. In the convention used here, \(V(\mu)=\mu^2\) is the variance function, while the response variance is \(\phi V(\mu)=\phi\mu^2\). For \(\mu=10\) and \(\phi=0.04\), the variance is \(0.04(100)=4\), not \(0.04^2(100)=0.16\).

A related mistake is to conclude that the standard deviation is quadratic in the mean. Taking the square root gives \(\sqrt{\phi\mu^2}=\sqrt{\phi}\mu\), because \(\mu>0\). The standard deviation grows linearly with the mean, even though the variance grows quadratically.