Gamma Log-Link Regression

Why Use a Log Link for a Gamma Response?

A gamma response model is useful when the outcome is continuous and strictly positive, such as a duration, cost, or concentration. Its mean must therefore be positive as well. A log link ensures this constraint automatically: whatever finite real-valued linear predictor the covariates produce, its exponential is positive.

This tutorial specifies the gamma model with a log link, obtains its fitted means, and derives the likelihood score for the regression coefficients. The focus is on the model calculation and score; the coefficient estimates themselves generally require solving the resulting equations using the observed data.

Model Specification

Suppose $Y_1,\ldots,Y_n$ are independent responses, each observed with a covariate vector $x_i\in\mathbb{R}^p$. Assume $y_i>0$, use a common dispersion parameter $\phi>0$, and let the coefficient vector be $\beta\in\mathbb{R}^p$. The gamma parameterization used here gives the response mean $\mu_i$ and variance $\phi\mu_i^2$.

Definition: A gamma log-link regression model specifies, for each observation $i$, $$ Y_i\sim \operatorname{Gamma}(\mu_i,\phi),\qquad E[Y_i]=\mu_i,\qquad \operatorname{Var}(Y_i)=\phi\mu_i^2, $$ with $$ \log(\mu_i)=\eta_i=x_i^\mathsf{T}\beta. $$ Equivalently, the fitted mean is $\mu_i=\exp(x_i^\mathsf{T}\beta)$. The gamma density, for $y_i>0$, is $$ f(y_i;\mu_i,\phi)= \frac{y_i^{1/\phi-1}\exp\{-y_i/(\phi\mu_i)\}} {\Gamma(1/\phi)(\phi\mu_i)^{1/\phi}}. $$

In this parameterization, the gamma shape is $1/\phi$ and the scale is $\phi\mu_i$. Their product is the mean, $(1/\phi)(\phi\mu_i)=\mu_i$, and the shape times the squared scale is the variance, $(1/\phi)(\phi\mu_i)^2=\phi\mu_i^2$. The design vector ordinarily includes a first component equal to 1 for the intercept. The model assumes the responses are conditionally independent given their covariates and the model parameters.

The log link is the function $g(\mu)=\log\mu$, defined for $\mu>0$, and its inverse is $g^{-1}(\eta)=\exp(\eta)$, defined for every real $\eta$. Thus every finite coefficient vector and finite covariate vector give a valid positive fitted mean. This link is different from the gamma canonical negative-inverse link: here it is the logarithm of the mean that is linear in the covariates.

Worked Example: Calculating Fitted Means

Suppose the response is a positive cost, measured in hundreds of dollars. Let $x$ be a continuous covariate and let $d$ be a binary indicator. Use the model $$ \log(\mu)=1.2+0.08x-0.35d. $$ Calculate the fitted mean for $x=5,d=0$; $x=5,d=1$; $x=10,d=0$; and $x=10,d=1$.

For $x=5,d=0$, the linear predictor is $1.2+0.08(5)-0.35(0)=1.60$, so $\hat\mu=\exp(1.60)\approx4.953$. For $x=5,d=1$, it is $1.2+0.08(5)-0.35(1)=1.25$, so $\hat\mu=\exp(1.25)\approx3.490$.

For $x=10,d=0$, the predictor is $1.2+0.08(10)=2.00$, giving $\hat\mu=\exp(2.00)\approx7.389$. For $x=10,d=1$, it is $1.2+0.08(10)-0.35=1.65$, giving $\hat\mu=\exp(1.65)\approx5.207$. Each prediction is positive, as required for a gamma mean. In the stated response units, the first fitted mean is about 495.30 dollars.

Deriving the Coefficient Score

For independent observations, the likelihood is the product of the gamma densities. Fix $\phi>0$ while differentiating with respect to $\beta$. This gives the coefficient score conditional on the dispersion value; it does not require treating $\phi$ as known for every other inferential purpose.

Theorem: For independent gamma responses with the log-link model above, the log-likelihood, up to terms not involving $\beta$, is $$ \ell(\beta)= -\frac{1}{\phi}\sum_{i=1}^n \left(\log\mu_i+\frac{y_i}{\mu_i}\right)+C, \qquad \mu_i=\exp(x_i^\mathsf{T}\beta), $$ where $C$ does not depend on $\beta$. Its score vector is $$ U(\beta)=\frac{\partial\ell(\beta)}{\partial\beta} =\frac{1}{\phi}\sum_{i=1}^n x_i \left(\frac{y_i}{\mu_i}-1\right). $$

Proof. Taking the logarithm of the gamma density gives $$ \log f(y_i;\mu_i,\phi)= \left(\frac{1}{\phi}-1\right)\log y_i -\frac{y_i}{\phi\mu_i} -\log\Gamma(1/\phi) -\frac{1}{\phi}\log\phi -\frac{1}{\phi}\log\mu_i. $$ The terms involving $y_i$ alone or $\phi$ alone do not depend on $\beta$. Summing over observations leaves the stated likelihood expression, with all such terms absorbed into $C$.

Since $\log\mu_i=x_i^\mathsf{T}\beta$, its gradient with respect to $\beta$ is $x_i$. Also, $\mu_i=\exp(x_i^\mathsf{T}\beta)$, so the chain rule gives $$ \frac{\partial\mu_i}{\partial\beta}=\mu_i x_i, \qquad \frac{\partial(1/\mu_i)}{\partial\beta}=-\frac{x_i}{\mu_i}. $$ Therefore, $$ \frac{\partial}{\partial\beta} \left[-\frac{1}{\phi}\left(\log\mu_i+\frac{y_i}{\mu_i}\right)\right] =-\frac{1}{\phi}\left(x_i-\frac{y_i x_i}{\mu_i}\right) =\frac{x_i}{\phi}\left(\frac{y_i}{\mu_i}-1\right). $$ Summing these gradients proves the score formula. All differentiations are valid for finite $\beta$, since $\mu_i>0$ and the log-likelihood is differentiable there. $\square$

The maximum-likelihood score equations set each component of $U(\beta)$ equal to zero. In matrix notation, if $X$ has rows $x_i^\mathsf{T}$, the same equation is $$ U(\beta)=\frac{1}{\phi}X^\mathsf{T}r,\qquad r_i=\frac{y_i}{\mu_i}-1. $$ The terms $r_i$ compare each observation with its fitted mean on a relative scale. In general, because the fitted means depend nonlinearly on $\beta$, the score equations do not give a separate closed-form solution for each coefficient.

Worked Example: Evaluating the Score at a Candidate Coefficient Vector

Consider an intercept and a binary covariate $z$, with $x_i=(1,z_i)^\mathsf{T}$. Let $\phi=0.5$, and evaluate the score at $\beta=(\log 2,\log 2)^\mathsf{T}$. The four observations are $(z,y)=(0,1),(0,2),(1,2),(1,4)$.

The fitted mean is $\mu=\exp(\log 2+z\log 2)$. Thus it is 2 when $z=0$ and 4 when $z=1$. The four ratios $y_i/\mu_i$, in the order listed, are $1/2,1,1/2,1$. The score has an intercept component and a $z$-coefficient component: $$ U_0=\frac{1}{0.5}\left[ \left(\frac12-1\right)+(1-1)+ \left(\frac12-1\right)+(1-1)\right]=-2, $$ $$ U_1=\frac{1}{0.5}\left[ 0\left(\frac12-1\right)+0(1-1)+ 1\left(\frac12-1\right)+1(1-1)\right]=-1. $$ Hence $U(\beta)=(-2,-1)^\mathsf{T}$, not the zero vector. This candidate coefficient vector does not satisfy the likelihood score equations. The calculation evaluates the score at the specified values; it does not by itself determine the maximum-likelihood estimate.

An Intercept-Only Case With a Closed-Form Estimate

A useful check on the score formula is the intercept-only model. Here every design vector is $x_i=1$, so all observations share a fitted mean $\mu=\exp(\beta_0)$. The score reduces to one scalar equation. In this special case it can be solved explicitly, and the solution is the arithmetic sample mean.

Theorem: Suppose $y_1,\ldots,y_n>0$ are independent gamma observations with common dispersion $\phi>0$ and an intercept-only log-link model. The likelihood has a unique global maximum over $\mu>0$ at $$ \hat\mu=\frac{1}{n}\sum_{i=1}^n y_i. $$

Proof. With $\beta_0=\log\mu$, the part of the log-likelihood that depends on $\mu$ is $$ \ell(\mu)=-\frac{1}{\phi}\left(n\log\mu+\frac{\sum_{i=1}^n y_i}{\mu}\right)+C. $$ Differentiating with respect to $\mu>0$ yields $$ \frac{d\ell}{d\mu} =\frac{\sum_{i=1}^n y_i-n\mu}{\phi\mu^2}. $$ The denominator is positive. The derivative is positive when $0<\mu<\bar y$, zero at $\mu=\bar y$, and negative when $\mu>\bar y$, where $\bar y=n^{-1}\sum_i y_i>0$. Thus the likelihood increases up to $\bar y$ and decreases after it, establishing a unique global maximum there. $\square$

Worked Example: Fitting an Intercept-Only Gamma Model

Suppose the observed positive responses are $2,4,6$, and the common dispersion is $\phi=0.25$. The intercept-only result gives $$ \hat\mu=\frac{2+4+6}{3}=4,\qquad \hat\beta_0=\log 4\approx1.3863. $$

Check the score directly. At $\hat\mu=4$, the ratios are $2/4=0.5$, $4/4=1$, and $6/4=1.5$, so $$ U(\hat\beta_0)=\frac{1}{0.25}\left[(0.5-1)+(1-1)+(1.5-1)\right] =4(-0.5+0+0.5)=0. $$ The score equation is satisfied. The derivative sign in the theorem also verifies that this stationary point is the unique global maximum, rather than merely a root of the score equation.

Practical Checks and a Common Mistake

The model specification gives a short sequence of checks when calculating fitted means or deriving a score:

Form the linear predictor.
Substitute the observed covariate values into $x_i^\mathsf{T}\beta$, including the intercept and any indicator coding.

Apply the inverse link.
Exponentiate the predictor to obtain $\mu_i=\exp(x_i^\mathsf{T}\beta)>0$.

Use the gamma score consistently.
For each observation calculate $y_i/\mu_i-1$, multiply by $x_i$, sum, and divide by $\phi$.

Common mistake: treating the linear predictor as the fitted mean. If $x_i^\mathsf{T}\beta=1.60$, it is incorrect to report a fitted mean of 1.60. The predictor is on the log-mean scale, so the fitted mean is $\exp(1.60)\approx4.953$. The correction is to apply the inverse link before reporting or using the fitted mean. A second useful check is that a finite log-link predictor can never yield a nonpositive fitted mean.

The log-link specification is valuable because it separates the unconstrained linear predictor from the positive response mean while preserving a direct likelihood calculation. For computation, the score formula shows exactly how observed responses, fitted means, covariates, and dispersion contribute to coefficient estimation. It also supplies a concrete diagnostic for arithmetic: a claimed solution should make the score components zero, subject to numerical tolerance.

Check Your Understanding

Use the model definition and score formula to answer these questions.

For a log-link gamma model with predictor $0.4+0.2x$, calculate the fitted mean when $x=3$.
State the gamma shape and scale in terms of $\mu_i$ and $\phi$, and verify their product gives the mean.
For one observation with $y_i=6$, $\mu_i=4$, and $\phi=0.5$, what is its scalar contribution before multiplication by $x_i$ in the score?
Why does a finite linear predictor guarantee a positive fitted mean in this model?
In an intercept-only model with observations $3,5,7$, what are the maximum-likelihood estimate of $\mu$ and the corresponding intercept?

What You'll Learn