Gamma Log-Link Regression Scores

Why the Gamma Log-Link Score Has Its Own Weighting

A regression score tells us how the log-likelihood changes when a coefficient changes. For a gamma model with a log link, each observation contributes according to its relative residual, not just its raw difference from the fitted mean. This distinction matters when fitted means vary substantially: the same absolute discrepancy can have different influence on the score at different mean levels.

We derive the score directly from the gamma likelihood, keeping the dispersion fixed. We then compare it with the Poisson log-link score and work through examples showing both the difference and a special case in which the two models give the same intercept estimate.

Model and Likelihood

Suppose \(Y_1,\ldots,Y_n\) are independent gamma observations with means \(\mu_i>0\) and common fixed dispersion \(\phi>0\). Use the gamma parameterization in which the shape is \(1/\phi\) and the scale is \(\phi\mu_i\). Thus \(\operatorname{Var}(Y_i)=\phi\mu_i^2\). For a log-link regression, let \(x_i\) be the \(p\)-dimensional predictor vector, \(\beta\) the coefficient vector, and

\eta_i=x_i^\mathsf{T}\beta,\qquad \mu_i=\exp(\eta_i).

Here \(y_i>0\) is an observed response. The density at \(y_i\), as a function of \(\mu_i\) with \(\phi\) fixed, is

f(y_i;\mu_i,\phi) = \frac{y_i^{1/\phi-1}\exp\{-y_i/(\phi\mu_i)\}} {\Gamma(1/\phi)(\phi\mu_i)^{1/\phi}}.

Independence makes the log-likelihood a sum of observation-level contributions. Terms that depend on the coefficients are the terms involving \(\mu_i\):

\ell(\beta) = \sum_{i=1}^n \left[ -\frac{y_i}{\phi\mu_i} -\frac{1}{\phi}\log(\mu_i) \right] +C,

where \(C\) does not depend on \(\beta\). This expression uses the specified gamma density and assumes every observed \(y_i\) is positive. The dispersion is fixed throughout the derivation; it is not being differentiated.

Definition: The coefficient score is the vector of partial derivatives of the log-likelihood with respect to the regression coefficients: \(U(\beta)=\partial\ell(\beta)/\partial\beta\). A score equation sets this vector equal to zero. In the gamma log-link model, the contribution from observation \(i\) will be proportional to \(x_i(y_i-\mu_i)/\mu_i\).

Deriving the Coefficient Score

First differentiate the coefficient-dependent log-likelihood contribution with respect to its linear predictor \(\eta_i\). Since \(\mu_i=\exp(\eta_i)\), we have \(d\mu_i/d\eta_i=\mu_i\). Applying the chain rule gives

\begin{aligned} \frac{\partial \ell_i}{\partial\eta_i} &= \frac{\partial}{\partial\eta_i} \left[-\frac{y_i}{\phi\mu_i}-\frac{1}{\phi}\log(\mu_i)\right]\\ &= \frac{y_i}{\phi\mu_i^2}\frac{\partial\mu_i}{\partial\eta_i} -\frac{1}{\phi}\frac{1}{\mu_i}\frac{\partial\mu_i}{\partial\eta_i}\\ &= \frac{y_i}{\phi\mu_i}-\frac{1}{\phi} = \frac{y_i-\mu_i}{\phi\mu_i}. \end{aligned}

Also, \(\partial\eta_i/\partial\beta=x_i\). Therefore each observation contributes \(x_i\) times its derivative with respect to \(\eta_i\), and summing the contributions yields the score.

Result: For independent gamma observations with fixed common dispersion \(\phi>0\), means \(\mu_i=\exp(x_i^\mathsf{T}\beta)\), and positive observed responses, the coefficient score is

U(\beta) = \sum_{i=1}^n x_i\frac{y_i-\mu_i}{\phi\mu_i} = \frac{1}{\phi}\sum_{i=1}^n x_i\left(\frac{y_i}{\mu_i}-1\right).

The score equation is \(U(\beta)=0\).

This derivation is a complete chain-rule calculation: the log-likelihood separates into a finite sum, and each term is differentiable for \(\mu_i>0\). The log link ensures that \(\mu_i\) remains positive for every finite coefficient vector. No limit or integral interchange is needed.

The form of the score gives its weighting interpretation. The raw residual is \(y_i-\mu_i\); dividing by \(\mu_i\) converts it to the relative residual \(y_i/\mu_i-1\). The fixed factor \(1/\phi\) scales the whole score. Consequently, at the same absolute residual, a smaller fitted mean produces a larger magnitude contribution than a larger fitted mean.

Worked Example: One Observation’s Gamma Score Contribution

Consider one observation with \(x_1=(1,2)^\mathsf{T}\), observed response \(y_1=6\), fixed dispersion \(\phi=0.2\), and current coefficients \(\beta=(\log 4,0)^\mathsf{T}\). Its linear predictor is \(\eta_1=\log 4\), so \(\mu_1=4\).

The residual is \(y_1-\mu_1=2\), and \(\phi\mu_1=0.2(4)=0.8\). The scalar multiplier on \(x_1\) in the gamma score is therefore \(2/0.8=2.5\). The score contribution is

x_1\frac{y_1-\mu_1}{\phi\mu_1} = \begin{pmatrix}1\\2\end{pmatrix}\frac{6-4}{0.2(4)} = \begin{pmatrix}2.5\\5\end{pmatrix}.

For comparison, a Poisson log-link observation with the same predictor vector and fitted mean has score contribution \(x_1(y_1-\mu_1)=(2,4)^\mathsf{T}\). The difference is the gamma factor \(1/(\phi\mu_1)\) multiplying the raw residual.

Comparison with the Poisson Log-Link Score

For independent Poisson responses with means \(\mu_i=\exp(x_i^\mathsf{T}\beta)\), differentiating the Poisson log-likelihood gives the coefficient score

U_{\mathrm{Pois}}(\beta)=\sum_{i=1}^n x_i(y_i-\mu_i).

The gamma log-link score instead is

U_{\mathrm{Gam}}(\beta) = \sum_{i=1}^n x_i(y_i-\mu_i)\frac{1}{\phi\mu_i}.

Thus the Poisson score multiplies each predictor vector by a raw residual. The gamma score multiplies it by a residual divided by its fitted mean and by the dispersion. Since \(\phi\) is common and fixed, multiplying the entire gamma score equation by \(\phi\) does not change its solutions. But it does not remove the observation-specific factor \(1/\mu_i\). When means differ, gamma and Poisson equations generally weight residuals differently.

Worked Example: Comparing Scores Across Three Fitted Means

Take three observations with covariate values \(z=(0,1,2)\), predictor vectors \(x_i=(1,z_i)^\mathsf{T}\), coefficients \(\beta=(\log 2,\log 2)^\mathsf{T}\), and observed responses \((y_1,y_2,y_3)=(1,6,4)\). The fitted means are \(\mu_i=\exp(\log 2+z_i\log 2)\), namely \(2,4,8\). Set \(\phi=0.5\).

For the gamma score, the scalar contributions before multiplying by \(x_i\) are

\frac{y_1-\mu_1}{\phi\mu_1}=\frac{-1}{0.5(2)}=-1,\quad \frac{y_2-\mu_2}{\phi\mu_2}=\frac{2}{0.5(4)}=1,\quad \frac{y_3-\mu_3}{\phi\mu_3}=\frac{-4}{0.5(8)}=-1.

The coefficient score is consequently

U_{\mathrm{Gam}}(\beta) = \begin{pmatrix}1\\0\end{pmatrix}(-1) + \begin{pmatrix}1\\1\end{pmatrix}(1) + \begin{pmatrix}1\\2\end{pmatrix}(-1) = \begin{pmatrix}-1\\-1\end{pmatrix}.

For the Poisson score, the raw residuals are \(-1,2,-4\). Hence

U_{\mathrm{Pois}}(\beta) = \begin{pmatrix}1\\0\end{pmatrix}(-1) + \begin{pmatrix}1\\1\end{pmatrix}(2) + \begin{pmatrix}1\\2\end{pmatrix}(-4) = \begin{pmatrix}-3\\-6\end{pmatrix}.

These are scores at the same coefficient vector and fitted means. Their different values result from the gamma score’s relative-residual weighting, not from a different link or a different set of predictors.

An Intercept-Only Score Equation

A useful special case makes the score equation especially transparent. With an intercept alone, \(x_i=1\) and every fitted mean equals \(\mu=\exp(\beta)\). The gamma score is

U(\beta) = \frac{1}{\phi}\sum_{i=1}^n\left(\frac{y_i}{\mu}-1\right) = \frac{1}{\phi}\left(\frac{\sum_{i=1}^n y_i}{\mu}-n\right).

Setting this score to zero gives \(\mu=n^{-1}\sum_i y_i\). This coincidence with the Poisson intercept estimate does not mean that the observation-level score contributions are the same: it follows because the common \(\mu\) factors out of the intercept-only equation.

Worked Example: Solving and Checking an Intercept-Only Gamma Equation

Let the observed responses be \(2,4,9\), with fixed dispersion \(\phi=0.5\), and fit an intercept-only gamma log-link model. Write \(\mu=\exp(\beta)\). The score is

U(\beta) = \frac{1}{0.5}\left(\frac{2+4+9}{\mu}-3\right) = 2\left(\frac{15}{\mu}-3\right).

The score equation gives \(15/\mu-3=0\), so \(\mu=5\), and therefore \(\hat\beta=\log 5\). Checking directly at \(\mu=5\), the relative residuals are

\frac{2-5}{5}=-0.6,\qquad \frac{4-5}{5}=-0.2,\qquad \frac{9-5}{5}=0.8.

Their sum is zero, so the score is \(2(-0.6-0.2+0.8)=0\), as required. The score is positive when \(0<\mu<5\), because \(15/\mu>3\), and negative when \(\mu>5\). Since \(\mu=\exp(\beta)\) increases with \(\beta\), the log-likelihood increases up to this root and decreases afterward; it is the unique maximum in the intercept parameter. The Poisson intercept equation also gives \(\mu=15/3=5\), although its individual contributions are the raw residuals \(-3,-1,4\), rather than the gamma relative residuals.

Common Mistake and Practical Interpretation

A common mistake is to write the gamma log-link score as \(\sum_i x_i(y_i-\mu_i)\), borrowing the Poisson formula because both models use a log link. The link determines how the mean changes with the linear predictor, but it does not determine the likelihood derivative by itself. The gamma likelihood’s mean-dependent terms produce the additional factor \(1/(\phi\mu_i)\). The correction is to differentiate the relevant likelihood contribution first, then apply the chain rule from \(\eta_i\) to \(\beta\).

This weighting is useful to keep in mind when interpreting a fitted gamma model. Two observations can have equal absolute residuals but different score contributions if their fitted means differ. Conversely, equal relative residuals give equal scalar score contributions when \(\phi\) is common. This is a statement about the coefficient score, not a claim that each observation’s final influence is determined by the residual alone: the predictor vector also matters.

Key takeaway: With fixed dispersion and a gamma log link, observation \(i\) contributes \(x_i(y_i-\mu_i)/(\phi\mu_i)\) to the coefficient score. Poisson log-link regression instead uses \(x_i(y_i-\mu_i)\). The gamma score is therefore based on relative residuals, and its weighting varies with the fitted mean.

Check Your Understanding

Use the likelihood and score forms in this tutorial to answer the following.

For one gamma observation, how does the coefficient score contribution change if its fitted mean doubles while its raw residual stays fixed?
Why does multiplying the full gamma score equation by the common fixed dispersion not make it a Poisson score equation?
For an intercept-only gamma model, show from the score equation why the fitted mean equals the sample arithmetic mean.
At the same fitted means, what distinguishes the scalar residual factors in the gamma and Poisson log-link scores?
In a regression with varying fitted means, why should one not infer that gamma and Poisson coefficient estimates will generally coincide just because both use a log link?