Gamma Identity-Link Regression

Identity-Link Gamma Regression

A gamma response is positive and often has variability that grows with its mean. Gamma regression models that relationship through a mean and variance structure; the link determines how the mean depends on the covariates. With an identity link, the mean itself is linear in the covariates. This gives a direct mean model, but also imposes an essential restriction: every fitted mean must be positive.

We derive the mean specification and coefficient score from the gamma likelihood, then use numerical examples to check the score and the positive-mean condition. The latter check must cover the covariate values where the model is intended to apply, not just the observations used to fit it.

Model Specification and Parameter Space

The density is defined for \(y_i>0\) and \(\mu_i>0\). In this parameterization its shape is \(1/\phi\) and its scale is \(\phi\mu_i\); hence its mean is \((1/\phi)(\phi\mu_i)=\mu_i\), and its variance is \((1/\phi)(\phi\mu_i)^2=\phi\mu_i^2\). The identity link is \(g(\mu)=\mu\), so the linear predictor equals the mean: \(\eta_i=\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta}=\mu_i\).

For observed design vectors, the allowable coefficient set is \(\mathcal{B}=\{\boldsymbol{\beta}:\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta}>0\text{ for every }i\}\). If predictions are required over a covariate set \(\mathcal{C}\), the relevant condition is instead \(\mathbf{x}^\mathsf{T}\boldsymbol{\beta}>0\) for every \(\mathbf{x}\in\mathcal{C}\). A coefficient vector can satisfy the observed-data condition while failing over a wider prediction range.

Deriving the Coefficient Score

For fixed \(\phi\), the gamma density for observation \(i\) is \[ f(y_i\mid\mu_i,\phi)= \frac{y_i^{1/\phi-1}\exp\{-y_i/(\phi\mu_i)\}} {\Gamma(1/\phi)(\phi\mu_i)^{1/\phi}}, \qquad y_i>0,\ \mu_i>0. \] Independence makes the log-likelihood the sum of the individual log-densities. The terms that depend on \(\boldsymbol{\beta}\), with \(\mu_i=\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta}\), are \[ \ell(\boldsymbol{\beta}) =\text{constant}-\frac{1}{\phi}\sum_{i=1}^n \left(\log\mu_i+\frac{y_i}{\mu_i}\right). \] The constant here may depend on the observed \(y_i\) and on fixed \(\phi\), but not on \(\boldsymbol{\beta}\).

To derive it, differentiate one summand with respect to its mean: \[ \frac{\partial}{\partial\mu_i} \left[-\frac{1}{\phi}\left(\log\mu_i+\frac{y_i}{\mu_i}\right)\right] =-\frac{1}{\phi}\left(\frac{1}{\mu_i}-\frac{y_i}{\mu_i^2}\right) =\frac{y_i-\mu_i}{\phi\mu_i^2}. \] Since \(\partial\mu_i/\partial\boldsymbol{\beta}=\mathbf{x}_i\), the chain rule gives the stated contribution for observation \(i\); adding the contributions gives the score. An interior maximum of the likelihood must satisfy \(\mathbf{U}(\boldsymbol{\beta})=\mathbf{0}\). This is a necessary score equation, not a guarantee that a finite maximum exists or that a particular solution is a maximum. \(\square\)

The factor \(1/\mu_i^2\) matters: the score equations are not the ordinary least squares normal equations. Observations with different fitted means contribute differently, even if their residuals \(y_i-\mu_i\) are equal.

Properties of the Score

The score’s conditional expectation is zero when the specified conditional mean is correct. This follows directly from the model mean and does not require the responses to be normally distributed.

Conditioning on the design vectors makes each \(\mathbf{x}_i\), \(\mu_i\), and \(\phi\) fixed. By linearity of conditional expectation, \[ E[\mathbf{U}(\boldsymbol{\beta})\mid\mathbf{x}_1,\ldots,\mathbf{x}_n] =\sum_{i=1}^n\mathbf{x}_i \frac{E[Y_i\mid\mathbf{x}_i]-\mu_i}{\phi\mu_i^2} =\sum_{i=1}^n\mathbf{x}_i\frac{\mu_i-\mu_i}{\phi\mu_i^2} =\mathbf{0}. \] Each denominator is defined because \(\phi>0\) and the model requires \(\mu_i>0\). \(\square\)

The conditional covariance matrix of the score also follows from the gamma variance. Independence makes the covariance of the sum equal to the sum of the individual covariance matrices. For each term, the covariance is \[ \operatorname{Cov}\left( \mathbf{x}_i\frac{Y_i-\mu_i}{\phi\mu_i^2} \middle|\mathbf{x}_i\right) =\frac{\mathbf{x}_i\mathbf{x}_i^\mathsf{T}}{\phi^2\mu_i^4} \operatorname{Var}(Y_i\mid\mathbf{x}_i) =\frac{\mathbf{x}_i\mathbf{x}_i^\mathsf{T}}{\phi\mu_i^2}. \] Therefore, the conditional score covariance is \(\sum_i\mathbf{x}_i\mathbf{x}_i^\mathsf{T}/(\phi\mu_i^2)\). This expression shows again that the score’s contributions depend on the fitted means as well as the residuals.

Checking Positivity Over a Covariate Range

For a single scalar covariate on a closed interval, the identity-link mean is linear, so its minimum occurs at an endpoint. Checking both endpoints is therefore enough to establish positivity throughout that interval. For multiple covariates, a range or set of combinations must be specified; checking each variable’s separate minimum and maximum without considering which combinations are possible may be insufficient. With a finite set of observed design rows, evaluate \(\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta}\) at every row.

Common Pitfall

• Assuming any linear predictor is an admissible gamma mean. A linear expression can be zero or negative. Check every observed fitted mean and, for predictions, the entire intended covariate range.
• Treating the response as Gaussian because the link is identity. The link specifies how the conditional mean depends on covariates. The response distribution here is gamma, with variance \(\phi\mu_i^2\).
• Using ordinary least squares score equations. The gamma coefficient score weights each residual by \(1/(\phi\mu_i^2)\). The ordinary least squares equations do not generally solve this score equation.