Worked Gamma Log-Link Regression

From a Fitted Model to Numerical Results

A gamma log-link regression turns a linear predictor into a positive fitted mean. In a worked analysis, the calculation does not end at the coefficients: we also need to quantify how dispersion affects uncertainty in fitted means and in comparisons between covariate values. This tutorial carries out those calculations and makes clear which uncertainties refer to the mean and which refer to a future response.

Assume independent responses \(Y_i\) with gamma distributions, means \(\mu_i\), and a common positive dispersion parameter \(\phi\), so that \(\operatorname{Var}(Y_i)=\phi\mu_i^2\). For a design vector \(x_i\) and coefficient vector \(\beta\), the log link specifies \(\log(\mu_i)=x_i^\mathsf{T}\beta\). Thus \(\hat{\mu}_i=\exp(x_i^\mathsf{T}\hat{\beta})\). The calculations below use the usual large-sample Wald approximation for coefficient uncertainty; the resulting intervals are approximate, not exact finite-sample intervals.

Worked Example: Fitted Means at Two Covariate Values

Why Dispersion Enters Coefficient Uncertainty

The covariance formula can be derived directly from the gamma likelihood. For this derivation, use the gamma density parameterized by mean \(\mu_i\) and dispersion \(\phi\):

With \(\log(\mu_i)=x_i^\mathsf{T}\beta\), terms in the log-likelihood that depend on \(\beta\) are \(y_i/(\phi\mu_i)\) with a negative sign and \(-\log(\mu_i)/\phi\). Differentiation gives the score contribution; differentiating once more gives the Hessian contribution:

Because \(E[Y_i/\mu_i]=1\), the expected negative Hessian for observation \(i\) is \(x_ix_i^\mathsf{T}/\phi\). Independence means the expected information contributions add. Summing over observations proves that the information matrix is \(X^\mathsf{T}X/\phi\). When \(X\) has full column rank, its inverse is \(\phi(X^\mathsf{T}X)^{-1}\). Under the usual regularity conditions for maximum likelihood asymptotics—an identifiable interior parameter, a correctly specified model, and a nonsingular limiting information matrix—this is the large-sample covariance of the coefficient estimator. Replacing \(\phi\) by an estimate gives the stated plug-in covariance estimate. \(\square\)

This derivation explains the role of dispersion: larger \(\phi\) multiplies coefficient variances, and hence standard errors, by a larger amount. In a routine fit, \(\phi\) may be estimated in several ways. The next example uses the Pearson estimate divided by residual degrees of freedom; it is an estimate, not a declaration that the true dispersion is known.

Worked Example: A Two-Group Fit and Its Dispersion-Adjusted Uncertainty

Uncertainty for Means and Contrasts

The calculations above follow from a general first-order result. Suppose \(\hat{\beta}\) is approximately normal with mean \(\beta\) and covariance matrix \(V\), and let \(g(\beta)\) be differentiable. A first-order Taylor expansion around \(\beta\) gives \(g(\hat{\beta})\approx g(\beta)+\nabla g(\beta)^\mathsf{T}(\hat{\beta}-\beta)\). The variance of this linear term is \(\nabla g(\beta)^\mathsf{T}V\nabla g(\beta)\). Substituting estimates for unknown quantities yields the usual delta-method variance estimate. For a log-link fitted mean at design vector \(x\), the gradient of \(\mu=\exp(x^\mathsf{T}\beta)\) is \(\mu x\). For a log mean ratio between two design vectors, define \(c=x_a-x_b\); the ratio is \(R=\exp(c^\mathsf{T}\beta)\), with log-ratio standard error \(\sqrt{c^\mathsf{T}Vc}\). These approximations require the coefficient estimator to have the stated large-sample normal approximation and the relevant covariance matrix to be estimable.

Worked Example: A Continuous-Covariate Contrast

Common Mistake: Confusing Mean Uncertainty with Response Variation

A fitted mean is an estimated model quantity, whereas a future gamma response also has intrinsic variation around that mean. For a future response at covariate vector \(x\), the model variance is \(\phi\mu^2\). The estimated uncertainty in the fitted mean instead comes from uncertainty in \(\hat{\beta}\); on the log scale its estimated variance is \(x^\mathsf{T}\widehat{\operatorname{Cov}}(\hat{\beta})x\). These quantities answer different questions and should not be substituted for one another.

• Wrong reasoning: “The standard error of the fitted mean is \(\sqrt{\hat{\phi}}\hat{\mu}\), because that is the gamma standard deviation.”
• Correction: \(\sqrt{\hat{\phi}}\hat{\mu}\) estimates the response standard deviation at a specified mean. For uncertainty in the estimated mean, first calculate the log-mean variance \(x^\mathsf{T}\widehat{\operatorname{Cov}}(\hat{\beta})x\), then use the delta method if a standard error on the response mean scale is wanted.
• Wrong reasoning: “A mean-ratio standard error can be calculated from the standard error of one coefficient in every contrast.”
• Correction: Use the whole contrast \(c\) and the full coefficient covariance matrix: the log-ratio variance is \(c^\mathsf{T}\widehat{\operatorname{Cov}}(\hat{\beta})c\). Covariances between coefficient estimates can matter.