Reciprocal-Mean Gamma Regression

The Positive Reciprocal Link

A gamma response has a positive mean, so a regression model must produce positive fitted means. With the positive reciprocal link, the model makes the reciprocal of the mean linear in the predictors. This gives a simple formula for obtaining a fitted mean, but also imposes a direct restriction on the range of the linear predictor: it must be positive.

The sign is important. The positive reciprocal link in this tutorial is $g(\mu)=1/\mu$. The canonical gamma link introduced earlier is the negative inverse, $g(\mu)=-1/\mu$. They have different signs and different admissible predictor ranges.

Definition: In a gamma generalized linear model with positive reciprocal link, the mean $\mu_i=E[Y_i]$ and linear predictor $\eta_i=\mathbf{x}_i^\top\boldsymbol{\beta}$ satisfy $$ g(\mu_i)=\frac{1}{\mu_i}=\eta_i, \qquad \mu_i=\frac{1}{\eta_i}. $$ Since $\mu_i>0$, an admissible linear predictor must satisfy $\eta_i>0$.

Here $Y_i$ is a positive gamma response, $\mathbf{x}_i$ is the predictor vector for observation $i$, and $\boldsymbol{\beta}$ is the coefficient vector. As usual for the gamma GLM, the conditional variance is $\operatorname{Var}(Y_i)=\phi\mu_i^2$, where $\phi>0$ is the dispersion parameter. The link specifies the mean; it does not change the gamma response distribution or its mean-variance relationship.

From Linear Predictor to Fitted Mean

Once the coefficients are specified or estimated, prediction follows in two steps: calculate the linear predictor and take its reciprocal. The reciprocal is a valid fitted mean only when the predictor is strictly positive. A zero predictor does not give a finite mean, and a negative predictor gives a negative value, which cannot be a gamma mean.

Worked Example: Calculating Two Fitted Means

Suppose a model has an intercept and a binary predictor $x$, with fitted coefficients $\hat{\beta}_0=0.4$ and $\hat{\beta}_1=0.2$. The linear predictor is $\hat{\eta}=0.4+0.2x$.

For $x=0$, $\hat{\eta}=0.4+0.2(0)=0.4$, so the fitted mean is $\hat{\mu}=1/0.4=2.5$. For $x=1$, $\hat{\eta}=0.4+0.2(1)=0.6$, so $\hat{\mu}=1/0.6=5/3$, approximately $1.667$. Both predictors are positive, and both fitted means are positive and finite.

The calculation is not $\hat{\mu}=\hat{\eta}$, nor is the link applied by taking a reciprocal of each observed response. The reciprocal operation converts the model's predictor into its fitted conditional mean.

The positivity requirement applies to each covariate vector at which the model is used. For fitting to observed data, all observed predictors must satisfy $\mathbf{x}_i^\top\boldsymbol{\beta}>0$. If predictions are needed over a larger covariate range, positivity must also hold throughout that range. Checking only the training observations does not guarantee valid extrapolated predictions.

Worked Example: Checking the Predictor Range

Consider the proposed predictor $\eta(x)=0.6-0.3x$, with $x$ taking values in the range from 0 to 2.

At $x=0$, $\eta=0.6$, giving $\mu=1/0.6=5/3$. At $x=1$, $\eta=0.3$, giving $\mu=1/0.3=10/3$. At $x=2$, $\eta=0$, so $1/\eta$ is not finite; this covariate value does not have an admissible fitted mean.

In fact, $\eta(x)>0$ requires $0.6-0.3x>0$, or $x<2$. Thus the proposed model is admissible on the portion of the specified range below 2, but not at $x=2$. If the observed data include $x=2$, these coefficients cannot be used as a valid fit for all those observations.

Likelihood Fitting Under the Positivity Constraint

For independent gamma observations with means $\mu_i$ and common dispersion $\phi$, the density can be written using shape $1/\phi$ and scale $\phi\mu_i$. Substituting $\mu_i=1/\eta_i$ makes the dependence of the likelihood on the regression coefficients explicit. For a fixed $\phi>0$, the log-likelihood, up to terms that do not depend on $\boldsymbol{\beta}$, is

\ell(\boldsymbol{\beta}) = \frac{1}{\phi}\sum_{i=1}^n \left(\log \eta_i-y_i\eta_i\right) +\text{constant}, \qquad \eta_i=\mathbf{x}_i^\top\boldsymbol{\beta}>0.

The constraint is part of the model, not an optional check after fitting. It restricts the coefficient vectors to those satisfying $\mathbf{x}_i^\top\boldsymbol{\beta}>0$ for every observation. This feasible set is an intersection of open half-spaces, so it is convex. Likelihood optimization must remain within it.

Theorem: For independent gamma responses with fixed $\phi>0$, the positive-reciprocal-link log-likelihood has score $$ \nabla_{\boldsymbol{\beta}}\ell(\boldsymbol{\beta}) = \frac{1}{\phi}\sum_{i=1}^n \mathbf{x}_i\left(\frac{1}{\eta_i}-y_i\right) $$ and Hessian $$ \nabla^2_{\boldsymbol{\beta}}\ell(\boldsymbol{\beta}) = -\frac{1}{\phi}\sum_{i=1}^n \frac{\mathbf{x}_i\mathbf{x}_i^\top}{\eta_i^2}. $$ Consequently, the log-likelihood is concave on the feasible set. If the design matrix has full column rank, it is strictly concave there. Any feasible stationary point is therefore the unique global maximizer.

Proof. For each observation, $\eta_i=\mathbf{x}_i^\top\boldsymbol{\beta}$, so its contribution depending on $\boldsymbol{\beta}$ is $\phi^{-1}(\log\eta_i-y_i\eta_i)$. Differentiating with respect to $\eta_i$ gives $\phi^{-1}(1/\eta_i-y_i)$. Since $\nabla_{\boldsymbol{\beta}}\eta_i=\mathbf{x}_i$, summing these derivatives gives the stated score. Differentiating again gives the stated Hessian.

For any vector $\mathbf{v}$, the Hessian's quadratic form is $$ \mathbf{v}^\top\nabla^2_{\boldsymbol{\beta}}\ell(\boldsymbol{\beta})\mathbf{v} = -\frac{1}{\phi}\sum_{i=1}^n \frac{(\mathbf{x}_i^\top\mathbf{v})^2}{\eta_i^2} \leq 0. $$ This proves concavity. If the design matrix has full column rank and $\mathbf{v}\neq\mathbf{0}$, then at least one $\mathbf{x}_i^\top\mathbf{v}\neq 0$. Because every $\eta_i>0$, the displayed sum is then strictly negative, proving strict concavity. A stationary point of a differentiable concave function on this convex feasible set is a global maximizer; strict concavity makes that maximizer unique. $\square$

The score equations set the displayed score to zero. They generally do not yield a closed-form solution when predictors vary across observations, so the coefficients are found by numerical likelihood optimization subject to the positivity constraints. The theorem establishes uniqueness if a feasible stationary point exists; it does not assert that a finite interior maximum must exist for every data set.

Worked Example: Fitting an Intercept-Only Model

Let $Y_1,Y_2,Y_3$ be independent gamma observations with a common mean $\mu=1/\beta_0$, where the intercept-only predictor is $\eta_i=\beta_0>0$. Suppose the observed values are $2,4,6$.

The score for $\beta_0$ is $$ \frac{1}{\phi}\left(\frac{3}{\beta_0}-(2+4+6)\right) = \frac{1}{\phi}\left(\frac{3}{\beta_0}-12\right). $$

Setting the score to zero gives $3/\beta_0=12$, hence $\hat{\beta}_0=3/12=0.25$. This is feasible because $0.25>0$. The fitted mean is $\hat{\mu}=1/0.25=4$, which equals the sample mean $(2+4+6)/3=4$. The Hessian is $-3/(\phi\beta_0^2)<0$, so this stationary point is the unique maximizer in the feasible intercept-only model.

Fitting a Model With a Categorical Predictor

A useful check on the coefficient calculations is a model with an intercept and a binary predictor. With two groups, this model assigns a separate mean to each group. The positive-reciprocal parameterization expresses each group’s linear predictor as the reciprocal of that group’s mean.

Worked Example: Estimating Two Group Means

Suppose the responses are $2$ and $4$ in group $x=0$, and $1$, $2$, and $3$ in group $x=1$. Use $\eta(x)=\beta_0+\beta_1x$, with both group predictors required to be positive.

The group-$0$ predictor is $\beta_0$. Its score equation is $2/\beta_0-(2+4)=0$, so $\hat{\beta}_0=2/6=1/3$. The group-$1$ predictor is $\beta_0+\beta_1$. Its score equation for that group predictor is $3/(\beta_0+\beta_1)-(1+2+3)=0$, giving $\hat{\beta}_0+\hat{\beta}_1=3/6=1/2$.

Thus $\hat{\beta}_1=1/2-1/3=1/6$. The fitted means are $\hat{\mu}(0)=1/(1/3)=3$ and $\hat{\mu}(1)=1/(1/2)=2$, equal to the observed group averages $(2+4)/2=3$ and $(1+2+3)/3=2$. Both fitted predictors, $1/3$ and $1/2$, satisfy the required positivity condition.

Common Pitfall: Treating Every Coefficient Vector as Valid

A common mistake is to calculate $\mathbf{x}_i^\top\hat{\boldsymbol{\beta}}$ for each row and take its reciprocal without first checking the sign. If any predictor is zero or negative, this does not produce a valid finite positive gamma mean. The correction is to enforce $\mathbf{x}_i^\top\boldsymbol{\beta}>0$ throughout fitting and to recheck the constraint at every covariate value where predictions will be reported.

The same sign check prevents confusion with the canonical negative-inverse link. Under the positive reciprocal link, a positive predictor corresponds to a positive mean. Under the negative-inverse link, the sign convention differs. Carrying the sign from one link into the other changes the model and its admissible region.

Key takeaway: In positive-reciprocal gamma regression, $\eta_i=\mathbf{x}_i^\top\boldsymbol{\beta}=1/\mu_i$. The fitted mean is $1/\eta_i$, and valid fitting and prediction require $\eta_i>0$ at every relevant covariate value.

Check Your Understanding

Use the link definition, likelihood score, and positivity condition to answer these questions.

If a fitted predictor is $0.8$, what is the fitted mean? What if the predictor is zero?
For $\eta(x)=1-0.25x$, determine the values of $x$ for which the fitted mean is admissible.
Write the likelihood score for an intercept-only model with $n$ observations and explain why its fitted predictor must be positive.
Why does full column rank of the design matrix make the log-likelihood strictly concave on the feasible set?
State the sign difference between the positive reciprocal link and the canonical gamma link.