Comparing Log-Linked Positive-Response Models

One Mean Structure, Different Response Models

Gamma, inverse-Gaussian, and Tweedie generalized linear models can all use the log link. This gives their conditional means the same convenient multiplicative interpretation: a one-unit covariate change multiplies the modeled mean by a constant factor. It does not, however, make the models distributionally equivalent. Their variance functions differ, and the Tweedie family with power between one and two can also assign positive probability to an exact zero.

The comparison is useful when the scientific target is a positive or nonnegative response and the mean relationship is naturally multiplicative. The link describes how the mean changes with predictors; the family describes the response distribution around that mean. Both choices matter, even when they produce the same fitted means.

Definition: For observation $i$, let $Y_i$ be the response and $x_i$ the covariate vector. A log-linked GLM has conditional mean $\mu_i=E[Y_i\mid x_i]$ and linear predictor $\eta_i=x_i^T\beta$, with $$ \log(\mu_i)=\eta_i \qquad\text{and hence}\qquad \mu_i=\exp(x_i^T\beta). $$ A variance-function specification has $\operatorname{Var}(Y_i\mid x_i)=\phi V(\mu_i)$, where $\phi>0$ is a dispersion parameter and $V$ is the family’s variance function.

For the comparison here, the gamma model has $V(\mu)=\mu^2$, the inverse-Gaussian model has $V(\mu)=\mu^3$, and a Tweedie model with fixed power $1<p<2$ has $V(\mu)=\mu^p$. The dispersion and regression coefficients are model parameters; the Tweedie power $p$ specifies how variance grows with the mean. These formulas concern conditional moments given the covariates.

What the Log Link Makes Common

Suppose $x_{ij}$ is a covariate whose coefficient is $\beta_j$, and compare two covariate vectors that differ by one unit in that covariate while all other entries stay fixed. Their linear predictors differ by $\beta_j$. Exponentiating gives a ratio of conditional means:

\frac{\mu(x_i+e_j)}{\mu(x_i)} = \frac{\exp((x_i+e_j)^T\beta)}{\exp(x_i^T\beta)} = \exp(\beta_j),

where $e_j$ adds one unit to covariate $j$. Thus $\exp(\beta_j)$ is the multiplicative mean ratio, and $100\{\exp(\beta_j)-1\}\%$ is the corresponding percentage change. This describes a change in the modeled conditional mean, not a constant change for every individual response and not necessarily a change in the mean among positive responses alone.

Worked Example: Interpreting a Shared Mean Effect

Suppose a log-linked model for a response has linear predictor $\eta=\beta_0+\beta_1 x$, with $\beta_0=\log(5)$ and $\beta_1=\log(1.2)$. Calculate the modeled means at $x=0$ and $x=1$, and interpret the coefficient.

At $x=0$, $\eta=\log(5)$, so $\mu(0)=\exp(\log(5))=5$. At $x=1$, $\eta=\log(5)+\log(1.2)=\log(6)$, so $\mu(1)=6$. The mean ratio is $6/5=1.2=\exp(\beta_1)$, a 20% increase.

This calculation is identical whether the response family is gamma, inverse-Gaussian, or Tweedie. The family changes the distribution and variance around the mean, not this interpretation of the coefficient. If the model is Tweedie and zeros are possible, the ratio is for the unconditional mean $E[Y\mid x]$, including zero outcomes.

Theorem: In any of these log-linked models, changing covariate $x_j$ by an amount $c$, with other covariates fixed, multiplies the conditional mean by $\exp(c\beta_j)$. This statement concerns the specified mean model and does not depend on which of the three variance functions is used.

Proof. Write the original linear predictor as $\eta=x^T\beta$. After changing $x_j$ by $c$, the new predictor is $\eta+c\beta_j$. The new mean divided by the original mean is therefore $\exp(\eta+c\beta_j)/\exp(\eta)=\exp(c\beta_j)$. $\square$

Different Variance Growth, Different Implications

Although all three models can have the same $\mu_i$, their conditional variances at that mean are different:

\begin{aligned} \text{Gamma:}\quad &\operatorname{Var}(Y_i\mid x_i)=\phi\mu_i^2,\\ \text{Inverse-Gaussian:}\quad &\operatorname{Var}(Y_i\mid x_i)=\phi\mu_i^3,\\ \text{Tweedie:}\quad &\operatorname{Var}(Y_i\mid x_i)=\phi\mu_i^p,\qquad 1<p<2. \end{aligned}

These equations describe how absolute variability changes as the conditional mean changes. A second useful comparison divides the variance by the squared mean. The resulting squared coefficient of variation is:

\frac{\operatorname{Var}(Y_i\mid x_i)}{\mu_i^2} = \begin{cases} \phi, & \text{Gamma},\\ \phi\mu_i, & \text{Inverse-Gaussian},\\ \phi\mu_i^{p-2}, & \text{Tweedie}. \end{cases}

For gamma, relative variability is constant across means when $\phi$ is fixed. For inverse-Gaussian, it increases with the mean. For Tweedie with $1<p<2$, it decreases with the mean. These are implications of the model specifications, not universal empirical laws. A fitted model can also include different dispersion estimates, so comparing variance at a particular mean should use each model’s own fitted $\phi$.

Worked Example: Comparing Variance at the Same Mean

At a covariate setting where $\mu=10$, compare the conditional variances predicted by a gamma model, an inverse-Gaussian model, and a Tweedie model with $p=1.5$. For a direct comparison, use $\phi=0.4$ in all three.

For the gamma model, $\operatorname{Var}(Y\mid x)=0.4(10^2)=40$. For the inverse-Gaussian model, it is $0.4(10^3)=400$. For the Tweedie model, it is $0.4(10^{1.5})=0.4(31.6228)\approx12.65$.

The common mean does not imply common uncertainty: the three variances are 40, 400, and approximately 12.65. Their squared coefficients of variation are respectively $40/100=0.4$, $400/100=4$, and $12.65/100\approx0.1265$. These differences affect the distributions a model regards as plausible around the same expected response.

The moment comparison alone does not fully specify a response distribution. Support and distributional form matter too. Gamma and inverse-Gaussian responses are strictly positive and continuous. A Tweedie response with $1<p<2$ is nonnegative and has a compound Poisson-gamma representation: it has an atom at zero as well as a continuous positive part. Thus this Tweedie model can be appropriate for responses with exact zeros, while gamma and inverse-Gaussian likelihoods do not assign probability to an exact zero.

How the Tweedie Zero Mass Arises

For $1<p<2$, one standard representation of a Tweedie response is a sum of a Poisson number of independent gamma amounts. Let $N$ be Poisson with mean $\lambda$, and, conditional on $N$, let $Y$ be the sum of $N$ independent gamma variables with shape $\alpha$ and scale $\theta$. The parameter choices that give mean $\mu$ and dispersion $\phi$ are:

\lambda=\frac{\mu^{2-p}}{\phi(2-p)},\qquad \alpha=\frac{2-p}{p-1},\qquad \theta=\phi(p-1)\mu^{p-1}.

If $N=0$, the sum is zero; if $N\geq1$, it is positive with probability one. Since $P(N=0)=\exp(-\lambda)$, the exact zero probability is $\exp(-\lambda)$. The derivation also verifies the mean and variance implied by this representation.

Result: Under the compound Poisson-gamma representation above, $E[Y]=\mu$, $\operatorname{Var}(Y)=\phi\mu^p$, and $P(Y=0)=\exp\{-\mu^{2-p}/[\phi(2-p)]\}$.

Derivation. A gamma variable with shape $\alpha$ and scale $\theta$ has mean $\alpha\theta$ and second moment $\alpha(\alpha+1)\theta^2$. For a Poisson sum, the mean is $\lambda\alpha\theta$, and the variance is $\lambda E[Z^2]$, where $Z$ is one gamma amount. Substituting the parameters gives:

\lambda\alpha\theta = \frac{\mu^{2-p}}{\phi(2-p)} \frac{2-p}{p-1} \phi(p-1)\mu^{p-1} =\mu,

and, because $\alpha+1=1/(p-1)$,

\lambda\alpha(\alpha+1)\theta^2 = \frac{\mu^{2-p}}{\phi(2-p)} \frac{2-p}{(p-1)^2} \phi^2(p-1)^2\mu^{2p-2} = \phi\mu^p.

Finally, $Y=0$ exactly when $N=0$, since each gamma amount is strictly positive. The Poisson probability of that event is $\exp(-\lambda)$, which gives the stated zero probability. $\square$

Worked Example: A Tweedie Model with Exact Zeros

Consider a Tweedie model with $\mu=0.2$, $\phi=0.4$, and $p=1.5$. Calculate its Poisson rate, probability of zero, variance, and mean conditional on a positive response.

The rate is $\lambda=0.2^{0.5}/\{0.4(0.5)\}=\sqrt{0.2}/0.2\approx2.236$. Therefore $P(Y=0)=\exp(-2.236)\approx0.107$. The variance is $0.4(0.2^{1.5})\approx0.0358$.

Since $E[Y]=0.2$ and $Y=0$ on the zero event, $E[Y]=P(Y>0)E[Y\mid Y>0]$. Here $P(Y>0)\approx1-0.107=0.893$, so $E[Y\mid Y>0]=0.2/0.893\approx0.224$. This positive-part mean is not the same as the unconditional mean $0.2$ modeled by the log link.

Choosing What the Comparison Tells You

The families make distinct assumptions, so a common link should not be used as a reason to treat them as interchangeable. A gamma model encodes positive continuous outcomes with variance proportional to the square of the mean. An inverse-Gaussian model encodes positive continuous outcomes with variance proportional to its cube. A Tweedie model with $1<p<2$ encodes a nonnegative response with a possible exact zero and variance proportional to an intermediate power of the mean.

These distinctions help explain why two candidate models may produce similar estimated mean trends but different predictive distributions and uncertainty. The selection question is not simply which link makes coefficients easiest to interpret. It is whether the response support and the way variability changes across the mean are credible for the data and scientific setting.

Common pitfall: “The log link is used in both models, so their coefficients and predictions mean the same thing.” The correction is to separate the mean from the full response distribution. If the same coefficient vector is used, the log link gives the same multiplicative mean ratios; the family still determines variance and support. Also, for a Tweedie model with zeros, $\exp(x_i^T\beta)$ is the unconditional mean, not the mean conditional on a positive response.

A practical comparison should therefore check the response support first, then consider whether the observed mean–variance relationship is compatible with the candidate variance function. No one variance pattern guarantees a family is correct, but ignoring these assumptions can yield an unsuitable predictive distribution even when the mean coefficients look interpretable.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

In a log-linked model, what multiplicative change in the conditional mean corresponds to increasing a covariate by two units?
At fixed $\phi$, how does the squared coefficient of variation change with $\mu$ in gamma, inverse-Gaussian, and Tweedie models with $1<p<2$?
Which of the three models described can assign positive probability to an exact zero, and why?
A Tweedie model has a log link and some responses are zero. What mean does $\exp(x_i^T\beta)$ represent?
Why does using the same log link not imply that two candidate models give the same predictive distribution?