Worked Log-Link Inverse-Gaussian Regression

A Small Regression, Worked From Start to Finish

Inverse-Gaussian regression is useful when a response is positive and its variability increases strongly with its mean. The calculations in a fitted model connect several quantities that should not be confused: the linear predictor, the fitted response mean, the mean ratio associated with a predictor, the likelihood of the observed responses, and the uncertainty in estimated coefficients.

We will use a binary treatment indicator to make each calculation visible. The model has a log link, so the fitted mean is positive and a treatment coefficient describes a multiplicative change in that mean. We will also calculate likelihood contributions and derive the Fisher-information formula used for dispersion-adjusted uncertainty.

Model: Let $Y_i$ be independent inverse-Gaussian responses with mean $\mu_i>0$ and common dispersion $\phi>0$, so that $\operatorname{Var}(Y_i)=\phi\mu_i^3$. Under the log link, $\log(\mu_i)=\eta_i=x_i^\mathsf{T}\beta$, and therefore $\mu_i=\exp(x_i^\mathsf{T}\beta)$. The density is $$ f(y_i;\mu_i,\phi)=\frac{1}{\sqrt{2\pi\phi y_i^3}}\exp\left\{-\frac{(y_i-\mu_i)^2}{2\phi\mu_i^2y_i}\right\},\qquad y_i>0. $$

For a binary predictor $x_i$, take $x_i=0$ for control and $x_i=1$ for treatment, with $x_i^\mathsf{T}\beta=\beta_0+\beta_1x_i$. Then the control mean is $\exp(\beta_0)$, and the treatment mean is $\exp(\beta_0+\beta_1)$. Their ratio is $\exp(\beta_1)$. These interpretations concern the conditional response means, not individual observed outcomes.

Worked Example: Fitting a Two-Group Log-Link Model

Suppose the control responses are $1,2,3$, and the treatment responses are $2,3,4$. The model has one mean for each group: $\mu_0=\exp(\beta_0)$ for control and $\mu_1=\exp(\beta_0+\beta_1)$ for treatment.

The fitted control mean is the arithmetic mean of the three control responses: $$ \hat{\mu}_0=\frac{1+2+3}{3}=2. $$ Similarly, the fitted treatment mean is $$ \hat{\mu}_1=\frac{2+3+4}{3}=3. $$

The coefficient estimates follow by solving the two log-link equations: $$ \hat{\beta}_0=\log(\hat{\mu}_0)=\log(2)\approx0.6931, \qquad \hat{\beta}_1=\log(\hat{\mu}_1)-\log(\hat{\mu}_0) =\log(3/2)\approx0.4055. $$

Thus the fitted mean for a control observation is $\exp(0.6931)=2$, while the fitted mean for a treatment observation is $\exp(0.6931+0.4055)=3$, up to rounding. The fitted treatment-to-control mean ratio is $\exp(\hat{\beta}_1)=1.5$: the estimated treatment mean is 50% higher. It does not follow that every treatment response is 50% higher than a corresponding control response.

Why the Two-Group Fit Gives These Means

In this particular model, each group has its own freely estimated mean. The arithmetic-mean result can be checked directly from the likelihood; it is not a general rule that fitted inverse-Gaussian means in every regression equal observed group averages.

Result: For independent inverse-Gaussian observations in a group with common mean $\mu$ and common positive dispersion $\phi$, the maximum-likelihood estimate of $\mu$ is the arithmetic sample mean, provided the observations are positive.

To prove this, retain only the terms in the group's log-likelihood that depend on $\mu$. For observations $y_1,\ldots,y_m>0$, this is $$ \ell(\mu)=C-\frac{1}{2\phi}\sum_{j=1}^{m}\left(\frac{y_j}{\mu^2}-\frac{2}{\mu}+\frac{1}{y_j}\right), $$ where $C$ does not depend on $\mu$. Differentiating for $\mu>0$ gives $$ \frac{d\ell}{d\mu} =\frac{\sum_{j=1}^{m}y_j-m\mu}{\phi\mu^3}. $$ The denominator is positive. Hence the log-likelihood increases when $\mu<\bar{y}$, decreases when $\mu>\bar{y}$, and has its unique maximum at $\hat{\mu}=\bar{y}$. Applying this argument separately to the two groups establishes the fitted means used above. $\square$

Calculating Log-Likelihood Contributions

For a fitted mean $\hat{\mu}_i$ and specified dispersion $\phi$, the individual log-likelihood contribution is the logarithm of the density evaluated at the observed response. Keeping both the normalizing term and the exponential term matters when calculating a likelihood. In particular, the normalizing term depends on the response and dispersion.

Worked Example: Evaluating One Inverse-Gaussian Contribution

Use $\phi=0.2$, and consider an observation $y=3$ with fitted mean $\hat{\mu}=2.5$. Substitution into the log-density gives $$ \ell_i=-\frac12\log(2\pi\phi)-\frac32\log(y) -\frac{(y-\hat{\mu})^2}{2\phi\hat{\mu}^2y}. $$

The normalizing part is $$ -\frac12\log(2\pi(0.2))-\frac32\log(3) =-\frac12\log(1.256637\ldots)-\frac32\log(3) \approx-1.76214. $$ The remaining part is $$ -\frac{(3-2.5)^2}{2(0.2)(2.5)^2(3)} =-\frac{0.25}{7.5} \approx-0.03333. $$ Therefore the contribution is $\ell_i\approx-1.79547$. For a data set of independent observations, the total log-likelihood is the sum of these individual contributions, each evaluated using its own observed response and fitted mean.

The contribution is a log density, not the logarithm of a probability assigned to the exact value $y=3$. For a continuous response, the probability of one exact value is zero; the density describes probability per unit of response near that value. The density formula also requires a positive response.

Fisher Information and Dispersion-Adjusted Uncertainty

Coefficient estimates alone do not quantify how precisely the regression has estimated a mean ratio. A standard large-sample uncertainty calculation uses the Fisher information. For independent observations with fixed covariate vectors $x_i$, the information depends on the fitted means and on the dispersion.

Result: If the observations are independent, have the inverse-Gaussian density above with common known $\phi>0$, and $\mu_i=\exp(x_i^\mathsf{T}\beta)$, the Fisher information for $\beta$ is $$ \mathcal{I}(\beta)=\frac{1}{\phi}\sum_i\frac{x_ix_i^\mathsf{T}}{\mu_i}. $$ The usual model-based large-sample covariance estimate is consequently $$ \widehat{\operatorname{Cov}}(\hat{\beta})\approx \hat{\phi}\left(\sum_i\frac{x_ix_i^\mathsf{T}}{\hat{\mu}_i}\right)^{-1}, $$ when the displayed matrix is invertible. Here $\hat{\phi}$ is the dispersion value used in the uncertainty calculation.

For the derivation, the observation log-density, differentiated with respect to $\mu_i$, has derivative $(y_i-\mu_i)/(\phi\mu_i^3)$. Since $d\mu_i/d\eta_i=\mu_i$ under the log link, the derivative with respect to $\eta_i$ is $$ \frac{\partial\ell_i}{\partial\eta_i} =\frac{y_i-\mu_i}{\phi\mu_i^2}. $$ As $\eta_i=x_i^\mathsf{T}\beta$, the score vector for $\beta$ is $x_i(y_i-\mu_i)/(\phi\mu_i^2)$. Its expectation is zero because $E[Y_i]=\mu_i$. Its expected outer product is $$ \frac{x_ix_i^\mathsf{T}}{\phi^2\mu_i^4}\operatorname{Var}(Y_i) =\frac{x_ix_i^\mathsf{T}}{\phi^2\mu_i^4}(\phi\mu_i^3) =\frac{x_ix_i^\mathsf{T}}{\phi\mu_i}. $$ Independence makes the information contributions add, proving the stated formula. Inverting the information gives the large-sample covariance approximation; replacing unknown quantities by fitted values gives its estimated version. $\square$

Worked Example: Standard Error for the Treatment Mean Ratio

Return to the six responses in the first example and use $\hat{\phi}=0.2$ as the dispersion value. For the two-group model, the design vectors are $(1,0)^\mathsf{T}$ for control and $(1,1)^\mathsf{T}$ for treatment. The fitted means are 2 and 3, with three observations at each mean.

The information matrix evaluated at the fitted means is $$ \widehat{\mathcal{I}} =\frac{1}{0.2} \begin{pmatrix} 3/2+3/3 & 3/3\\ 3/3 & 3/3 \end{pmatrix} =5 \begin{pmatrix} 2.5&1\\ 1&1 \end{pmatrix}. $$ The determinant of the inner matrix is $2.5(1)-1(1)=1.5$, so its inverse is $$ \begin{pmatrix} 2.5&1\\ 1&1 \end{pmatrix}^{-1} =\frac{1}{1.5} \begin{pmatrix} 1&-1\\ -1&2.5 \end{pmatrix}. $$ Therefore the estimated covariance matrix is $$ \widehat{\operatorname{Cov}}(\hat{\beta}) =0.2\left[ \begin{pmatrix} 2.5&1\\ 1&1 \end{pmatrix}^{-1} \right] = \begin{pmatrix} 0.13333&-0.13333\\ -0.13333&0.33333 \end{pmatrix}. $$ The estimated standard error of $\hat{\beta}_1$ is $\sqrt{0.33333}\approx0.5774$. Since $\exp(\beta_1)$ is the treatment-to-control mean ratio, this is uncertainty on the log-ratio scale. The dispersion enters directly: using a different dispersion estimate would multiply the covariance matrix by that value, with the fitted means and design held fixed.

Reading the Results With Care

The fitted ratio of 1.5 describes a contrast between conditional means. It is not a ratio of variances: under this model the variance is $\phi\mu^3$, so both the mean and the common dispersion determine response variability. The fitted mean is also not a prediction that each future response will equal that mean.

The covariance calculation is a model-based large-sample approximation. Its standard errors depend on the specified or estimated dispersion and on the assumed inverse-Gaussian variance relationship. In a small data set, a normal approximation for a coefficient may be rough; the calculation should be reported as an approximation, not an exact finite-sample guarantee.

Common Mistakes

Interpreting a log-link coefficient additively on the response scale. The treatment coefficient $0.4055$ is a change in log mean; exponentiating it gives the mean ratio $1.5$.
Dropping the density's normalizing term. The terms involving $\log(\phi)$ and $\log(y_i)$ belong in a log-likelihood contribution, even though they do not depend on the fitted mean.
Using a variance formula without dispersion. Here $\operatorname{Var}(Y_i)=\phi\mu_i^3$, so the coefficient covariance includes the dispersion factor $\phi$.
Calling the standard error of $\hat{\beta}_1$ the standard error of the ratio. It is on the log-ratio scale. The ratio is $\exp(\hat{\beta}_1)$, and its uncertainty must be transformed if reported on the ratio scale.
Applying the arithmetic-mean fit to every regression. It holds here because each group has its own unrestricted mean; more constrained designs generally require fitting the model rather than taking separate group averages.

Key takeaway: In a two-group inverse-Gaussian regression with a log link, the group means determine the intercept and log mean ratio. The likelihood evaluates each positive response under its fitted mean, while coefficient uncertainty uses Fisher information and must account for dispersion.

Check Your Understanding

Use the model and calculations above to answer the following questions.

If the fitted control and treatment means are 4 and 5, what are the intercept, treatment coefficient, and treatment-to-control mean ratio?
For a response $y_i=2$, fitted mean $\mu_i=2$, and dispersion $\phi=0.1$, which part of the log-likelihood contribution is zero, and which parts remain?
Why does the fitted mean in the two-group example equal the arithmetic mean within each group?
In the Fisher-information formula, what happens to the coefficient covariance approximation if the dispersion is doubled while the fitted means and design remain unchanged?
What does the standard error of a binary-predictor coefficient measure when the model uses a log link?