Inverse-Gaussian Regression Likelihood

From an Inverse-Gaussian Model to a Regression Likelihood

In inverse-Gaussian regression, the mean is allowed to vary across observations, while the response distribution also depends on dispersion. A likelihood calculation must account for both: the fitted mean for each observation determines how its response contributes, and the dispersion controls the scale of those contributions. Keeping these parts visible makes the likelihood easier to calculate and helps prevent comparisons that accidentally discard a term that depends on a parameter.

Let \(Y_1,\ldots,Y_n\) be conditionally independent responses given their covariates. Write \(y_i>0\) for the observed response and \(\mu_i>0\) for its modelled mean. Use a common dispersion parameter \(\phi>0\). The inverse-Gaussian density in this mean-dispersion parameterization is

f(y_i;\mu_i,\phi) = \frac{1}{\sqrt{2\pi\phi y_i^3}} \exp\left\{-\frac{(y_i-\mu_i)^2}{2\phi\mu_i^2 y_i}\right\}, \qquad y_i>0.

In a regression model, \(\mu_i\) is determined by the covariates and regression coefficients. For a specified link \(g\), this is expressed as \(g(\mu_i)=\eta_i=\mathbf{x}_i^{\mathsf T}\boldsymbol{\beta}\), with parameter values restricted so that every \(\mu_i\) is positive. The likelihood below is conditional on the observed covariates; its arguments are the regression coefficients \(\boldsymbol{\beta}\) and the dispersion \(\phi\).

Definition: For independent inverse-Gaussian responses with positive means \(\mu_i\) and common dispersion \(\phi>0\), the likelihood is the product of the observation densities evaluated at the observed responses. The log-likelihood is its natural logarithm. The mean-model contribution depends on the fitted means, while the dispersion contribution depends on \(\phi\); these contributions are linked through a term divided by \(\phi\).

Deriving the Log-Likelihood

Conditional independence means that the joint density is the product of the individual densities. Taking its logarithm converts the product into a sum. The resulting expression is the central calculation for this tutorial.

Result: For fixed observations \(y_1,\ldots,y_n>0\), the independent-observation inverse-Gaussian regression log-likelihood is

\begin{aligned} L(\boldsymbol{\beta},\phi) &=\prod_{i=1}^{n} f(y_i;\mu_i,\phi),\\ \ell(\boldsymbol{\beta},\phi) &=-\frac{n}{2}\log(2\pi\phi) -\frac{3}{2}\sum_{i=1}^{n}\log y_i -\frac{1}{2\phi} \sum_{i=1}^{n}\frac{(y_i-\mu_i)^2}{\mu_i^2 y_i}. \end{aligned}

To verify the expression, independence gives the product in the first line. For each \(i\), taking the logarithm of the density gives

\log f(y_i;\mu_i,\phi) =-\frac{1}{2}\log(2\pi\phi) -\frac{3}{2}\log y_i -\frac{(y_i-\mu_i)^2}{2\phi\mu_i^2 y_i}.

Summing this identity over \(i=1,\ldots,n\) gives the displayed log-likelihood: the first term occurs \(n\) times, and the other terms sum over observations. This proves the formula. \(\square\)

It is useful to name the nonnegative mean-model quantity

Q(\boldsymbol{\mu};\mathbf{y}) = \sum_{i=1}^{n}\frac{(y_i-\mu_i)^2}{\mu_i^2 y_i}, \qquad \boldsymbol{\mu}=(\mu_1,\ldots,\mu_n).

Then the log-likelihood is \(-\frac{n}{2}\log(2\pi\phi)-\frac{3}{2}\sum_i\log y_i-Q(\boldsymbol{\mu};\mathbf{y})/(2\phi)\). The response-only term \(-\frac{3}{2}\sum_i\log y_i\) is fixed when the observed data are fixed. The term \(-\frac{n}{2}\log(2\pi\phi)\), in contrast, depends on dispersion and cannot be treated as a constant if \(\phi\) is being compared.

An equivalent expression for each summand in \(Q\) is

\frac{(y_i-\mu_i)^2}{\mu_i^2 y_i} =\frac{1}{y_i}-\frac{2}{\mu_i}+\frac{y_i}{\mu_i^2}.

This follows by expanding \((y_i-\mu_i)^2=y_i^2-2y_i\mu_i+\mu_i^2\) and dividing each term by \(\mu_i^2y_i\). The squared-difference form is often clearer for hand calculations; the expanded form can make the dependence on \(\mu_i\) more apparent. Either way, each contribution is nonnegative because its squared-difference numerator is nonnegative and its denominator is positive.

Worked Example: Calculating a Log-Likelihood from Fitted Means

Suppose the observed responses are \(y=(1,2,4)\), the regression model gives fitted means \(\mu=(1,2,2)\), and the common dispersion is \(\phi=0.5\). Calculate the log-likelihood. The fitted means are treated as specified values from a model; no claim is made here that they can vary independently in a particular regression design.

First calculate the three terms in \(Q\):

\begin{aligned} q_1&=\frac{(1-1)^2}{1^2(1)}=0,\\ q_2&=\frac{(2-2)^2}{2^2(2)}=0,\\ q_3&=\frac{(4-2)^2}{2^2(4)}=\frac{4}{16}=0.25. \end{aligned}

Thus \(Q=0+0+0.25=0.25\). The data-only logarithm sum is \(\sum_i\log y_i=\log 1+\log 2+\log 4=\log 8\). Substituting \(n=3\) and \(\phi=0.5\) into the log-likelihood gives

\begin{aligned} \ell &=-\frac{3}{2}\log(2\pi\cdot 0.5) -\frac{3}{2}\log 8 -\frac{0.25}{2(0.5)}\\ &=-\frac{3}{2}\log\pi-\frac{3}{2}\log 8-0.25\\ &\approx -1.717095-3.119162-0.25\\ &\approx -5.086257. \end{aligned}

The first two log terms are determined by the sample size, dispersion, and observed responses; the final term reflects the discrepancy between the responses and their fitted means, scaled by dispersion.

What Can Be Omitted in a Likelihood Comparison?

When comparing mean vectors for the same observed data at the same fixed dispersion, the first two terms in the log-likelihood do not change with the candidate means. In that restricted comparison, the ordering of the log-likelihoods is determined entirely by \(Q\).

Result: Fix the observed responses and \(\phi>0\). For any two candidate mean vectors \(\boldsymbol{\mu}^{(A)}\) and \(\boldsymbol{\mu}^{(B)}\) with positive components,

\ell(\boldsymbol{\mu}^{(A)},\phi)-\ell(\boldsymbol{\mu}^{(B)},\phi) = \frac{ Q(\boldsymbol{\mu}^{(B)};\mathbf{y})- Q(\boldsymbol{\mu}^{(A)};\mathbf{y}) }{2\phi}.

To prove the identity, write the log-likelihood for each candidate using the formula already derived. Both log-likelihoods contain the same term \(-\frac{n}{2}\log(2\pi\phi)\) and the same term \(-\frac{3}{2}\sum_i\log y_i\); these cancel when the first is subtracted from the second. The remaining difference is

-\frac{Q(\boldsymbol{\mu}^{(A)};\mathbf{y})}{2\phi} +\frac{Q(\boldsymbol{\mu}^{(B)};\mathbf{y})}{2\phi},

which is the stated expression. Since \(2\phi>0\), if candidate A has a smaller \(Q\) than candidate B, its log-likelihood is larger. This conclusion holds for this fixed-dispersion comparison; it does not license dropping the dispersion-dependent term when comparing different values of \(\phi\). \(\square\)

Worked Example: Comparing Two Candidate Mean Vectors

Use the same responses \(y=(1,2,4)\) and dispersion \(\phi=0.5\). Compare the fitted means \(\boldsymbol{\mu}^{(A)}=(1,2,2)\) with \(\boldsymbol{\mu}^{(B)}=(1,2.5,4)\).

For candidate A, the calculation above gave \(Q_A=0.25\) and \(\ell_A\approx-5.086257\). For candidate B, the first and third responses equal their fitted means, so their contributions are zero. The second contribution is

\frac{(2-2.5)^2}{(2.5)^2(2)} =\frac{0.25}{12.5} =0.02.

Therefore \(Q_B=0.02\). The log-likelihood difference is

\ell_B-\ell_A =\frac{Q_A-Q_B}{2\phi} =\frac{0.25-0.02}{1} =0.23.

Thus \(\ell_B=\ell_A+0.23\approx-4.856257\), which is larger than \(\ell_A\). This verifies the comparison directly from the formula and from the fact that \(Q_B<Q_A\).

Dispersion Is Part of the Likelihood

The dispersion enters in two places: through \(-\frac{n}{2}\log(2\pi\phi)\) and through \(-Q/(2\phi)\). Consequently, a likelihood comparison that changes \(\phi\) must retain both terms. Holding fitted means and responses fixed makes \(Q\) fixed, but it does not make the entire likelihood contribution involving dispersion constant.

Worked Example: Evaluating Two Dispersion Values

Return to \(y=(1,2,4)\) and \(\mu=(1,2,2)\), for which \(Q=0.25\). Evaluate the log-likelihood at \(\phi=1\), and compare it with the value at \(\phi=0.5\).

At \(\phi=1\), the three terms are

\begin{aligned} \ell(\phi=1) &=-\frac{3}{2}\log(2\pi) -\frac{3}{2}\log 8 -\frac{0.25}{2}\\ &\approx -2.756816-3.119162-0.125\\ &\approx -6.000978. \end{aligned}

At \(\phi=0.5\), the earlier calculation gave approximately \(-5.086257\). The values differ because both the logarithmic dispersion term and the \(Q/(2\phi)\) term change. In particular, using only the discrepancy term would not give the correct log-likelihood comparison.

Observation-Level Contributions and a Common Pitfall

For bookkeeping, define the contribution from observation \(i\) as

\ell_i =-\frac{1}{2}\log(2\pi\phi) -\frac{3}{2}\log y_i -\frac{(y_i-\mu_i)^2}{2\phi\mu_i^2y_i}.

Then \(\ell=\sum_i\ell_i\). This decomposition is useful for checking arithmetic: calculate each observation's contribution, add them, and verify that the result agrees with the grouped log-likelihood formula. It also makes clear that independence is what permits the joint likelihood to factor into observation-level densities. If the observations are not independent, this product calculation does not describe their joint likelihood.

Common mistake: A learner may say, “The response-only terms are constants, so I can remove every term that looks constant and compare only \(Q\), even if dispersion changes.” The correction is to specify which parameters are changing. The term \(-\frac{3}{2}\sum_i\log y_i\) is fixed for a given dataset, but \(-\frac{n}{2}\log(2\pi\phi)\) changes with \(\phi\). Only in a comparison of mean vectors at the same fixed dispersion can all terms other than \(-Q/(2\phi)\) be canceled.

A related bookkeeping error is to omit the factor \(y_i\) from the denominator of the squared-difference term. The correct quantity is \((y_i-\mu_i)^2/(\mu_i^2y_i)\), as follows directly from the inverse-Gaussian density. Keeping the density visible until the likelihood has been written is a reliable way to avoid losing either this factor or the power \(y_i^3\) in the density's normalizing term.

Key takeaway: For independent inverse-Gaussian responses, the log-likelihood is the sum of the density contributions. Its fitted-mean term is summarized by \(Q\), while dispersion appears both in a logarithmic term and in the scaling of \(Q\). Terms may be canceled only when they are constant across the parameter values being compared.

Check Your Understanding

Use the likelihood formula and state which quantities are held fixed in each comparison.

For one observation with response \(y\), fitted mean \(\mu\), and dispersion \(\phi\), write its log-likelihood contribution.
For fixed responses and fixed dispersion, explain why a candidate mean vector with smaller \(Q\) has a larger log-likelihood.
For \(y=3\) and \(\mu=2\), calculate the observation's contribution to \(Q\).
Which log-likelihood terms change when the dispersion changes but the responses and fitted means do not?
Why does the product of the individual densities give the joint likelihood in this regression setup?