Why the Link Changes the Scoring Weights
Fisher scoring for a generalized linear model can be expressed as a weighted least-squares update on a working response. The response distribution determines the variance function, while the link determines how changes in the mean translate into changes in the linear predictor. In an inverse-Gaussian GLM, these ingredients combine to give link-specific weights that can differ greatly at the same fitted mean.
This tutorial derives the working response and weights for independent inverse-Gaussian observations, then compares log, identity, and canonical links in numerical iterations. Throughout, treat the dispersion parameter as fixed and positive; its estimation is a separate question.
Deriving the Working Response and Weight
Assume the current means are in the interior of the parameter space, the link is differentiable with \(g'(\mu_i)\ne 0\), and the design matrix has the needed rank for the update being computed. Write \(\mathbf{x}_i\) for the predictor vector for observation \(i\). The inverse-Gaussian log-likelihood contribution, up to terms independent of \(\mu_i\), is
Differentiating with respect to \(\mu_i\) gives \(\partial\ell_i/\partial\mu_i=(y_i-\mu_i)/(\phi\mu_i^3)\). Since \(d\mu_i/d\eta_i=1/g'(\mu_i)\) and \(d\eta_i/d\boldsymbol{\beta}=\mathbf{x}_i\), the score for the regression coefficients is
The expected information follows from the variance of the score contribution. Conditional on the predictors, \(E[Y_i-\mu_i]=0\) and \(E[(Y_i-\mu_i)^2]=\phi\mu_i^3\). Independence makes cross-observation covariances zero, so the expected information matrix is
A Fisher-scoring step solves \(\mathcal{I}(\boldsymbol{\beta})\Delta\boldsymbol{\beta}=\mathbf{U}(\boldsymbol{\beta})\), then sets \(\boldsymbol{\beta}^{\mathrm{new}}=\boldsymbol{\beta}+\Delta\boldsymbol{\beta}\). Define \(z_i=\eta_i+(y_i-\mu_i)g'(\mu_i)\) and \(w_i=1/\{\phi\mu_i^3[g'(\mu_i)]^2\}\). Then \(w_i(z_i-\eta_i)=(y_i-\mu_i)/\{\phi\mu_i^3g'(\mu_i)\}\). Substituting this identity into the score and information equations gives the weighted least-squares form.
To verify the equivalence, the left-hand matrix is exactly the expected information because \(w_i\) is its observation-specific multiplier. The right-hand side equals the score because its \(i\)th contribution is \(\mathbf{x}_i w_i(z_i-\eta_i)\). The normal equations for weighted least squares have these same left- and right-hand sides when \(\boldsymbol{\beta}^{\mathrm{new}}=\boldsymbol{\beta}+\Delta\boldsymbol{\beta}\). Thus the working-response representation is precisely a Fisher-scoring step, not an unrelated approximation. \(\square\)
Weights Under Three Common Links
The inverse-Gaussian variance function is \(V(\mu)=\mu^3\), so the link-specific weight follows by differentiating the link and substituting into \(w=1/\{\phi V(\mu)[g'(\mu)]^2\}\). For the canonical link, use the natural parameter \(\eta=-1/(2\mu^2)\), whose derivative is \(g'(\mu)=\mu^{-3}\).
| Link | \(g(\mu)\) | \(g'(\mu)\) | Fisher-scoring weight |
|---|---|---|---|
| Log | \(\log\mu\) | \(1/\mu\) | \(1/(\phi\mu)\) |
| Identity | \(\mu\) | \(1\) | \(1/(\phi\mu^3)\) |
| Canonical | \(-1/(2\mu^2)\) | \(\mu^{-3}\) | \(\mu^3/\phi\) |
The canonical-link weight grows with the cube of the mean, whereas the identity-link weight decreases with its cube. The log-link weight decreases in proportion to the reciprocal of the mean. These formulas are the expected-information weights at the current fit; they are not fixed constants attached to the response distribution.
Worked Example: Evaluating the Three Weights
Suppose the current fitted mean is \(\mu=2\) and the dispersion is \(\phi=0.5\). Calculate the three weights.
For the log link, \(w=1/(\phi\mu)=1/(0.5\cdot2)=1\).
For the identity link, \(w=1/(\phi\mu^3)=1/(0.5\cdot 2^3)=1/4=0.25\).
For the canonical link, \(w=\mu^3/\phi=2^3/0.5=16\).
All three calculations use the same \(\mu\) and \(\phi\), but the weights range from \(0.25\) to \(16\). The difference comes entirely from the link derivative. A larger weight means that the observation contributes more to the local weighted least-squares step at this iteration.
Reading a Fisher-Scoring Iteration
In an intercept-only model with one observation, the weighted least-squares update sets the new predictor equal to the working response: \(\eta^{\mathrm{new}}=z\). This simple case isolates the link’s effect on both the step and the corresponding updated mean.
Worked Example: Comparing One-Step Updates Across Links
Take one inverse-Gaussian observation \(y=7/3\), dispersion \(\phi=0.5\), and current mean \(\mu=2\). The starting predictor depends on the link. Calculate each working response, weight, and updated mean.
Log link. The current predictor is \(\eta=\log 2\), and \(g'(2)=1/2\). Therefore \(z=\log 2+(7/3-2)(1/2)=\log 2+1/6\). In this intercept-only one-observation update, \(\eta^{\mathrm{new}}=z\), so \(\mu^{\mathrm{new}}=\exp(\log 2+1/6)=2\exp(1/6)\approx2.3627\). The weight at the current fit is \(1/(\phi\mu)=1\).
Identity link. Here \(\eta=2\) and \(g'(2)=1\), so \(z=2+(7/3-2)=7/3\). The updated mean is \(\mu^{\mathrm{new}}=7/3\approx2.3333\), and the current weight is \(1/(\phi\mu^3)=1/(0.5\cdot8)=0.25\).
Canonical link. The current predictor is \(\eta=-1/(2\cdot2^2)=-1/8\), and \(g'(2)=2^{-3}=1/8\). Thus \(z=-1/8+(7/3-2)(1/8)=-1/8+1/24=-1/12\). Solving \(-1/(2[\mu^{\mathrm{new}}]^2)=-1/12\) gives \([\mu^{\mathrm{new}}]^2=6\), hence \(\mu^{\mathrm{new}}=\sqrt{6}\approx2.4495\). The current weight is \(\mu^3/\phi=8/0.5=16\). The updated canonical predictor remains negative, as required by the canonical parameterization.
The three updates start from the same mean and use the same observation, but they move differently because each link maps a change in mean to a different change in predictor. The weights, meanwhile, describe the local information scale at the starting fit.
Worked Example: Updating an Intercept From Two Working Responses
Consider two observations \(y_1=1\) and \(y_2=4\) in an intercept-only inverse-Gaussian model. Let \(\phi=0.5\), use the log link, and start with the common mean \(\mu_1=\mu_2=2\), so the common predictor is \(\eta=\log 2\).
For the log link, \(g'(2)=1/2\). The working responses are \(z_1=\log 2+(1-2)/2=\log 2-1/2\) and \(z_2=\log 2+(4-2)/2=\log 2+1\). The current weight for each observation is \(1/(\phi\mu)=1\).
With an intercept-only design, the weighted least-squares estimate is the weighted average of the working responses. Hence \(\beta^{\mathrm{new}}=(z_1+z_2)/2=\log 2+1/4\). The updated mean is \(\exp(\beta^{\mathrm{new}})=2\exp(1/4)\approx2.5681\). As a check, the two residuals from the starting mean are \(-1\) and \(2\), whose average is \(1/2\); multiplying by the log-link derivative \(1/2\) gives the predictor increment \(1/4\), as obtained from the working responses.
Conditions, Interpretation, and a Common Pitfall
An iteration uses means and weights evaluated at the current coefficient estimate. The next coefficient estimate changes the predictors, means, working responses, and weights; these must be recomputed at the next iteration. In particular, the weights in the table do not give a single permanent weighting of the observations.
The canonical link has an additional practical constraint: its predictor must lie in the natural-parameter domain \(\eta<0\), corresponding to \(\mu>0\). Even if a current fit is valid, a computed working-response update can leave that domain. The numerical example above stays valid because its updated predictor is \(-1/12\). In an actual fitting procedure, the model’s parameter-space constraints must be respected during iteration.
- Pitfall: using the variance function without the link derivative. One might write \(w=1/(\phi\mu^3)\) for every link because \(\phi\mu^3\) is the response variance. That is only the identity-link weight.
- Correction. The weighted least-squares scale is the variance of the response transformed locally to the predictor scale. Since \(d\eta/d\mu=g'(\mu)\), the weight is \(1/\{\phi\mu^3[g'(\mu)]^2\}\). Substituting each link derivative gives its correct weight.
- Pitfall: treating \(z_i\) as an observed response. The working response depends on the current fitted mean and changes at each iteration. It is a computational device for the local scoring step, not a newly measured outcome.
A common software convention factors the common \(1/\phi\) out of all the weights. When the same dispersion applies to every observation, this common factor cancels from the coefficient solution of the weighted least-squares normal equations. It still matters for the scale of the information matrix, so the convention should be checked before interpreting reported weights.
Check Your Understanding
Use the derivation and examples to answer these questions.
- For a log link, derive the working response from the general formula and state how the weight varies with the current mean.
- At \(\mu=3\) and \(\phi=0.2\), calculate the identity-link and canonical-link Fisher-scoring weights.
- Explain why the canonical-link weight increases with \(\mu\), even though the inverse-Gaussian variance also increases with \(\mu\).
- In an intercept-only model, why is the updated predictor the weighted average of the working responses?
- What parameter-space condition must a canonical-link predictor satisfy, and why should it be checked after an update?