Starting Values for Positive-Mean GLMs

Why Positive-Mean Models Need Deliberate Starting Values

Gamma and inverse-Gaussian GLMs model responses whose means must be positive. An iterative fitting algorithm therefore needs a starting point that does more than look numerically convenient: its linear predictors must lie in the domain of the chosen link, and the resulting fitted means must be valid. Setting every coefficient to zero is not a universal solution. For an identity link it gives a zero mean, and for reciprocal or canonical links it can make the inverse link undefined.

A practical baseline is the arithmetic mean of the observed positive responses. It gives a positive common mean, and in an intercept-only model it is also the maximum likelihood estimate for both families. To initialize a regression model with an intercept, transform that baseline using the model’s link, put the result in the intercept coefficient, and set the slopes to zero. This produces a constant, valid starting mean across observations.

Response Summary and Link Domain

Suppose \(y_1,\ldots,y_n\) are observed responses from a gamma or inverse-Gaussian GLM, with \(y_i>0\) for every \(i\). Define their arithmetic mean by

\bar y=\frac{1}{n}\sum_{i=1}^{n}y_i.

Since each response is positive, \(\bar y>0\). For a link \(g\), the baseline linear predictor is \(\eta_0=g(\bar y)\), and the corresponding mean is \(g^{-1}(\eta_0)=\bar y\), provided the link is evaluated in its domain. The link determines which values of \(\eta_0\) are allowed.

Link and use	Predictor for baseline mean \(m>0\)	Predictor domain	Inverse link
Log, both families	\(\log m\)	All real numbers	\(\exp(\eta)\)
Identity, both families	\(m\)	\(\eta>0\)	\(\eta\)
Reciprocal, both families	\(1/m\)	\(\eta>0\)	\(1/\eta\)
Gamma canonical link	\(-1/m\)	\(\eta<0\)	\(-1/\eta\)
Inverse-Gaussian canonical link	\(-1/(2m^2)\)	\(\eta<0\)	\(\sqrt{-1/(2\eta)}\)

The reciprocal link \(g(\mu)=1/\mu\) and the gamma canonical link \(g(\mu)=-1/\mu\) differ by a sign. The inverse-Gaussian canonical link is \(g(\mu)=-1/(2\mu^2)\). Keeping these definitions distinct matters: a predictor valid for one convention may be invalid for another.

Definition: A valid starting value is an initial coefficient vector whose linear predictors lie in the domain of the selected link for every observation, so that every inverse-link evaluation gives a finite, positive mean. The link-specific domain must be checked before fitting begins.

Worked Example: Gamma Starting Values Under Several Links

Suppose four positive gamma responses are \(2,3,4,7\), and the design has an intercept. Their sum is \(2+3+4+7=16\), so \(\bar y=16/4=4\). Initialize the intercept to \(g(4)\) and all slopes to zero.

With the log link, \(\beta_0=\log 4\), giving \(\eta_i=\log 4\) and \(\mu_i=\exp(\log 4)=4\) for every observation. With the identity link, \(\beta_0=4\), so each \(\mu_i=4>0\). With the reciprocal link, \(\beta_0=1/4\), so each \(\mu_i=1/(1/4)=4\). With the gamma canonical link, \(\beta_0=-1/4\), so each \(\mu_i=-1/(-1/4)=4\).

Each intercept is inside its link’s domain and produces the same positive baseline mean. The coefficients differ because they are on different link scales.

Worked Example: Inverse-Gaussian Canonical Starting Value

Take inverse-Gaussian responses \(1,2,3,6\). Their sum is \(12\), so \(\bar y=12/4=3\). Under the canonical link, set the intercept to \(\beta_0=-1/(2\bar y^2)=-1/(2\cdot9)=-1/18\), and set the slopes to zero.

The predictor is \(\eta_i=-1/18<0\), which is in the canonical link’s domain. Applying the inverse link gives \(\mu_i=\sqrt{-1/(2(-1/18))}=\sqrt{9}=3\) for each observation. By contrast, a zero intercept would give \(\eta_i=0\), where the inverse link is undefined.

Building a Starting Coefficient Vector

Write the linear predictor as \(\eta_i=x_i^\mathsf{T}\beta\), where \(x_i\) is the design vector for observation \(i\). If the design contains an intercept and no offset, choosing the baseline \(\bar y\) gives the simple initialization

\beta_0^{(0)}=g(\bar y),\qquad \beta_j^{(0)}=0\quad\text{for each slope }j.

Then \(x_i^\mathsf{T}\beta^{(0)}=g(\bar y)\) for every observation, because the intercept column is one and every slope is zero. Thus \(\mu_i^{(0)}=g^{-1}(g(\bar y))=\bar y\). This is a baseline, not a claim that the final fitted means should be equal. The algorithm can subsequently update the slopes to reflect covariate relationships.

When a model includes an offset, the predictor also contains that offset. The intercept must then be chosen with the offset in mind; simply copying \(g(\bar y)\) into the intercept may not give the intended constant predictor. Initialization with offsets requires the offset-aware construction appropriate to that model.

Worked Example: Zero Slopes in a Regression Design

Suppose the responses are \(2,4,6,8\), and the model has an intercept and one covariate with values \(-1,0,1,2\). The response mean is \((2+4+6+8)/4=5\). For the log-link model, choose \(\beta_0^{(0)}=\log 5\) and \(\beta_1^{(0)}=0\). Each predictor is \(\eta_i=\log 5+x_i\cdot0=\log 5\), so each initial mean is \(5\).

For the identity link, choose \((\beta_0^{(0)},\beta_1^{(0)})=(5,0)\), giving means of \(5\). For the reciprocal link, choose \((1/5,0)\), also giving means of \(5\). In all three cases, the nonzero covariate values cause no change to the initial means because their coefficients are zero. The validity check is still link-specific: the predictors are respectively unrestricted, positive, and positive.

Why the Arithmetic Mean Is a Useful Baseline

The response mean is not merely convenient. In an intercept-only model, where every observation has the same unknown mean \(\mu>0\), it maximizes the likelihood for either family, provided the responses are positive and the dispersion \(\phi>0\) is fixed. This result explains why the arithmetic mean is a natural starting summary even when the eventual model includes covariates.

Theorem: Let \(y_1,\ldots,y_n>0\) be independent observations from an intercept-only gamma GLM with variance \(\phi\mu^2\), or an intercept-only inverse-Gaussian GLM with variance \(\phi\mu^3\), where \(\phi>0\) is fixed. In either model, the likelihood as a function of the common mean \(\mu>0\) is maximized at \(\hat\mu=\bar y\).

Proof for the gamma model. Under the mean-dispersion parameterization, the density of one observation is

f(y_i;\mu,\phi)= \frac{y_i^{1/\phi-1}\exp[-y_i/(\phi\mu)]} {\Gamma(1/\phi)(\phi\mu)^{1/\phi}},\qquad y_i>0.

Terms in the log-likelihood that depend on \(\mu\) are \(\ell(\mu)=-\frac{n}{\phi}\log\mu-\frac{1}{\phi\mu}\sum_i y_i+C\), where \(C\) does not depend on \(\mu\). Differentiating gives

\ell'(\mu) =-\frac{n}{\phi\mu}+\frac{\sum_i y_i}{\phi\mu^2} =\frac{n(\bar y-\mu)}{\phi\mu^2}.

The denominator is positive. Hence \(\ell'(\mu)>0\) for \(0<\mu<\bar y\), equals zero at \(\bar y\), and is negative for \(\mu>\bar y\). The log-likelihood increases up to \(\bar y\) and decreases thereafter, so its unique maximum is \(\bar y\).

Proof for the inverse-Gaussian model. Its density is

f(y_i;\mu,\phi)= (2\pi\phi y_i^3)^{-1/2} \exp\!\left[-\frac{(y_i-\mu)^2}{2\phi\mu^2y_i}\right], \qquad y_i>0.

Using \((y_i-\mu)^2/(\mu^2y_i)=y_i/\mu^2-2/\mu+1/y_i\), differentiate the log-likelihood with respect to \(\mu\):

\ell'(\mu) =\sum_{i=1}^n\left(\frac{y_i}{\phi\mu^3}-\frac{1}{\phi\mu^2}\right) =\frac{n(\bar y-\mu)}{\phi\mu^3}.

Again, the denominator is positive, so the derivative is positive below \(\bar y\), zero at \(\bar y\), and negative above it. The log-likelihood therefore has its unique maximum at \(\bar y\). This proves the result for both families. \(\square\)

In a regression model with covariates, \(\bar y\) is generally a starting baseline, not the maximum likelihood estimate for every fitted mean. The theorem concerns a single common mean. It justifies the baseline choice but does not replace fitting the regression model.

Transform the Summary, Not Each Response

For a link-scale intercept, the correct construction is \(g(\bar y)\): first calculate a positive response-scale summary, then transform it. In general, this is not the same as averaging the individually transformed responses. For the log link, the latter operation gives the logarithm of the geometric mean, which can produce a substantially different baseline.

Worked Example: Arithmetic Mean Before the Log Transformation

Consider two positive responses, \(1\) and \(9\). Their arithmetic mean is \(\bar y=(1+9)/2=5\), so the log-link starting intercept is \(\log 5\), and its inverse-link mean is \(\exp(\log 5)=5\).

If instead we average the transformed responses, we get \((\log 1+\log 9)/2=\log 3\). The corresponding inverse-link mean is \(\exp(\log 3)=3\), not \(5\). Thus averaging on the link scale changes the baseline. For the stated response-mean initialization, calculate \(\bar y\) first and then apply the selected link.

Domain Checks and Common Mistakes

The starting vector should be checked by evaluating its predictors and inverse-link means for every observation. With an intercept and zero slopes, the predictor is constant, so the check reduces to confirming that \(g(\bar y)\) is in the selected link’s domain. A finite predictor is not enough if it is outside that domain.

Setting all coefficients to zero without checking the link. The log link gives a valid mean of \(1\), but identity and reciprocal links give an invalid predictor of zero, and canonical links have an undefined inverse at zero. Use the transformed positive baseline instead.
Using the wrong canonical-link sign or scale. The gamma canonical link is \(-1/\mu\), while the inverse-Gaussian canonical link is \(-1/(2\mu^2)\). Check the family and link definition before transforming.
Averaging transformed responses and calling the result the transformed mean. As the example shows, these operations need not agree. Calculate the response-scale summary first.
Ignoring the response support. The standard gamma and inverse-Gaussian densities above require \(y_i>0\). Replacing a zero by an arbitrary small positive number changes the data and is not justified by this initialization rule.
Treating a baseline start as a final model conclusion. Zero slopes make all initial means equal; they do not assert that covariates have no association with the response.

Key takeaway: For positive responses, begin with \(\bar y\), set the intercept to \(g(\bar y)\), and set slopes to zero when the design has an intercept and no offset. Then verify that the predictors satisfy the chosen link’s domain and that the inverse link returns positive means.

Check Your Understanding

Answer each question by checking both the response-scale summary and the link domain.

For gamma responses \(2,4,6\), what are the log-link and reciprocal-link intercept starting values?
For an inverse-Gaussian model with baseline mean \(m=2\), calculate the canonical-link predictor and verify its sign.
Why does setting the intercept and all slopes to zero fail as a general initialization rule for identity and canonical links?
With responses \(1\) and \(9\), compare the log of the arithmetic mean with the mean of the log responses. Which gives the arithmetic-mean baseline?
What does the intercept-only maximum likelihood result establish, and why does it not imply that a regression model’s fitted means must all equal \(\bar y\)?