Starting Coefficients for Noncanonical Links

Why Build Starting Coefficients on the Link Scale?

A GLM solver needs an initial coefficient vector, but coefficients do not have a natural default value that works for every link and design matrix. Setting every coefficient to zero, for example, gives the same initial linear predictor for every observation. The resulting mean may be implausible or poorly matched to the response. For a noncanonical link, a useful alternative is to begin with plausible means, transform them to the linear-predictor scale, and find coefficients that reproduce those transformed values as closely as the design permits.

This construction is a way to provide a finite and interpretable starting point. It does not maximize the likelihood, guarantee convergence, or replace checks for boundary fits. The response-based means are provisional targets, not final fitted means.

Constructing a Link-Scale Target

The target means must be inside the domain of the link and must correspond to valid means for the response distribution. For a Bernoulli model with a probit link, for example, \(m_i\) must lie strictly between 0 and 1: the inverse probit of either endpoint is infinite. With an identity link for Bernoulli data, fitted means must also be probabilities, so predictors that produce values outside \([0,1]\) are invalid.

A simple way to avoid using observed zeros and ones as exact Bernoulli means is to shrink each observed response toward the overall response mean. For binary responses \(y_i\in\{0,1\}\), let \(\bar y=n^{-1}\sum_i y_i\) and choose \(0<\lambda<1\). Define

If \(0<\bar y<1\), every \(m_i\) is strictly between 0 and 1. This is one possible regularization of the starting targets, not a uniquely correct choice. More elaborate targets can come from grouped summaries, a simple preliminary trend, or other scientifically plausible means.

Thus smoothing the individual targets in this way makes them finite on the link scale without changing their average. It does not imply that the eventual fitted GLM will have exactly this average; the design and link determine the fitted means.

Solving for the Coefficients

Stack the row vectors \(x_i^\mathsf{T}\) into the \(n\times p\) design matrix \(X\), and let \(t\) be the vector with entries \(t_i=g(m_i)\). If there are at least as many independent design directions as coefficients and \(X\) has full column rank, the least-squares starting vector is

This is the exact solution when the target vector \(t\) is in the column space of \(X\); otherwise it gives the closest fitted linear predictors in squared Euclidean distance. With chosen positive weights \(w_i\), one can instead minimize \(\sum_i w_i(t_i-x_i^\mathsf{T}\beta)^2\). These weights describe how closely to match the provisional targets; they are not automatically the GLM's likelihood or working weights.

Proof. Since \(X\) has full column rank and \(W\) has positive diagonal entries, for every nonzero \(a\in\mathbb{R}^p\),

Here \(Xa\ne 0\) by full column rank, and a positive diagonal \(W\) makes the weighted squared length of any nonzero vector positive. Therefore \(X^\mathsf{T}WX\) is invertible. Differentiating the quadratic objective gives

Setting this gradient to zero gives the normal equations \(X^\mathsf{T}WX\beta=X^\mathsf{T}Wt\), whose unique solution is the stated \(\hat\beta\). The positive definite quadratic has a unique minimum there. Substituting the solution into the normal equations gives \(X^\mathsf{T}W(t-X\hat\beta)=0\), the weighted residual orthogonality condition. \(\square\)

If \(X\) is rank deficient, the displayed inverse does not exist and the coefficients are not uniquely determined by this least-squares problem. A generalized inverse or a reduced parameterization can produce a solution, but the rank deficiency should be diagnosed rather than hidden. In an intercept-only model, the construction reduces to choosing a single transformed target; initializing intercepts from response margins is the special case where the target mean is the response mean.

Worked Examples

A Practical Construction and Its Limits

A disciplined workflow separates the choice of targets from the coefficient calculation. The calculation is often simple; the substantive judgment is whether the chosen targets are plausible and valid for the response distribution.

A common mistake is to transform the raw binary outcomes directly and regress \(g(y_i)\) on the predictors. For a probit link, this attempts to evaluate \(\Phi^{-1}(0)\) and \(\Phi^{-1}(1)\), which are infinite, so it does not produce finite coefficient targets. The correction is to construct interior target means first—for instance, by shrinkage toward the response mean—and only then apply the link.

Another mistake is to interpret the least-squares solution as a GLM estimate. It minimizes discrepancies between chosen transformed targets and linear predictors, not the GLM negative log-likelihood. It is useful because it supplies a reasoned initial vector; the likelihood-based fitting procedure must still determine the final coefficients. Finally, a good starting vector cannot cure separation, rank deficiency, or a likelihood with no finite maximizer. It can help the numerical routine begin in a plausible region, but it is not a substitute for diagnosing the model and the fit.