Initializing Intercepts from Response Margins

Why Initialize an Intercept From the Response Margin?

An iterative GLM fit needs starting coefficients. A useful baseline is an intercept-only fit: it begins with a common fitted mean and matches that mean to the observed response margin. This gives the optimizer a starting point on the scale of the data, rather than an arbitrary intercept that might imply implausible probabilities or count means.

The formulas depend on the response distribution and link. For Bernoulli and binomial responses with a logit link, the intercept matches the overall success proportion. For Poisson responses with a log link, it matches the overall count mean. A known offset changes the matching equation: with nonconstant offsets, the intercept generally has to be found by solving an equation rather than by applying a single inverse link to the raw margin.

Bernoulli and Binomial Responses Without Offsets

Let \(Y_i\) be a Bernoulli response, with \(Y_i=1\) for a success and \(Y_i=0\) for a failure. Under a logistic intercept-only model, every observation has success probability \(p=\operatorname{expit}(\alpha)=1/(1+\exp(-\alpha))\). If \(S=\sum_{i=1}^n Y_i\) successes are observed, the log-likelihood, apart from terms not involving \(\alpha\), is \(S\log p+(n-S)\log(1-p)\). Its score is \(S-np\). Thus, when \(0<S<n\), the score is zero at \(p=S/n\), giving the starting value

Here the subscript \(0\) labels the intercept-only starting value. If \(S=0\), the score is negative for every finite \(\alpha\), and the likelihood increases as \(\alpha\) decreases without bound. If \(S=n\), it increases as \(\alpha\) increases without bound. In either case there is no finite intercept-only maximum; the corresponding limiting intercept is \(-\infty\) or \(+\infty\).

For grouped binomial data, let \(Y_i\) be the number of successes among \(m_i\) trials, where \(m_i\) is a nonnegative integer and \(0\leq Y_i\leq m_i\). Write \(S=\sum_iY_i\) and \(M=\sum_i m_i\). With a common probability \(p\), the score for \(\alpha\) is \(S-Mp\). If \(0<S<M\), setting this score to zero gives \(p=S/M\), and hence

The Bernoulli formula is the special case \(m_i=1\) for every observation. The grouped formula uses the pooled success proportion, not the unweighted average of the group-specific proportions. A group with more trials contributes more information to the pooled margin.

Poisson Responses Without Offsets

Suppose \(Y_i\) are independent Poisson responses and the intercept-only model uses the log link, so that \(E[Y_i]=\mu=\exp(\alpha)\). The log-likelihood, apart from constants, is \(\ell(\alpha)=\sum_i\{Y_i\alpha-\exp(\alpha)\}\). Differentiating gives the score \(\sum_iY_i-n\exp(\alpha)\). If the total count is positive, setting the score to zero gives

The intercept is the log of the sample mean count, not the sample mean count itself. If every observed count is zero, the score is \(-n\exp(\alpha)<0\) for every finite \(\alpha\); the likelihood increases as \(\alpha\) tends to \(-\infty\), so there is no finite intercept-only maximum.

Adding a Known Offset

An offset is a known term added to the linear predictor with coefficient fixed at one. With an offset \(o_i\), the intercept-only predictor is \(\eta_i=\alpha+o_i\). For a logistic binomial model, the fitted success probability in group \(i\) is \(p_i(\alpha)=\operatorname{expit}(\alpha+o_i)\). The intercept score is therefore

The starting value solves \(h(\alpha)=S\). When the offsets differ, there is generally no formula that turns the pooled proportion \(S/M\) directly into \(\alpha\), because each group has a different probability even at the same intercept.

Proof. Each term \(m_i\operatorname{expit}(\alpha+o_i)\) is nondecreasing in \(\alpha\), and it is strictly increasing whenever \(m_i>0\). Since \(M>0\), at least one \(m_i\) is positive, so their sum \(h\) is strictly increasing. It is continuous because it is a finite sum of continuous functions. Since the offsets are finite, \(\operatorname{expit}(\alpha+o_i)\) tends to 0 as \(\alpha\) tends to \(-\infty\), and to 1 as \(\alpha\) tends to \(+\infty\). It follows that \(h(\alpha)\) tends to 0 and \(M\), respectively. For \(0<S<M\), the intermediate value theorem gives a finite \(\alpha\) with \(h(\alpha)=S\); strict increase makes that solution unique. \(\square\)

For a Poisson log-link model with offset, \(\mu_i(\alpha)=\exp(\alpha+o_i)\). The score equation is

If \(\sum_iY_i>0\), every \(\exp(o_i)\) is positive and finite, so the unique solution is

The denominator is the sum of the known exposure multipliers on the log scale. This expression reduces to \(\log(\sum_iY_i/n)\) when every offset is zero. If the total count is zero, no finite Poisson intercept solves the score equation; the likelihood approaches its supremum as \(\alpha\) tends to \(-\infty\).

Using the Intercept as a Starting Value

In a model with covariates, these calculations provide a baseline rather than a claim that the full model has been fitted. Setting slope coefficients to zero leaves the intercept-only predictor, and the margin-based intercept gives the corresponding starting means. With no offset this baseline mean is constant; with offsets it varies according to the known offset while the intercept adjusts the overall level.

For the offset logistic binomial case, the monotone equation can be solved numerically. A bracket can be expanded until its lower endpoint gives a fitted total below \(S\) and its upper endpoint gives one above \(S\); bisection then converges because the function is continuous and strictly increasing. In practice, a safeguarded root solver can use the same monotonicity. The relevant target is the total fitted successes, not an unweighted average of group probabilities.

Common Mistakes

• Using the wrong grouped-binomial margin. Averaging \(Y_i/m_i\) without weights treats a group with one trial like a group with many trials. Use \(\sum_iY_i/\sum_i m_i\).
• Applying the no-offset formula when offsets vary. The pooled success proportion does not generally determine an offset logistic intercept by a simple logit. Solve \(\sum_i m_i\operatorname{expit}(\alpha+o_i)=S\).
• Using the Poisson mean as the intercept. With a log link, the intercept is the logarithm of the fitted mean, not the mean itself.
• Assuming every group has positive trial count. A zero-trial group contributes zero to both the total trial count and the score. For uniqueness in the interior-margin result, it is enough that \(M>0\), which ensures at least one positive \(m_i\).
• Ignoring boundary totals. All failures, all successes, or an all-zero Poisson sample do not yield a finite intercept-only maximum. Do not substitute a boundary proportion into a logarithm and treat the result as finite.