Intercept Columns and Constant Shifts

Why an Intercept Needs Its Own Column

A GLM linear predictor assigns a value \(\eta_i\) to each observation \(i\). The model needs a way to include a baseline component even when no covariate value is contributing to that component. A design-matrix column containing only ones provides exactly that: multiplying every entry by the intercept coefficient gives the same contribution to every observation.

This uniform contribution is useful because observations can have different covariate values and different linear predictors, while still sharing a common baseline. The intercept does not make the predictors identical; it shifts them all together on the linear-predictor scale.

To isolate this role, write \(a_i\) for all contributions to observation \(i\)'s linear predictor other than the intercept. This notation does not specify how those other contributions are constructed; it simply lets us focus on what the intercept does. With the intercept included, the predictor for observation \(i\) is \(a_i+\beta_0\). In an intercept-only model, there are no other contributions, so \(a_i=0\) for every observation.

Worked Examples

What the Intercept Column Guarantees

The column’s entries are identical, so the coefficient multiplies the same number in every row. This simple fact yields a precise shift result. Suppose the non-intercept contributions \(a_i\) are held fixed and the intercept changes from \(\beta_0\) to \(\beta_0+\delta\), where \(\delta\) is any real number for which the model is defined. Denote the predictors before and after the change by \(\eta_i\) and \(\eta_i'\).

Proof. By the definition of \(a_i\), the original predictor is \(\eta_i=a_i+\beta_0\). After the intercept change, it is \(\eta_i'=a_i+(\beta_0+\delta)\). Subtracting gives

This holds for each \(i\), so the shift is common to all observations. For any pair \(i,j\),

Thus every pairwise difference is preserved. \(\square\)

This result separates two effects that are easy to confuse. An intercept change moves the whole collection of linear predictors by the same amount, but it does not change the spacing among them. It also does not necessarily move response means by the same amount. If the inverse link is \(g^{-1}\), then the means before and after the change are \(g^{-1}(\eta_i)\) and \(g^{-1}(\eta_i+\delta)\). Their difference depends on \(\eta_i\) unless the inverse link has a special form over the relevant values.

From a Link-Scale Shift to a Response-Scale Change

The distinction can be stated generally. Let \(\mu_i=g^{-1}(\eta_i)\), where \(g^{-1}\) is the inverse link on the model’s allowed range. After an intercept change of \(\delta\), the new mean is \(g^{-1}(\eta_i+\delta)\). This expression is the exact consequence for the mean; whether the change is additive, multiplicative, or otherwise depends on the link.

Derivation. Under the logistic link, \(\mu_i=1/(1+\exp(-\eta_i))\). Its odds are

After adding \(\delta\) to the intercept, the linear predictor is \(\eta_i+\delta\), so the new odds are \(\exp(\eta_i+\delta)=\exp(\delta)\exp(\eta_i)\). The factor \(\exp(\delta)\) is the same for every observation. This proves the common multiplicative change in odds. It does not imply a common difference in probabilities, as the numerical logistic example showed.

For an intercept-only model, all the other contributions \(a_i\) are zero. The linear predictors are therefore all equal to \(\beta_0\), and applying the same inverse link to each gives equal means. With other contributions present, the intercept still shifts all predictors equally, but the predictors and means need not be equal across observations.

Common Mistake: Putting the Intercept in Only One Row

A frequent design-matrix error is to write \(1\) in the intercept column for the first observation and \(0\) for all remaining observations. That does not represent a shared intercept. It gives the intercept coefficient to the first row alone, because the contribution in row \(i\) is the coefficient multiplied by that row’s column entry.

For example, if the entries were \((1,0,0)\) and \(\beta_0=0.6\), the contributions would be \((0.6,0,0)\), not \((0.6,0.6,0.6)\). The correction is to put \(1\) in every observation’s intercept-column entry. Then every row receives \(\beta_0\), and changing the coefficient shifts every linear predictor by the same amount.