From Factor Levels to a Linear Predictor
A factor assigns each observation to one of a finite set of levels. In a generalized linear model (GLM), indicator columns let the linear predictor take a different value at each level. Reference coding expresses those level-specific values using one baseline and a set of differences from it. The central interpretive point is that these differences are on the linear-predictor scale, not necessarily on the response scale.
Suppose a factor has \(K\) levels, numbered \(1,\ldots,K\), and choose level 1 as the reference. For observation \(i\), let \(x_{ik}=1\) if it belongs to level \(k\), and \(x_{ik}=0\) otherwise. The reference level is represented by the intercept alone; each other level has an additional indicator coefficient.
A factor-only model has no other predictors: all observations at the same level have the same modeled mean and linear predictor. The word “cell” here means a factor level. We will call \(\alpha_k\) the linear predictor for level \(k\). The response mean for that level is \(g^{-1}(\alpha_k)\), when the inverse link is defined there. With a nonlinear link, \(\alpha_k\) is not itself the response mean.
The Coefficients Are Differences From the Reference
The level-specific predictors can be written as \(\alpha_1,\ldots,\alpha_K\). Comparing the reference-coded expression with the predictor for each level gives the conversion directly: the intercept is the reference-level predictor, and each other coefficient is that level’s predictor minus the reference predictor.
Proof. For an observation in level 1, every indicator \(x_{ik}\), \(k\geq2\), is zero. Substitution into the model gives \(\eta_i=\beta_0\), so \(\alpha_1=\beta_0\). For an observation in level \(k\geq2\), exactly the indicator \(x_{ik}\) equals 1 among the non-reference indicators, so \(\eta_i=\beta_0+\beta_k\), giving \(\alpha_k=\beta_0+\beta_k\). Solving these equations yields \(\beta_0=\alpha_1\) and \(\beta_k=\alpha_k-\alpha_1\). Conversely, given any \(\alpha_1,\ldots,\alpha_K\), setting the coefficients to these values reproduces every \(\alpha_k\). Therefore the parameterizations are equivalent. \(\square\)
This equivalence is algebraic: it concerns the form of the model. If every level is represented in the data, the intercept column and the \(K-1\) indicator columns are linearly independent. To see this, suppose a linear combination of those columns is zero for every observation. Evaluating it at level 1 forces the intercept’s coefficient in that combination to be zero. Evaluating it at each other level then forces the coefficient of that level’s indicator to be zero. If a level has no observations, its predictor cannot be learned from the data without additional assumptions, even though the parameterization can still be written down.
In GLM terminology, “cell means” often refers to the level-specific response means \(\mu_k\). The reference-coded coefficients are not generally differences between those means. Rather, if \(\eta_k=g(\mu_k)\), then \(\beta_0=g(\mu_1)\) and \(\beta_k=g(\mu_k)-g(\mu_1)\). The link determines how to translate a coefficient difference into a response-scale comparison.
Worked Examples
Worked Example: Gaussian Model With an Identity Link
Suppose \(Y\) is modeled as Gaussian with identity link, and a factor has levels A, B, and C, with A as the reference. The observations are A: \(2,4,6\); B: \(5,7\); and C: \(1,4,7\). Find the fitted level means and reference-coded coefficients.
With the identity link, \(\eta_k=\mu_k\). In this factor-only Gaussian model, the fitted mean for each level is its sample mean. The calculations are $$ \bar{Y}_A=\frac{2+4+6}{3}=4,\qquad \bar{Y}_B=\frac{5+7}{2}=6,\qquad \bar{Y}_C=\frac{1+4+7}{3}=4. $$
Thus the level-specific predictors are \(\alpha_A=4\), \(\alpha_B=6\), and \(\alpha_C=4\). A is the reference, so $$ \beta_0=\alpha_A=4,\qquad \beta_B=\alpha_B-\alpha_A=6-4=2,\qquad \beta_C=\alpha_C-\alpha_A=4-4=0. $$
The fitted predictor is \(\eta_i=4+2x_{iB}+0x_{iC}\). It gives 4 for A, \(4+2=6\) for B, and \(4+0=4\) for C. Here the link is the identity, so the coefficients also are differences in response means: B’s fitted mean is 2 higher than A’s, while C’s is equal to A’s.
Worked Example: Binomial Model With a Logit Link
Three groups have binomial outcomes, with A as the reference: A has 8 successes in 20 trials, B has 12 in 20, and C has 18 in 25. Use a logit link and find the reference-coded coefficients. Assume independent trials and a separate modeled success probability for each group.
For a group with \(y\) successes in \(m\) trials, the likelihood as a function of its probability \(p\), apart from a constant not involving \(p\), is \(p^y(1-p)^{m-y}\). For \(0<y<m\), its log-likelihood is maximized at \(\hat p=y/m\). Therefore, $$ \hat p_A=\frac{8}{20}=0.4,\qquad \hat p_B=\frac{12}{20}=0.6,\qquad \hat p_C=\frac{18}{25}=0.72. $$
The logit of a probability \(p\) is \(\log\{p/(1-p)\}\). The fitted level-specific predictors are $$ \hat\alpha_A=\log\!\left(\frac{0.4}{0.6}\right)=\log\!\left(\frac{2}{3}\right)\approx-0.4055, $$ $$ \hat\alpha_B=\log\!\left(\frac{0.6}{0.4}\right)=\log\!\left(\frac{3}{2}\right)\approx0.4055, \qquad \hat\alpha_C=\log\!\left(\frac{0.72}{0.28}\right)=\log\!\left(\frac{18}{7}\right)\approx0.9445. $$
Subtracting the reference predictor gives $$ \hat\beta_0=\hat\alpha_A\approx-0.4055,\qquad \hat\beta_B=\hat\alpha_B-\hat\alpha_A=\log\!\left(\frac{9}{4}\right)\approx0.8109, $$ $$ \hat\beta_C=\hat\alpha_C-\hat\alpha_A=\log\!\left(\frac{27}{7}\right)\approx1.3499. $$
The intercept is the estimated log-odds for A. The B coefficient is B’s estimated log-odds minus A’s; exponentiating it gives an odds ratio of \(9/4=2.25\). For C, the odds ratio relative to A is \(27/7\approx3.857\). Neither coefficient is a difference in success probabilities: for example, B’s probability is \(0.6\), which is \(0.2\) higher than A’s \(0.4\), while its logit-scale difference is about \(0.8109\).
Worked Example: Poisson Model With a Log Link
Suppose independent count observations follow a Poisson model with a log link. For reference level A, the observed counts are \(1,2,3\); for level B, they are \(4,5,6\). Find the fitted means and the reference-coded coefficients.
For a group of independent Poisson observations with common mean \(\mu\), the log-likelihood, apart from terms not involving \(\mu\), is $$ \ell(\mu)=\left(\sum_i y_i\right)\log(\mu)-n\mu. $$ Its derivative is \(\sum_i y_i/\mu-n\), which is zero at the maximum when \(\hat\mu=\sum_i y_i/n\). Thus $$ \hat\mu_A=\frac{1+2+3}{3}=2,\qquad \hat\mu_B=\frac{4+5+6}{3}=5. $$
Under the log link, the level predictors are \(\hat\alpha_A=\log(2)\) and \(\hat\alpha_B=\log(5)\). The reference-coded coefficients are $$ \hat\beta_0=\log(2)\approx0.6931,\qquad \hat\beta_B=\log(5)-\log(2)=\log\!\left(\frac{5}{2}\right)\approx0.9163. $$
The model is \(\eta_i=\log(2)+0.9163x_{iB}\). For B, the predictor is approximately \(0.6931+0.9163=1.6094=\log(5)\), as required. Exponentiating the coefficient gives \(\exp(\hat\beta_B)=2.5\), the ratio of B’s fitted mean count to A’s. The coefficient itself is a difference in log means, not a difference of 3 in the means.
What the Interpretation Depends On
Reference coding always expresses differences on the linear-predictor scale. What those differences imply for the response depends on the link:
| Link | Reference coefficient | Response-scale comparison |
|---|---|---|
| Identity | \(\mu_k-\mu_1\) | Difference in means |
| Logit | Difference in logits | Exponentiated coefficient is an odds ratio |
| Log | \(\log(\mu_k)-\log(\mu_1)\) | Exponentiated coefficient is a mean ratio |
These interpretations follow from applying the link to each level’s mean. For a logit link, \(\eta_k=\log\{\mu_k/(1-\mu_k)\}\), so \(\eta_k-\eta_1\) is the log of the odds ratio. For a log link, \(\eta_k-\eta_1=\log(\mu_k/\mu_1)\), so exponentiating gives the mean ratio. With an identity link, the predictor is the mean itself, and subtracting predictors gives a difference in means.
Common Mistakes and Why Coding Matters
A frequent mistake is to interpret every factor coefficient as a difference in response means. That is correct for the identity link, but not for a logit or log link. Another is to call \(\beta_k\) the predictor for level \(k\): the predictor is \(\beta_0+\beta_k\); the coefficient alone is its difference from the reference predictor. To recover a level’s response mean, first add the intercept and the appropriate coefficient, then apply the inverse link.
Reference coding is useful because it makes comparisons to a chosen baseline explicit while still allowing a distinct modeled mean at every factor level. The same fitted level predictors could be expressed using a different reference, but the numerical coefficients and their comparisons would then be expressed relative to that chosen level. For interpreting a particular coefficient, always identify the reference level and the link function.
Check Your Understanding
Use the reference-level and link-scale interpretations to answer the following questions.
- A three-level factor has level predictors \(-1\), \(0.5\), and \(2\), with the first level as reference. What are the intercept and the two factor coefficients?
- Under an identity link, what response-scale comparison does a non-reference coefficient represent?
- Under a logit link, what does the exponential of a reference-coded coefficient represent?
- A Poisson log-link model has a coefficient of \(\log(3)\) for level B relative to A. What is the fitted mean ratio, and is the coefficient itself a mean difference?
- Why can a factor level with no observations have a predictor that is defined algebraically but not estimable from the data alone?