Inspecting Factor Contrasts With Model Matrix

Why Inspect the Model Matrix?

A factor in an R formula is not passed to a regression algorithm as a set of words. R translates it into numeric columns, and the values in those columns determine the parameterization used by the fitted model. A coefficient can only be interpreted correctly after checking which columns were created and what each column represents.

The function model.matrix() makes that translation visible. It is useful when a coefficient seems inconsistent with the factor labels, when a reference level is unclear, or when a model uses contrasts other than the usual treatment coding. The central task is to inspect the matrix rather than infer its structure from the formula alone.

Definition: A model matrix is the numeric design matrix constructed from a model formula and data. Each row corresponds to an observation; columns represent the intercept, numeric predictors, and encoded factor terms. A factor’s contrasts specify the numeric pattern used to represent its levels. The model matrix, together with a coefficient vector, defines the model’s linear predictor.

For a model with a linear predictor, the relationship is $\eta=X\beta$, where $X$ is the model matrix and $\beta$ is the coefficient vector. In a linear model, $\eta$ is the fitted mean. In a generalized linear model, $\eta$ is the linear predictor on the link scale. Inspecting $X$ tells you what the coefficients multiply; it does not, by itself, establish that the model assumptions are appropriate.

Inspecting a Treatment-Coded Factor

For an unordered factor, standard R installations usually use treatment contrasts by default. Under treatment coding with an intercept, the first factor level is represented by zeros in all factor columns. Each remaining level gets its own indicator column. The precise defaults can be changed by R options or by a factor’s contrast settings, so inspect the object used in the model rather than assume the default.

Worked Example: Finding the Reference Level and Indicator Columns

A hypothetical pricing analysis uses a factor called plan with levels Base, Plus, and Premier. The data contain these four rows, in order: Base, Plus, Premier, Plus. Assume treatment coding with an intercept and Base as the reference. Identify the model-matrix columns and calculate the linear predictor for each row if the fitted coefficients are 120, 18, and −7.

Method. Write one row for each observation. The intercept column is 1 for every row. The Base row has zeros in both factor columns; Plus and Premier each have a 1 in their corresponding column.

X = \begin{bmatrix} 1 & 0 & 0\\ 1 & 1 & 0\\ 1 & 0 & 1\\ 1 & 1 & 0 \end{bmatrix}, \qquad \beta = \begin{bmatrix} 120\\ 18\\ -7 \end{bmatrix}.

The columns are (Intercept), planPlus, and planPremier. The all-zero factor row is Base, so Base is the reference. Multiply each row by the coefficient vector: Base gives $120+0(18)+0(-7)=120$; Plus gives $120+1(18)+0(-7)=138$; Premier gives $120+0(18)+1(-7)=113$; and the final Plus row gives 138.

A compact R check is model.matrix(~ plan, data = dat). The factor level order and contrast settings must be the ones used to form the stated design. Common mistake avoided: treating the first column after the intercept as the reference-level indicator. Under this coding, the reference level has no separate indicator column; it is represented by the intercept and zeros in the factor columns.

For a fitted model, the matrix can be inspected directly, for example with X <- model.matrix(fit). If you are constructing it from a formula, use model.matrix(~ plan, data = dat). The matrix has useful attributes: attr(X, "contrasts") reports the contrasts used for factor terms, and attr(X, "assign") records which formula term generated each column. In a simple formula with an intercept followed by one factor, the intercept column is assigned to term 0 and the factor columns to that factor’s term. These attributes help check how the formula was expanded; the column names and values still need to be read in context.

Reading Sum-Coded Columns

Treatment coding is not the only possible parameterization. With sum coding for a factor with $k$ levels and an intercept, R creates $k-1$ columns. For the usual sum-contrast construction, the first $k-1$ levels have successive rows of the identity matrix, and the last level has −1 in every contrast column. Consequently, the contrast values across levels sum to zero in each column. The reference-level interpretation from treatment coding does not apply: the columns represent deviations subject to a constraint, not indicators against a baseline.

Worked Example: Checking Sum Contrasts Against Group Means

A hypothetical study records a response for two observations in each of three plan groups, ordered Basic, Standard, Premium. The responses are 9 and 11 for Basic, 13 and 15 for Standard, and 18 and 20 for Premium. Use sum contrasts with an intercept to find the fitted coefficients for a model containing only this factor, then verify the group fitted means.

Method. First calculate each group mean: Basic has $(9+11)/2=10$, Standard has $(13+15)/2=14$, and Premium has $(18+20)/2=19$. A model with a separate fitted mean for each factor level reproduces these group means.

X = \begin{bmatrix} 1 & 1 & 0\\ 1 & 1 & 0\\ 1 & 0 & 1\\ 1 & 0 & 1\\ 1 & -1 & -1\\ 1 & -1 & -1 \end{bmatrix}.

Let the coefficient vector be $(\alpha,\beta_1,\beta_2)^\mathsf{T}$. The three fitted group means must satisfy $\alpha+\beta_1=10$, $\alpha+\beta_2=14$, and $\alpha-\beta_1-\beta_2=19$. Adding the three equations gives $3\alpha=43$, so $\alpha=43/3$. Then $\beta_1=10-43/3=-13/3$, and $\beta_2=14-43/3=-1/3$.

Check the fitted means: Basic is $43/3-13/3=30/3=10$; Standard is $43/3-1/3=42/3=14$; Premium is $43/3+13/3+1/3=57/3=19$. The two rows within each group receive that group’s fitted mean. Coefficients are shown exactly as fractions; their decimal forms, if needed, are rounded only at the final step to four decimal places: $\alpha=14.3333$, $\beta_1=-4.3333$, and $\beta_2=-0.3333$.

In R, inspect the contrast matrix with contrasts(plan) and the expanded design with model.matrix(~ plan, data = dat), after assigning contrasts(plan) <- contr.sum(3). Common mistake avoided: calling the last level the reference because it has negative entries. Those entries enforce the sum constraint; they do not make Premium a treatment-coded baseline.

Why Different Contrasts Can Give the Same Fitted Values

A change of contrasts can change the coefficient values without changing the set of fitted values available to the model. This holds when the alternative coding spans the same model space and the design has the required rank. For example, a factor-only model with an intercept and $k-1$ independent contrasts can represent any set of $k$ group means, whichever full-rank contrast scheme is used.

The reason is a change of coordinates. If two design matrices satisfy $X_{\text{new}}=X_{\text{old}}A$ for an invertible matrix $A$, define $\beta_{\text{new}}=A^{-1}\beta_{\text{old}}$. Then:

X_{\text{new}}\beta_{\text{new}} = X_{\text{old}}A A^{-1}\beta_{\text{old}} = X_{\text{old}}\beta_{\text{old}}.

Thus the same linear predictor can be represented with different coefficients. This conclusion requires equivalent model spaces; it does not say that every change to a formula leaves fitted values unchanged. Dropping a factor column, changing which observations are included, or using a rank-deficient design can alter what the model can represent.

Worked Example: Verifying a Different Treatment Reference

Use the same three hypothetical group means from the previous example: Base has fitted mean 10, Plus has fitted mean 14, and Premier has fitted mean 19. This time use treatment coding with Plus as the reference. Identify the columns, calculate the coefficients, and check all three fitted means.

Method. The treatment-coded design has an intercept and indicators for the nonreference levels Base and Premier. Its rows, in Base, Plus, Premier order, are $(1,1,0)$, $(1,0,0)$, and $(1,0,1)$. The intercept is the fitted mean for Plus, so it is 14. The Base coefficient is the Base-minus-Plus difference, $10-14=-4$; the Premier coefficient is the Premier-minus-Plus difference, $19-14=5$.

\beta = \begin{bmatrix} 14\\ -4\\ 5 \end{bmatrix}, \qquad X\beta = \begin{bmatrix} 14-4\\ 14\\ 14+5 \end{bmatrix} = \begin{bmatrix} 10\\ 14\\ 19 \end{bmatrix}.

In R, one way to request this coding is to set contrasts(plan) <- contr.treatment(levels(plan), base = 2) and then inspect model.matrix(~ plan, data = dat). With levels ordered Base, Plus, Premier, the second level is Plus. Common mistake avoided: retaining the old treatment-coded coefficients after changing the reference. The fitted group means remain the same, but the intercept and factor coefficients must be expressed in the new parameterization.

A Reliable Inspection Workflow

When a factor term appears in a model, check its coding in the same data and model context used for fitting. A displayed formula alone does not reveal whether a factor has been relevelled, assigned custom contrasts, or affected by global contrast options. A short inspection sequence prevents those details from being guessed.

Check the factor levels.
Use levels(dat$plan) to see the level names and their order. Do not infer the order from how rows happen to appear in the data.

Check the contrasts.
Use contrasts(dat$plan) for a factor with assigned contrasts. If no factor-specific contrast matrix is set, the session’s defaults can matter.

Build or extract the model matrix.
Use model.matrix() with the fitted model or the exact formula and data. Read the column names together with the numeric rows.

Match columns to coefficients.
Check coefficient names against the model-matrix columns. For an intercept model, identify which level is all zeros under treatment coding, or identify the contrast constraint under sum coding.

One frequent exam trap is to see a factor with levels A, B, and C and assume that the coefficient called factorB is always the B-versus-A difference. That interpretation is justified only if the model matrix confirms treatment coding with A as the reference. Under sum coding, custom contrasts, or a different level order, the column’s entries—and therefore the coefficient’s meaning—can differ. State the coding evidence before interpreting a coefficient.

Key takeaway: Use model.matrix() to verify the actual numeric columns, and use the factor levels and contrast settings to explain them. Column names are clues, not a substitute for inspecting the matrix values.

Check Your Understanding

For each problem, identify the relevant coding and show how the matrix or coefficients follow from it.

A factor has levels A, B, C in that order and uses treatment coding with an intercept, with A as reference. Give the three model-matrix rows for A, C, B, in that observation order, using columns $(\text{Intercept}, \text{factorB}, \text{factorC})$. (Answer: $(1,0,0)$, $(1,0,1)$, $(1,1,0)$.)
A three-level factor uses sum contrasts in level order Low, Mid, High. Write the contrast values for each level using two contrast columns. (Answer: Low $(1,0)$, Mid $(0,1)$, High $(-1,-1)$.)
A treatment-coded factor has reference level Rural. The fitted intercept is 32, the Urban coefficient is 6, and the Remote coefficient is −4. Calculate the fitted linear predictor for each level. (Answer: Rural 32, Urban 38, Remote 28.)
A factor’s three fitted group means are 7, 11, and 18 in level order A, B, C. Under sum contrasts, calculate the intercept and the two coefficients. (Answer: Intercept 12; coefficients $-5$ and $-1$.)
A treatment-coded three-level factor has levels East, Central, West, with Central as the reference. State the factor-column names and the row for West, assuming an intercept. (Answer: columns for East and West; the West row is $(1,0,1)$ in column order $(\text{Intercept}, \text{factorEast}, \text{factorWest})$.)

What You'll Learn