Why Matrix Construction Matters
Penalized regression software works with numeric predictor columns, not with the mixed data frame an analyst may use to store ages, regions, product types, and other fields. In R, model.matrix() converts a formula and data frame into a numeric design matrix. Its choices determine how categorical predictors become columns, whether a constant column is included, and what the resulting column names are.
These are not cosmetic details. A training matrix and a prediction matrix must represent the same predictors in the same columns. If a factor is encoded differently, or columns are supplied in a different order, predictions can be wrong or the software can reject the input. The matrix is part of the fitted model’s specification and must be treated as such.
For a typical Gaussian, binomial, or Poisson fit with glmnet, the predictor input is a numeric matrix \(x\), and the response is supplied separately as \(y\). glmnet includes an intercept by default for these model types. Therefore, a common workflow is to create a formula matrix with an intercept, then remove its \((\text{Intercept})\) column before passing the predictors to glmnet. Do not assume this convention applies to every model type or fitting option: for example, the Cox model does not use an intercept in the same way.
How Factors Become Numeric Columns
A factor is categorical, so its labels cannot be entered directly as numeric predictor values. Under treatment coding, one level is selected as the reference level. The matrix contains one indicator column for each other level. An observation at the reference level has zero in all those indicator columns; an observation in another level has a one in the corresponding column.
For example, if the levels of region are Coastal, Inland, and Mountain, with Coastal as reference, model.matrix() creates columns named regionInland and regionMountain. The exact reference and contrast settings should be explicit or saved from training rather than left to an analyst’s changing R options.
This encoding describes differences from a baseline; it does not make the category labels quantitative. A coefficient on regionInland, for instance, is an adjustment relative to Coastal under the specified model. For a penalized model, the factor indicators are predictor columns subject to the chosen penalty, while glmnet’s default intercept is not penalized. The precise penalty behavior is separate from the act of constructing the matrix.
Worked Example: Encoding a Factor and Removing the Matrix Intercept
A hypothetical insurer models a numeric claim-cost outcome using policyholder age and region. Three training records have ages 40, 55, and 35, and regions Inland, Coastal, and Mountain, respectively. Use Coastal as the factor reference. Construct the predictor matrix that would be supplied to a glmnet fit with its default intercept.
Method. Make the factor levels explicit, build the formula matrix, and then remove its intercept column. This keeps factor encoding reproducible and avoids supplying both a matrix constant and glmnet’s default intercept.
The R setup is: tr$region <- factor(tr$region, levels = c("Coastal", "Inland", "Mountain")), followed by form <- ~ age + region and X0 <- model.matrix(form, data = tr). With the stated levels, Coastal is the treatment reference.
The numeric age column is unchanged. The two region columns are:
Thus the full matrix, including the formula intercept, is:
Remove the first column with X <- X0[, colnames(X0) != "(Intercept)", drop = FALSE]. The result has three columns: age, regionInland, and regionMountain. Row 2 is Coastal, so both region indicators are zero. A check is that the matrix has three rows, one for each record, and contains only numeric predictor columns.
Common mistake avoided: Passing all of X0 to a default glmnet fit adds a constant predictor column as well as glmnet’s own intercept. Remove the matrix constant when using the default intercept; if the fitting configuration deliberately disables the intercept, decide explicitly whether a matrix constant is required.
Intercepts, Contrasts, and Column Names
A formula such as ~ age + region includes a model-matrix intercept. A formula such as ~ age + region - 1 removes it. With a factor, removing the intercept typically changes the factor representation to one column per level. These are alternative parameterizations, not interchangeable columns to mix casually. For the usual glmnet workflow with its own intercept, treatment coding with the matrix intercept removed is a transparent default.
The column names encode how each predictor was constructed. They are also the keys used to align new data. Interactions and transformed terms can create additional names; their construction should be fixed by the same formula and contrast settings used for training. Never rely on column position alone when checking a matrix.
A short justification explains why treatment coding works. Suppose a linear predictor has an intercept \(\beta_0\), and a factor has reference level \(A\) and other levels \(B\) and \(C\). Treatment indicators are \(z_B=1\) for level \(B\) and zero otherwise, and \(z_C=1\) for level \(C\) and zero otherwise. The factor portion of the predictor is:
For level \(A\), both indicators equal zero, so \(\eta=\beta_0\). For level \(B\), \(z_B=1\) and \(z_C=0\), so \(\eta=\beta_0+\beta_B\). For level \(C\), \(\eta=\beta_0+\beta_C\). Thus the coefficient for each indicator is the model’s contrast against the reference on the linear-predictor scale. This also explains why an intercept and all \(K\) factor indicators are redundant in an unpenalized model: their columns sum to the constant column. Using \(K-1\) indicators with an intercept avoids that exact linear dependence.
Worked Example: Reading the Encoded Columns as Model Terms
A hypothetical pricing model uses age and a three-level plan factor with levels Basic, Plus, and Premium. Basic is the reference. A fitted linear predictor has intercept 2, age coefficient 0.1, Plus coefficient 0.7, and Premium coefficient -0.4. Find the model-matrix row and linear predictor for a 30-year-old with a Premium plan.
Method. Under treatment coding, the columns are (Intercept), age, planPlus, and planPremium. For Premium, planPlus is zero and planPremium is one. The row is therefore \((1,30,0,1)\).
Multiply the row by the coefficient vector in the same order:
The factor contribution is the Premium adjustment relative to Basic, not a numerical score assigned to the word Premium. The check is that a Basic policy at the same age would have row \((1,30,0,0)\) and predictor \(2+3=5\); the Premium predictor is lower by \(0.4\), as its coefficient specifies.
Common mistake avoided: Treating Basic, Plus, and Premium as values 1, 2, and 3 would impose a single linear trend across category labels. Treatment coding instead estimates the stated level-specific contrasts.
Constructing Aligned Prediction Matrices
Prediction data must use the training representation. In particular, a factor in new data should be interpreted using the training factor levels and contrast matrix. If a prediction set simply drops a level that happened not to occur in that set, it must not cause the corresponding training column to disappear. Conversely, a genuinely unseen category is not automatically a known level: it must be handled deliberately, such as by applying a pre-defined category rule or stopping for review.
A reliable pattern is to save the training model frame and full model matrix, then use their factor levels and contrast settings when building the new matrix. The following R expressions illustrate the sequence for a formula containing age and region:
form <- ~ age + region; mf_train <- model.frame(form, data = tr, na.action = na.fail); X_train_full <- model.matrix(form, data = mf_train). Save factor levels with xlev <- lapply(mf_train[vapply(mf_train, is.factor, logical(1))], levels). For new data, use tt <- delete.response(terms(form)) and mf_new <- model.frame(tt, data = newdata, na.action = na.fail, xlev = xlev), then build X_new_full <- model.matrix(tt, data = mf_new, contrasts.arg = attr(X_train_full, "contrasts")).
Remove the intercept from both matrices using the same rule, then verify the column sets. A direct alignment check is setequal(colnames(X_train), colnames(X_new)). If it is TRUE, reorder prediction columns with X_new <- X_new[, colnames(X_train), drop = FALSE]. If it is FALSE, stop and investigate rather than filling or discarding columns without justification. The drop = FALSE argument preserves a matrix when only one predictor column remains.
The row order must also remain associated with the corresponding observations. Matrix construction does not by itself guarantee that a separately stored response or identifier has been reordered in the same way. Retain row identifiers or verify row counts and ordering as part of the data-preparation check.
Worked Example: A Training Level Missing From the Training Rows
A hypothetical model uses age and region, with levels Coastal, Inland, and Mountain, and Coastal as reference. The training records contain Coastal and Inland observations, but no Mountain observations. A later prediction record is for Mountain. Show how to maintain aligned columns, and explain what the encoding does and does not establish.
Method. Declare all three levels before building the training matrix. For example, use tr$region <- factor(tr$region, levels = c("Coastal", "Inland", "Mountain")). With form <- ~ age + region, treatment coding yields columns age, regionInland, and regionMountain after removing the matrix intercept.
Suppose the two training rows have ages 40 and 55, with regions Coastal and Inland. Their predictor matrix is:
The Mountain column exists, but is zero in every training row. For a new 50-year-old Mountain record, the aligned row is \((50,0,1)\). The training and prediction matrices now have the same three columns in the same order.
This alignment is necessary, but it does not create evidence about the Mountain effect. Since its training column is all zero, the model cannot learn a Mountain contrast from these records; a penalized fit will not obtain a data-supported Mountain coefficient from that column. The appropriate conclusion is not that the coding failed, but that the training data lack observations for that level. The check is both structural and substantive: the columns match, while the Mountain training column has no variation.
Common mistake avoided: Building each matrix independently can omit a factor column when a level is absent in one data set. Supplying a test matrix with a different set or order of columns is not repaired by matching columns by position.
Practical Checks and Exam Traps
Before fitting or predicting, check the matrix as an object rather than assuming that a successful function call means the representation is correct. Useful checks include:
- Confirm that the response is not among the predictor columns.
- Confirm that all predictor columns are numeric and that there are no unintended missing values.
- Inspect the intercept setting and remove or retain the matrix intercept consistently with the fitting configuration.
- Compare the training and prediction column names, then reorder prediction columns to the training order.
- Review factor levels, reference levels, and any levels with no training observations.
One exam trap is to treat matching dimensions as proof of matching meaning. Two matrices may have the same number of columns while representing different categories in different positions. Another is to see a zero-filled indicator for a level absent from training and assume that the model has estimated that level’s effect. The first is an alignment failure; the second is an information limitation. Both require more than checking the matrix dimensions.
Check Your Understanding
Work through each matrix or encoding decision, then compare your result with the answer.
- A factor zone has levels East, North, and West, with East as reference. For observations North then West, state the treatment-coded columns and the two rows after removing the intercept. (Answer: Columns zoneNorth, zoneWest; rows \((1,0)\), \((0,1)\).)
- A model uses a default glmnet intercept, and model.matrix() returns columns (Intercept), age, and channelOnline. Which columns belong in the predictor matrix? (Answer: age and channelOnline; remove (Intercept).)
- A factor has levels Bronze, Silver, Gold, with Bronze as reference. For a Gold record, give the row of factor indicators under treatment coding. (Answer: \((0,1)\) for channelSilver and channelGold, respectively.)
- The training matrix columns are age, planPlus, planPremium. A new matrix has columns planPremium, age, planPlus. What should be done before prediction? (Answer: Reorder the new matrix to age, planPlus, planPremium, after confirming the column names represent the same encoded terms.)
- The training factor levels are A, B, and C, with A as reference, but no training record has level C. What does a C prediction row encode, and what limitation remains? (Answer: Its factor indicators are \((0,1)\) for B and C; the C training column is all zero, so the C effect is not learned from the training data.)