Preparing an R Design Matrix for Ridge

Why Prepare the Design Matrix?

A regression formula describes relationships in terms of variables, but a ridge calculation operates on numbers arranged in a matrix. In R, model.matrix() translates a formula and data frame into that numeric representation. The result is not just a formatting step: an unexpected intercept, an unused factor level, or a constant predictor can change the columns supplied to a later fitting procedure.

This tutorial focuses on constructing and auditing the matrix. It does not fit a ridge model. The key task is to verify what each matrix column represents, how many rows and columns it has, and whether any columns are constant or redundant.

Definition: For $n$ observations and $p$ predictor columns, a design matrix $X$ is an $n \times p$ numeric matrix. Row $i$ represents observation $i$; column $j$ represents one numeric predictor term, such as a measured variable or an indicator created from a factor. In R, model.matrix() constructs this matrix from a formula and data.

For example, model.matrix(~ age + smoker, data = dat) makes columns for the intercept, the numeric predictor age, and the factor predictor smoker. The formula’s terms determine which predictors are represented; R’s factor contrasts determine how categorical levels become columns. Always inspect the output rather than assuming the column names or count.

Constructing and Reading a Matrix

With the usual formula convention, ~ includes an intercept unless the formula explicitly removes it. A factor with two levels typically contributes one indicator column under treatment contrasts: one level is the reference, and the column indicates the other level. The reference level is represented by a zero in that indicator column, not by a separate column of ones.

Worked Example: Building a Matrix with a Binary Factor

A hypothetical pricing study has four records. The numeric predictor is driver experience in years. The factor channel is recorded as Direct or Partner, with Direct as the reference level. Construct the design matrix from ~ experience + channel, report its dimensions and columns, and describe each row.

Method. Use model.matrix() on the predictor formula, then inspect dim() and colnames(). The data are: experience $=(1, 2, 3, 4)$ and channel $=(\text{Direct}, \text{Partner}, \text{Direct}, \text{Partner})$. Set the factor levels explicitly so the intended reference is clear.

The R construction is dat$channel <- factor(dat$channel, levels = c("Direct", "Partner")), followed by X <- model.matrix(~ experience + channel, data = dat). The default intercept contributes a column of ones. Direct is the first level, so the factor column is channelPartner, equal to 1 for Partner and 0 for Direct.

X = \begin{bmatrix} 1 & 1 & 0 \\ 1 & 2 & 1 \\ 1 & 3 & 0 \\ 1 & 4 & 1 \end{bmatrix}

The columns, in order, are (Intercept), experience, and channelPartner. There are 4 rows and 3 columns, so dim(X) returns c(4, 3). The first row means intercept $1$, experience $1$, and Partner indicator $0$. The second means intercept $1$, experience $2$, and Partner indicator $1$. The third and fourth rows follow the same coding rule.

Check. The intercept column contains four ones; the experience column reproduces the four numeric values; and the Partner column is $0,1,0,1$. This method avoids the common mistake of expecting separate indicator columns for both levels when the model includes an intercept.

Useful inspections include dim(X), colnames(X), head(X), and str(X). The first gives the row and column counts; the second identifies the terms; the third displays rows; and the fourth checks the object’s structure. For a design matrix intended for numerical modelling, confirm that it is a numeric matrix and not a data frame containing factor columns.

A formula can include a response, as in outcome ~ experience + channel; model.matrix() returns the predictor representation, not a column for the response. Keep track of the observations used to construct the matrix, especially if missing values cause rows to be omitted. The response and matrix rows must refer to the same observations in the same order.

Factor Levels and Unexpected Columns

The factor’s levels—not just the values visible in a few printed rows—affect the resulting columns. Explicitly setting the levels makes the intended coding easier to inspect. A factor may also retain a level that has no observations in the current data. With treatment contrasts, that unused level can produce a column that is zero for every row.

Worked Example: Finding an Unused Factor Level

A hypothetical service study has four records with region values East, West, East, West. The factor was previously defined with levels East, West, and Central. Construct the matrix for ~ region and identify any column that needs attention before further modelling.

Method. Preserve the declared level order, build the matrix, and inspect each factor column. East is the reference level. The output has an intercept and one treatment-contrast column for each non-reference level: regionWest and regionCentral.

X = \begin{bmatrix} 1 & 0 & 0 \\ 1 & 1 & 0 \\ 1 & 0 & 0 \\ 1 & 1 & 0 \end{bmatrix}

There are 4 rows and 3 columns. regionWest is $0,1,0,1$, matching the observed regions. regionCentral is $0,0,0,0$, because no record is Central. A direct check is colSums(X): the column sums are $4, 2, 0$.

If Central is not a level needed for this modelling data, remove unused levels before constructing the matrix, for example with dat$region <- droplevels(dat$region). Rebuilding the matrix then gives two columns: the intercept and regionWest. Its dimensions are $4 \times 2$, and its rows are $(1,0)$, $(1,1)$, $(1,0)$, and $(1,1)$.

Check. The zero column is a consequence of the unused level, not evidence of four Central observations or of an observed effect. This method avoids treating a column name as proof that the corresponding category occurs in the data.

A different issue arises when a factor has only one level in the data being modelled. There is no contrast to estimate between levels. R may report that contrasts cannot be applied to a factor with fewer than two levels. Check the observed values and levels; do not try to interpret a missing contrast column as a meaningful predictor effect.

Intercepts, Constant Columns, and Redundancy

A constant numeric predictor has the same value in every row. When its value is 1 and the formula also includes an intercept, its column duplicates the intercept column exactly. More generally, any constant column is a linear combination of an intercept column. Such redundancy means that the columns are not linearly independent, which can make coefficient representations non-unique.

Ridge regression conventionally leaves the intercept unpenalized. In a workflow where the fitting function supplies its own intercept, the input predictor matrix is commonly prepared without the explicit (Intercept) column. Removing that column does not automatically solve every design-matrix issue: a constant predictor column can remain and should be identified separately. The exact matrix expected depends on the fitting interface, so check its convention before passing the matrix onward.

Worked Example: Detecting a Constant Predictor

A hypothetical renewal study has three observations. tenure is $1,2,3$ years, and active is recorded as 1 for every observation. Build the matrix for ~ tenure + active. Identify the constant column, determine the matrix rank, and describe what remains after removing only the explicit intercept.

Method. Construct the matrix, inspect its columns, and compare its column count with its rank. The formula creates columns in this order: intercept, tenure, active.

X = \begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 1 \\ 1 & 3 & 1 \end{bmatrix}

The dimensions are $3 \times 3$. The intercept values are $1,1,1$; tenure values are $1,2,3$; and active values are $1,1,1$. The sums are $3,6,3$, respectively. The first and third columns are identical, so there are only two linearly independent columns: the intercept pattern and the varying tenure pattern. Thus the matrix rank is 2, less than its 3 columns; in R, qr(X)$rank returns 2.

Removing only (Intercept) leaves the $3 \times 2$ matrix with rows $(1,1)$, $(2,1)$, and $(3,1)$. The remaining active column is still constant. A check such as apply(X, 2, function(z) length(unique(z))) returns $1,3,1$ for the original columns and reveals both constant columns.

Check. The matrix has three columns but rank two because two columns are duplicates. This method avoids the incorrect assumption that dropping the generated intercept also removes every constant predictor.

A Justification for the Matrix Representation

The matrix form follows directly from the way a regression linear predictor is assembled. For the first worked example, write the predictor for observation $i$ as an intercept, a coefficient multiplying experience, and a coefficient multiplying the Partner indicator:

\eta_i = \beta_0 + \beta_1\,\text{experience}_i + \beta_2\,I(\text{channel}_i=\text{Partner}).

Here $I(\text{condition})$ equals 1 when the condition is true and 0 otherwise. The $i$th row of $X$ is therefore $(1,\text{experience}_i,I(\text{channel}_i=\text{Partner}))$. Multiplying that row by the coefficient vector gives

(1,\text{experience}_i,I(\text{channel}_i=\text{Partner})) \begin{bmatrix} \beta_0\\ \beta_1\\ \beta_2 \end{bmatrix} = \eta_i.

Stacking these row-by-row equalities gives $\boldsymbol{\eta}=X\boldsymbol{\beta}$. This explains why each encoded term must correspond to a numeric column and why a mistaken or redundant column changes the model representation. The argument describes a linear predictor; it does not require that the response itself be normally distributed.

Practical Audit Before Ridge Fitting

A short, deliberate audit catches many errors before fitting. Use the formula you intend to model, build the matrix from the intended data, and inspect its structure. When a model expects a predictor matrix without an intercept, remove the intercept by its name rather than assuming it is always in a particular position.

Confirm the input rows.
Check that the data contain the intended observations and that missing-value handling has not changed the rows used for the response.

Build the matrix.
Use model.matrix() with the predictor terms and factor levels that match the intended model specification.

Inspect dimensions and names.
Use dim(X) and colnames(X); compare the resulting terms with the formula and expected factor contrasts.

Check columns and constants.
Inspect the first rows, factor-level columns, and columns with only one unique value. Investigate all-zero and duplicate columns.

Match the fitting convention.
Determine whether the downstream procedure expects an explicit intercept in the matrix or estimates an intercept separately.

For a formula with an intercept and $k$ factor levels under treatment contrasts, the factor ordinarily contributes $k-1$ columns when all levels are represented and the factor has at least two levels. This count is a useful check, not a substitute for inspecting the actual output: unused levels, changed contrasts, or formula terms can alter what appears.

Common Mistakes

Assuming the formula has no intercept. An ordinary R formula includes one unless it is explicitly removed. Check for (Intercept) in colnames(X).
Expecting one column per factor level when an intercept is present. Treatment coding uses a reference level; with two levels, the usual result is one indicator column plus the intercept.
Keeping an unused level without checking its column. An unused factor level can correspond to an all-zero column. Verify observed level counts and inspect the matrix.
Removing the intercept and stopping the audit. A separate numeric predictor may still be constant or redundant. Check every column, not only the intercept.
Ignoring row alignment. If missing data cause observations to be omitted during matrix construction, the response must use the matching rows in the same order.

Key takeaway: A design matrix should be treated as an object to verify, not as an automatic by-product to trust. Confirm its dimensions, names, factor coding, row alignment, and constant columns before using it in a ridge workflow.

Check Your Understanding

For each problem, reason from the specified formula and factor levels before checking the answer.

A data set has three observations with factor delivery taking values Mail, App, Mail, and levels ordered Mail then App. What are the dimensions and columns of model.matrix(~ delivery, data = d)? (Answer: $3 \times 2$; (Intercept) and deliveryApp.)
A data set has four rows, numeric score, and factor zone with levels North and South. What are the dimensions and columns of model.matrix(~ 0 + zone + score, data = d)? (Answer: $4 \times 3$; zoneNorth, zoneSouth, and score.)
A factor has declared levels Oak, Pine, Birch, but the observed values are Oak, Pine, Oak. Under ~ tree with Oak as the reference, which factor column is all zero, and what dimensions result if the unused level is dropped before construction? (Answer: treeBirch is all zero; after dropping Birch, the matrix has 3 rows and 2 columns: intercept and treePine.)
For three rows, numeric rate is $2,4,6$ and numeric constant is $7,7,7$. What is the rank of the matrix from ~ rate + constant, and what remains after deleting only the intercept column? (Answer: rank 2 for the $3 \times 3$ matrix; a constant constant column remains in the $3 \times 2$ matrix.)
A response has eight rows, while one predictor is missing in exactly one row and the default missing-value action omits incomplete cases. How many rows will the resulting model matrix have, and what alignment check is needed? (Answer: 7 rows; ensure the response is restricted to those same seven observations in the same order.)

What You'll Learn