Constructing Factorial Design Matrices

From Factor Levels to Matrix Rows

A factorial model is specified in terms of factor levels and terms such as main effects and interactions. A design matrix makes that specification concrete: each row records the predictor values for one observation, and each column corresponds to one coefficient in the linear predictor. Constructing the rows carefully is important because an incorrect indicator or product column changes the model being fitted.

Consider a factor \(A\) with levels \(A_1,A_2\) and a factor \(B\) with levels \(B_1,B_2,B_3\). We use reference coding with \(A_1\) as the reference level of \(A\) and \(B_1\) as the reference level of \(B\). The reference level is represented by zeros in that factor’s indicator columns.

Definition: For observation \(i\), define \(a_i=1\) if its \(A\)-level is \(A_2\), and \(a_i=0\) if its \(A\)-level is \(A_1\). Define \(b_{2i}=1\) if its \(B\)-level is \(B_2\), and \(b_{2i}=0\) otherwise; define \(b_{3i}=1\) if its \(B\)-level is \(B_3\), and \(b_{3i}=0\) otherwise. With an intercept, both main effects, and their interaction, the row for observation \(i\) is \((1,a_i,b_{2i},b_{3i},a_i b_{2i},a_i b_{3i})\).

The interaction columns are products of the corresponding main-effect indicators. There is no separate interaction column for \(A_1\) or \(B_1\): when either factor is at its reference level, the relevant product is zero. Writing the coefficients in the same order as the columns gives the linear predictor

\eta_i=\beta_0+\beta_Aa_i+\beta_{B2}b_{2i}+\beta_{B3}b_{3i} +\beta_{AB2}a_ib_{2i}+\beta_{AB3}a_ib_{3i}.

The design matrix \(X\) is obtained by calculating this row for every observation, in the same order as the observations in the data. Thus its dimensions are \(n\) rows by 6 columns. The response distribution and link determine how the linear predictor is used in a GLM; they do not change these factor-indicator calculations.

The Complete Two-by-Three Layout

For clarity, first list each of the six possible combinations once. The column labels state the coefficient associated with each column.

Combination	Intercept	\(A_2\)	\(B_2\)	\(B_3\)	\(A_2:B_2\)	\(A_2:B_3\)
\(A_1,B_1\)	1	0	0	0	0	0
\(A_1,B_2\)	1	0	1	0	0	0
\(A_1,B_3\)	1	0	0	1	0	0
\(A_2,B_1\)	1	1	0	0	0	0
\(A_2,B_2\)	1	1	1	0	1	0
\(A_2,B_3\)	1	1	0	1	0	1

For example, the \(A_2,B_2\) row has \(a_i=1\) and \(b_{2i}=1\), so its \(A_2:B_2\) product is 1. Its \(B_3\) indicator and \(A_2:B_3\) product are both 0. In contrast, the \(A_1,B_2\) row has \(b_{2i}=1\) but \(a_i=0\), so the interaction product is 0.

In compact form, with rows in the displayed order, the matrix is

X= \begin{pmatrix} 1&0&0&0&0&0\\ 1&0&1&0&0&0\\ 1&0&0&1&0&0\\ 1&1&0&0&0&0\\ 1&1&1&0&1&0\\ 1&1&0&1&0&1 \end{pmatrix}.

Worked Example: Building and Using the Six-Row Matrix

Suppose the coefficient vector, ordered to match the table, is \(\beta=(1.2,0.5,-0.3,0.4,0.2,-0.6)^\mathsf{T}\). Construct the predictor for each combination and check it by multiplying the row by \(\beta\).

For \(A_1,B_1\), the row is \((1,0,0,0,0,0)\), giving \(\eta=1.2\). For \(A_1,B_2\), the row is \((1,0,1,0,0,0)\), giving \(\eta=1.2-0.3=0.9\). For \(A_1,B_3\), the row is \((1,0,0,1,0,0)\), giving \(\eta=1.2+0.4=1.6\).

For \(A_2,B_1\), the row is \((1,1,0,0,0,0)\), giving \(\eta=1.2+0.5=1.7\). For \(A_2,B_2\), the row is \((1,1,1,0,1,0)\), giving \(\eta=1.2+0.5-0.3+0.2=1.6\). For \(A_2,B_3\), the row is \((1,1,0,1,0,1)\), giving \(\eta=1.2+0.5+0.4-0.6=1.5\).

The direct matrix product gives the same result:

X\beta=(1.2,\;0.9,\;1.6,\;1.7,\;1.6,\;1.5)^\mathsf{T}.

This check catches a common implementation error: a coefficient vector and matrix columns can both be numerically valid while being in different orders. The multiplication is meaningful only when each coefficient is paired with its stated column.

Construction for Observations in Any Order

The six-row table is a lookup table, not a requirement that the data appear in that order. For each actual observation, identify its factor levels, write its indicators, multiply them to obtain the interaction entries, and place the resulting row in that observation’s position. If a combination occurs multiple times, its row occurs multiple times as well.

Fix the references and column order.
Here the references are \(A_1,B_1\), and the columns are intercept, \(A_2,B_2,B_3,A_2:B_2,A_2:B_3\).

Code the factor levels.
For each observation, assign \(a_i\), \(b_{2i}\), and \(b_{3i}\) according to its levels. Exactly one of the three \(B\) patterns \((0,0),(1,0),(0,1)\) applies.

Form the product columns.
Calculate \(a_ib_{2i}\) and \(a_ib_{3i}\) from the indicators in that same row.

Preserve observation order.
Append the row where that observation occurs in the data. Do not sort the matrix independently of the response or other observation-level quantities.

Worked Example: Encoding Repeated Combinations

Six observations appear in this order: \((A_2,B_3),(A_1,B_2),(A_2,B_1),(A_2,B_3),(A_1,B_1),(A_1,B_2)\). Build \(X\) in that order.

The rows follow directly from the six-combination lookup table. Both observations in \(A_2,B_3\) receive \((1,1,0,1,0,1)\); both in \(A_1,B_2\) receive \((1,0,1,0,0,0)\). The other two rows are \((1,1,0,0,0,0)\) and \((1,0,0,0,0,0)\), in the positions corresponding to \(A_2,B_1\) and \(A_1,B_1\). Therefore

X= \begin{pmatrix} 1&1&0&1&0&1\\ 1&0&1&0&0&0\\ 1&1&0&0&0&0\\ 1&1&0&1&0&1\\ 1&0&0&0&0&0\\ 1&0&1&0&0&0 \end{pmatrix}.

Using the coefficient vector from the previous example, the predictors are obtained row by row as \(1.5,0.9,1.7,1.5,1.2,0.9\). The repeated rows produce repeated predictors because the linear predictor depends on the factor combination and coefficients, not on the observation’s position in the data.

Why the Product Columns Represent Interaction

The product columns are not additional indicators for new factor levels. They modify the predictor only when both relevant nonreference indicators are 1. This can be checked directly by comparing the four cell predictors involving \(A_1,A_2\) and \(B_1,B_2\).

Result: In the reference-coded model above, the difference-in-differences across \(A_1,A_2\) and \(B_1,B_2\) equals \(\beta_{AB2}\). Across \(A_1,A_2\) and \(B_1,B_3\), it equals \(\beta_{AB3}\).

To verify the first equality, the four predictors from their matrix rows are

\eta_{11}=\beta_0,\quad \eta_{12}=\beta_0+\beta_{B2},\quad \eta_{21}=\beta_0+\beta_A,\quad \eta_{22}=\beta_0+\beta_A+\beta_{B2}+\beta_{AB2}.

Subtracting the \(A_1\)-to-\(A_2\) change at \(B_1\) from that change at \(B_2\) gives

(\eta_{22}-\eta_{12})-(\eta_{21}-\eta_{11}) =(\beta_A+\beta_{AB2})-\beta_A =\beta_{AB2}.

For \(B_3\), replace \(\eta_{12},\eta_{22},\beta_{B2},\beta_{AB2}\) with \(\eta_{13},\eta_{23},\beta_{B3},\beta_{AB3}\). The same subtraction gives \((\eta_{23}-\eta_{13})-(\eta_{21}-\eta_{11})=\beta_{AB3}\). Thus each product column encodes how the \(A\) contrast changes at that nonreference level of \(B\). \(\square\)

Rank Check for the Complete Layout

When every factor combination is represented, the six rows in the lookup table identify six distinct cell predictors. The matrix has full column rank, as the following direct argument shows.

Result: The displayed \(6\) by \(6\) design matrix has rank \(6\).

To prove the claim, suppose \(X\beta=0\). The row for \(A_1,B_1\) implies \(\beta_0=0\). The row for \(A_1,B_2\) then implies \(\beta_{B2}=0\), and the row for \(A_1,B_3\) implies \(\beta_{B3}=0\). The \(A_2,B_1\) row implies \(\beta_A=0\). Next, the \(A_2,B_2\) row gives \(\beta_0+\beta_A+\beta_{B2}+\beta_{AB2}=0\), so \(\beta_{AB2}=0\). Finally, the \(A_2,B_3\) row gives \(\beta_0+\beta_A+\beta_{B3}+\beta_{AB3}=0\), so \(\beta_{AB3}=0\). Thus the only vector in the null space is the zero vector, which proves that the six columns are linearly independent. \(\square\)

This rank check applies to the six-row complete layout shown here. In a data set with repeated observations, the rows repeat and the number of rows increases, but the same six distinct row patterns are present if all combinations occur. The point of the check is to connect the coding to the available information: a column is useful only insofar as its values across observations distinguish its coefficient from the others.

Worked Example: A Grouped Binomial Design Matrix

Suppose a study records successes and trials for six groups, one at each factor combination. The groups are listed in the order \(A_1B_1,A_2B_1,A_1B_3,A_2B_2,A_1B_2,A_2B_3\), with success and trial counts \((2,10),(4,10),(5,10),(6,10),(3,10),(7,10)\). Construct the predictor matrix for a model with the specified main effects and interaction terms.

The response counts do not determine the design rows; the factor levels do. Applying the lookup table in the supplied order gives

X= \begin{pmatrix} 1&0&0&0&0&0\\ 1&1&0&0&0&0\\ 1&0&0&1&0&0\\ 1&1&1&0&1&0\\ 1&0&1&0&0&0\\ 1&1&0&1&0&1 \end{pmatrix}.

For example, the fourth group is \(A_2,B_2\), so both \(a_i\) and \(b_{2i}\) equal 1; the other \(B\) indicator is 0, and the interaction products are 1 and 0. Its row is therefore \((1,1,1,0,1,0)\). The first group is at both reference levels, so its row is \((1,0,0,0,0,0)\). This illustrates a general practical check: construct \(X\) from the factor labels and then align its rows with the response counts in exactly the same group order.

Common Mistakes

Making a column for every level as well as an intercept. With reference coding and an intercept, use indicators only for nonreference levels: one for \(A_2\), and two for \(B_2,B_3\).
Using the wrong interaction indicator. The \(A_2:B_2\) entry is \(a_ib_{2i}\), not \(a_i+b_{2i}\). It equals 1 only when both indicators equal 1.
Reordering rows without reordering the response. The row order of \(X\) must match the observation or group order used for the response.
Assuming the outcome changes the coding. A Gaussian, binomial, or Poisson response may use this same factor design; the factor levels determine these columns.

Key takeaway: Fix the reference levels and column order, code each observation’s factor indicators, form the interaction columns as products, and preserve the data order. The resulting row is the direct record of which terms contribute to that observation’s linear predictor.

Check Your Understanding

Use reference levels \(A_1\) and \(B_1\), with columns ordered as intercept, \(A_2,B_2,B_3,A_2:B_2,A_2:B_3\).

Write the design row for an observation at \(A_1,B_3\).
Write the design row for an observation at \(A_2,B_2\), and explain why exactly one of the interaction columns is 1.
If a data set has observations in the order \(A_2B_1,A_1B_1,A_2B_1\), write its three-row design matrix.
For \(A_2,B_3\), write the full expression for the linear predictor using the six coefficients.
Explain why the response counts in a grouped binomial data set do not change the factor-coded entries of \(X\).