Common Design-Matrix Errors

Why Matrix Bookkeeping Matters

A design matrix is more than a compact way to store predictor values. Its rows identify observations, its columns identify predictors, and its coefficient vector must follow that same column order. A matrix product can be arithmetically valid while pairing a predictor value with the wrong observation or coefficient. Those bookkeeping errors change the linear predictors without necessarily producing an obvious calculation warning.

For \(n\) observations and \(p\) columns, write the design matrix as \(X\), the coefficient vector as \(\beta\), and the linear predictor vector as \(\eta=X\beta\). The \(i\)th entry \(\eta_i\) is formed from row \(i\) of \(X\) and the coefficient associated with each column. In the examples below, labels are included explicitly so that the arithmetic can be checked against the intended data structure.

Definition: A design matrix \(X\) has one row per observation and one column per modeled term. Its entry \(x_{ij}\) is the value of term \(j\) for observation \(i\). With a coefficient vector \(\beta\) whose entries are ordered to match the columns of \(X\), the linear predictor is \(\eta=X\beta\), an \(n\)-entry vector. An intercept is represented by a column of ones.

For example, if the columns are ordered as intercept, \(x\), and \(z\), then the coefficient vector must be ordered as intercept coefficient, \(x\) coefficient, and \(z\) coefficient. Each row calculation is a dot product: multiply the entries in that row by their corresponding coefficients and add the products.

Worked Audits of Common Errors

Worked Example: A Predictor Column Is Misaligned With the Observations

Three patients, A, B, and C, have predictor values \(x=(0,1,2)\) and \(z=(1,2,0)\), respectively. Use the model \(\eta_i=2+0.5x_i-z_i\). The intended row order is A, B, C, and the columns are intercept, \(x\), \(z\).

The correctly assembled design matrix and coefficient vector are

X= \begin{pmatrix} 1&0&1\\ 1&1&2\\ 1&2&0 \end{pmatrix}, \qquad \beta= \begin{pmatrix} 2\\ 0.5\\ -1 \end{pmatrix}.

Calculate each row explicitly. For A, \(2+0(0.5)+1(-1)=1\). For B, \(2+1(0.5)+2(-1)=0.5\). For C, \(2+2(0.5)+0(-1)=3\). Thus \(\eta=(1,0.5,3)^\mathsf{T}\), in the order A, B, C.

Suppose the \(z\) values are accidentally entered in the order B, A, C, giving the column \((2,1,0)^\mathsf{T}\), while the rows are still labelled A, B, C. The erroneous matrix produces \(2+0(0.5)+2(-1)=0\) for A and \(2+1(0.5)+1(-1)=1.5\) for B; C remains \(3\). The product is now \((0,1.5,3)^\mathsf{T}\), which assigns the wrong \(z\) value to each of A and B. The fix is to align every predictor column to the same observation order before multiplication.

Worked Example: Column Order and Coefficient Order Disagree

Suppose the intended columns are intercept, \(x\), \(z\), with coefficients \((1,0.2,2)^\mathsf{T}\). The two rows are \((1,0,3)\) and \((1,1,5)\). Calculate the predictors using the stated matching order:

\begin{pmatrix} 1&0&3\\ 1&1&5 \end{pmatrix} \begin{pmatrix} 1\\ 0.2\\ 2 \end{pmatrix} = \begin{pmatrix} 1+0(0.2)+3(2)\\ 1+1(0.2)+5(2) \end{pmatrix} = \begin{pmatrix} 7\\ 11.2 \end{pmatrix}.

Now reorder the predictor columns as intercept, \(z\), \(x\). The rows become \((1,3,0)\) and \((1,5,1)\). To preserve the same model, reorder the coefficients to \((1,2,0.2)^\mathsf{T}\). The row calculations are \(1+3(2)+0(0.2)=7\) and \(1+5(2)+1(0.2)=11.2\), as before.

If the reordered matrix is mistakenly multiplied by the original coefficient vector \((1,0.2,2)^\mathsf{T}\), the results are instead \(1+3(0.2)+0(2)=1.6\) and \(1+5(0.2)+1(2)=4\). The arithmetic is valid, but the \(x\) coefficient has been applied to the \(z\) column and vice versa. Always check the column labels and coefficient labels together.

Worked Example: Omitting the Intercept Column

Consider \(\eta_i=2+3x_i\) for observations with \(x=0,1,2\). With the intercept included, the matrix and coefficient vector are

X= \begin{pmatrix} 1&0\\ 1&1\\ 1&2 \end{pmatrix}, \qquad \beta= \begin{pmatrix} 2\\ 3 \end{pmatrix}, \qquad X\beta= \begin{pmatrix} 2\\ 5\\ 8 \end{pmatrix}.

If the column of ones is removed, the remaining design matrix has one column, \((0,1,2)^\mathsf{T}\). Multiplying that matrix by the original two-entry coefficient vector is undefined: the number of columns and number of coefficient entries do not match. If instead the coefficient vector is shortened to \((3)^\mathsf{T}\), the product is \((0,3,6)^\mathsf{T}\). This is a different model, \(\eta_i=3x_i\), with no intercept; in particular, its predictor at \(x=0\) is \(0\), not \(2\).

Thus an omitted intercept is not repaired by casually dropping a coefficient. First determine whether the intended model includes a constant term; then build its column and coefficient entry consistently.

When Reordering Preserves the Model

Reordering is not automatically an error. It is harmless when the relevant labels and quantities are reordered together. The following two facts make that distinction precise. A permutation matrix is a square matrix obtained by reordering the columns of an identity matrix; it satisfies \(P^\mathsf{T}P=PP^\mathsf{T}=I\).

Theorem: Suppose \(X\) is an \(n\)-by-\(p\) design matrix and \(\beta\) is a \(p\)-entry coefficient vector. If the columns of \(X\) are reordered by a permutation matrix \(P\), then \(X'=XP\) and \(\beta'=P^\mathsf{T}\beta\) give the same linear predictor: \(X'\beta'=X\beta\).

Proof. By the definitions of \(X'\) and \(\beta'\), and associativity of matrix multiplication,

X'\beta'=(XP)(P^\mathsf{T}\beta) =X(PP^\mathsf{T})\beta =XI\beta =X\beta.

The middle equality uses \(PP^\mathsf{T}=I\), which holds because a permutation matrix only reorders the coordinates. Hence the predictors are unchanged when the coefficient entries are reordered to match the columns. \(\square\)

Theorem: Suppose \(X\) has \(n\) rows, \(\beta\) has \(p\) entries, and \(Q\) is an \(n\)-by-\(n\) permutation matrix. Reordering the rows gives \(X'=QX\) and \(\eta'=X'\beta=Q\eta\). The numerical predictors are the same values in the correspondingly reordered observation order.

Proof. Since \(\eta=X\beta\), associativity gives

\eta'=X'\beta=(QX)\beta=Q(X\beta)=Q\eta.

Left multiplication by \(Q\) permutes the entries of a vector in the same way that it permutes the rows of a matrix. Therefore, each observation retains its predictor when its row and its associated label are moved together. If row labels or outcome records are not reordered with the design matrix, the displayed entries may be assigned to the wrong observations. \(\square\)

These results give a practical diagnostic: a consistent reordering changes only the arrangement of the calculation, not which predictor belongs to which observation. By contrast, moving a column without its coefficient, moving a predictor value without its row label, or removing a column changes the pairings used to form the dot products.

A Practical Audit Before Multiplication

Check the observation key.
Write the intended observation order beside the rows. Confirm that every predictor value in a row belongs to that same observation.

Check the column labels.
Record the column order explicitly, including the intercept if the model has one.

Check the coefficient labels.
Match each coefficient to its predictor column. A coefficient vector is not self-describing unless its order is known.

Check dimensions and sample rows.
An \(n\)-by-\(p\) matrix requires a \(p\)-entry coefficient vector and produces an \(n\)-entry predictor vector. Compute at least one row by hand to verify the pairings and arithmetic.

Common Mistakes and Why They Matter

Assuming a valid matrix product must be the intended one. Compatible dimensions only establish that multiplication is possible. They do not confirm that rows, columns, and coefficients have the intended meanings.
Sorting one predictor column independently. This can detach its values from their observations. Sort or reorder whole records, or use a stable observation identifier to realign the column.
Changing the column order but not the coefficient order. The resulting dot products apply coefficients to the wrong predictors. Reorder both, as in the permutation result.
Omitting the intercept but retaining the old coefficient vector. The dimensions no longer match. Dropping the intercept coefficient as well produces a no-intercept model, which generally gives different predictors.
Reordering rows but leaving outcomes or labels behind. The numerical vector may be correctly permuted while its entries are reported for the wrong cases. Row order must be tracked consistently across all observation-level quantities.

These checks matter before any later calculation based on the linear predictor. A wrong \(\eta_i\) can be carried into subsequent model calculations, and checking those later calculations will not reveal which row or coefficient was originally mismatched. Matrix bookkeeping is therefore part of specifying the model, not merely a formatting preference.

Key takeaway: For every calculation \(\eta=X\beta\), verify three alignments: rows with observations, columns with modeled terms, and coefficient entries with columns. Reordering is safe only when all associated labels and values move together; omitting an intended intercept changes the model.

Check Your Understanding

For each question, state what you would inspect and, where requested, calculate the result.

A design matrix has rows labelled A, B, C, but one predictor column was copied in order C, A, B. What information must be used to correct the column?
The columns of \(X\) are intercept, \(u\), \(v\), while the entries of \(\beta\) are labelled intercept, \(v\), \(u\). Is the product dimensionally valid, and why can it still be wrong?
For \(X=\begin{pmatrix}1&2\\1&4\end{pmatrix}\) with columns intercept and \(x\), calculate \(X\beta\) for \(\beta=(3,0.5)^\mathsf{T}\). What does the first row represent?
Explain why dropping the column of ones from an intercept model and dropping the matching coefficient do not generally preserve the linear predictors.
When rows of \(X\) are permuted, what other observation-level records must be reordered if the predictors are to remain assigned to the correct cases?

What You'll Learn