Design-Matrix Columns as Vectors in R n

From a Matrix Column to a Vector Across Observations

A design matrix is often introduced as a rectangular array of numbers: rows correspond to observations, columns correspond to predictors, and multiplying the matrix by a coefficient vector gives the linear predictors. A useful next step is to view each entire column as one vector. Its entries are the values of one predictor across all \(n\) observations, in their fixed order. This viewpoint makes the matrix product more than a compact calculation: it shows exactly how each predictor contributes to the vector of linear predictors.

This is a statement about the organization and arithmetic of the linear predictor. It applies regardless of which response distribution or link function is used in the GLM. The response model determines how the linear predictor is used; here the focus is how that predictor vector is assembled from the design matrix.

Definition: Let \(X\) be an \(n \times p\) design matrix, with \(n\) observations and \(p\) columns. Its \(j\)th column is the vector \(\mathbf{x}_j \in \mathbb{R}^n\) whose \(i\)th entry is the design-matrix entry \(x_{ij}\). For a coefficient vector \(\boldsymbol{\beta}=(\beta_1,\ldots,\beta_p)^\mathsf{T}\), the vector of linear predictors is \(\boldsymbol{\eta}=X\boldsymbol{\beta}\in\mathbb{R}^n\); its \(i\)th entry is \(\eta_i\), the linear predictor for observation \(i\).

The order of entries matters. The first entry of every column refers to observation 1, the second to observation 2, and so on. Thus \(\mathbf{x}_j\) is not just a list of predictor values detached from the data: it is the list aligned with the same observation order used by every other column and by \(\boldsymbol{\eta}\).

The matrix also has a row-wise interpretation. Row \(i\) collects the \(p\) design entries for observation \(i\), while column \(j\) collects the values of predictor \(j\) across all observations. These are two views of the same entries. Rows support the calculation for one observation; columns expose the contribution of a predictor across the whole sample.

The Column-Decomposition Result

For each observation, the usual matrix product takes a row of \(X\) and forms its dot product with the coefficient vector. Collecting those \(n\) results gives \(\boldsymbol{\eta}\). Equivalently, each coefficient scales one design column, and the scaled columns are added as vectors in \(\mathbb{R}^n\).

Theorem: If \(X\) has columns \(\mathbf{x}_1,\ldots,\mathbf{x}_p\) and \(\boldsymbol{\beta}=(\beta_1,\ldots,\beta_p)^\mathsf{T}\), then

X\boldsymbol{\beta}=\sum_{j=1}^{p}\beta_j\mathbf{x}_j.

The \(i\)th component of this vector is \(\eta_i=\sum_{j=1}^{p}x_{ij}\beta_j\).

Proof. By the definition of matrix multiplication, the \(i\)th component of \(X\boldsymbol{\beta}\) is \(\sum_{j=1}^{p}x_{ij}\beta_j\). The \(i\)th component of \(\sum_{j=1}^{p}\beta_j\mathbf{x}_j\) is also \(\sum_{j=1}^{p}\beta_j x_{ij}\), because the \(i\)th component of \(\mathbf{x}_j\) is \(x_{ij}\). These components agree for every \(i=1,\ldots,n\). Two vectors in \(\mathbb{R}^n\) with equal components are equal, proving the identity. \(\square\)

The result is simply the matrix product written one column at a time. It says that coefficients are scalar weights and design columns are vectors being weighted. The sum is vector addition: at each observation, the corresponding predictor contributions are added to produce that observation's linear predictor.

Worked Example: Combining an Intercept and Two Numerical Predictors

Suppose there are four observations, and the model has an intercept, a predictor \(x_1\), and a predictor \(x_2\). Let

X= \begin{pmatrix} 1&0&2\\ 1&1&1\\ 1&2&0\\ 1&3&2 \end{pmatrix}, \qquad \boldsymbol{\beta}= \begin{pmatrix} 2\\ -1\\ 0.5 \end{pmatrix}.

The three columns, read down the observations, are

\mathbf{x}_1= \begin{pmatrix}1\\1\\1\\1\end{pmatrix}, \qquad \mathbf{x}_2= \begin{pmatrix}0\\1\\2\\3\end{pmatrix}, \qquad \mathbf{x}_3= \begin{pmatrix}2\\1\\0\\2\end{pmatrix}.

The column decomposition gives

\boldsymbol{\eta} =2\mathbf{x}_1-\mathbf{x}_2+0.5\mathbf{x}_3 = \begin{pmatrix}2\\2\\2\\2\end{pmatrix} + \begin{pmatrix}0\\-1\\-2\\-3\end{pmatrix} + \begin{pmatrix}1\\0.5\\0\\1\end{pmatrix} = \begin{pmatrix}3\\1.5\\0\\0\end{pmatrix}.

Check the same result observation by observation. For observation 1, \(2(1)-1(0)+0.5(2)=3\). For observation 2, \(2(1)-1(1)+0.5(1)=1.5\). For observation 3, \(2(1)-1(2)+0.5(0)=0\). For observation 4, \(2(1)-1(3)+0.5(2)=0\). The vector result and the four row-wise calculations agree.

The intercept column is the vector of four ones, so its contribution is \(2\) at every observation. The other two columns vary across observations, and their weighted contributions vary accordingly. A negative coefficient reverses the direction of the column's contribution; it does not change which observations the entries refer to.

Why the Vector Sum Gives the Same Predictors as the Row Calculation

The column expression and row expression are two equivalent ways to organize one calculation. The row view asks, “What is the predictor for observation \(i\)?” The column view asks, “What vector of contributions does coefficient \(\beta_j\) add across all observations?” Keeping these views connected is especially useful when checking a matrix calculation or interpreting an intercept.

Result: For each observation \(i\), the \(i\)th entry of the column sum \(\sum_{j=1}^{p}\beta_j\mathbf{x}_j\) equals the dot product of row \(i\) of \(X\) with \(\boldsymbol{\beta}\). Therefore, computing all row-wise linear predictors and computing the weighted column sum produce the same vector.

Derivation. Let \(\mathbf{r}_i^\mathsf{T}=(x_{i1},\ldots,x_{ip})\) denote row \(i\) of \(X\). The row calculation is

\mathbf{r}_i^\mathsf{T}\boldsymbol{\beta} =\sum_{j=1}^{p}x_{ij}\beta_j.

The \(i\)th entry of the column sum is the sum of the \(i\)th entries of its scaled columns:

\left(\sum_{j=1}^{p}\beta_j\mathbf{x}_j\right)_i =\sum_{j=1}^{p}\beta_j(\mathbf{x}_j)_i =\sum_{j=1}^{p}\beta_j x_{ij}.

The two expressions are equal term by term. This holds for every observation \(i\), so the row calculations collectively equal the column sum as vectors in \(\mathbb{R}^n\). \(\square\)

Worked Example: A Binary-Predictor Column Across Observations

A study records five observations. Let \(g_i=1\) for the treatment group and \(g_i=0\) for the control group, and let \(t_i\) be a numerical predictor. Suppose the design matrix and coefficients are

X= \begin{pmatrix} 1&0&2\\ 1&1&2\\ 1&0&4\\ 1&1&4\\ 1&1&6 \end{pmatrix}, \qquad \boldsymbol{\beta}= \begin{pmatrix} 1\\ 2\\ 0.5 \end{pmatrix}.

The columns are \(\mathbf{x}_1=(1,1,1,1,1)^\mathsf{T}\), \(\mathbf{x}_2=(0,1,0,1,1)^\mathsf{T}\), and \(\mathbf{x}_3=(2,2,4,4,6)^\mathsf{T}\). The second column is the treatment indicator across all five observations: its zero entries align with control observations and its one entries align with treatment observations.

Weight each column by its coefficient and add:

\boldsymbol{\eta} =\mathbf{x}_1+2\mathbf{x}_2+0.5\mathbf{x}_3 = \begin{pmatrix}1\\1\\1\\1\\1\end{pmatrix} + \begin{pmatrix}0\\2\\0\\2\\2\end{pmatrix} + \begin{pmatrix}1\\1\\2\\2\\3\end{pmatrix} = \begin{pmatrix}2\\4\\3\\5\\6\end{pmatrix}.

For example, observation 4 has row \((1,1,4)\), so its predictor is \(1(1)+2(1)+0.5(4)=5\), matching the fourth component. Observation 3 has row \((1,0,4)\), giving \(1(1)+2(0)+0.5(4)=3\), matching the third component. The treatment column adds 2 to the predictor for observations 2, 4, and 5, and adds 0 to the other two; this is the same contribution represented by the vector \(2\mathbf{x}_2\).

Intercepts and Predictor Contributions as Vectors

If a model includes an intercept, its design column consists of \(n\) ones, \(\mathbf{1}_n=(1,\ldots,1)^\mathsf{T}\). Multiplying this column by an intercept coefficient \(\beta_0\) gives \(\beta_0\mathbf{1}_n\), which shifts every component of \(\boldsymbol{\eta}\) by the same amount. A numerical predictor column \(\mathbf{x}\), by contrast, contributes \(\beta\mathbf{x}\); observations with different predictor values receive different contributions.

Worked Example: Separating the Baseline Shift From a Predictor Contribution

Suppose three observations have an intercept column and a predictor column \(\mathbf{x}=(1,3,5)^\mathsf{T}\), with intercept \(\beta_0=-1\) and slope \(\beta_1=0.4\). Then

\boldsymbol{\eta} =-1 \begin{pmatrix}1\\1\\1\end{pmatrix} +0.4 \begin{pmatrix}1\\3\\5\end{pmatrix} = \begin{pmatrix}-1\\-1\\-1\end{pmatrix} + \begin{pmatrix}0.4\\1.2\\2.0\end{pmatrix} = \begin{pmatrix}-0.6\\0.2\\1.0\end{pmatrix}.

The first vector is the same baseline shift for every observation. The second vector contains the predictor's observation-specific contributions. Checking by rows gives \(-1+0.4(1)=-0.6\), \(-1+0.4(3)=0.2\), and \(-1+0.4(5)=1.0\). This check confirms both the arithmetic and the alignment of the entries.

Common Mistake: Treating a Column as One Number

A design-matrix column is not a single predictor value, and it is not a coefficient. It is an \(n\)-component vector of predictor values. The coefficient is a scalar that multiplies that whole vector. For instance, if \(\mathbf{x}_j=(2,4,1)^\mathsf{T}\) and \(\beta_j=3\), the contribution is \((6,12,3)^\mathsf{T}\), not just the number \(3\) or the number \(4\).

Another common error is to combine entries from different observations. In the fourth observation's predictor, use the fourth entry of every column; do not pair the fourth entry of one column with the third entry of another. The fixed row order is what makes each component of the column sum correspond to one observation.

The vector viewpoint matters because it gives a direct way to audit a calculation: scale each column by its coefficient, add the vectors, and verify selected components against their row-wise dot products. It also makes the effect of a coefficient visible across the dataset, while keeping clear that the resulting \(\boldsymbol{\eta}\) is a vector of predictors, not a vector of responses.

Key takeaway: Each design-matrix column is a vector in \(\mathbb{R}^n\), indexed in observation order. The coefficient for that column scales the vector, and summing all scaled columns yields \(\boldsymbol{\eta}=X\boldsymbol{\beta}\). Component \(i\) of that sum is exactly the linear predictor for observation \(i\).

Check Your Understanding

Use the column-vector view to answer each question.

If \(X\) has \(n\) rows and \(p\) columns, how many entries does each design-column vector have, and what does its \(i\)th entry represent?
Write \(X\boldsymbol{\beta}\) as a sum of scaled columns when \(X\) has three columns.
For an intercept column of all ones with coefficient \(\beta_0\), describe its contribution to every component of \(\boldsymbol{\eta}\).
Suppose \(\mathbf{x}_1=(1,0,2)^\mathsf{T}\), \(\mathbf{x}_2=(3,1,0)^\mathsf{T}\), and the coefficients are \(2\) and \(-1\). Find \(\boldsymbol{\eta}\), then verify its second component using the corresponding matrix row.
Explain why the column entries must remain in the same observation order when forming the vector sum.