Building Response, Design, and Coefficient Matrices

From Observations to Matrices

In a multivariate regression model, each case contributes a vector of responses, rather than a single response. For example, a clinical study might record both systolic blood pressure and a laboratory marker for each patient. The model uses a common design matrix to describe how the expected values of all outcomes depend on the predictors. Constructing the matrices carefully makes the correspondence between cases, predictors, outcomes, and coefficients explicit.

Let \(n\) be the number of cases, \(q\) the number of outcomes, and \(p\) the number of design columns. Use the general multivariate linear model notation \(Y=XB+U\). The rows of the three matrices must follow the same case order: row \(i\) of \(Y\) contains case \(i\)'s outcomes, and row \(i\) of \(X\) contains that case's predictor values.

Definition: The response matrix \(Y\) has \(n\) rows and \(q\) columns, with one row per case and one column per outcome. The design matrix \(X\) has \(n\) rows and \(p\) columns, with one row per case and one column per model term or predictor column. The coefficient matrix \(B\) has \(p\) rows and \(q\) columns. The error matrix \(U\) has the same dimensions as \(Y\). Thus \(Y=XB+U\) has dimensions \(n\times q=(n\times p)(p\times q)+(n\times q)\).

Write the outcome vector for case \(i\) as the column vector \(y_i\in\mathbb{R}^q\), and write the corresponding design row as \(x_i^{\mathsf T}\in\mathbb{R}^{1\times p}\). Then the \(i\)th row of \(Y\) is \(y_i^{\mathsf T}\), and the \(i\)th row of \(X\) is \(x_i^{\mathsf T}\). If \(u_i\) is the case's error vector, the row-wise model equation is \(y_i^{\mathsf T}=x_i^{\mathsf T}B+u_i^{\mathsf T}\). This notation fixes the orientation: outcomes are columns in \(Y\), and the coefficients for one outcome form a column of \(B\).

Building the Three Matrices

To build \(Y\), first decide which measured variables are outcomes and set a fixed column order. Each row then contains the outcomes for one case in that order. To build \(X\), choose the model terms and their coding, and apply the same column order to every case. An intercept is represented by a column of ones. A continuous predictor contributes its recorded value in each case's row. A categorical predictor is represented by one or more numerical columns according to the chosen coding.

Once the columns of \(X\) are specified, each row of \(B\) gives the coefficients associated with one design column, across all \(q\) outcomes. For instance, with an intercept and one continuous predictor, the first row of \(B\) contains the intercept coefficients for the outcomes; the second contains their slopes. A coefficient is not a single number when there are multiple outcomes: it is one entry of \(B\), indexed by a predictor column and an outcome column.

Worked Example: Two Outcomes and One Continuous Predictor

Three patients have an age measurement \(x\), systolic pressure, and a laboratory measurement. Use the outcome order pressure, laboratory measurement, and the predictor order intercept, age. Suppose the observed values are:

Patient	Age \(x\)	Pressure	Laboratory measurement
1	40	128	5.2
2	55	142	6.1
3	65	151	6.8

There are \(n=3\) cases and \(q=2\) outcomes. Enter each patient's two outcomes in one row, in the stated order:

Y= \begin{pmatrix} 128&5.2\\ 142&6.1\\ 151&6.8 \end{pmatrix}_{3\times 2}.

The design has an intercept column followed by age, so \(p=2\):

X= \begin{pmatrix} 1&40\\ 1&55\\ 1&65 \end{pmatrix}_{3\times 2}, \qquad B= \begin{pmatrix} \beta_{01}&\beta_{02}\\ \beta_{11}&\beta_{12} \end{pmatrix}_{2\times 2}.

The first outcome's intercept and age slope are \(\beta_{01}\) and \(\beta_{11}\); the second outcome's are \(\beta_{02}\) and \(\beta_{12}\). Multiplication gives

XB= \begin{pmatrix} \beta_{01}+40\beta_{11}&\beta_{02}+40\beta_{12}\\ \beta_{01}+55\beta_{11}&\beta_{02}+55\beta_{12}\\ \beta_{01}+65\beta_{11}&\beta_{02}+65\beta_{12} \end{pmatrix}_{3\times 2}.

For example, the first row predicts both outcomes for patient 1. The model equation allows observed outcomes to differ from these fitted mean values through the corresponding row of \(U\). The dimension check is \(Y:3\times2\), \(X:3\times2\), \(B:2\times2\), and hence \(XB:3\times2\).

Deriving the Row-Wise Model Equation

The matrix model is not merely a compact way to list data: its multiplication encodes one regression equation per case and outcome. The following derivation makes the indexing precise.

Result: If \(X\) is \(n\times p\), \(B\) is \(p\times q\), and \(U\) is \(n\times q\), then the \(i,j\) entry of \(Y=XB+U\) is \(Y_{ij}=\sum_{k=1}^{p}X_{ik}B_{kj}+U_{ij}\). Equivalently, the predicted value for case \(i\) and outcome \(j\) is the sum of that case's \(p\) design values multiplied by the corresponding coefficients for outcome \(j\).

Proof. By the definition of matrix multiplication, the \((i,j)\) entry of \(XB\) is \(\sum_{k=1}^{p}X_{ik}B_{kj}\): row \(i\) of \(X\) is paired with column \(j\) of \(B\). Adding the \((i,j)\) entry \(U_{ij}\) gives the displayed formula for \(Y_{ij}\). In vector form, collect the \(q\) entries for a fixed case \(i\); the resulting row is \(x_i^{\mathsf T}B+u_i^{\mathsf T}\). This also shows why the \(i\)th rows of \(X\), \(Y\), and \(U\) must refer to the same case. \(\square\)

Worked Example: A Categorical Group and a Continuous Predictor

Five patients belong to one of three treatment groups, A, B, or C. For each patient, record age \(z\) and two outcomes, symptom score and biomarker. Use treatment A as the reference group, with indicator columns \(I_B\) and \(I_C\), where \(I_B=1\) for group B and \(0\) otherwise, and likewise for \(I_C\). Let the design-column order be intercept, \(I_B\), \(I_C\), age.

Patient	Group	Age \(z\)	Symptom score	Biomarker
1	A	40	12	4.8
2	B	50	10	5.1
3	C	60	9	5.6
4	B	45	11	5.0
5	A	55	13	5.3

There are \(n=5\) cases, \(q=2\) outcomes, and \(p=4\) design columns. Applying the coding to each patient gives

X= \begin{pmatrix} 1&0&0&40\\ 1&1&0&50\\ 1&0&1&60\\ 1&1&0&45\\ 1&0&0&55 \end{pmatrix}_{5\times4}, \qquad Y= \begin{pmatrix} 12&4.8\\ 10&5.1\\ 9&5.6\\ 11&5.0\\ 13&5.3 \end{pmatrix}_{5\times2}.

The coefficient matrix has one row for each design column and one column for each outcome:

B= \begin{pmatrix} \beta_{01}&\beta_{02}\\ \beta_{B1}&\beta_{B2}\\ \beta_{C1}&\beta_{C2}\\ \beta_{z1}&\beta_{z2} \end{pmatrix}_{4\times2}.

For a group-A patient aged 40, the model's two predicted responses are \(\beta_{01}+40\beta_{z1}\) and \(\beta_{02}+40\beta_{z2}\). For a group-B patient aged 50, they are \(\beta_{01}+\beta_{B1}+50\beta_{z1}\) and \(\beta_{02}+\beta_{B2}+50\beta_{z2}\). Thus the indicator coefficients represent group differences from A at the same age, under this additive design. The check is \(X:5\times4\), \(B:4\times2\), so \(XB:5\times2\), matching \(Y\).

Intercepts and Reference-Coded Factors

For a categorical predictor with \(g\) levels, treatment or reference coding with an intercept uses \(g-1\) indicator columns. The omitted level is the reference. Each included level receives an indicator that is one for that level and zero otherwise. In a model without interactions, the corresponding coefficient row gives that level's difference from the reference in each outcome, holding the other design variables fixed. The intercept row gives the reference-level prediction when the continuous predictors in the design are zero.

Result: With an intercept and \(g-1\) reference-coded indicators, the group contribution to the linear predictor is the intercept for the reference level and the intercept plus the relevant indicator coefficient for each other level.

Derivation. For a reference-level case, all \(g-1\) indicators equal zero. In the row product \(x_i^{\mathsf T}B\), only the intercept coefficient and any other nonzero predictor contributions remain. For a case in included level \(r\), its indicator is one and all other group indicators are zero, so its group contribution is the intercept coefficient plus the coefficient for indicator \(r\). Subtracting the reference-level group contribution cancels the intercept and leaves exactly that indicator coefficient. This calculation applies separately to every outcome column of \(B\).

Worked Example: Checking Reference-Level Predictions

Suppose there are three groups A, B, and C, an intercept, and no other predictors. There are two outcomes. Under reference coding with A omitted, the design matrix and coefficient matrix have the forms

X= \begin{pmatrix} 1&0&0\\ 1&1&0\\ 1&0&1 \end{pmatrix}, \qquad B= \begin{pmatrix} \beta_{01}&\beta_{02}\\ \beta_{B1}&\beta_{B2}\\ \beta_{C1}&\beta_{C2} \end{pmatrix}.

The three rows of \(XB\) are, in group order A, B, C:

XB= \begin{pmatrix} \beta_{01}&\beta_{02}\\ \beta_{01}+\beta_{B1}&\beta_{02}+\beta_{B2}\\ \beta_{01}+\beta_{C1}&\beta_{02}+\beta_{C2} \end{pmatrix}.

For outcome 1, the group-B minus group-A prediction is \((\beta_{01}+\beta_{B1})-\beta_{01}=\beta_{B1}\). For outcome 2 it is \((\beta_{02}+\beta_{B2})-\beta_{02}=\beta_{B2}\). The same subtraction for group C gives \(\beta_{C1}\) and \(\beta_{C2}\). Here \(n=3\), \(p=3\), and \(q=2\), so \(XB\) has dimensions \(3\times2\), as required.

Why Matrix Construction Matters

The matrices preserve two kinds of alignment. First, every row refers to the same case across \(X\) and \(Y\); sorting one table without applying the same permutation to the other breaks the intended pairing. Second, every column has a fixed meaning: changing the outcome order changes the interpretation of the columns of \(B\), and changing the design-column order changes the interpretation of its rows. Dimensions can reveal some construction errors, but they cannot detect a silent reordering that leaves the dimensions unchanged.

Coding choices also determine what individual coefficients mean. With an intercept and all \(g\) group indicators, the group columns sum to the intercept column. The design columns are then linearly dependent, so the separate coefficients are not uniquely determined by the model equation. Using \(g-1\) indicators avoids that particular redundancy. The aim here is not to select a coding for every analysis, but to ensure the chosen coding is stated and represented consistently.

Common Mistakes

Putting outcomes in rows and cases in columns without adjusting the model. In \(Y=XB+U\), the convention is \(n\) cases by \(q\) outcomes. Check that \(Y\) and \(XB\) have matching dimensions.
Using one coefficient row per outcome. Coefficients are arranged with design columns as rows and outcomes as columns, so \(B\) is \(p\times q\).
Including an intercept and every category indicator without checking redundancy. With \(g\) levels, the \(g\) indicators sum to the intercept column. Use a stated coding convention and check the design columns.
Reading an indicator coefficient as an absolute group mean. Under reference coding with an intercept, it is a difference from the reference level, conditional on other predictors—not the group's full predicted response.
Reordering cases in only one matrix. Matching dimensions do not guarantee that the rows still refer to the same cases.

Key takeaway: State the case order, outcome order, and design-column order before writing the matrices. Then verify \(Y\) is \(n\times q\), \(X\) is \(n\times p\), \(B\) is \(p\times q\), and each row of \(XB\) corresponds to the same case as the matching row of \(Y\).

Check Your Understanding

Use the matrix dimensions and the predictor coding to justify each answer.

A study has 80 cases, three outcomes, and a design with an intercept and two predictors. What are the dimensions of \(Y\), \(X\), and \(B\)?
For a four-level treatment factor with an intercept, how many reference-coded indicator columns are needed? What does an included indicator coefficient represent?
If the first two columns of \(Y\) are reversed while \(X\) is unchanged, what must also change for the coefficient matrix to retain its intended interpretation?
Explain why an intercept and indicators for all three levels of a categorical factor are linearly dependent.
With outcomes in columns, what does row \(i\) of \(XB\) represent, and which row of the response matrix must it align with?