Organizing Observations Into Rows and Predictors Into Columns

Why Rows and Columns Need a Fixed Meaning

A design matrix is more than a rectangular arrangement of numbers. Its organization tells us which predictor values belong to which observation. A row is assigned to one observation, and a column is assigned to one predictor. If either assignment is unclear, a correctly computed value can still be attached to the wrong case or variable.

In a generalized linear model, the linear predictor is built using the predictor values for each observation. The design matrix records those values in a consistent format. This tutorial focuses on its dimensions and indexing: how many rows and columns it has, and what the entry at a given row and column means. It does not depend on the form of the response distribution.

The two indices answer different questions. The first index, \(i\), identifies which observation; the second, \(j\), identifies which predictor. The entry \(x_{ij}\) is their intersection. The row can be written \((x_{i1},\ldots,x_{ip})\), and the column for predictor \(j\) consists of \((x_{1j},\ldots,x_{nj})\).

The observation index is not necessarily a patient ID, a row number in the original spreadsheet, or a time point. It is an index for the observations as they are organized for the analysis. The predictor index is similarly an index for the chosen predictors, not a numerical measurement. Labels such as “age” and “baseline score” tell us what columns mean; the indices \(j=1,2,\ldots,p\) give them positions.

Worked Examples of Indexing

Dimensions and Index Sets

The indexing can be made explicit by defining the observation-index set \(I=\{1,\ldots,n\}\) and the predictor-index set \(J=\{1,\ldots,p\}\). Each matrix entry has an index pair \((i,j)\) in \(I\times J\). This description is useful when checking whether a proposed matrix has the right size or whether an entry has been placed in the right location.

Proof. There are \(n\) possible choices for \(i\). For each fixed \(i\), there are \(p\) possible choices for \(j\), so there are \(p\) index pairs in that row. Summing over all \(n\) row indices gives \(p+\cdots+p\), with \(n\) terms, or \(np\) pairs. Since each pair \((i,j)\) identifies one entry \(x_{ij}\), there are exactly \(np\) entries. \(\square\)

This gives a simple dimensional check. If a data set has \(n=6\) observation records and \(p=2\) predictors, its design matrix should contain \(12\) recorded predictor values. A missing value or an extra column may require a deliberate data-handling decision, but it does not change what an \(n\)-by-\(p\) array means.

Keeping Responses and Predictor Rows Aligned

In a data set with a response, the response for observation \(i\) must stay associated with row \(i\) of the design matrix. A response vector can be indexed as \(y_1,\ldots,y_n\), where \(y_i\) is the response for the same observation whose predictor values form row \(i\). This is a bookkeeping requirement: a row containing one patient's predictors cannot be paired with another patient's response.

Proof. Let \(\pi\) be a permutation of \(\{1,\ldots,n\}\), meaning that it reorders the \(n\) observation indices without dropping or repeating any index. After reordering, place the original row \(i\) in new position \(\pi(i)\), and place its response \(y_i\) in that same new position. At position \(\pi(i)\), the response is therefore still \(y_i\), and the predictor row is still the row containing the values \(x_{i1},\ldots,x_{ip}\). Their pairing is unchanged. If only the predictor rows are moved, the response at a position generally remains from its former observation; if only the responses are moved, the predictor row generally remains from its former observation. Either operation can therefore mismatch the paired records. \(\square\)

For a concrete check, suppose the first two records are \((y_1,\text{row }1)=(0,(52,18,10))\) and \((y_2,\text{row }2)=(1,(61,22,15))\). Swapping the records means the new first position contains response \(1\) and row \((61,22,15)\), while the new second position contains response \(0\) and row \((52,18,10)\). The two original pairings remain intact. Swapping the rows alone would instead put \((61,22,15)\) beside \(0\), changing which response is associated with those predictor values.

Common Mistakes and a Practical Check

• Reversing the index meanings. In \(x_{ij}\), \(i\) indexes the observation (the row), while \(j\) indexes the predictor (the column). State both meanings before interpreting an entry.
• Counting subjects when the rows are records. With repeated visits, the number of rows may be the number of visits rather than the number of people. First identify the observational unit for the data arrangement.
• Counting the response as a predictor by default. The \(p\) columns here are the specified predictor columns. A response recorded in the same source table is not automatically one of them.
• Reordering one part of a record. The reasoning “the rows are in a different order, so the response list can stay where it is” is wrong if the responses remain indexed by their old positions. Move each response with its corresponding predictor row, or retain a stable identifier to verify the match.
• Using the wrong number of columns. Count the predictors in the model specification, not every field that happens to appear in the raw data table. Each selected predictor contributes a column of values for the observations.

A reliable setup check is to name the observational unit, list the predictors in a fixed order, record the number of observations and predictors, and verify several entries by their row and column labels. Then check that each response is still paired with the predictors from the same observation. These checks prevent indexing errors before any model fitting begins.