Why Multi-Level Predictors Need a Coding Rule
A categorical predictor can have several levels, such as four underwriting tiers or three types of service plan. Regression requires numerical inputs, so we represent each observation's category with indicator variables. The coding must distinguish the categories while avoiding redundant columns in the design matrix.
Suppose a categorical predictor has \(k\) levels, and the regression includes an intercept. The standard dummy-coding rule is to select one level as the reference category and create one indicator column for each of the other \(k-1\) levels. The reference level is represented by zeros in all those indicator columns.
For a model with only this categorical predictor, the mean response for observation \(i\) is written
Here \(\beta_0\) is the mean response for the reference category. For a nonreference category \(j\), its mean is \(\beta_0+\beta_j\), so \(\beta_j\) is the difference between that category's mean and the reference category's mean. If other predictors are included, their terms are added to the model; the indicator columns still identify the categorical level.
The indicators are not measurements of category size or order. For example, coding a category as level 1, 2, or 3 and entering those numbers as one quantitative predictor would impose a particular linear trend across categories. Dummy coding instead allows a separate mean difference for each nonreference category.
Building the Design Matrix
A design matrix has one row per observation and one column per model coefficient. Its first column is the intercept, and each remaining column corresponds to a predictor term. To construct a row for a categorical predictor, write 1 in the indicator column for the observation's category and 0 in the other indicator columns. For an observation in the reference category, write 0 in every indicator column.
The arrangement of the nonreference columns must be stated, because each coefficient is attached to a particular category. For example, with levels Bronze, Silver, and Gold and Bronze as reference, the columns should be identified as intercept, Silver indicator, and Gold indicator. A Silver observation then has row \((1,1,0)\), not \((1,0,1)\).
Worked Example: Three Categories and Their Indicator Columns
A hypothetical insurer groups policy applications into Basic, Standard, and Enhanced tiers. A response \(Y\) is measured for two applications in each tier. Use Basic as the reference category, construct the design matrix, and find the least-squares coefficients for the model with only the tier predictor.
The observed responses, in the stated order, are Basic: 8 and 10; Standard: 13 and 15; Enhanced: 18 and 20. The design-matrix columns are intercept, Standard indicator, and Enhanced indicator. Thus the six rows are
For Basic, the fitted group mean is \(\widehat\beta_0\). The two Basic observations have mean \((8+10)/2=9\), so \(\widehat\beta_0=9\). The Standard mean is \((13+15)/2=14\). Since its fitted mean is \(\widehat\beta_0+\widehat\beta_{\text{Standard}}\), the Standard coefficient is \(14-9=5\). The Enhanced mean is \((18+20)/2=19\), giving \(\widehat\beta_{\text{Enhanced}}=19-9=10\).
Therefore the coefficient vector, in the column order shown, is \((9,5,10)^\mathsf{T}\). The fitted mean for Basic is 9, for Standard is \(9+5=14\), and for Enhanced is \(9+10=19\). The residuals within each group are \(-1,1\); the residual sum of squares is \(1+1+1+1+1+1=6\). This checks that the fitted values are the three sample means. The method is to encode the categories first, then fit the group means; a common mistake is to treat the tier labels as numerical scores and impose an unrequested trend.
Why One Fewer Indicator Than Categories?
With an intercept, the indicator columns for all \(k\) categories would be redundant. For each observation, exactly one of the \(k\) category indicators equals 1. Therefore the sum of all category-indicator columns equals the intercept column. This is an exact linear dependence, so the design matrix cannot have full column rank.
The usual least-squares coefficient formula \(\widehat{\boldsymbol\beta}=(X^\mathsf{T}X)^{-1}X^\mathsf{T}\mathbf y\) requires \(X\) to have full column rank, so that \(X^\mathsf{T}X\) is invertible. When the columns are redundant, the fitted values may still be determined, but the individual coefficients are not uniquely identified by that formula. Using \(k-1\) indicators with an intercept removes this particular redundancy, provided the categories are represented in the data and other predictors do not introduce additional dependencies.
The reason the one-factor model estimates group means can be shown directly. Let \(n_j\) be the number of observations in category \(j\), and let \(\bar y_j\) be their sample mean. For a proposed fitted value \(\mu_j\) in that category, the sum of squared residuals within it is
To verify the equality, write \(y_i-\mu_j=(y_i-\bar y_j)+(\bar y_j-\mu_j)\), square, and sum. The cross-product term is zero because \(\sum_{i:\,j(i)=j}(y_i-\bar y_j)=0\). The first term on the right does not depend on \(\mu_j\), while the second is nonnegative and is minimized at \(\mu_j=\bar y_j\). With one intercept and \(k-1\) indicators, the model can assign an unrestricted fitted mean to every category: \(\mu_1=\beta_0\) and \(\mu_j=\beta_0+\beta_j\) for \(j\ge 2\). Least squares therefore sets each fitted category mean to its sample mean, when each category has observations.
Worked Example: Four Levels With a Stated Column Order
For a hypothetical portfolio analysis, an analyst records response values for policies assigned to Central, East, North, or West regions. The observations are Central: 4 and 6; East: 7 and 9; North: 3 and 5; West: 10 and 12. Use Central as the reference and order the indicator columns East, North, West. Construct \(X\) and calculate the least-squares coefficient vector.
There are four levels, so the model has three indicator columns plus the intercept. Listing rows in the same order as the response vector gives
The Central mean is \((4+6)/2=5\), so \(\widehat\beta_0=5\). The East mean is \((7+9)/2=8\), giving \(\widehat\beta_{\text{East}}=8-5=3\). The North mean is \((3+5)/2=4\), giving \(\widehat\beta_{\text{North}}=4-5=-1\). The West mean is \((10+12)/2=11\), giving \(\widehat\beta_{\text{West}}=11-5=6\).
Thus the coefficient vector in the specified column order is \((5,3,-1,6)^\mathsf{T}\). Checking by category gives fitted means \(5\), \(5+3=8\), \(5-1=4\), and \(5+6=11\), exactly the four sample means. The negative North coefficient is not a negative predicted response; it is a difference of \(-1\) relative to the Central mean. The common mistake avoided here is assigning a coefficient to the wrong region by overlooking the stated indicator-column order.
Adding Other Predictors
A categorical predictor can be included alongside quantitative predictors. In an additive model, the design matrix contains the intercept, the quantitative predictor columns, and the \(k-1\) indicator columns. Each row combines the measured quantitative values with the correct indicator pattern. For example, a row in the reference category has zeros in all category-indicator columns but still contains its observed quantitative predictor values.
The coding rule alone does not guarantee a unique least-squares coefficient vector. For a unique solution, \(X\) must have full column rank. At a minimum, every category whose separate mean or effect is to be estimated must occur in the data. Also, the other design-matrix columns must not be exact linear combinations of existing columns. When a category is absent, its indicator cannot be estimated from these observations.
Worked Example: Three Categories Alongside a Quantitative Predictor
A hypothetical risk study records a response \(Y\), a quantitative score \(x\), and a three-level review class A, B, or C. Use A as the reference, with the model \(Y=\beta_0+\beta_xx+\beta_BD_B+\beta_CD_C+\varepsilon\). The observations \((\text{class},x,Y)\) are \((A,0,2)\), \((A,1,5)\), \((B,0,6)\), \((B,1,9)\), \((C,0,1)\), and \((C,1,4)\). Construct the design matrix and solve for the coefficients.
The columns are intercept, \(x\), B indicator, and C indicator. In the given observation order,
The first row gives \(\beta_0=2\) for the exact equations because \(x=0\) and both indicators are zero. Comparing the second and first rows, \(x\) increases from 0 to 1 while the other columns stay fixed, and \(Y\) increases from 2 to 5. Thus \(\beta_x=5-2=3\). In the third row, \(x=0\) and \(D_B=1\), so \(6=\beta_0+\beta_B=2+\beta_B\), giving \(\beta_B=4\). The fourth row checks this result: \(2+3+4=9\), matching its response. The fifth row gives \(1=2+\beta_C\), so \(\beta_C=-1\). The sixth row checks: \(2+3-1=4\).
The solution is \((\widehat\beta_0,\widehat\beta_x,\widehat\beta_B,\widehat\beta_C)=(2,3,4,-1)\), and each fitted value equals its observed response, so the residuals are all zero. The matrix has independent columns: the A rows distinguish the intercept from \(x\), and the B and C rows then distinguish the two indicators. This exact-fit example illustrates how to place both predictor types in the matrix; with noisy observations, least squares estimates the coefficients by minimizing the total squared residuals rather than matching every response exactly.
Common Mistakes and Checks
An efficient way to review a proposed coding is to check the number and meaning of its columns before estimating anything. For \(k\) categories with an intercept, count \(k-1\) indicator columns, confirm that exactly one indicator is 1 for each nonreference observation, and confirm that all indicators are 0 for each reference observation. Then check that the rows align with the response observations.
- Including all \(k\) indicators with an intercept. The indicator columns sum to the intercept column, creating exact redundancy. Drop one indicator and name its category as the reference.
- Using \(k\) indicator columns but no intercept without saying so. That is a different parameterization. Do not combine it with the \(k-1\)-indicator rule for a model that includes an intercept.
- Using category labels as numerical scores. This treats the differences between coded numbers as meaningful and imposes structure that ordinary dummy coding does not impose.
- Changing the reference without changing coefficient labels. The fitted model may be represented with a different reference, but the coefficient meanings change. Always record which level is omitted.
- Assuming every coefficient can be estimated because a column was written down. A level with no observations has no information in the data to estimate its effect.
Worked Example: Detecting Redundancy in a Proposed Matrix
A hypothetical survey uses categories Day, Evening, and Overnight. An analyst includes an intercept and all three category indicators. For a Day observation, the proposed row is \((1,1,0,0)\); for Evening it is \((1,0,1,0)\); for Overnight it is \((1,0,0,1)\). Identify the issue and give a corrected coding with Day as reference.
The proposed columns are intercept, Day, Evening, and Overnight. Adding the three indicator columns row by row gives \((1,1,1)\) down those columns: for each observation exactly one category indicator equals 1. Therefore, as columns, Day + Evening + Overnight equals the intercept column. The four columns are linearly dependent, and the coefficient vector is not uniquely determined.
With Day as the reference, omit its indicator column. The corrected columns are intercept, Evening, and Overnight. The rows become Day \((1,0,0)\), Evening \((1,1,0)\), and Overnight \((1,0,1)\). There are now \(3-1=2\) indicator columns, and the direct dependence caused by including every category indicator alongside the intercept is removed. The correction follows the \(k-1\) rule; merely keeping all four columns and attempting to interpret their coefficients separately does not resolve the redundancy.
Check Your Understanding
For each problem, write the requested coding or coefficient result before checking the answer.
- A three-level predictor has levels Low, Medium, and High. Low is the reference. State the indicator-column names and the design row for a High observation. (Answer: columns \(D_{\text{Medium}},D_{\text{High}}\); High row including the intercept is \((1,0,1)\).)
- A model with an intercept has five categories. How many indicator columns are needed, and what happens if all five category indicators are included? (Answer: four indicators; all five with an intercept create exact linear dependence because their columns sum to the intercept column.)
- For categories Cedar (reference), Maple, and Pine, the sample means are 12, 17, and 9. Give the one-factor dummy-coded least-squares coefficients in the column order intercept, Maple, Pine. (Answer: \((12,5,-3)^\mathsf{T}\).)
- A four-level predictor uses A as reference and indicator columns B, C, D. An observation is in C and has quantitative predictor \(x=2.5\). Give its design row for the column order intercept, \(x\), B, C, D. (Answer: \((1,2.5,0,1,0)\).)
- A three-level predictor has reference category A and indicators B and C. The data contain observations in A and B but none in C. Which category effect cannot be estimated from these observations, and why? (Answer: the C effect cannot be estimated because its indicator is zero for every observed row, so the data provide no information about its coefficient.)