Why Combine Quantitative and Categorical Predictors?
A response can vary with both a numerical measurement and membership in a category. For example, a researcher might study how an outcome score changes with hours of practice while accounting for which training program each participant attended. A model using only practice hours could overlook systematic differences between programs; a model using only program could overlook the relationship with practice.
Multiple regression can include both kinds of predictors. In this tutorial, we focus on an additive model: it estimates a quantitative slope and adjusted group differences, while assuming that the quantitative slope is the same across groups. The word “adjusted” means that a group comparison is made at the same value of the quantitative predictor. It does not, by itself, imply a causal effect.
Reading the Model’s Coefficients
Set \(D_i=0\) to obtain the reference-group regression line, and set \(D_i=1\) to obtain the other group’s line:
Both lines have slope \(\beta_1\). Their vertical separation is \(\beta_2\), whatever value of \(x_i\) is used. This equal-slope condition is part of the additive model, not a conclusion that follows automatically from observing two groups. A model that allows group-specific slopes is a different model.
The model’s usual assumptions concern the errors \(\varepsilon_i\): their conditional means are zero, observations are independent, and their conditional variances are constant across predictor values and groups. For the usual exact small-sample regression tests and intervals, errors are also assumed to be normally distributed. The mean relationship must be adequately represented by a straight-line quantitative trend and additive group differences. The observations should cover predictor values relevant to the comparisons; if groups have little or no overlap in \(x\), an adjusted comparison may rely heavily on extrapolation.
Worked Example: Estimating a Common Slope and an Adjusted Group Difference
Worked Example: Estimating a Common Slope and an Adjusted Group Difference
Suppose a hypothetical training evaluation records a score \(Y\) and practice hours \(x\) for participants in two programs. Program A is the reference group, and \(D=1\) identifies Program B. The following data are hypothetical:
| Program | Practice hours \(x\) | Observed scores \(Y\) |
|---|---|---|
| A | 0 | 49, 51 |
| A | 1 | 54, 56 |
| A | 2 | 59, 61 |
| B | 0 | 55, 57 |
| B | 1 | 60, 62 |
| B | 2 | 65, 67 |
The model is \(Y_i=\beta_0+\beta_1x_i+\beta_2D_i+\varepsilon_i\). Each program has observations at all three practice-hour values, so there is overlap for comparing the groups across this observed range. The pattern of cell averages is linear: Program A’s averages are 50, 55, and 60, while Program B’s are 56, 61, and 66. At each practice-hour value, the group-average difference is 6 points. The example therefore supports the additive pattern in these data; in a real analysis, residual plots and subject-matter knowledge would also be relevant to assessing the model assumptions. Treating participants’ observations as independent would require that no participant contributes multiple dependent observations.
To verify the least-squares slope, the overall mean of \(x\) is \(\bar{x}=1\), since there are four observations at each of \(x=0,1,2\). The slope numerator is \(\sum (x_i-\bar{x})(Y_i-\bar{Y})\). The overall score mean is \(\bar{Y}=58\). The four scores at \(x=0\) sum to 212, and the four scores at \(x=2\) sum to 252; the four at \(x=1\) have zero multiplier. Thus
The slope denominator is 8: the eight observations at \(x=0\) or \(x=2\) each contribute \(1\) to \(\sum(x_i-\bar{x})^2\), while the four observations at \(x=1\) each contribute \(0\). Therefore,
The estimated adjusted difference is 6 points. For instance, at \(x=1\), Program A’s mean is \((54+56)/2=55\) and Program B’s mean is \((60+62)/2=61\), so the difference is \(61-55=6\). Because each program has the same number of observations at each \(x\), this difference is consistent across the three observed values.
At \(x=0\), Program A’s average is 50, so the fitted reference intercept is \(\hat{\beta}_0=50\). The fitted equation is
The observed scores at each group-and-\(x\) combination are one point below and one point above their cell average. The residuals therefore sum to zero within every such combination. Consequently, they sum to zero within each group and their \(x\)-weighted sums are zero as well. The residuals are orthogonal to the model’s intercept, \(x\), and \(D\) columns, which are the least-squares normal-equation conditions. This verifies that the displayed coefficients are the least-squares fit. The fitted values describe conditional mean scores; they do not establish that attending Program B causes a six-point increase.
For a participant in Program B with 1.5 practice hours, the predicted score is
At the same practice hours in Program A, the prediction is \(50+5(1.5)+6(0)=57.5\) points. The predicted difference is \(63.5-57.5=6\) points, matching the adjusted group coefficient.
More Than Two Categories
A categorical predictor with \(k\) levels is represented using \(k-1\) indicator variables when the model includes an intercept. One category is chosen as the reference; each remaining indicator marks membership in one of the other categories. Each indicator coefficient then compares its category with the reference, adjusting for the quantitative predictors in the model. This follows the reference-category coding used for categorical predictors more generally.
Worked Example: Interpreting a Three-Program Model
Suppose a hypothetical analysis uses Program A as the reference and fits the following model for an outcome score, with practice hours \(x\):
where \(D_B=1\) for Program B and \(0\) otherwise, and \(D_C=1\) for Program C and \(0\) otherwise. The indicators are mutually exclusive, so Program A has \(D_B=D_C=0\), Program B has \(D_B=1,D_C=0\), and Program C has \(D_B=0,D_C=1\). The equation implies a common estimated slope of \(-2.4\) score points per additional practice hour in all three programs. It estimates that, at the same number of practice hours, Program B’s mean score is 8.0 points higher than Program A’s, while Program C’s is 5.5 points lower.
For \(x=3\) hours, the fitted mean for Program A is \(72-2.4(3)=72-7.2=64.8\) points. For Program B it is \(72-2.4(3)+8.0=72.8\) points. For Program C it is \(72-2.4(3)-5.5=59.3\) points. These are model-based fitted means at three hours, not guarantees about individual scores. Their differences are adjusted comparisons at the same practice time.
As with the two-group model, interpreting these coefficients requires a suitable linear, additive mean model and adequate data support at the predictor values being compared. A coefficient describes an adjusted association in the model; a causal interpretation needs additional design and assumptions.
Adjusted Comparisons and Raw Group Means
Raw group means can differ partly because groups have different distributions of the quantitative predictor. An additive model separates the fitted group difference from the fitted quantitative trend, provided its assumptions and data support are reasonable. The next example makes the distinction concrete.
Worked Example: Why Adjustment Can Change a Group Comparison
Consider hypothetical data from two groups with outcome \(Y\) and a quantitative predictor \(x\):
| Group | \(x\) values | Corresponding \(Y\) values |
|---|---|---|
| A (reference) | 0, 1, 2, 3 | 40, 43, 46, 49 |
| B | 1, 2, 3, 4 | 48, 51, 54, 57 |
Group A’s mean \(x\) is \((0+1+2+3)/4=1.5\), and its mean outcome is \((40+43+46+49)/4=44.5\). Group B’s mean \(x\) is \((1+2+3+4)/4=2.5\), and its mean outcome is \((48+51+54+57)/4=52.5\). The raw difference in outcome means is \(52.5-44.5=8.0\) points.
The data follow \(Y=40+3x+5D\) exactly, where \(D=1\) for Group B. For example, Group A at \(x=2\) has fitted outcome \(40+3(2)=46\); Group B at \(x=2\) has fitted outcome \(40+3(2)+5=51\). The adjusted group difference at a common \(x\) is 5 points. The remaining 3 points in the raw mean difference correspond to the one-unit difference in mean \(x\), multiplied by the 3-point-per-unit slope: \(3(2.5-1.5)=3\). Thus \(8=5+3\).
The groups overlap in \(x\) from 1 to 3, so the comparison at \(x=2\) is within their observed range. The perfect linear pattern makes the additive model exact for these illustrative values. Real data would not generally fall exactly on a line, and model assumptions and other differences between groups still matter. In particular, the adjusted difference is not automatically causal.
Common Mistakes and Practical Limits
- Calling the group coefficient an unadjusted difference. In the additive model, \(\beta_2\) compares conditional means at the same \(x\), not necessarily the groups’ raw averages.
- Giving the slope a group-specific meaning. Without an interaction term, the model uses one common quantitative slope. Do not claim that the slope differs by group.
- Misreading the intercept. The intercept is the reference group’s fitted mean at \(x=0\). If zero is outside the meaningful or observed range, it may not be practically useful.
- Treating adjustment as proof of causation. Controlling for a measured predictor does not necessarily remove confounding or other differences between groups.
- Ignoring lack of overlap. If one group is observed only at low \(x\) values and another only at high values, an adjusted comparison at a shared \(x\) may be extrapolated rather than supported by observations from both groups.
The additive model is useful because it summarizes a quantitative trend and adjusted category comparisons in one equation. Its simplicity comes with a substantive restriction: it represents each group with the same slope. Interpret its coefficients in context, check that comparisons are supported by the data, and distinguish a model-based association from a causal conclusion.
Check Your Understanding
Try these questions using the definitions and examples above.
- In the model \(\widehat{Y}=18+2.5x-4D\), where \(D=1\) identifies Group B and Group A is the reference, interpret the slope and group coefficient in context-free terms.
- Using that same model, calculate the fitted values for Groups A and B at \(x=6\). What is their fitted difference?
- Why does an additive model with a categorical indicator give the two groups the same quantitative slope?
- A three-level categorical predictor is included in a model with an intercept. How many indicators are used, and what does each non-reference coefficient compare?
- Two groups have different raw outcome means and different mean values of \(x\). Give one reason an adjusted group difference might not equal the raw difference.
- Why is a causal interpretation of an adjusted group coefficient not justified merely because another predictor appears in the regression model?