Tutorials › Exam PA - Predictive Analytics › Worked Example With Multiple Categorical Predictors

Home  /  Tutorials  /  Exam PA / Categorical Encoding and Interactions

Exam PA · Categorical Encoding and Interactions · Tutorial 260 of 966

Worked Example With Multiple Categorical Predictors

Use R factor models to calculate and interpret fitted values for combinations of two categorical predictors, with and without an interaction.

Intermediate 10 min read

What You'll Learn

  • Set and identify reference levels for two categorical predictors in R
  • Translate model coefficients into fitted values for factor combinations
  • Compare additive and interaction-model predictions across cells
  • Interpret an interaction as a change in a category difference
  • Recognize common calculation and coding errors

Why Multiple Categorical Predictors Need Careful Interpretation

A model with two categorical predictors can describe an outcome across several groups at once. For example, a hypothetical insurer might study how claim-processing time varies by submission channel and document type. The key is to distinguish the observed groups from the model’s parameterization: coefficients describe differences relative to chosen reference levels, while predictions describe the fitted outcome for a particular combination of factor levels.

This tutorial uses ordinary least squares regression with a quantitative response. The same ideas about factor coding apply in other model families, but the interpretation of predictions would then be on the model’s link scale or response scale as appropriate. Here, all predictions are fitted mean response values in the response’s original units.

Definition: A factor is a categorical predictor with a specified set of levels. In reference-level (treatment) coding, one level of each factor is selected as the reference. The model represents other levels with indicator variables that equal 1 when the observation is at that non-reference level and 0 otherwise. With two factors, an interaction allows the difference associated with one factor to vary according to the level of the other.

Reference-Level Parameterization

Begin with two factors, \(A\) and \(B\), each having two levels. Let \(A_0\) and \(B_0\) be their reference levels, and \(A_1\) and \(B_1\) their non-reference levels. Write \(I_A=1\) for \(A_1\) and 0 for \(A_0\); define \(I_B\) similarly. The additive model and interaction model are:

$$ \text{Additive: } E[Y\mid A,B]=\beta_0+\beta_A I_A+\beta_B I_B $$
$$ \text{Interaction: } E[Y\mid A,B]=\beta_0+\beta_A I_A+\beta_B I_B+\beta_{AB}I_AI_B $$

These equations describe the conditional mean. For an ordinary least squares model, observations are represented as \(Y=E[Y\mid A,B]+\varepsilon\), with errors having conditional mean zero. The usual inferential results for linear regression additionally require suitable assumptions such as independent errors and, for conventional exact small-sample tests, normally distributed errors with constant variance. The fitted values can still be calculated without those inferential assumptions, but their reliability depends on the data and model being appropriate.

The subscripts in a cell prediction indicate whether each factor is at its non-reference level: the first index indicates whether \(A\) is at its non-reference level and the second indicates whether \(B\) is at its non-reference level. Thus:

$$ \begin{aligned} \hat{y}_{00}&=\hat{\beta}_0,\\ \hat{y}_{10}&=\hat{\beta}_0+\hat{\beta}_A,\\ \hat{y}_{01}&=\hat{\beta}_0+\hat{\beta}_B,\\ \hat{y}_{11}&=\hat{\beta}_0+\hat{\beta}_A+\hat{\beta}_B+\hat{\beta}_{AB}. \end{aligned} $$

In the additive model, \(\hat{\beta}_{AB}=0\), so the predicted difference between \(A_1\) and \(A_0\) is the same at both levels of \(B\). With an interaction, the difference at \(B_0\) is \(\hat{\beta}_A\), while the difference at \(B_1\) is \(\hat{\beta}_A+\hat{\beta}_{AB}\). The interaction coefficient is therefore a difference between differences.

Why the interaction is a difference between differences: Subtract the prediction for \(A_0,B_0\) from the prediction for \(A_1,B_0\). The result is \(\hat{\beta}_A\). At \(B_1\), subtract the prediction for \(A_0,B_1\) from the prediction for \(A_1,B_1\); the result is \(\hat{\beta}_A+\hat{\beta}_{AB}\). Subtract the first difference from the second and the main effect cancels, leaving \(\hat{\beta}_{AB}\). Equivalently, \(\hat{y}_{11}-\hat{y}_{01}-\hat{y}_{10}+\hat{y}_{00}=\hat{\beta}_{AB}\).

In R, the formula `A * B` includes both main effects and their interaction; it is shorthand for `A + B + A:B`. The formula `A + B` includes only additive effects. R uses factor levels and contrasts to construct model columns. To make coefficient interpretation transparent, set the factor levels deliberately, then inspect the fitted model and predictions rather than assuming the reference level.

Worked Predictions for Two Factors

Worked Example: Additive and Interaction Predictions for Processing Time

A hypothetical operations team records processing days for claims by submission channel (Standard or Express) and document type (Paper or Digital). The three observed outcomes in each cell are: Standard-Paper, 10, 12, 14; Express-Paper, 15, 17, 19; Standard-Digital, 13, 15, 17; Express-Digital, 24, 26, 28. Use Standard and Paper as reference levels. Calculate the interaction-model predictions, the additive-model predictions, and the difference between the two models’ predictions in each cell.

Method. First calculate each cell mean. The means are \(12\), \(17\), \(15\), and \(26\), respectively. With the interaction model, treatment coding allows a separate fitted mean for each of the four combinations:

$$ \hat{y}_{00}=12,\quad \hat{y}_{10}=17,\quad \hat{y}_{01}=15,\quad \hat{y}_{11}=26. $$

The reference-level coefficients follow directly. The intercept is 12. The Express coefficient is \(17-12=5\), and the Digital coefficient is \(15-12=3\). The interaction coefficient is \(26-12-5-3=6\). Therefore the fitted interaction model is

$$ \widehat{\text{days}}=12+5I_{\text{Express}}+3I_{\text{Digital}}+6I_{\text{Express}}I_{\text{Digital}}. $$

For the additive model, the design is balanced, with the same number of observations in every cell. Its fitted cell predictions can be calculated as row mean plus column mean minus the overall mean. The Standard mean is \((12+15)/2=13.5\), and the Express mean is \((17+26)/2=21.5\). The Paper mean is \((12+17)/2=14.5\), the Digital mean is \((15+26)/2=20.5\), and the overall mean is \((12+17+15+26)/4=17.5\). Thus:

$$ \begin{aligned} \hat{y}_{00}^{\text{add}}&=13.5+14.5-17.5=10.5,\\ \hat{y}_{10}^{\text{add}}&=21.5+14.5-17.5=18.5,\\ \hat{y}_{01}^{\text{add}}&=13.5+20.5-17.5=16.5,\\ \hat{y}_{11}^{\text{add}}&=21.5+20.5-17.5=24.5. \end{aligned} $$

The additive predictions have the same Express-versus-Standard difference at both document types: \(18.5-10.5=8\) days and \(24.5-16.5=8\) days. The interaction-model differences are \(17-12=5\) days for Paper and \(26-15=11\) days for Digital, a difference between differences of \(11-5=6\) days. This agrees with the interaction coefficient. The comparison also checks the calculations: additive predictions reproduce the row and column means, while the interaction model reproduces all four observed cell means. A common mistake is to assume that the additive intercept must equal the reference cell mean; here it is 10.5, not 12, because the additive model compromises across cells.

Reading Coefficients and Predictions in R

For this model, R code can explicitly set the intended reference levels and fit both specifications:

$$ \texttt{claims\$channel <- factor(claims\$channel, levels=c("Standard","Express"))} $$
$$ \texttt{claims\$document <- factor(claims\$document, levels=c("Paper","Digital"))} $$

Then `lm(days ~ channel + document, data=claims)` fits the additive model, while `lm(days ~ channel * document, data=claims)` includes the interaction. In R’s default treatment coding, the first listed factor level is the reference. A call such as `predict(fit, newdata=...)` calculates fitted values for the specified combinations; the `newdata` columns must use the same variable names and compatible factor levels as the model data.

A displayed coefficient is not itself a prediction for every group. The intercept is the fitted value for the reference combination, and a main-effect coefficient is a change from that reference when the other factor is also at reference. In an interaction model, a main effect is not an across-the-board difference: it is the difference at the other factor’s reference level. To obtain another cell’s fitted value, add every coefficient whose indicator is 1 for that cell.

Worked Example: Interpreting a Fitted Two-Factor Model

Suppose a hypothetical retention model predicts annual account value in dollars using plan (Basic or Premium) and service region (Urban or Rural). Basic and Urban are reference levels. The fitted interaction-model coefficients are: intercept 40, Premium 9, Rural \(-4\), and Premium:Rural 7. Calculate the four fitted values and the Premium-versus-Basic difference in each region.

Method. Use an indicator equal to 1 for Premium and another equal to 1 for Rural. The fitted mean is \(40+9I_{\text{Premium}}-4I_{\text{Rural}}+7I_{\text{Premium}}I_{\text{Rural}}\).

For Basic-Urban, both indicators are 0, so the prediction is \(40\). For Premium-Urban, only the Premium indicator is 1, giving \(40+9=49\). For Basic-Rural, only the Rural indicator is 1, giving \(40-4=36\). For Premium-Rural, both indicators are 1, giving \(40+9-4+7=52\).

The Premium-versus-Basic difference in Urban is \(49-40=9\). In Rural it is \(52-36=16\), which also equals the Premium coefficient plus the interaction coefficient, \(9+7=16\). The interaction coefficient is \(16-9=7\), the change in the plan difference across regions. The calculation avoids the common error of reading the Premium coefficient, 9, as the Premium difference in every region.

More Than Two Levels in a Factor

The same logic extends when a factor has more than two levels. With three levels of \(A\), reference coding uses one indicator for each non-reference level. If \(B\) has two levels, the interaction model includes an interaction coefficient for each non-reference \(A\)-level paired with the non-reference \(B\)-level. Each cell prediction is the intercept plus the main-effect and interaction coefficients whose indicators are active. The number of coefficients changes, but the reference-cell principle does not.

Worked Example: Three Plans Across Two Distribution Channels

A hypothetical pricing analyst models a quantitative annual service cost by plan (Basic, Silver, Gold) and distribution channel (Direct, Broker). Basic and Direct are the reference levels. The fitted coefficients are: intercept 100, Silver 12, Gold 30, Broker 8, Silver:Broker \(-3\), and Gold:Broker 5. Calculate all six fitted cell values, then compare the additive model’s predictions, which omit the two interaction terms.

Method. The interaction-model prediction is \(100+12I_{\text{Silver}}+30I_{\text{Gold}}+8I_{\text{Broker}}-3I_{\text{Silver}}I_{\text{Broker}}+5I_{\text{Gold}}I_{\text{Broker}}\). For Direct, the Broker indicator is zero: Basic is 100, Silver is \(100+12=112\), and Gold is \(100+30=130\). For Broker, Basic is \(100+8=108\); Silver is \(100+12+8-3=117\); Gold is \(100+30+8+5=143\).

The additive model retains the intercept and main effects but sets both interaction contributions to zero. Its Direct predictions remain 100, 112, and 130. Its Broker predictions are Basic \(100+8=108\), Silver \(100+12+8=120\), and Gold \(100+30+8=138\). Therefore, the interaction model changes the Silver-Broker prediction by \(-3\) and the Gold-Broker prediction by \(+5\), relative to the additive model. These are model-based fitted values, not evidence by themselves that plan or channel causes a change in cost.

Common Calculation Traps

Rounding intermediate values can change a reported contrast even when each individual cell prediction looks reasonable. For example, suppose the fitted means across three hypothetical channels are 11, 13, and 17 for Basic, and 15, 19, and 21 for Plus. The average Plus mean is \(55/3\), and the average Basic mean is \(41/3\). Their exact difference is:

$$ \frac{55}{3}-\frac{41}{3}=\frac{14}{3}=4.666\ldots\approx4.667 $$

A mistaken calculation rounds the averages first to 18.333 and 13.667, then subtracts to get 4.666. The correction is to retain exact values (or sufficient precision) until the final rounding. This matters whenever cell summaries are combined into a contrast.

  • Mixing up reference and non-reference indicators. Under the definitions used here, an indicator is 1 at the non-reference level. Check each cell against that definition before adding coefficients.
  • Interpreting a main effect as universal. In an interaction model, a main-effect coefficient gives a difference when the other predictor is at its reference level.
  • Adding an interaction to the wrong cells. An interaction term contributes only when both of its indicators equal 1.
  • Assuming an additive model matches every observed cell mean. It generally does not; it estimates a compromise that respects the additive structure.
  • Forgetting the factor reference in R. Inspect factor levels or set their order explicitly before interpreting coefficients.
Key takeaway: Identify the reference levels, translate each factor combination into its active indicators, and add exactly those coefficients to obtain a fitted value. In an interaction model, the interaction coefficient measures how a category difference changes across the other factor’s levels.

Check Your Understanding

For each problem, calculate the requested values from the stated reference coding. Retain exact values until rounding is needed.

  1. A model has factors Route (Local reference, Regional non-reference) and Package (Standard reference, Priority non-reference). Its coefficients are intercept 22, Regional 6, Priority \(-3\), and Regional:Priority 4. What are the four fitted values? (Answer: Local-Standard 22; Regional-Standard 28; Local-Priority 19; Regional-Priority 29.)
  2. In the same model, what is the Regional-versus-Local difference for Standard and for Priority, and what does the interaction coefficient represent? (Answer: Standard difference 6; Priority difference 10; the difference between those differences is 4.)
  3. A balanced two-by-two table has cell means 8 for A0B0, 14 for A1B0, 10 for A0B1, and 20 for A1B1. Calculate the additive-model prediction for A1B1 and the interaction-model difference between differences. (Answer: Additive prediction 18; interaction difference between differences 4.)
  4. A model has factor Tier (Entry reference, Plus, Select) and Source (Web reference, Agent). Coefficients are intercept 50, Plus 5, Select 11, Agent 4, Plus:Agent \(-2\), and Select:Agent 3. What are the Plus-Agent and Select-Agent predictions? (Answer: Plus-Agent 57; Select-Agent 68.)
  5. For problem 4, what predictions would the additive model give for Plus-Agent and Select-Agent, and how do they differ from the interaction predictions? (Answer: Additive predictions 59 and 65; the interaction predictions are 2 lower for Plus-Agent and 3 higher for Select-Agent.)

Previous

Next

Back to Tutorials
Clinical Biostats