Marginal Effects for Categorical Predictors

Why Compare Predictions?

When a predictor is categorical, changing its value means moving from one category to another, not making an infinitesimally small numerical change. For example, a person may move from the reference group to a treatment group, or from one program format to another. The relevant quantity is the difference between the fitted means for those categories at the same values of the other predictors.

That difference can depend on the covariate profile. If the regression model includes an interaction between group and a continuous predictor, the fitted group difference changes as that continuous predictor changes. In that setting, an indicator coefficient by itself generally does not answer the question, “How different are the fitted outcomes for these groups at this profile?”

Definition: At a specified profile $z$ of the other predictors, the discrete change from category $g'$ to category $g$ is the difference between their fitted conditional means: $$ \Delta_{g,g'}(z)=\hat{\mu}(g,z)-\hat{\mu}(g',z). $$ It is a contrast in the outcome’s units. The profile $z$ is held fixed in both predictions.

Model and Calculation

Consider a linear regression with a binary categorical predictor $G$, coded $0$ for the reference category and $1$ for the other category, and a continuous predictor $x$. A model that allows the group difference to vary with $x$ is

E(Y_i\mid G_i,x_i)=\beta_0+\beta_1x_i+\beta_2G_i+\beta_3G_ix_i.

Here, $\beta_0+\beta_1x$ is the conditional mean for the reference category. For the category coded $1$, the conditional mean is $\beta_0+\beta_1x+\beta_2+\beta_3x$. Subtracting the first fitted mean from the second gives the group contrast at a chosen $x$:

\Delta_{1,0}(x) =(\beta_0+\beta_1x+\beta_2+\beta_3x)-(\beta_0+\beta_1x) =\beta_2+\beta_3x.

In practice, replace the unknown coefficients with their fitted estimates. The calculation is the same: predict for each category at the chosen covariate values, then subtract. If the model has other predictors, set them to the same specified values in both predictions. Terms that do not involve the categorical predictor may cancel in the subtraction, but calculating both predictions makes the comparison clear and helps catch errors.

The model’s conditional mean must represent the pattern being analyzed well enough for the fitted contrast to be meaningful. In particular, include an interaction when the model is intended to allow the categorical difference to vary with a covariate. The usual linear regression framework also assumes errors have conditional mean zero; independence and constant error variance are commonly used for conventional uncertainty calculations. Those assumptions concern the model and its statistical inference, not the arithmetic of subtracting two fitted values. Interpret a contrast cautiously when the specified covariate profile is poorly represented in the data.

Worked Example: A Binary Group Difference at a Chosen Score

Suppose a hypothetical model predicts a follow-up symptom score, measured in points, from treatment group and baseline score $x$. The fitted model is

\hat{Y}=50+2x-5G+1.5Gx,

where $G=0$ denotes the comparison group and $G=1$ denotes the treatment group. Calculate the fitted treatment-minus-comparison difference for a baseline score of $x=4$.

First predict the comparison-group mean at $x=4$:

\hat{Y}_{G=0}=50+2(4)-5(0)+1.5(0)(4)=58\text{ points}.

Then predict the treatment-group mean at the same score:

\hat{Y}_{G=1}=50+2(4)-5(1)+1.5(1)(4)=50+8-5+6=59\text{ points}.

The discrete change is $59-58=1$ point. Thus, at a baseline score of 4, the model predicts a treatment-group follow-up score 1 point higher than the comparison-group score. The indicator coefficient $-5$ is not this contrast at $x=4$: it is the model’s treatment-minus-comparison difference at $x=0$. The interaction adds $1.5(4)=6$ points to that difference, giving $-5+6=1$ point.

Comparing More Than Two Categories

A categorical predictor may have three or more levels. Choose the two categories relevant to the question and compare their fitted means. With reference-category coding, each nonreference category has its own indicator. If the model also interacts those indicators with $x$, each category’s contrast with the reference can vary with $x$.

For example, let $A$ be the reference category, with indicators $D_B$ and $D_C$ for categories $B$ and $C$. A model with category-by-$x$ interactions can be written as

\hat{Y}=\hat{\beta}_0+\hat{\beta}_1x +\hat{\beta}_B D_B+\hat{\beta}_C D_C +\hat{\beta}_{Bx}D_Bx+\hat{\beta}_{Cx}D_Cx.

At a given $x$, the fitted difference between $B$ and $A$ is $\hat{\beta}_B+\hat{\beta}_{Bx}x$, and the fitted difference between $C$ and $A$ is $\hat{\beta}_C+\hat{\beta}_{Cx}x$. To compare $B$ directly with $C$, subtract their predictions, or subtract their two contrasts with $A$. State the comparison and the profile explicitly: “$B$ minus $C$ at $x=5$” is more informative than saying only “the effect of category.”

Worked Example: Comparing Three Program Formats

In a hypothetical evaluation, a model predicts a test score $Y$, in points, from study hours $x$ and program format. Format $A$ is the reference category. The fitted equation is

\hat{Y}=20+3x-4D_B+2D_C+1.2D_Bx-0.5D_Cx,

where $D_B=1$ for format $B$, $D_C=1$ for format $C$, and both indicators are zero for format $A$. Compare formats $B$ and $C$ with format $A$ at $x=5$ study hours.

For format $A$, both indicators are zero:

\hat{Y}_A=20+3(5)=35\text{ points}.

For format $B$, $D_B=1$ and $D_C=0$:

\hat{Y}_B=20+3(5)-4+1.2(5)=20+15-4+6=37\text{ points}.

For format $C$, $D_B=0$ and $D_C=1$:

\hat{Y}_C=20+3(5)+2-0.5(5)=20+15+2-2.5=34.5\text{ points}.

At 5 study hours, the fitted $B-A$ difference is $37-35=2$ points, and the fitted $C-A$ difference is $34.5-35=-0.5$ points. The $B-C$ difference is $37-34.5=2.5$ points. These are three distinct comparisons; none should be described simply as “the program effect” without naming the comparison and the value of $x$.

Holding a Full Profile Fixed

The same prediction-and-subtraction method applies when there are additional covariates. A profile specifies values for all predictors other than the categorical level being compared. For a binary predictor, compute one fitted mean with $G=0$ and another with $G=1$, leaving every other profile value unchanged. Their difference is the fitted discrete change at that profile.

If a covariate enters the model additively and has no interaction with $G$, its contribution is identical in the two predictions and cancels. It still matters to describe the profile and to calculate each prediction correctly. If an interaction is present, the relevant covariate value can change the contrast. This is why “hold everything else fixed” is not just a phrase: it defines the comparison being calculated.

Worked Example: A Group Contrast With an Additional Covariate

Suppose a hypothetical model predicts waiting time in minutes from service plan $G$, queue load $x$, and a workload measure $z$:

\hat{Y}=12+1.1x+0.8z-3G+0.4Gx.

Here $G=0$ denotes the standard plan and $G=1$ denotes the priority plan. Calculate the priority-minus-standard fitted difference at queue load $x=5$ and workload $z=2$.

For the standard plan:

\hat{Y}_{G=0}=12+1.1(5)+0.8(2)-3(0)+0.4(0)(5) =12+5.5+1.6=19.1\text{ minutes}.

For the priority plan:

\hat{Y}_{G=1}=12+1.1(5)+0.8(2)-3(1)+0.4(1)(5) =12+5.5+1.6-3+2=18.1\text{ minutes}.

The discrete change is $18.1-19.1=-1.0$ minute. At this profile, the model predicts a waiting time 1 minute lower under the priority plan. Equivalently, subtracting the equations cancels the common terms $12+1.1(5)+0.8(2)$, leaving $-3+0.4(5)=-1.0$ minute. The workload value is held at 2 for both predictions, even though its contribution cancels in this particular model.

The interaction also shows why the specified queue load matters. At a different $x$, the contrast would be $-3+0.4x$, not necessarily $-1.0$ minute. The result is a fitted association under the stated model; it does not by itself establish that changing service plans would cause the predicted change.

Why the Calculation Matters

An indicator coefficient is tied to the model’s coding and, when interactions are present, to the profile where the interacting covariates equal zero. Predictions provide a direct route to the comparison of interest and make the profile visible. This is especially important when zero is not a meaningful or typical value of a covariate. The desired contrast can still be calculated at a meaningful value, provided the model includes the relevant terms.

A discrete change is not a derivative. For a continuous predictor, a slope describes a rate of change per unit; for a categorical predictor, the contrast compares two fitted levels. Nor is a contrast automatically an average over a population: it applies to the profile used to obtain the predictions. If the question asks about different profiles, calculate the corresponding prediction differences separately.

Common Mistake

Wrong reasoning: “The treatment indicator coefficient is $-5$, so the fitted treatment group has an outcome 5 points lower for everyone.” This ignores the interaction. In the first example, the fitted difference at $x=4$ is $-5+1.5(4)=1$, not $-5$.

Correction: Identify the two categories and the covariate profile, predict the fitted mean for each category at that same profile, and subtract in the stated order. Interpret the result as a fitted difference in outcome units, not as a causal effect unless the study design and assumptions support a causal interpretation.

Key takeaway: For a categorical predictor, a marginal effect at a specified profile is a discrete contrast between fitted means. Make the category comparison explicit, hold the other predictors fixed, include any relevant interactions, and report the difference in the outcome’s units.

Check Your Understanding

For each question, identify the comparison and profile before calculating or interpreting a contrast.

In the model $\hat{Y}=8+2x-3G+0.5Gx$, find the fitted $G=1$ minus $G=0$ difference at $x=4$.
Why is the coefficient of $G$ alone not generally the group difference at $x=4$ in a model containing $Gx$?
A predictor has categories $A$, $B$, and $C$. What two predictions would you subtract to obtain the fitted $B-C$ contrast at a specified covariate profile?
If an additional covariate enters additively, with no interaction with the category indicator, what happens to its contribution when the two fitted predictions are subtracted at the same profile?
Explain why a fitted contrast at one specified covariate profile should not automatically be described as an average group difference.