A Binary-Factor Coefficient as a Group Difference

What Does the Indicator Coefficient Measure?

A regression coefficient for a numerical predictor often describes the estimated change in the response for a one-unit increase in that predictor. For a two-level categorical predictor, the values zero and one are codes for groups, not measurements on a scale. The indicator coefficient therefore has a different, direct interpretation: it compares the mean response in the group coded one with the mean response in the group coded zero.

The zero-coded group is the reference group. The other group is the coded-one group. In the two-group regression, the indicator coefficient is the coded-one group’s fitted mean minus the reference group’s fitted mean. Its sign tells which fitted mean is larger; its size gives the difference in the response’s units.

The Two-Group Model

Suppose the response \(Y_i\) is quantitative and each observation belongs to one of two groups. Choose a coding in which the reference group has \(X_i=0\) and the other group has \(X_i=1\). The model is

Here, \(\beta_0\) is the model mean response when \(X_i=0\), and \(\beta_1\) is the difference between the model mean responses at \(X_i=1\) and \(X_i=0\). The error \(\varepsilon_i\) is the difference between an observation’s response and its model mean. For the model mean to represent each group’s mean response, assume \(E(\varepsilon_i\mid X_i)=0\). For conventional inference from sampled observations, independence and an appropriate sampling design also matter. Equal error variances or normally distributed errors are not needed for the algebraic identity between the least-squares coefficient and the observed group-mean difference.

To connect the coefficient to group means, define \(\mu_0=E(Y\mid X=0)\) and \(\mu_1=E(Y\mid X=1)\). Under the stated model,

The same difference applies to the fitted means in a sample. In a regression containing just this indicator and an intercept, least squares fits the sample mean in each group. Consequently, the estimated coefficient is exactly the coded-one group’s sample mean minus the reference group’s sample mean.

Why the Least-Squares Coefficient Is a Difference

A short derivation shows why the result holds, including when the group sizes are unequal. Let \(n_0\) and \(n_1\) be the numbers of observations in the reference and coded-one groups, and let \(\bar{Y}_0\) and \(\bar{Y}_1\) be their sample means. Both groups must have at least one observation for their means, and the coefficient contrast, to be estimated.

For a group coded zero, the model’s fitted value is \(\beta_0\). For a group coded one, it is \(\beta_0+\beta_1\). The least-squares criterion can therefore be written as

Set \(a_0=\beta_0\) and \(a_1=\beta_0+\beta_1\). These are simply the fitted constants for the two groups, and any pair \((a_0,a_1)\) corresponds to one pair of regression coefficients: \(\beta_0=a_0\) and \(\beta_1=a_1-a_0\). Thus least squares separately chooses the constant that minimizes the squared deviations in each group.

For a group with observations \(Y_1,\ldots,Y_m\), its sum of squared deviations from a constant \(a\) can be decomposed as

The first term on the right does not depend on \(a\). The second term is smallest when \(a=\bar{Y}\). Applying this result separately to the two groups gives \(\hat{a}_0=\bar{Y}_0\) and \(\hat{a}_1=\bar{Y}_1\). Therefore,

This is an exact least-squares identity for a regression with an intercept and one two-group indicator. No assumption that the group sizes are equal appears in the derivation.

Reading the Sign and Units

The subtraction order follows the coding: coded-one mean minus reference-group mean. A positive coefficient means the fitted mean is higher in the coded-one group. A negative coefficient means it is lower. A coefficient of zero means the two fitted group means are equal. The units are the units of \(Y\), because the coefficient is a difference in responses, not a change in the indicator’s measurement.

For example, if \(Y\) is measured in millimetres of mercury, the coefficient is in millimetres of mercury. If \(Y\) is a test score in points, the coefficient is in points. Do not describe the coefficient as a percentage or a percent change unless a model and outcome scale specifically support that interpretation.

Reading a Coefficient in a Regression Output

A regression table may report the coefficient without listing the two group means. Its interpretation still follows from the coding and the outcome’s units. For example, suppose a hypothetical software output reports an indicator coefficient of \(-2.7\) hours for \(X=1\) versus \(X=0\). The estimated difference is already in the order coded-one minus reference: the coded-one group’s fitted mean is 2.7 hours lower than the reference group’s fitted mean. The number is not the coded-one group’s mean on its own.

What the Identity Does—and Does Not—Say

The equality \(\hat{\beta}_1=\bar{Y}_1-\bar{Y}_0\) is a statement about least-squares estimation in a model with an intercept and one binary predictor. It does not say that every regression coefficient is a raw group-mean difference. Adding other predictors changes the model being fit, and the coefficient then describes a comparison conditional on those predictors. Here the focus is the two-group model alone.

The identity also does not depend on equal group sizes. Each group’s fitted constant is its own sample mean, whether one group has many more observations than the other or not. However, the amount of uncertainty in an estimated difference can depend on group sizes and variability. The coefficient identity gives the point estimate; it is not, by itself, an uncertainty interval or a test.

Common Mistake: Treating the Coefficient as a Group Mean

A common error is to read a coefficient of 8 points as saying that the coded-one group has an average score of 8 points. That confuses a difference with a level. The coefficient compares the groups; its value may be negative, zero, or larger than either group mean. To interpret it correctly, identify which group is coded one, subtract the reference-group mean from the coded-one mean, and report the result in the response’s units.

Another error is to turn an observed group difference into a causal claim. Coding a group with an indicator does not make the groups comparable in every other way. Unless the study design and relevant assumptions justify a causal interpretation, describe the coefficient as an estimated difference or association, not as an effect caused by group membership.