Binary Regressors and Group-Mean Differences

Why a Binary Regressor Is a Group Comparison

A regression coefficient can be easier to interpret when a regressor marks group membership. If $D_i$ is zero for one group and one for another, a regression with an intercept and $D_i$ alone does not merely summarize an association between a numeric variable and an outcome. Its fitted values are the two groups’ sample means, and its coefficient is their difference. This exact result makes the regression’s reference group and units especially transparent.

Adding other regressors changes the comparison. The coefficient on $D_i$ is then an adjusted contrast in a linear projection: it compares fitted outcomes for the two values of $D_i$ while holding the included regressors fixed. It is generally not the raw difference between the groups’ observed outcome means, and by itself it does not establish a causal effect.

Definition: A binary regressor $D_i$ takes only the values zero and one. The group with $D_i=0$ is the reference group; the group with $D_i=1$ is the indicator group. In a regression with an intercept and no other regressors, the coefficient on $D_i$ is the difference between the indicator group’s sample mean outcome and the reference group’s sample mean outcome, provided both groups are represented.

The Two-Group Regression

For observations $i=1,\ldots,n$, consider the ordinary least squares regression

Y_i=\beta_0+\beta_1D_i+u_i,

where $Y_i$ is the outcome and $D_i\in\{0,1\}$. Let $n_0$ and $n_1$ be the numbers of observations with $D_i=0$ and $D_i=1$, respectively, and assume $n_0>0$ and $n_1>0$. Write their sample mean outcomes as $\bar{Y}_0$ and $\bar{Y}_1$. These are group-specific means, not additional regressors.

The fitted value is $\hat{\beta}_0$ for every observation in the reference group and $\hat{\beta}_0+\hat{\beta}_1$ for every observation in the indicator group. OLS chooses these two fitted values to minimize the total squared residuals. The derivation below shows why the minimizing values are precisely the two group means.

Theorem: In an OLS regression with an intercept and one binary regressor, with at least one observation in each group, $$ \hat{\beta}_0=\bar{Y}_0,\qquad \hat{\beta}_1=\bar{Y}_1-\bar{Y}_0. $$ Thus the fitted value for each group is that group’s sample mean.

Derivation: OLS Fits Each Group Mean

For any proposed intercept $a$ and coefficient $b$, the sum of squared residuals can be separated into the contributions from the two groups:

Q(a,b)=\sum_{i:D_i=0}(Y_i-a)^2 +\sum_{i:D_i=1}(Y_i-a-b)^2.

At an OLS minimum, the derivative with respect to $b$ must be zero. Only the second group’s fitted value depends on $b$, so this condition says that the residuals in the indicator group sum to zero:

\frac{\partial Q}{\partial b} =-2\sum_{i:D_i=1}(Y_i-a-b)=0 \quad\Longrightarrow\quad a+b=\bar{Y}_1.

The derivative with respect to $a$ sets the sum of residuals across both groups to zero. Since the indicator-group residuals already sum to zero, the reference-group residuals must also sum to zero:

\frac{\partial Q}{\partial a} =-2\sum_{i:D_i=0}(Y_i-a) -2\sum_{i:D_i=1}(Y_i-a-b)=0 \quad\Longrightarrow\quad a=\bar{Y}_0.

Combining the two conditions gives $b=\bar{Y}_1-\bar{Y}_0$. The result is an algebraic property of this least-squares specification; it does not require a distributional assumption about the outcomes or errors.

Worked Example: Recovering Two Group Means

Suppose a hypothetical sample records an outcome in points for two groups. The reference group ($D=0$) has outcomes 4, 6, and 8. The indicator group ($D=1$) has outcomes 9 and 11. Calculate the OLS intercept and coefficient from a regression of the outcome on an intercept and $D$.

The group sizes are $n_0=3$ and $n_1=2$. The sample means are

\bar{Y}_0=\frac{4+6+8}{3}=6,\qquad \bar{Y}_1=\frac{9+11}{2}=10.

Both groups are represented, so the two-group result applies. The intercept is the reference group mean, $ \hat{\beta}_0=6 $ points, and the indicator coefficient is

\hat{\beta}_1=\bar{Y}_1-\bar{Y}_0=10-6=4\text{ points}.

The fitted value is 6 points for every $D=0$ observation and $6+4=10$ points for every $D=1$ observation. The coefficient describes a four-point difference in sample means; it does not, on its own, say that group membership caused that difference.

Reference Group and Indicator Coding

The intercept depends on which group is coded zero. The coefficient’s sign depends on the order of the group comparison. Changing the coding does not change the underlying fitted group means, but it changes which mean is the intercept and which difference the slope reports.

Worked Example: Reversing the Indicator

In a hypothetical assessment, the mean score is 22 points in a comparison group and 26 points in an alternative-coaching group. Let $D=1$ mark alternative coaching and $D=0$ mark the comparison group. What does the coefficient report, and what changes if the indicator is reversed?

With the stated coding, the intercept is the comparison-group mean, 22 points, and the coefficient is

\hat{\beta}_1=26-22=4\text{ points}.

Now define $R=1-D$, so that $R=1$ marks the comparison group. The regression’s fitted value can be written in either coding:

\hat{\beta}_0+\hat{\beta}_1D =\hat{\beta}_0+\hat{\beta}_1(1-R) =(\hat{\beta}_0+\hat{\beta}_1)-\hat{\beta}_1R.

Consequently, the intercept in the reversed coding is 26 points, the mean for the group now coded zero, and the coefficient on $R$ is $22-26=-4$ points. Fitted values remain 22 and 26 points. The sign change reflects the reversed comparison, not a change in the data.

Adding Regressors: An Adjusted Linear-Projection Contrast

Now include controls $X_i$, collected in a vector, alongside the intercept and binary regressor. The sample regression is

Y_i=\beta_0+\beta_DD_i+X_i'\gamma+u_i.

In this additive specification, the difference between the two fitted values at the same value of $X_i$ is $\hat{\beta}_D$. This is an adjusted contrast because it compares the model’s fitted values while holding the included controls fixed. It is a linear-projection statement: OLS chooses coefficients to minimize sample squared prediction errors. The specification need not equal the true conditional mean for the coefficient to be defined.

The Frisch–Waugh–Lovell theorem makes the adjustment precise. Let $Z$ be the matrix containing the intercept and all included controls, and suppose the relevant regressions are identified. Residualize $D$ and $Y$ by regressing each on $Z$ and retaining the residuals. Denote these residual vectors by $\tilde{D}$ and $\tilde{Y}$. Then the coefficient on $D$ in the full regression is the slope from regressing $\tilde{Y}$ on $\tilde{D}$ with no additional intercept:

\hat{\beta}_D =\frac{\tilde{D}'\tilde{Y}}{\tilde{D}'\tilde{D}}.

In words, this coefficient relates the part of the binary indicator not linearly explained by the controls to the part of the outcome not linearly explained by the controls. If the controls perfectly explain $D$, then $\tilde{D}'\tilde{D}=0$: there is no residual variation in the indicator with which to estimate a separate coefficient.

With no controls beyond the intercept, residualizing both variables gives the ordinary two-group mean difference. With additional controls, the residualized variables generally differ from the original group labels and outcomes. Therefore, the adjusted coefficient generally does not equal the raw difference in group means.

Worked Example: A Raw Difference and an Adjusted Contrast

Consider these hypothetical observations. $D$ indicates membership in group 1, $X$ is a control, and $Y$ is an outcome in units of $Y$.

Observation	$D$	$X$	$Y$
1	0	0	10
2	0	1	12
3	1	1	17
4	1	2	20

The raw group means are $\bar{Y}_0=(10+12)/2=11$ and $\bar{Y}_1=(17+20)/2=18.5$. Thus the unadjusted mean difference is $18.5-11=7.5$ outcome units. To find the coefficient in the regression with an intercept, $X$, and $D$, apply the residualized-variable formula.

First regress $D$ on an intercept and $X$. The fitted values are $0, 0.5, 0.5, 1$, so the residualized indicator is $\tilde{D}=(0,-0.5,0.5,0)'$. Next regress $Y$ on an intercept and $X$. The fitted line is $9.75+5X$, giving fitted values $9.75,14.75,14.75,19.75$ and residuals $\tilde{Y}=(0.25,-2.75,2.25,0.25)'$. Therefore,

\hat{\beta}_D =\frac{0(0.25)+(-0.5)(-2.75)+(0.5)(2.25)+0(0.25)} {0^2+(-0.5)^2+(0.5)^2+0^2} =\frac{2.5}{0.5}=5.

The adjusted coefficient is 5 outcome units, not the raw 7.5-unit difference. In the fitted additive model, the predicted outcome for $D=1$ is 5 units higher than for $D=0$ at the same $X$. For example, the fitted regression is $ \hat{Y}=9.75+2.5X+5D $. This is a model-based adjusted association, not evidence by itself that changing group membership would change the outcome by 5 units.

What Adjustment Does—and Does Not—Mean

Including controls changes the contrast by asking the regression to account for linear relationships between the outcome, the indicator, and those controls. If group membership is related to a control that also predicts the outcome, the raw group means can differ partly because the groups have different control values. The adjusted coefficient instead uses the remaining linear variation in the indicator after accounting for the included controls.

This description does not guarantee that the groups are comparable in every relevant way. The coefficient depends on which controls and functional forms are included, and it can be sensitive to limited overlap in the groups’ control values. In particular, an additive regression imposes a common difference between $D=0$ and $D=1$ across values of the controls. If the actual fitted relationship has different group differences at different control values, a single coefficient does not report all those differences.

A causal interpretation requires additional assumptions and a design that supports causal inference. The algebraic identity for the unadjusted coefficient and the residualization formula for the adjusted coefficient are properties of OLS; neither identity alone establishes causation.

Common Mistakes

Reversing the group order. Saying the coefficient is “the reference mean minus the indicator mean” gives the wrong sign. With $D=1$ for the indicator group, the coefficient is $\bar{Y}_1-\bar{Y}_0$.
Calling the intercept the overall mean. In the two-group regression, the intercept is the mean for the group coded zero, not generally the overall sample mean.
Assuming controls leave the raw difference unchanged. Once regressors are added, the coefficient is a partialled-out linear-projection contrast; it generally differs from the raw group-mean difference.
Interpreting an adjusted association as a causal effect. “Adjusted for these controls” describes the regression specification. It does not by itself show that confounding has been removed or that group membership caused the outcome difference.

Key takeaway: With an intercept and a single binary regressor, the OLS coefficient is exactly the difference between the two sample group means, in the outcome’s units. With additional regressors, it becomes an adjusted linear-projection contrast defined by residual variation in the indicator and outcome.

Check Your Understanding

Try these questions before reviewing the derivations.

A regression uses an intercept and a binary indicator. The reference-group mean is 12 and the indicator-group mean is 17. What are the intercept and indicator coefficient, and what are their units?
What changes in the intercept and coefficient if the indicator’s zero and one labels are reversed? What does not change?
In a regression with controls, what are $\tilde{D}$ and $\tilde{Y}$ in the Frisch–Waugh–Lovell expression for the indicator coefficient?
Why can the coefficient from a regression with controls differ from the raw difference in group means?
Why does an adjusted coefficient not, by itself, establish a causal effect?

Observation	\(D\)	\(X\)	\(Y\)
1	0	0	10
2	0	1	12
3	1	1	17
4	1	2	20