The Difference in Means as a Regression

Why a Difference in Means Can Be Written as a Regression

In a two-arm randomized study, a basic summary of the outcome is the difference between the treatment-group mean and the control-group mean. The same comparison can be expressed as a regression: code treatment assignment as 1 and control assignment as 0, then regress the outcome on that indicator. This connection is useful because it shows exactly what the regression coefficient means, while providing a framework that can later accommodate other predictors.

Here the regression has only an intercept and a binary treatment indicator. Its treatment coefficient is not a change in outcome for a one-unit increase along a continuous scale. It is the estimated change between the two groups, in the direction set by the coding.

The Two-Arm Regression Model

The model implies a separate mean for each assignment group. When \(T_i=0\), the model mean is \(\beta_0\). When \(T_i=1\), it is \(\beta_0+\beta_1\). Thus \(\beta_1\) is treatment mean minus control mean on the model scale. The sample regression coefficients \(b_0\) and \(b_1\) estimate the corresponding quantities.

Before interpreting a fitted regression, distinguish the model error from the fitted residual. The model error \(\varepsilon_i\) is defined relative to the model’s population or theoretical mean. The residual \(e_i\) is calculated after fitting the sample coefficients. They are related, but they are not the same quantity.

What Is Required, and What Is Not

For the coefficient identity proved below, both groups must be represented in the data, and ordinary least squares must fit an intercept and the 0–1 treatment indicator. The identity is an algebraic fact about the observed outcomes. It does not require equal group sizes, equal variances, normally distributed outcomes, or a causal interpretation.

Additional conditions matter for other claims. A conventional model-based standard error relies on assumptions about the errors, commonly independent errors with a common variance; exact small-sample tests may also assume normally distributed errors within groups. If observations are dependent, the uncertainty calculation must account for that dependence. The algebraic equality of the point estimate and difference in means still holds, but it does not make an inappropriate standard error appropriate.

In a randomized study, treatment assignment is randomized to support a causal comparison of assigned groups. Interpreting the observed mean difference as an estimate of a causal treatment effect also requires a suitable study setup, including a well-defined assignment and outcome and no interference between participants. Regression algebra alone does not establish those design conditions.

Deriving the Coefficients

Ordinary least squares chooses the intercept and treatment coefficient to minimize the sum of squared residuals. It is helpful to rewrite the model in terms of its two fitted group means: \(\mu_0=\beta_0\) for control and \(\mu_1=\beta_0+\beta_1\) for treatment. This is only a change of parameterization, since \(\beta_0=\mu_0\) and \(\beta_1=\mu_1-\mu_0\).

Each group mean can be optimized separately. For any group with observations \(y_1,\ldots,y_m\) and sample mean \(\bar{y}\), the following identity holds:

To verify it, write \(y_j-c=(y_j-\bar{y})+(\bar{y}-c)\), square and sum. The cross-product is zero because \(\sum_j(y_j-\bar{y})=0\). The final term, \(m(c-\bar{y})^2\), is nonnegative and is minimized when \(c=\bar{y}\). Applying this result separately to the control and treatment terms shows that the least-squares fitted control mean is the control sample mean and the fitted treatment mean is the treatment sample mean.

The notation \(\bar{Y}_0\) and \(\bar{Y}_1\) here means the sample mean outcome among observations with \(T_i=0\) and \(T_i=1\), respectively. The fitted treatment coefficient is exactly the treatment-minus-control difference in sample means.

Fitted Values and Residuals

With a binary predictor, all participants in a group receive the same fitted value: that group’s sample mean. Their residuals are their observed outcomes minus that fitted group mean. The residuals within each group sum to zero, which is a consequence of fitting a separate mean for each group.

Unequal Group Sizes Do Not Change the Result

The difference-in-means identity does not depend on a balanced design. Each group’s fitted value is still its own sample mean, so the fitted coefficient is still the treatment mean minus the control mean.

The Direction Depends on the Coding

The reference group determines the intercept, and the order of subtraction determines the sign of the treatment coefficient. With treatment coded 1 and control coded 0, the coefficient is treatment minus control. Reversing the indicator reverses the comparison. The underlying fitted group means do not change.

Why This Connection Matters

The binary-indicator regression provides a compact way to express a two-group comparison. Its intercept gives the reference group’s mean, while its treatment coefficient gives the difference from that reference group. This makes the result straightforward to report in outcome units and clarifies how changing the coding changes the coefficient’s interpretation.

The coefficient identity should not be confused with an inference result. The equality \(b_1=\bar{Y}_1-\bar{Y}_0\) tells us what the point estimate is. It does not determine a valid standard error, confidence interval, or test without considering the study design and the assumptions used for uncertainty estimates. Nor does writing the difference as a regression automatically adjust for other variables: this model contains only the intercept and treatment indicator.