Comparing Raw and Adjusted Treatment Estimates

Why Two Treatment Estimates Can Differ

In a randomized study, a straightforward estimate of the treatment contrast is the difference between the two groups’ observed outcome means. A second estimate comes from a regression that adjusts for a baseline outcome. Randomization makes treatment assignment independent of baseline characteristics in the assignment process, but it does not force the treatment and control groups to have exactly the same baseline mean in every realized sample. The two estimates can therefore differ even when the study was properly randomized.

The difference is often systematic rather than mysterious: the ANCOVA estimate accounts for the observed baseline mean difference, using the fitted relationship between baseline and follow-up outcomes. The adjustment may move the estimate up or down, depending on which group started higher and whether higher baseline values tend to predict higher follow-up values.

Setup and model: Let $T_i=1$ for a participant assigned to treatment and $T_i=0$ for a participant assigned to control. Let $X_i$ be the participant’s baseline outcome and $Y_i$ the follow-up outcome. The common-slope ANCOVA model is $$ Y_i=\beta_0+\beta_1T_i+\beta_2X_i+\varepsilon_i. $$ The model uses one baseline slope for both groups and includes no treatment-by-baseline interaction. For its usual mean interpretation, assume the conditional mean is described by this model, so $E(\varepsilon_i\mid T_i,X_i)=0$. The baseline measure must be recorded before treatment assignment. Randomization supports a causal comparison of assigned groups; it does not itself guarantee exact baseline balance in the realized sample.

Write $\bar{Y}_1$ and $\bar{Y}_0$ for the observed follow-up means in the treatment and control groups, and $\bar{X}_1$ and $\bar{X}_0$ for their baseline means. The raw, or unadjusted, estimate is the observed treatment mean minus the observed control mean. The adjusted estimate is the fitted coefficient of $T_i$ in the ANCOVA model.

Definition: The raw difference in means is $D_{\text{raw}}=\bar{Y}_1-\bar{Y}_0$. The ANCOVA treatment estimate is $\hat{\beta}_1$, the fitted coefficient for treatment in the common-slope model. It compares the fitted follow-up means for treatment and control at the same baseline value. Under the common-slope model, that fitted contrast is the same at every baseline value.

The Exact Relationship Between the Estimates

In a finite sample, the following identity connects the two estimates:

\hat{\beta}_1 = (\bar{Y}_1-\bar{Y}_0) - \hat{\beta}_2(\bar{X}_1-\bar{X}_0) = D_{\text{raw}}-\hat{\beta}_2(\bar{X}_1-\bar{X}_0).

Here $\hat{\beta}_2$ is the fitted common baseline slope. The identity says that ANCOVA subtracts from the raw difference the part associated with the observed baseline mean difference, as estimated by the baseline slope. It is an exact algebraic result for this fitted model—not an approximation that holds only in large samples.

A Short Derivation

The least-squares fit includes an intercept and a treatment indicator. Consequently, the residuals sum to zero within each treatment group: the fitted values average to the observed outcome mean in each group. Averaging the fitted model separately for the two groups gives

\bar{Y}_1=\hat{\beta}_0+\hat{\beta}_1+\hat{\beta}_2\bar{X}_1, \qquad \bar{Y}_0=\hat{\beta}_0+\hat{\beta}_2\bar{X}_0.

Subtract the control-group equation from the treatment-group equation. The intercept cancels, leaving

\bar{Y}_1-\bar{Y}_0 = \hat{\beta}_1+\hat{\beta}_2(\bar{X}_1-\bar{X}_0).

Rearranging yields the identity above. This also clarifies when the estimates coincide: if the groups have the same observed baseline mean, then $\bar{X}_1-\bar{X}_0=0$, and $\hat{\beta}_1=D_{\text{raw}}$. They also coincide if the fitted baseline slope is zero. Otherwise, their difference depends on the product of the fitted slope and the baseline mean difference.

Worked Example: Higher Baseline Mean in the Treatment Group

Suppose a hypothetical randomized study has two participants in each group. Their baseline and follow-up outcomes, measured in points, are:

Group	Baseline $X$	Follow-up $Y$
Control	0, 2	10, 14
Treatment	4, 6	20, 24

The group means are $\bar{X}_0=1$, $\bar{X}_1=5$, $\bar{Y}_0=12$, and $\bar{Y}_1=22$. Therefore, the raw difference is

D_{\text{raw}}=22-12=10\text{ points}.

The fitted baseline slope is $\hat{\beta}_2=2$ points of follow-up outcome per baseline point. In these data, both groups’ follow-up outcomes rise by 4 points when baseline rises by 2 points, so the common-slope line has slope $4/2=2$. Using the relationship between estimates,

\hat{\beta}_1 =10-2(5-1) =10-8 =2\text{ points}.

The treatment group began 4 baseline points higher, and the fitted slope associates each baseline point with 2 follow-up points. The adjusted contrast is thus 8 points smaller than the raw contrast. In this hypothetical sample, the estimated common treatment contrast is 2 follow-up points after adjustment, compared with an observed raw difference of 10 points.

How the Direction of Adjustment Depends on the Data

The sign of the adjustment matters. If higher baseline values predict higher follow-up values, then $\hat{\beta}_2$ is positive. When the treatment group also has a higher baseline mean, the product $\hat{\beta}_2(\bar{X}_1-\bar{X}_0)$ is positive, so subtracting it makes the adjusted estimate smaller than the raw estimate. If the treatment group has a lower baseline mean instead, the product is negative, and subtracting it makes the adjusted estimate larger.

The sign need not always be positive: the baseline-outcome relationship could be negative in a particular setting. The reliable approach is to retain the signs in the formula, rather than memorizing that adjustment always increases or always decreases a treatment estimate.

Worked Example: Lower Baseline Mean in the Treatment Group

Consider another hypothetical study with the following data, measured in points:

Group	Baseline $X$	Follow-up $Y$
Control	4, 6	18.5, 21.5
Treatment	2, 4	22.5, 25.5

The baseline means are $\bar{X}_0=5$ and $\bar{X}_1=3$. The follow-up means are $\bar{Y}_0=20$ and $\bar{Y}_1=24$, so

D_{\text{raw}}=24-20=4\text{ points}.

Within each group, follow-up outcomes increase by 3 points across a 2-point baseline increase. Thus the fitted common baseline slope is $\hat{\beta}_2=3/2=1.5$. The adjusted estimate is

\hat{\beta}_1 =4-1.5(3-5) =4-1.5(-2) =4+3 =7\text{ points}.

The baseline mean is 2 points lower in the treatment group. With a positive fitted baseline slope, that imbalance pulls the raw difference downward relative to the adjusted contrast. The adjustment therefore raises the estimate from 4 to 7 follow-up points.

Interpreting Both Estimates

The raw difference describes the observed difference between the two randomized groups’ follow-up means, without using baseline outcomes in the calculation. Under random assignment, it is a natural estimate of the average treatment contrast for the study population. The ANCOVA coefficient describes the fitted treatment-versus-control contrast at a common baseline value, under the common-slope model. When the treatment effect is constant across baseline values and the model’s mean structure is appropriate, both estimates target that same treatment contrast, though they can differ in a particular sample.

Randomization is important for causal interpretation: it makes assignment independent of participants’ potential outcomes before assignment, in expectation over the randomization. But a particular allocation can leave one group with a higher baseline mean by chance. Such a difference is not, by itself, evidence that randomization failed or that treatment assignment caused the baseline difference. ANCOVA uses the observed baseline-outcome relationship to account for the part of the follow-up difference associated with that sample imbalance.

Worked Example: Equal Baseline Means

In a third hypothetical study, both groups have baseline values 4 and 6. The control follow-up values are 12 and 16, and the treatment follow-up values are 15 and 19. Each group has a baseline mean of 5. The follow-up means are 14 and 17, respectively, so

D_{\text{raw}}=17-14=3\text{ points}.

The fitted common baseline slope is $\hat{\beta}_2=2$, since a 2-point increase in baseline corresponds to a 4-point increase in follow-up within either group. The baseline mean difference is zero, so

\hat{\beta}_1 =3-2(5-5) =3\text{ points}.

The two estimates are identical here, even though the baseline slope is not zero: the groups have no baseline mean difference for the fitted slope to adjust.

What a Difference Does—and Does Not—Tell You

A gap between the raw and adjusted estimates tells you that the baseline mean difference and fitted baseline slope combine to shift the treatment estimate in this sample. It does not, by itself, show that either estimate is erroneous. Nor does the size of the gap alone determine which estimate is more precise, or whether the treatment effect is statistically distinguishable from zero. Those questions require appropriate uncertainty estimates and inference, not just comparison of point estimates.

The comparison also depends on the specified model. The identity above applies to the common-slope ANCOVA model with an intercept and a binary treatment indicator. If the treatment contrast varies with baseline, a single common treatment coefficient does not describe the contrast at every baseline value. In that case, the model and interpretation need to account for that variation rather than treating one coefficient as a universal contrast.

Common Mistake

Wrong reasoning: “The study was randomized, so the baseline means must be equal. Therefore, if raw and adjusted estimates differ, the calculation or randomization must be wrong.”
Correction: Randomization balances baseline characteristics in expectation across assignments; it does not enforce exact equality in each realized sample. Check the actual baseline mean difference and fitted baseline slope, then use $\hat{\beta}_1=D_{\text{raw}}-\hat{\beta}_2(\bar{X}_1-\bar{X}_0)$ to understand the change.
Wrong reasoning: “The adjusted estimate is automatically the true treatment effect because it uses more information.”
Correction: Adjustment gives a model-based estimate. Its interpretation depends on the common-slope model and its assumptions; neither the adjustment nor the numerical difference alone proves that an estimate is correct.

Key takeaway: In a common-slope ANCOVA, the adjusted treatment estimate equals the raw difference in follow-up means minus the fitted baseline slope multiplied by the observed baseline mean difference. This exact relationship explains how the estimates can differ in a finite randomized sample while keeping their interpretations distinct.

Check Your Understanding

Use the relationship between the estimates to answer each question.

A study has a raw treatment-minus-control difference of 6 points, a fitted baseline slope of 0.8 follow-up points per baseline point, and a treatment-minus-control baseline mean difference of 2 points. What is the adjusted estimate?
If the treatment group has a lower baseline mean and the fitted baseline slope is positive, will the adjusted estimate be larger or smaller than the raw difference? Explain using the signs.
Can the raw and adjusted estimates differ if the groups have identical observed baseline means? Explain.
In words, what does the ANCOVA treatment coefficient compare under the common-slope model?
Why does a difference between the two estimates not, by itself, establish that the randomization was flawed?