Why Compare Raw and Adjusted Differences?
Two groups can have different average outcomes partly because the groups have different values of another relevant variable. For example, patients in two care groups may differ in their average baseline score. A comparison of their outcome means describes the groups as observed; a comparison at the same baseline score addresses a different question. Neither calculation is automatically a causal effect.
This tutorial compares those two quantities using a quantitative predictor \(X\) and a binary group indicator \(G\). We define \(G=0\) for the reference group and \(G=1\) for the other group. Throughout, differences are calculated as group 1 minus group 0, so a positive value means a higher outcome in group 1.
Raw and Adjusted Differences
For a basic adjusted comparison, use an additive linear regression model:
Here \(Y\) is the quantitative outcome, \(\beta_0\) is the model intercept, \(\beta_G\) is the group difference at a fixed value of \(X\), and \(\beta_X\) is the common slope relating \(X\) to the mean outcome in the two groups. The word additive means that this model has no group-by-\(X\) interaction: the slope for \(X\) is the same for both groups.
The fitted equation replaces the unknown coefficients with their estimates:
At \(X=x^*\), the fitted outcome for group 0 is \(b_0+b_Xx^*\), and for group 1 it is \(b_0+b_G+b_Xx^*\). Subtracting the first from the second gives \(b_G\). Thus, in this additive model, the adjusted difference is the same at every chosen \(x^*\); the common predictor contribution cancels.
This comparison relies on the model being a reasonable description of the mean outcome over the values being compared, including its common-slope assumption. A common value \(x^*\) is most informative when it is within the range of observed \(X\) values in both groups. If the model is used to compare groups far outside their observed ranges, the result relies on extrapolation. The calculations below describe fitted associations; the model alone does not establish that group membership causes an outcome difference.
Worked Example: A Raw Difference Larger Than the Adjusted Difference
Suppose a hypothetical program evaluation records a score \(Y\), in points, and a quantitative baseline measure \(X\), in points. Group 0 is the comparison group and group 1 is the program group. The six observations are:
| Group | \(X\) (points) | \(Y\) (points) |
|---|---|---|
| 0 | 0 | 10 |
| 0 | 2 | 14 |
| 0 | 4 | 18 |
| 1 | 2 | 19 |
| 1 | 4 | 23 |
| 1 | 6 | 27 |
Raw difference. The group 0 mean outcome is \((10+14+18)/3=14\) points. The group 1 mean is \((19+23+27)/3=23\) points. Therefore, the raw difference is \(23-14=9\) points, with group 1 higher.
Adjusted difference. These values lie exactly on the additive fitted equation \(\hat{Y}=10+5G+2X\). For example, at \(G=1\) and \(X=4\), the fitted value is \(10+5(1)+2(4)=23\) points, matching that observation. Choose \(x^*=3\) points, which lies within the observed \(X\) range of each group. For group 0, the fit is \(10+5(0)+2(3)=16\) points. For group 1, it is \(10+5(1)+2(3)=21\) points. The adjusted difference is \(21-16=5\) points.
The raw difference is 9 points, whereas the difference at the same baseline value is 5 points. Group 1's mean \(X\) is 4, compared with 2 in group 0. Because the fitted slope is 2 outcome points per predictor point, that 2-point difference in mean \(X\) accounts for \(2(4-2)=4\) points of the raw gap in this example. The remaining 5 points are the adjusted group difference.
Why the Two Calculations Differ
In an additive model, the group predictions differ by the group term when \(X\) is held fixed. But the raw means do not hold \(X\) fixed: each group is averaged over its own observed predictor values. If group 1 tends to have larger \(X\), and \(X\) is positively related to \(Y\), that difference can contribute positively to the raw group difference. With a negative slope, the contribution goes in the opposite direction.
For ordinary least squares with an intercept and the group indicator included, the model's residuals sum to zero within each group. Consequently, the sample mean outcome in each group equals the model prediction at that group's mean \(X\). This gives a useful exact relationship for the fitted additive model:
Here \(\bar{Y}_g\) and \(\bar{X}_g\) are the observed outcome and predictor means in group \(g\). The relationship says that the raw difference consists of the adjusted group difference plus the fitted contribution associated with the groups' difference in mean \(X\). It is an identity for this fitted model, not a general guarantee that the model captures every important difference between groups.
Worked Example: Calculate the Difference at a Shared Predictor Value
In a hypothetical study of study time and quiz scores, suppose an additive regression fit predicts quiz score \(Y\), in points, from group \(G\) and study time \(X\), in hours:
The observed group means are \(\bar{X}_0=10\) hours and \(\bar{X}_1=14\) hours, while the mean quiz scores are \(\bar{Y}_0=57\) points and \(\bar{Y}_1=67\) points. The raw difference is \(67-57=10\) points.
Now compare the groups at \(x^*=12\) hours. The group 0 fitted score is \(42+4(0)+1.5(12)=60\) points. The group 1 fitted score is \(42+4(1)+1.5(12)=64\) points. Their adjusted difference is \(64-60=4\) points. The same answer follows directly from the group term \(b_G=4\).
Check the calculation against the group means: the predictor mean difference is \(14-10=4\) hours, and the fitted slope contributes \(1.5(4)=6\) points. The raw difference is \(4+6=10\) points, matching the observed means. At another shared value, say 14 hours, the fitted scores would be \(42+1.5(14)=63\) and \(42+4+1.5(14)=67\), still 4 points apart. This constancy depends on the additive common-slope model.
Worked Example: Report Both Differences in Context
Consider a hypothetical comparison of two teaching formats. Let \(Y\) be a final assessment score in points and \(X\) be a baseline assessment score in points. Format 0 is the reference group; format 1 is the other group. The data are:
| Format | \(X\) (points) | \(Y\) (points) |
|---|---|---|
| 0 | 35 | 67 |
| 0 | 45 | 73 |
| 1 | 45 | 76 |
| 1 | 55 | 82 |
The raw mean for format 0 is \((67+73)/2=70\) points, and for format 1 it is \((76+82)/2=79\) points. The raw difference, format 1 minus format 0, is \(79-70=9\) points.
The observations fit \(\hat{Y}=46+3G+0.6X\). The group 0 mean baseline score is \((35+45)/2=40\); the group 1 mean is \((45+55)/2=50\). At the shared value \(x^*=45\) points, the fitted outcome for format 0 is \(46+3(0)+0.6(45)=73\) points, and for format 1 it is \(46+3(1)+0.6(45)=76\) points. The adjusted difference is \(76-73=3\) points.
The 10-point difference between the groups' mean baseline scores contributes \(0.6(50-40)=6\) points to the raw difference under this fitted model. Adding the 3-point adjusted difference gives \(3+6=9\) points, the observed raw difference. A suitable report would say: “The raw mean final score was 9 points higher in format 1. At a common baseline score of 45 points, the fitted difference was 3 points higher in format 1.” These are descriptions of the observed data and fitted association, not evidence by themselves that changing teaching format would raise scores.
Common Mistakes and Interpretation
- Calling the raw difference adjusted. Subtracting two group means compares the groups as observed, including any difference in their predictor values. To describe an adjusted comparison, calculate both fitted outcomes at the same \(X\).
- Changing the subtraction order. If the chosen direction is group 1 minus group 0, keep that order in both calculations. Reversing it changes the sign, not the size of the comparison.
- Forgetting units. A group difference in the outcome is measured in outcome units. If \(Y\) is a score in points, the difference is in points; the slope's units are outcome points per unit of \(X\).
- Assuming the adjusted difference is causal. Adjustment describes a comparison conditional on the included predictor and the specified model. It does not, on its own, rule out other differences between groups or establish cause and effect.
- Using a common value without checking support. A fitted comparison at an \(X\) value far beyond the observations in one group depends on extrapolation. Prefer a shared value represented in both groups when the data permit it.
- Assuming the adjusted difference is constant under every model. The cancellation giving a common difference holds for the additive model shown here. If the group difference varies with \(X\), a single constant adjusted difference will not describe every predictor value.
Check Your Understanding
Use group 1 minus group 0 as the direction of comparison.
- Group 1 has mean outcome 84 points and group 0 has mean outcome 79 points. What is the raw difference, and what does its sign mean?
- For \(\hat{Y}=20+6G+2X\), calculate the adjusted group difference at \(X=5\). Show both fitted outcomes.
- In an additive fit, \(b_G=3\), \(b_X=0.5\), \(\bar{X}_1=18\), and \(\bar{X}_0=10\). What raw difference does the model imply?
- Why can a raw group difference differ from the adjusted group difference when \(X\) predicts \(Y\)?
- What should you check before interpreting a fitted group comparison at a chosen common value of \(X\)?