Why Compare Adjustment Sets?
A regression coefficient for an exposure can change when covariates are added to the model. That change may be informative: it can show how the exposure-outcome association depends on which variables are held fixed. But a coefficient moving closer to zero, or changing less than expected, does not by itself tell us which model is correct. Covariates must be chosen for a substantive reason, not because a particular set produces a preferred estimate.
This tutorial compares several models for the same exposure and outcome. The comparisons are most useful when they use the same observations, exposure and outcome definitions, units, and functional form. Otherwise, a changed estimate may reflect more than a changed adjustment set.
Models and Conditions for a Fair Comparison
Let \(Y_i\) be an outcome, \(X_i\) an exposure, and \(Z_{ij}\) the \(j\)th covariate for observation \(i\). A linear regression model with adjustment set \(S\) can be written as
The fitted exposure coefficient, \(\hat{\beta}_X\), is the estimated change in the model's mean outcome for a one-unit increase in exposure while the included covariates are held fixed. This interpretation relies on the model's linear form: in particular, it assumes a constant exposure slope over the range being modeled and no exposure-by-covariate interaction unless one is included. An exposure coefficient's units are outcome units per exposure unit.
Ordinary least squares (OLS) selects coefficients to minimize the sum of squared residuals. For the coefficient comparison to have a clear meaning, the models should use the same complete observations and consistent coding. The included predictors must not be exact linear combinations of one another, so that their coefficients can be distinguished. A correctly specified conditional mean is also important for interpreting the fitted coefficients as describing that mean. These conditions do not ensure a causal interpretation.
A causal interpretation needs additional support from the study design and subject-matter knowledge. For example, the adjustment set must address relevant common causes without conditioning on variables that would introduce bias, and the exposure must precede the outcome. Earlier material on the backdoor criterion and candidate adjustment sets provides a framework for that causal reasoning. Here, the estimates are compared as adjusted associations; their movement alone cannot verify causal assumptions.
A Small Hypothetical Data Set
Suppose a hypothetical group of eight participants has a weekly training exposure \(X\), a task-score outcome \(Y\), a pre-training skill score \(A\), and a prior-practice score \(B\). Both \(A\) and \(B\) are recorded before the training exposure. They are plausible candidates for adjustment because prior skill and practice might relate to both training and task performance. The following deliberately small data set makes the calculations transparent.
| Participant | Training hours \(X\) | Pre-training skill \(A\) | Prior practice \(B\) | Task score \(Y\) |
|---|---|---|---|---|
| 1 | 11 | 52 | 42 | 78 |
| 2 | 11 | 52 | 40 | 76 |
| 3 | 11 | 50 | 42 | 74 |
| 4 | 11 | 50 | 40 | 72 |
| 5 | 9 | 50 | 40 | 68 |
| 6 | 9 | 50 | 38 | 66 |
| 7 | 9 | 48 | 40 | 64 |
| 8 | 9 | 48 | 38 | 62 |
All variables are complete for all eight participants, so each model below uses the same sample. The model includes an intercept and treats the exposure and scores as continuous. The observations are constructed to have a linear pattern, but the exposure takes only two values; these data cannot establish that a linear exposure relationship is appropriate more generally. The example illustrates coefficient comparison, not a real study or a causal conclusion.
Worked Example: Fitting the Exposure-Only Model
Start with the unadjusted linear regression of task score \(Y\) on training hours \(X\). For simple linear regression, the OLS slope is the sum of the products of deviations from the means divided by the sum of squared exposure deviations.
Here, \(\bar{x}=10\) hours and \(\bar{y}=70\) task-score points. The centered exposure values are four \(1\)s and four \(-1\)s. The centered outcome values are \(8, 6, 4, 2, -2, -4, -6,\) and \(-8\). Thus,
The fitted exposure-only association is \(5\) task-score points per additional training hour. The intercept is \(70-(5)(10)=20\), so the fitted equation is \(\hat{Y}=20+5X\). This model does not hold skill or prior practice fixed; its slope summarizes the unadjusted association in these data.
Comparing the Adjustment Sets
The next models add pre-training skill, prior practice, or both. In this constructed data set, the centered values of \(X\), \(A\), and \(B\) follow a simple pattern. Write \(u\) for the centered training-hours vector, \(v\) for a centered pattern in skill, and \(w\) for a centered pattern in prior practice. Each vector has four \(1\)s and four \(-1\)s, and every pair has products that sum to zero. The centered predictor vectors are \(u\) for \(X\), \(u+v\) for \(A\), and \(u+w\) for \(B\). The centered outcome vector is \(5u+2v+w\).
These relations let us check the adjusted fits directly. In OLS, a proposed residual must have zero sum of products with each included predictor for the coefficients to be the fitted coefficients. The vector \(w\) is orthogonal to both \(u\) and \(u+v\), and \(2v\) is orthogonal to both \(u\) and \(u+w\). The three centered predictors in the full model are linearly independent, so their coefficients can be distinguished.
| Model | Adjustment set | Estimated training coefficient | Interpretation in this model |
|---|---|---|---|
| 1 | None | 5 | 5 task-score points per additional training hour, unadjusted |
| 2 | Pre-training skill \(A\) | 3 | 3 points per additional hour, conditional on skill |
| 3 | Prior practice \(B\) | 4 | 4 points per additional hour, conditional on prior practice |
| 4 | Pre-training skill \(A\) and prior practice \(B\) | 2 | 2 points per additional hour, conditional on both covariates |
Worked Example: Calculating Coefficient Changes
Compare each adjusted estimate with the unadjusted estimate of \(5\) task-score points per training hour. Define a change as the adjusted estimate minus the unadjusted estimate.
In this data set, adding skill is associated with a \(2\)-point decrease in the estimated slope; adding prior practice is associated with a \(1\)-point decrease; and adding both is associated with a \(3\)-point decrease. All four estimates remain positive. Each coefficient is still measured in task-score points per training hour, but the conditioning differs across models. The differences are descriptive comparisons among fitted models, not tests that one coefficient is statistically different from another.
The changes are consistent with the covariates carrying information related to training and the outcome. They do not, by themselves, prove that either covariate is a confounder or establish the size of any causal effect. Those conclusions require subject-matter and causal reasoning as well as a correctly specified analysis.
Worked Example: Comparing the Two-Covariate Model With Each One-Covariate Model
Now compare the model adjusting for both variables with each model adjusting for only one. The change from the skill-only model to the model with both covariates is \(2-3=-1\) task-score point per training hour. The change from the prior-practice-only model to the model with both is \(2-4=-2\) points per hour.
These comparisons describe what adding the other covariate does, conditional on the covariate already included. They do not imply that the full model is automatically preferable. For instance, if prior practice were measured after training began, its timing could make adjustment inappropriate for a question about the total effect of training. If it were a pre-exposure common cause, the causal argument might instead support including it. The variable's role and timing matter; the coefficient change alone cannot settle them.
What Coefficient Movement Can and Cannot Tell You
A useful comparison reports which covariates were added and how the exposure coefficient changed, with its units and the conditioning stated. A coefficient that changes substantially can signal that the estimated association is sensitive to adjustment. It can motivate closer examination of the covariates, their measurement, and the causal structure. Conversely, a small change does not demonstrate that confounding is absent: relevant common causes may be unmeasured or measured imperfectly.
The substantive question should guide which models are compared. For example, a model with pre-exposure skill might be motivated by the possibility that skill relates to both training participation and task performance. A separate model with prior practice might be motivated by a similar theory. Those are reasons to examine the models, not guarantees that their assumptions are satisfied. The appropriate set depends on the target association or causal question and on the variables' roles and timing.
For a fair descriptive comparison, keep the outcome, exposure, units, coding, sample, and general functional form fixed while changing the adjustment set. If a covariate has missing values and its addition removes participants, the coefficient difference mixes a change in adjustment with a change in sample. It then cannot be attributed to adjustment alone. Also, if the exposure relationship is nonlinear or differs across covariate groups, a single common-slope model may not capture the relationship of interest.
Common Mistake
- Wrong reasoning: “The coefficient became smaller after adding two covariates, so the full adjusted model must be the correct one.”
- Correction: A changed coefficient tells us that the fitted association depends on the included covariates. It does not establish that those covariates belong in the model, that important variables have not been omitted, or that the remaining estimate is causal. Justify adjustment using the study question, timing, and subject-matter knowledge.
- Another warning: Do not describe a conditional coefficient as the same quantity as an unadjusted coefficient. Both may be valid summaries of different associations, but they answer different questions about the data.
Check Your Understanding
Answer each question using the definitions and calculations in this tutorial.
- In a model that includes training hours and pre-training skill, what does the training coefficient describe?
- In the worked data, calculate the change from the unadjusted estimate to the model adjusted for prior practice. State the units.
- Why is it important for models being compared to use the same observations?
- Does a decrease in the exposure coefficient after adjustment prove that the added covariate is a confounder? Explain.
- Give one reason a model adjusted for both covariates might not be appropriate for a particular causal question.