How the Adjustment Set Changes a Coefficient

Why the Adjustment Set Matters

A regression coefficient is not a label attached to a predictor once and for all. Its meaning depends on the other predictors included in the model. For example, a model describing exam score from study time alone summarizes one association. A model that also includes a prior score summarizes the study-time association among students with the same prior score, according to that model.

These estimates can differ, even when both models use the same outcome, predictor, and observations. That difference is not automatically an error or proof that one estimate is the “correct” one. The models ask different conditional questions. To interpret or compare a coefficient, name the predictors included alongside it.

Definition: An adjustment set is the set of additional predictors included in a regression model alongside the predictor of interest. In an additive regression model, the coefficient of a predictor is its fitted change in the outcome for a one-unit increase in that predictor, with the other included predictors held fixed. It is therefore conditional on the model’s particular adjustment set.

For a predictor \(X\), an outcome \(Y\), and adjustment set \(A\), write the estimated coefficient as \(\hat{\beta}_{X\mid A}\). This notation highlights that the estimate for \(X\) is tied to the included set \(A\). If \(A\) changes, the coefficient may change. The coefficient’s units still come from the outcome and \(X\); what changes is the conditional comparison it summarizes and, potentially, its numerical value.

What Changes When a Predictor Is Added?

An ordinary least-squares model chooses coefficients to minimize the sum of squared differences between observed outcomes and fitted outcomes. With the same observations, adding a predictor changes the set of fitted equations available to the model. The new model chooses coefficients to minimize squared errors for that expanded equation; it does not have to preserve the coefficient previously assigned to \(X\).

This gives a direct reason that the coefficient can change: each model fits a different equation, with a different set of terms sharing the task of describing variation in \(Y\). In a model with a numeric predictor \(X\) and one added numeric predictor \(Z\), the coefficient of \(X\) is the value that minimizes the total squared error jointly with the coefficient of \(Z\). In a model without \(Z\), the coefficient of \(X\) is chosen without that term. The two optimization problems are not the same.

\begin{aligned} \text{Model without } Z:\quad &Y_i=\beta_0+\beta_X X_i+\varepsilon_i,\\ \text{Model with } Z:\quad &Y_i=\beta_0+\beta_X X_i+\beta_Z Z_i+\varepsilon_i. \end{aligned}

The second equation describes the fitted association between \(X\) and \(Y\) at a fixed value of \(Z\), within the model. It does not, by itself, say that changing \(X\) causes \(Y\) to change. Nor does adding \(Z\) guarantee that the estimated coefficient of \(X\) becomes larger, smaller, or more precise. Those are questions about the data and the model, not rules that follow merely from including another predictor.

Worked Example: Adding Prior Score to a Study-Time Model

Suppose a clearly hypothetical dataset contains 20 students. For each student, \(Y\) is a later exam score in points, \(X\) is study time in hours, and \(Z\) is a prior score in points. The same 20 students are used in both models. Assume the additive linear descriptions are reasonable for these data and that \(X\) and \(Z\) are not perfectly collinear, so both slopes can be estimated.

Suppose the centered sums calculated from these observations are \(S_{XX}=40\), \(S_{ZZ}=100\), \(S_{XZ}=20\), \(S_{XY}=80\), and \(S_{ZY}=100\). In the model using study time alone, the estimated slope is:

\hat{\beta}_{X}=\frac{S_{XY}}{S_{XX}} =\frac{80}{40}=2.00.

The model using both study time and prior score gives the study-time coefficient:

\hat{\beta}_{X\mid Z} =\frac{S_{XY}S_{ZZ}-S_{ZY}S_{XZ}} {S_{XX}S_{ZZ}-S_{XZ}^{\,2}} =\frac{(80)(100)-(100)(20)}{(40)(100)-20^2} =\frac{6000}{3600} \approx 1.667.

The denominator is positive because \(40(100)-20^2=3600>0\); in these summaries, \(X\) and \(Z\) are not perfectly collinear. The study-time estimate is 2.00 exam points per study hour in the model without prior score, and about 1.667 exam points per study hour at the same prior score in the model with prior score. These are two model-specific fitted associations, not two calculations of a single coefficient that must have one fixed value.

Compare Like With Like

A useful comparison keeps the outcome, predictor coding, units, and observations the same, then states how the adjustment set differs. If the observations change as well as the adjustment set, the coefficient difference may partly reflect which cases were analyzed. For example, a model that omits a predictor may use observations with missing values on that predictor, while a model that includes it may exclude those observations. In that situation, the comparison does not isolate the impact of changing the adjustment set.

Also check what the model actually says is held fixed. An additive model with a numeric predictor and several controls assigns one common slope to that predictor across the included predictor values. Its coefficient is not a separate estimate for every subgroup. Models that include different controls define different conditional associations; models with interactions can define still more specific comparisons. The coefficient must be interpreted from the equation that was fitted, not from a general phrase such as “adjusted for the other variables.”

Worked Example: Comparing a Group Coefficient Across Nested Models

In a hypothetical analysis of 60 students, \(Y\) is a final exam score in points and \(G\) indicates participation in an optional tutoring program (\(G=1\) for participants and \(G=0\) otherwise). A prior exam score is also recorded. Suppose all three models use the same 60 students, the same coding, and the same outcome. Assume the fitted additive forms are reasonable and that the included predictors are not perfectly collinear.

Suppose the estimated tutoring coefficients are 5.2 points in the model with \(G\) alone, 4.1 points in the model with \(G\) and prior score, and 4.4 points in the model with \(G\), prior score, and attendance. The change from the first model to the second is:

\hat{\beta}_{G\mid \text{prior}}-\hat{\beta}_{G} =4.1-5.2=-1.1\text{ points}.

Thus, in the model including prior score, the fitted difference in final exam scores between participants and nonparticipants with the same prior score is 4.1 points. In the model with \(G\) alone, the fitted difference is 5.2 points without holding prior score fixed. Adding attendance to the second model changes the tutoring coefficient by \(4.4-4.1=0.3\) points. The 4.4-point estimate is conditional on both prior score and attendance.

The model summaries establish these estimates; the differences do not establish that tutoring caused an increase in scores. The predictor is participation status, and the models describe fitted associations under their respective adjustments. Without a design and assumptions that support causal interpretation, “the adjusted association” is more accurate than “the effect.”

Each Estimate Answers Its Own Conditional Question

When two adjustment sets are plausible, it can be informative to report both coefficients rather than selecting one because it is larger, smaller, or closer to a preferred value. A clear report identifies the outcome, the predictor, the estimate and its units, and the adjustment set. It can then explain that the estimates differ across specifications. This makes the comparison transparent without suggesting that the model with more predictors is automatically better.

Worked Example: Two Different Adjustment Sets

Suppose a hypothetical dataset includes the same 75 patients, outcome \(Y\) as a symptom score in points, and predictor \(X\) as weekly exercise time in hours. One additive model includes age as its only adjustment predictor; another includes age and a measure of sleep duration. Assume both models provide reasonable additive descriptions, the predictors can be estimated without perfect collinearity, and both are fitted to the same patients.

Suppose the exercise-time coefficients are 0.8 symptom-score points per hour in the age-adjusted model and 0.6 points per hour in the model adjusted for both age and sleep duration. The numerical difference is:

\hat{\beta}_{X\mid \text{age, sleep}} -\hat{\beta}_{X\mid \text{age}} =0.6-0.8=-0.2\text{ points per hour}.

The first coefficient describes the fitted exercise-time association at the same age, according to the age-adjusted model. The second describes it at the same age and sleep duration, according to the other model. The difference of \(-0.2\) points per hour summarizes how the two estimates compare; it is not a coefficient in either model and does not establish why the estimates differ.

A suitable report would be: “The estimated exercise-time coefficient was 0.8 symptom-score points per hour in the model adjusted for age and 0.6 points per hour in the model adjusted for age and sleep duration, using the same patients.” This names both adjustment sets and avoids implying that one is the unique adjusted association.

Common Mistake: Treating “Adjusted” as One Universal Quantity

A common mistake is to say, “The adjusted coefficient is 4.1,” without naming the adjustment set, and then to treat that number as though adjustment had produced the one definitive coefficient. The correction is to specify the model: “The estimated coefficient is 4.1 points, adjusting for prior score.” If another model also adjusts for attendance, its coefficient is conditional on that larger set and should be reported as a separate estimate.

Likewise, a coefficient changing after adding a predictor does not by itself show that the new estimate is more accurate, less biased, or more causally meaningful. Those judgments require subject-matter reasoning and appropriate assumptions beyond simply comparing the numbers. For either model, the coefficient describes the fitted relationship conditional on that model’s predictors. If the data or model do not support the intended interpretation, the wording should not claim more than the estimate establishes.

Key takeaway: An adjusted coefficient is always conditional on a specified adjustment set. For a useful comparison, keep the outcome, predictor, coding, and observations fixed; state which predictors differ; and interpret each estimate in the context of its own model.

Check Your Understanding

Use the model-specific interpretation throughout your answers.

In an additive model with outcome \(Y\), predictor \(X\), and control \(Z\), what does the coefficient of \(X\) describe?
A coefficient is 3.0 units in a model with \(X\) alone and 2.4 units in a model with \(X\) and \(Z\), fitted to the same observations. Calculate the difference and describe what can be concluded.
Why is it important to use the same observations when comparing coefficients from models with different adjustment sets?
Does adding a control necessarily make the coefficient of the predictor of interest smaller? Explain.
Write a sentence that reports a coefficient while explicitly naming its adjustment set and units.