Omitted Predictors and Changes in Regression Coefficients

Why a Regression Coefficient Can Change

A regression coefficient describes a relationship within a particular model. In a simple linear regression, the slope summarizes the fitted change in the response associated with a one-unit increase in the predictor. In multiple regression, a coefficient describes the fitted change associated with a one-unit increase in that predictor while the other included predictors are held fixed. These are different descriptions, so their numerical values need not agree.

Suppose a model relates exam score to hours studied. Adding a measure of prior preparation may change the estimated study-hours coefficient. That change can arise because prior preparation is associated with both study hours and exam score. The model containing prior preparation separates the study-hours association from the part shared with preparation; the model without it does not.

This comparison is useful for understanding how a model's estimates depend on which predictors it includes. It does not, by itself, show that either predictor causes the response. A regression coefficient is an adjusted or unadjusted association according to the model; causal interpretation requires additional justification.

The Omitted-Predictor Relationship

Consider a response \(Y\), a predictor \(X_1\) whose coefficient is of interest, and another predictor \(X_2\). Suppose the population relationship is represented by the two-predictor linear model below. The error term \(\varepsilon\) represents variation not captured by the predictors.

Y=\beta_0+\beta_1X_1+\beta_2X_2+\varepsilon

For the standard omitted-variable formula, assume the conditional mean is correctly represented by this linear form, and that \(E(\varepsilon\mid X_1,X_2)=0\). Also assume \(X_1\) has nonzero variance. The coefficient \(\beta_1\) is the conditional linear association between \(X_1\) and \(Y\), holding \(X_2\) fixed; \(\beta_2\) is the corresponding association for \(X_2\). If a simple linear regression of \(Y\) on \(X_1\) is used instead, its population slope is

\widetilde{\beta}_1 = \beta_1+ \beta_2\frac{\operatorname{Cov}(X_1,X_2)}{\operatorname{Var}(X_1)}

The added term is the omitted-variable component. The ratio \(\operatorname{Cov}(X_1,X_2)/\operatorname{Var}(X_1)\) is the slope from the population linear regression of \(X_2\) on \(X_1\). Thus the change depends on two things: how \(X_2\) is associated with the response after accounting for \(X_1\), and how \(X_2\) varies with \(X_1\).

Key result: Omitting \(X_2\) changes the population slope for \(X_1\), relative to the conditional coefficient \(\beta_1\), when both \(\beta_2\ne 0\) and \(\operatorname{Cov}(X_1,X_2)\ne 0\). The direction of the change is the sign of the product \(\beta_2\operatorname{Cov}(X_1,X_2)\). If either factor is zero, this formula gives no shift in the population slope.

Why the Formula Holds

The result follows from taking the covariance of the model with \(X_1\). Under the stated assumption, the error has zero covariance with \(X_1\). The intercept also has zero covariance with \(X_1\), so

\operatorname{Cov}(X_1,Y) = \beta_1\operatorname{Var}(X_1) + \beta_2\operatorname{Cov}(X_1,X_2)

The slope in the population simple linear regression of \(Y\) on \(X_1\) is \(\operatorname{Cov}(X_1,Y)/\operatorname{Var}(X_1)\). Dividing the covariance equation by \(\operatorname{Var}(X_1)\) gives

\widetilde{\beta}_1 = \beta_1+ \beta_2\frac{\operatorname{Cov}(X_1,X_2)}{\operatorname{Var}(X_1)}

The same reasoning has an exact sample counterpart when both models are fitted by ordinary least squares with an intercept to the same observations. In that case, the reduced-model slope equals the full-model slope for \(X_1\) plus the full-model slope for \(X_2\) multiplied by the slope from regressing \(X_2\) on \(X_1\). This identity explains a coefficient difference in a given data set; it does not guarantee that the population assumptions hold or that either coefficient is causal.

Worked Example: Preparation and Study Hours

In a hypothetical set of 100 students, exam score is measured in points, study time in hours, and prior preparation on an index scale. The same 100 students are used in each regression. Suppose the fitted model for exam score using both predictors is

\widehat{Y}=42+3.0X_1+6.0X_2

Here \(X_1\) is study time in hours and \(X_2\) is the prior-preparation index. A separate regression of prior preparation on study time has slope \(0.30\) index units per study hour. For these same observations, the exact sample relationship gives the reduced-model study slope:

b_{\text{reduced}} = b_{1,\text{full}}+ b_{2,\text{full}}(\text{slope of }X_2\text{ on }X_1)

Substituting the fitted slopes gives

b_{\text{reduced}} = 3.0\frac{\text{points}}{\text{hour}} + 6.0\frac{\text{points}}{\text{index unit}} \left(0.30\frac{\text{index unit}}{\text{hour}}\right) = 3.0+1.8 = 4.8\frac{\text{points}}{\text{hour}}

Thus the reduced model's study slope is 4.8 points per hour, whereas the full model's adjusted study coefficient is 3.0 points per hour. The added term is positive because preparation has a positive fitted association with score and is positively associated with study time in this sample. In the full model, the study coefficient describes the fitted association among students with the same preparation index; the reduced slope also reflects the part of preparation that tends to accompany additional study time.

The Direction of a Coefficient Change

The omitted-variable formula is especially helpful because it gives the direction of the change, not just the fact that a change may occur. If \(X_2\) has a positive conditional association with \(Y\) and tends to be larger when \(X_1\) is larger, omitting \(X_2\) shifts the \(X_1\) slope upward. If one of those associations is negative and the other positive, the shift is downward. A downward shift can make a negative coefficient more negative, make a positive coefficient smaller, or even change its sign.

Worked Example: Age and Exercise in a Blood-Pressure Model

Imagine a hypothetical observational sample of adults. Let \(Y\) be systolic blood pressure in mmHg, \(X_1\) be weekly exercise hours, and \(X_2\) be age in years. Suppose a fitted model including both predictors gives a coefficient of \(-1.5\) mmHg per exercise hour for \(X_1\), and \(+0.8\) mmHg per year for age. Suppose the fitted slope from regressing age on weekly exercise is \(-0.50\) years per exercise hour.

The sample omitted-variable relationship gives the exercise slope from a regression leaving age out:

b_{\text{reduced}} = -1.5\frac{\text{mmHg}}{\text{hour}} + 0.8\frac{\text{mmHg}}{\text{year}} \left(-0.50\frac{\text{year}}{\text{hour}}\right)

The second term is \(-0.40\) mmHg per hour, so

b_{\text{reduced}}=-1.5-0.40=-1.90\frac{\text{mmHg}}{\text{hour}}

In this example, leaving age out makes the exercise slope more negative: \(-1.90\) rather than \(-1.50\) mmHg per hour. The reason is that age has a positive fitted association with blood pressure, while age and exercise are negatively associated. This is a description of the specified regressions and their associations. It does not establish that increasing exercise would reduce an individual's blood pressure by either amount.

A Coefficient Change Is Not a Causal Verdict

In observational data, adding a predictor can change a coefficient without resolving whether the included predictor has a causal effect. A baseline measurement may help account for differences that existed before an exposure, but there may still be unmeasured factors, measurement error, or an unsuitable model form. In other settings, adjusting for a variable can change the question being answered rather than improve an estimate of the same causal quantity.

For example, a variable measured after an exposure may lie on a pathway through which that exposure affects the response. Controlling for it can remove part of an association that would be included in a total-effect question. The coefficient shift alone does not tell us whether a variable is a confounder, a mediator, or an inappropriate adjustment variable. Those judgments depend on the study design, timing, subject-matter knowledge, and a clearly stated causal question.

Worked Example: Tutoring and Baseline Exam Score

Suppose a hypothetical school dataset records students' tutoring hours, baseline exam scores, and final exam scores. Let \(X_1\) be tutoring hours, \(X_2\) be baseline score in points, and \(Y\) be final score in points. Using the same students, a model with both predictors estimates a tutoring coefficient of \(0.4\) final-score points per hour and a baseline-score coefficient of \(0.8\) final-score points per baseline point. Suppose the slope from regressing baseline score on tutoring hours is \(1.0\) baseline point per hour.

The tutoring slope when baseline score is omitted is

b_{\text{reduced}} = 0.4\frac{\text{final points}}{\text{hour}} + 0.8\frac{\text{final points}}{\text{baseline point}} \left(1.0\frac{\text{baseline point}}{\text{hour}}\right) = 1.2\frac{\text{final points}}{\text{hour}}

The reduced slope is 1.2 points per hour, compared with 0.4 points per hour after adjustment for baseline score. The formula describes how the estimates relate for these fitted models. It does not prove that tutoring raises final scores by 0.4 or 1.2 points per hour. Students who choose or receive tutoring may differ in ways the model does not measure, and the causal interpretation depends on how tutoring was assigned and what other factors were accounted for.

How to Compare Models Carefully

A coefficient comparison is most interpretable when the two models use the same observations, outcome, predictor coding, and measurement units, and differ only in the predictor or predictors under study. If adding a variable also changes the sample because of missing data, or if the model changes its functional form or interaction terms, a coefficient difference can have several explanations. Check what changed before attributing the movement to omission of one predictor.

When only \(X_2\) is added to a model containing \(X_1\), the two-predictor formula offers a direct explanation. With several included and omitted predictors, the change generally depends on their relationships with one another and with the response; it is not always captured by a single product. The simple formula remains a useful guide to the central idea, but should not be applied mechanically to a more complicated model.

Identify the coefficient and its meaning.
Name the response, the predictor of interest, the units, and whether the coefficient is from a reduced or adjusted model.

Check what differs between models.
Confirm whether both regressions use the same observations and whether the only change is the addition or removal of the predictor being examined.

Assess the two ingredients of the shift.
Ask whether the omitted predictor is associated with the response conditional on the included predictor and whether it is associated with the included predictor.

Interpret the change as model-dependent.
State how the fitted association changes, then separately consider whether study design and assumptions support any causal interpretation.

Common Mistakes

Assuming correlation alone guarantees a coefficient shift. If \(X_2\) is associated with \(X_1\) but has no conditional association with \(Y\), then \(\beta_2=0\) and the formula gives no shift.
Assuming an omitted predictor always changes a coefficient. If it has a conditional association with \(Y\) but is uncorrelated with \(X_1\), the formula also gives no shift in the population slope.
Calling every coefficient change omitted-variable bias. The term describes a shift relative to a specified coefficient under a model and assumptions. Changes in sample, coding, or model form can also change estimates.
Treating the adjusted coefficient as causal because another predictor was added. Adjustment changes the modeled association; it does not establish that all relevant confounding has been addressed.
Comparing coefficients without their units or model context. A slope must be described in terms of the response, the predictor, and which other predictors are held fixed.

Key takeaway: In the two-predictor linear model, omitting \(X_2\) shifts the slope for \(X_1\) by \(\beta_2\operatorname{Cov}(X_1,X_2)/\operatorname{Var}(X_1)\). Use this relationship to explain an association-driven coefficient change, not to claim causation.

Check Your Understanding

Answer each question using the model and context specified.

In the omitted-variable formula, what two conditions make the shift in the population slope for \(X_1\) nonzero?
A predictor \(X_2\) has a positive conditional coefficient, and its association with \(X_1\) is negative. In which direction does omitting \(X_2\) shift the slope for \(X_1\)?
A reduced-model slope is 5.0, and the full-model coefficient for \(X_1\) is 3.5. What does the difference say about the fitted associations, and what does it not establish?
Why is it important to check whether both models use the same observations before interpreting a coefficient change?
Give one reason an adjusted regression coefficient may still fail to have a causal interpretation.