Why Regression Adjustment Requires Care
Adding variables to a regression model can change the estimated association between an exposure and an outcome. That change may be useful: adjustment for an appropriate common cause can help address confounding. But a changed coefficient is not, by itself, evidence that the model is better, that bias has been removed, or that the association is causal. The purpose of adjustment must be decided from the study question and subject-matter knowledge, not from a mechanical rule.
This tutorial focuses on four frequent errors: adjusting for every recorded variable, choosing covariates by their p-values, controlling for variables measured after the exposure, and treating adjustment as automatic causal identification. The central habit is to ask what role each variable plays and what comparison the fitted coefficient actually describes.
What an Adjusted Coefficient Says
For example, the model
describes the mean outcome as a linear function of the exposure and included covariates. To use ordinary least squares estimates and interpret them as estimates of the model’s coefficients, the model’s conditional mean should be adequately represented by this form, observations should be independent as appropriate to the study design, and the predictors should not have exact linear dependence. The error is assumed to have mean zero conditional on the modeled predictors. These conditions concern the regression model; they do not establish that the exposure was assigned independently of potential causes of the outcome.
For a binary exposure coded 0 and 1, \(\hat{\beta}_X\) is the fitted difference in mean outcome between the two exposure groups at the same values of the included covariates. For a continuous exposure, it is the fitted mean change per one-unit increase, holding those covariates fixed. The interpretation depends on the coding, units, and model form.
Worked Example: Why “Adjust for Everything” Can Mislead
Suppose a hypothetical clinic dataset records whether adults received a new care program (\(X=1\) if yes, \(X=0\) if no) and their symptom score three months later (\(Y\), points; higher scores mean worse symptoms). The analysts also recorded baseline symptom score, age, clinic, and a symptom measure taken one month after program assignment. In this hypothetical analysis, the unadjusted fitted difference is \(-2.40\) points. A model adjusting for baseline score and age gives \(-1.65\) points, while a model that also includes the one-month symptom measure gives \(-0.92\) points. Treat these as illustrative regression outputs, not results from a real study.
The exposure is binary, so each coefficient is a fitted difference between program and no-program groups. In the baseline-adjusted model, the estimate is \(-1.65\) points. Its sign indicates a lower fitted symptom score for the program group at the same baseline score and age. Adding the post-assignment symptom measure changes the estimate by
That arithmetic describes a change in the coefficient, not proof that the second model is less biased. The one-month symptom measure occurs after assignment; it may partly reflect the program’s effect. Adjusting for it changes the comparison and can remove part of a pathway through which the program affects the later score. The baseline-adjusted model and the model including the post-assignment measure therefore need not answer the same question.
Nor should clinic be included merely because it is available. Whether clinic is relevant depends on how program assignment and outcomes arose, and on the target comparison. A variable’s presence in the dataset is not a reason to adjust for it. First establish its timing and plausible role; then choose a model consistent with the question.
Do Not Select Covariates by Their P-Values
A p-value for a covariate tests a particular null hypothesis about its regression coefficient, conditional on the other included variables and the model assumptions. It does not test whether that variable is a confounder, whether it should be included, or whether adjustment for it improves causal identification. A true common cause can have a large p-value in a particular sample; a variable that is not a confounder can have a small p-value.
Selecting covariates by a threshold such as \(p<0.05\) makes the adjustment set depend on sample-specific evidence about outcome prediction rather than on the variable’s role and timing. It can leave a relevant common cause unadjusted or include a variable for an unsuitable reason. The earlier material on subject-matter covariate selection provides the relevant principle: determine the intended adjustment set from the scientific structure of the problem.
Worked Example: A Small P-Value Does Not Make a Covariate Necessary
In a hypothetical study of an exercise program (\(X\)) and six-month blood pressure (\(Y\), mmHg), investigators fit a regression model with age and baseline blood pressure. The exposure coefficient is \(1.80\) mmHg. A recorded variable indicating the participant’s preferred appointment time has a conditional p-value of \(0.004\); adding it changes the exposure coefficient to \(1.76\) mmHg. A subject-matter variable indicating a prior diagnosis has a conditional p-value of \(0.31\); adding it instead changes the exposure coefficient to \(1.52\) mmHg. These are stipulated outputs from a hypothetical analysis.
The changes are
for appointment preference, and
for prior diagnosis. Selecting only variables with \(p<0.05\) would include appointment preference but exclude prior diagnosis. Yet the p-values do not tell us whether either variable is a common cause of program participation and blood pressure. The larger coefficient change for diagnosis is also not enough, on its own, to prove that it is a confounder; that judgment requires knowledge of the variables’ relationships and timing.
The correct conclusion is not “include diagnosis because it changes the coefficient” or “include appointment preference because its p-value is small.” Investigators should justify the adjustment set using the study question and subject-matter knowledge, then describe the coefficient and its limitations. P-values are not a substitute for that reasoning.
Post-Exposure Adjustment Can Change the Target
A post-exposure variable is measured or occurs after the exposure. If the exposure can affect that variable, the variable may be a mediator: part of a pathway from exposure to outcome. Adjusting for it can block some of that pathway. It can also create bias in some settings, such as when the variable is a common effect of the exposure and another cause of the outcome. The direction and size of the resulting change are not reliably diagnosed by looking only at the coefficient.
A short algebraic argument illustrates why a mediator can change an exposure coefficient. Suppose, for simplicity, that the exposure \(X\) affects a mediator \(M\), and the outcome follows \(Y=\tau X+\gamma M+\varepsilon\). Suppose also that \(M=aX+u\), where \(u\) is unrelated to \(X\) in this simplified illustration and the error has conditional mean zero. Substituting the mediator equation into the outcome equation gives
Thus the exposure slope in a model that does not include \(M\) is \(\tau+\gamma a\) under these simplified conditions, while a model that includes \(M\) separates the coefficient \(\tau\) from the pathway represented by \(\gamma a\). The two coefficients answer different questions within this model. This algebra does not show that either coefficient is a causal effect in real data: that interpretation requires additional design and causal assumptions.
Worked Example: Adjusting for a Mediator Is Not a Neutral Step
Consider a hypothetical evaluation of a coaching program (\(X=1\) for offered coaching, \(X=0\) otherwise) and a later fitness score (\(Y\), points). The program may increase weekly exercise (\(M\), hours per week), which may in turn improve fitness. Suppose a model adjusting for baseline fitness estimates an exposure coefficient of \(3.60\) points, while adding weekly exercise reduces it to \(1.40\) points.
The coefficient difference is
It is tempting to call the smaller coefficient “the corrected effect.” That is not justified by the calculation. If coaching changes weekly exercise, the second model conditions on a variable on a possible pathway from coaching to fitness; it no longer describes the same overall comparison as the first model. The value \(-2.20\) points is the change between two fitted coefficients, not an estimate of bias.
The appropriate model depends on the question. If the question concerns the overall association of coaching with later fitness, adjustment for a post-program exercise measure may remove part of the association of interest. If a question specifically concerns an association conditional on exercise, that conditional quantity may be relevant, but interpreting it causally requires more assumptions than fitting the regression. Timing and purpose should be stated before interpreting the adjusted coefficient.
Adjustment Is Not Automatic Causal Identification
Regression adjustment can control for measured covariates in the fitted model. It cannot guarantee that all confounding has been removed. A causal interpretation requires, among other things, an appropriate set of pre-exposure covariates that suffices to control confounding, correct measurement and adequate model specification, and exposure comparisons that are possible across relevant covariate patterns. Unmeasured common causes, poor measurement, incorrect functional form, and unsuitable adjustment can all undermine a causal claim.
In particular, “adjusted for age, baseline severity, and income” names variables in a model; it does not prove that the groups are otherwise comparable as if exposure had been assigned at random. The result remains an adjusted association unless the design and assumptions justify a causal interpretation. Association alone does not establish that changing the exposure would change the outcome.
Worked Example: An Adjusted Association Is Not Proof of Cause
In a hypothetical observational comparison, people who use a sleep app report an average of \(0.80\) more hours of sleep per night than nonusers after adjustment for age and work schedule. The fitted coefficient is \(0.80\) hours per night. At equal modeled age and work schedule, the app-user group has a fitted mean sleep duration higher by \(0.80\) hours per night.
The coefficient is in the outcome’s units, hours per night. It does not establish that using the app increases sleep by \(0.80\) hours. For example, motivation to improve sleep could influence both app use and sleep duration, but it is not among the named covariates. The regression calculation adjusts for age and work schedule, not for every possible common cause. The warranted statement is that app use is associated with higher sleep duration conditional on the included variables; a causal claim would require additional justification.
This interpretation also follows the fitted model’s assumptions: the conditional mean relation must be adequately modeled, and the recorded variables and observations must support the analysis. Even if those regression conditions hold, they do not by themselves establish that all common causes of app use and sleep have been controlled.
A Practical Audit Before Interpreting an Adjusted Model
State the exposure, outcome, coding, and units so the coefficient’s scale is clear.
For each proposed covariate, ask whether it is pre-exposure, post-exposure, or plausibly a common cause, mediator, or other variable.
Do not include a variable solely because it was recorded, has a small p-value, or changes the exposure coefficient.
Describe the modeled conditional association first. Make a causal claim only when the study design and assumptions support it.
Common Mistakes
- Wrong: “We had the variable, so we adjusted for it.” Correction: Availability is not a scientific reason to condition on a variable; consider its timing and role.
- Wrong: “The covariate had \(p=0.03\), so it is a confounder.” Correction: A p-value tests a coefficient under a model; it does not classify the variable’s causal role.
- Wrong: “The adjusted coefficient is smaller, so the unadjusted result was biased.” Correction: A coefficient change can reflect confounding, a changed target, model features, or other causes. Its direction alone does not diagnose bias.
- Wrong: “We adjusted for several variables, so the exposure effect is causal.” Correction: Adjustment does not ensure that relevant common causes were measured or appropriately controlled, nor that other causal assumptions hold.
Check Your Understanding
Try these questions before moving on.
- In a multiple regression, what does the exposure coefficient mean when the exposure is coded 0 and 1?
- Why does a small p-value for a covariate fail to show that it is a confounder?
- A variable is measured after exposure and may be affected by exposure. What is one reason adjusting for it may change the question answered by the model?
- An adjusted coefficient differs from the unadjusted coefficient. Why does that difference alone not reveal which estimate is less biased?
- What additional kind of justification is needed before describing an adjusted association as a causal effect?