The Causal Claim Pitfall in Observational Regression

Why an Observational Association Is Not Automatically an Effect

A regression may show that people who exercise more tend to have lower blood pressure. That pattern is an association: it describes how the variables vary together in the observed data. Saying that exercise lowers blood pressure makes a stronger claim. It says that changing exercise would change blood pressure, rather than merely that exercise and blood pressure are related.

That distinction matters whenever the predictor was observed rather than assigned. People who exercise more may differ in age, health, income, or other characteristics that also relate to blood pressure. A regression can describe an association accurately and still fail to isolate an effect. Its numerical precision, a small p-value, or a good fit does not by itself resolve that problem.

What a Causal Claim Means

For a simple causal question, let \(A\) indicate whether a unit receives an exposure or treatment: \(A=1\) means exposed and \(A=0\) means unexposed. Let \(Y(1)\) denote that unit’s outcome under exposure and \(Y(0)\) its outcome without exposure. These are called potential outcomes. For one unit, only one of the two is observed: the unit’s actual outcome is \(Y=Y(1)\) if \(A=1\), and \(Y=Y(0)\) if \(A=0\).

A common population causal target is the average treatment effect, the mean difference between the two potential outcomes across the target population. It is a comparison of outcomes under two conditions, not simply a difference between two observed groups.

The central difficulty is that the same person cannot, at the same time, be observed both exposed and unexposed under otherwise identical circumstances. Observational regression compares different units and tries to make that comparison informative about the unobserved alternative outcomes. Whether it succeeds depends on the study design and assumptions, not just on the regression calculation.

Assumptions Needed for a Causal Interpretation

For a treatment or exposure comparison, a useful starting point is to ask whether the observed groups provide a credible comparison of the potential outcomes. Several conditions are important. They are substantive requirements about how the data arose and what was measured; fitting a regression does not make them true.

• Temporal order: the exposure must occur before the outcome it is claimed to affect. If timing is unclear, the outcome may have influenced the exposure, or both may have changed together.
• Exchangeability: after accounting for relevant measured characteristics, the exposed and unexposed groups must be comparable in their potential outcomes. In practice, this requires that important common causes of exposure and outcome are not left unaccounted for. This is often described as no unmeasured confounding.
• Positivity: within the kinds of people being compared, there must be a real chance of observing each exposure condition. If, for example, every person in a subgroup is exposed, the data cannot directly compare exposed and unexposed people in that subgroup.
• Consistency and a well-defined exposure: the exposure condition should have a clear meaning, and a person’s observed outcome under their actual exposure should correspond to the relevant potential outcome. If “treatment” covers substantially different versions, a single causal contrast may not be well defined.
• Suitable measurement and inclusion: exposure, outcome, and important comparison characteristics should be measured well enough for the intended question, and study participation should not create a misleading comparison. Measurement error or selective inclusion can distort an association and complicate causal interpretation.

These conditions explain why knowing the design is essential. Random assignment, when properly carried out, can support exchangeability by making assignment independent of potential outcomes on average. In an observational study, exposure is not assigned at random, so comparability instead depends on the design, the measured information, and assumptions about what has or has not been controlled. A regression adjustment can account for included variables under suitable conditions; it cannot guarantee that relevant unmeasured differences are absent.

A Short Justification: When an Observed Difference Equals an Effect

The distinction can be stated precisely for a binary exposure. The observed difference in mean outcomes compares those actually exposed with those actually unexposed. By contrast, the average treatment effect compares both potential outcomes across the same target population. If exposure groups are exchangeable, then the average potential outcome under either condition is the same within the group that actually received that condition as it is in the full target population. If consistency also holds, the observed outcome in each group is the potential outcome under its actual condition. Together, these facts give:

The first equality uses consistency. The second uses exchangeability: \(Y(1)\) and \(Y(0)\) must each have the same mean across exposure groups as in the target population. Without that comparability, the observed difference can mix an exposure effect with pre-existing group differences, as in the age example. This is a justification for the assumptions, not a test that proves they hold in a particular observational data set.

When measured characteristics \(C\) are used to make groups more comparable, the causal argument must still be conditional on those characteristics: within relevant values of \(C\), the exposure groups need to be exchangeable, and both exposure conditions need to occur. The analysis must then combine comparisons in a way that represents the target population. Merely including a variable in a regression does not establish these conditions, and an adjusted regression coefficient need not automatically equal the population effect of interest.

Timing, Selection, and the Strength of the Wording

Some observational associations are especially difficult to interpret because the direction of influence is uncertain. Suppose people who report more cups of coffee per day also report fewer hours of sleep. The observed contrast can be calculated and described, but it does not establish that coffee reduced sleep. People with demanding schedules might both drink more coffee and sleep less; alternatively, people who slept poorly might drink more coffee the next day. If exposure and outcome were measured at the same time, the data may not clarify which came first.

Selection can pose a related problem. If entry into a study depends on both the exposure and the outcome, analyzing only those who entered may create or distort an association. A regression among recruited participants still describes the analyzed sample, but its relationship may not represent the target population or a causal effect there. Recruitment and inclusion therefore belong in the interpretation, alongside the regression output.

Use wording that matches the evidence. “Was associated with,” “was higher among,” and “predicted” can describe observed relationships, provided the context is clear. “Caused,” “increased,” “reduced,” and “led to” assert an effect and need a design and assumptions that support that claim. A statistically significant association does not change this distinction: a p-value addresses compatibility with a specified statistical null model, not whether confounding, reverse direction, or selection has been ruled out.

Common Mistake: Treating Adjustment as Proof

• Wrong reasoning: “The regression included age and income, so its exposure coefficient is the causal effect.”
• Correction: Adjustment can address differences represented by the measured variables under appropriate modeling and design conditions. It does not show that all important common causes were measured, that measurements are accurate, that timing is correct, or that selection is harmless.
• Wrong reasoning: “The association is strong and statistically significant, so the predictor caused the outcome.”
• Correction: Strength and statistical significance describe aspects of the observed association and its sampling uncertainty. They do not establish causal identification.