Tutorials › VEE Mathematical Statistics › Regression Association and Causal Claims

Home  /  Tutorials  /  VEE Mathematical Statistics / Simple Linear Regression Model

VEE Mathematical Statistics · Simple Linear Regression Model · Tutorial 721 of 880

Regression Association and Causal Claims

Interpret regression slopes carefully by separating what an observed association says from what a causal claim requires.

Intermediate 12 min read

What You'll Learn

  • Interpret a regression slope as a change in the model’s conditional mean, not automatically as a causal effect.
  • Define potential outcomes and the average treatment effect.
  • Recognize how a confounder can alter an observed treatment–outcome association.
  • State key conditions for causal interpretation from randomized or observational data.
  • Explain why unequal assignment probabilities can make an unweighted difference in means target the wrong quantity.

What a Regression Slope Does—and Does Not—Say

A regression slope summarizes a relationship between variables under a specified model. In a study of an optional training program, for example, a positive slope between participation and later productivity describes an association in the observed data. It does not, by itself, show that the program caused productivity to rise. People who choose training may differ from those who do not in experience, motivation, or other factors that also affect productivity.

The distinction matters whenever the question changes from “How are these variables related?” to “What would happen if we intervened?” Regression describes patterns in the data. A causal claim concerns outcomes under alternative interventions and needs additional assumptions or a suitable study design.

Definition: In the simple linear regression model \(Y=\beta_0+\beta_1X+\varepsilon\), with \(E(\varepsilon\mid X)=0\), the conditional mean is \(E(Y\mid X=x)=\beta_0+\beta_1x\). The slope \(\beta_1\) describes the change in that model’s conditional mean for a one-unit increase in \(X\). This is a model-based association; the equation alone does not say that intervening to change \(X\) would change \(Y\) by \(\beta_1\).

The condition \(E(\varepsilon\mid X)=0\) says that, at each value of \(X\), the model’s average error is zero. It supports the conditional-mean interpretation of the slope. If an omitted factor is related to \(X\) and helps predict \(Y\), that condition may fail for a model that leaves out the factor. Even when the condition holds as a statistical description, it does not turn an observational comparison into an intervention.

Why the Slope Is an Association

For a population with finite variances and \(\operatorname{Var}(X)>0\), the slope of the best linear predictor of \(Y\) from \(X\) is \(\operatorname{Cov}(X,Y)/\operatorname{Var}(X)\). Under the stated conditional-mean model, this slope equals \(\beta_1\). The identity explains how the slope reflects the joint distribution of observed \(X\) and \(Y\); it does not identify what would happen under a change to \(X\).

$$ \begin{aligned} Y&=\beta_0+\beta_1X+\varepsilon,\qquad E(\varepsilon\mid X)=0,\\ \operatorname{Cov}(X,Y) &=\operatorname{Cov}\bigl(X,\beta_0+\beta_1X+\varepsilon\bigr)\\ &=\beta_1\operatorname{Var}(X)+\operatorname{Cov}(X,\varepsilon)\\ &=\beta_1\operatorname{Var}(X). \end{aligned} $$

For the last equality, \(E(\varepsilon\mid X)=0\) implies \(E(\varepsilon)=0\) and \(E(X\varepsilon)=E[X E(\varepsilon\mid X)]=0\). Therefore \(\operatorname{Cov}(X,\varepsilon)=0\). Dividing by the positive \(\operatorname{Var}(X)\) gives the slope identity. Nothing in this derivation compares the same unit’s outcome under two alternative values imposed on \(X\); it is a statement about association in the model.

Potential Outcomes and Causal Effects

A causal question requires a comparison between outcomes under alternative treatments. For a binary treatment, let \(Y(1)\) be the outcome a unit would have under treatment and \(Y(0)\) the outcome that unit would have without treatment. The individual treatment effect is \(Y(1)-Y(0)\). Only one of these outcomes is observed for each unit, so the individual effect cannot generally be calculated from the observed record alone.

Definition: The average treatment effect in a population is \(E[Y(1)-Y(0)]\). Let \(T\) indicate the treatment actually assigned, with \(T=1\) for treatment and \(T=0\) for control. Under consistency, the observed outcome is \(Y=T Y(1)+(1-T)Y(0)\). Consistency means that the outcome under the treatment actually received equals the corresponding potential outcome. A no-interference condition means one unit’s treatment does not change another unit’s potential outcomes.

Random assignment can support causal comparisons because it makes treatment assignment independent of potential outcomes under the assignment design. In observational data, treatment choice may depend on factors that also predict the outcome. Such a factor is a confounder when it contributes to the observed treatment–outcome association and makes the groups differ in outcome-relevant ways apart from treatment.

For observational data, adjustment for measured covariates \(C\) can identify an average treatment effect if, among other requirements, the potential outcomes are independent of treatment conditional on \(C\), both treatment options are possible at the covariate values being compared, and consistency and no interference hold. In that case, the effect is obtained by comparing conditional outcome means within covariate groups and averaging those contrasts over the population’s covariate distribution. This is a demanding set of conditions: adjusting for a variable does not establish that all relevant confounding has been removed.

$$ \begin{aligned} \{Y(1),Y(0)\}&\perp T\mid C,\\ 0<P(T=1\mid C)&<1,\\ E[Y(1)-Y(0)] &=E_C\!\left[E(Y\mid T=1,C)-E(Y\mid T=0,C)\right]. \end{aligned} $$

The first line is conditional exchangeability: within levels of \(C\), treatment assignment carries no further information about the potential outcomes. The second is positivity: each treatment option has a positive chance at the covariate values included in the target population. If these assumptions fail, the displayed adjustment formula need not identify the causal effect. A regression coefficient adjusted for \(C\) is not automatically causal; its interpretation also depends on the model form and the causal assumptions.

Worked Example: A Confounder Changes the Observed Comparison

Worked Example: A Confounder Changes the Observed Comparison

A hypothetical employer compares an optional skills course with no course. Prior skill is recorded as low or high. Among participants, 8 had low prior skill and had a mean productivity score of 60; 2 had high prior skill and had a mean of 80. Among nonparticipants, 2 had low prior skill and had a mean of 65; 8 had high prior skill and had a mean of 85. Compute the overall difference in mean scores and the differences within each skill group. Explain what the results establish.

Method. Treat participation as a binary predictor \(T\). For a binary predictor, the slope in a regression with an intercept is the treated-group mean minus the untreated-group mean. First calculate those means from the counts and group averages.

The participant mean is \( (8(60)+2(80))/10=(480+160)/10=64 \). The nonparticipant mean is \( (2(65)+8(85))/10=(130+680)/10=81 \). Thus the unadjusted difference, participant minus nonparticipant, is \(64-81=-17\) score points.

Within the low-skill group, the difference is \(60-65=-5\). Within the high-skill group, it is \(80-85=-5\). The observed association is therefore negative both overall and within each recorded skill group, but the groups have very different skill compositions: 8 of 10 participants had low prior skill, compared with 2 of 10 nonparticipants.

Interpretation. The overall regression slope is \(-17\) points for the specified unadjusted binary-predictor model. The two within-group differences are each \(-5\) points. These are descriptive comparisons, not proof that the course reduces productivity. Prior skill is associated with both participation patterns and productivity, so the crude comparison mixes the course association with group composition. Even the within-skill differences would be causal only with further assumptions—for example, that relevant treatment selection is ignorable within each skill group.

Check. The participant total is 640 across 10 people, and the nonparticipant total is 810 across 10 people. Their means are 64 and 81, confirming the overall difference of \(-17\). The method avoids treating an adjusted or unadjusted association as a causal result without stating the needed assumptions.

Randomization Helps, but the Assignment Design Matters

Under a completely randomized design that assigns a fixed number of units to treatment, with every unit having the same probability of assignment, the difference in sample means is unbiased for the finite-population average treatment effect. “Unbiased” means that its expectation over the random assignment mechanism equals that target; it does not mean the estimate equals the target in every realized sample.

Worked Example: Equal-Probability Complete Random Assignment

In a hypothetical study, four clinics have potential outcome pairs \((Y(1),Y(0))\) equal to \((30,20)\), \((35,30)\), \((50,40)\), and \((55,50)\). Exactly two clinics are assigned treatment, with all six possible pairs of treated clinics equally likely. Find the finite-population average treatment effect, and examine the difference in means for one assignment.

Method. Calculate each clinic’s treatment effect, average them, then compute the observed difference for a specified assignment. The four effects are \(30-20=10\), \(35-30=5\), \(50-40=10\), and \(55-50=5\). Their average is \((10+5+10+5)/4=7.5\).

Suppose clinics 1 and 2 are assigned treatment. Their observed outcomes are 30 and 35, with treated mean \((30+35)/2=32.5\). Clinics 3 and 4 are controls, so their outcomes are 40 and 50, with control mean \((40+50)/2=45\). The observed difference is \(32.5-45=-12.5\), which is not the population effect of 7.5.

The six possible differences, in the order of treated pairs \(12,13,14,23,24,34\), are \(-12.5,0,7.5,7.5,15,27.5\). Their average is \(45/6=7.5\). Thus this design makes the difference in means unbiased, while the particular estimate for pair 12 differs substantially from the target.

Check. Each clinic has probability \(2/4=1/2\) of treatment and the same probability of control. The assignment mechanism, not an assumption that one observed estimate must equal the effect, justifies the expectation result. The method avoids confusing unbiasedness across assignments with exact accuracy in one study.

Equal assignment probabilities are important for that unweighted result. Random assignment in a broader sense does not guarantee that an unweighted difference in means estimates the population average treatment effect when assignment probabilities vary across units.

Worked Example: Unequal Assignment Probabilities

Consider two units assigned so exactly one receives treatment. Unit 1 is selected with probability 0.9 and unit 2 with probability 0.1. Their potential outcomes are \(Y_1(0)=Y_2(0)=0\), \(Y_1(1)=10\), and \(Y_2(1)=0\). Compare the population average treatment effect with the expected unweighted difference in means.

Method. The two individual effects are \(10-0=10\) and \(0-0=0\), so the population average treatment effect is \((10+0)/2=5\).

If unit 1 is treated, the observed treated mean is 10 and the control mean is 0, giving a difference of 10. If unit 2 is treated, both observed outcomes are 0, giving a difference of 0. Therefore the expected unweighted difference is \(0.9(10)+0.1(0)=9\), not 5.

With known assignment probabilities, a probability-weighted estimator can target the finite-population effect. Here it is \( \frac{1}{2}\sum_{j=1}^{2}\left[T_jY_j/p_j-(1-T_j)Y_j/(1-p_j)\right] \), where \(p_1=0.9\) and \(p_2=0.1\). If unit 1 is treated, the estimate is \(\frac12(10/0.9)=5.5556\) to four decimals. If unit 2 is treated, it is 0. Its expectation is \(0.9(5.5556)+0.1(0)=5\) before rounding, the average treatment effect.

Check. The unweighted estimate targets 9 under this design, whereas the weighted estimator has expectation 5. The weighted estimate in an individual assignment need not itself equal 5. This calculation avoids the trap of treating “randomly assigned” as sufficient justification for every unweighted comparison; the estimator must match the assignment design.

Interpreting Evidence and Avoiding Causal Overstatement

A regression result is most useful when its interpretation states the model and the scope of the evidence. For an observational study, say that a one-unit increase in \(X\) is associated with a \(\beta_1\)-unit change in the model’s conditional mean of \(Y\), holding included predictors fixed when appropriate. Do not replace “associated with” by “causes” unless the study design and assumptions justify that step.

Common threats to a causal interpretation include confounding by unmeasured variables, incorrect adjustment, lack of overlap between treatment groups, and a model that does not represent the relevant conditional means. Randomization reduces treatment-selection confounding in expectation, but does not guarantee identical groups in a particular sample, eliminate measurement problems, or ensure that results generalize to a different population.

  • Association: a pattern in the joint distribution of observed variables, summarized by a slope under a specified model.
  • Causation: a contrast between potential outcomes under alternative interventions, supported by design and assumptions.
  • Adjustment: a way to compare units with similar measured covariates; it does not rule out unmeasured confounding by itself.
  • Randomization: a design feature whose implications depend on its assignment probabilities and the estimator used.
Key takeaway: A regression slope describes an association specified by a statistical model. A causal interpretation requires a defined intervention, a suitable design or explicit identification assumptions, and an estimator appropriate to that design. Random assignment can support causal inference, but an unweighted difference in means targets the population average effect only under suitable assignment conditions, such as equal assignment probabilities.

Check Your Understanding

For each problem, distinguish a model-based association from a causal conclusion.

  1. In an observational study, a regression of annual medical spending on the number of clinic visits has slope 120 dollars per visit. State the model-based interpretation and one reason it does not alone establish causation. (Answer: In the specified model, one additional visit is associated with a 120-dollar increase in conditional mean spending; illness severity could increase both visits and spending.)
  2. A binary treatment group has mean outcome 18 and a control group has mean 15. What is the unadjusted binary-predictor regression slope, and what does it describe? (Answer: 3 outcome units; it describes the observed difference in group means, not necessarily a causal effect.)
  3. In a population, 40% receive treatment. The treatment group mean outcome is 26 and the control group mean is 20. A measured risk group is unevenly distributed across treatment status. What must be addressed before interpreting the difference of 6 causally? (Answer: Compare or adjust appropriately for the risk-group distribution, and justify conditional exchangeability, positivity, consistency, and no interference; the crude difference alone is not causal evidence.)
  4. Two units have treatment effects 8 and 2. Exactly one is treated, with each unit equally likely to be selected. What is the population average treatment effect, and what is the expected unweighted difference in means? (Answer: The average treatment effect is \((8+2)/2=5\), and the expected difference in means is 5 under equal-probability assignment.)
  5. Two units have control outcomes 0 and 0 and treatment outcomes 12 and 0. Exactly one is treated; unit 1 is selected with probability 0.75 and unit 2 with probability 0.25. Find the population average treatment effect and the expected unweighted difference in means. (Answer: The population average treatment effect is 6; the expected unweighted difference is \(0.75(12)+0.25(0)=9\), so it does not target 6.)

Previous

Next

Back to Tutorials
Clinical Biostats