Causal Effect Heterogeneity Versus Statistical Interaction

Introduction

A treatment may have different effects in different groups of people. That is a claim about how causal effects vary across groups. A regression model may also include an interaction between treatment and a covariate. That is a statement about the form of a statistical model. The two ideas can be related, but they are not interchangeable.

The distinction matters because causal effects require a defined intervention and a defined comparison, while a model interaction depends on how an outcome is represented and which scale the model uses. A treatment can have equal effects across groups on one scale and unequal effects on another. Conversely, an interaction in observed data need not represent causal effect heterogeneity if treatment assignment is confounded.

Definition: Let $A$ denote treatment, with levels 0 and 1, and let $Y_i(a)$ be the outcome individual $i$ would have under treatment level $a$. Let $Z_i$ be a baseline covariate defining groups. The subgroup-average causal effect for group $z$, on the outcome-difference scale, is $$ \Delta(z)=E[Y_i(1)-Y_i(0)\mid Z_i=z]. $$ Causal effect heterogeneity across two groups $z_1$ and $z_0$, on this scale, is the contrast $\Delta(z_1)-\Delta(z_0)$. It is not simply a difference between observed outcomes in the groups.

This notation assumes that the treatment and outcome are well defined. For simplicity, the examples below use a setting with no interference between individuals. If one person's outcome can depend on others' assignments, potential outcomes must instead be indexed by the relevant exposure condition or assignment regime; the causal comparison must then specify those conditions as well.

Causal Heterogeneity and Statistical Interaction

A causal effect is defined by comparing potential outcomes under interventions. A statistical interaction is defined within a model for observed data. For example, a model might describe the conditional mean outcome as a function of treatment and a baseline covariate. On a chosen scale, the model may impose that the treatment contrast is constant across covariate values, or allow that contrast to vary. The latter is an interaction on that model's scale.

To interpret a model-based interaction causally, additional conditions are needed. The model's outcome and treatment contrasts must correspond to the causal estimand of interest, and the observed data must identify that estimand. In a randomized study, randomization can support identification of group-specific causal contrasts, subject to consistency, appropriate treatment definition, and adequate assignment probability within the groups. In an observational study, identification generally also requires that relevant confounders are controlled, along with positivity and consistency. A model term by itself establishes none of these conditions.

The word “interaction” is also scale-dependent. A risk difference compares two risks by subtraction; a risk ratio compares them by division. The same pair of causal risks can have an equal treatment effect across groups under subtraction and unequal effects under division. Therefore a claim of heterogeneity must name the causal contrast and its scale.

Worked Example: Comparing Subgroup Risk Differences

Consider a hypothetical randomized trial with a binary outcome. Treatment was randomized within each of two baseline groups. In group A, 8 of 12 treated participants and 6 of 12 control participants had the outcome. In group B, 2 of 11 treated participants and 1 of 10 control participants had the outcome. Use the observed risks to estimate each group's risk difference and their difference.

The risk difference in group A is $$ \widehat{\Delta}(A)=\frac{8}{12}-\frac{6}{12}=\frac{2}{3}-\frac{1}{2}=\frac{1}{6}\approx 0.1667. $$ The estimated treatment risk is about 16.67 percentage points higher than the control risk in group A.

In group B, the risk difference is $$ \widehat{\Delta}(B)=\frac{2}{11}-\frac{1}{10} =\frac{20-11}{110}=\frac{9}{110}\approx 0.08182. $$ The estimated treatment risk is about 8.182 percentage points higher than the control risk in group B.

The estimated heterogeneity contrast, group A minus group B, should be calculated from the unrounded values: $$ \widehat{\Delta}(A)-\widehat{\Delta}(B) =\frac{1}{6}-\frac{9}{110} =\frac{55-27}{330} =\frac{14}{165}\approx 0.08485. $$ Thus the estimated risk difference is about 8.485 percentage points larger in group A. Rounded to two decimal places as a percentage-point contrast, this is 8.48 percentage points. These point estimates alone do not establish that the underlying causal effects differ: the groups are small, so sampling uncertainty may be substantial. Randomization supports causal interpretation of the subgroup contrasts in expectation, but does not make noisy estimates exact.

The Estimand and the Outcome Scale

The estimand is the precise causal quantity a study aims to learn. When comparing effects across groups, it includes both the target groups and the contrast used to summarize the outcome. “Does treatment work differently by group?” is incomplete until the outcome scale is specified. For a binary outcome, common choices include comparing risks by subtraction or comparing them by division. Those choices answer different questions.

Worked Example: The Same Risks, Different Scale Conclusions

Suppose, hypothetically, that the causal risks under treatment and control are as follows. Assume these risks refer to the same target population within each group and are identified under a suitable design or set of assumptions.

Baseline group	Risk under control	Risk under treatment
Group 0	0.10	0.30
Group 1	0.50	0.70

On the risk-difference scale, the treatment effects are $$ \Delta(0)=0.30-0.10=0.20,\qquad \Delta(1)=0.70-0.50=0.20. $$ The risk-difference heterogeneity contrast is $0.20-0.20=0$: the treatment increases risk by 20 percentage points in each group.

On the risk-ratio scale, the corresponding ratios are $$ \frac{0.30}{0.10}=3.00,\qquad \frac{0.70}{0.50}=1.40. $$ The relative effects differ across groups, even though the risk differences are equal. This is not a contradiction: the two scales define different effect comparisons. A conclusion about heterogeneity is meaningful only when it states which contrast is being compared.

This example does not imply that one scale is universally correct. Absolute differences can be useful when the question concerns changes in outcome probability; relative contrasts can be useful for other scientific or decision-making questions. The appropriate estimand depends on the question. Detailed interpretation of particular additive, multiplicative, and model-based scales requires care beyond simply checking whether a model contains an interaction term.

Why an Observed Interaction Need Not Be Causal

An observed-data model describes patterns in the data actually collected. If treatment assignment is associated with prognosis, observed treatment-outcome contrasts may combine causal effects with differences in who received treatment. Those assignment patterns can differ across groups, creating apparent variation in observed associations even when causal effects are constant.

Worked Example: Different Observed Contrasts With Constant Causal Effects

Consider a hypothetical observational study with a binary baseline group $Z$ and an unmeasured prognostic factor $U$, classified as low or high risk. Suppose untreated outcome risks are 0.10 for low-risk individuals and 0.50 for high-risk individuals. Treatment lowers each individual's risk by 0.10, so the risks under treatment are 0.00 and 0.40, respectively. The treatment effect is therefore $-0.10$ for both levels of $U$ and for both groups $Z$.

Suppose treatment selection differs by group. In group $Z=0$, 80% of treated individuals but 20% of untreated individuals are high risk. The observed treated risk is $$ 0.80(0.40)+0.20(0.00)=0.32, $$ and the observed untreated risk is $$ 0.20(0.50)+0.80(0.10)=0.18. $$ The observed treatment-minus-control difference is $0.32-0.18=0.14$.

In group $Z=1$, suppose 20% of treated individuals but 80% of untreated individuals are high risk. The observed treated risk is $$ 0.20(0.40)+0.80(0.00)=0.08, $$ and the observed untreated risk is $$ 0.80(0.50)+0.20(0.10)=0.42. $$ The observed difference is $0.08-0.42=-0.34$. Thus the observed contrasts vary sharply across $Z$, although the stipulated causal effect is $-0.10$ in both groups. The discrepancy arises because treatment groups have different mixes of the prognostic factor, and those mixes vary by $Z$. Without adequate control of confounding, an interaction in an observed-data model would not establish causal effect heterogeneity.

A Short Identification Justification

The connection between an observed subgroup contrast and a causal subgroup effect can be stated precisely. Let $Y$ be the observed outcome and $A$ the observed treatment. Under consistency, $Y=Y(a)$ for individuals who receive $A=a$. If treatment is conditionally exchangeable within group $Z=z$, meaning $Y(a)$ is independent of $A$ given $Z=z$, then

E[Y\mid A=a,Z=z] =E[Y(a)\mid A=a,Z=z] =E[Y(a)\mid Z=z].

The first equality follows from consistency; the second follows from conditional exchangeability. If positivity also holds, so that both treatment levels are possible within the group, the observed conditional means identify the group-specific causal means. Subtracting them identifies the subgroup-average risk difference:

E[Y\mid A=1,Z=z]-E[Y\mid A=0,Z=z] =E[Y(1)-Y(0)\mid Z=z].

Without exchangeability, the key second equality need not hold. In a randomized study, random assignment can supply exchangeability, but the treatment contrast still has to match the target intervention and population. With interference, the potential outcomes and assignment conditions must be defined for the exposure regime of interest rather than treated as if each person's assignment were isolated.

Common Mistakes and Practical Interpretation

Equating an interaction term with causal heterogeneity. A model term describes a pattern under a model's assumptions and scale. Causal interpretation additionally requires a causal estimand and identification assumptions.
Calling effects “the same” without naming a scale. Equal risk differences do not imply equal risk ratios. State the contrast before describing heterogeneity.
Comparing separate significance tests. A statistically significant estimate in one subgroup and a non-significant estimate in another do not, by themselves, show that the subgroup effects differ. The relevant target is a direct contrast between effects, with its uncertainty assessed appropriately.
Interpreting subgroup associations as causal effects. Different treatment assignment or prognostic composition across groups can produce observed differences even when causal effects are constant.
Ignoring the target population. Group-specific causal means refer to the people in each group as defined. Changing the population or the distribution over which outcomes are averaged can change the estimand.

Key takeaway: Causal effect heterogeneity is variation in a specified causal contrast across defined groups. Statistical interaction is a feature of a chosen model and scale. To connect the two, define the estimand, specify the scale and target population, and justify why the observed data identify the causal quantities.

Check Your Understanding

Use the definitions and examples above to answer the following questions.

What is the difference between a subgroup-average causal effect and an interaction in a statistical model?
In the worked example with two risk groups, why was there no heterogeneity on the risk-difference scale but heterogeneity on the risk-ratio scale?
In the observational example, what caused the observed treatment contrasts to differ across groups despite a constant stipulated causal effect?
Which identification condition justifies replacing a conditional mean among treated individuals with a potential-outcome mean for the group?
Why does finding a significant subgroup result and a non-significant result in another subgroup not, by itself, establish effect heterogeneity?