Causal Effect Modification Across Populations

Why the Same Conditional Effects Can Produce Different Averages

A treatment can have the same causal effect for people in each of several clinically meaningful groups in two populations, yet its overall average effect can differ between those populations. The reason may be composition: the populations contain different proportions of people in the groups where the treatment effect is larger or smaller. Understanding this distinction is central to comparing effects across a study population and a population of interest.

The comparison is about causal effects, not merely observed outcome rates. A difference in observed outcomes between populations could have many explanations. Here we isolate one possibility: the conditional causal effects are stable across populations, but the distribution of a variable that modifies those effects differs.

Definition: Let $G$ indicate population membership, with $G=S$ for the source population and $G=T$ for the target population. For treatment levels $A=1$ and $A=0$, let $Y^1$ and $Y^0$ be the potential outcomes under those treatment levels. The population-specific average causal effect is $$ \Delta_g=E[Y^1-Y^0\mid G=g], \qquad g\in\{S,T\}. $$ For a covariate $V$, the conditional average causal effect at $V=v$ is $$ \tau_g(v)=E[Y^1-Y^0\mid V=v,G=g]. $$ The conditional effects are stable across populations when $\tau_S(v)=\tau_T(v)$ for the relevant values of $v$. This equality does not require the distributions of $V$ to be the same.

The sign in $\Delta_g$ is important: it is the mean potential outcome under treatment minus the mean potential outcome under control. For an adverse binary outcome, a treatment that lowers risk therefore has a negative risk difference. A positive risk-reduction magnitude can also be reported, but it is the negative of this defined estimand.

How Population Composition Enters the Average

Suppose $V$ has a finite set of categories. Within population $g$, divide the population into those categories and average the conditional effects using the category proportions in that population. The law of total expectation gives

\Delta_g =E\!\left[E[Y^1-Y^0\mid V,G=g]\mid G=g\right] =\sum_v \tau_g(v)P(V=v\mid G=g).

This identity says that a population-average effect depends on both the conditional causal effects and the population's distribution of $V$. If the conditional effects are stable, write their shared value as $\tau(v)$. Then

\Delta_T-\Delta_S =\sum_v \tau(v)\left\{P(V=v\mid G=T)-P(V=v\mid G=S)\right\}.

The difference in average effects is thus a weighted sum of differences in stratum proportions. It can be nonzero even though there is no source-target difference in the conditional effect within any stratum. For a continuous $V$, the corresponding sums are integrals over its population-specific distributions.

Key result: If conditional causal effects are stable across populations, then source and target average effects can still differ whenever the distributions of effect-modifying covariates differ in a way that changes the weighted average. If the conditional effect is constant across all values of $V$, changing the distribution of $V$ alone does not change the average effect.

Worked Calculations

Worked Example: Different Proportions in Two Risk Strata

Suppose a hypothetical treatment is evaluated for a binary adverse outcome. In both populations, the treatment risk difference is $-2$ percentage points in the lower-risk stratum and $-8$ percentage points in the higher-risk stratum. Assume these conditional causal effects are stable across the two populations. The source population is 80% lower-risk and 20% higher-risk; the target population is 25% lower-risk and 75% higher-risk. Calculate both average effects.

Source population. The source average effect is the weighted sum of the conditional effects, using source proportions:

\Delta_S=0.80(-2)+0.20(-8) =-1.6-1.6=-3.2\text{ percentage points}.

Target population. The target proportions place more weight on the higher-risk stratum:

\Delta_T=0.25(-2)+0.75(-8) =-0.5-6.0=-6.5\text{ percentage points}.

The difference between the defined average effects is $$ \Delta_T-\Delta_S=-6.5-(-3.2)=-3.3\text{ percentage points}. $$ Thus the target average risk difference is 3.3 percentage points more negative. Equivalently, if reporting risk-reduction magnitudes $R_g=-\Delta_g$, the source reduction is 3.2 points and the target reduction is 6.5 points, so the target reduction magnitude is 3.3 points larger. These are two sign conventions for describing the same calculations.

Worked Example: Composition Can Reverse the Average Effect

Consider two strata in a hypothetical study. In stratum A, the adverse-outcome risk is 10% under control and 14% under treatment, so the conditional risk difference is $14\%-10\%=+4$ percentage points. In stratum B, the risks are 30% under control and 20% under treatment, so the conditional risk difference is $20\%-30\%=-10$ percentage points. Suppose these stratum-specific causal effects are the same in source and target populations. Stratum A makes up 80% of the source and 20% of the target; stratum B makes up the remaining proportion in each.

For the source population:

\Delta_S=0.80(4)+0.20(-10) =3.2-2.0=+1.2\text{ percentage points}.

For the target population:

\Delta_T=0.20(4)+0.80(-10) =0.8-8.0=-7.2\text{ percentage points}.

The source average effect is a 1.2-point increase in adverse-outcome risk, while the target average effect is a 7.2-point decrease. This reversal does not require the conditional effects to change between populations. It occurs because the stratum with a beneficial effect is much more common in the target, while the stratum with a harmful effect is much more common in the source. The example describes population-average causal estimands; it does not, by itself, establish that any particular observed-data analysis identifies them.

Worked Example: A Distribution Difference That Does Not Change the Average

Suppose a hypothetical treatment has a risk difference of $-5$ percentage points in each of two strata, and this conditional effect is stable across populations. The source population is 70% in stratum C and 30% in stratum D. The target is 20% in stratum C and 80% in stratum D.

The source effect is

\Delta_S=0.70(-5)+0.30(-5) =-3.5-1.5=-5\text{ percentage points}.

The target effect is

\Delta_T=0.20(-5)+0.80(-5) =-1.0-4.0=-5\text{ percentage points}.

The distributions differ substantially, but the average effects are equal because both strata have the same conditional effect. A distribution difference matters for this additive average only when it reweights different conditional effects.

A Short Derivation for a Binary Modifier

When $V$ is binary, denote its values by 0 and 1, and let $p_g=P(V=1\mid G=g)$. If conditional effects are stable, denote them by $\tau(0)$ and $\tau(1)$. The population average in group $g$ is

\Delta_g=(1-p_g)\tau(0)+p_g\tau(1).

Subtract the source average from the target average and collect terms:

\begin{aligned} \Delta_T-\Delta_S &=(1-p_T)\tau(0)+p_T\tau(1)\\ &\quad-\{(1-p_S)\tau(0)+p_S\tau(1)\}\\ &=(p_T-p_S)\tau(1)-(p_T-p_S)\tau(0)\\ &=(p_T-p_S)\{\tau(1)-\tau(0)\}. \end{aligned}

This expression makes two requirements for a composition-driven difference explicit. The proportion with $V=1$ must differ between populations, and the conditional effects must differ between $V=1$ and $V=0$. If either factor is zero, the population-average effects are equal under the stated stability condition.

Interpretation and Scope

Effect modification is defined relative to a chosen effect scale. The examples use the risk-difference scale, where effects are differences in outcome risks. Conditional effects may vary across groups on that scale without varying in the same way on a relative scale, such as a risk ratio. Therefore, a claim that effects are stable or modified should name the scale and the outcome contrast being compared.

The composition identity is a statement about estimands. It does not show that conditional effects are actually stable, nor does it tell us whether they can be learned from available data. Those are separate scientific and identification questions. In particular, knowing the source effect and the target distribution of a covariate is not enough to establish the target effect unless the relevant conditional causal effects are appropriately comparable across populations.

The distinction matters in practice because an average effect describes a particular population. An effect estimated for a trial sample need not equal the average effect for a target population, even when treatment effects within clinically relevant strata are unchanged. Reporting the conditional effects and the relevant population proportions can make the reason for an average difference transparent. It also prevents a population-average contrast from being mistaken for a universal effect applying equally to every group.

Common Mistakes

Confusing stable conditional effects with equal average effects. Stability within strata does not imply equal marginal effects if the strata have different proportions and different effects.
Dropping the sign of the effect. With $\Delta_g=E[Y^1-Y^0\mid G=g]$, a treatment that reduces an adverse outcome has a negative risk difference. If reporting a positive reduction magnitude, define it separately as $R_g=-\Delta_g$.
Calling every population difference effect modification. A covariate distribution difference changes the average through this mechanism only when conditional effects vary with that covariate on the chosen scale.
Treating the weighted-average identity as proof of transportability. The identity describes what follows if the conditional causal effects are stable and the distributions are known; it does not establish those conditions or identify effects from observed data.

Key takeaway: Under stable conditional causal effects, a source-target difference in average effects can arise entirely from different distributions of effect modifiers. The direction and size of that difference depend on both how the populations differ in composition and how the conditional effects vary across modifier levels.

Check Your Understanding

Use the definitions and calculations above to answer the following questions.

On the risk-difference scale, treatment lowers adverse-outcome risk by 3 percentage points in one stratum and 9 points in another. What are the corresponding values of $E[Y^1-Y^0\mid V=v]$?
If the target population has a larger proportion in the stratum with the more negative conditional effect, what direction of change in the average risk difference would you expect, assuming conditional effects are stable?
For a binary modifier, suppose $p_T=p_S$ but the conditional effects differ between modifier levels. What does the binary-modifier decomposition imply about $\Delta_T-\Delta_S$?
For a binary modifier, suppose $p_T\ne p_S$ but $\tau(1)=\tau(0)$. What does the decomposition imply about the population-average effects?
Why should a report specify the effect scale when describing whether treatment effects are modified across strata?