Model Effect Modification in Observational Data

Why Model Effect Modification?

An exposure may have different associations with an outcome in different groups. For example, a program might be associated with a larger change in symptom score among people with severe baseline symptoms than among people with mild symptoms. A single exposure coefficient can conceal this pattern. Regression can represent it with an exposure-by-covariate interaction, which allows the exposure contrast to vary across covariate values.

In observational data, however, a fitted interaction is first a feature of the model for observed outcomes. It is not automatically evidence that the exposure has different causal effects across groups. That interpretation also depends on how the exposure groups compare, which population is being described, and whether the causal identification assumptions are credible.

Definition: Effect modification occurs when an exposure contrast differs across levels or values of a covariate, on a specified outcome scale. For a binary exposure \(A\), outcome \(Y\), and covariate \(Z\), the causal additive effect in stratum \(Z=z\) is \(E[Y(1)-Y(0)\mid Z=z]\). It is modified by \(Z\) on the additive scale if this contrast varies with \(z\). The potential outcome \(Y(a)\) is the outcome that would be observed under exposure level \(a\).

The scale matters. A difference in mean outcomes is an additive contrast; a ratio of means or risks is a multiplicative contrast. Effects may vary across strata on one scale but not another. Therefore, a statement such as “there is effect modification” is incomplete unless it identifies the contrast being compared.

A Regression Model With an Interaction

Begin with a binary exposure \(A\), a binary baseline covariate \(Z\), and a quantitative outcome \(Y\). Code \(A=1\) for exposed and \(A=0\) for unexposed, and code \(Z=1\) and \(Z=0\) for its two strata. A linear model for the conditional mean is

E(Y\mid A,Z)=\beta_0+\beta_A A+\beta_Z Z+\beta_{AZ}AZ.

The product \(AZ\) is the interaction term. The model says that the expected outcome is linear in these coded predictors and permits the exposure contrast to differ by \(Z\). Its conditional means are

\begin{aligned} E(Y\mid A=0,Z=0)&=\beta_0,\\ E(Y\mid A=1,Z=0)&=\beta_0+\beta_A,\\ E(Y\mid A=0,Z=1)&=\beta_0+\beta_Z,\\ E(Y\mid A=1,Z=1)&=\beta_0+\beta_A+\beta_Z+\beta_{AZ}. \end{aligned}

Subtracting the unexposed mean from the exposed mean within each stratum gives the model-based contrasts:

\begin{aligned} \Delta_0&=E(Y\mid A=1,Z=0)-E(Y\mid A=0,Z=0)=\beta_A,\\ \Delta_1&=E(Y\mid A=1,Z=1)-E(Y\mid A=0,Z=1)=\beta_A+\beta_{AZ}. \end{aligned}

Thus, \(\beta_A\) is the contrast for \(Z=0\), not an overall exposure effect. The interaction coefficient \(\beta_{AZ}\) is the difference between the two stratum-specific contrasts, \(\Delta_1-\Delta_0\). The main-effect coefficient \(\beta_Z\) compares the covariate strata among the unexposed; it is not an exposure effect.

Worked Example: Reading Stratum-Specific Contrasts

Suppose a hypothetical observational analysis models 12-week symptom score \(Y\), measured in points, using exposure to a support program \(A\) and an indicator \(Z\) for high baseline symptom severity. The fitted conditional-mean equation is

\widehat{E}(Y\mid A,Z)=50-4A+1.5Z-3AZ.

For participants with lower baseline severity (\(Z=0\)), the fitted contrast is \(\hat{\Delta}_0=\hat{\beta}_A=-4\) points. The model predicts a score 4 points lower for exposed than unexposed participants in this stratum.

For participants with high baseline severity (\(Z=1\)), the contrast is \(\hat{\Delta}_1=\hat{\beta}_A+\hat{\beta}_{AZ}=-4+(-3)=-7\) points. The fitted exposure contrast is 7 points lower in this stratum.

The difference between the contrasts is \(\hat{\Delta}_1-\hat{\Delta}_0=-7-(-4)=-3\) points, equal to the interaction coefficient. These are model-based associations. The calculations alone do not establish that the program caused either score difference.

More Covariates and Population-Average Contrasts

In an observational analysis, a model will often include adjustment covariates \(C\) as well as the possible effect modifier \(Z\). For instance, a model might include an exposure-by-\(Z\) product while adjusting for age and baseline health. With continuous \(Z\), a linear exposure interaction allows the exposure contrast to change linearly with \(Z\); more flexible functions may be needed if that form is implausible.

A conditional contrast at a particular covariate profile is not necessarily the contrast for a whole population. To describe a target population, predict each person’s outcome under each exposure level, keep that person’s covariates fixed, calculate the difference, and average those differences over the target population. This is standardization. It makes the population being averaged over explicit.

Definition: Let \(C\) denote measured covariates and let \(m(a,z,c)=E(Y\mid A=a,Z=z,C=c)\) be the fitted conditional mean. A standardized contrast for stratum \(Z=z\), over a specified target distribution of \(C\), is the average of \(m(1,z,C)-m(0,z,C)\) in that distribution. An overall standardized contrast also averages over the target distribution of \(Z\).

Worked Example: Standardizing Contrasts Across Covariate Profiles

Consider a hypothetical regression model for a quantitative outcome, in units called points. Let \(A\) be a binary exposure, \(Z\) a binary baseline stratum, and \(C\) a binary adjustment covariate. Suppose the fitted model is

\widehat{m}(A,Z,C)=10+2A+Z+4C+3AZ-AC.

There is no \(ZC\) or three-way interaction in this model. Its exposure contrast at a particular \(Z,C\) profile is \(2+3Z-C\): subtracting the prediction at \(A=0\) from the prediction at \(A=1\) cancels the terms that do not involve \(A\).

For \(Z=0,C=0\), the predictions are 10 points when \(A=0\) and \(12\) when \(A=1\), giving a contrast of \(12-10=2\) points. For \(Z=0,C=1\), they are \(14\) and \(15\), giving \(15-14=1\) point. Suppose the target population within \(Z=0\) is 60% \(C=0\) and 40% \(C=1\). Its standardized contrast is

0.60(2)+0.40(1)=1.60\text{ points}.

For \(Z=1,C=0\), the predictions are \(11\) and \(16\), so the contrast is \(5\) points. For \(Z=1,C=1\), they are \(15\) and \(19\), so the contrast is \(4\) points. If the target population within \(Z=1\) is 25% \(C=0\) and 75% \(C=1\), its standardized contrast is \(0.25(5)+0.75(4)=4.25\) points.

Finally, suppose the target population is 70% \(Z=0\) and 30% \(Z=1\). The overall standardized contrast is

0.70(1.60)+0.30(4.25)=1.12+1.275=2.395\text{ points}.

This overall result depends on the stated target-population weights. Using a different distribution of \(Z\) or \(C\) can give a different average, even when the fitted model is unchanged.

When Can Heterogeneity Be Interpreted Causally?

The regression contrasts above describe fitted differences in observed conditional means. To interpret them as contrasts between potential outcomes, the analysis needs assumptions for causal identification. In an observational setting, adjusting for measured covariates is useful only if the chosen covariates and model support a valid comparison.

Consistency and a well-defined exposure. For a person who receives exposure level \(a\), the observed outcome equals \(Y(a)\). The exposure levels must be defined clearly enough that these potential outcomes are meaningful.
No interference. One person’s exposure does not change another person’s outcome, unless such spillovers are explicitly included in the causal question.
Conditional exchangeability. Within levels of the measured adjustment covariates, exposure assignment is independent of the potential outcomes: \(Y(a)\) is independent of \(A\) given those covariates. If this fails because relevant confounding remains, a fitted interaction can reflect confounding rather than causal effect modification.
Positivity. Each exposure level has a positive probability within every covariate profile represented in the target population. Without exposed and unexposed observations in a stratum, the within-stratum contrast cannot be learned from that stratum’s observed data without extrapolation.
A suitable outcome model and target population. The conditional mean model must estimate the needed predictions adequately, and the averaging weights must represent the population to which the contrast is intended to apply.

These conditions concern different issues: causal assumptions justify identifying potential-outcome contrasts from observed data, while model adequacy concerns estimating those contrasts with the chosen regression. A well-fitting model cannot repair unmeasured confounding or a lack of overlap. Conversely, valid causal assumptions do not guarantee that an overly restrictive interaction model captures the relevant pattern.

The adjustment set should use covariates measured before exposure when the goal is to estimate total exposure effects. Adjusting for a variable affected by exposure can change the estimand or introduce bias; covariate choice should follow the causal question rather than a search for whichever model yields a preferred interaction.

Worked Example: An Interaction Estimate Is Not a Causal Conclusion

Suppose a hypothetical study finds fitted score contrasts of \(-2\) points for one baseline group and \(-6\) points for another. The estimated difference in contrasts is \(-6-(-2)=-4\) points. This is evidence, within the fitted model, that the observed exposure association differs between groups on the additive score scale.

Now suppose an important pre-exposure health factor was not measured and is more common among exposed people in the second group. If that factor also predicts symptom scores, exposed and unexposed people may not be comparable within that group. Conditional exchangeability then fails for the measured adjustment set. The \(-4\)-point difference could partly or entirely reflect that imbalance, so it cannot be interpreted as a causal difference merely because it is an interaction coefficient.

A causal interpretation would require a defensible adjustment set that controls confounding, adequate exposure overlap in both groups and relevant covariate profiles, a suitable model, and a clearly stated target population. Even if these conditions are plausible, the result should be described as an estimated causal contrast with uncertainty, not as proof that the exposure has a different effect in every individual.

Uncertainty, Scale, and Common Pitfalls

An interaction estimate has sampling uncertainty. In the binary-\(Z\) model, \(\hat{\beta}_{AZ}\) estimates the difference between the two fitted stratum-specific contrasts. A confidence interval for this difference should use its standard error, which accounts for the uncertainty and covariance of the fitted coefficients. A small interaction estimate does not establish identical effects; a wide interval may include both little difference and differences that matter in practice. Likewise, statistical evidence for an interaction does not by itself establish causal heterogeneity.

Interaction also depends on the scale. In the linear mean model, the product term represents a difference in mean differences. For a binary outcome, a model on the risk scale describes differences in risks, while a model on a different scale describes a different contrast. Always state what is being compared and in what units.

Mistake: interpreting \(\beta_A\) as the overall effect when an interaction is present. In the binary-\(Z\) model, \(\beta_A\) is the contrast specifically at \(Z=0\). Calculate \(\beta_A+\beta_{AZ}\) for \(Z=1\), then standardize if an overall target-population contrast is needed.
Mistake: treating an interaction coefficient as automatically causal. It describes a modeled difference in observed conditional associations. Causal interpretation requires the identification assumptions and a suitable model.
Mistake: reporting an average without saying which population was averaged over. State the target population or the weights used; averages can change when its covariate distribution changes.
Mistake: assuming a non-significant interaction proves no effect modification. A test may have limited precision. Report the stratum-specific estimates and their uncertainty, and distinguish lack of strong evidence from evidence of no important difference.

Key takeaway: An exposure-by-covariate term allows a regression contrast to vary across covariate strata. Calculate the stratum-specific contrasts from the model, average predictions over an explicitly defined target population when needed, and reserve causal language for analyses whose assumptions are defensible.

Check Your Understanding

Use the model and ideas in this tutorial to answer the following questions.

In a model with a binary exposure \(A\), binary covariate \(Z\), and product term \(AZ\), what is the exposure contrast for \(Z=1\)?
In the model \(\widehat{E}(Y\mid A,Z)=30+5A-2Z+4AZ\), calculate the fitted exposure contrasts for both values of \(Z\), and state their difference in outcome units.
Why can an overall standardized contrast change when the target population’s distribution of the effect-modifying covariate changes?
Name two assumptions needed to interpret an observational stratum-specific contrast causally, and briefly explain what each protects against.
Why does a fitted interaction not, by itself, demonstrate causal effect modification?