Qualitative and Quantitative Effect Modification

Why the Direction of an Effect Matters

A treatment can have effects that differ across subgroups in more than one way. It might help every subgroup, but help some more than others. Or it might help one subgroup and harm another. The first pattern is a difference in magnitude; the second is a difference in direction. Distinguishing them matters when describing who may benefit, who may be harmed, and what a subgroup analysis actually shows.

This distinction is defined relative to a causal contrast and its null value. For a risk difference, the null is zero; for a risk ratio, it is one. Before calling effects qualitative or quantitative, specify the outcome, the treatment contrast, the subgroup, and the effect scale. A pattern classified on one scale need not have the same magnitude pattern on another.

Definition: Quantitative effect modification occurs when causal effects differ in magnitude across subgroups but do not reverse direction: the subgroup effects lie on the same side of the null value of the chosen contrast. Qualitative effect modification (also called qualitative heterogeneity or a crossover pattern) occurs when effects lie on opposite sides of that null, indicating opposite directions in different subgroups. An effect exactly at the null is a boundary case, not by itself a reversal.

Define the Subgroup-Specific Causal Effect First

Let \(A\) denote treatment assignment, with \(A=1\) for treatment and \(A=0\) for control. Let \(S\) be a subgroup characteristic measured before treatment, and let \(Y(1)\) and \(Y(0)\) be the outcomes the same individual would have under treatment and control. In the usual no-interference setting, the average causal effect in subgroup \(s\), on the additive scale, is

\tau_s = E[Y(1)-Y(0)\mid S=s].

This is a contrast between the subgroup's average potential outcome under treatment and its average potential outcome under control. It is not, in general, the observed difference between whichever people happened to receive treatment and control. For a binary adverse outcome, one useful contrast is the subgroup risk difference:

\tau_s = P(Y(1)=1\mid S=s)-P(Y(0)=1\mid S=s).

Here negative values mean treatment lowers the risk of the adverse outcome, and positive values mean it raises the risk. The null is zero. If the outcome is one for a favorable event instead, the substantive interpretation of positive and negative values changes; the coding must be stated.

A subgroup-specific ratio has a different null. For example, for an adverse binary outcome with positive control risk, the risk ratio is \(RR_s=P(Y(1)=1\mid S=s)/P(Y(0)=1\mid S=s)\), whose null is one. Values below one indicate lower risk under treatment, while values above one indicate higher risk. The direction must always be interpreted against the outcome coding and the contrast's null, not by looking at the number alone.

Identification conditions: A causal interpretation requires well-defined treatment conditions and consistency: an individual's observed outcome equals the potential outcome under the treatment actually received. In a randomized study, treatment must be randomized within the subgroup, so assignment is independent of the potential outcomes conditional on \(S=s\), and both treatment conditions must be possible in each subgroup. In observational data, conditional exchangeability given measured covariates, positivity, and appropriate adjustment are needed instead. If one person's treatment can affect another person's outcome, the potential outcomes must also specify the relevant exposure mapping; the simple notation above assumes no interference.

From Potential Outcomes to Subgroup Contrasts

The distinction between the causal target and its estimate is important. In a randomized study, the difference in observed mean outcomes by assignment within a subgroup estimates the corresponding causal effect. The justification is short. Under randomization within \(S=s\), \(A\) is independent of the potential outcomes in that subgroup. Consistency then equates the observed outcome with the relevant potential outcome in each arm:

E[Y\mid A=1,S=s]-E[Y\mid A=0,S=s] =E[Y(1)\mid S=s]-E[Y(0)\mid S=s] =E[Y(1)-Y(0)\mid S=s].

Thus, the observed within-subgroup difference identifies \(\tau_s\) in the randomized setting under the stated conditions. In a finite sample, it is an estimate and may differ from the population effect through sampling variability. A descriptive difference between subgroups in observed outcomes is not enough: treatment allocation and the identification assumptions determine whether the within-subgroup contrasts have a causal interpretation.

Worked Example: Different Magnitudes, Same Direction

In a hypothetical randomized trial, a higher score on a functional outcome is better. Treatment is randomized within each of two prespecified baseline subgroups, and the mean score is recorded at follow-up. Suppose there are 40 treated and 40 control participants in each subgroup, with these sample means:

Subgroup	Mean under treatment	Mean under control
Group A	62 points	58 points
Group B	71 points	70 points

For Group A, the estimated treatment contrast is \(62-58=4\) points. For Group B, it is \(71-70=1\) point. Both estimates are positive, so both point in the favorable direction on this outcome scale. Their magnitudes differ by \(4-1=3\) points. This is a non-reversing, quantitative pattern: the estimated benefit is larger in Group A, but there is no estimated direction reversal.

Because assignment was randomized within each subgroup, the within-subgroup mean differences estimate subgroup-specific causal effects, subject to the usual consistency and no-interference conditions. The calculations do not show that the true effects are exactly 4 and 1 points; they are sample estimates. Nor do they establish that the difference in effects is precisely 3 points in the target population.

Worked Example: A Reversal in Adverse-Event Risk

Consider a hypothetical randomized study of a treatment for an adverse event, where lower event risk is better. Within each of two baseline subgroups, 100 people are assigned to treatment and 100 to control. Suppose the observed event counts are:

Subgroup	Treatment events	Control events
Group A	20 of 100	30 of 100
Group B	28 of 100	20 of 100

For Group A, the estimated treatment risk is \(20/100=0.20\), and the control risk is \(30/100=0.30\). The estimated risk difference is

\widehat{\tau}_A=0.20-0.30=-0.10.

This is a 10-percentage-point lower adverse-event risk under treatment in Group A. For Group B, the estimated risks are \(28/100=0.28\) under treatment and \(20/100=0.20\) under control, so

\widehat{\tau}_B=0.28-0.20=0.08.

This is an 8-percentage-point higher adverse-event risk under treatment in Group B. The estimates lie on opposite sides of the risk-difference null, zero. They therefore show a qualitative, direction-reversing pattern: treatment appears beneficial in Group A and harmful in Group B. The interpretation is causal only because the example stipulates randomization within each subgroup, along with well-defined treatment, consistency, and no interference. The estimates alone do not establish how certain the population effects are.

Zero Effects and the Boundary Between Patterns

Not every difference in subgroup effects is a strict reversal. Suppose one subgroup's effect is positive and another's is exactly zero. There is a difference in magnitude, but the zero effect does not point in the opposite direction. It is therefore a boundary case rather than qualitative effect modification. This distinction is useful because “not beneficial in both groups” does not necessarily mean “harmful in one group.”

Worked Example: One Subgroup at the Null

In a hypothetical randomized study, larger values on a continuous outcome are better. Treatment is randomized within two baseline subgroups, with 60 participants in each arm of each subgroup. Suppose the mean outcome is 12 points under treatment and 9 under control in Group A, and 8 points under both treatment and control in Group B.

The estimated effects are \(12-9=3\) points in Group A and \(8-8=0\) points in Group B. The first is favorable and the second is at the null. This is not a qualitative reversal: no subgroup estimate indicates harm. It is a difference in magnitude that reaches the null in Group B. As in the other examples, randomization within subgroup supports a causal interpretation of the contrasts, but finite-sample estimates do not prove that the population effect in Group B is exactly zero.

A Practical Classification

A careful classification starts by fixing the causal contrast and its null value. Then compare the subgroup-specific causal effects, not just outcome levels or treatment-arm averages. The sequence below helps keep the direction and magnitude questions separate.

Specify the outcome and contrast.
State how the outcome is coded and whether the effect is, for example, a mean difference, risk difference, or risk ratio.

Identify the null and direction.
For a difference, the null is zero; for a ratio, it is one. Decide which side represents benefit or harm in context.

Compare subgroup-specific causal contrasts.
Effects on the same side of the null but of different magnitudes are non-reversing; effects on opposite sides indicate qualitative heterogeneity.

Check identification before interpreting causally.
Use randomization within subgroup or justify the observational exchangeability, positivity, and consistency conditions needed for the contrasts.

This classification is about the pattern of causal effects on the chosen scale. It is not determined by whether treatment and control outcomes differ across subgroups, because baseline outcome levels can differ even when treatment effects are equal. It is also not a claim that an observed pattern must be real: estimates can be noisy, and uncertainty must be considered before drawing firm conclusions.

Common Mistakes and Why They Matter

Calling any difference in magnitude a reversal. If estimated effects are \(+4\) and \(+1\), they differ in size but are both on the same side of the null. The correction is to check direction first, then describe magnitude.
Using the wrong null value. A ratio of 1 is the no-effect value, not zero. Compare ratios with 1 and differences with 0.
Comparing subgroup outcome levels instead of effects. One subgroup may have higher risks under both treatment and control while having the same treatment effect. Compare the treatment-versus-control contrast within each subgroup.
Treating observed subgroup differences as causal automatically. If treatment assignment is confounded within a subgroup, its observed treatment-control contrast need not equal the subgroup causal effect. State and justify the identification conditions.
Defining subgroups after treatment. A post-treatment characteristic can be affected by treatment, so comparisons within its levels need not represent effects in baseline-defined populations. Define subgroup membership before treatment for the estimand described here.
Equating an estimate at the null with proof of no effect. A sample contrast of zero may reflect sampling variability. It is a point estimate, not proof that the population effect is exactly null.

Key takeaway: Quantitative effect modification means subgroup effects differ in magnitude without reversing direction; qualitative effect modification means they fall on opposite sides of the chosen contrast's null. These labels describe causal subgroup contrasts only when those contrasts are identified, and the scale and outcome coding must be explicit.

Check Your Understanding

For each question, state the contrast and direction carefully before classifying the pattern.

On an outcome where higher values are better, subgroup estimates are \(+5\) and \(+2\) points. Is this a reversal? What kind of pattern does it suggest?
For an adverse event, subgroup risk differences are \(-0.06\) and \(+0.03\). What does each sign mean, and how is the pattern classified?
A subgroup risk ratio is 0.8. What is its null value, and what does 0.8 indicate when the outcome is adverse?
One subgroup has an estimated effect of \(-2\) points and another has an estimated effect of zero. Why is this not a strict direction reversal?
Why is a difference in outcome levels between subgroups insufficient to establish effect modification?
Name one condition needed to interpret observed treatment-control contrasts within randomized subgroups as estimates of causal effects.