Data-Adaptive Subgroup Discovery

Why Subgroup Discovery Is Data-Adaptive

A subgroup analysis asks whether treatment effects differ across groups of units. The groups might be defined by age, baseline severity, or combinations of measured characteristics. When the groups are specified before outcomes are examined, their effects can be estimated and compared directly. In data-adaptive subgroup discovery, the observed outcomes also help decide which groups to examine. That flexibility can reveal patterns missed by a short prespecified list, but it creates a serious interpretive hazard: the group that looks most promising in the discovery data may look less impressive in new data.

The issue is not only computational. A selected subgroup is a causal target only if it is defined using appropriate covariates and the treatment comparison identifies causal effects. Even then, searching over many possible groups can make the selected estimate optimistic. It is useful to keep these two questions separate: does the comparison identify a causal effect for the group? and does the reported estimate honestly reflect the search that selected the group?

Definition: Let $X$ denote pretreatment covariates and let $Y(1)$ and $Y(0)$ be the potential outcomes under treatment and control. A covariate-defined subgroup is $G=\{X\in\mathcal{G}\}$, for a rule $\mathcal{G}$ specified using $X$. Its average treatment effect is $$ \tau_G=E[Y(1)-Y(0)\mid X\in\mathcal{G}]. $$ A subgroup discovery procedure searches over a collection of such rules and uses the observed data to choose one or more groups.

For a causal interpretation, treatment and control outcomes must be comparable within the target group. In a randomized trial, random assignment provides this comparability, subject to consistency and no interference between units. In observational data, a standard sufficient set of assumptions is conditional exchangeability, $ (Y(1),Y(0))\perp W\mid X$, positivity, $0<P(W=1\mid X)<1$, and consistency, where $W$ is treatment assignment. A subgroup should be defined from pretreatment characteristics; defining it using a post-treatment variable can change the estimand and introduce bias.

These assumptions address identification, not the search itself. Randomization does not prevent a large search from selecting an unusually favorable noise pattern. Likewise, a sophisticated search algorithm cannot repair unmeasured confounding in observational data.

Estimating and Comparing Subgroup Effects

In a randomized trial, a simple estimator for a fixed subgroup is the treated mean minus the control mean within that group. Its causal interpretation follows because assignment is randomized and the subgroup membership is determined from pretreatment covariates. For observational data, adjustment for covariates may be needed; the appropriate estimator depends on the design and assumptions.

Evidence that treatment is effective in one subgroup is not, by itself, evidence that its effect differs from the effect in another. The relevant comparison is the difference between subgroup effects. For two groups $G_1$ and $G_2$, heterogeneity on the mean-difference scale is represented by $\tau_{G_1}-\tau_{G_2}$. This distinction matters because one subgroup can have a statistically clearer estimate simply because it is larger or less variable, even when its underlying effect is no different.

Worked Example: Estimating Effects in a Prespecified Split

Consider a hypothetical randomized trial of a treatment intended to improve a test score, measured in points. Before examining outcomes, investigators define a subgroup as participants aged 50 years or younger. Among 40 treated participants in this subgroup, the mean score improvement is 12.4 points; among 40 control participants, it is 10.1 points. In the older subgroup, the corresponding means are 15.2 points among 40 treated participants and 14.1 points among 40 controls.

Because assignment is randomized and age is measured before treatment, the within-subgroup mean contrasts estimate the subgroup average treatment effects:

\widehat{\tau}_{\text{younger}}=12.4-10.1=2.3\text{ points}, \qquad \widehat{\tau}_{\text{older}}=15.2-14.1=1.1\text{ points}.

The estimated difference in effects is $2.3-1.1=1.2$ points. The data therefore suggest a larger mean treatment effect in the younger subgroup by 1.2 points. This arithmetic does not establish that the effects truly differ: uncertainty in both estimates must be considered. Nor does a comparison of separate significance tests answer the heterogeneity question; the target contrast is the difference between the two estimated effects.

Tree-Based Searches

A tree-based search recursively divides the covariate space. At each step it considers candidate splits, such as age at a threshold or baseline severity above a threshold, and evaluates whether the resulting leaves appear to have different treatment effects. A common broad objective is to reward between-leaf variation in estimated effects, sometimes with additional penalties or adjustments for uncertainty and sample size. The precise splitting criterion varies by method, but the core task is to choose covariate-defined groups that appear to separate treatment effects.

Trees can represent interactions and nonlinear boundaries without requiring the analyst to specify every combination in advance. Their leaves also yield readable rules, such as “age 50 or younger and baseline severity above a threshold.” But a search that repeatedly tests possible splits on the same outcomes can favor chance differences. Small leaves are especially vulnerable to noisy estimates, and a greedy split selected early can affect all subsequent splits. Earlier material on causal forests and honest causal forests develops related adaptive partitioning and sample-separation ideas; here the concern is the subgroup search and the interpretation of its selected pattern.

Worked Example: Comparing Two Candidate Tree Splits

Suppose a hypothetical randomized trial has 80 participants, with 40 treated and 40 controls. Consider two possible splits, each producing leaves of 40 participants. For candidate split A, the estimated treatment effects are 4 points in the left leaf and 0 points in the right leaf. For candidate split B, they are 3 points and 1 point. Assume the overall estimate is 2 points for either balanced split.

For illustration, use the between-leaf score $Q=\sum_g (n_g/n)(\widehat{\tau}_g-\widehat{\tau})^2$, where $n_g$ is the number of participants in leaf $g$, $n=80$, and $\widehat{\tau}=2$ is the overall estimate. For split A:

Q_A=\frac{40}{80}(4-2)^2+\frac{40}{80}(0-2)^2 =0.5(4)+0.5(4)=4.

For split B:

Q_B=\frac{40}{80}(3-2)^2+\frac{40}{80}(1-2)^2 =0.5(1)+0.5(1)=1.

Under this illustrative criterion, split A is preferred because its estimated effects are more separated. This score is a search device, not a test proving that the true effects differ. If many possible thresholds and covariates were compared on these same outcomes, the winning score could partly reflect random noise. The calculation also assumes the proposed splits yield the stated leaf sizes and estimates; in an actual analysis those values must be computed from the trial data.

Score-Based Searches

A score-based approach first constructs a quantity whose conditional expectation represents a treatment effect, then searches for covariate-defined groups with high or low average scores. One simple illustration is available in a randomized trial with treatment probability $e(X)$, known from the design. Under consistency, randomization, and positivity, define the inverse-probability score

\Gamma=\frac{WY}{e(X)}-\frac{(1-W)Y}{1-e(X)}.

Conditional on $X$, the treated term has expectation $E[Y(1)\mid X]$, and the control term has expectation $E[Y(0)\mid X]$. Therefore $E[\Gamma\mid X]=E[Y(1)-Y(0)\mid X]$. Averaging $\Gamma$ over a covariate-defined group targets that group’s average treatment effect. This score can be noisy for an individual, especially when treatment probabilities are near zero or one, so score-based searches generally require care in their construction and evaluation. More elaborate methods use adjusted or orthogonal pseudo-outcomes, as covered in earlier material on orthogonal pseudo-outcomes and cross-fitting.

A score-based search may rank units by estimated conditional effects, scan candidate thresholds in that ranking, or evaluate a predefined family of subgroup rules using average scores or related statistics. Compared with a tree, it can use an existing effect score to organize the search and may be convenient for evaluating many candidate rules. Its results depend on the quality of that score and on the candidate groups examined. Neither approach eliminates the need to account for the fact that the same outcomes may have guided selection.

Worked Example: A Score Search and Selection Optimism

In a hypothetical randomized trial, suppose a score procedure produces estimates for two candidate subgroups. Both true subgroup effects are 1.0 point, but because of sampling noise the estimated score averages are 1.8 points for group A and 0.7 points for group B. If the discovery rule selects the group with the largest estimated effect, group A is chosen and its reported discovery estimate is 1.8 points—0.8 points above its true effect in this hypothetical setup.

A simple calculation shows why selection can create optimism even when each candidate estimate is unbiased before selection. Suppose two estimates have the same true effect $\tau$, and their errors $\epsilon_1,\epsilon_2$ are independent $N(0,\sigma^2)$. The selected estimate is $\tau+\max(\epsilon_1,\epsilon_2)$. Using $\max(a,b)=(a+b+|a-b|)/2$:

E[\max(\epsilon_1,\epsilon_2)] =\frac{1}{2}E[|\epsilon_1-\epsilon_2|] =\frac{\sigma}{\sqrt{\pi}}>0.

For example, if $\sigma=1.5$ points, the expected excess from selecting the larger of these two estimates is $1.5/\sqrt{\pi}\approx0.846$ points. The independence and equal-effect assumptions make this a simplified illustration, not a universal bias formula. With overlapping subgroups, estimates will generally be correlated, and a larger or more complex search changes the amount of optimism. The essential point remains: choosing the maximum uses the noise as well as the signal.

How to Interpret a Discovered Subgroup

The discovery result should be described as a data-selected pattern, not as confirmation that a treatment works only for the selected group. If the search used outcomes to choose a threshold, those same outcomes cannot be treated as if the threshold had been fixed in advance when assessing how strong the evidence is. Reporting only the selected effect hides the search process and can make a chance pattern appear stable.

A practical design separates discovery from assessment. For example, one portion of the data can be used to choose a subgroup rule, and an independent portion can estimate the effect for that fixed rule. The assessment sample must not have influenced the choice of rule. This separation reduces the particular optimism caused by reusing discovery outcomes, although it does not guarantee high precision, eliminate confounding, or make a poorly chosen subgroup substantively meaningful. In observational work, the causal identification assumptions still apply in each stage.

Tree and score procedures also answer somewhat different practical questions. A tree directly proposes a sequence of covariate rules; a score-based method uses treatment-effect scores to search or rank candidate groups. Both can be useful for exploration, but both should report the search strategy, the variables and thresholds considered, and how the selected group was assessed. The goal is not merely to find a striking estimate, but to establish a credible, interpretable difference in treatment effects.

Common Mistake

Pitfall: “The subgroup with the largest estimated effect must be the subgroup that benefits most.” This treats the selected estimate as though its group had been specified independently of the outcomes. The correction is to recognize that selection favors estimates with positive noise as well as genuine signal. State how the subgroup was discovered, distinguish exploratory estimates from independent assessment, and interpret the result under the causal assumptions and uncertainty appropriate to the design.

Check Your Understanding

Use these questions to check both the causal and the adaptive parts of subgroup discovery.

Write the definition of the average treatment effect for a covariate-defined subgroup $G$. Why should subgroup membership generally be based on pretreatment covariates?
In a randomized trial, one subgroup has an estimated effect of 5 points and another has an estimated effect of 2 points. What contrast addresses whether the effects differ?
What does a tree-based splitting score summarize, and why is a large score not by itself evidence that the true subgroup effects differ?
In a randomized trial with known treatment probability, why can the conditional expectation of the inverse-probability score represent the conditional treatment effect?
Explain, in your own words, why selecting the largest of several unbiased estimates can produce an optimistic reported estimate.