Fixed Baseline Subgroup Estimands

Why Fix the Subgroup Before Treatment?

A trial may ask whether treatment effects differ for patients with a particular baseline characteristic, such as biomarker status or disease severity. The subgroup target must be defined using information available before randomization. This makes membership a stable property of the trial population: treatment cannot change who belongs to the subgroup. The question is then precise: what is the average effect of the specified treatment contrast among participants in this fixed baseline subgroup?

This is different from comparing participants grouped by what happened after randomization—for example, those who responded, discontinued treatment, or experienced an adverse event. Such events can be affected by treatment. A comparison within their observed categories may therefore compare different sets of people in the treatment arms, rather than estimate an effect in one fixed subgroup.

The earlier material on standardizing effects to a fixed population provides the general idea of averaging conditional effects over a specified population distribution. Here the target is a particular baseline-defined subgroup, and care is needed if randomization probabilities vary across strata within that subgroup.

Definition: Let $S$ be a characteristic measured before randomization, and let $s$ denote a specified subgroup value or category (for example, biomarker-positive). Let $Y(1)$ and $Y(0)$ be the outcomes that would be observed under the two treatment conditions defined for the estimand. The fixed baseline subgroup treatment effect on the mean-difference scale is $$ \Delta_s = E[Y(1)\mid S=s]-E[Y(0)\mid S=s] =E[Y(1)-Y(0)\mid S=s]. $$ The subgroup rule, treatment conditions, outcome, summary measure, and relevant intercurrent-event strategy must be specified. The subgroup rule is fixed before treatment assignment and does not depend on a post-randomization outcome.

For a binary outcome, the same definition can use a risk difference: the probability of the outcome under treatment minus the probability under control, conditional on $S=s$. A ratio-scale measure can also be defined, but its scale must be stated because treatment-effect heterogeneity can look different on different scales. The examples below use mean differences.

Identification and Randomization Strata

Write $A$ for randomized treatment assignment. If assignment is randomized within the subgroup and both treatment conditions have positive assignment probability there, then the observed mean in each arm within that subgroup identifies its corresponding potential-outcome mean. In symbols, conditional randomization and consistency give $E[Y\mid A=a,S=s]=E[Y(a)\mid S=s]$, for $a=0,1$. Consistency means that a participant assigned to condition $a$ has observed outcome $Y=Y(a)$, under the specified treatment condition and outcome strategy.

That direct equality requires the arm comparison to represent the target subgroup's composition. If randomization probabilities vary across finer baseline strata $Z$ within $S=s$, the participants assigned to each arm can have different distributions of $Z$. The arm-specific pooled means then need not identify the marginal potential-outcome means for the target subgroup. Instead, estimate conditional means within the relevant strata and average them using the target subgroup's stratum distribution.

E[Y(a)\mid S=s] =\sum_z E[Y(a)\mid S=s,Z=z]P(Z=z\mid S=s) =\sum_z E[Y\mid A=a,S=s,Z=z]P(Z=z\mid S=s).

The second equality follows when assignment is randomized conditional on $S,Z$, treatment positivity holds within the strata used, and consistency holds. The weights $P(Z=z\mid S=s)$ describe the target subgroup, not the observed distribution of $Z$ in a particular treatment arm. For a continuous $Z$, the sum is replaced by an integral. Subtracting the standardized mean under $a=0$ from the one under $a=1$ identifies $\Delta_s$.

Identification conditions: The subgroup definition and outcome strategy are clear; treatment assignment is randomized, or otherwise conditionally exchangeable, within the baseline strata used; each treatment condition has positive probability in those strata; and observed outcomes are consistent with the assigned treatment condition. If the estimand requires outcomes that are missing, additional assumptions and an appropriate missing-data method are needed. Randomization alone does not make missing outcomes observed.

Worked Examples

Worked Example: A Prespecified Biomarker Subgroup

In a hypothetical randomized trial, the primary outcome is a symptom score at 12 weeks, with lower scores indicating fewer symptoms. Biomarker-positive status is measured before randomization. Among biomarker-positive participants, 40 are assigned to the new treatment and 38 to control. The observed mean scores are 8.2 and 10.1, respectively. Assume complete outcome measurement and random assignment within biomarker status.

The estimand is the mean 12-week score under the new treatment minus the mean score under control among participants who were biomarker-positive at baseline. In this design, assignment within the subgroup and consistency allow the two conditional arm means to estimate the corresponding subgroup potential-outcome means.

Calculation. The estimated subgroup effect is $$ \widehat{\Delta}_{+}=\bar{x}_{1,+}-\bar{x}_{0,+} =8.2-10.1=-1.9\text{ score points}. $$

The point estimate indicates that the new-treatment mean is 1.9 score points lower than the control mean in the baseline biomarker-positive subgroup. It is not, by itself, evidence that the treatment effect differs between biomarker-positive and biomarker-negative participants; that is a comparison of subgroup effects and requires its own estimate and uncertainty assessment. No inference about statistical significance follows from these means alone.

The identification argument relies on the stated randomization and complete measurement assumptions. If allocation probabilities or other randomization features vary across baseline strata within biomarker-positive participants, the pooled arm means may not target the same stratum mixture; standardization may then be needed.

Worked Example: Unequal Allocation Across Baseline Strata

Consider a hypothetical trial population in which a fixed baseline subgroup $S=s$ consists of two equally common randomization strata, $Z=L$ and $Z=H$. The outcome is measured in arbitrary units. In each stratum, the mean outcome is 0 under either treatment condition in $L$, and 10 under either condition in $H$. Randomization assigns 90% to treatment in $L$ and 10% to treatment in $H$. The table gives the resulting hypothetical counts and arm means.

Stratum	Target proportion	Treatment: n and mean	Control: n and mean
L	0.50	90; mean 0	10; mean 0
H	0.50	10; mean 10	90; mean 10

Within each stratum, the treatment-minus-control contrast is 0. Thus the fixed subgroup effect, standardized to its equal mixture of strata, is also 0. But the pooled treatment and control groups have different stratum compositions.

Pooled comparison. The treatment mean is $(90\times0+10\times10)/100=1$. The control mean is $(10\times0+90\times10)/100=9$. The unadjusted treatment-minus-control contrast is $1-9=-8$ units. It does not estimate the target subgroup effect in this example.

Standardized comparison. For either treatment condition, the target subgroup mean is $0.50\times0+0.50\times10=5$ units. The standardized contrast is $5-5=0$ units, as required by the stratum-specific means. The discrepancy arises from composition, not from a treatment effect.

The example uses fixed stratum-specific means and counts to make the arithmetic transparent. In an actual analysis, the means would be estimated with sampling variability. The key design check is whether the arm-specific means represent the target stratum distribution; conditional randomization within each stratum does not, by itself, ensure that pooled arm means do so.

Worked Example: Why an Observed-Response Group Is Not a Baseline Subgroup

In a hypothetical trial, response is assessed after randomization and is affected by treatment. The endpoint is a symptom score, with lower values better. In the treatment arm, 60 participants respond and their mean endpoint is 8; 40 do not respond and their mean is 5. In control, 40 respond with mean 9 and 60 do not respond with mean 6.

The observed comparison among responders is $8-9=-1$ score point, treatment minus control. But treatment-arm responders and control-arm responders are not necessarily the same type of participants: response occurs after assignment, and the response rates differ (60% versus 40%). Thus this contrast is not the treatment effect among a fixed subgroup defined before randomization.

For context, the overall observed treatment mean is $(60\times8+40\times5)/100=6.8$; the overall control mean is $(40\times9+60\times6)/100=7.2$. Their difference is $6.8-7.2=-0.4$ score point. The responder-only contrast of $-1$ and this overall contrast answer different questions. Neither calculation turns the post-randomization response categories into a fixed baseline subgroup.

A valid baseline subgroup analysis would instead specify a pre-randomization characteristic, such as biomarker status, and compare treatment conditions within that fixed category. Asking for effects among people who would respond under either treatment condition is a different causal target and requires additional assumptions; observed responders alone do not identify it.

A Short Justification of Standardization

The standardization equation follows by partitioning the subgroup according to baseline stratum $Z$. For either treatment condition $a$, the law of total expectation gives the first equality below. Conditional randomization and consistency give the second. Taking the difference between $a=1$ and $a=0$ yields the fixed subgroup effect:

\begin{aligned} E[Y(1)-Y(0)\mid S=s] &=\sum_z \{E[Y(1)\mid S=s,Z=z]-E[Y(0)\mid S=s,Z=z]\}P(Z=z\mid S=s)\\ &=\sum_z \{E[Y\mid A=1,S=s,Z=z]-E[Y\mid A=0,S=s,Z=z]\}P(Z=z\mid S=s). \end{aligned}

The target weights are the same for the two treatment conditions because both means refer to the same fixed subgroup. Replacing them with each arm's observed stratum proportions changes the target whenever those proportions differ. The derivation also shows why all required treatment-by-stratum cells matter: if a treatment condition has zero probability in a stratum with positive target weight, its conditional mean is not identified there from this design.

Diagnostics, Comparisons, and Limitations

Before interpreting a subgroup estimate, check that the baseline characteristic and subgroup rule are unambiguous, that the rule was not chosen after inspecting treatment outcomes, and that outcome measurement and intercurrent-event handling match the estimand. Then inspect the number of participants and outcome observations by treatment arm and relevant randomization stratum. Very small cells produce imprecise estimates; a missing treatment arm in a target stratum is a positivity failure, not a problem that ordinary standardization can solve.

A subgroup treatment effect is not the same quantity as the overall trial effect. The overall effect averages over its own target population, while $\Delta_s$ averages only over participants meeting the fixed subgroup rule. Nor does a difference between subgroup point estimates automatically establish effect modification: sampling uncertainty matters, and the comparison of effects should be made explicitly on a stated scale.

Approach	Target it answers	Main caution
Fixed baseline subgroup contrast	Treatment effect among participants meeting a prespecified pre-randomization rule	Requires adequate treatment representation and outcome information in the target subgroup
Overall population contrast	Average treatment effect over the defined overall target population	Does not by itself give the effect in a particular subgroup
Comparison within an observed post-randomization group	Descriptive contrast among people who exhibited an event after assignment	Group membership can differ by treatment and is not a fixed baseline target

Subgroup analyses can also be unstable when many characteristics or cutpoints are explored. Selecting a subgroup because it showed the most favorable observed result can exaggerate the apparent effect and makes ordinary uncertainty statements misleading unless the selection is accounted for. A prespecified subgroup definition improves interpretability, but it does not guarantee high precision or remove the need to report how the subgroup was chosen.

Common Mistake

Common mistake: “Randomization was conducted within each baseline stratum, so the pooled treatment mean and pooled control mean automatically estimate the target subgroup means.” This is not generally true when allocation probabilities differ across strata. Each arm can contain a different mixture of strata, as the two-stratum example shows. Compare conditional means within strata and standardize them to the target subgroup distribution, or use another estimator justified by the design. Pooled arm means are suitable only when additional conditions ensure they represent the same target mixture.

Key takeaway: A fixed baseline subgroup estimand is a treatment contrast among people classified using pre-randomization information. Its definition fixes the target; its identification depends on randomization, positivity, consistency, outcome handling, and—when needed—standardization to the subgroup's target stratum distribution.

Check Your Understanding

Answer each question before reading the brief solutions below.

A subgroup is defined by a laboratory value measured before randomization. What feature makes its membership fixed for the treatment comparison?
Within a target subgroup, two equally common baseline strata have conditional treatment effects of 2 and 6 units. What is the standardized effect if both strata have target weight 0.50?
Why might pooled treatment and control means fail to estimate the target subgroup means when allocation probabilities differ by stratum?
In a trial, participants are grouped by whether they experienced an adverse event after randomization. Is a treatment contrast within the observed event group a fixed baseline subgroup effect? Explain.

Solutions. (1) The laboratory value is measured before treatment assignment, and the subgroup rule is specified using that baseline value, so assignment cannot change membership. (2) The standardized effect is $0.50\times2+0.50\times6=4$ units. (3) The arms may contain different mixtures of baseline strata; pooled means can therefore reflect composition differences as well as conditional outcomes. Standardize conditional means to the target subgroup's stratum weights when needed. (4) No. The adverse event is post-randomization and may be affected by treatment, so the observed event groups can contain different participants across arms; the contrast is not an effect in a fixed baseline-defined subgroup.