Why Adjust Within Covariate Strata?
When treatment assignment differs across groups with different outcome risks, the crude difference in observed outcome means can combine two things: differences in treatment and differences in group composition. If the relevant pretreatment covariates are discrete, one direct way to adjust is to compare treated and untreated units within each covariate stratum, then average those conditional means using the covariate distribution of the population of interest.
The averaging weights matter. The resulting contrast describes the target population whose covariate distribution supplies those weights. In particular, conditional means observed in a study population do not automatically describe a separate target population. Using them for that target requires justified assumptions that link the two populations.
Assumptions and the Standardization Formula
The target average treatment effect is a contrast between mean potential outcomes, not merely a difference between observed groups. To identify it by stratification, the following conditions are needed for the relevant target population and covariate strata.
- Consistency: for a unit with \(A=a\), the observed outcome equals that unit’s potential outcome under the received treatment: \(Y=Y(a)\). The treatment must represent a sufficiently well-defined intervention.
- Conditional exchangeability: \(Y(a)\perp A\mid L\) for \(a=0,1\). Within each stratum, treatment assignment must not remain associated with the potential outcome being estimated.
- Positivity: \(P(A=a\mid L=l)>0\) for both \(a=0,1\) in every stratum with positive target weight. Both treatment groups must be represented there in the population supplying the conditional means.
- Appropriate target-population means: the conditional means used must pertain to the target population. If conditional means are estimated in a different study population, additional assumptions are needed to transport them to the target.
When the observed conditional means come from the target population, consistency and conditional exchangeability connect them to the potential-outcome means. For either treatment level \(a\), exchangeability gives \(E_T[Y(a)\mid L=l]=E_T[Y(a)\mid A=a,L=l]\). Consistency then gives \(E_T[Y(a)\mid A=a,L=l]=E_T[Y\mid A=a,L=l]\). Averaging over target strata yields the identification formula:
The last expression is equivalently \(\tau^T=\sum_l\{m_1(l)-m_0(l)\}P_T(L=l)\), when the \(m_a(l)\) are conditional means for the target population. With data, conditional means are estimated from the observed treatment-by-stratum cells, and target weights are estimated or otherwise specified for the target population. This is an estimation step; it does not make the assumptions true.
A short justification for the weights is that the target mean is a mixture across its strata. By the law of total expectation, \(E_T[Y(a)]=\sum_l E_T[Y(a)\mid L=l]P_T(L=l)\). Thus each stratum-specific mean is weighted by how common that stratum is in the target, not by how common it happens to be among treated units or untreated units.
Worked Example: Standardizing Two Stratum-Specific Comparisons
Suppose a hypothetical study records a binary outcome, where 1 means improvement. The covariate \(L\) has two pretreatment strata, lower-risk and higher-risk. The target population is the study population, with half in each stratum. All four treatment-by-stratum cells have observations, so the sample exhibits positivity in both strata.
| Stratum | Treated: improved / total | Usual care: improved / total | Target weight |
|---|---|---|---|
| Lower-risk | 32 / 40 | 6 / 10 | 0.50 |
| Higher-risk | 5 / 10 | 16 / 40 | 0.50 |
The observed conditional improvement proportions are \(m_1(\text{lower})=32/40=0.80\), \(m_0(\text{lower})=6/10=0.60\), \(m_1(\text{higher})=5/10=0.50\), and \(m_0(\text{higher})=16/40=0.40\). The target standardized means are:
The standardized difference is \(0.650-0.500=0.150\), or 15.0 percentage points. Under the assumptions above, this estimates a target-population causal mean contrast. The observed-data calculation alone is not evidence that exchangeability holds; that assumption needs substantive justification.
For comparison, the crude treated proportion is \((32+5)/(40+10)=37/50=0.740\), and the crude usual-care proportion is \((6+16)/(10+40)=22/50=0.440\). Their crude difference is \(37/50-22/50=15/50=0.300\), or 30.0 percentage points. Here the treated group contains more lower-risk units than the usual-care group. The crude difference therefore uses different stratum mixtures for its two means; it is not the same standardized comparison.
Choosing the Target Weights
The weights encode whose mean effect is being summarized. They might be the proportions of strata in the study population, in a specified service population, or in another explicitly defined target. If treatment effects differ across strata, changing those weights can change the overall contrast even when every stratum-specific comparison stays the same. This is not a calculation error: it reflects a different target.
The formula can also be written by first calculating each stratum-specific difference, \(m_1(l)-m_0(l)\), and then weighting those differences by \(p_T(l)\). This makes clear why a standardized contrast is not generally the difference between two means averaged over different treatment-group compositions. Both treatment-specific means must use the same target weights.
Worked Example: A Separate Target Population
Consider a hypothetical source study with two age strata. Its observed conditional improvement proportions are \(0.70\) for treated and \(0.50\) for usual care among younger participants, and \(0.60\) and \(0.30\), respectively, among older participants. Suppose a separate target population is 80% younger and 20% older.
The source study’s conditional means can be used for the target calculation only if the required transportability assumption is justified: within each age stratum, the source and target populations have the same conditional potential-outcome means for each treatment level. One also needs consistency, conditional exchangeability in the source study, and treatment positivity in the relevant source strata. The target’s stratum proportions must be known or estimated appropriately.
Under those assumptions, the target standardized treated mean is \( (0.70)(0.80)+(0.60)(0.20)=0.56+0.12=0.680\). The target standardized usual-care mean is \((0.50)(0.80)+(0.30)(0.20)=0.40+0.06=0.460\). Therefore, the target contrast is \(0.680-0.460=0.220\), or 22.0 percentage points.
The 80% and 20% weights are used for both treatment levels because they describe the target population, not the treatment mix in the source study. If the conditional potential-outcome means do not transport, these arithmetic steps still produce a number, but that number is not thereby identified as the target causal contrast.
Diagnostics and Failure Modes
For discrete covariates, begin with a table of counts for every treatment-by-stratum cell. This exposes empty cells and very small cells, both of which matter. An empty cell violates sample support for estimating a conditional mean directly. A very small cell may technically be nonempty but yield an unstable mean, so the standardized estimate can be highly sensitive to one or two observations. Also check that all target strata are represented in the data used for conditional means, and that the weights are nonnegative and sum to one.
An empirical cell proportion is not a diagnostic for conditional exchangeability. Similar observed outcomes or covariate distributions do not establish that all relevant causes of both treatment and outcome have been measured. Subject-matter knowledge is needed to assess whether \(L\) is sufficient for the stated exchangeability assumption. Covariates must be pretreatment variables for this adjustment to address treatment assignment as specified.
Worked Example: An Empty Cell and What It Prevents
Suppose a hypothetical target population has two equally common strata, \(L=A\) and \(L=B\). In stratum \(A\), all 20 observed units received treatment, and 14 improved; no usual-care units were observed. In stratum \(B\), 8 of 20 treated units improved and 6 of 20 usual-care units improved. Assume the observed conditional means are otherwise appropriate for the target and focus on the consequence of the missing cell.
The treated mean is observed in both strata: \(m_1(A)=14/20=0.70\) and \(m_1(B)=8/20=0.40\). Its standardized value is \((0.70)(0.50)+(0.40)(0.50)=0.550\). The usual-care mean in stratum \(B\) is \(6/20=0.30\), but \(m_0(A)\) cannot be estimated from these observations.
Let \(q\) denote the unknown mean usual-care potential outcome in stratum \(A\). Since the outcome is binary, \(0\le q\le1\). The target usual-care mean would be \((0.50)q+(0.50)(0.30)=0.50q+0.15\), and the target contrast would be \(0.550-(0.50q+0.15)=0.400-0.50q\). As \(q\) ranges from 0 to 1, this contrast ranges from \(-0.100\) to \(0.400\). The observed data do not determine a unique contrast.
This is a positivity failure: \(P(A=0\mid L=A)=0\) in the observed population. Exchangeability cannot supply the missing conditional outcome mean without additional information or assumptions. A numerical answer obtained by silently dropping stratum \(A\), or by inventing its missing mean, would answer a different question or rely on an unstated assumption.
How This Adjustment Relates to Alternatives
A crude difference compares the observed treatment groups as they are, so their covariate distributions can differ. Direct stratified standardization instead compares treatment groups within strata and combines the results using one declared target distribution. This makes the target explicit and permits inspection of each stratum’s contribution.
Matching and inverse probability weighting are other approaches to constructing adjusted comparisons, while regression adjustment can model conditional outcome means. These approaches differ in how they use the data and encode assumptions; none removes the need for a defensible adjustment set, adequate treatment support, and a clearly stated target. With a modest number of discrete strata, direct standardization is especially transparent. If many covariates generate numerous sparse combinations, direct cell means may be unreliable or unavailable; more structured methods may be considered, while recognizing that their additional modeling choices do not create information in unsupported strata.
A common mistake is to use the treated group’s stratum proportions to average treated means and the untreated group’s proportions to average untreated means, then call the difference “adjusted.” That comparison retains different covariate mixtures in its two terms. The correction is to choose the target distribution first and use the same \(p_T(l)\) weights for both treatment-specific means. Another mistake is to apply source-study conditional means to a separate target population as if the target weights alone guaranteed validity. The correction is to establish the needed transportability assumptions or use conditional means that are valid for the target.
Check Your Understanding
Answer each question, then check the solutions below.
- Two strata have target weights 0.25 and 0.75. Their estimated treatment effects are 0.12 and 0.04. What is the standardized target effect?
- Why must both treatment-specific means use the same target-stratum weights?
- A source study has treatment and control observations in every stratum. Does that alone justify using its conditional means for a separate target population? State the additional issue.
- In one target stratum, no untreated units are observed. Which assumption is at issue, and why can the missing mean not be calculated by stratification?
Solutions. (1) The standardized effect is \((0.25)(0.12)+(0.75)(0.04)=0.030+0.030=0.060\). (2) The same weights make both means refer to the same target covariate distribution; different weights would mix treatment differences with composition differences. (3) No. The source conditional potential-outcome means must also be valid for the target, requiring justified transportability assumptions or target-population conditional means. (4) Positivity is violated for the untreated treatment level in that stratum. With no untreated observations there, the observed conditional mean does not identify the missing potential-outcome mean, so the weighted target contrast is not determined by stratification alone.