Weighting Within Strata to Standardize Covariates

Why Standardize Within Strata?

Subclassification groups people with similar estimated propensity scores and compares treatment outcomes within each group. But treatment and control participants in a stratum can still have different distributions of measured covariates. If a prognostic covariate is more common in one treatment arm, the within-stratum outcome comparison may partly reflect that composition difference rather than treatment.

Within-stratum weighting addresses this problem by reweighting each treatment arm to have the same specified covariate distribution inside each stratum. The resulting treatment contrast is standardized to that distribution. Stratum-specific contrasts can then be combined using weights that represent the target population’s distribution across strata. The target must be stated: an estimate for the original study population, for treated participants, or for another population can require different weights.

Definition: Let $A\in\{0,1\}$ denote treatment, $Y$ the outcome, $X$ measured pretreatment covariates, and $S$ a propensity-score stratum. Define a target population by stratum probabilities $q_s$, where $q_s\geq 0$ and $\sum_s q_s=1$, and by a target covariate distribution $F_s^*(x)$ within each stratum. The target standardized mean under treatment $a$ is $$ \mu_a^*=\sum_s q_s\int E(Y\mid A=a,X=x,S=s)\,dF_s^*(x), $$ and the target standardized treatment contrast is $\tau^*=\mu_1^*-\mu_0^*$. For a continuous outcome, this contrast is in the outcome’s units.

The distribution $F_s^*$ is the same for treatment and control within a stratum; that common distribution is what makes their means comparable on a common covariate basis. The $q_s$ values determine how much each stratum contributes to the target. For example, using each stratum’s share of the original sample and its observed covariate distribution targets that sample. Using the treated participants’ stratum shares and covariate distribution targets the treated population.

Assumptions and the Weighting Formula

The calculation is straightforward, but its causal interpretation depends on assumptions. First, treatment and outcomes must be well defined and consistent: a participant’s observed outcome equals the potential outcome under the treatment received. Interference between participants is assumed absent, or otherwise adequately handled. Second, conditional exchangeability must hold for the adjustment information used: $Y(a)\mathrel{\perp} A\mid X,S$ for each $a$. Because $S$ is formed from the propensity score, it does not replace the need for sufficient adjustment for confounding. Third, positivity requires both treatment arms to be represented wherever the target distribution places covariate mass: $P(A=a\mid X=x,S=s)>0$ for target-relevant $x,s$. The chosen target distributions and stratum weights must also correspond to the population the analysis intends to describe.

For a given arm $a$ and stratum $s$, let $f_{a,s}(x)$ be the covariate density among observed participants with $A=a,S=s$, and let $f_s^*(x)$ be the target density. A density-ratio weight is

w_{a,s}(x)=\frac{f_s^*(x)}{f_{a,s}(x)}.

For discrete covariates, densities are replaced by probabilities. In a sample, the weighted outcome mean in arm $a$, stratum $s$, is commonly estimated with a normalized weighted mean:

\widehat{\mu}_{a,s}^* = \frac{\sum_{i:S_i=s,\,A_i=a} w_{a,s}(X_i)Y_i} {\sum_{i:S_i=s,\,A_i=a} w_{a,s}(X_i)}.

The standardized treatment contrast within stratum $s$ is $\widehat{\tau}_s^*=\widehat{\mu}_{1,s}^*-\widehat{\mu}_{0,s}^*$. Combining contrasts gives $\widehat{\tau}^*=\sum_s q_s\widehat{\tau}_s^*$, using target stratum weights $q_s$, or estimates of them if they are not known. The same $q_s$ must be used for both treatment arms when estimating one target contrast.

Why Density-Ratio Weighting Works

The reason for the weights is that they transform the covariate distribution in an observed treatment arm into the target distribution. In the population, the weighted mean outcome in arm $a$, stratum $s$, is

\frac{E\{w_{a,s}(X)Y\mid A=a,S=s\}} {E\{w_{a,s}(X)\mid A=a,S=s\}} = \frac{\int \frac{f_s^*(x)}{f_{a,s}(x)} E(Y\mid A=a,X=x,S=s) f_{a,s}(x)\,dx} {\int \frac{f_s^*(x)}{f_{a,s}(x)}f_{a,s}(x)\,dx} = \int E(Y\mid A=a,X=x,S=s)f_s^*(x)\,dx.

The denominator equals $\int f_s^*(x)\,dx=1$. Thus, after weighting, the mean averages the arm-specific conditional outcome means over the target covariate distribution, not the arm’s original covariate distribution. Under consistency, exchangeability, and positivity, those conditional means identify the corresponding potential-outcome means for the target population. In finite samples, estimated density ratios and normalized weighted means introduce sampling and estimation error; the identity explains the target of the procedure, not a guarantee that an estimate is exact.

Worked Example: Standardizing One Stratum to a Common Covariate Mix

Consider a hypothetical stratum with a binary covariate $X$. The target distribution has $P^*(X=1)=0.40$ and $P^*(X=0)=0.60$. Among 30 treated participants, 20 have $X=0$ and 10 have $X=1$. Their mean outcomes are 5 and 9 units, respectively. Among 50 control participants, 30 have $X=0$ and 20 have $X=1$, with mean outcomes 4 and 7 units. Assume that both arms contain participants in both covariate categories, and that these conditional means are suitable for the stated target under the causal assumptions above.

For treated participants, the observed covariate proportions are $20/30=0.6667$ for $X=0$ and $10/30=0.3333$ for $X=1$. The weights are therefore

w_{1}(0)=\frac{0.60}{20/30}=0.9000,\qquad w_{1}(1)=\frac{0.40}{10/30}=1.200.

The normalized weighted treated mean is

\widehat{\mu}_{1}^* = \frac{(20)(0.9000)(5)+(10)(1.200)(9)} {(20)(0.9000)+(10)(1.200)} = \frac{90+108}{18+12} = 6.600\text{ units}.

The control covariate proportions are $30/50=0.60$ and $20/50=0.40$, already equal to the target proportions. Both control weights are 1, so

\widehat{\mu}_{0}^* = \frac{(30)(1)(4)+(20)(1)(7)}{30+20} = \frac{260}{50} = 5.200\text{ units}.

The standardized contrast is $6.600-5.200=1.400$ outcome units. It compares the two arms after placing both on the same 60% $X=0$, 40% $X=1$ covariate mix. This calculation does not establish that the target is causally identified by itself; the relevant exchangeability and positivity assumptions remain necessary.

Combining Strata for a Chosen Target

Within-stratum standardization and across-stratum combining answer different questions. The first sets the covariate mix within each stratum. The second determines the target population’s mix of strata. If treatment effects differ across strata, changing $q_s$ can change the overall estimate even when every stratum-specific standardized contrast stays the same.

Worked Example: Combining Two Standardized Stratum Contrasts

In a hypothetical study, the target population assigns 70% of its members to stratum 1 and 30% to stratum 2. Within stratum 1, the target covariate distribution is $P^*(X=1\mid S=1)=0.25$. The treatment and control outcome means are 10 and 8 units for $X=0$, and 14 and 11 units for $X=1$. The standardized means are

\widehat{\mu}_{1,1}^*=(0.75)(10)+(0.25)(14)=11.00,\qquad \widehat{\mu}_{0,1}^*=(0.75)(8)+(0.25)(11)=8.750,

so $\widehat{\tau}_1^*=11.00-8.750=2.250$ units. Within stratum 2, the target distribution is $P^*(X=1\mid S=2)=0.60$. The corresponding treatment and control means are 16 and 13 units for $X=0$, and 20 and 17 units for $X=1$. Thus,

\widehat{\mu}_{1,2}^*=(0.40)(16)+(0.60)(20)=18.40,\qquad \widehat{\mu}_{0,2}^*=(0.40)(13)+(0.60)(17)=15.40,

and $\widehat{\tau}_2^*=3.000$ units. The overall target contrast is

\widehat{\tau}^*=(0.70)(2.250)+(0.30)(3.000)=1.575+0.900=2.475\text{ units}.

The target places more weight on stratum 1, so its contrast contributes more to the overall result. This is not an arithmetic average of the two stratum effects unless the target assigns equal weight to the strata. The calculation requires that each stratum-specific comparison be supported by treatment and control observations over the covariate values used in that stratum’s target distribution.

Choosing a Target and Checking the Calculation

A target should be specified before interpreting the estimate. For an original-population target, one natural choice is the original covariate distribution within each stratum and the original population’s stratum proportions. For an ATT target, use the treated population’s covariate distribution and stratum proportions; the treated arm then already has that target distribution, while controls are reweighted toward it. Other choices are possible, but they describe different estimands. An estimate standardized to the treated population is not automatically an estimate for the full population.

Diagnostics should examine both the weighting step and the target. Within each stratum, compare the weighted covariate distributions in the two arms with the specified target distribution. For a binary covariate, check whether the weighted proportion of $X=1$ in each arm is close to its target proportion. For continuous covariates, compare distributions as well as means: matching means does not ensure that tails or nonlinear features match. Check the number of observations in every target-relevant covariate category and treatment arm. A category absent from one arm cannot be recovered by finite weighting.

Also inspect the weight distribution separately by arm and stratum. A few very large weights can make the nominal sample size misleading. One summary is the effective sample size,

n_{\mathrm{eff}}=\frac{\left(\sum_i w_i\right)^2}{\sum_i w_i^2}.

It equals the sample size when all weights are equal and becomes smaller as weights become more concentrated. A small effective sample size signals that the standardized estimate relies heavily on a limited number of observations. It is a diagnostic, not a universal pass-or-fail threshold. Trimming or changing weights may reduce instability, but generally changes the weighting scheme and can change the target; it should not be treated as a cost-free repair.

Worked Example: Standardizing Controls to the Treated Population

Suppose a hypothetical single-stratum study has 100 treated participants: 30 with $X=0$ and 70 with $X=1$. Their mean outcomes are 5 and 10 units. The target is the treated population, so its covariate distribution is 30% $X=0$ and 70% $X=1$. There are 100 controls, of whom 70 have $X=0$ and 30 have $X=1$; their mean outcomes are 3 and 8 units. All four treatment-by-covariate groups are present.

The treated outcomes already have the target covariate mix. Their target mean is

\widehat{\mu}_{1}^*=(0.30)(5)+(0.70)(10)=1.50+7.00=8.500\text{ units}.

For controls, the weights are $0.30/0.70=0.4286$ for $X=0$ and $0.70/0.30=2.333$ for $X=1$. The weighted control mean is

\widehat{\mu}_{0}^* = \frac{(70)(0.4286)(3)+(30)(2.333)(8)} {(70)(0.4286)+(30)(2.333)} \approx \frac{90+560}{30+70} = 6.500\text{ units}.

The standardized contrast is $8.500-6.500=2.000$ units for the treated-population target. The control weights substantially increase the contribution of $X=1$, because that covariate is more common among treated participants than controls. The large relative weight is a reason to inspect overlap and effective sample size before relying on the estimate.

Alternatives, Limitations, and a Common Mistake

Unweighted subclassification compares treatment and control outcomes within strata and combines those contrasts, but it retains each arm’s observed covariate mix inside a stratum. Within-stratum weighting explicitly targets a common mix. Outcome-model standardization instead models the conditional mean $E(Y\mid A,X,S)$, predicts outcomes under each treatment for the target covariate distribution, and averages those predictions. Weighting and outcome modeling make different modeling and stability demands; neither eliminates the need to define a target and justify confounding adjustment. Matching is another way to improve comparability by selecting or pairing observations, but it can discard observations and thereby change which population is represented.

A common mistake is to use the observed treated proportion in each stratum as a weight for the treated mean and the observed control proportion as a weight for the control mean, then subtract the results. That compares treatment and control outcomes averaged over different covariate distributions. The correction is to use the same specified $F_s^*$ for both arms within each stratum, then use the same target $q_s$ weights to combine the stratum-specific means or contrasts.

Standardization cannot create information where the data have no support. It also cannot correct unmeasured confounding, errors in covariate measurement, or a poorly specified target. Estimated propensity-score strata are constructed from observed data, and broad subclasses may contain meaningful covariate differences even when their scores are similar. The standardized result should therefore be reported with its target population, covariates, stratum definitions, weighting procedure, and diagnostics, rather than as a generic treatment effect.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

Within one stratum, the target proportion with $X=1$ is 0.30. In the control arm, 40% have $X=1$. What are the density-ratio weights for controls with $X=1$ and $X=0$?
Why must the target covariate distribution within a stratum be the same for both treatment arms when forming a standardized contrast?
Two strata have standardized contrasts of 1 and 4 outcome units. The target assigns them probabilities 0.25 and 0.75. What is the combined contrast?
What does it imply if a target-relevant covariate category has no control participants in a stratum?
Does using treated participants’ covariate distribution as the target make the resulting estimate an estimate for the entire original population? Explain.

Solutions. (1) The target-to-observed proportion ratios are $0.30/0.40=0.750$ for $X=1$, and $0.70/0.60=1.167$ for $X=0$. (2) Using a common distribution makes both arm-specific means averages over the same covariate composition; otherwise their difference can reflect different mixes as well as treatment. (3) The combined contrast is $(0.25)(1)+(0.75)(4)=0.25+3.00=3.25$ outcome units. (4) Positivity fails for that category in the control arm, so weighting cannot identify its control outcome mean from the observed data without additional assumptions or a different target. (5) No. It targets the treated population’s covariate distribution, which generally differs from the original population’s; the target must be named when interpreting the estimate.