Overlap Weighting for Limited Comparability

Why Target the Overlap Population?

A trial and an external control cohort may contain patients with very different baseline profiles. When only a few trial participants resemble many external controls—or vice versa—a treatment-effect estimate for the whole combined population can depend heavily on extrapolation. Overlap weighting instead targets patients whose measured covariates make them plausible members of either cohort. The resulting effect is not automatically an effect for all trial-eligible patients; its interpretation depends on the population that receives weight.

Let \(A=1\) indicate a patient assigned to the trial treatment and \(A=0\) indicate an external control. Let \(X\) be the measured baseline covariates used to model cohort membership, and define the propensity score \(e(X)=P(A=1\mid X)\). In this setting, it is the probability of being in the trial-treated group rather than the external-control group, conditional on \(X\). It is not necessarily a randomized treatment-assignment probability: membership in the two cohorts may reflect different recruitment, care, or data processes.

The factor \(e(X)\{1-e(X)\}\) is largest at \(e(X)=0.5\) and approaches zero as the score approaches 0 or 1. Thus the target population gives little aggregate mass to covariate profiles that strongly distinguish the cohorts. This statement concerns the target distribution, not every individual weight: a trial-treated patient with a low score can have a high overlap weight, as can an external control with a high score.

Assumptions and the Target Effect

Write \(Y(1)\) and \(Y(0)\) for the potential outcomes under the trial treatment and control, respectively. A causal interpretation of an overlap-weighted contrast requires consistency (the observed outcome equals the potential outcome under the treatment actually received), no interference between patients, and conditional exchangeability: given the measured \(X\), cohort membership is independent of the potential outcomes. Because external controls are not generally randomized together with trial participants, exchangeability is a substantive assumption, not a consequence of weighting.

The covariates should be measured before treatment, harmonized across sources, and sufficient to address important confounding. Follow-up, outcome definitions, and eligibility should also be aligned well enough that the compared outcomes have the same intended meaning. The propensity model must adequately represent cohort membership in the population being analyzed. Finally, the target must have nonzero overlap: there must be covariate profiles represented in both groups. Profiles with \(e(X)=0\) or \(e(X)=1\) have zero target density and cannot support a comparison.

Let \(f(X)\) denote the covariate density in the combined population of interest. The overlap target density is

The corresponding average treatment effect is the average of \(Y(1)-Y(0)\) over this distribution. It describes the treatment effect for a covariate population concentrated where both cohorts have representation. It does not describe patients outside that overlap, and its value can differ from an effect standardized to the trial population.

Why the Two Group Weights Target the Same Population

A short derivation shows why the group-specific weights produce a common covariate distribution. Let \(\pi=P(A=1)\). By the definition of conditional probability, the covariate densities in the two groups are \(f(X\mid A=1)=e(X)f(X)/\pi\) and \(f(X\mid A=0)=\{1-e(X)\}f(X)/(1-\pi)\). Weighting the treated covariate density by \(1-e(X)\) gives a density proportional to \(e(X)\{1-e(X)\}f(X)/\pi\). Normalizing it removes the constant \(1/\pi\). Weighting the control density by \(e(X)\) likewise gives a density proportional to \(e(X)\{1-e(X)\}f(X)/(1-\pi)\); normalization removes \(1/(1-\pi)\). Both therefore yield \(f_{\mathrm{OW}}(X)\), provided the propensity score is correctly specified for the population being analyzed.

In data, replace \(e(X_i)\) with an estimated score \(\hat e_i\). For an outcome \(Y\), the weighted difference in means is

The first term is the weighted mean outcome among trial-treated patients; the second is the weighted mean among external controls. Their difference is oriented as treatment minus control. For a continuous outcome, it has the outcome's units; for a binary outcome, it is a risk difference on the probability scale. This calculation defines a weighted contrast, but causal interpretation still rests on the assumptions above.

Worked Estimation With Two Covariate Profiles

The next example shows how individual weights give the two groups the same weighted profile distribution. All data are hypothetical. Suppose there are two baseline profiles, \(X=1\) and \(X=0\), and the fitted scores are \(0.8\) and \(0.2\), respectively.

Diagnostics, Alternatives, and Limitations

Before interpreting a weighted result, inspect the estimated score distributions by group and check whether the propensity model is plausible for the clinical and data-generation context. After weighting, examine whether important baseline covariates are similar across groups; overlap weighting does not guarantee balance for every covariate when the score model is misspecified or too restrictive. Review the influence of individual observations and the resulting weighted sample sizes. A bounded weight (between 0 and 1) avoids the arbitrarily large weights possible with inverse probability weighting, but it does not ensure a precise estimate or adequate information.

The target itself depends on the covariates and the cohort-membership model. If the groups differ in how patients were sampled or selected, the fitted \(e(X)\) and resulting overlap population may describe the analyzed mixture rather than a broader clinical population. A deliberate change in the relative number sampled from each source can therefore change the target unless the sampling design is accounted for. Report the covariates used, how cohort membership was modeled, and the population the weights are intended to represent.

Inverse probability weighting and matching are alternatives with different targets and practical behavior. Inverse probability weighting can target a broader population, such as the treated group, but may rely heavily on observations with extreme scores. Matching pairs or selects patients based on similarity and may discard unmatched observations; its target depends on the matching design. Overlap weighting is useful when the scientific question concerns patients with covariate profiles represented in both groups, not when the primary aim is to estimate an effect for all trial-eligible patients regardless of comparability.

Overlap weighting cannot repair unmeasured confounding, outcome definition differences, incorrect follow-up alignment, or selection bias that remains after conditioning on \(X\). Nor does a weighted estimate automatically account for uncertainty in the estimated propensity scores. Uncertainty procedures should reflect that the scores were estimated, rather than treating the calculated weights as known constants. A clinically clear report distinguishes the estimate's target population from the trial population and states the assumptions needed for a causal interpretation.

Common Pitfall

A related mistake is to see weights bounded by 1 and conclude that all instability or bias has been solved. Bounded weights reduce one source of numerical instability, but misspecified propensity scores can leave covariate imbalance, and unmeasured differences between cohorts can remain. The remedy is to check the modeling and data assumptions, report the target precisely, and limit conclusions to what the comparison supports.