Trimming and Restriction When Historical Controls Lack Overlap

Why Restrict an Analysis When Historical Controls Lack Overlap?

Historical controls can add information when a new treatment arm opens, but they may not represent every patient enrolled in the target period. For example, eligibility, referral patterns, or the mix of disease severity may have changed. If the historical data contain no patients with some target-period covariate profiles, an adjusted analysis cannot learn the historical control outcome for those profiles directly. A regression model might still produce a prediction there, but that prediction is an extrapolation.

Restriction, also called trimming in this setting, addresses the support problem by limiting the analysis to covariate profiles represented in both sources. It does not make historical and current patients exchangeable, and it does not recover an effect for patients who are removed. Its purpose is narrower: to avoid relying on unsupported predictions and to define clearly the population for which the comparison is made.

Here, the target-period group is the group assigned to the experimental treatment in period \(C\), and historical controls received control in period \(H\). If \(Y(1)\) and \(Y(0)\) denote the potential outcomes under treatment and control, respectively, a useful target is the average effect among retained target-period patients:

For a binary outcome, \(\tau_S\) is a risk difference, measured in probability units or percentage points. Other effect measures require their own estimand and assumptions; restriction alone does not determine the scale of the effect.

Assumptions and the Restricted Comparison

The analysis requires more than overlap. First, covariates and outcomes must be defined comparably across periods, including a compatible outcome window and follow-up. Second, conditional on the measured covariates, historical controls must provide a valid estimate of the control outcome that target-period patients would have experienced. In potential-outcome notation, this requires no residual period difference in the control outcome after conditioning on \(X\), for the covariate profiles being analyzed. Third, each retained target profile must have historical control support. Finally, the treatment outcome among the retained target-period patients must be observed or identified by the target-period design, commonly randomization.

These assumptions clarify what restriction can and cannot accomplish. Restriction addresses lack of measured-covariate support by excluding profiles for which historical controls are absent or too sparse. It cannot remove bias from unmeasured differences between periods, such as a change in clinical practice not captured by \(X\). Nor does a common range of a few covariates guarantee that the full multivariate profiles overlap.

Within \(S\), one can compare the target-period treated outcome with a control outcome standardized to the covariate distribution of retained target-period patients. Let \(m_1(x)=E\{Y\mid C,A=1,X=x\}\) and let \(m_{0,H}(x)=E\{Y\mid H,A=0,X=x\}\). Under the assumptions above, the latter estimates the target-period control mean conditional on \(X=x\). The standardization is:

In a sample, this averages the covariate-specific differences over the retained target-period treated patients. For binary outcomes, it is the difference between the observed or modeled treatment risk and the historical control risk standardized to those retained patients. The earlier material on standardizing effects to a common target population gives the general standardization idea; the key here is that the target distribution itself is restricted to the overlap region.

Defining and Checking the Restriction

The support rule should be chosen using covariates measured before treatment and specified without using outcome results. In a small discrete example, support can be assessed directly by cross-tabulating covariate patterns by period. With continuous or many covariates, analysts may examine distributions, multivariate profiles, or an estimated period-membership score \(q(X)=P(C\mid X)\). Values of \(q(X)\) near zero indicate profiles concentrated in the historical period; values near one indicate profiles concentrated in the target period. A rule such as retaining \(0.10\le q(X)\le0.90\) can be an operational trimming rule, but it is not a universal definition of adequate overlap. It should be justified by design context and checked against the covariate distributions themselves.

A practical restriction process is:

Diagnostics should look for more than a single score range. Tabulate retained and excluded patients by important covariates, inspect distributions and joint patterns, and check the number of historical controls supporting each retained region. Very few controls in a region can make the estimated control outcome unstable even if the region is not literally empty. Also examine how estimates and retained populations change under reasonable alternative thresholds. Large changes can signal that the conclusion depends heavily on the chosen support rule.

How Trimming Changes Interpretation

Restriction changes the target population by design. If treatment effects vary across covariate profiles, the restricted estimate can differ from the effect in the full target-period population even when both estimates are unbiased for their respective targets. Excluded patients are not evidence of no treatment effect; their effect is simply not identified by this restricted historical-control comparison.

Restriction differs from weighting, which keeps a broader target group but gives some observations more influence; severe lack of overlap can make such weights unstable or place high demands on the data. Restriction avoids estimating effects in unsupported regions, at the cost of changing the population and discarding data. A concurrent-control-only analysis is another alternative: it avoids relying on historical control outcomes, but may be less precise or address a narrower enrollment period. The choice depends on the intended estimand, support, and credibility of the assumptions; no method can manufacture information in a region where one source has no observations.

Common Mistakes and Limitations

• Reporting the restricted estimate as an all-patient effect. The estimate is for target-period patients retained in \(S\). Report the excluded count and describe the retained population.
• Treating a score cutoff as proof of exchangeability. A period score summarizes measured covariates; it cannot establish that all relevant temporal differences have been measured.
• Keeping a region with almost no historical controls. Nonzero support may still be too weak for stable estimation. Report cell counts and uncertainty, not only a visual or score-based overlap summary.
• Choosing the cutoff to obtain a preferred result. Outcome-guided trimming risks making the analysis depend on the observed result. Specify the rule from design considerations and show sensitivity to reasonable alternatives.
• Assuming restriction always improves validity. It can reduce extrapolation but can also sharply reduce sample size and precision. It does not fix outcome measurement differences, unmeasured period effects, or incompatible follow-up.