Weighting Nonconcurrent Controls to Match Current Patients

Why Weight Nonconcurrent Controls?

In a platform trial, an arm may recruit patients during a period when its randomized control arm is no longer available. Nonconcurrent controls can provide information about outcomes under control, but their covariate distribution may differ from that of patients eligible during the treatment arm’s recruitment period. Weighting addresses this measured imbalance by giving historical controls unequal influence so that, after weighting, their measured baseline covariates represent the current-period target.

The goal is not simply to make two groups look alike on any available variables. The target population must first be specified: here it is the patients eligible during the treatment arm’s recruitment period, represented in practice by observed current-period patients. The weights then align the historical controls to that target on selected baseline covariates. As in standardizing effects to a common target population, the choice of target matters; weighting is one way to carry out the alignment.

Definition: Let \(S=H\) indicate membership in the nonconcurrent historical control cohort and \(S=T\) membership in the current-period target cohort. Let \(X\) denote measured baseline covariates, and let \(Y\) be the outcome measured under control. A density-ratio weight for a historical control with covariates \(X=x\) is proportional to \(f_T(x)/f_H(x)\), the target covariate density divided by the historical-control covariate density. A normalized weighted control mean estimates the control outcome mean for the target population when the required assumptions hold.

From Covariate Ratios to a Weighted Mean

For discrete covariate strata, the weight for a historical control in stratum \(k\) can be calculated as the target proportion in that stratum divided by the historical-control proportion there. Thus, a stratum that is common in the target but uncommon among historical controls receives a larger weight. A stratum that is less common in the target receives a smaller weight.

For individual historical controls with weights \(w_i\), the normalized weighted mean is

\widehat{\mu}_{0,T}^{\,w} = \frac{\sum_{i:S_i=H} w_iY_i} {\sum_{i:S_i=H} w_i}.

This is a weighted average of the observed historical control outcomes. The denominator normalizes the weights, so multiplying every weight by the same positive constant does not change the estimate. When the current-period treatment mean is estimated for the same target population, a treatment-minus-control contrast can be formed by subtracting this weighted control mean from that treatment mean.

Key assumption: Conditional on the measured covariates \(X\), the historical control outcomes must be transportable to the target period: the expected control outcome for patients with covariates \(X=x\) must not differ merely because they entered in the historical rather than current period. Weighting can balance measured \(X\); it cannot remove bias from unmeasured differences or changes in care and outcome measurement that remain after conditioning on \(X\).

A useful justification follows directly from the density ratio. If \(f_H(x)>0\) wherever \(f_T(x)>0\), and \(w(x)=f_T(x)/f_H(x)\), then for any outcome function \(g\),

\frac{E_H\{w(X)g(X,Y)\}}{E_H\{w(X)\}} = \frac{\int \frac{f_T(x)}{f_H(x)}E\{g(X,Y)\mid X=x,S=H\}f_H(x)\,dx} {\int \frac{f_T(x)}{f_H(x)}f_H(x)\,dx} = \int E\{g(X,Y)\mid X=x,S=H\}f_T(x)\,dx.

The denominator equals one for an exact density ratio. Taking \(g(X,Y)=Y\), and assuming the conditional control outcome mean is transportable from \(H\) to \(T\), the final expression is the target-period mean outcome under control. In a finite sample, estimated weights and observed outcomes replace the population quantities, so the result is an estimate rather than an exact identity.

Worked Example: Exact Weights for Two Risk Strata

Hypothetical data. Suppose the current target has 60% high-risk and 40% low-risk patients. Among 100 nonconcurrent controls, 30 are high-risk and 70 are low-risk. Their mean symptom scores are 14 points in the high-risk stratum and 8 points in the low-risk stratum. Lower scores are preferable. Assume these two baseline strata capture the covariate differences being addressed.

The historical proportions are 0.30 and 0.70. The stratum weights are therefore \(w_{\text{high}}=0.60/0.30=2.00\) and \(w_{\text{low}}=0.40/0.70=0.5714\).

The weighted high-risk contribution to the denominator is \(30(2.00)=60\), and its outcome contribution is \(30(2.00)(14)=840\). For low-risk controls, these contributions are \(70(0.5714)=40.00\) and \(70(0.5714)(8)=320.0\), using the unrounded ratio \(0.40/0.70\) for the arithmetic. Thus,

\widehat{\mu}_{0,T}^{\,w} = \frac{840+320}{60+40} = 11.60\text{ points}.

The weighted historical controls estimate a mean control symptom score of 11.60 points for a population with the target’s 60:40 risk distribution. This changes the representation of the historical controls; it does not change any individual’s recorded score. If, hypothetically, the current treatment mean in that same target were 9.00 points, the treatment-minus-control estimate would be \(9.00-11.60=-2.60\) points. That contrast is interpretable as a difference in mean scores only if the target, outcome definition, follow-up, and transportability assumptions are appropriate.

Estimating Weights With Cohort Membership

When \(X\) contains several variables, one practical approach is to model cohort membership using the pooled current and historical records. Fit a model for \(P(S=T\mid X)\), then use the estimated odds of current versus historical membership as weights for historical controls. The odds are proportional to the density ratio when the cohort sampling scheme is ignorable within covariate values. The factor involving the proportions sampled from each cohort is constant across historical controls; it cancels from a normalized weighted mean. This is why normalized odds weights can be used without knowing that constant.

The membership model must be suited to the covariates and their relationships with cohort membership. A model that omits important nonlinearities or interactions can produce weights that do not adequately align the target and historical covariate distributions. The fitted model is not evidence that temporal differences in outcomes have been removed: it estimates measured covariate alignment, not exchangeability by itself.

Worked Example: Weighted Mean From Fitted Membership Odds

Hypothetical data. A fitted cohort-membership model gives estimated current-to-historical odds of 0.1353, 0.4493, and 0.1653 for three covariate patterns. There are 10 historical controls in each pattern, with mean outcomes of 12, 8, and 10 points, respectively. Suppose the model is adequate for these measured covariates and there is support for all three patterns in both cohorts.

Each historical control receives the odds for their pattern. Since there are the same number of historical controls in each pattern, the common factor of 10 in the weighted numerator and denominator cancels. Using the displayed rounded odds consistently, the numerator after cancellation is \(0.1353(12)+0.4493(8)+0.1653(10)=1.6236+3.5944+1.6530=6.8710\). The corresponding denominator is \(0.1353+0.4493+0.1653=0.7499\). Therefore,

\widehat{\mu}_{0,T}^{\,w} = \frac{6.8710}{0.7499} \approx 9.163\text{ points}.

The weighted estimate is a mean control outcome of about 9.163 points for the modeled current-period covariate distribution. This calculation uses the rounded odds printed above in both numerator and denominator. With unrounded fitted odds, the corresponding unrounded sums should be used throughout before rounding the final estimate.

Diagnostics and Failure Modes

Weighting should be evaluated as an empirical procedure, not assumed to have succeeded because a model was fitted. Before interpreting the outcome estimate, compare target patients and weighted historical controls on the covariates used to construct the weights. Useful summaries include stratum proportions for categorical covariates and means or distributional summaries for continuous covariates. Check important covariate relationships as well as separate marginal summaries: matching each variable’s mean does not guarantee that combinations of variables are balanced.

Inspect the distribution of the weights. A few very large weights indicate that a small number of historical controls represent many target patients. Such weights can make the estimate highly sensitive to individual outcomes. One diagnostic is the effective sample size,

n_{\mathrm{eff}} = \frac{\left(\sum_i w_i\right)^2}{\sum_i w_i^2}.

This equals the actual sample size when all weights are equal and decreases as weights become more unequal. It is a diagnostic for concentration of weighted information, not a guarantee of valid inference or a replacement for an uncertainty calculation that accounts for the analysis design.

Worked Example: Effective Sample Size and a Dominant Weight

Hypothetical data. Six historical controls have weights \(1,1,2,2,4,10\), and outcomes \(9,10,8,11,7,15\) points. Their total weight is \(1+1+2+2+4+10=20\). The weighted outcome sum is \(1(9)+1(10)+2(8)+2(11)+4(7)+10(15)=235\), so the weighted mean is \(235/20=11.75\) points.

The sum of squared weights is \(1+1+4+4+16+100=126\). Hence

n_{\mathrm{eff}} = \frac{20^2}{126} = \frac{400}{126} \approx 3.175.

Although six controls were observed, the unequal weighting gives an effective sample size of about 3.175. The control with weight 10 contributes 150 of the 235 weighted outcome units, so the estimate is sensitive to that person’s outcome. This example does not by itself establish bias, but it signals that precision and robustness require close attention.

Other failure modes include using covariates measured after treatment assignment, defining strata differently across periods, and ignoring changes in eligibility, follow-up, or outcome ascertainment. These problems can invalidate the comparison or give the weights the wrong target. Missing overlap is especially consequential: if target patients have covariate values absent from historical controls, the density ratio is not estimable there from the observed historical data. Weighting cannot create information about unsupported parts of the target.

Large weights and poor overlap should prompt scrutiny of the estimand and the data before reporting a result. Decisions to alter the target or the analysis are substantive choices, not merely numerical fixes. Weight truncation or restriction changes what the analysis represents and must not be treated as a way to preserve the original target automatically.

How Weighting Differs From Other Comparisons

Direct standardization estimates outcomes within covariate strata and averages those estimates using target-population proportions. With discrete strata and sufficient data, this can produce the same target mean as ratio weighting. Weighting instead reweights the observed historical controls, which can be convenient with many covariates or a fitted membership model. Regression adjustment models the outcome conditional on covariates and predicts or averages outcomes for the target. It relies on outcome-model assumptions rather than, or in combination with, a model for cohort membership.

Unweighted pooling of current and historical controls does not generally estimate the current-period control mean when the covariate distributions differ. Even if weighting achieves measured covariate balance, it does not guarantee that calendar-time confounding is absent. For example, standard care may change over time in ways not captured by \(X\), or the outcome may be recorded differently. The assumptions for using historical controls therefore remain central to interpretation.

Key takeaway: Weight historical controls by how much more common their measured covariate pattern is in the target than in the historical cohort. Then verify covariate balance, examine weight concentration, and interpret the weighted outcome only under adequate overlap and a defensible assumption of conditional transportability across periods.

Check Your Understanding

Try each problem, then compare your reasoning with the solutions below.

The target has 40% patients in a covariate stratum, while historical controls have 20% there. What is the density-ratio weight for that stratum? Explain its meaning.
Why does multiplying every weight by the same positive constant leave a normalized weighted mean unchanged?
Six historical controls have weights \(1,1,2,2,4,10\). Calculate their effective sample size and explain what it indicates.
State one reason why covariate weighting cannot, by itself, remove bias due to calendar time.

Solutions. (1) The weight is \(0.40/0.20=2\). Each historical control in that stratum receives twice the relative representation it would have under equal weighting, because the stratum is twice as prevalent in the target as in the historical cohort. (2) If every \(w_i\) is multiplied by \(c>0\), then \(\sum_i cw_iY_i/\sum_i cw_i=c\sum_iw_iY_i/(c\sum_iw_i)=\sum_iw_iY_i/\sum_iw_i\). (3) The sum of weights is 20 and the sum of squared weights is 126, so \(n_{\mathrm{eff}}=20^2/126\approx3.175\). Weight concentration means the six observations contribute roughly as much information, by this diagnostic, as about 3.175 equally weighted observations; it is not a guarantee of precision. (4) For example, standard care may have changed in a way not measured by the included baseline covariates. Weighting aligns measured covariates, not unmeasured temporal differences in outcomes.