Real-World Controls in a Small Oncology Trial

Why Adjusted Real-World Controls Need Careful Borrowing

A small oncology trial may enroll too few patients to estimate the control response rate precisely. A real-world cohort can add information, but its patients may differ from trial participants in prognosis, eligibility, treatment setting, or outcome assessment. Propensity adjustment can align measured baseline characteristics; it cannot guarantee that the cohorts are comparable in every relevant way.

The aim here is to estimate the response probability under the control regimen for a clearly defined trial-eligible population. Real-world records contribute information about that target only to the extent that the relevant differences between sources have been measured, adjusted, and judged plausible. The resulting posterior is conditional on those modeling and comparability assumptions.

Target Population, Propensity Adjustment, and Assumptions

For a single-arm trial with external controls, propensity adjustment usually models the probability of belonging to the trial rather than the real-world cohort, conditional on baseline covariates. The propensity score is \(e(X)=P(S=1\mid X)\), where \(S=1\) indicates trial enrollment and \(X\) is a vector of measured pretreatment characteristics. Weighting or matching real-world patients using this score aims to make their covariate distribution resemble that of the trial population. The exact weights depend on the target population and design; the analysis must state how they were constructed.

The adjustment supports borrowing under substantive assumptions. First, conditional exchangeability requires that, after conditioning on the measured covariates used in adjustment, source membership does not remain associated with the potential control response. Second, positivity requires real-world patients with relevant covariate patterns to be available for the target population. Third, the propensity model and the endpoint definitions must be sufficiently well specified. In particular, response must be defined consistently, with comparable assessment windows and ascertainment. These conditions are not established merely by fitting a propensity model.

Balance and overlap are practical diagnostics. Compare covariate distributions before and after weighting or matching, including clinically important prognostic factors and nonlinear features where relevant. Standardized mean differences close to zero are useful summaries; an absolute value below 0.1 is a common heuristic, not proof of adequate adjustment. Inspect propensity-score overlap and the distribution of weights. Poor overlap, very large weights, or substantial remaining imbalances indicate that an adjusted estimate may be unstable or may depend heavily on extrapolation.

From Weighted Outcomes to Effective Information

Let \(w_i\) be the nonnegative adjustment weight for real-world patient \(i\), and let \(Y_i\) be that patient's binary response outcome. The weighted response rate is \(q_w=\sum_i w_iY_i/\sum_i w_i\). This is the adjusted point estimate, but the nominal number of records can exaggerate how much independent information the weighted cohort supplies. A common summary is the Kish effective sample size:

The justification is a variance comparison. For independent outcomes with a common variance, the variance of a normalized weighted mean is proportional to \(\sum_i w_i^2/(\sum_i w_i)^2\). An unweighted mean based on \(n_{\mathrm{eff}}\) observations has variance proportional to \(1/n_{\mathrm{eff}}\). Equating these expressions gives the formula above. Equal weights yield \(n_{\mathrm{eff}}=n\); unequal weights reduce it. This is a useful calibration, not a claim that weighting creates a literal sample of that many patients.

A simple Bayesian approximation uses the adjusted rate and effective sample size to form fractional effective response and nonresponse counts. With a Beta prior, these update the prior parameters directly. Because fractional counts are a pseudo-likelihood construction, the analysis should identify it as an approximation and examine whether conclusions depend on the borrowing strength. It is not the only possible model for external-control data.

Sensitivity to Residual Imbalance

Even after adjustment, a measured or unmeasured difference may leave the real-world response rate systematically above or below the target trial control rate. A transparent sensitivity analysis can represent this discrepancy as a shift \(\delta\) on the log-odds scale. In this analysis, the adjusted real-world rate is shifted to a target rate \(q_\delta\) by adding \(\delta\) to its log odds:

Here \(\delta=0\) means no residual shift; a positive value assumes the target trial control rate is higher than the adjusted real-world rate, and a negative value assumes it is lower. The corresponding odds ratio is \(e^\delta\). For this simple sensitivity calculation, replace \(q_w\) by \(q_\delta\) in the fractional effective counts and update the same prior. The chosen values of \(\delta\) should be justified using clinical knowledge or external evidence, rather than selected because they produce a preferred conclusion.

Diagnostics, Alternatives, and Limitations

A credible borrowing analysis reports more than the adjusted response rate. At minimum, document the target population, endpoint definition and observation window, covariates used in the propensity model, weighting or matching procedure, balance before and after adjustment, overlap, weight behavior, and effective sample size. Compare the adjusted estimate with the unadjusted real-world rate as a diagnostic. A large difference can reflect successful adjustment, but it can also reveal sensitivity to modeling choices. Refit with reasonable alternative propensity specifications and report whether the conclusion changes.

Propensity-adjusted borrowing differs from using an unadjusted real-world cohort: the latter does not address measured covariate differences. It also differs from a randomized concurrent control, which protects the treatment comparison through random assignment rather than relying on cross-source exchangeability. Historical-control borrowing, including power-prior approaches, offers another way to calibrate external information; the introductory material on power-prior borrowing treats that alternative. None of these methods eliminates the need to define the target population and assess comparability.

The effective-sample-size update compresses potentially complex weighted data into one rate and one information measure. It does not fully represent uncertainty in estimated propensity scores, correlations induced by matching, outcome misclassification, or uncertainty about unmeasured confounding. Kish effective sample size can be an imperfect variance summary, particularly with complex designs. The log-odds shift analysis is likewise a sensitivity device: it assumes a common shift and does not estimate residual bias from the observed data. These limitations should temper interpretation, especially when the trial is very small and external information has substantial influence.