Why External Controls Require Special Care in Rare Diseases
When a disease is rare, a randomized control arm may be difficult to recruit, ethically challenging, or too small to provide a precise estimate. Natural-history studies, registries, and clinical records can provide information about outcomes without the investigational treatment. But a larger historical data set does not automatically make a comparison reliable. Differences in who was eligible, how outcomes were measured, and when care was delivered can create apparent treatment effects unrelated to treatment.
A Bayesian analysis can represent uncertainty in both trial and external-control estimates and can make the amount of borrowing explicit. It cannot, by itself, remove bias from a poorly aligned comparison. The key task is to establish what the external data can reasonably tell us about the outcomes that comparable trial participants would have experienced without treatment.
Conditions for a Credible Rare-Disease Comparison
Begin by defining the target population, treatment contrast, outcome, and time horizon. For example, the target may be patients meeting the trial’s eligibility criteria at baseline, and the outcome may be progression by 24 weeks. A historical estimate for a broader population or a different follow-up time does not directly answer that question.
Several conditions matter. Eligibility criteria and baseline disease severity should be sufficiently aligned, or differences should be handled by a justified adjustment to the target population. The external cohort should have adequate overlap with trial participants on important prognostic characteristics; if some trial participants have no comparable external patients, adjustment cannot reliably supply their missing counterfactual outcomes. Treatment history, supportive care, calendar time, follow-up, and endpoint ascertainment also need scrutiny. The earlier material on endpoint harmonization and transport effects provides standard tools for addressing these issues; here the emphasis is on deciding whether their assumptions are plausible in a rare-disease setting.
The outcome definitions should have the same clinical meaning and be assessed over comparable windows. A registry entry that records an event only when a patient returns to a specialist may not be comparable to scheduled trial assessments. Missingness and censoring also matter: if outcome observation depends on health status, a simple comparison of observed outcomes can be biased. A Bayesian probability model does not make such observation processes ignorable.
Even after careful alignment, exchangeability remains an assumption: conditional on measured baseline characteristics and the design choices, the external control outcomes should represent the outcomes the trial participants would have had without the investigational treatment. Unmeasured prognostic differences can violate this assumption. As in the earlier material on external-control selection bias and quantitative bias analysis, investigators should identify plausible sources of bias and assess how large they would need to be to alter conclusions.
A Simple Bayesian Borrowing Model
For a binary outcome, let \(p_C\) be the probability of the outcome under control and \(p_T\) the probability under treatment in the target population. Suppose the trial observes \(y_T\) outcomes among \(n_T\) treated participants, and the external data observe \(y_E\) outcomes among \(n_E\) comparable controls. A transparent starting model is binomial sampling with beta priors. For illustration, take independent uniform priors, \(p_T\sim\mathrm{Beta}(1,1)\) and \(p_C\sim\mathrm{Beta}(1,1)\). These priors are examples, not universally appropriate choices.
One way to limit the contribution of external observations is a fixed power weight \(w\), between 0 and 1. The external likelihood is raised to the power \(w\), so its event and non-event counts contribute \(wy_E\) and \(w(n_E-y_E)\) to the beta posterior. This fractional-count construction is an analysis choice; it does not turn the historical cohort into randomized data or correct systematic differences.
The result follows by multiplying the uniform beta prior density, proportional to 1, by the powered binomial likelihood. The resulting kernel is \(p_C^{wy_E}(1-p_C)^{w(n_E-y_E)}\), which is the kernel of the stated beta distribution. The posterior mean is its first shape parameter divided by the sum of the two shape parameters. For the treatment arm, under the same uniform prior, \(p_T\mid\text{data}\sim\mathrm{Beta}(1+y_T,1+n_T-y_T)\).
The fixed weight makes a useful sensitivity parameter: \(w=1\) uses all external likelihood information, while \(w=0\) uses none. A value between them discounts the external data. Choosing \(w\) is not a substitute for checking compatibility, and results should be examined across defensible choices. A later treatment of prior-data conflict will consider more adaptive borrowing approaches.
Worked Example: Discounting a Natural-History Control
Hypothetical data. In a natural-history cohort, 2 of 20 eligible patients experience a specified event by 24 weeks. In a small trial, 6 of 8 treated patients meet the response endpoint by 24 weeks. Assume the endpoint and eligibility criteria have been harmonized for this illustration, use independent uniform beta priors, and set \(w=0.5\) for the external likelihood.
The treatment posterior is \(\mathrm{Beta}(1+6,1+8-6)=\mathrm{Beta}(7,3)\), with mean \(7/(7+3)=0.7000\). For the external control, the weighted event count is \(0.5(2)=1\), and the weighted non-event count is \(0.5(18)=9\). Thus the control posterior is \(\mathrm{Beta}(2,10)\), with mean \(2/12=0.1667\).
The posterior mean difference in response probability, treatment minus control, is \(0.7000-0.1667=0.5333\), or 53.33 percentage points. This is a posterior mean contrast under the stated model, not proof that treatment caused the difference. The posterior variances are \(7(3)/[10^2(11)]=0.01909\) and \(2(10)/[12^2(13)]=0.01068\). If the posteriors are independent, the difference has variance \(0.01909+0.01068=0.02977\), giving a posterior standard deviation of approximately \(0.1725\). The substantial uncertainty is important despite the large difference in posterior means.
The calculation assumes that the external outcomes can inform the trial’s control probability. If the natural-history cohort used a different response definition, had materially different disease severity, or assessed patients less often, the numerical posterior would not repair that mismatch.
Aligning the External Outcome to the Trial Population
When prognosis varies across baseline characteristics, a crude external rate may not represent the trial population. One practical approach is to estimate control outcomes within relevant strata and average them using the trial population’s stratum proportions. This is direct standardization: it targets the trial’s observed covariate distribution, provided the strata are sufficiently comparable and have external data support.
In a Bayesian analysis, each stratum-specific rate is uncertain. If the stratum posteriors are independent, the variance of the standardized rate is the sum of the stratum variances multiplied by the squared target-population weights. This carries external sampling uncertainty into the target estimate rather than treating estimated rates as known.
Worked Example: Standardizing for Baseline Severity
Hypothetical data. A trial includes 6 patients with mild disease and 4 with severe disease. By a fixed follow-up time, 2 of 10 treated patients have an adverse event. In an aligned external cohort, 2 of 20 mild patients and 8 of 20 severe patients have the event. Use independent \(\mathrm{Beta}(1,1)\) priors for all event probabilities, and treat the trial’s severity proportions, 0.6 and 0.4, as the target weights.
The treatment posterior is \(\mathrm{Beta}(3,9)\), with mean \(3/12=0.2500\). The mild external posterior is \(\mathrm{Beta}(3,19)\), with mean \(3/22=0.1364\). The severe external posterior is \(\mathrm{Beta}(9,13)\), with mean \(9/22=0.4091\). The posterior mean of the standardized external event risk is therefore \(0.6(0.1364)+0.4(0.4091)=0.2455\).
The posterior mean risk difference, treatment minus standardized control, is \(0.2500-0.2455=0.0045\), or about 0.45 percentage points. This small mean contrast is not evidence of equivalence: the trial is small and both estimates are uncertain. For a \(\mathrm{Beta}(a,b)\) distribution, the variance is \(ab/[(a+b)^2(a+b+1)]\). The mild and severe posterior variances are \(57/11132=0.00512\) and \(117/11132=0.01051\). Thus the standardized control variance is \(0.6^2(0.00512)+0.4^2(0.01051)=0.00352\). The treatment variance is \(27/[12^2(13)]=0.01442\). Assuming independent posteriors, the risk-difference standard deviation is \(\sqrt{0.01442+0.00352}=0.1340\).
This is a wide uncertainty relative to the estimated contrast. The calculation also treats the target weights as fixed and assumes adequate comparability within severity strata. If important prognostic factors remain imbalanced, severity standardization alone is not sufficient.
Diagnostics, Alternatives, and Limitations
Before interpreting a borrowed estimate, inspect whether the external data actually resemble the trial cohort in eligibility, baseline prognosis, care, calendar time, follow-up, and endpoint recording. Check the number of external patients in each relevant subgroup; large overall sample size can conceal little or no support in a clinically important stratum. Review outcome missingness and censoring patterns, and ask whether they could depend on prognosis. The earlier material on calendar-time drift, posterior predictive checks, and benchmarking against concurrent controls supplies relevant diagnostic approaches.
A useful diagnostic is sensitivity to reasonable modeling choices. Recompute estimates using little or no borrowing, alternative defensible weights, and appropriate baseline adjustment. Large changes indicate that the conclusion depends materially on external information or on modeling assumptions. They are a reason to report that dependence, not a reason to select the most favorable result. When available, a small concurrent randomized arm can anchor the comparison and help reveal discrepancies between trial and external controls; it does not eliminate the need to assess selection and measurement differences.
| Approach | Strength | Key limitation |
|---|---|---|
| Concurrent randomized control | Randomization supports comparability in expectation. | May be difficult to recruit and can leave substantial uncertainty in a rare disease. |
| External control with no borrowing | Makes the separation between trial and historical evidence explicit. | Does not use external outcomes to improve precision for the control rate. |
| Discounted external borrowing | Uses some external information while allowing reduced contribution. | The chosen discount does not correct bias and may strongly influence results. |
| Adjusted or standardized external comparison | Can address measured population differences relevant to prognosis. | Requires sufficient overlap and adequate measurement of prognostic factors. |
The central limitation is that external-control evidence is generally not protected by randomization. Unmeasured confounding, changes in care, informative outcome observation, and selection into data sources may remain even after careful adjustment. Bayesian posterior uncertainty quantifies uncertainty under the specified model; it does not automatically include uncertainty from every possible source of bias. A credible report therefore distinguishes model-based uncertainty from the additional uncertainty due to design and data limitations.
Common Pitfall
Worked Example: Sensitivity to the Borrowing Weight
Hypothetical data. Suppose 3 of 30 external patients experience an adverse event, while 4 of 12 treated trial patients experience it. Use independent uniform beta priors. Compare fixed borrowing weights \(w=1\), \(w=0.25\), and \(w=0\), where the contrast is the posterior mean treatment risk minus the posterior mean control risk.
The treatment posterior is \(\mathrm{Beta}(5,9)\), with mean \(5/14=0.3571\). At \(w=1\), the external posterior is \(\mathrm{Beta}(4,28)\), with mean \(4/32=0.1250\); the mean contrast is \(0.3571-0.1250=0.2321\). At \(w=0.25\), the weighted external counts are \(0.75\) events and \(6.75\) non-events. The posterior is \(\mathrm{Beta}(1.75,7.75)\), with mean \(1.75/9.5=0.1842\), and the contrast is \(0.3571-0.1842=0.1729\). At \(w=0\), the external data contribute nothing and the uniform prior gives a control posterior mean of \(0.5000\); the contrast is \(0.3571-0.5000=-0.1429\).
The direction of the posterior mean contrast changes across these choices. This does not establish that one weight is correct; it shows that the data do not support a weight-insensitive conclusion. The analysis would need a justified borrowing strategy, transparent sensitivity results, and cautious interpretation. In particular, the \(w=0\) result is based on the prior alone for the control probability, not on an observed concurrent control group.
Check Your Understanding
Answer each question, then compare with the solutions below.
- What does a power weight of zero mean for the contribution of external outcomes?
- An external registry measures progression at 12 months, while the trial’s primary outcome is progression at 24 weeks. Why is the registry rate not automatically a control estimate for the trial?
- In a binary-outcome model, an external cohort has 4 events and 16 non-events, and \(w=0.5\). With a \(\mathrm{Beta}(1,1)\) prior, what is the resulting posterior mean control risk?
- Why can a narrow posterior interval still fail to provide a credible estimate of the trial’s control outcome?
Solutions. (1) It sets the external likelihood contribution to zero; the control posterior is determined by its prior and any other data in the model. (2) The endpoint and follow-up horizons differ, so the observed event probabilities do not necessarily represent the same outcome. A justified transformation or harmonized estimate would be needed. (3) The weighted counts are 2 events and 8 non-events, giving \(\mathrm{Beta}(3,9)\) and mean \(3/(3+9)=0.2500\). (4) A narrow interval describes uncertainty under the specified probability model. It cannot rule out bias from noncomparability, unmeasured prognostic factors, or differential outcome ascertainment.