Why Ascertainment Must Be Comparable
An external-control comparison can be carefully adjusted for measured patient characteristics and still be misleading if the outcome was not sought or classified in the same way across data sources. A trial may assess response at prespecified visits and use blinded central review; a routine-care cohort may record response only when a clinician documents it. An event can then be missed, detected later, or classified differently, even when the underlying clinical experience is the same.
The task is not to make records look alike. It is to state the clinical outcome of interest precisely, establish how each source operationalizes it, and assess what differences in observation could do to the treatment contrast. This is especially important when borrowing an external control: an apparent difference in response, progression, or adverse events can reflect ascertainment as well as treatment.
Specify the Target Before Comparing Records
A usable endpoint specification identifies the event or state, the population and time origin, the time window, and the rule for classifying the outcome. For a binary response endpoint, it might specify confirmed response by 24 weeks after treatment initiation, requiring two qualifying assessments at least four weeks apart. For a time-to-event endpoint, it should also specify the event date rule and how censoring is handled. A shared label such as “response” or “progression-free survival” is not enough if the underlying rules differ.
Compare each source on at least three dimensions:
- Definition: Are the qualifying findings, thresholds, confirmation requirements, and handling of ambiguous records the same?
- Assessment schedule: Are assessments planned at comparable times, and what happens when visits are missed or delayed?
- Adjudication: Who classifies the endpoint, with what information, and blinded to which treatment or source details?
A common operational definition can sometimes be applied retrospectively to both sources by re-reviewing source records. That is more persuasive than simply relabeling existing fields. If necessary records are absent in one source, the endpoint may not be harmonizable for that analysis. It is then better to report the limitation or narrow the estimand than to assume that undocumented events did not occur.
Worked Example: Comparing Response Definitions
Suppose a hypothetical trial defines response as a complete or partial response confirmed at a later assessment by week 24. An external oncology registry records the clinician’s best response by week 24, without requiring confirmation. The trial reports 42 responses among 140 patients; the registry reports 39 among 130.
The recorded proportions are \(42/140=0.300\), or 30.0%, and \(39/130=0.300\), also 30.0%. The equality of these proportions does not establish comparable response rates. The trial’s confirmation rule can exclude transient responses that the registry includes. Conversely, the registry may miss responses that were not entered in routine documentation.
A defensible harmonization plan would first determine whether the registry has assessment dates and findings that allow the trial’s confirmation rule to be applied. If it does, both sources can be reclassified under that rule. If it does not, the two reported proportions remain different operational endpoints: their numerical equality does not make them the same clinical measure.
Quantifying Differential Misclassification
For a binary endpoint at a fixed time, let \(Y=1\) indicate that the endpoint truly occurred under the target definition and \(Z=1\) indicate that the source recorded it as occurring. In source \(g\), define sensitivity \(Se_g=P(Z=1\mid Y=1)\) and specificity \(Sp_g=P(Z=0\mid Y=0)\). If the true endpoint risk is \(p_g=P(Y=1)\), the recorded risk \(q_g=P(Z=1)\) satisfies
The first term is the fraction of the population with a true event that is recorded as positive; the second is the fraction without a true event that is incorrectly recorded as positive. If \(Se_g+Sp_g>1\), solving for the true risk gives
This correction is only as credible as its inputs. It assumes that the stated sensitivity and specificity apply to the population, endpoint definition, and follow-up window being analyzed. It also assumes that the validation information used to estimate them is appropriate for each source. The corrected risk must lie between 0 and 1; a result outside this range signals incompatible inputs or substantial uncertainty, not a risk to truncate silently.
Worked Example: Correcting a Risk Contrast for Different Ascertainment
Consider a hypothetical study comparing a trial treatment group with an external control group. At a fixed follow-up time, the trial records 48 events among 200 patients and the external cohort records 60 among 200. Suppose an independent validation review supports sensitivity 0.90 and specificity 0.98 for the trial, and sensitivity 0.70 and specificity 0.95 for the external cohort.
The recorded risks are \(q_T=48/200=0.240\) and \(q_C=60/200=0.300\). The uncorrected risk difference, treatment minus control, is \(0.240-0.300=-0.060\), or 6.0 percentage points fewer recorded events in the treatment group.
For the trial group,
For the external controls,
The corrected treatment-minus-control risk difference is \(0.250-0.3846=-0.1346\), approximately 13.5 percentage points fewer events in the treatment group. Under the assumed validation values, less sensitive ascertainment in the external cohort had concealed some control events, making the uncorrected contrast closer to zero. This calculation quantifies one possible ascertainment effect; it does not address confounding or other differences between the cohorts.
Schedules and Adjudication Change What Is Seen
Assessment frequency is one mechanism for changing sensitivity. A patient whose event occurs between visits may be detected at the next visit, detected late, or never documented. For a fixed-time binary endpoint, this can be represented through the source-specific sensitivity in the preceding model, provided the endpoint and time window are truly the same. For time-to-event analyses, the timing of detection also matters: an event known only to have occurred between two assessments is interval-censored, not necessarily known to have occurred on the first positive visit date.
Adjudication can affect both sensitivity and specificity. A central reviewer applying a prespecified rule may classify borderline cases differently from a treating clinician. Blinding matters because knowledge of treatment or data source can influence ambiguous judgments. When feasible, blinded re-adjudication of records from both sources using one written rule is a strong harmonization strategy. It cannot recover missing images, laboratory results, or clinical notes that were never collected.
Worked Example: A Less Frequent Schedule and Missed Events
In a hypothetical comparison, the true 6-month event risks are 0.20 in the treated group and 0.35 in the external-control group. Assume specificity is 1.00 in both sources, so no event is falsely recorded among patients without an event. Trial visits detect 95% of true events by 6 months, while the external records detect 70%.
With perfect specificity, the recorded risks are \(q=Se\times p\). Thus the trial records \(0.95\times0.20=0.190\), and the external cohort records \(0.70\times0.35=0.245\). The recorded treatment-minus-control difference is \(0.190-0.245=-0.055\), or 5.5 percentage points fewer events.
The true difference under the assumptions is \(0.20-0.35=-0.15\), or 15 percentage points fewer events. The less frequent external assessment misses a larger share of control events, attenuating the observed contrast. This example assumes the 6-month target is common to both groups and that the stated detection probabilities are applicable; if the assessment windows differ, the recorded risks may not even refer to the same time-specific endpoint.
A Bayesian Model for Ascertainment Uncertainty
Sensitivity and specificity are rarely known exactly. Treating their estimates as fixed can make a corrected treatment contrast appear more certain than the evidence warrants. A Bayesian model can represent uncertainty in the true risks and in ascertainment parameters together. For example, conditional on group \(g\), one can model the number of recorded events \(K_g\) among \(n_g\) patients as
Priors for \(p_g\), \(Se_g\), and \(Sp_g\) should reflect the target population and relevant validation evidence. The posterior then describes uncertainty in the risks and their contrast after accounting for observed counts and the assumed observation process. If validation data are available, they can inform the ascertainment parameters through a likelihood of their own. If they are absent, strong conclusions about correction may be impossible; a sensitivity analysis over plausible ascertainment values is more transparent than presenting a single adjusted estimate as established fact.
Worked Example: How the Ascertainment Model Changes Posterior Support
Consider a deliberately small hypothetical Bayesian illustration for an external-control event risk. There are 50 patients and 16 recorded events. For clarity, suppose specificity is 1, sensitivity is known to be 0.80, and the prior assigns probability 0.50 to each of two possible true risks, 0.25 and 0.50. This discrete prior is illustrative, not a recommended default.
Under true risk 0.25, the recorded-event probability is \(0.80\times0.25=0.20\); under true risk 0.50, it is \(0.80\times0.50=0.40\). The likelihood ratio favoring risk 0.50 over risk 0.25 is
Because the prior odds are 1, the posterior odds are approximately 3.70, giving posterior probability \(3.70/(1+3.70)=0.787\) for the risk of 0.50. If the same recorded data were instead modeled as perfectly ascertained, the likelihood ratio favoring 0.50 over 0.25 would be
That model gives posterior probability \(0.0675/(1+0.0675)=0.0633\) for risk 0.50. The data are identical; the observation model changes their interpretation. In an applied analysis, uncertainty in sensitivity and specificity would also be propagated, rather than fixed at 0.80 and 1.00 as in this illustration.
Diagnostics, Alternatives, and Limitations
Useful diagnostics start with the data-generating process, not just the final event counts. Tabulate assessment times and missing visits by source; compare the proportion with records sufficient to apply the common rule; inspect the timing of events relative to assessments; and examine disagreement between routine classifications and blinded re-adjudication where both are available. Validation samples should be checked for representativeness: a sensitivity estimate from complete, clear-cut cases may not apply to patients with sparse or ambiguous records.
A practical comparison of approaches is:
| Approach | When it is useful | Main limitation |
|---|---|---|
| Reclassify both sources under one operational definition | Underlying records contain the required measurements and dates | Cannot recover information that was never collected |
| Use a misclassification model with validation data | Sensitivity and specificity can be estimated for each source | Results depend on validation relevance and model assumptions |
| Report a sensitivity analysis across plausible ascertainment values | Ascertainment parameters are uncertain or only indirectly supported | Does not identify which values are correct |
| Restrict or redefine the analysis endpoint | A shared, supportable endpoint can be measured in a narrower subset or window | May change the target question and reduce usable data |
A common mistake is to treat equal assessment schedules or matching endpoint names as proof of equal ascertainment. The correction is to compare the actual rules, opportunities for observation, and classification process. Another mistake is to “correct” an observed risk using sensitivity and specificity from a different population without checking applicability. The formulas do not make weak validation evidence strong.
Misclassification correction also does not remove confounding, selection bias, calendar-time differences, or informative missingness. Nor does a common endpoint definition ensure that outcome ascertainment is independent of treatment or cohort. If event capture differs in ways not represented by the model, a corrected estimate may still be biased. Report the operational definitions, validation basis, assumptions, and sensitivity analyses alongside the contrast.
Check Your Understanding
Try each question, then compare your reasoning with the solutions.
- A trial requires response confirmation, while an external registry records best response without confirmation. What must be checked before comparing their response proportions?
- An external cohort has recorded event risk \(q=0.28\), sensitivity 0.70, and specificity 0.96. Calculate the corrected risk under the misclassification model.
- Why can less frequent follow-up make a treatment appear more effective when events are otherwise missed?
- What is gained by including uncertainty in sensitivity and specificity in a Bayesian model?
Solutions. (1) Check whether the registry contains enough dated assessments and findings to apply the trial confirmation rule to both sources; otherwise, the endpoints differ operationally. (2) \(p=(0.28+0.96-1)/(0.70+0.96-1)=0.24/0.66=0.364\), approximately 36.4%, conditional on the assumed sensitivity and specificity. (3) If fewer control events are detected, the recorded control risk can be biased downward, shrinking the treatment-minus-control difference and potentially making treatment appear less beneficial; the direction depends on which group has less detection. (4) It prevents uncertain validation estimates from being treated as exact and allows their uncertainty to contribute to posterior uncertainty in the risks and treatment contrast.