Pediatric External Controls From Registries

Why Pediatric Registry Controls Need Special Scrutiny

A registry can offer valuable information about the expected course of a rare pediatric disease, but a large sample does not by itself make it a suitable control group. Children’s prognosis, treatment, and measurement can change substantially with age and development. A registry may also enroll children under different eligibility rules, record outcomes at different intervals, or reflect a different era of care. These differences can make registry outcomes an unreliable representation of what trial participants would have experienced without the trial treatment.

The central task is to assess whether the registry and trial describe comparable children at a comparable point in their disease course, under sufficiently similar care and outcome measurement. This assessment comes before interpreting a Bayesian comparison: a statistical model can represent uncertainty in the observed data, but it cannot make an ineligible registry population exchangeable with the trial population or make differently ascertained outcomes equivalent.

Let \(S=1\) indicate participation in the trial and \(S=0\) indicate inclusion in the registry. Let \(X\) denote measured baseline characteristics, \(A\) the treatment strategy, and \(Y\) the endpoint over a specified follow-up period. A common target is the outcome risk under control treatment in the trial-eligible population, \(E(Y^0\mid S=1)\), where \(Y^0\) is the outcome that would occur under control treatment. The registry can inform this quantity only under assumptions about eligibility, measurement, overlap, and conditional comparability. The target population and follow-up period must be stated; “children with the disease” is generally not specific enough.

Eligibility, Index Dates, and Age Overlap

Age eligibility should be checked against the trial protocol, not inferred from the registry’s broad label of “pediatric.” Identify the age at the trial’s treatment assignment or other prespecified index date, the age at registry entry, and the ages represented over follow-up. A child’s age at diagnosis may differ from age at treatment initiation, and using different time origins can align children at different disease stages. Registry inclusion should likewise be anchored to a date that corresponds to the trial’s index date as closely as the data permit.

The comparison also needs age overlap. If the trial includes infants but the registry contains only school-age children, adjustment cannot recover infant control outcomes from registry observations alone. Even where ages overlap, broad categories can conceal meaningful differences: infancy, early childhood, and adolescence may carry different prognoses and care pathways. Age may need to be represented continuously or in clinically justified bands, with enough observations in each relevant range to estimate outcomes.

Developmental Covariates and Prognostic Comparability

Chronological age is not a complete description of pediatric development. Depending on the disease and endpoint, relevant baseline covariates may include growth or body size, pubertal status, organ function, developmental or functional status, disease severity, age at onset, prior disease trajectory, and previous treatment. The useful covariates are those that can affect prognosis, treatment selection, or outcome measurement and that differ between the registry and trial populations. A long list of variables is not a substitute for a clinically justified assessment.

Measure these characteristics at a comparable baseline. A developmental measure recorded after treatment starts may be affected by treatment or disease progression; treating it as a baseline adjustment variable can distort the comparison. Check how each variable was defined, when it was recorded, and whether the registry captures it with comparable completeness. Missingness can itself vary by source or severity. If the registry records functional status only for children attending specialist visits, for example, the recorded subset may not represent all eligible registry children.

Standardization makes one part of the comparability problem explicit. If \(X\) is a set of measured baseline covariates and registry outcomes can be estimated within relevant covariate profiles, the target control risk can be represented as

This expression requires, among other conditions, comparable outcome definitions, sufficient overlap in \(X\), and conditional exchangeability: after conditioning on \(X\), the registry control outcome must represent the trial population’s control outcome. It also requires that registry participants classified as controls actually received the relevant control strategy. Standardization adjusts for measured differences; it does not demonstrate that all important differences have been measured.

Treatment Era and Outcome Ascertainment

A registry may span years in which treatment standards, supportive care, diagnostic practice, or referral patterns changed. These calendar-time differences matter when they affect control outcomes or the kinds of children observed. Earlier material on calendar-time drift and external-control selection bias provides relevant concepts; the pediatric assessment should ask whether the registry’s control-care era is clinically compatible with the trial’s era and whether the registry includes patients selected under different referral or eligibility practices.

Record the dates of diagnosis, registry enrollment, treatment initiation, and outcome measurement where available. Compare actual control-care strategies, not just calendar years. A change in supportive care could alter event risk even if the named disease treatment is unchanged. Restricting to a compatible era may improve comparability but reduce sample size and precision. If treatment era and patient characteristics change together, an apparent era effect may also reflect changes in who entered the registry.

Outcome ascertainment requires a separate check. Compare endpoint definitions, assessment schedules, instruments, thresholds, adjudication procedures, and the handling of death, withdrawal, or loss to follow-up. A registry based on routine visits may detect events less often than a trial with scheduled assessments. Conversely, intensive trial surveillance may identify mild events that routine care misses. Harmonization must address both what counts as an event and how likely it is to be observed.

Assessment Strategy and Alternatives

A useful assessment makes each key assumption visible and checks whether the data can support it. The following sequence helps organize the review without treating any one adjustment as a guarantee of validity.

Restriction to a clinically compatible age range or care era is easier to explain than extrapolating across non-overlapping groups, but it can discard information and leave a small control sample. Matching or weighting can align measured distributions when overlap is adequate; extreme weights signal that a small number of registry observations are being asked to represent much of the trial population. Standardization is useful when outcome estimates are available for relevant covariate profiles, but it also depends on those profiles being sufficiently populated. A concurrent randomized control arm remains the strongest direct way to protect against many measured and unmeasured baseline differences. A concurrent external registry can reduce calendar-time mismatch but still differs from the trial in selection and measurement.

In a Bayesian analysis, registry observations may contribute information to the model for control outcomes, but posterior uncertainty is conditional on the data model and its assumptions. A narrow posterior distribution cannot compensate for a violated eligibility restriction, a missing developmental subgroup, or differential outcome detection. Therefore, assess design compatibility and data quality alongside model-based uncertainty. Sensitivity analyses should vary assumptions that are uncertain and consequential, such as outcome ascertainment or the impact of an incompatible era, rather than merely repeat the same analysis with minor computational changes.

Diagnostics, Pitfalls, and Limitations

Warning signs include trial ages outside the registry’s observed range, empty or sparse developmental subgroups, large or unstable adjustment weights, substantial differences in missingness, and event rates that change sharply across registry periods. Differences in endpoint frequency may also reflect visit schedules or definitions rather than clinical outcomes. These signals do not automatically invalidate a registry, but they identify where the proposed comparison relies on extrapolation or additional assumptions.

• Using a broad pediatric label as an eligibility check. “Pediatric” can include ages outside the trial’s range. Apply the actual age and clinical criteria at a comparable index date.
• Assuming age adjustment captures development. Children of the same age can differ in growth, puberty, function, and disease stage. Include relevant, reliably measured developmental covariates where supported.
• Treating a measured risk as a comparable risk. A different assessment schedule can change the probability an event is recorded. Harmonize the endpoint and evaluate ascertainment differences.
• Equating calendar-time adjustment with exchangeability. Modeling year cannot by itself remove changes in care, referral, eligibility, or unmeasured prognosis.
• Trusting a precise estimate despite poor overlap. Precision within the observed registry data does not validate predictions for trial subgroups that the registry scarcely represents.

The available registry may not measure some developmental factors or may lack the detail needed to reconstruct trial eligibility. Such limitations should be treated as uncertainty about the comparison, not hidden by a more elaborate model. The defensible conclusion may be that the registry provides contextual or supportive evidence but cannot serve as a credible stand-alone control for the intended target population.