Why Harmonization Comes Before Borrowing
A power prior can reduce the influence of historical control data, but discounting is not a substitute for making sure the data represent the same outcome in a comparable population. If “event” means different things across trials, or if one trial observes outcomes for a shorter period, the historical likelihood may describe a different quantity from the current-control likelihood. Combining them can produce a precise posterior for an ill-defined mixture rather than useful information about the current control outcome.
Harmonization is the process of checking and, where justified, transforming the recorded data so the historical and current controls refer to a shared outcome, coding convention, assessment window, and eligibility-defined population. Some differences can be corrected directly from records. Others require a model or cannot be resolved with the available information. The key is to establish comparability before applying a power-prior discount.
For a binary outcome, a common reference model is \(Y_j\mid\theta\sim\operatorname{Binomial}(n_j,\theta)\), where \(Y_j\) counts participants with the defined event in study \(j\), \(n_j\) is the number assessed, and \(\theta\) is the event probability for the target outcome and population. A power prior uses a historical likelihood contribution such as \(L_0(\theta\mid D_0)^a\), with discount \(a\), alongside the current-control likelihood. This construction is meaningful only if both likelihoods concern compatible data-generating quantities, or if differences have been handled explicitly in the model.
What Must Match
Begin by specifying the target outcome in words precise enough that two reviewers could apply it to the same record and reach the same classification. Then audit the historical and current data on four dimensions. These checks are linked: a difference in eligibility can change who is at risk, while a difference in follow-up can change whether an event is observed.
- Endpoint: Are the same clinical construct and event criteria being measured? Check the event components, the relevant baseline or index date, and how events are ascertained. A shared label does not guarantee a shared endpoint.
- Coding: Does the same value represent the event in both data sets? Confirm the meaning of every category, including missing, unknown, and not-applicable codes. Recode from source records when possible; never infer coding from a label alone.
- Follow-up: Is the outcome assessed over the same period from a comparable time origin? A cumulative event indicator at 30 days is not interchangeable with one at 90 days.
- Eligibility: Do the historical participants meet the current trial’s relevant inclusion and exclusion criteria? If not, determine whether the historical records contain enough information to identify and retain a comparable subset.
The goal is not to erase every study difference. It is to decide whether the data can be represented as observations of the same target outcome in the target population, and to document what changes were made to reach that representation. Harmonization cannot repair an outcome that was never measured, recover follow-up that was not observed, or recreate participant-level eligibility information missing from an aggregate summary.
A Likelihood Justification for Requiring Common Coding
The reason coding matters can be seen directly from the binomial likelihood. Suppose \(Y\) counts adverse events among \(n\) participants and \(\theta\) is the adverse-event probability. The likelihood, ignoring a factor that does not depend on \(\theta\), is:
If a historical study instead codes favorable outcomes as 1, and \(S\) is the number coded 1, then \(S=n-Y\). Treating \(S\) as though it counted adverse events gives a likelihood proportional to \(\theta^{n-Y}(1-\theta)^Y\), which is the likelihood for the complementary event probability, not for the adverse-event probability \(\theta\). Raising this likelihood to a power \(a\) reduces its influence but does not change which event its parameter describes.
More generally, let \(R_j\) denote the raw record in study \(j\), and let \(h_j(R_j)\) be the documented transformation that maps it to the common outcome \(Z_j\). For straightforward borrowing, the transformed historical and current data should support the same outcome model for \(Z\), conditional on any modeled covariates. If the transformations do not produce observations of the same target, the analyst must not assume that a smaller power-prior weight makes the mismatch harmless.
Worked Example: Reversing a Binary Outcome Code
Suppose a hypothetical historical control group has 120 participants. Its data dictionary records favorable outcome as 1, and 96 participants have code 1. The current control group has 90 participants, with adverse event coded as 1: 18 have an adverse event and 72 do not. Assume the source records verify that favorable and adverse are mutually exclusive and exhaustive categories for this endpoint.
To use adverse event as the common event, recode the historical records: the historical adverse-event count is \(120-96=24\), and the non-event count is \(96\). The current counts are already in the desired coding. The harmonized data therefore have historical \(Y_0=24\) out of \(n_0=120\) and current \(Y_c=18\) out of \(n_c=90\).
For illustration, take a \(\operatorname{Beta}(1,1)\) prior for the common event probability and a fixed historical power \(a=0.5\). With compatible binomial likelihoods, the posterior is:
The posterior mean is \(31/(31+121)=31/152\approx0.2039\). This is a model-based estimate of the adverse-event probability under the assumed common outcome and population. The calculation is justified only if the endpoint, follow-up, and relevant eligibility are also compatible; correct coding alone is insufficient.
Follow-Up and Eligibility Need Their Own Checks
A follow-up mismatch is not usually fixed by changing a label. If the historical binary outcome indicates whether an event occurred by day 90 and the current outcome indicates whether it occurred by day 30, the two event probabilities are different cumulative risks. Even when the event definition is identical, later observation gives more opportunity to record an event. A common target time must be chosen for the analysis and supported by the available records.
Worked Example: Aligning a Follow-Up Cutoff
In a hypothetical study, the historical control records show 28 first events by day 90 among 100 participants; the event dates are available, and 17 events occurred by day 30. The current control records show 20 first events by day 90 among 90 participants, of which 12 occurred by day 30. Suppose all participants in both groups have outcome ascertainment through day 30, and the planned target is event by day 30.
The harmonized 30-day binary counts are therefore \(17/100\) historically and \(12/90\) currently. The corresponding observed proportions are \(17/100=0.1700\) and \(12/90=0.1333\). The 90-day counts, \(28/100\) and \(20/90\), must not be substituted into a model for the 30-day endpoint.
This restriction is defensible because the event dates identify which outcomes occurred by the common cutoff, and all participants were observed through that cutoff. If only the two 90-day totals were available, the 30-day counts could not be recovered from those totals alone. In that case, the data do not support this direct harmonization.
Eligibility differences create a related problem. A historical overall event rate may not describe the current trial’s target population when the historical trial enrolled participants who would be excluded today. Individual records can sometimes be filtered using the current criteria. An aggregate event total generally cannot be reliably filtered unless it includes the relevant eligibility-specific breakdowns.
Worked Example: Restricting to Comparable Eligibility
A hypothetical historical control group contains 100 participants. Applying the current trial’s documented eligibility criteria to the available individual records retains 60 participants, of whom 9 experienced the target event. The other 40 participants would not have been eligible for the current trial; 14 of them experienced the event. A current control group has 60 eligible participants, with 8 events.
For the stated target population, the historical subset is \(9/60=0.1500\), and the current proportion is \(8/60=0.1333\). The historical overall rate, \((9+14)/100=23/100=0.2300\), is not the rate for the eligible subset. Borrowing based on the overall total would include outcomes from participants outside the target eligibility group.
The restriction is supported here because eligibility can be determined for each historical participant and the outcome is available for the retained subset. If only the aggregate total of 23 events among 100 participants were known, with no eligibility-specific data, the eligible subset’s event count could not be identified. The aggregate rate should not be presented as if it were the eligible-population rate.
A Practical Decision Process
State the event, coding convention, time origin and follow-up cutoff, and population to which the current control estimand refers.
Use protocols, data dictionaries, and records to identify differences in definitions, codes, observation windows, and eligibility.
Recode verified labels, derive a shared follow-up outcome from observed event times, or restrict records when the required variables are available. Do not invent unavailable information.
Confirm that counts reconcile before and after transformations, record exclusions and unresolved differences, and decide whether the resulting data support the intended borrowing model.
A useful audit table can record, for each study, the endpoint source and definition, code meanings, time origin and assessment cutoff, eligibility criteria, available participant-level variables, and the resulting action. Keep the original records and transformations traceable. Unexpected counts after recoding, an unclassified code, or a retained participant with incomplete eligibility information is a reason to pause and investigate, not a reason to choose a discount that hides the discrepancy.
Alternatives, Diagnostics, and Limits
| Approach | When it is appropriate | Main caution |
|---|---|---|
| Harmonize and borrow | Differences can be resolved from reliable records, yielding a common outcome and target population. | Document each transformation; apparent similarity is not enough. |
| Restrict to a common subset | Participant-level eligibility and outcome data identify the subset relevant to the current trial. | Restriction can reduce historical information and may leave a small or selected subset. |
| Use a model that represents the difference | Differences are measured and can be represented by an appropriate outcome or covariate structure. | A model must be justified by the available data; merely adding a study indicator does not automatically make distinct estimands comparable. |
| Do not borrow the incompatible data | The discrepancy cannot be resolved, or the common outcome or target population cannot be identified. | This forgoes potential information but avoids treating non-comparable outcomes as observations of one quantity. |
Common warning signs include a historical event rate that changes sharply after a code audit, inconsistent numbers of participants eligible at the common cutoff, and records whose outcome codes cannot be interpreted from the source documentation. These checks are diagnostics of data compatibility, not tests that can prove two studies are exchangeable. Even after successful harmonization, differences between studies may remain and should be considered in the broader borrowing analysis.
A power-prior discount controls the contribution of a historical likelihood under a specified model. It does not convert a 90-day endpoint into a 30-day endpoint, infer an unrecorded code definition, or remove outcomes from historically ineligible participants when only aggregate data are available. When a discrepancy cannot be resolved, describe it plainly and consider a justified alternative rather than treating discounting as a repair.
Common Mistake
Correction: A discount scales the historical likelihood; it does not change the outcome represented by that likelihood. First determine whether the historical records can be recoded and restricted to the same endpoint, coding, follow-up, and target population. If not, the data cannot be treated as directly comparable through discounting alone.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A historical study codes “no event” as 1, while the current study codes “event” as 1. What must be checked before recoding the historical data?
- Historical event-by-day-90 totals are available, but no event dates or day-30 totals are reported. Can those totals be used directly for an event-by-day-30 analysis?
- A historical study has 40 participants who fail current eligibility criteria, but only its overall event count is available. Can the eligible-subset event rate be calculated from the overall count alone?
- Why does choosing a smaller historical power-prior discount not resolve incompatible endpoint definitions?
Solutions. (1) Verify from source documentation that the categories have the stated meanings and form a complete, unambiguous coding of the same endpoint; then recode consistently. (2) No. The 90-day total does not reveal how many events occurred by day 30, so it cannot directly supply the 30-day count. (3) No. Without eligibility-specific outcomes or participant-level records, the event count among eligible participants is unknown. (4) The discount reduces the historical likelihood’s influence but leaves its outcome definition unchanged; incompatibility must be resolved or the data must not be directly borrowed.