What Does Transfer Across Sites Mean?
A PBPK model may predict well in one study and be intended for use in settings that differ in investigators, clinical sites, sampling procedures, or analytical laboratories. External validation across sites asks whether the model’s predictions remain informative in those independent settings. The question is not simply whether two data tables look alike: each observed result must be compared with the prediction for the conditions that actually produced it.
Site and protocol differences matter because they can change either the pharmacokinetics or how it is observed. A different infusion duration changes the concentration-time profile. A different sampling schedule may miss a peak. A different assay may report systematically different concentrations even when the underlying samples are equivalent. These possibilities should be identified and evaluated rather than hidden by pooling all observations.
The model and parameter set must be frozen before examining validation outcomes. The studies must be independent in the relevant sense: validation outcomes, participants, or measurements must not have been used to choose the model, tune its parameters, or make analysis decisions that depend on those outcomes. The earlier material on calibration and independent validation and on data leakage establishes why this separation matters.
Define What Is Being Compared
Before comparing sites, state the prediction target. It may be a concentration at a specified time, a profile over a specified interval, or another prespecified quantity. Define the population, dose, route, formulation, dosing schedule, time origin, sampling times, and concentration units. A prediction at two hours after a bolus is not automatically comparable with a measurement two hours after the start of a prolonged infusion.
For site \(j\) and protocol \(k\), write \(C_{\mathrm{pred},jk}(t)\) for the model-predicted concentration at time \(t\), conditional on the protocol and relevant study inputs, and \(C_{\mathrm{obs},jk}(t)\) for the observed concentration. The indices are reminders that a prediction should reflect the site’s actual protocol when that protocol changes the simulated conditions. They do not imply that the model is separately fitted at each site.
Some site differences are part of the intended transfer question; others obstruct a fair comparison. Differences in participant characteristics may be part of the target setting, while differences in dose timing or sample handling may make two measurements represent different conditions. Make this distinction explicit. Do not silently treat unmatched conditions as if they were identical.
Assay Sources and Measurement Comparability
An assay difference can create apparent disagreement even if the underlying concentrations are the same. Relevant details include assay platform, calibration standard, laboratory, sample matrix, sample processing, and lower and upper measurement limits. Record these details for each study. A change of laboratory alone does not prove that measurements are incompatible, and the same nominal units do not prove that they are interchangeable.
When an independently established assay bridge is available, it can support a conversion to a common measurement scale. For example, if a bridge study supports the relationship \(C_{\mathrm{raw}}=a+bC_{\mathrm{common}}\), then \(C_{\mathrm{common}}=(C_{\mathrm{raw}}-a)/b\), for \(b\ne 0\). This conversion is defensible only if the bridge applies to the relevant assay, matrix, concentration range, and procedures. Its uncertainty should be acknowledged. Estimating a correction from the validation observations themselves risks using the test data to improve apparent agreement.
Worked Example: Comparing Results From Two Assay Sources
Suppose two independent sites study the same hypothetical dose and collect samples at four hours after dosing. Both report concentrations in mg/L. Site A uses the common assay scale. At Site B, an independently conducted bridge study supports \(C_{\mathrm{raw}}=0.20+1.10C_{\mathrm{common}}\) over the concentrations of interest. The fixed PBPK prediction for the matched conditions is 8.00 mg/L. Site A reports 8.10 mg/L, and Site B reports 9.44 mg/L.
For Site A, no conversion is needed. Its observed value differs from the prediction by \(8.10-8.00=0.10\) mg/L.
For Site B, apply the bridge: \[ C_{\mathrm{common}}=\frac{9.44-0.20}{1.10} =\frac{9.24}{1.10} =8.40\ \text{mg/L}. \] On the common scale, the difference from the matched prediction is \(8.40-8.00=0.40\) mg/L. The raw Site B value would have suggested a difference of \(9.44-8.00=1.44\) mg/L, but that comparison mixes measurement scales.
The bridge makes the comparison more interpretable; it does not prove the model is accurate at either site. The bridge’s range and uncertainty, the independence of its data, and other protocol differences still need to be reported. If the bridge did not apply to these samples, the appropriate conclusion would be that assay comparability is unresolved.
Match Predictions to Protocols Before Comparing Sites
A model prediction should reproduce the relevant administered regimen and observation times. If an infusion lasts one hour at one site and three hours at another, simulate each duration rather than imposing one common regimen. If time is recorded from the end of infusion at one site and from dose initiation at another, first determine whether the records can be mapped to a common time origin. If they cannot, present the comparisons separately.
A useful justification follows directly from the meaning of a protocol-conditioned prediction. Let \(P_A\) and \(P_B\) denote two protocols. In general, the PBPK model permits \(C_{\mathrm{pred}}(t;P_A)\ne C_{\mathrm{pred}}(t;P_B)\). Comparing an observation under \(P_B\) with \(C_{\mathrm{pred}}(t;P_A)\) therefore combines model disagreement with the effect of changing the protocol. Simulating \(P_B\) removes that avoidable mismatch; it does not establish that every remaining difference is caused by the model, because measurement and sampling differences may remain.
Worked Example: Different Infusion Durations and Time Origins
Consider two hypothetical independent studies of the same drug and nominal dose. At Site A the dose is infused over one hour, and concentrations are timed from infusion start. At Site B the dose is infused over three hours, and concentrations are timed from infusion end. At a recorded time of two hours, the observed concentrations are 6.2 mg/L at Site A and 5.1 mg/L at Site B.
The recorded times do not represent the same point in the dosing schedule. Two hours after infusion start at Site A is one hour after its infusion ended. Two hours after infusion end at Site B is five hours after its infusion started. A direct comparison of 6.2 and 5.1 mg/L would mix different infusion durations and different elapsed times.
Suppose the frozen model, run with the actual protocols, predicts 6.0 mg/L at Site A’s recorded time and 5.0 mg/L at Site B’s recorded time. The observation-prediction differences are \(6.2-6.0=0.2\) mg/L at Site A and \(5.1-5.0=0.1\) mg/L at Site B. These are protocol-matched comparisons; the raw site-to-site difference of \(6.2-5.1=1.1\) mg/L is not evidence by itself of a site effect or a change in model performance.
This comparison is interpretable only if dose, infusion duration, time origins, concentration definitions, and assays have been recorded correctly. If the time origin at Site B cannot be confirmed, the apparent protocol-matched comparison should be qualified rather than treated as definitive.
Keep Uncertainty and Site-Level Evidence Visible
A site-specific comparison is more informative than a pooled number when site conditions may differ. Report the number of participants, available sampling times, observed summaries and their uncertainty, corresponding model predictions, and relevant study differences by site. A pooled summary can conceal a setting in which predictions are less suitable, or make a large site appear to dominate simply because it enrolled more participants.
Distinguish sampling uncertainty in an observed summary from uncertainty in the model prediction. For a sample mean, a standard error describes sampling variability under the assumptions used to calculate it; it does not include uncertainty in the model parameters, assay bridge, or protocol records. Conversely, a model prediction interval, if available, has its own stated sources of uncertainty. Do not treat either interval as if it automatically includes all sources.
Worked Example: Interpreting Different Site-Level Evidence
Suppose a fixed PBPK model predicts a mean concentration of 10.0 mg/L at a prespecified time under the matched protocol at each of two hypothetical sites. At Site A, \(n=16\), the observed mean is 9.6 mg/L, and the sample standard deviation is 2.0 mg/L. At Site B, \(n=25\), the observed mean is 7.8 mg/L, and the sample standard deviation is also 2.0 mg/L. Assume independent participants within each site and use a \(t\)-interval for each site mean.
The standard error is \(SE=s/\sqrt{n}\). At Site A, \(SE=2.0/\sqrt{16}=0.500\) mg/L. Using \(t_{0.975,15}=2.131\), the 95% interval is \[ 9.6\pm 2.131(0.500)=9.6\pm1.066, \] or approximately 8.53 to 10.67 mg/L.
At Site B, \(SE=2.0/\sqrt{25}=0.400\) mg/L. Using \(t_{0.975,24}=2.064\), the 95% interval is \[ 7.8\pm2.064(0.400)=7.8\pm0.826, \] or approximately 6.97 to 8.63 mg/L.
These intervals describe uncertainty in each site’s observed mean under the stated sampling assumptions. They do not include model uncertainty or assay-bridge uncertainty. The Site A interval includes the point prediction of 10.0 mg/L, whereas the Site B interval does not. This contrast is a reason to inspect the Site B protocol, assay, population, and data handling carefully; it is not, by itself, proof of why the results differ or a formal test of model validity. A conclusion should consider the prespecified acceptance criteria and the full uncertainty relevant to the prediction.
Document Differences and Diagnose Comparability Problems
A concise site-by-site record makes it possible to distinguish evidence about model transfer from evidence about study conduct. Include the study and site identifiers; participant population; dose, route, formulation, and schedule; time origin and sampling schedule; sample collection and processing; assay source and units; handling of missing or out-of-range measurements; model version and parameter set; and any transformations or external assay bridges. Explain which differences are expected to affect the prediction and which affect measurement comparability.
Several patterns should trigger investigation before a broad transfer claim is made. A disagreement concentrated in one assay source may point to a measurement-scale issue. A discrepancy that follows different infusion durations may reflect protocol mismatch. Sparse sampling near a rapidly changing concentration can make a nominal time-point comparison fragile. Unclear time stamps, inconsistent units, or an undocumented conversion prevent a secure interpretation even if the numerical results appear close.
These are diagnostic clues, not automatic explanations. Do not remove a site because its results are inconvenient, adjust its observations until they agree with the model, or use validation outcomes to revise assay conversions. If a discrepancy leads to model changes, those changes require new independent validation data for an unbiased assessment.
Approaches, Limitations, and a Common Pitfall
There are two common ways to organize the evidence. A site-stratified presentation shows matched predictions and observed results separately for each study or site. It preserves heterogeneity and highlights where transfer may be limited, but can be imprecise when sites are small. A pooled presentation summarizes results across sites and can be useful when the studies share a defensible target and measurement scale. Pooling, however, can obscure site-specific problems and should not substitute for documenting differences. When protocols or assays are not comparable, separate reporting is often more defensible than a single combined summary.
External validation cannot establish transfer to every unobserved site, protocol, population, or assay. The strength of a transfer claim depends on how well the validation settings represent the intended use and whether the relevant sources of variation have been documented. A close match in one site does not guarantee performance elsewhere, and a discrepancy at one site does not automatically invalidate every application.
Common pitfall: “The pooled observations are close to the pooled prediction, so the model transfers across sites.” This reasoning can be wrong because large sites may dominate the pooled result, assay offsets may cancel, or protocol mismatches may be averaged together. Instead, match predictions to actual conditions, present site-level evidence, explain measurement differences, and qualify the conclusion according to the sites and protocols actually evaluated.
Record the model version, parameter set, intended use, and quantities to be predicted before reviewing validation outcomes.
Document the site, population, dosing and sampling protocol, time origin, assay, and data handling relevant to each comparison.
Simulate the actual protocol and use only justified, independently established measurement conversions.
Show observed results, matched predictions, uncertainty, and unresolved differences before making a bounded transfer claim.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A site measures time from the end of infusion while the simulation output is indexed from infusion start. What must be checked before comparing concentrations?
- An assay bridge was estimated using the validation study’s own samples. Why is this a concern?
- Why might a pooled comparison appear acceptable even when one site’s results are poorly matched?
- At a site, \(n=9\) and \(s=3.0\) mg/L. Calculate the standard error of the sample mean. What uncertainty does it describe?
- A prediction matches one independent site, but that site used the same assay and protocol as the calibration studies. What can be concluded about transfer to different assays and protocols?
Solutions. (1) Confirm the time origins and map the recorded sample time to the simulation time under the actual infusion schedule; if that cannot be done, the comparison is not securely aligned. (2) The conversion may be tuned to the validation outcomes, making agreement look better and compromising independence. Use an applicable bridge established independently, or report measurement comparability as unresolved. (3) Large sites can dominate the pooled summary, and opposing discrepancies across sites can cancel. Show site-level results and conditions. (4) \(SE=s/\sqrt{n}=3.0/\sqrt{9}=1.00\) mg/L. It describes sampling variability of the sample mean under the assumptions used; it does not include model or assay uncertainty. (5) The result supports performance only in the evaluated setting. It does not establish transfer to different assays or protocols that were not evaluated.