Why Discordant Predictions Need Adjudication
An integrated development program may contain several models that address connected but nonidentical questions. A PBPK model may predict exposure from physiology and drug properties; a population PK model may estimate exposure from clinical concentration data; an exposure-response model may predict an outcome conditional on exposure; and a trial simulation may predict outcomes after combining exposure, patient characteristics, dosing, and trial design. Different numerical predictions are not, by themselves, evidence that one model is defective. They may reflect different targets, conditions, assumptions, data, or uncertainty.
Adjudicating discordant predictions means identifying which of these explanations accounts for a disagreement, and whether the remaining difference matters for the decision at hand. It is not a procedure for selecting a winner by averaging predictions. First establish that the predictions can be compared; then locate the source of any material discrepancy and document what remains unresolved. The earlier material on cross-model posterior predictive checks and residual patterns across model handoffs provides useful diagnostics, but this task focuses on tracing disagreement across the evidence chain.
Make the Predictions Comparable Before Explaining Them
Start by writing each model output in words and units. For example, “mean AUC from zero to 24 hours after the fifth dose, in adults with normal renal function receiving 100 mg once daily” is more useful than “exposure.” For a clinical outcome, specify the endpoint, its assessment time, and whether the prediction is conditional on a fixed exposure or averaged over a population’s exposure distribution. A conditional prediction and a population-average prediction may differ even when both models are correct.
Check that the scenario passed between models is the scenario each receiving model actually uses. Relevant details can include dose timing, adherence, covariate distributions, dose adjustments, missing-data rules, and whether variability represents between-person differences or uncertainty about model parameters. These are not interchangeable. In particular, the earlier material on separating variability from knowledge uncertainty matters here: a predictive distribution for individual outcomes is generally wider than uncertainty about a population mean.
If the targets are aligned, quantify the difference and its uncertainty. Let \(A\) and \(B\) denote predictions of the same scalar quantity, and let \(D=A-B\). When the prediction estimates are approximately jointly normal, with standard errors \(SE_A\) and \(SE_B\) and correlation \(\rho\), the variance of their difference is
This follows from \(\operatorname{Var}(A-B)=\operatorname{Var}(A)+\operatorname{Var}(B)-2\operatorname{Cov}(A,B)\), and \(\operatorname{Cov}(A,B)=\rho SE_A SE_B\). The standardized difference \(D/SE_D\), where \(SE_D=\sqrt{\operatorname{Var}(D)}\), summarizes the size of the gap relative to uncertainty under these assumptions. It is a diagnostic, not a probability that either model is correct. If the joint distribution is non-normal, quantities are nonlinear, or the prediction is a tail probability, comparing paired simulation draws or intervals on the target scale is usually more informative than applying this normal approximation.
Worked Example: Comparing Two Predictions of Mean AUC
Suppose two hypothetical models predict mean AUC over the same 24-hour interval, after the same dose, in the same target population. The PBPK prediction is 24.0 mg·h/L with uncertainty \(SE_A=2.0\) mg·h/L; the population PK prediction is 20.0 mg·h/L with \(SE_B=2.0\) mg·h/L. Suppose their prediction estimates have correlation \(\rho=0.50\), because they use some shared information. Assume their uncertainties are approximately jointly normal and refer to uncertainty in the predicted mean, not the full variability of individual patients.
The difference is \(D=24.0-20.0=4.0\) mg·h/L. Its standard error is
Thus \(D/SE_D=4.0/2.0=2.00\). The gap is two standard errors on the stated approximation. This flags a difference to investigate; it does not establish that the PBPK prediction is wrong or provide a formal test that the models are equivalent. If the correlation had been ignored, the calculated standard error would be \(\sqrt{8}=2.83\) mg·h/L, understating the standardized difference. The comparison is meaningful only because the estimand and scenario were aligned and the dependence was included.
Trace the Source in a Controlled Sequence
Once comparability is established, trace the disagreement from the output back through the handoff. A useful practice is to change one explanation at a time, holding the others fixed. This helps distinguish a difference caused by the target or interface from one caused by model structure, parameter values, or evidence. It also makes the investigation auditable: each explanation should predict what changes in the output if it is correct.
Record the endpoint, units, population, dose, time horizon, and conditioning variables for each prediction. Recalculate or restate outputs on a shared target where possible.
Compare the quantity actually passed between models with the quantity expected by the receiving model. Verify units, transformations, covariate distributions, and whether dependence and uncertainty were preserved.
Compare predictions at intermediate quantities, such as concentration-time profiles before comparing AUC, or exposure distributions before comparing trial outcomes. The first point where outputs separate narrows the search.
List the influential assumptions and ask whether the relevant data identify them, validate them, or leave them extrapolated. Check whether either model is being used outside its supported population or exposure range.
Report what discrepancy remains, its uncertainty, the explanation best supported by evidence, and whether resolving it could change the development decision. Keep unresolved disagreement visible.
Worked Example: Conditional and Trial-Average Response Predictions
A hypothetical exposure-response model predicts response probability using \(p(C)=1/(1+\exp[-(-2+0.04C)])\), where \(C\) is exposure in a stated concentration unit. An analyst reports the prediction at \(C=80\), while a trial simulation averages outcomes over a population in which half the patients have \(C=80\) and half have \(C=40\). Assume the endpoint and assessment time are identical, and the exposure-response relationship applies to both exposure levels.
At \(C=80\), the linear predictor is \(-2+0.04(80)=1.2\), so \(p(80)=1/(1+\exp(-1.2))=0.7685\), or 76.85%. At \(C=40\), it is \(-2+0.04(40)=-0.4\), so \(p(40)=1/(1+\exp(0.4))=0.4013\), or 40.13%.
The simulated population-average response probability is
The trial simulation therefore predicts 58.49%, about 18.36 percentage points below the 76.85% conditional prediction at \(C=80\). This is not evidence of model conflict: the first number describes patients fixed at \(C=80\), while the second averages over two exposure levels. The calculation assumes the stated exposure distribution and response model are appropriate; if the simulated exposure distribution, endpoint, or time point differs from the intended trial, that interface must be checked separately.
Follow Assumptions Back to Their Evidence
When predictions remain discordant after alignment, identify which assumptions have the leverage to produce the observed gap. In a PBPK versus population PK comparison, inspect the predicted concentration-time profile, bioavailability, clearance pathways, and covariate effects before focusing on a summary such as AUC. Ask whether clinical concentration data cover the relevant dose, population, and interaction conditions. A mechanistic parameter may be scientifically plausible yet weakly identified by the available evidence.
For exposure-response versus trial-simulation disagreement, compare the exposure distribution, the response curve over that distribution, and the trial’s endpoint and design assumptions. If the simulation predicts outcomes beyond the exposure range supporting the response model, the result is extrapolation, not direct confirmation. The earlier material on exposure estimation error and carrying parameter covariance across model handoffs is relevant: measurement error or omitted dependence can distort downstream comparisons even when component models are individually well fitted.
A practical diagnostic is to compare intermediate predictions on common scales. If two models predict similar concentration-time profiles but different AUCs, first check integration windows and calculations. If their profiles diverge before the outcome model is involved, investigate PK assumptions and data support. If PK outputs agree but trial outcomes differ, investigate the exposure distribution, response function, endpoint definition, or trial implementation. A discrepancy that first appears at a specific handoff is a useful clue, although it does not by itself prove the handoff is the cause.
Worked Example: Tracing a Drug-Interaction Prediction
Consider a hypothetical drug for which a PBPK model predicts an AUC ratio of 2.14 with versus without an inhibitor, while a clinical population PK analysis estimates a ratio of 1.30 (95% interval 1.10 to 1.54). Assume both ratios refer to the same dose, steady-state condition, population, and AUC definition. To illustrate a mechanistic audit, suppose the PBPK calculation uses the simplified relation \(R=1/[(1-f_m)+f_m/(1+x)]\), where \(f_m=0.80\) is the fraction of baseline clearance through the inhibited pathway and \(x=I/K_i\) is the unbound inhibitor concentration relative to its inhibition constant. This relation assumes linear PK, reversible inhibition, and no other inhibitor effects.
With the PBPK assumption \(x=2.0\),
The estimate is close to the stated PBPK prediction of 2.14. To see what value of \(x\) would correspond to the clinical estimate under this simplified relation, solve \(1/1.30=0.20+0.80/(1+x)\). Since \(1/1.30=0.7692\), \(0.80/(1+x)=0.7692-0.20=0.5692\), so \(1+x=0.80/0.5692=1.405\), and \(x=0.405\). The clinical estimate is therefore compatible with a substantially smaller effective \(I/K_i\) than the PBPK assumption, under the stated assumptions.
This calculation identifies a testable lead, not a final explanation. The analyst should check whether the unbound inhibitor concentration, inhibition constant, inhibited fraction of clearance, interaction conditions, and clinical analysis target are supported by evidence. The clinical interval includes 1.30, but that alone does not prove the mechanism or validate every assumption in the population PK model. If the simplified relation omits relevant pathways, solving it cannot establish the true \(I/K_i\).
Diagnostics, Alternatives, and Limits
Warning signs include mismatched units or time windows, an unexplained shift at a model handoff, exposure values outside the response model’s data range, implausible intermediate predictions, and uncertainty intervals that omit shared inputs. Check calculations, code, and data transformations before interpreting a scientific explanation. If uncertainty is estimated by simulation, compare draws generated under matched scenarios; independent draws can conceal dependence and exaggerate or understate the uncertainty of a difference.
The main alternatives serve different purposes. A component-level diagnostic asks where predictions first diverge; targeted data collection asks which uncertain assumption can be tested; and scenario analysis asks whether the decision changes across plausible assumptions. These approaches can support adjudication but do not automatically resolve it. Formal model weighting or averaging answers a different question—how to combine predictive contributions—and should not substitute for discovering that two models predict different estimands or use incompatible inputs. A disagreement can also remain genuinely unresolved when available data do not distinguish competing assumptions.
The standardized difference in the first example is limited by its normality and uncertainty assumptions. For nonlinear endpoints, asymmetric distributions, or near-boundary probabilities, report the difference on the decision-relevant scale and inspect the full simulation distributions. Nor does a small discrepancy prove adequacy: both models may share a misspecified assumption or lack support in the target setting. The goal is a defensible account of the discrepancy, not a forced declaration that one prediction is correct.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Two models predict mean clearance for different populations. What must be done before treating the numerical difference as model discordance?
- Two estimates have standard errors 1.5 and 2.0 units and correlation 0.40. Calculate the standard error of their difference.
- An exposure-response model reports response at one fixed exposure, while a trial simulation reports a population average. Name one reason the two can differ without model conflict.
- In the interaction example, what does the implied \(x=0.405\) establish, and what does it not establish?
Solutions. (1) Define a common estimand and align population, treatment, time horizon, endpoint, and conditioning information; otherwise the difference may be due to different targets. (2) \(SE_D=\sqrt{1.5^2+2.0^2-2(0.40)(1.5)(2.0)}=\sqrt{2.25+4.00-2.40}=\sqrt{3.85}=1.962\) units. (3) The simulation averages the conditional response over a distribution of exposures, whereas the other prediction is conditional on a single exposure; nonlinearity means these need not be equal. (4) Under the simplified relation and fixed \(f_m=0.80\), it is the value of \(I/K_i\) that reproduces the clinical point estimate. It does not prove that this is the true inhibitor exposure-to-potency ratio or that the simplified mechanism is complete.