Why Model Transport Requires More Than a New Prediction
A pharmacometric model is often developed in one patient population and then considered for use in another. The populations may differ in age, body size, organ function, disease severity, treatment, or study conduct. A change in average predicted exposure or response does not by itself show that the model has failed: it may simply reflect a different mix of patients. The key question is whether the model still describes the relationship between patient characteristics and the outcome in the new population.
Transporting a model means using its specified relationships and parameter values to make predictions for a target population, while assessing whether those predictions remain appropriate there. It is not a claim that the populations are identical. It is a claim that the model’s relevant conditional relationships can be applied to the target patients, possibly after a justified recalibration.
The model’s application also depends on consistent definitions and measurements. For example, a covariate measured at baseline in the source data should have a comparable meaning and timing in the target data. A change in assay, outcome definition, dosing protocol, or sampling design can change what the model input or outcome represents. A numerical prediction is not transportable merely because the same variable names appear in two data sets.
Assumptions and a Decomposition of Population Differences
A useful first distinction is between a change in covariate distribution and a change in the conditional outcome relationship. Write \(f_S(x)\) and \(f_T(x)\) for the covariate distributions in the source and target populations. The source model’s average prediction for a population with covariate distribution \(f\) is the average of \(m_S(x)\) over that distribution. Comparing source and target averages gives:
Here \(m_T(x)=E(Y\mid X=x,T)\) is the target population’s conditional mean. The first term is the change attributable to the covariate mix, using the source relationship. The second is the difference between target and source conditional relationships, averaged over the target covariate distribution. The identity follows by adding and subtracting \(\int m_S(x)f_T(x)\,dx\) from \(E_T(Y)=\int m_T(x)f_T(x)\,dx\).
This decomposition is informative only when the covariates include the important effect modifiers and there is adequate overlap: target patients should generally have covariate values represented in the source data. If the target contains a subgroup outside the source model’s support, the source relationship there is an extrapolation, not an empirically checked transport. The assumption that the conditional relationship is stable after accounting for relevant covariates is substantive; it cannot be established by algebra alone.
Worked Example: A Changed Covariate Mix Changes Mean Clearance
Worked Example: A Changed Covariate Mix Changes Mean Clearance
Suppose a hypothetical source model predicts clearance from body weight as \(CL=5.00(WT/70)^{0.75}\) L/h. The source and target populations contain only patients weighing 70 or 84 kg, but the proportions differ. The model is assumed, for this illustration, to describe clearance equally well at both weights in both populations.
At 70 kg, the model predicts \(5.00\) L/h. At 84 kg:
The increase from 70 to 84 kg is approximately \(5.733-5.00=0.733\) L/h, or \((0.73265/5.00)\times100\%=14.7\%\). These are individual model predictions, not observed target-population results.
Now suppose 80% of the source population weighs 70 kg and 20% weighs 84 kg. Its mean model prediction is \(0.80(5.00)+0.20(5.73265)=5.14653\), or approximately 5.147 L/h. If the target proportions are 20% and 80%, respectively, its mean model prediction is \(0.20(5.00)+0.80(5.73265)=5.58612\), or approximately 5.586 L/h. The target mean prediction is therefore about \(5.58612-5.14653=0.43959\) L/h higher, or 0.440 L/h rounded to three decimals.
This difference follows from the changed weight distribution under the same conditional model. It is not evidence of model failure. That conclusion depends on the example’s stated assumption that the weight-clearance relationship applies in both populations; target observations would be needed to evaluate that assumption in practice.
Checking Target Predictions and Choosing an Adjustment
When target-population outcomes are available, compare predictions with observations on the outcome scale and in clinically meaningful subgroups. Define a residual as observed outcome minus predicted outcome. Residuals near zero on average are not sufficient by themselves: opposing errors across subgroups can cancel. Check the overall bias, the spread of errors, trends across predictions and covariates, and patterns tied to population membership or study conditions. Use uncertainty intervals appropriate to the data and design; a small target sample can make an apparently reassuring pattern inconclusive.
Recalibration adjusts a limited part of the prediction rule while retaining its main structure. For a continuous outcome, one simple option is an intercept adjustment: add a constant to all predictions. Another is a linear recalibration of observed outcomes on predictions, with an intercept and slope. These changes may correct systematic offset or scale error, but they should not be used to conceal subgroup-specific errors. A structural revision changes the conditional model itself, for example by adding a missing covariate effect or interaction. Evidence for such a revision should be evaluated against the target data and scientific rationale, with the increased complexity and its uncertainty considered.
Worked Example: Estimating a Mean Offset in Target Data
In a hypothetical target-population check, four patients have predicted clearances of 4.0, 5.0, 6.0, and 7.0 L/h. Their observed clearances are 5.0, 6.3, 6.9, and 8.2 L/h. Assume the same measurement definition is used for predictions and observations.
The observed-minus-predicted residuals are \(1.0,\ 1.3,\ 0.9,\) and \(1.2\) L/h. Their mean is:
An intercept-only adjustment based on these data adds 1.10 L/h to each prediction, giving 5.10, 6.10, 7.10, and 8.10 L/h. The adjusted residuals are \(-0.10,\ 0.20,\ -0.20,\) and \(0.10\) L/h, whose mean is zero. This arithmetic shows how the offset is chosen; it does not establish that the adjustment will work for future patients. Four observations provide limited evidence, and the adjustment should be evaluated on separate data or with an appropriate validation design before clinical use.
The mean residual is the intercept that minimizes the sum of squared residuals when the only permitted adjustment is a common offset. If the adjusted prediction is \(p_i+c\), the residual is \(r_i-c\). Differentiating the sum of squared residuals with respect to \(c\) gives:
This result justifies the calculation, but not the assumption that a constant offset is scientifically adequate. Residual patterns determine whether that simple adjustment is plausible.
Worked Example: A Shared Offset Cannot Repair a Subgroup Pattern
Consider a hypothetical target population with two groups. The source model predicts 5.0 L/h for each patient. Observed clearances are 5.0 and 5.1 L/h in the reference group, and 6.4 and 6.6 L/h in a second group defined by a disease characteristic that the model does not include. Assume that measurements are comparable between groups.
The reference-group mean residual is \(\{(5.0-5.0)+(5.1-5.0)\}/2=0.05\) L/h. The second-group mean residual is \(\{(6.4-5.0)+(6.6-5.0)\}/2=1.50\) L/h. The overall mean residual is \((0+0.1+1.4+1.6)/4=0.775\) L/h.
Adding the overall offset of 0.775 L/h makes the adjusted predictions 5.775 L/h for everyone. The adjusted residuals are \(-0.775,\ -0.675,\ 0.625,\) and \(0.825\) L/h. Their overall mean is zero, but the reference group remains systematically overpredicted and the second group underpredicted. A single average correction has concealed rather than resolved the group pattern.
This pattern motivates investigation of a group-specific covariate effect or another population difference. The four observations are only illustrative: they are not sufficient to establish an interaction or support a structural change. A larger, representative target data set, appropriate uncertainty assessment, and scientific justification would be needed.
Diagnostics, Alternatives, and Limits
A transport assessment should ask not just whether an overall summary agrees, but where and how predictions fail. Useful checks include plotting residuals against predicted values and relevant covariates; summarizing bias and prediction error by clinically important subgroups; checking target covariate ranges against source ranges; and checking for changes in assay, dosing, sampling, or outcome definition. Patterns may signal a missing relationship, a scale problem, inconsistent measurement, or a target group with too little source support. No single diagnostic proves the cause.
The main practical approaches answer different questions. Applying the unchanged model tests its current predictions. Recalibration makes a limited adjustment to systematic error. Structural revision changes the model’s conditional relationship. A model developed or refit using target data may be suitable when evidence shows that the source model is inadequate, but it requires enough target information and independent evaluation. The earlier material on external validation with an independent trial concerns prediction checks in new data; transport adds the specific need to distinguish population composition from changes in conditional relationships.
| Approach | What changes? | When it may be appropriate | Main caution |
|---|---|---|---|
| Apply unchanged model | Nothing in the model | Source assumptions and target prediction checks are credible | Covariate support and residual patterns still need assessment |
| Recalibrate | A limited offset or scale relationship | Target data show a systematic error that a simple adjustment can describe | Overall agreement can hide subgroup error; validate the adjustment |
| Revise model structure | Conditional effects or relationships | Target evidence and scientific reasoning indicate a missing or changed relationship | Additional complexity can overfit sparse target data |
| Develop a target model | Model parameters and possibly structure | The source model is unsuitable and target data are adequate | Requires sufficient data and evaluation beyond the fitting sample |
Transport conclusions are limited by the target data’s size and representativeness, the range of covariates observed, and the reliability of measurements. Lack of an obvious residual pattern in a small sample is not evidence that meaningful errors are absent. If important target covariate values have no source analogues, prediction depends on extrapolation and should be presented with corresponding caution. Any recalibration or revision should preserve the intended use, state its data basis, and be checked on data not used to choose that adjustment whenever feasible.
Common Mistake
- Wrong reasoning: “The target mean prediction is higher than the source mean, so the model is biased in the target.”
- Correction: A higher mean can arise solely from a changed covariate distribution, as in the clearance example. Compare conditional predictions at relevant covariate values and, when possible, assess residuals against target observations before attributing the difference to model failure.
- Wrong reasoning: “The overall mean residual is zero after adding an offset, so the target model is calibrated.”
- Correction: Subgroup residuals can remain systematically different even when their overall mean is zero. Inspect error patterns and assess the adjustment on appropriate validation data.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A target population has more patients with a covariate associated with higher predicted clearance. The conditional clearance model is assumed unchanged. Should the target mean prediction necessarily equal the source mean? Explain.
- In a target data set, residuals are observed minus predicted. Four residuals are \(-0.2,\ 0.1,\ 0.3,\) and \(0.2\) L/h. What intercept adjustment minimizes the sum of squared residuals, and what is its direction?
- After an overall offset adjustment, residuals average \(-0.4\) L/h in one subgroup and \(+0.4\) L/h in another. What does the zero overall average establish, and what does it not establish?
- Why is prediction for target patients outside the source covariate range more uncertain than prediction for patients within that range?
Solutions. (1) No. If the model is unchanged but the target covariate distribution differs, the average of the same conditional predictions can change. (2) The mean residual is \((-0.2+0.1+0.3+0.2)/4=0.10\) L/h, so the minimizing offset is to add 0.10 L/h to each prediction. (3) It establishes only that the subgroup residual averages cancel in the overall average. It does not establish adequate predictions within either subgroup; the opposite signs suggest subgroup-specific error that should be investigated. (4) The source data do not directly test the model relationship at those covariate values. Predictions there rely on extrapolation, so support from the source data for the assumed conditional relationship is weaker.