Why Validate Across Doses and Regimens?
A QSP model can reproduce data used to estimate its parameters yet make poor predictions at a dose or schedule that was not used during calibration. The relevant question is not simply whether the model fits a response at one dose. It is whether the locked model predicts the sequence of events from drug exposure, through target engagement, to pharmacodynamic (PD) response under conditions that were withheld from model development.
Dose and regimen are distinct challenges. A higher dose tests whether the model captures how responses change over the dose range. A different schedule tests whether it captures how the timing and shape of exposure affect target engagement and downstream response. The same total dose over a dosing interval does not guarantee the same concentration-time profile, target engagement, or PD trajectory.
Let \(d\) denote dose level and \(r\) denote regimen. Write \(\hat{C}_{d,r}(t)\), \(\hat{E}_{d,r}(t)\), and \(\hat{R}_{d,r}(t)\) for model-predicted drug concentration, target engagement, and PD response at time \(t\). The corresponding observations can be written \(C_{d,r,\mathrm{obs}}(t)\), \(E_{d,r,\mathrm{obs}}(t)\), and \(R_{d,r,\mathrm{obs}}(t)\). Here \(E\) might be fractional occupancy or another measured engagement quantity, and \(R\) must have a clearly defined scale, such as cytokine concentration or percentage inhibition.
What Must Be Held Fixed
A meaningful predictive comparison requires more than labeling a dose as “withheld.” Before comparing predictions with data, check that the model, parameter values, and analysis choices were fixed without using those validation outcomes. Dose amount, route, administration times, and relevant covariates must be represented correctly in the simulation. Observation times, units, baseline definitions, and assay scales must also be aligned between predictions and measurements.
The comparison assumes that validation observations are informative about the population and experimental context for which the claim is made. For example, if the model predicts a typical response but the validation sample is enriched for a subgroup with different disease severity, a mismatch may reflect population composition as well as model structure. Measurement uncertainty matters too: a small difference relative to assay variability is not equivalent to a consistent discrepancy beyond that variability.
A useful comparison follows the biological chain rather than relying on a single summary. Exposure checks ask whether the simulated concentration profile is plausible. Engagement checks ask whether the model translates exposure into the expected degree and duration of target interaction. PD checks ask whether the model predicts the downstream response, including its timing. These are complementary checks, not independent proofs: a discrepancy in one layer can propagate to later layers.
Comparing Dose Levels
Dose-level validation should examine the pattern across the withheld range, not just whether one high-dose prediction is close. A concentration-time summary such as area under the concentration curve (AUC) can help compare total exposure, while peak concentration and trough concentration can reveal differences that AUC alone hides. For engagement and PD, compare both meaningful summaries and trajectories at times that can reveal onset, peak, and recovery.
For a positive observed AUC, one descriptive measure of signed relative discrepancy is
A positive value means the model overpredicts AUC; a negative value means it underpredicts. This ratio is not useful when the observed quantity is zero or near zero, where an absolute difference on a meaningful scale is preferable. Neither measure supplies a universal pass threshold: interpretation depends on the intended use, measurement uncertainty, and the consequences of prediction error.
Worked Example: A Dose-Dependent Divergence
Suppose a hypothetical model was calibrated without data from three dose levels. The validation assay reports AUC in mg·h/L, target engagement as a percentage, and PD suppression as a percentage from baseline. The table gives the observed and predicted summaries.
| Dose | AUC predicted / observed | Engagement predicted / observed | PD suppression predicted / observed |
|---|---|---|---|
| 10 mg | 20 / 21 mg·h/L | 32 / 30% | 18 / 20% |
| 30 mg | 60 / 57 mg·h/L | 65 / 62% | 48 / 45% |
| 60 mg | 120 / 99 mg·h/L | 82 / 72% | 70 / 60% |
At 10 mg, the AUC discrepancy is
so predicted AUC is about 4.76% below observed AUC. At 60 mg,
so predicted AUC is about 21.2% above observed AUC. The engagement and PD predictions also exceed the observations at 60 mg, by 10 and 10 percentage points, respectively. The pattern is more informative than a single overall average: agreement is closer at the lower dose, while the highest dose shows overprediction at all three layers. This suggests a possible high-dose limitation in the model or its inputs, but these summaries alone do not identify the cause. The conclusion is that the high-dose predictions are less consistent with these hypothetical validation data, not that a particular mechanism has been disproved.
Why Regimen Shape Matters
A schedule changes when concentrations rise and fall, not merely how much drug is administered in total. Even if two schedules have the same dose summed over an interval, a nonlinear exposure-to-engagement relationship can produce different average engagement. For example, consider the illustrative relationship \(E(C)=C/(K+C)\), where \(K\) is a positive concentration scale. Compare two equally long periods: one with concentration \(C=0\) for half the time and \(C=4K\) for the other half, and another with constant concentration \(C=2K\). Both have mean concentration \(2K\). Their mean engagement values are
Thus equal mean concentration does not imply equal mean engagement under this nonlinear relationship. In a QSP model, downstream response dynamics can add further schedule dependence: response may lag behind engagement, persist after concentration declines, or recover on a different timescale. These properties must be evaluated against time-course data rather than inferred from total dose alone.
Worked Example: Same Total Dose, Different Schedule
In a hypothetical 28-day validation experiment, one regimen administers 40 mg on day 1; the other administers 10 mg on days 1, 8, 15, and 22. Both deliver 40 mg over 28 days. The model and parameters were fixed before either regimen's observations were examined. The following are hypothetical summaries over the 28-day interval.
| Regimen | AUC predicted / observed | Maximum concentration predicted / observed | Mean engagement predicted / observed | Day-28 PD suppression predicted / observed |
|---|---|---|---|---|
| 40 mg once | 40 / 38 mg·h/L | 8.0 / 7.6 mg/L | 44 / 40% | 36 / 33% |
| 10 mg weekly | 40 / 39 mg·h/L | 2.8 / 2.7 mg/L | 68 / 66% | 57 / 55% |
For the 40 mg once regimen, the relative AUC discrepancy is
or 5.26% above the observed AUC. For the weekly regimen it is
or 2.56% above observed AUC. Despite similar interval AUCs, the weekly schedule has a lower maximum concentration and higher mean engagement and day-28 suppression in both predictions and observations. The model captures that schedule contrast in these summaries, although it overpredicts mean engagement by 4 percentage points and day-28 suppression by 3 percentage points for the single-dose regimen. A reasonable conclusion is that the model reproduces several regimen-level differences in this example, while the remaining discrepancies warrant examination. Equal total dose, by itself, would not have established this result.
Use the Three Layers to Interpret Mismatch
The sequence of exposure, engagement, and PD can help narrow down where a prediction departs from data. If concentration predictions are already discrepant, later engagement and PD differences may follow from an input or exposure-model mismatch. If exposure is close but engagement is not, the exposure-to-target relationship or engagement measurement deserves attention. If both exposure and engagement are close but PD differs, the mismatch may lie in the downstream response relationship, response timescale, or PD measurement. These are hypotheses for follow-up, not diagnoses established by the comparisons alone.
Time courses matter because two predictions can agree on an endpoint while disagreeing on the path to that endpoint. Compare observations and predictions at matched times, including times selected to capture onset, peak, and recovery. Sampling too sparsely can make a missed peak or delayed recovery invisible. When measurements use different units or scales, examine each on its own interpretable scale rather than combining them into an unqualified single score.
Worked Example: Similar Exposure, Different PD Trajectory
A hypothetical withheld regimen gives an observed 72-hour concentration AUC of 44 mg·h/L and a model prediction of 45 mg·h/L. At five observation times, target engagement and PD suppression are recorded as follows. All percentages are relative to their stated assay scales.
| Time (h) | Engagement predicted / observed (%) | PD suppression predicted / observed (%) |
|---|---|---|
| 0 | 0 / 0 | 0 / 0 |
| 6 | 60 / 58 | 5 / 4 |
| 24 | 45 / 43 | 22 / 18 |
| 48 | 22 / 24 | 35 / 30 |
| 72 | 10 / 12 | 28 / 26 |
The PD prediction errors, calculated as predicted minus observed, are 0, 1, 4, 5, and 2 percentage points. Their root mean squared error across these five times is
For engagement, the errors are 0, 2, 2, -2, and -2 percentage points, giving
The engagement trajectory and AUC are fairly close in these summaries, while the model overpredicts PD suppression at 24 and 48 hours. This pattern points attention toward the downstream engagement-to-PD relationship or its timing. It does not prove that this relationship is the cause: assay error, sparse sampling, or population differences may also contribute. The calculation summarizes the observed time points and should not be mistaken for evidence about unobserved times.
Diagnostics, Alternatives, and Limitations
Before interpreting a mismatch biologically, check whether it could arise from the validation setup. Confirm the dose units and administration times, the interval used for AUC, the baseline and direction of the PD scale, the assay's quantification range, and the alignment of observation times. Check that the simulation uses the intended covariates and that the data being called withheld did not influence parameter selection or model revisions. A seemingly systematic high-dose error can also be created by a unit conversion or an incorrect dosing event.
There are several useful comparison strategies, each answering a different question.
| Approach | Useful for | Limitation |
|---|---|---|
| Endpoint or interval summaries | Comparing AUC, peak, trough, or response at a prespecified time | Can hide timing errors and trajectory shape |
| Time-course overlay at matched times | Seeing onset, peak, duration, and recovery discrepancies | Interpretation depends on sampling density and measurement uncertainty |
| Separate exposure, engagement, and PD comparisons | Locating which layer first shows a mismatch | Layer-by-layer agreement does not establish a unique causal explanation |
| One combined score across layers | Providing a compact summary when scales and weights are justified | Can conceal a serious error in one layer or depend strongly on arbitrary weighting |
Predictions may also vary because parameters, inputs, or observations are uncertain. A single simulated trajectory compared with noisy measurements can give a misleading impression of precision. Where uncertainty information is available, show it alongside predictions and measurements and interpret discrepancies in that context. Do not claim success merely because a trajectory passes through one observed point, or failure solely because of a small difference that is within known measurement variability.
No finite set of dose levels and schedules validates a model everywhere. Results support only the conditions actually tested and the nearby conditions that the evidence reasonably informs. Extrapolation beyond the tested range, to a new population or biological setting, or to a substantially different regimen requires additional justification and evidence.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A model predicts an AUC of 54 mg·h/L, while the observed AUC is 60 mg·h/L. Calculate the signed relative discrepancy and interpret its direction.
- Two schedules have the same total dose over 28 days, but one produces a higher observed mean engagement. Does this alone show that either regimen's PD prediction is wrong? Explain.
- Exposure and engagement trajectories agree reasonably well with observations, but predicted PD suppression peaks 12 hours earlier than observed. Which layer should be investigated first, and why?
- Why is an overall average error across concentration, engagement, and PD percentages potentially misleading?
Solutions. For question 1, \(D_{\mathrm{AUC}}=(54-60)/60=-0.100\), so predicted AUC is 10.0% below observed AUC. For question 2, no. Different schedules can produce different engagement, and this observation alone does not compare predicted with observed PD; the PD trajectories must be checked directly. For question 3, investigate the downstream response dynamics and PD measurement first, because the upstream exposure and engagement comparisons are reasonably consistent while the timing discrepancy appears in PD. This is a focused diagnostic, not proof of a unique cause. For question 4, the quantities have different meanings and may have different scales, units, and uncertainty. Combining them without justified scaling and weighting can let agreement in one layer conceal a consequential mismatch in another.