QSP Model Verification and Validation

1. What Do Verification and Validation Mean in QSP?

Quantitative systems pharmacology (QSP) models combine mechanistic representations of physiology, disease biology, drug action, and treatment response. Because these models may contain hundreds of equations, parameters, feedback loops, and interacting biological processes, evaluating whether the model is trustworthy is a major part of QSP modeling.

Two related but distinct questions must be separated:

• Verification: Did we implement the mathematical model correctly?
• Validation: Does the model provide an adequate representation of the biological system for its intended use?

2. Start With the Context of Use

QSP model evaluation should begin with the question the model is intended to answer. This is often described as the context of use (COU).

The COU defines what the model is expected to do, under what conditions, and what decision or scientific question the model will support.

A model does not need to reproduce every biological detail to be useful. The appropriate level of evaluation depends on what the model is being used to do.

3. What Is Model Verification?

Model verification asks whether the implemented model correctly represents the mathematical specification that the modeler intended to implement.

In QSP, verification can involve several layers because a model may exist simultaneously as a conceptual description, a set of equations, computer code, and a numerical simulation.

Verification is therefore closely related to software quality assurance and numerical analysis. It is not primarily a test of whether the biological assumptions are correct.

4. Practical Verification Tests

Several simple tests can reveal implementation errors before a model is compared with experimental data.

4.1 Dimensional consistency

Every equation should be dimensionally coherent. For example, if a state variable represents concentration, then every term contributing to its time derivative must have concentration-per-time units.

If \(C\) has units of amount/volume and time is measured in hours, then \(dC/dt\) has units of amount/(volume·hour). Therefore \(k_{\mathrm{in}}\) must have those same units, while \(k_{\mathrm{out}}\) must have units of \(1/\text{hour}\).

4.2 Limiting-case tests

Set parameters to biologically or mathematically meaningful limiting values and determine whether the model behaves as expected.

• Set a drug dose to zero.
• Set an efficacy parameter to zero.
• Remove a feedback mechanism.
• Set a degradation rate to zero.
• Examine behavior when an input becomes very large or very small.

4.3 Conservation checks

For systems in which mass, cell number, or another quantity should be conserved, verify that the model preserves the relevant balance.

4.4 Known-solution tests

If a simplified version of the model has an analytical solution, compare the numerical implementation with that solution.

This provides a particularly strong test because the expected trajectory is known independently of the numerical solver.

4.5 Solver sensitivity

Repeat simulations using tighter numerical tolerances or a different appropriate solver. Important conclusions should not change simply because the numerical settings changed within a reasonable range.

5. What Is QSP Model Validation?

Validation concerns whether a model provides an adequate representation of the real biological system for a specified purpose.

In QSP, validation is more nuanced than asking whether predicted and observed values are statistically identical. Mechanistic models may integrate knowledge from many sources, and the purpose may involve predictions in situations for which direct observations are limited.

Validation can therefore involve evidence at multiple levels:

• Validation of individual biological submodels.
• Validation of system-level behavior.
• Comparison with experimental observations.
• Comparison with clinical observations.
• Prediction of data that were not used for model calibration.
• Assessment of whether predicted qualitative behaviors are biologically plausible.

Published QSP evaluation frameworks emphasize that model evaluation should combine multiple sources of evidence rather than rely on a single statistical test or acceptance criterion.

6. Calibration Is Not the Same as Validation

Calibration uses available data to estimate or adjust model parameters so that the model reproduces selected observations.

Validation evaluates how well the resulting model performs against evidence that was not simply used to determine those same parameters, or evaluates other biologically relevant behaviors for the intended use.

7. Validate the Parts and the Whole

QSP models are hierarchical. A large model may contain modules for disease progression, immune-cell dynamics, target biology, drug exposure, receptor binding, downstream signaling, and clinical outcomes.

Validation can therefore occur at several levels.

Agreement at one level does not automatically establish credibility at every other level. A validated biological submodel can still behave unexpectedly when embedded in a larger network.

8. Qualitative Validation

Not every important model prediction is a single numerical endpoint. QSP models often make predictions about direction, ordering, thresholds, saturation, feedback, and system behavior.

Examples include:

• Increasing drug exposure should increase target occupancy within a relevant concentration range.
• Increasing target inhibition should reduce a downstream pathway activity when the modeled biology predicts that relationship.
• A negative feedback loop should oppose uncontrolled accumulation of a state variable.
• A saturable process should approach a limiting behavior at sufficiently high input.
• Removing an active treatment should allow the system to move toward an appropriate untreated trajectory.

These checks can be valuable even when quantitative experimental data are sparse.

9. Quantitative Comparison With Data

When suitable data are available, model predictions can be compared quantitatively with observations.

The comparison should reflect the endpoint and measurement process. Useful approaches may include:

• Overlaying predicted trajectories and observed time-course data.
• Comparing predicted and observed summary measures.
• Examining prediction intervals.
• Assessing bias and systematic deviations.
• Comparing dose-response relationships.
• Evaluating clinically relevant thresholds.
• Comparing distributions across virtual populations and observed populations.

A simple prediction error for observation \(y_i\) and prediction \(\hat y_i\) is:

But the numerical magnitude of an error must be interpreted in context. An error of 2 units may be negligible for one endpoint and unacceptable for another.

For this reason, QSP evaluation should be tied to the context of use and the consequences of prediction error rather than relying automatically on a universal numerical threshold.

10. Sensitivity Analysis as Part of Model Evaluation

Sensitivity analysis examines how changes in model inputs or parameters affect model outputs.

For an output \(Y\) and parameter \(\theta\), a local sensitivity can be expressed conceptually as:

In large QSP models, global sensitivity analysis can be particularly useful because parameters interact nonlinearly and local derivatives may not capture the behavior across the relevant parameter space.

Sensitivity analysis can answer questions such as:

• Which parameters have the greatest influence on the prediction?
• Which biological assumptions drive the model outcome?
• Which measurements would be most valuable to collect?
• Which uncertain parameters need better characterization?
• Is a decision driven by a small number of influential assumptions?

Sensitivity analysis is therefore not merely a technical exercise. It helps connect model uncertainty to experimental priorities and decision risk.

11. Identifiability and Model Credibility

A parameter can be biologically meaningful but poorly identifiable from the available data.

Structural identifiability asks whether a parameter can theoretically be determined uniquely from ideal observations under the model structure.

Practical identifiability asks whether the available data are sufficiently informative to estimate the parameter with useful precision.

This distinction matters because a model may appear to fit data while several parameter combinations produce essentially equivalent predictions.

12. Quantifying Model Uncertainty

QSP models contain multiple sources of uncertainty. Treating all uncertainty as if it came from a single parameter-estimation step can give a misleading picture of predictive confidence.

One useful conceptual representation is:

The terms should not literally be added as a universal mathematical formula; rather, the expression emphasizes that multiple uncertainty sources can contribute to a prediction.

13. Why External Data Matter

The strongest test of predictive performance often occurs when the model is confronted with data that were not used to calibrate it.

External evaluation may involve:

• A different experiment.
• A different study.
• A different treatment regimen.
• A different dose level.
• A different disease state.
• A different patient population.
• A prospective prediction generated before the new data become available.

Prospective prediction can be especially informative because the model's prediction is established before the outcome is observed.

14. Worked Example: Verifying and Validating a Simple QSP Module

Consider a simplified QSP module describing the turnover of a biomarker \(B\). The biomarker is produced at a constant rate and eliminated proportionally to its concentration:

Suppose the baseline biomarker concentration is expected to be \(100\) units and the degradation rate is \(0.10\ \mathrm{h}^{-1}\).

Step 1: Derive the baseline production rate

At steady state, \(dB/dt=0\), so:

Step 2: Verification through the baseline test

Substituting the values into the implemented equation:

The model therefore preserves the intended baseline state. If the computer implementation produced a substantial nonzero derivative at these conditions, that would indicate a problem with the equation, parameterization, units, or code.

Step 3: Apply a hypothetical perturbation

Suppose a treatment reduces biomarker production by 50%. The new production rate becomes:

The new steady-state concentration is:

Step 4: Qualitative validation

If biological knowledge indicates that treatment should reduce this biomarker by decreasing its production, the model predicts the expected direction: the biomarker moves from approximately 100 toward 50 units.

Step 5: Quantitative validation

Suppose an independent experiment reports a steady-state biomarker concentration of approximately 55 units under the same treatment condition. The model prediction of 50 units is reasonably close in this illustrative example, but whether the difference is acceptable depends on the measurement uncertainty, biological variability, and context of use.

Step 6: Test a new time-course prediction

The solution following the treatment change at time zero is:

Therefore:

At \(t=10\) hours:

The model thus predicts both the eventual response and its time course.

15. Should a QSP Model Have a Single Pass/Fail Threshold?

Usually, there is no universal numerical threshold that determines whether a QSP model is "validated."

The appropriate evaluation criteria depend on the intended use, endpoint, data quality, biological variability, prediction horizon, and consequences of an incorrect prediction.

For example, an evaluation plan might specify:

• Which outputs are considered decision-relevant.
• Which experimental datasets will be used for evaluation.
• Which qualitative behaviors must be reproduced.
• Which prediction intervals should contain a specified proportion of observations.
• Which systematic biases would materially affect the decision.
• Which sensitivity or uncertainty analyses are required.
• What model limitations must be documented before use.

Published QSP evaluation frameworks emphasize defining the model's aims, scope, and risk before selecting the evaluation analyses.

16. Can a QSP Model Be Overfit?

Yes. Mechanistic interpretation does not make a model immune to overfitting.

Overfitting can occur when parameters or mechanisms are adjusted primarily to reproduce a limited dataset without sufficient biological justification.

Warning signs include:

• Many poorly identifiable parameters.
• Large numbers of ad hoc correction factors.
• Excellent calibration but poor external predictions.
• Strong dependence on one particular dataset.
• Mechanisms added solely to improve a numerical fit without independent support.
• Predictions that change dramatically under small parameter perturbations.

One reason QSP modelers emphasize mechanistic justification is that the purpose of the model is often to extrapolate to new interventions or biological scenarios. A model that merely interpolates existing observations may not provide the desired predictive value.

17. A Good Fit Is Not the Same as a Good Prediction

A QSP model can reproduce existing data without necessarily being reliable for an untested scenario.

Consider two models that both reproduce an observed treatment-response curve. One model may attribute the response to direct pathway inhibition, while another attributes it to an indirect feedback mechanism.

If both mechanisms fit the available data, the models may produce very different predictions when the treatment is changed.

18. A Practical QSP Verification and Validation Workflow

1. Define the context of use. Specify the population, intervention, endpoint, time horizon, and intended decision.
2. Document the model specification. Record equations, assumptions, parameters, units, initial conditions, and boundary conditions.
3. Verify the implementation. Check equations, units, code, numerical solvers, inputs, outputs, and simulation workflows.
4. Perform limiting-case tests. Test zero inputs, extreme values, steady states, conservation relationships, and simplified model forms.
5. Establish baseline behavior. Confirm that untreated or reference simulations reproduce known system behavior.
6. Perform sensitivity analysis. Identify parameters and assumptions that materially influence decision-relevant outputs.
7. Assess identifiability. Determine whether influential parameters can actually be estimated or constrained.
8. Calibrate the model. Use appropriate data and document parameter-estimation decisions.
9. Evaluate qualitative behavior. Confirm that the model reproduces established biological relationships and expected perturbation responses.
10. Perform quantitative evaluation. Compare predictions with appropriate experimental and clinical observations.
11. Evaluate independent data. Where possible, test predictions using data that were not used for calibration.
12. Quantify uncertainty. Propagate relevant parameter, structural, variability, and scenario uncertainty into decision-relevant predictions.
13. Document limitations. Clearly state where evidence is strong, where assumptions dominate, and where the model should not be extrapolated.
14. Reassess credibility when the COU changes. A model qualified for one purpose may require additional evaluation for a substantially different purpose.

19. What Should Be Documented?

Credibility depends not only on the analyses performed but also on whether the evidence can be understood and reproduced.

A QSP model evaluation package should clearly document:

• The scientific question and context of use.
• The model equations and biological assumptions.
• Sources for parameters and biological mechanisms.
• Units and parameter definitions.
• Initial conditions and boundary conditions.
• Software and solver versions.
• Verification tests and their results.
• Calibration datasets and estimation methods.
• Evaluation datasets and their relationship to calibration data.
• Sensitivity and identifiability analyses.
• Uncertainty analyses.
• Quantitative and qualitative validation results.
• Known limitations and excluded mechanisms.
• Model versioning and changes between model iterations.

Good documentation makes it possible for another modeler to understand not only what the model predicts, but why those predictions should or should not be trusted for a particular purpose.

20. Verification, Validation, and Model Credibility

In model-informed drug development, the ultimate objective is not to establish that a model is universally "true." The practical objective is to establish sufficient evidence that the model is credible for the decision or scientific question for which it will be used.

This distinction is particularly important for QSP because models often integrate heterogeneous evidence across biological scales. Some components may be strongly supported by experimental measurements, while others may rely more heavily on mechanistic assumptions or indirect evidence.

A useful conceptual hierarchy is:

The 2026 FDA draft guidance on QSP-based MABEL dose selection similarly describes verification as checking the model equations and coding implementation and emphasizes that validation should be driven by the context of use and model risk. Because this document is draft guidance, its recommendations are nonbinding and should not be treated as final regulatory requirements.

21. Common Verification and Validation Mistakes

• Calling a good fit "validation." A good calibration fit is evidence about calibration, not automatically about predictive validity.
• Skipping unit checks. A model can produce numerically plausible results while containing incompatible units.
• Ignoring solver behavior. Numerical artifacts can be mistaken for biological dynamics.
• Validating only the final endpoint. An apparently correct endpoint can conceal incorrect intermediate trajectories.
• Using the same data for calibration and validation. This weakens the independence of the validation evidence.
• Ignoring structural uncertainty. Multiple mechanistic structures may be consistent with the available data.
• Ignoring identifiability. Parameters may appear precise while being weakly constrained by the data.
• Using arbitrary acceptance thresholds. Criteria should be tied to the context of use and decision consequences.
• Validating modules but not the integrated system. Correct submodels do not guarantee correct emergent behavior.
• Failing to document assumptions. Hidden assumptions make model credibility difficult to evaluate.
• Assuming validation is permanent. New data, mechanisms, software changes, or a new context of use may require additional evaluation.

Where to Go Next

A natural progression is to study QSP model uncertainty quantification, followed by sensitivity analysis, structural and practical identifiability, virtual populations, model calibration, and prospective simulation.

The next tutorial can build directly on the concepts introduced here by examining how uncertainty in QSP parameters and model structure propagates into uncertainty in predictions and decision-relevant outcomes.

References

1. Braakman S, et al. Evaluation framework for systems models. CPT: Pharmacometrics & Systems Pharmacology. 2022;11:123–137. doi:10.1002/psp4.12755.
2. Friedrich CM. A model qualification method for mechanistic physiological QSP models to support model-informed drug development. CPT: Pharmacometrics & Systems Pharmacology. 2016;5:43–53. doi:10.1002/psp4.12056.
3. Marshall S, et al. Best Practices to Maximize the Use and Reuse of Quantitative and Systems Pharmacology Models: Recommendations From the United Kingdom Quantitative and Systems Pharmacology Network. CPT: Pharmacometrics & Systems Pharmacology. 2019;8:371–380.
4. McGinnity DF, et al. Assessing the performance of QSP models: biology as the driver for validation. Journal of Pharmacokinetics and Pharmacodynamics. 2023.
5. Sokolov V, Peskov K, Helmlinger G. A Framework for Quantitative Systems Pharmacology Model Execution. Handbook of Experimental Pharmacology. 2025;289:75–120. doi:10.1007/164_2024_738.
6. U.S. Food and Drug Administration. Quantitative Systems Pharmacology (QSP)-Based Dose Selection for Minimum Anticipated Biological Effect Level (MABEL) in First-in-Human (FIH) Trials. Draft Guidance. June 2026.