Why Shared Assumptions Matter
Two QSP models can represent different biological explanations for the same response. A comparison is informative only if differences in their fit or interpretation can reasonably be attributed to those explanations. If one model is given a more favorable starting state, a different assay calibration, or a selectively chosen subset of observations, then the comparison no longer isolates the biological difference.
Fairness does not mean forcing competing models to have identical biological structures. Those structures are the hypotheses being compared. It means holding fixed, or handling by the same defensible rule, the evidence and non-mechanistic assumptions that should not themselves distinguish the hypotheses. The earlier material on mechanistic hypotheses as explicit model structures provides the context: here the focus is on controlling comparison conditions so those structures are not confounded with setup choices.
The practical audit has four parts: the data being explained, the initial conditions for the model states, the mapping from latent model states to measured observations, and the handling of uncertainty and missingness. A choice can be shared by fixing it, or by estimating it separately from common evidence under the same rule. It need not always have the same numerical value if the biological setup genuinely differs.
Separate the Mechanism from the Comparison Conditions
Write each model conceptually as a biological state process followed by an observation process. For model \(m\), let \(x_m(t;\theta_m,x_{0,m})\) denote its latent state trajectory, determined by its mechanism, parameters \(\theta_m\), and initial state \(x_{0,m}\). Let \(g(\cdot;\phi_m)\) map that state to a measured quantity, and let the observed data be \(y\). The prediction is \(g(x_m(t;\theta_m,x_{0,m});\phi_m)\). Competing models may differ in their state equations; the data and observation rule generally should not differ merely because the mechanism does.
A useful comparison begins by specifying which elements are shared and why. For example, the same measured subjects, dose histories, sampling times, assay units, and residual-error definition normally apply to both models. Initial states should be based on the same baseline measurements or the same initialization rule. If an assumption truly must differ between models, state the scientific reason and examine whether the conclusion depends on that difference.
The reason is straightforward: observed disagreement can be caused by either the state process or the observation process. With a common observation mapping, a difference in predicted observations traces back to the differing state trajectories (and their model-specific parameters). If the mapping changes too, the source of the difference is ambiguous.
Here the same observation mapping parameter \(\phi\) and error term assumptions are used for each model, while the mechanistic structure is indexed by \(m\). This does not claim the models are correct, nor does it guarantee that the data can distinguish them. It makes the intended comparison interpretable.
Initial Conditions Must Be Evidence-Based
An initial condition is the state from which the model simulation begins. It may be directly measured, fixed by a protocol, derived from a measured baseline, or estimated. The key question is whether a model-specific initial value is supported by information that is independent of the outcome being compared. If each mechanism is allowed a different unconstrained starting value, one can compensate for a mechanistic difference simply by moving the starting point.
When baseline states are measured, use the same values and units in each model, allowing only transformations required by the model definitions. When they are not directly observed, estimate them from the same baseline data using comparable rules and account for the uncertainty. If the experimental groups truly begin in different states, that difference is not a nuisance to erase: represent it using the measured group-specific evidence in every relevant model.
Worked Example: A Starting Value Can Imitate a Mechanism
Suppose two hypothetical models describe the decline of a biomarker \(C(t)\) after a perturbation, with time in days and concentration in units per liter. Model A uses \(C_A(t)=C_A(0)e^{-0.15t}\), while Model B uses \(C_B(t)=C_B(0)e^{-0.25t}\). A baseline measurement gives \(C(0)=45.0\) units per liter for the same experimental condition. Assume exponential decline is an adequate description for this calculation and that the baseline measurement is used identically in both models.
With the shared initial condition \(C_A(0)=C_B(0)=45.0\), their day-2 predictions are:
The predictions differ because the stipulated decline rates differ, not because the models began at different baselines. Now suppose Model B is instead assigned \(C_B(0)=55.0\). Its day-2 prediction becomes \(55.0e^{-0.50}\approx33.36\) units per liter, almost the same as Model A's prediction. That apparent agreement is created by the altered starting value. Unless the 55.0 value is supported by baseline evidence, it cannot be treated as a fair mechanistic comparison.
For a separate initialization calculation, if a hypothetical trajectory is known to be 45.0 units per liter at day 2 and follows \(C(t)=C(0)e^{-0.20t}\), back-extrapolation gives \(C(0)=45.0e^{(0.20)(2)}=45.0e^{0.4}\approx67.13\) units per liter. Forward substitution checks the result: \(67.13e^{-0.4}\approx45.0\) units per liter. The value is not 67.04; the unrounded exponential factor is approximately 1.4918. Such a back-calculated state is justified only if the assumed trajectory and time origin are appropriate, and the same rule is applied to both models.
Keep the Observation Process Comparable
QSP models often predict latent quantities that are not measured directly. An assay may introduce a scale factor, offset, censoring threshold, or measurement error. These components can materially change the predicted data even when the underlying biology is unchanged. If one mechanism is evaluated using one observation mapping and another mechanism using a more favorable mapping, the result combines biological and measurement differences.
Use the same assay definition, units, calibration evidence, and error assumptions when the models concern the same measurements. If a model has a genuinely different mapping because it represents a distinct analyte or experimental method, make that difference explicit and support it with the relevant calibration data. Do not silently assign model-specific offsets or scales merely to improve agreement.
Worked Example: An Assay Offset Can Hide a State Difference
Consider one hypothetical assay with the shared measurement rule \(y=1.5x+2\), where \(x\) is the latent biomarker concentration in units per liter and \(y\) is the reported assay value in the assay's calibrated units. At a sampling time, Model A predicts \(x_A=12.0\) units per liter and Model B predicts \(x_B=10.0\) units per liter. Assume the assay is linear over this range and that the same offset and scale apply to both predictions.
The corresponding assay predictions are \(y_A=1.5(12.0)+2=20.0\) and \(y_B=1.5(10.0)+2=17.0\). If the hypothetical observed value is 20.0, Model A has zero residual and Model B has a residual of \(20.0-17.0=3.0\) assay units under this simple comparison.
If Model B is allowed a different offset of 5, its reported prediction becomes \(1.5(10.0)+5=20.0\). It now matches the observation, but the comparison no longer isolates the biological state prediction: the offset has changed too. A model-specific offset could be legitimate if independent assay-calibration evidence supports it, but that is a different scientific claim and must be treated as such. Under the stated common-assay assumption, the shared offset is 2, so the prediction remains 17.0.
Use the Same Data and Evaluation Rules
A model comparison also requires a common evidentiary basis. The models should ordinarily be evaluated against the same observations, with the same units, time alignment, subject or experimental-group definitions, and treatment of missing or censored values. A point absent from one model's evaluation is not evidence that the model explains it. Nor should repeated measurements be counted differently unless the data structure and error model justify that treatment.
For a simple illustration, suppose errors are independent, normally distributed with mean zero and a common known standard deviation of 1 assay unit. Under these assumptions, comparing the sum of squared residuals on the same observations is a transparent summary of discrepancy. It is not a complete model-selection procedure; it only illustrates why the data and scoring rule must match.
Worked Example: A Common Data Set Gives an Interpretable Fit Summary
Suppose three hypothetical observations at days 1, 2, and 3 are 8, 12, and 16 assay units. Model A predicts 9, 11, and 15; Model B predicts 8, 13, and 15. Assume the same three observations, same units, and independent normal errors with mean zero and standard deviation 1 assay unit for both models.
For Model A, the residuals \(y_i-\hat y_i\) are \(-1, 1, 1\), so the sum of squared residuals is \((-1)^2+1^2+1^2=3\) assay-units squared. For Model B, the residuals are \(0,-1,1\), giving \(0^2+(-1)^2+1^2=2\) assay-units squared. Under this stated error model, Model B has the smaller squared discrepancy on these three shared observations.
This result alone does not establish that Model B's biological mechanism is true. It summarizes fit to a very small hypothetical data set under specified assumptions. If Model A had been scored on only the first two points while Model B was scored on all three, the two sums would not be directly comparable: they would refer to different evidence. Likewise, changing the weights or error variances for one model would change the comparison rule.
A Practical Audit and Common Failure Modes
Before interpreting a difference between competing models, document which components are common, which are allowed to differ, and the evidence supporting each difference. The following sequence helps distinguish a biological comparison from a comparison of setup choices.
Confirm that both models address the same measurements, subjects or groups, time points, doses, units, and handling of missing values.
Trace every initial state to a measured baseline, protocol-defined value, or shared estimation rule. Flag any model-specific value that is not independently supported.
Check assay transformations, offsets, scales, censoring, and residual assumptions. Identify any differences and the calibration evidence that justifies them.
State what differs biologically and what is held common. Describe a fit advantage as conditional on those assumptions, not as proof that a mechanism is correct.
Warning signs include model-specific baselines chosen after viewing the endpoint, different observation units, unexplained offsets, inconsistent exclusions, and a fit summary computed over different observations. Residual patterns can also reveal that the assumed observation process is inadequate; the appropriate response is to investigate the measurement or error assumptions, not to declare the biological mechanism established.
The main alternative is not to make every component identical, but to justify differences explicitly. For example, distinct assays may require different calibration parameters, and treatment groups may have different measured baselines. In such cases, use the appropriate group- or assay-specific information in each model while retaining a common rule for using that information. Sensitivity analyses can show whether a conclusion changes under plausible shared assumptions. However, no comparison can remove uncertainty caused by sparse measurements, unobserved states, or weak information about the observation process.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Two models use the same equations except that one starts from a biomarker baseline of 30 units and the other from 36 units. What must be checked before interpreting their endpoint predictions?
- A model predicts a latent value of 4, and an assay reports \(y=2x+1\). What is the assay prediction? Why would assigning a different offset to a competing model require justification?
- Why is a residual sum of squares based on four observations not directly comparable with one based on six different observations?
- Give one example of a scientifically justified difference in an assumption and explain how to use it without confounding the mechanism comparison.
Solutions. (1) Check whether the baselines are measured, protocol-defined, or estimated by the same rule. If the same condition has one measured baseline, use that evidence consistently; otherwise a starting-state difference may imitate a mechanistic effect. (2) The prediction is \(2(4)+1=9\) assay units. A different offset changes the observation mapping as well as the biological model, so it needs independent calibration support. (3) The sums summarize discrepancy over different data and have different numbers and identities of terms; a difference may reflect the evidence set rather than model behavior. (4) Distinct assays may have distinct calibrated offsets. Use each assay's calibration evidence in the relevant model and state clearly that the observation mapping differs for this experimentally supported reason.