Why Calibration Order Matters
A QSP model may be calibrated against data from binding assays, biomarker time courses, and clinical endpoints. Different sources can inform different parameters, but they may also depend on the same parameters. The calibration strategy matters especially when estimates from one source are frozen before another source is considered: later data then cannot revise those shared estimates, and uncertainty from the earlier stage may be lost.
The choice is not simply between a “good” simultaneous fit and a “bad” staged fit. Staging can be efficient and scientifically sensible when parameter dependencies are well separated, or when uncertainty is explicitly passed between stages. Simultaneous fitting directly balances all included sources, but it still depends on defensible source weights, a suitable model, and adequate parameter identifiability.
A Common Objective for Multiple Sources
Let \(\theta\) denote the model parameters, and let source \(s\) provide observation vector \(y_s\). The model prediction for that source is \(f_s(\theta)\). For a standard illustrative case, suppose measurement errors are independent across sources, normally distributed with mean zero, and have known within-source covariance matrix \(\Sigma_s\). Then the negative log-likelihood, apart from constants that do not depend on \(\theta\), is the weighted residual objective
The covariance matrix determines how discrepancies are weighted. A source with less measurement uncertainty can contribute more strongly, and correlated observations within a source are not treated as independent. If sources share correlated errors, the covariance structure must include those cross-source correlations rather than assuming independence. In practice, calibration may use another justified likelihood or loss; the key comparison is whether the staged and simultaneous procedures optimize compatible objectives and carry forward the same information.
A staged procedure often optimizes only part of this objective at a time. For example, it may first fit \(\theta_A\) to source A, fix that estimate, and then fit \(\theta_B\) to source B. This is not generally equivalent to minimizing \(J(\theta_A,\theta_B)\) jointly. By contrast, if stages repeatedly re-optimize parameter blocks against the full objective until convergence, that is an iterative joint optimization strategy, not a one-pass procedure that permanently freezes early estimates.
Worked Example: A Shared Parameter and Unequal Source Precision
Suppose two hypothetical assays both inform a single shared potency parameter \(\theta\), measured in nM. Source A reports \(y_A=10\) nM with known standard deviation \(2\) nM, and source B reports \(y_B=14\) nM with known standard deviation \(1\) nM. Assume each observation has prediction \(f_s(\theta)=\theta\), and the errors are independent and normally distributed.
The joint objective is proportional to the sum of squared residuals divided by their variances. Differentiating it with respect to \(\theta\), or solving for the weighted mean, gives
The standard error under this known-variance model is
If source A is fitted first and its estimate, \(10\) nM, is frozen, the resulting value cannot accommodate source B’s more precise observation. At \(\theta=10\) nM, source B’s residual is \(14-10=4\) nM, or four of its stated standard deviations. If source B is fitted afterward by replacing the estimate, the final value becomes \(14\) nM and source A is no longer balanced. The simultaneous estimate \(13.2\) nM is a precision-weighted compromise. These calculations rely on the stated prediction model, independent Gaussian errors, and known standard deviations; a shared parameter must genuinely represent the same quantity on the same scale in both sources for this pooling to be meaningful.
How Uncertainty Passes Between Stages
A staged estimate is not just a point value. If a downstream prediction depends on an earlier estimated parameter, its uncertainty depends on uncertainty in that parameter as well as on uncertainty in the later-stage data. Freezing an intermediate estimate and treating it as exact omits the first component.
Worked Example: Propagating Uncertainty to a Downstream Parameter
Consider two hypothetical measurements, in common biomarker-contribution units. Source A measures an upstream contribution \(\theta\): \(y_A=2.0\) with known standard error \(0.3\). Source B measures a total contribution predicted as \(\theta+\phi\): \(y_B=5.0\) with known standard error \(0.2\). Assume independent, normally distributed measurement errors and this stated linear prediction model.
The point estimates are \(\hat{\theta}=2.0\) and \(\hat{\phi}=5.0-2.0=3.0\). A plug-in staged calculation that treats \(\hat{\theta}\) as exact assigns \(\phi\) a standard error of only \(0.2\). But \(\hat{\phi}=y_B-y_A\), so independence gives
The joint covariance matrix for \((\hat{\theta},\hat{\phi})\) follows from \(\hat{\theta}=y_A\) and \(\hat{\phi}=y_B-y_A\):
The negative covariance arises because a larger estimate of \(\theta\), with the total \(y_B\) held fixed, implies a smaller estimate of \(\phi\). A sound staged analysis could retain the uncertainty in \(\theta\) and propagate it into \(\phi\); the point estimate need not change. The \(0.2\) standard error is appropriate only conditional on treating \(\theta\) as known, not as an unconditional uncertainty measure for \(\phi\).
Why Joint Uncertainty Can Be Larger
The effect of freezing parameters can be summarized through a local quadratic approximation to the objective near its minimum. Partition parameters into an earlier-stage subset \(\theta_A\) and a later-stage subset \(\theta_B\). Write the approximate curvature, or information matrix, in blocks as \(H\). If it is positive definite, the local joint covariance is approximately \(H^{-1}\).
The off-diagonal block \(B\) represents coupling: changes in one parameter subset can be compensated, at least partly, by changes in the other. When \(\theta_A\) is treated as fixed, the conditional covariance approximation for \(\theta_B\) is \(C^{-1}\). When uncertainty in \(\theta_A\) is included, the \(\theta_B\) block of the joint covariance is instead
This result follows by block inversion: solving \(H\binom{u}{v}=\binom{0}{I}\) gives \(Au+Bv=0\), hence \(u=-A^{-1}Bv\). Substituting into the second block equation yields \((C-B^\mathsf{T}A^{-1}B)v=I\). Thus the lower-right block of \(H^{-1}\) is the displayed expression. Since \(B^\mathsf{T}A^{-1}B\) is positive semidefinite, this joint covariance is at least as large as \(C^{-1}\) in the positive-semidefinite ordering, when the required inverses exist. The difference is uncertainty associated with estimating the coupled earlier parameters. This is a local approximation; it relies on adequate identifiability, a well-behaved optimum, and a sufficiently accurate quadratic approximation.
Worked Example: A Shared Parameter Can Make a Stage Order Fail
Suppose two hypothetical sources each have an observation with known standard deviation \(1\). Source A measures a pathway output predicted as \(\theta+\phi\) and reports \(10\) units. Source B measures an output predicted as \(\theta\) and reports \(4\) units. Assume independent normal errors and that both parameters are measured in units consistent with these predictions.
Simultaneous fitting minimizes
The two predictions can match both observations exactly: \(4-\theta=0\) gives \(\hat{\theta}=4\), and \(10-\theta-\phi=0\) then gives \(\hat{\phi}=6\). The residual for each source is zero.
Now consider a poor one-pass stage: fit \(\theta\) to source A while fixing \(\phi=0\), then freeze \(\theta\). This gives \(\hat{\theta}=10\). Source B’s residual is then \(4-10=-6\) units, six stated standard deviations. Since \(\theta\) has been frozen, the later stage cannot correct that shared-parameter error. A dependency-aware staged order would instead fit \(\theta\) from source B first, then fit \(\phi\) from source A conditional on \(\theta=4\); here it recovers the same point estimates as the joint fit. The example shows that staged calibration can work when the sequence respects parameter dependencies, but an arbitrary order and premature freezing can create severe misfit.
For this linear model with known unit variances, the design matrix has rows \((1,1)\) and \((1,0)\). The joint covariance is
Thus the approximate standard errors are \(1\) unit for \(\hat{\theta}\) and \(\sqrt{2}=1.414\) units for \(\hat{\phi}\), with covariance \(-1\) square units. The exact match of two observations does not imply that the parameters are known without uncertainty.
Choosing and Checking a Calibration Strategy
Simultaneous calibration is a natural default when sources share parameters or when the scientific question concerns their combined constraints. It ensures that a parameter informed by several sources is optimized against all those sources at once. Staging can still be appropriate when a parameter is genuinely specific to an early data source, when later steps depend on an identifiable earlier estimate, or when computation requires staged initialization. In those cases, be explicit about whether earlier parameters are fixed, updated, or represented with uncertainty in later steps.
A useful alternative to irreversible staging is blockwise optimization: update one parameter subset at a time while evaluating the full multi-source objective, and repeat until the objective and parameter estimates stabilize. Under suitable regularity conditions this can reach a joint optimum, although nonlinearity and multiple local optima mean convergence should be checked rather than assumed. Another option is to pass a distribution or covariance for earlier estimates into downstream calculations instead of passing only a point estimate.
Diagnostics should match the chosen strategy. Compare source-specific residuals and standardized residuals after the final calibration, inspect whether shared parameters move substantially when another source is added, and check whether predictions for a source excluded from a stage remain compatible with that source’s observations. The earlier material on leave-one-source-out calibration checks provides one way to assess sensitivity to source inclusion. A low total objective alone can conceal a poor fit to one source, particularly when sources have different scales or uncertainties.
- Order sensitivity: large changes in estimates when the stage order changes suggest coupling or an unstable stage-specific estimate.
- Residual conflict: systematic residuals in one source after freezing parameters indicate that the stage sequence may prevent a shared parameter from reconciling the data.
- Uncertainty collapse: implausibly narrow downstream uncertainty can occur when an estimated upstream value is treated as exact.
- Weak identification: large parameter changes with little objective change indicate that sources may not distinguish parameter combinations well. Simultaneous fitting does not create information that the data do not contain.
Neither strategy repairs a misspecified biological model, inappropriate measurement-error assumptions, or incompatible data scales. Simultaneous fitting can also allow a large or precise source to dominate if the objective’s weighting is not justified. Staged fitting can be more transparent about source-specific roles, but its apparent simplicity should not be mistaken for a complete treatment of uncertainty. The calibration record should state the objective, source weights or covariance assumptions, which parameters were fixed or shared, and whether uncertainty was propagated.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- Two independent sources measure the same parameter: \(y_1=6\) with standard deviation \(1\), and \(y_2=10\) with standard deviation \(2\). What is the simultaneous weighted estimate?
- A later endpoint prediction is \(\theta+\phi\). If \(\theta\) was estimated from earlier data, why can fixing it at its estimate understate uncertainty in \(\phi\)?
- In the block information matrix, what does a nonzero off-diagonal block \(B\) indicate?
- When can iterative blockwise fitting be equivalent in aim to simultaneous calibration, and what should be checked?
Solutions. (1) The inverse-variance weighted estimate is \((6/1^2+10/2^2)/(1/1^2+1/2^2)=(6+2.5)/1.25=6.8\) units. (2) The endpoint depends on the uncertain \(\theta\) as well as on the later data; treating \(\theta\) as exact omits its variance and any covariance with \(\phi\). (3) It indicates local coupling between parameter subsets: changes in one subset can be associated with changes in the other while fitting the observations. (4) It can pursue a joint optimum when each update evaluates the full objective and parameter blocks are repeatedly updated. Check that the objective and estimates stabilize and inspect for local-optimum or order sensitivity; convergence should not simply be presumed.