Why Source Influence Is Not the Same as Residual Size
A joint calibration can fit several data sources at once, but the sources need not contribute equally to the fitted parameters or to predictions. A source may have conspicuous residuals yet have little effect on a parameter if the model is insensitive to that parameter in the source’s experimental setting. Conversely, a source with modest residuals may strongly constrain a parameter combination and therefore substantially affect the fit.
The distinction matters when interpreting a calibrated QSP model. The residual contribution describes disagreement between a source and the fitted model, scaled according to the calibration objective. Influence describes how the fit would respond to a change in that source’s weight or information. Neither quantity alone establishes that a source is reliable, representative, or biologically correct.
The factor of one-half is a convenient convention for derivatives; it does not change which parameter vector minimizes the objective. The weights should reflect the calibration choices, including measurement uncertainty and any source-level weighting used to avoid double counting. If residuals have covariance, \(W_s\) typically incorporates its inverse, so the contribution is not simply a sum of raw squared residuals.
Local Influence From a Change in Source Weight
Suppose \(\hat{\theta}\) is an interior local minimum of a smooth joint objective and its Hessian \(H=\nabla^2 F(\hat{\theta})\) is nonsingular. To ask how the fit responds when source \(s\) gains weight, consider \(F_\delta(\theta)=F(\theta)+\delta Q_s(\theta)\), where \(\delta\) is a small change from its current weight. Differentiating the stationarity condition at \(\delta=0\) gives a local sensitivity.
This derivative is a local diagnostic: it estimates the parameter change per unit increase in source weight near the fitted calibration. A source’s gradient \(g_s\) describes the direction in which its own contribution would move the parameters. At a joint optimum, the total gradient is zero, \(\sum_s g_s=0\), but an individual source gradient need not be zero. Other sources balance it.
For a small weight change, the fitted parameter shift is approximately \(\Delta\theta\approx-H^{-1}g_s\Delta\delta\). For a prediction \(h(\theta)\), its corresponding local change is approximately \(\Delta h\approx\nabla h(\hat{\theta})^T\Delta\theta\). Thus parameter sensitivity and prediction sensitivity are related but not identical: a parameter can shift appreciably while a particular prediction barely changes, or a small parameter shift can matter greatly for a sensitive prediction.
Worked Example: Residual Contributions Are Not Influence
Worked Example: Residual Contributions Are Not Influence
Consider a hypothetical one-parameter calibration with three sources reporting values \(2\), \(3\), and \(5\) for the same model quantity \(\theta\). Assign weights \(1\), \(1\), and \(8\), respectively, and use \(Q_s(\theta)=\frac{1}{2}w_s(\theta-y_s)^2\). Find the joint estimate and compare the residuals and objective contributions.
The joint objective is minimized at its weighted mean:
Define each residual as \(r_s=\hat{\theta}-y_s\). The residuals are \(2.5\), \(1.5\), and \(-0.5\), respectively. Their absolute values are \(2.5\), \(1.5\), and \(0.5\): source 3 has the smallest absolute residual at the joint fit.
The weighted objective contributions are:
Source 1 has the largest contribution under this particular objective, despite source 3’s large weight. But residual contributions alone still do not quantify how a change in source weight moves the estimate. The source gradients are \(g_s=w_s(\hat{\theta}-y_s)\), giving \(g_1=2.5\), \(g_2=1.5\), and \(g_3=-4\); they sum to zero, as required at the joint optimum. Increasing source 3’s weight infinitesimally moves \(\theta\) in the direction \(-g_3\), upward. The example shows why neither raw residual size nor weighted residual contribution should be treated as a complete influence measure.
Parameter Shifts and Prediction Sensitivity
A direct way to describe influence is to state a perturbation and report the resulting parameter or prediction shift. For example, reducing a source’s weight by a small fraction is a perturbation in the negative-weight direction. The local derivative above estimates the response. A finite change can also be evaluated by optimizing the perturbed objective; this provides a direct comparison but costs more and can expose nonlinear effects that a derivative cannot capture.
Worked Example: A Local Parameter Shift From Source 3
Continue the one-parameter example. The full objective is \(F(\theta)=\frac{1}{2}\sum_s w_s(\theta-y_s)^2\), so its Hessian is \(H=1+1+8=10\). For source 3, \(g_3=8(4.5-5)=-4\). Estimate the parameter shift if its weight is reduced by \(10\%\), so \(\Delta\delta=-0.10\).
The local formula gives:
The local approximation predicts a decrease of \(0.04\) units in \(\theta\). The sign is sensible: reducing the weight on the largest reported value pulls the weighted estimate downward.
For comparison, the exact estimate after changing source 3’s weight from \(8\) to \(7.2\) is:
The exact shift is \(4.4565-4.5=-0.04348\) units, close to the local estimate of \(-0.04\). The agreement is plausible because the weight change is modest and this weighted-mean model is smooth. It does not guarantee similar accuracy for a large perturbation or a strongly nonlinear QSP model.
For interpretation, report the perturbation alongside the shift: “A 10% reduction in this source’s weight is estimated to lower the fitted parameter by 0.04 units locally.” Avoid reporting a parameter shift without specifying what was changed. Also inspect predictions in relevant units. If \(h(\theta)\) is a predicted biomarker concentration in nanomoles per litre, report its estimated shift in nanomoles per litre rather than only describing a change in an abstract parameter.
Worked Example: Influence on a Model Prediction
Consider a hypothetical joint calibration with two scaled parameters \(\theta_1,\theta_2\). Three source contributions are:
The last contribution has twice the curvature of a unit-weight contribution. Setting the summed gradient to zero gives \(\hat{\theta}=(1,1.5)^T\). The total Hessian is \(H=4I\), where \(I\) is the \(2\) by \(2\) identity matrix. The gradient of source C at the joint fit is \(g_C=(-2,-1)^T\). If its contribution is reduced by a small fraction \(\varepsilon\), the perturbation is \(F-\varepsilon Q_C\); its local parameter response is \(H^{-1}g_C\varepsilon\):
Suppose the prediction of interest is \(h(\theta)=\theta_1+2\theta_2\), in scaled prediction units. Its gradient is \((1,2)^T\), so its local change is:
Thus a small reduction of source C’s contribution is predicted to lower this output by approximately \(1.0\varepsilon\) scaled prediction units. This is an output-specific influence statement; other outputs can respond differently. The parameter scales and prediction scale must be stated in an applied analysis so that numerical magnitudes can be interpreted.
Diagnostics, Alternatives, and Limitations
A useful influence assessment combines several views rather than ranking sources by a single number. Inspect source-specific residual patterns and objective contributions, then examine parameter shifts and changes in predictions under explicit weight perturbations. The earlier material on source-specific residual patterns and measurement conflict helps distinguish a large source contribution from a coherent conflict or a pattern suggesting model misfit; influence itself does not resolve that distinction.
The Hessian-based derivative is computationally efficient and identifies local directions, but relies on local curvature. Finite weight perturbations or refitting provide a more direct measure of actual response at the chosen perturbation size, but are more computationally demanding and can depend on optimization settings. Residual summaries are simpler and useful for describing fit, but do not indicate how strongly a source constrains the fitted solution. These approaches answer related, not interchangeable, questions.
- Check scaling and conditioning. Parameters with very different units or scales can make raw shift magnitudes misleading. A nearly singular Hessian can make influence estimates highly sensitive to small numerical or data changes.
- Check local accuracy. Compare the derivative-based estimate with a finite perturbation when the shift matters. A large discrepancy can indicate nonlinearity, changing curvature, or movement to a different optimum.
- Check nuisance parameters. If a source informs source-specific assay or batch parameters as well as shared biological parameters, inspect the influence on both. A shift in a nuisance parameter need not imply the same shift in a biological prediction.
- Check the prediction target. Parameter influence depends on which parameter is reported, while scientific consequences depend on the outputs and conditions relevant to the decision.
These diagnostics have limits. A high-influence source is not necessarily erroneous, and a low-influence source is not necessarily unimportant for a biological mechanism or a particular prediction. Influence can depend on the chosen weights, parameterization, local optimum, and prediction target. It is a property of the fitted model and the specified perturbation, not a causal claim about the source or a test of whether the source is true.
Check Your Understanding
Answer each question, then compare with the concise solutions.
- At a joint optimum, must each source-specific gradient \(g_s\) equal zero? Explain.
- A source has a large weighted residual contribution but a small local parameter shift. Give one reason this can occur.
- If the weight of a source is reduced, what sign should be used for the weight perturbation in \(F_\delta=F+\delta Q_s\)?
- Why can a large parameter shift have little effect on one prediction?
- Name two conditions that can make a Hessian-based local influence estimate unreliable.
Solutions. (1) No. The source gradients sum to zero at the joint optimum, but individual gradients can be nonzero and balanced by other sources. (2) The source may have a substantial residual contribution but weak sensitivity in directions that affect the fitted parameters, or the joint curvature may strongly resist movement. (3) Use \(\delta<0\), since the source contribution is being reduced. (4) The prediction may be insensitive to the shifted parameter combination, so \(\nabla h(\hat{\theta})^T\Delta\theta\) is small. (5) Examples include a nearly singular Hessian, strong nonlinearity over the perturbation, and movement to a different local optimum.