Distinguishing Measurement Conflict from Model Misfit

Why the Cause of Disagreement Matters

When a QSP model is calibrated to multiple data sources, disagreement between observations and predictions can have at least two different explanations. The sources may measure the same biological quantity differently, or the model may fail to represent a biological pattern that appears across sources. These explanations imply different next steps: a measurement problem may call for calibration or assay review, whereas shared structured residuals may motivate reconsidering the mechanistic representation.

Residuals alone do not prove which explanation is correct. A source-specific pattern is evidence about measurement only when the study design and controls support that interpretation. Conversely, a pattern repeated across assays is consistent with shared model misfit, but can also arise from a measurement bias common to all assays. The objective is therefore to compare explicit predictions of the two explanations against controls, replicates, and matched experimental conditions.

Assumptions and a Diagnostic Decomposition

This distinction is most informative when sources have measurements at overlapping biological conditions, controls that identify how instruments respond, and replicates that characterize within-condition variability. The model prediction must correspond to the quantity actually measured, or a stated measurement mapping must connect the prediction to the observed scale. For example, a model may predict free concentration while an assay reports total concentration; treating those as interchangeable would manufacture apparent residual structure.

A simple diagnostic representation separates a source pattern from a condition pattern:

Here \(a_s\) represents a source-associated residual component, \(h_c\) a condition-associated component common across sources, and \(\varepsilon_{sc}\) remaining variation. This is a diagnostic decomposition, not automatically a validated biological model. A constraint such as \(\sum_s a_s=0\) is needed to define the reference level of the source components. If every source measures a different condition, \(a_s\) and \(h_c\) cannot be separated from the observations alone: source and condition are confounded.

A useful justification comes from comparing sources at the same condition. Suppose the model prediction is shared, and observations on the comparable scale satisfy \(y_{sc}=f_c(\theta)+a_s+\varepsilon_{sc}\). Subtracting observations from two sources cancels the common model prediction:

Thus a consistent contrast across matched conditions can point to a source difference without requiring the model prediction to be correct. But this cancellation is valid only if the sources measure the same underlying quantity under comparable conditions. With a source-specific scale transformation, the raw difference need not be constant; controls are then needed to estimate or assess the transformation.

Residual size should be judged relative to uncertainty, not just by visual distance. If an observation has standard error \(SE\), a simple standardized residual is \(r/SE\). For correlated observations, the residual vector’s covariance matters: treating repeated measurements from one sample as independent can make a pattern appear more precise than it is. Replicates estimate measurement variability under their experimental conditions; they do not by themselves establish that the mechanistic model is correct.

Use Controls to Test Measurement Explanations

A reference control has a known or independently characterized value, so it can reveal whether sources differ even when the biological model is not at issue. For a control value \(x\), a simple affine measurement model is \(y_s=a_s+b_s x+\epsilon\), where \(a_s\) is an offset, \(b_s\) a scale factor, and \(\epsilon\) measurement error. One control level can inform an offset only if the scale is already known; multiple levels are needed to assess a linear scale difference. A visibly curved control response would call the affine assumption into question.

Controls should span the range relevant to the biological samples and, where appropriate, use a matrix comparable to those samples. A control measured only at a low concentration may not reveal a source difference at high concentrations. Likewise, a reference material in buffer may not expose a matrix effect in plasma. Control-based adjustment also carries uncertainty: it does not turn noisy measurements into exact values.

Look for Replicated Residual Structure

After considering the measurement process, inspect residuals by condition, time, dose, and source. A source pattern that is approximately stable across matched conditions, and is also visible in controls, is more consistent with measurement conflict. A residual pattern that changes with biological condition in a similar way across sources is more consistent with shared model misfit. These are evidential patterns, not rules that uniquely identify cause.

Replicate patterns help distinguish a stable shift from ordinary measurement scatter. If replicate values within a condition vary widely, a difference between source means may be poorly determined. If replicates cluster tightly but their mean repeatedly departs from the model prediction, random measurement noise alone is less persuasive as an explanation. However, shared sample handling, common reagents, or common preprocessing can induce correlated errors that replicate counts do not remove.

When a Measurement Scale Difference Is Not an Offset

Source disagreement need not be constant across the measurement range. An offset-only correction can leave residual structure if sources differ in scale. Controls at multiple known levels can reveal this distinction. The following example uses an exact hypothetical calibration pattern to make the arithmetic transparent; real control data would have uncertainty that should be carried into subsequent interpretation.

Practical Diagnostic Sequence

Alternatives, Failure Modes, and Limits

A residual plot is a useful first check because it displays direction and structure, but it does not identify measurement behavior on its own. Control-based calibration directly investigates assay response, but depends on controls being relevant to the biological samples and on the chosen calibration form being adequate. Replicate summaries quantify repeatability, but cannot detect a bias shared by every replicate. Comparing source contrasts at matched conditions helps cancel a common model prediction, but fails if the sources measure different quantities or if their measurement mappings vary with condition.

A common mistake is to say, “Source B has larger residuals, so the biological model is wrong for source B.” This reasoning confuses the observed scale with the underlying mechanism. If controls show a source-specific offset or gain, the source pattern may arise before the model’s biological prediction is compared with the data. The correction is to inspect measurement behavior and matched conditions first, then assess whether residual structure remains across sources on a comparable scale.

The reverse mistake is also possible: applying a source correction until residuals look small and concluding that the model is adequate. A correction can conceal a shared biological discrepancy, especially when source and condition are confounded or when the correction is estimated from too few controls. Flexible corrections can also absorb real differences between experimental conditions. Preserve the distinction between what controls identify and what the mechanistic data support.

The strongest conclusions require crossed evidence: controls measured by multiple sources, biological conditions shared across sources, and replicates sufficient to assess variability. If these are absent, report the ambiguity and identify the additional measurement that would be most informative. A control spanning the disputed range can test a scale explanation; a second source measured at the discrepant biological condition can help separate source from condition effects.