Why Separate Assay Noise From Model Discrepancy?
A PBPK prediction and an observed concentration will rarely match exactly. Part of the difference may arise because the assay produces variable readings of the same sample. Another part may reflect a prediction that misses the underlying concentration. These sources of error matter for different reasons: better assay precision can reduce measurement uncertainty, whereas a persistent prediction error may point to a model limitation, an input mismatch, or a problem in how the validation comparison was constructed.
Replicate measurements help quantify assay repeatability, but they do not by themselves prove that a PBPK model is adequate or identify the cause of a discrepancy. The key is to compare the observed prediction error with an estimate of assay noise on the same scale, while recognizing what the replicate design does and does not measure.
An Error Model for Replicate Measurements
For positive concentrations, a log scale is often useful because assay errors and prediction errors are commonly interpreted proportionally. Let \(M_i\) be the fixed PBPK prediction for sample \(i\), and let \(Y_{ij}\) be replicate assay result \(j\) for that sample. A simple working model is
Here, \(\delta_i\) is the log-scale discrepancy between the underlying sample concentration and the PBPK prediction, while \(\varepsilon_{ij}\) is replicate-level assay error. For replicate measurements to estimate assay repeatability in this model, assume the errors have mean zero, a common variance \(\sigma_a^2\), and are independent across replicates. The model also assumes that replicates share the same underlying sample concentration and that no assay trend or shared bias is being mistaken for random noise.
The log prediction residual for an individual replicate is \(\log(Y_{ij}/M_i)\). Across replicates, its variation estimates assay noise, while its average includes the sample’s discrepancy. The replicate mean on the log scale is \(\overline{\log Y_i}\); its estimated assay standard error is \(s_{a,i}/\sqrt{k_i}\), where \(s_{a,i}\) is the sample standard deviation of the \(k_i\) log replicate results. This standard error describes uncertainty in the replicate mean due to repeatability. It is not an estimate of uncertainty in the PBPK prediction.
Deriving the Replicate-Mean Comparison
The following result explains why replication makes the assay component of the comparison smaller. Suppose the log-scale assay errors are independent, have mean zero, and have variance \(\sigma_a^2\). Average the log measurements for sample \(i\):
The variance of the average assay error is the sum of the replicate variances divided by \(k_i^2\). Independence makes the covariance terms zero, so
Thus, the standard deviation of the replicate-average assay error is \(\sigma_a/\sqrt{k_i}\). More replicates improve the precision of the measured sample mean. They do not make a nonzero \(\delta_i\) disappear: averaging reduces random assay noise, not systematic model discrepancy.
In practice, \(\sigma_a\) is estimated from replicate results. A residual much larger than the estimated replicate variation is evidence that repeatability alone is an implausible explanation, conditional on the assumptions above. It is not, by itself, proof that the PBPK structure is wrong. Before drawing that conclusion, consider assay bias, sample identity, matrix effects, protocol deviations, and uncertainty in the prediction.
Worked Example: One Prediction Error Versus Replicate Precision
Hypothetical data. A PBPK model predicts a concentration of 12.0 mg/L for one sample. Four assay replicates of that sample are 10.0, 10.4, 9.8, and 10.2 mg/L. Assume the measurements are independent replicate determinations, the sample is homogeneous, and the assay has no relevant shared bias.
The replicate mean is \((10.0+10.4+9.8+10.2)/4=10.1\) mg/L. The deviations from the mean are \(-0.1, 0.3, -0.3,\) and \(0.1\) mg/L; their squared sum is \(0.20\) \((\mathrm{mg/L})^2\). Therefore,
The observed-to-predicted ratio is \(10.1/12.0=0.8417\), so the observed mean is 15.8% below the prediction. On the log scale, the residual is
For a lognormal measurement model, the estimated log-scale assay standard deviation corresponding to a CV of 0.02556 is approximately \(\sqrt{\log(1+0.02556^2)}=0.02555\). The residual magnitude is about \(0.1724/0.02555=6.75\) assay standard deviations. This comparison suggests that random repeatability error alone is not a plausible explanation for the discrepancy, under the stated assumptions. It does not account for prediction uncertainty, biological variation, or a shared assay bias.
Patterns Across Samples Matter
A single large residual can arise from several causes. Across multiple validation samples, the direction and structure of residuals add important evidence. Errors that vary around zero without a consistent pattern may be more compatible with random variation than errors that are consistently positive, consistently negative, or concentrated at particular times or conditions. Earlier material on observed-predicted plots and time-structured residuals provides useful ways to inspect those patterns; the assay comparison adds a scale for judging whether replicate noise could plausibly account for them.
Worked Example: Repeated Bias Across Validation Samples
Hypothetical data. Five independent samples have PBPK predictions of 10, 20, 5, 12, and 8 mg/L. For each sample, duplicate assay results have a mean 10% above the prediction. Assume a prior precision assessment supports a log-scale assay standard deviation of 0.020 per replicate, the replicate errors are independent, and there is no shared assay bias. The duplicate means are therefore 11, 22, 5.5, 13.2, and 8.8 mg/L.
For every sample, the log residual is \(\log(1.10)=0.09531\). The assay standard error of each duplicate log mean is \(0.020/\sqrt{2}=0.01414\). The residual is \(0.09531/0.01414=6.74\) such standard errors from zero for each sample. The mean log residual across all five is also 0.09531. If the only source of variation were independent assay noise, its standard error across these five duplicate means would be
The common positive direction is difficult to attribute to independent repeatability noise alone. It is consistent with a systematic discrepancy, but does not establish its cause. For example, a shared calibration bias could shift every measurement in the same direction and would not be revealed by repeatability estimates from duplicate readings. Nor does the calculation include uncertainty in the predictions or biological variation across samples.
What Replicates Can and Cannot Establish
A replicate design estimates only the variability represented by its repeated operations. Re-assaying aliquots from one prepared sample estimates a different component from independently collecting and processing multiple samples. If replicates include sample preparation, they may reflect both preparation and assay variability. The analysis should describe what was replicated rather than label every source of variation “assay error.”
Repeatability estimates can also be imprecise when there are few replicates. A small estimated standard deviation from two or three readings is not a guarantee that the assay is highly precise. If assay precision changes with concentration, one pooled standard deviation may be inappropriate; assess precision at relevant concentration levels or use a justified concentration-dependent error model. On a proportional scale, log residuals and CVs can be more interpretable than absolute differences. On an additive scale, compare differences and standard deviations in concentration units.
If prediction uncertainty and other independent uncertainty components are known on the same scale, they can be incorporated into a total uncertainty calculation. For example, under independence, a log-scale residual variance might include assay-mean variance, prediction variance, and a separately justified biological component. Do not add a component twice, and do not use assay replication as a substitute for estimating sources it does not measure.
| Comparison | Useful when | Main limitation |
|---|---|---|
| Replicate precision versus prediction residual | The question is whether random assay repeatability alone could plausibly explain a discrepancy. | Does not capture shared bias, prediction uncertainty, or unmeasured biological variation. |
| Observed-predicted and residual plots | The question concerns direction, magnitude, or structure of errors across samples or time. | Patterns diagnose a problem but do not identify which error source caused it. |
| Total predictive uncertainty comparison | Relevant uncertainty components have been estimated and can be combined on a compatible scale. | Can be misleading if components are omitted, dependent, or poorly quantified. |
Diagnostics and a Common Pitfall
Before interpreting the comparison, check that replicates truly share the same underlying sample, that the replicate protocol matches the validation assay, and that units and scales agree. Inspect replicate results for outliers, trends, or concentration-dependent spread. Compare multiple samples for consistent signed residuals and time- or condition-related patterns. If the assay has known calibration or matrix biases, repeatability alone is not an adequate measure of total measurement error.
Common mistake: “The prediction residual is within one replicate standard deviation, so the model is validated.” This reasoning confuses a small observed error with evidence of model adequacy. Assay variability can obscure a real discrepancy, especially with few replicates, and assay precision says nothing by itself about untested conditions or prediction uncertainty. The correction is to report the residual and the assay uncertainty in context, assess patterns across validation samples, and limit the conclusion to whether the observed error is distinguishable from the quantified assay component under stated assumptions.
Worked Example: A Wide Assay Interval Does Not Prove Agreement
Hypothetical data. Four replicate assay results for one sample are 9, 11, 10, and 10 mg/L. Assume independent, normally distributed additive assay errors with a stable variance and no shared bias. The mean is 10 mg/L. The sample standard deviation is \(\sqrt{2/3}=0.8165\) mg/L, so the standard error is \(0.8165/\sqrt{4}=0.4082\) mg/L.
Using a two-sided 95% \(t\) interval with \(df=3\) and \(t^*=3.182\), the interval for the mean assay result is
A PBPK prediction of 11 mg/L lies within this interval. This means these four replicates do not resolve the difference between the measured mean and that prediction using this interval. It does not establish that the prediction is correct: the interval is wide, the replicate count is small, and the interval does not include prediction uncertainty or other sources of variation.
Check Your Understanding
Answer each question, then compare with the solutions below.
- What source of variation is estimated by replicate assays of aliquots from one sample, and name two sources they do not estimate.
- Four independent log-scale replicates have a sample standard deviation of 0.03. What is the estimated standard error of their log-scale mean?
- Why can duplicate measurements all be consistently high even when within-sample assay repeatability is excellent?
- A prediction lies inside a confidence interval for the replicate mean. Does that establish model adequacy? Explain.
- Why should an analyst inspect residual patterns across samples instead of relying only on a single residual-to-assay-SD comparison?
Solutions. (1) They estimate within-sample repeatability under the replicated conditions. They do not estimate, for example, between-subject biological variability or PBPK prediction uncertainty. (2) The standard error is \(0.03/\sqrt{4}=0.015\) on the log scale. (3) A shared assay bias can affect all replicates in the same direction without increasing their within-sample spread. (4) No. The interval describes uncertainty in the replicate mean under its assumptions; it may be wide and does not establish prediction accuracy or account for all uncertainty sources. (5) A consistent direction or time-related pattern can indicate systematic discrepancy that a single residual comparison cannot reveal, while one residual alone may reflect chance variation or an unmeasured source of error.