Why Below-Quantification Results Need Special Treatment
External validation asks how well a locked model predicts data from a new population or setting. When an assay reports a concentration as below the lower limit of quantification (LLOQ), it does not report the concentration itself. It tells us that the latent concentration is below a known threshold. Treating that record as a measured zero, as the LLOQ, or as a value such as half the LLOQ changes the data rather than evaluating the prediction against what was observed.
The distinction matters most in the low-concentration part of a profile, where predictions may be used to assess terminal decline, washout, or the probability that a sample will be quantifiable. A sound validation method retains the interval information in each below-quantification record and evaluates whether the model assigned appropriate probability to that event.
Prediction Distributions and Censored Likelihoods
For each external record, the locked model should provide a predictive distribution for the concentration under the relevant dose, covariates, time, and sampling design. This is a distribution for the future observation, including the variability appropriate to the prediction target; it is not just a typical concentration curve. The earlier material on population and conditional prediction distinguishes prediction targets. Here the key requirement is that the chosen target match the external observation being assessed, and that no parameters be re-estimated using the validation outcomes.
Suppose the predictive cumulative distribution function for a concentration is \(F_Y(y)\), with density \(f_Y(y)\). For a quantified value \(y\), its likelihood contribution is the predictive density at that value. For a below-quantification record, the contribution is the probability of the interval known to contain the value:
This follows directly from censoring: all possible concentrations from zero up to \(L\) are compatible with the reported result. Their probabilities add, so the contribution is the integral of the density over that interval:
For a quantified observation, the contribution is a density, not a point probability. A complete predictive log score therefore uses \(\log f_Y(y)\) for quantified values and \(\log F_Y(L)\) for censored values. Alternatively, when the model is formulated for log concentration \(W=\log Y\), use the density of \(W\) for quantified values and \(P(W<\log L)\) for below-quantification values. The likelihood must be expressed consistently on the chosen scale; comparing density scores requires the same response scale and units.
Worked Examples
Worked Example: Scoring One Below-Quantification Result
In a hypothetical external validation, the LLOQ is \(0.50\) mg/L. A locked model predicts that the probability a sample is below this LLOQ is \(0.291\), and therefore the probability it is quantifiable is \(1-0.291=0.709\). The external assay reports the sample as BLQ. Calculate its censored likelihood contribution and negative log score.
The observation is the event \(Y<0.50\) mg/L. Its likelihood contribution is the predicted probability of that event:
The negative log score for the observed BLQ event is
This score uses the probability of the event actually observed. For comparison, if the sample had instead been quantified, the probability of that status would be \(0.709\), and its status-only negative log score would be \(-\log(0.709)=0.3439\). A smaller negative log score means the model assigned greater probability to the observed event. Neither calculation claims that the true concentration equalled the LLOQ or any other substituted value.
Worked Example: Checking Predicted BLQ Frequency
In a hypothetical validation set, 20 samples share the same assay threshold and each has a model-predicted probability of \(0.30\) of being BLQ. Eight samples are reported BLQ. Compare the observed and predicted event frequencies and calculate the Brier score for the BLQ indicator.
The predicted number of BLQ results is \(20(0.30)=6\). The observed proportion is \(8/20=0.400\), compared with a predicted proportion of \(0.300\). The standardized difference from the predicted count, using the binomial standard deviation under the stated common probability, is
This descriptive difference is less than one such standard deviation. It is not, by itself, evidence that the model is calibrated or miscalibrated; this small hypothetical sample gives limited information.
For a binary event, the Brier score is the mean squared difference between the event indicator and its predicted probability. For each of the eight BLQ results the squared error is \((1-0.30)^2=0.49\). For each of the twelve quantified results it is \((0-0.30)^2=0.09\). Thus
The Brier score evaluates prediction of quantification status only. It discards the exact values for quantified observations, so it should complement rather than replace a score that uses their concentration information.
Worked Example: Combining a Quantified Value and a BLQ Record
Consider two independent hypothetical external records scored on the log-concentration scale. For a quantified record, the observed concentration is \(1.20\) mg/L and the predictive distribution for \(W=\log Y\) is normal with mean \(\eta=0\) and standard deviation \(\sigma=0.40\). A second record is BLQ at \(L=0.50\) mg/L, with a predictive log-concentration distribution having median \(0.80\) mg/L and standard deviation \(0.50\). Calculate the log-likelihood contributions on the log scale and their sum.
For the quantified record, \(w=\log(1.20)=0.1823\), so \(z=(w-\eta)/\sigma=0.1823/0.40=0.4558\). The normal density of \(W\) gives
For the BLQ record, the mean of \(W\) is the log of the stated median, \(\eta=\log(0.80)\). The standardized threshold is
The standard normal cumulative probability is \(\Phi(-0.9400)=0.1736\), so the censored log-likelihood contribution is \(\log(0.1736)=-1.751\). The total log likelihood for these two independent records, using the log-concentration scale for the quantified density, is
The quantified value contributes a density on the log scale, while the BLQ record contributes an event probability. Both records are retained without inventing an exact value for the censored concentration.
Diagnostics, Alternatives, and Limitations
A useful first diagnostic is to compare observed BLQ frequencies with predicted probabilities. Group records into prespecified or scientifically meaningful ranges of predicted BLQ probability, then compare the observed proportion in each range with its average predicted probability. Also examine whether discrepancies vary with time after dose, predicted concentration, or relevant subgroups. The expected number of BLQ observations across records is the sum of their individual predicted probabilities; when probabilities differ, this is more informative than comparing the observed rate with a single overall probability.
A second diagnostic is the censored predictive log score across the complete validation set. For independent records, add the quantified log-density contributions and censored log-probability contributions, then report the total or the average per record. A more negative log score indicates that the observed data received less predictive support. Scores can be unstable when a model assigns a probability near zero to an observed event, so report the number of records and inspect individual contributions rather than relying only on an average. Model comparisons require the same validation records, response scale, and observation definitions.
The following alternatives are common, but they answer different questions or discard information:
| Approach | What it does | Main problem for validation |
|---|---|---|
| Replace BLQ by zero, LLOQ, or a fraction of LLOQ | Turns an interval into an assumed exact value | Creates artificial residuals and may bias summaries or model comparisons |
| Delete BLQ records | Scores only quantified observations | Can make performance look better by omitting difficult low-concentration predictions |
| Score quantification status only | Compares predicted and observed BLQ/quantified events | Does not evaluate how close quantified concentrations are to their predictions |
| Use censored likelihood contributions | Uses the full density for quantified values and interval probability for BLQ values | Depends on a suitable predictive distribution and correctly recorded censoring thresholds |
Censored likelihood evaluation does not recover the unobserved exact concentrations. Its conclusions are limited by the assay’s threshold: two predictions that differ substantially below the LLOQ may both assign nearly all probability to the BLQ interval and thus be hard to distinguish from those records. It is also possible for apparent disagreement to arise from an incorrect threshold, an inappropriate predictive distribution, or a mismatch between the validation protocol and the model’s prediction target. These issues should be investigated rather than addressed by substituting values. Assay and protocol differences require separate assessment.
Common Mistake
Wrong reasoning: “The sample was BLQ, so its concentration was zero; I will calculate the residual as zero minus the prediction.” This treats an unknown concentration in an interval as an exact measurement and can make residual-based summaries depend heavily on an arbitrary convention. Correction: use the model’s predicted probability that the concentration lies below the recorded LLOQ. If a residual diagnostic is used, its definition must explicitly account for censoring; a conventional residual calculated from an invented value is not an observed prediction error.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- A hypothetical model assigns probability \(0.12\) to a concentration being below the sample’s LLOQ. What likelihood contribution applies if the result is BLQ?
- Why is replacing a BLQ value by the LLOQ not an exact observation?
- For 30 records with common predicted BLQ probability \(0.20\), what is the expected number of BLQ records?
- What information is lost if validation scores only whether each record is BLQ or quantified?
Solutions. (1) The censored likelihood contribution is \(P(Y<L)=0.12\); its negative log score is \(-\log(0.12)=2.120\). (2) The result states only that the latent concentration is below the LLOQ, not that it equals the LLOQ. (3) The expected count is \(30(0.20)=6\). (4) Status-only scoring ignores the observed concentration values among quantified records, so it cannot assess their predictive accuracy.