Why the Assay Threshold Must Follow the Sample
A BQL record does not mean that the concentration is zero, nor does it identify one exact concentration. It means that the sample’s result was below the lower limit of quantification (LLOQ) that applied to that measurement. If an assay is changed, improved, or replaced during a study, that threshold may differ across samples. An NLME analysis must therefore connect each record to the threshold used for that sample, rather than automatically assigning one LLOQ to the whole study.
The distinction matters even when the pharmacokinetic model is unchanged. The likelihood of a BQL observation depends on how far the model predicts the concentration to be below or above the applicable threshold. Changing the threshold changes that probability. Treating a new, lower threshold as if it had always applied can make BQL records appear less likely; applying an old, higher threshold to later records can make them appear more likely.
Observation Model With a Sample-Specific Threshold
To make the likelihood contribution precise, begin with an additive normal residual-error model. Conditional on the individual’s random effects and the model prediction, suppose
where \(\eta_i\) denotes the random effects for individual \(i\), and \(\sigma\) is the residual-error standard deviation in concentration units. This calculation assumes that the residual distribution is correctly specified, \(C_{ij}\) is the prediction at the sample time under the model, and the applicable threshold \(L_{ij}\) is known. It also assumes that the BQL status is determined by comparing the assay result with that threshold. Conditional on the random effects, observations are treated as independent in the usual likelihood construction.
For a BQL record, the exact value of \(Y_{ij}\) is unobserved. The information available is the event \(Y_{ij}<L_{ij}\). Its conditional likelihood contribution is the probability of that event:
where \(\Phi\) is the standard normal cumulative distribution function. A quantified observation at value \(y_{ij}\) contributes the normal density evaluated at that value. For a complete individual record, the conditional contributions are multiplied; the population likelihood then integrates over the distribution of the random effects. Thus, the LLOQ enters separately for every censored record, even when several records belong to the same individual.
The key step follows directly from standardization. Under the stated model, \((Y_{ij}-C_{ij})/\sigma\) has a standard normal distribution. Therefore,
This justification also shows why a study-wide threshold is not a harmless simplification: substituting the wrong \(L_{ij}\) changes the standardized cutoff and hence the probability assigned to the observed censoring event.
Worked Example: Assigning the Threshold After an Assay Change
Suppose a hypothetical study used assay A through study day 14, with an LLOQ of 0.50 mg/L. Starting on day 15, assay B was used, with an LLOQ of 0.20 mg/L. A participant has samples collected on days 12, 16, and 20. The records are BQL, 0.31 mg/L, and BQL, respectively. Assume the laboratory record confirms which assay produced each result.
The day-12 sample was measured with assay A, so its BQL event is \(Y_{i1}<0.50\) mg/L. Both later samples were measured with assay B: the day-16 quantified value is 0.31 mg/L, and the day-20 BQL event is \(Y_{i3}<0.20\) mg/L. The applicable thresholds are therefore 0.50, 0.20, and 0.20 mg/L, in sample order.
The quantified value is above assay B’s LLOQ because \(0.31>0.20\), so it is compatible with a quantified record. In the likelihood, the first and third records use their respective BQL probabilities; the second uses the observation density at 0.31 mg/L. It would be incorrect to use 0.50 mg/L for all three records just because that was the initial assay’s threshold, or to use 0.20 mg/L for the day-12 record just because it was the final threshold.
If the dates alone do not establish which assay was used—for example, if samples were re-run or analyzed in batches—the threshold assignment must follow the result actually reported for that sample. A calendar boundary is useful only when it correctly represents the laboratory’s assay and reporting history.
How Much Can a Wrong Threshold Change the Likelihood?
The direction of the change is transparent from the BQL probability. For fixed \(C_{ij}\) and \(\sigma\), increasing \(L_{ij}\) increases \((L_{ij}-C_{ij})/\sigma\), and therefore increases the probability of a BQL result. Conversely, using a lower threshold assigns a smaller BQL probability. This is a statement about the conditional likelihood for the specified prediction and error model; it does not, by itself, determine the direction or size of a fitted parameter change.
Worked Example: Two BQL Samples With Different Thresholds
Consider two hypothetical BQL samples from one individual. Conditional on the individual’s random effects, suppose both have predicted concentration \(C=1.2\) mg/L and additive residual standard deviation \(\sigma=1.0\) mg/L. The first sample’s applicable LLOQ is 0.8 mg/L and the second’s is 0.4 mg/L. Under conditional independence, their joint conditional contribution is the product of their BQL probabilities.
For the first sample, the standardized cutoff is \((0.8-1.2)/1.0=-0.4\), so its contribution is \(\Phi(-0.4)\approx0.3446\). For the second, the cutoff is \((0.4-1.2)/1.0=-0.8\). However, to match the illustrative comparison of unequal assay thresholds used here, take the second sample’s prediction to be 1.6 mg/L: its cutoff is then \((0.4-1.6)/1.0=-1.2\), and its contribution is \(\Phi(-1.2)\approx0.1151\). The correct joint conditional contribution is
Now suppose an analyst mistakenly uses 0.8 mg/L for both records. The second contribution is assigned \(\Phi(-0.8)\approx0.2119\), not \(\Phi(-0.4)\), because the second sample’s prediction remains 1.6 mg/L. The resulting joint contribution is approximately \(0.3446\times0.2119=0.0730\), overstating the correct contribution. If instead the analyst uses 0.4 mg/L for both, the first contribution becomes \(\Phi((0.4-1.2)/1.0)=\Phi(-0.8)\approx0.2119\); the joint contribution is approximately \(0.2119\times0.1151=0.0244\), understating it.
The calculation illustrates why an error’s impact depends on both the threshold and each sample’s model prediction. A numerical comparison that changes the threshold while silently changing neither prediction nor standardized cutoff can give the wrong result. Always calculate \( (L_{ij}-C_{ij})/\sigma \) for each record before multiplying contributions.
A useful interpretation is that the wrong threshold changes the information attributed to the observed BQL status. It does not turn the result into an observed concentration. The model is still using the event that the latent measurement fell below a limit; the analyst has simply specified the wrong event if the threshold was misassigned.
Worked Example: Combining a Quantified Result and a BQL Result
Suppose a hypothetical individual has two conditionally independent records. For the first, assay A has LLOQ 0.50 mg/L, the model predicts \(C=0.60\) mg/L, \(\sigma=0.20\) mg/L, and the quantified observation is \(y=0.72\) mg/L. For the second, assay B has LLOQ 0.20 mg/L, the prediction is \(C=0.35\) mg/L, and the record is BQL.
The first record contributes the normal density \(f(0.72)=\{0.20\sqrt{2\pi}\}^{-1}\exp[-(0.72-0.60)^2/(2(0.20)^2)]\). Here the exponent is \(-0.12^2/0.08=-0.18\), giving a density of approximately \(1.666\) per mg/L. The observed value is above its assay threshold because \(0.72>0.50\).
The second record contributes the probability of falling below its own threshold: \(\Phi((0.20-0.35)/0.20)=\Phi(-0.75)\approx0.2266\). The product is approximately \(1.666\times0.2266=0.3774\) per mg/L, conditional on the random effects. The density carries inverse-concentration units; the BQL probability is dimensionless. The result is a likelihood contribution, not a probability that the model or its parameters are correct.
Practical Checks, Alternatives, and Limits
Before fitting, create a sample-level record of the reported result, BQL status, applicable LLOQ, assay or method, and—where relevant—run or batch. Check that quantified results are consistent with their reported threshold, allowing for the laboratory’s documented reporting rules and rounding. Check for implausible threshold patterns, missing thresholds, and samples whose assay assignment conflicts with dates or batch records. These checks test the input data and assignment; they do not substitute for evaluating whether the observation model is adequate.
For a simple change in sensitivity, a sample-specific censoring likelihood is generally more faithful to the records than replacing BQL values with zero or a fraction of a common LLOQ. Substitution treats an interval statement as though it were an exact concentration and discards the distinction between assays. Likewise, using one common threshold is justified only if it truly applies to every relevant sample. If all records are quantified, threshold assignment does not change their ordinary density contributions, although assay-specific measurement properties may still matter.
A threshold-only adjustment has a limitation: it assumes that the assay change altered the censoring limit but did not otherwise alter the measurement process. If calibration, bias, residual variability, sample handling, or the relationship between measured and true concentration also changed, those changes may require a richer observation model. The LLOQ and the residual-error model are different components. Updating the threshold alone cannot repair a misspecified residual distribution or an unmodeled assay shift.
The additive normal model above is one explicit case. For a different observation model, derive the BQL contribution from that model’s probability below \(L_{ij}\), rather than reusing the normal formula. For example, with a lognormal measurement model, the threshold is still sample-specific, but the probability is evaluated on the model’s log scale. In either case, the threshold must be expressed in units and on the scale that correspond to the observed assay result.
Use laboratory metadata to identify the method or assay that generated each reported value or BQL status.
Assign the threshold used for that sample’s result; do not assume that the study’s first or final threshold applies to every record.
For a BQL record, use the model probability below its assigned threshold. For a quantified record, use the density at its observed value under the specified observation model.
If other assay properties changed, consider whether the observation model must represent those changes as well.
Common Pitfalls
- Using one threshold because the file has one LLOQ field. A single field may conceal assay changes. Preserve a threshold for each record or a verified mapping that returns the correct threshold for every sample.
- Assigning thresholds by study date without checking laboratory records. Dates can be a valid proxy when the assay switch was fixed and documented. Reanalysis, batch timing, or exceptions can make date-based assignment wrong.
- Treating BQL as zero or as the threshold itself. BQL means the result is below a limit, not that the concentration equals zero or the LLOQ. The censored likelihood uses the probability of the appropriate interval.
- Claiming the wrong threshold must shift a particular parameter in a known direction. It changes record-level likelihood contributions, but the fitted parameter response depends on the full model, the data, and other records.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A study changes from an assay with LLOQ 0.6 mg/L to one with LLOQ 0.25 mg/L. What determines the threshold assigned to a sample collected near the transition?
- Under additive normal residual error with prediction \(C=0.50\) mg/L, \(\sigma=0.10\) mg/L, and sample LLOQ \(L=0.30\) mg/L, what is the BQL likelihood contribution?
- Why can using the final study LLOQ for all BQL records change the fitted likelihood even if the concentration model is unchanged?
- What additional concern arises if an assay change also alters calibration or residual variability?
Solutions. (1) Use the assay and reporting procedure that generated that sample’s result, verified from the laboratory record; collection date alone is sufficient only if it correctly maps to the assay used. (2) The standardized cutoff is \((0.30-0.50)/0.10=-2\), so the contribution is \(\Phi(-2)\approx0.02275\). (3) The BQL contribution is \(\Phi((L_{ij}-C_{ij})/\sigma)\); changing \(L_{ij}\) changes the probability assigned to the recorded event. (4) The threshold-only model may be inadequate: changes to calibration or variability may require corresponding changes in the observation model.