Tutorials › Nonlinear Mixed-Effects Models (NLME) › M4: Substituting the LLOQ

Below-Quantification-Limit Observations · Tutorial 290 of 797

M4: Substituting the LLOQ

You will implement LLOQ substitution and assess how its artificial exact values affect model residuals and parameter estimates.

Advanced 12 min read

What You'll Learn

  • Define the M4 substitution rule for below-quantification-limit records.
  • Distinguish an exact-value density contribution from a censored-event probability.
  • Calculate substituted residuals and a simple parameter estimate.
  • Trace how substitution can affect the fitted slope of a nonlinear concentration-time profile.
  • Identify residual patterns and modeling conditions that make M4 results concerning.

What M4 Does to a BQL Record

A below-quantification-limit (BQL) result does not report an exact concentration. It reports that the assay result fell below a stated lower limit of quantification (LLOQ). M4 replaces that threshold-bound record with the LLOQ itself and then fits the model as if the replacement were an observed value. The rule is easy to implement, but the replacement is not a recovered measurement: it is an imposed value at the boundary.

Write \(L_{ij}\) for the LLOQ applying to observation \(j\) on individual \(i\). Let \(y_{ij}\) denote a quantified result, and let \(F_{ij}(\theta,\eta_i)\) be the model-predicted response at that observation time. Here \(\theta\) denotes population parameters and \(\eta_i\) the individual-level random effects. M4 constructs an analysis value \(y^{M4}_{ij}\) as follows.

$$ y^{M4}_{ij} = \begin{cases} y_{ij}, & \text{if the result is quantified},\\ L_{ij}, & \text{if the result is BQL}. \end{cases} $$

The resulting data are passed to the chosen NLME fitting procedure as if every \(y^{M4}_{ij}\) were an exact observation. For example, under an additive normal residual model, the conditional residual is \(e^{M4}_{ij}=y^{M4}_{ij}-F_{ij}(\theta,\eta_i)\). Its sign and size are calculated using the substituted LLOQ, not the unknown assay result that generated the BQL report.

Definition: M4 (LLOQ substitution) replaces each BQL observation with its applicable LLOQ and analyzes that replacement as an exact response. It is a data-substitution rule, not a statement that the underlying concentration equals the LLOQ.

What the Fit Treats as Data

Under a conditional additive normal residual model, suppose \(Y_{ij}\), before applying the assay reporting rule, has distribution \(Y_{ij}\mid\eta_i\sim N(F_{ij}(\theta,\eta_i),\sigma^2)\). For a quantified record, the conditional likelihood contribution is the normal density evaluated at its measured value. Under M4, a BQL record contributes that same density evaluated at \(L_{ij}\):

$$ L^{M4}_{ij}(\theta,\eta_i,\sigma) = \frac{1}{\sqrt{2\pi}\sigma} \exp\left[-\frac{\left(y^{M4}_{ij}-F_{ij}(\theta,\eta_i)\right)^2}{2\sigma^2}\right]. $$

This contribution is not the likelihood contribution for the information “the result was below \(L_{ij}\).” Under the stated normal model, that event has probability \(P(Y_{ij}<L_{ij}\mid\eta_i)=\Phi((L_{ij}-F_{ij})/\sigma)\), where \(\Phi\) is the standard normal cumulative distribution function. The event probability accounts for all possible assay results below the limit; the M4 density assigns its contribution to one exact value at the limit.

In an NLME analysis, individual random effects are handled according to the model’s likelihood procedure. For example, a marginal likelihood integrates the conditional likelihood over the random-effects distribution. M4 changes the response used in that procedure; it does not remove the random effects or otherwise change their distributional specification. The distinction between a conditional density at a value and a probability for a censored event remains.

Worked Example: Residual and Likelihood at One BQL Record

Suppose a hypothetical observation has an LLOQ of \(1.00\) mg/L, is reported BQL, and has a model prediction of \(0.60\) mg/L. Assume an additive normal residual model with \(\sigma=0.20\) mg/L. Calculate the M4 residual and compare the exact-value and censored-event contributions.

M4 residual. The substituted response is \(y^{M4}=1.00\) mg/L, so \(e^{M4}=1.00-0.60=+0.40\) mg/L. Its standardized residual is \(0.40/0.20=2.00\).

M4 density contribution. At the substituted value, the density is \(\frac{1}{\sqrt{2\pi}(0.20)}\exp[-(0.40)^2/(2(0.20)^2)]\). Since the exponent is \(-2\), this is approximately \(0.2700\) per mg/L.

Censored-event contribution. The probability of a result below the LLOQ under this prediction is \(\Phi((1.00-0.60)/0.20)=\Phi(2.00)\approx0.9772\). This probability and the M4 density are different quantities, with different units and interpretations. The comparison shows why replacing a BQL event with an exact value changes the likelihood information presented to the fit.

How Substitution Changes Parameter Estimates

A fit adjusts parameters to make its modeled responses compatible with the values it is given. M4 therefore makes every BQL record exert the influence of an exact response at the threshold. In a least-squares fit, for example, the contribution from a BQL record is proportional to \((L_{ij}-F_{ij})^2\). In a likelihood fit, the corresponding density is evaluated at \(L_{ij}\). The direction and size of a parameter shift depend on the model, sampling times, residual-error model, other observations, and the number and pattern of BQL records.

For a declining concentration profile, if the model prediction at a late time is below the LLOQ, substituting the LLOQ creates a response larger than that prediction. A fit may respond by predicting higher late concentrations, which can correspond to a slower decline. This is a possible pattern, not a guaranteed direction: parameters are estimated jointly, and the influence of a record depends on the full design and likelihood.

Worked Example: A Mean Estimate After Substitution

Consider three hypothetical responses modeled by a common mean \(\mu\), with independent additive normal errors having a fixed common standard deviation. The first and third results are quantified at \(1.20\) and \(1.40\) mg/L. The second is BQL at an LLOQ of \(1.00\) mg/L. Find the M4 estimate of \(\mu\).

Under M4, the analysis values are \(1.20,\ 1.00,\ 1.40\) mg/L. With a common known variance, maximizing the normal likelihood for a common mean is equivalent to minimizing the sum of squared deviations; the minimizing mean is the arithmetic average:

$$ \hat{\mu}_{M4} =\frac{1.20+1.00+1.40}{3} =\frac{3.60}{3} =1.20\ \text{mg/L}. $$

At this estimate, the three residuals are \(0.00,\ -0.20,\ +0.20\) mg/L, and the sum of squared residuals is \(0^2+(-0.20)^2+(0.20)^2=0.0800\ (\text{mg/L})^2\). The BQL record contributes a residual of \(-0.20\) mg/L because M4 treats it as exactly \(1.00\) mg/L. Its unknown underlying assay result could have been anywhere below \(1.00\) mg/L. Thus, the arithmetic is the M4 estimate, not evidence that the actual BQL response equaled the LLOQ.

The same reasoning applies in nonlinear models, although their parameter estimates generally do not have a simple arithmetic formula. The next example uses a log-concentration model and makes the fitting rule explicit so the effect can be calculated directly.

Worked Example: Substitution in a Declining Concentration Profile

Suppose a hypothetical one-compartment profile is \(C(t)=10\exp(-kt)\) mg/L, with \(t\) in hours and the intercept fixed at \(10\) mg/L. There is a quantified result of \(4.00\) mg/L at \(t=1\) h and a BQL result at \(t=3\) h with LLOQ \(1.00\) mg/L. For illustration, estimate \(k\) by equally weighted least squares on log concentrations, applying M4 to the BQL record.

After substitution, the log responses are \(\ln(4.00)\) at \(t=1\) and \(\ln(1.00)=0\) at \(t=3\). The model on this scale is \(\ln C(t)=\ln(10)-kt\). With the intercept fixed, minimizing squared log residuals gives \( \hat{k}=-\sum_j t_j(\ln y^{M4}_j-\ln(10))/\sum_j t_j^2 \). Substitution gives

$$ \hat{k}_{M4} =-\frac{1\ln(4.00/10)+3\ln(1.00/10)}{1^2+3^2} =-\frac{-0.9163-6.9078}{10} =0.7824\ \text{h}^{-1}. $$

The fitted prediction at \(t=3\) h is \(10\exp[-(0.7824)(3)]\approx0.9563\) mg/L, just below the LLOQ. The substituted value pulls the fit toward a higher late concentration than a still-lower actual response would. In this particular fixed-intercept, equal-weight log-scale calculation, a lower late response would pull the estimated decline rate upward. The example illustrates a possible parameter effect under stated conditions; it does not establish a universal bias direction for NLME fits.

Assumptions, Diagnostics, and Limitations

M4 requires the analyst to know which records are BQL and the applicable LLOQ for each such record. The substitution must be on the scale used for the analysis: when the model is fit to log concentrations, the substituted value is \(\ln(L_{ij})\), not \(L_{ij}\) inserted directly as a log response. The model’s residual-error specification should also be appropriate for the analyzed scale. If limits differ between records, use the relevant limit for each record rather than assigning one threshold indiscriminately.

These implementation conditions do not make the substitution statistically exact. M4 effectively imposes the additional data assumption that a BQL response can be represented by its LLOQ. That assumption is generally not warranted by the report itself, which only establishes that the result fell below the limit. The censored-likelihood approach instead uses the probability of that event under the observation model, as developed in the earlier material on censored likelihoods.

Several checks can reveal how strongly M4 shapes a fit. Inspect residuals by BQL status and sampling time, and look for an artificial pile-up of analysis values exactly at the limit. Compare predictions near and below the LLOQ with the substituted values; repeated positive residuals at late times, for example, can indicate that the model is being pulled upward there. Review changes in the relevant parameter estimates and their uncertainty when the BQL records are included. A stable optimizer or a successful covariance step does not establish that the substituted observations are appropriate.

M4 may be convenient when a simple analysis is required, but it discards the distinction between “below the limit” and “equal to the limit.” It can distort residual patterns and influence population parameters, individual predictions, and uncertainty estimates. The impact may be small when BQL records are few and have little leverage, or material when many such records occur in informative parts of the profile. A censored likelihood is the more direct way to represent threshold information under a specified observation model; substitution is a different approximation, not an equivalent computational shortcut.

Common mistake: “The assay reported BQL, so entering the LLOQ gives the observed concentration.” The correction is that the assay reported an event, \(Y<L\), not the exact value \(Y=L\). M4 deliberately treats the boundary as an exact analysis value; interpret resulting residuals and estimates as consequences of that approximation.

Check Your Understanding

Answer each question, then compare with the solutions below.

  1. A record is BQL at \(L=0.50\) mg/L, and the model prediction is \(0.70\) mg/L. What M4 value and raw residual are used?
  2. Under an additive normal model, why is a BQL event contribution not the normal density evaluated at the LLOQ?
  3. Two quantified responses are \(2.0\) and \(2.4\) mg/L, and one response is BQL at \(1.0\) mg/L. What is the common-mean M4 estimate under independent normal errors with fixed common variance?
  4. In a declining profile, why might substituting the LLOQ affect an estimated elimination rate?
  5. Name one diagnostic that can reveal the influence of M4, and explain what it checks.

Solutions. (1) M4 uses \(0.50\) mg/L, giving residual \(0.50-0.70=-0.20\) mg/L. (2) A BQL report identifies the event \(Y<L\), whose contribution is \(P(Y<L)\); a density at \(L\) instead treats one exact response as observed. (3) The substituted values are \(2.0,\ 2.4,\ 1.0\) mg/L, so the estimate is their mean, \(5.4/3=1.80\) mg/L. (4) The substituted late response may be higher than the underlying response, pulling predictions upward at that time and changing the fitted decline; the direction depends on the full model and data. (5) Plot residuals against time by BQL status. A cluster or repeated pattern among substituted records can show that the boundary values are influencing the fit in a systematic way.