Predictive Checks for the Frequency of BQL Samples

Why Check the Frequency of BQL Samples?

A model can fit quantified concentrations reasonably well and still predict too few or too many below-quantification-limit (BQL) records. This matters because the pattern of BQL observations carries information about whether the model reproduces concentrations near and below the assay’s reporting threshold. A single overall BQL percentage can hide important failures: predictions may agree on average but miss the rise in BQL frequency late in time, at a particular dose, or in one subgroup.

A predictive check compares the observed BQL pattern with patterns generated under the fitted model. The central requirement is that simulated concentrations must be classified using the same sample-specific lower limits of quantification (LLOQs) used for the corresponding observed samples. Otherwise, the comparison confounds model predictions with a change in the censoring rule.

In the observed data, \(B_{ij}\) is known from the reported record: a BQL result contributes 1 and a quantified result contributes 0. A predictive check generates replicated concentrations under the fitted model, applies the same censoring rule to each replicated sample, and calculates replicated BQL frequencies. It does not replace BQL records with numerical values such as zero or a fraction of the LLOQ.

Constructing a Design-Matched Predictive Check

Let the fitted NLME model specify typical concentration predictions, between-subject variability, and residual error. To generate a replicated dataset, retain the study’s relevant design information: subject covariates, dose and dosing history, sampling times, and the LLOQ attached to each sample. Generate new random effects and residual errors according to the model, then apply the sample’s LLOQ to its simulated concentration. This process is repeated to create many replicated datasets.

For a chosen group \(g\), such as a time window or dose group, let \(n_g\) be the number of evaluable samples and \(B_{ij}^{\mathrm{obs}}\) the observed BQL indicator. The observed frequency is

For replicate \(m\), define the simulated frequency in the same group by

The distribution of \(r_g^{(1)},\ldots,r_g^{(M)}\) is the model-based reference distribution for that check. Its median describes a typical frequency under the fitted model; its quantiles show the range of frequencies that can arise from model and sampling variability. The observed rate can be compared with that distribution, for example by checking whether it falls inside a central 90% simulation interval.

Why Simulation Provides the Reference Distribution

A simple calculation helps explain the check. If each sample had a known probability \(p_{ij}\) of being BQL, then the expected number of BQL records in group \(g\) would be the sum of those probabilities. The expected frequency would therefore be their average:

For example, if every sample in a group had BQL probability 0.30, the expected frequency would be 0.30. In an NLME model, however, those probabilities depend on predicted concentrations, covariates, random effects, residual error, and the individual LLOQs. In addition, samples from the same subject are generally dependent through shared random effects. Thus a binomial calculation based on one common probability can give an incorrect reference range.

Simulation handles these features directly: it generates subject-level random effects and within-subject observations under the fitted model, then applies each sample’s threshold. The simulated frequency distribution consequently reflects the model’s predicted variation and, when implemented with the correct subject-level structure, the dependence among records. This is a model-based check, not a distribution-free test.

Stratifying by Dose and Subgroup

The same calculation can be repeated within dose groups or clinically relevant subgroups. Define groups before examining the results where practical, and report each rate with its numerator and denominator. A rate of 8/20 is not as precise as 80/200, even though both equal 40%; denominators make that distinction visible. If subjects contribute multiple records to a group, the simulation should preserve that clustering rather than treating every record as an independent subject.

Subgroup checks can reveal a pattern masked by an overall rate. They should be interpreted as diagnostics of model adequacy, not as causal evidence that a covariate produced a difference. A discrepancy may reflect an inadequately described concentration-time profile, residual error, subgroup composition, an assay or data-processing issue, or an incorrectly represented design feature.

Assumptions, Diagnostics, and Alternatives

A useful predictive check requires that the fitted model and simulation procedure represent the intended data-generating process. At minimum, the analyst should verify the following:

• The observed and simulated records use the same scheduled samples, group definitions, and evaluability rules.
• Each simulated sample is censored at the LLOQ belonging to its matching observed sample, including any assay-specific changes over time.
• The simulation includes the model components relevant to prediction, such as residual error and between-subject variability. Random effects should be generated or conditioned on according to the stated purpose of the check.
• Missing or excluded records are handled consistently. A sample that was never collected is not automatically a BQL sample; the denominator should follow a clearly stated rule.
• There are enough replicates for stable estimates of the reference distribution. With \(M=500\), for example, each 5th percentile is determined by only about 25 replicates in the lower tail, so extreme quantiles remain imprecise.

A population-level predictive check generally generates new subject random effects and residual errors under the fitted population model. A conditional check may instead condition on subject-specific information, but that answers a different question and can make predictions appear more tailored to the observed subjects. State which approach was used. If parameter uncertainty is important, it can be incorporated by drawing parameter values from an appropriate uncertainty distribution or by a bootstrap procedure that refits the model. A simulation envelope based only on one fixed parameter estimate does not fully account for parameter-estimation uncertainty.

Inspect discrepancies by time, dose, and subgroup, and also inspect the overall rate. A model may pass a pooled check while failing in one stratum. Conversely, a single stratum can appear discrepant by chance when many strata are examined; the number and selection of checks should therefore be reported. Predictive intervals are not confidence intervals for a population BQL probability and do not guarantee that a specified fraction of future observed rates will fall inside them unless the full repeated-sampling procedure and its coverage have been established.

Frequency checks complement, rather than replace, other diagnostics. Residual diagnostics when values are censored examine aspects of the model’s residual behavior; frequency checks focus on whether the model reproduces the occurrence of the censoring event. Likelihood-based assessment uses the observed data and the specified censoring model to estimate or compare models, while predictive checks ask whether replicated data resemble the observed pattern. No single approach establishes adequate fit across all features.

Practical Workflow