Why Examine Predictions Within Pediatric Subgroups?
A population model may predict concentrations well on average and still miss systematically for a clinically relevant group. For example, overprediction in one age range can cancel underprediction in another when results are pooled. Such cancellation can make an overall summary look reassuring even though the model’s predictions—and any dose recommendation based on them—are poorly calibrated for a subgroup.
Subgroup diagnostics compare measured concentrations with predictions for children grouped by age, weight, or renal function. Their purpose is to find patterns that a pooled summary could hide. A subgroup pattern is a diagnostic signal, not proof of a biological mechanism and not, by itself, a dose-adjustment rule.
For dose-recommendation diagnostics, predictions should be generated from the intended population model and the recorded regimen and sampling time. A population prediction should not be updated using the same concentration that is then assessed against it: doing so can pull the prediction toward the observation and conceal a systematic miss. All comparisons also require compatible concentration units and a clear definition of the prediction being evaluated.
Summaries That Reveal Direction and Size
Within a subgroup, the mean prediction error (MPE) is the average of the signed errors. It retains direction, so positive and negative errors can cancel. The mean relative error expresses that signed average relative to each prediction. The mean absolute relative error (MARE) averages the magnitudes of the relative errors and describes typical proportional discrepancy without direction. These summaries answer different questions and should not be substituted for one another.
A useful complementary summary is the geometric mean observed-to-predicted ratio. Define \(d_i=\ln(C_{\mathrm{obs},i}/C_{\mathrm{pred},i})\). Then \(\exp(\bar d)\) is the geometric mean ratio: a value above 1 indicates that observations tend to exceed predictions, and a value below 1 indicates that they tend to fall below predictions.
This identity follows from the logarithm rule that a sum of logarithms is the logarithm of a product. A ratio of 1 corresponds to zero average log-ratio; a ratio of 1.20 corresponds to a geometric average observation 20% above prediction. Ratios require positive concentrations. When values are zero or below quantification, do not insert an arbitrary small value without a justified analysis plan; use an approach suited to those data.
Worked Example: Comparing Predictions Across Age Strata
Suppose hypothetical measurements from two prespecified age strata have the following mean concentrations. All measurements were taken at comparable post-dose times under the recorded regimens. The model predictions are population predictions based on the children’s covariates and dosing histories.
| Age stratum | Mean observed | Mean predicted |
|---|---|---|
| 12 to less than 24 months | 8.40 mg/L | 7.50 mg/L |
| 6 to less than 12 years | 6.10 mg/L | 6.40 mg/L |
For the younger stratum, the difference of the means is \(8.40-7.50=0.90\) mg/L. The ratio of the means is \(8.40/7.50=1.12\), so the mean observed concentration is 12.0% higher than the mean predicted concentration. This is evidence of underprediction in this descriptive summary.
For the older stratum, the difference is \(6.10-6.40=-0.30\) mg/L and the ratio is \(6.10/6.40=0.9531\). The mean observed concentration is about 4.69% below the mean predicted concentration, indicating modest overprediction in this summary. Because the directions differ, a pooled average could hide both patterns. These aggregate calculations are descriptive; individual-level errors and subgroup sample sizes are needed to assess how consistent the pattern is.
Stratification and Comparability
Choose age, weight, and renal-function strata before inspecting the diagnostic results when possible. Clinically meaningful cutpoints can make a pattern interpretable, but arbitrary cutpoints can create unstable or misleading contrasts. Preserve the continuous covariate values as well: a plot of errors or observed-to-predicted ratios against age, weight, or renal function can reveal a gradual trend that broad groups conceal.
A valid comparison also requires that observed and predicted concentrations refer to comparable conditions. In particular, check the dose amount, dose interval, time since the most recent dose, formulation, adherence information when available, concentration units, and the method used to estimate renal function. Different sampling-time distributions across strata can produce apparent differences because concentrations change over time even when the model is adequate. A subgroup with later samples, for example, should not be compared as if its samples were taken at the same point in the dosing interval.
For each stratum, report the number of children and measurements, the prediction-error summary, and the spread of the errors. Plotting individual observations against predictions or plotting residuals against the covariate can show outliers, curvature, increasing spread, and clusters. If a child contributes repeated measurements, those observations are not independent. Summarize at the child level or use an uncertainty method that accounts for clustering; treating every measurement as an independent child can make the apparent evidence too precise.
Worked Example: Separating Directional Bias From Error Magnitude
Consider hypothetical concentration measurements from two weight strata. For each child, compute \(r_i=(C_{\mathrm{obs},i}-C_{\mathrm{pred},i})/C_{\mathrm{pred},i}\). Concentrations are in mg/L.
| Weight stratum | Observed / predicted (mg/L) | Relative errors |
|---|---|---|
| Below 20 kg | 10.0 / 9.0; 8.0 / 8.5; 12.0 / 11.0 | 11.11%; −5.882%; 9.091% |
| 20 kg or more | 6.0 / 6.5; 7.2 / 8.0; 5.5 / 6.0 | −7.692%; −10.00%; −8.333% |
For the below-20-kg group, the signed mean relative error is \((11.11-5.882+9.091)/3=4.773\%\). Its MARE is \((11.11+5.882+9.091)/3=8.694\%\). The signed result suggests modest average underprediction, while the MARE indicates that the average absolute proportional discrepancy is larger.
For the group at or above 20 kg, the signed mean relative error is \((-7.692-10.00-8.333)/3=-8.675\%\). Its MARE is \((7.692+10.00+8.333)/3=8.675\%\). All three observations are below prediction, so the direction is consistent in this small example. The results suggest that predictions may be biased in opposite directions across the two strata; three children per group are not enough to establish a stable population pattern.
Interpreting a Renal-Function Pattern
A trend across renal-function strata may be relevant to a dose recommendation if renal function is associated with drug disposition and the predictions are systematically wrong under the evaluated regimens. However, the diagnostic does not identify the cause. A trend might reflect an inadequate covariate relationship, differences in dose or sampling time, errors in renal-function measurement, or a small and unrepresentative subgroup. The observed pattern should prompt checks and model evaluation, not an immediate dose change.
Worked Example: Detecting Increasing Underprediction With Renal Impairment
Suppose hypothetical, comparable-time subgroup summaries are as follows. Each row represents a different renal-function stratum, and the observed and predicted values are mean concentrations in mg/L.
| Renal-function stratum | Mean observed | Mean predicted | Observed / predicted |
|---|---|---|---|
| Higher function | 5.20 | 5.00 | 1.040 |
| Moderately reduced | 8.10 | 6.00 | 1.350 |
| Severely reduced | 12.0 | 7.50 | 1.600 |
The ratios are \(5.20/5.00=1.040\), \(8.10/6.00=1.350\), and \(12.0/7.50=1.600\). Thus, the subgroup mean observations are 4.00%, 35.0%, and 60.0% above their corresponding predictions. The increasing ratios are a signal of increasingly strong underprediction across these strata in this hypothetical data set.
Before connecting the pattern to dosing, check subgroup sizes, individual data, the renal-function calculation, dose histories, and sampling times. If the pattern remains after those checks, the model’s renal-function relationship and the resulting exposure predictions merit further evaluation. The summaries alone do not establish that renal impairment caused the discrepancy or specify a safe replacement dose.
Diagnostics, Alternatives, and Limitations
Subgroup summaries are most informative when paired with plots and a prespecified interpretation. Look for a consistent sign of error, a trend in relative error across ordered strata, unusually large individual discrepancies, and changes in error spread. A few extreme values can dominate an arithmetic mean, so inspect the observations rather than relying on one number. If the subgroup is small, describe the result as uncertain and avoid strong claims based on a striking ratio alone.
An overall prediction-error summary is a useful alternative when the question is overall performance, but it can conceal subgroup-specific bias. Continuous covariate plots avoid dependence on cutpoints and can show trends, but they may be harder to summarize and can be noisy with sparse data. Formal subgroup comparisons can quantify uncertainty, but they do not correct poor comparability or turn an exploratory pattern into causal evidence. If intervals are reported, they should reflect the sampling structure, including repeated observations within children where relevant.
The diagnostics assess predictions for the population and conditions represented by the data. Good average agreement does not ensure that every child’s concentration is predicted accurately, and a mismatch in a subgroup does not by itself establish that a particular dose recommendation is unsafe or ineffective. These summaries should be considered alongside the intended exposure target, clinical context, and the uncertainty in the underlying model. They are checks on model performance, not substitutes for clinical judgment or a complete dose evaluation.
Common Pitfall
Correction: Opposing subgroup errors can cancel in the pooled mean. Compare observed and predicted concentrations within clinically relevant strata, examine individual error patterns, and check that dose and sampling conditions are comparable. A subgroup discrepancy is a reason to investigate, not an automatic instruction to change the dose.
Specify the prediction, measurement, covariates, regimen, sampling time, and concentration units being evaluated.
Use interpretable age, weight, or renal-function groups, and report how many children and measurements contribute to each.
Calculate signed errors or ratios as well as an absolute-error measure; inspect the individual data and plots.
Check timing, dose history, data quality, renal-function measurement, repeated observations, and subgroup size before linking a pattern to dosing.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- In a subgroup, \(C_{\mathrm{obs}}=9.0\) mg/L and \(C_{\mathrm{pred}}=10.0\) mg/L. Calculate the error and relative error. Is the model underpredicting or overpredicting?
- Why can a pooled mean error near zero coexist with important subgroup discrepancies?
- Three relative errors are \(+10\%\), \(-10\%\), and \(+10\%\). Calculate the mean relative error and MARE. What does each summarize?
- A renal-function subgroup appears to have higher observed concentrations than predicted, but its samples were taken later after dosing than samples in the comparison group. What should be checked before interpreting the subgroup contrast?
Solutions. (1) \(e=9.0-10.0=-1.0\) mg/L and \(r=-1.0/10.0=-0.10=-10.0\%\). The observation is below prediction, so this is overprediction by the model. (2) Positive errors in one subgroup can be offset by negative errors in another when pooled; the overall signed mean can therefore be near zero despite subgroup bias. (3) The mean relative error is \((10-10+10)/3=3.33\%\), indicating slight average underprediction. The MARE is \((10+10+10)/3=10.0\%\), indicating the average absolute proportional discrepancy. (4) Check whether dose histories and sampling times are comparable and whether the recorded times are accurate. Because concentration changes over the dosing interval, the timing difference could contribute to the apparent contrast; it should be addressed before attributing the pattern to renal function.