Why Plug-In Estimates Understate Translational Risk

Why a Single Scaled Prediction Is Incomplete

A translational scaling equation can turn animal data into a human prediction, but the equation’s parameters are estimated from finite, imperfect evidence. Substituting their estimates produces a useful point prediction; it does not show how the prediction changes across plausible parameter values. This matters in drug development because a human prediction is often used to inform decisions under uncertainty, not merely to report a central value.

A plug-in prediction is not necessarily biased downward, and uncertainty does not guarantee that the prediction will miss in one particular direction. The central issue is that the single number suppresses uncertainty. Depending on the model’s nonlinearity, the plug-in value can differ from the average prediction over parameter uncertainty as well. The introductory material on worked PK-PD translation with uncertainty provides context for translational predictions; here the focus is on why the point-estimate substitution alone is incomplete.

Definition: Let \(g(\theta)\) be a human prediction from a translational model with uncertain parameter vector \(\theta\). The plug-in prediction is \(g(\hat{\theta})\), obtained by substituting the estimated parameter vector \(\hat{\theta}\). A prediction distribution represents the variation in \(g(\theta)\) as uncertain inputs vary, and may also include other specified sources of predictive variation. A prediction interval is an interval constructed from that distribution to contain a stated proportion of predictions under the model and its assumptions.

These quantities answer different questions. The plug-in result is the model output at the selected parameter estimates. An interval based on parameter uncertainty describes how that output varies as the parameters vary. A predictive interval for a future human outcome may additionally include residual or between-individual variation, if those sources belong to the prediction target. An interval that includes parameter uncertainty alone should not be described as covering all possible variation in human outcomes.

How Parameter Uncertainty Enters a Prediction

A useful local approximation follows by expanding \(g(\theta)\) around \(\hat{\theta}\). Write \(\mu=E(\theta)\), \(\Sigma=\operatorname{Cov}(\theta)\), and let \(\nabla g\) and \(\nabla^2 g\) denote the gradient and Hessian of \(g\). Taking expectations of the second-order Taylor expansion gives:

\begin{aligned} g(\theta) \approx {} & g(\hat{\theta}) +\nabla g(\hat{\theta})^{T}(\theta-\hat{\theta})\\ &+\frac{1}{2}(\theta-\hat{\theta})^{T} \nabla^2g(\hat{\theta})(\theta-\hat{\theta}),\\[4pt] E\{g(\theta)\}\approx {} & g(\hat{\theta}) +\nabla g(\hat{\theta})^{T}(\mu-\hat{\theta})\\ &+\frac{1}{2}\operatorname{tr}\!\left[ \nabla^2g(\hat{\theta}) \left\{\Sigma+(\mu-\hat{\theta})(\mu-\hat{\theta})^{T}\right\} \right]. \end{aligned}

The justification is that \(E(\theta-\hat{\theta})=\mu-\hat{\theta}\), while \(E\{(\theta-\hat{\theta})(\theta-\hat{\theta})^{T}\} =\Sigma+(\mu-\hat{\theta})(\mu-\hat{\theta})^{T}\). Thus, the first-order term vanishes only when the uncertainty distribution is centered at \(\hat{\theta}\). Under that centering assumption, the second-order correction is \(\frac{1}{2}\operatorname{tr}\{\nabla^2g(\hat{\theta})\Sigma\}\). If the prediction is locally linear, its curvature term is zero; if it is curved, parameter uncertainty can shift its mean away from the plug-in value.

The corresponding first-order approximation to the variance is \( \operatorname{Var}\{g(\theta)\}\approx \nabla g(\hat{\theta})^T\Sigma\nabla g(\hat{\theta})\). This shows why the covariance structure matters: correlated parameter estimates can either increase or decrease uncertainty in a particular prediction. The approximation is most useful when uncertainty is reasonably concentrated near \(\hat{\theta}\) and the model is smooth there. Strong nonlinearity, skewed parameter distributions, or substantial uncertainty can make it inadequate.

Worked Example: Propagating Uncertainty in an Allometric Prediction

Suppose a hypothetical animal study gives clearance \(CL_A=0.80\) L/h in an animal weighing \(W_A=0.25\) kg. A simple allometric prediction for a 70 kg human is \(CL_H=CL_A(W_H/W_A)^b\), with estimated exponent \(b=0.75\). Suppose the uncertainty in \(a=\log(CL_A/[1\text{ L/h}])\) has standard deviation 0.100, the uncertainty in \(b\) has standard deviation 0.0800, and their correlation is \(-0.500\). Treat these as a centered joint uncertainty distribution and use a first-order variance approximation on the log scale.

The log prediction is \(Y=\log(CL_H/[1\text{ L/h}])=a+bx\), where \(x=\log(70/0.25)=\log(280)=5.63479\). The plug-in prediction is:

\begin{aligned} \hat{Y} &=\log(0.80)+0.75\log(280)\\ &=-0.223144+0.75(5.63479)\\ &=4.00295,\\ \widehat{CL}_H&=\exp(4.00295)\text{ L/h}=54.76\text{ L/h}. \end{aligned}

The covariance of \(a\) and \(b\) is \(\operatorname{Cov}(a,b)=(-0.500)(0.100)(0.0800)=-0.00400\). Since \(Y=a+bx\), its variance is:

\begin{aligned} \operatorname{Var}(Y) &\approx (0.100)^2+(5.63479)^2(0.0800)^2 +2(5.63479)(-0.00400)\\ &=0.0100+0.2032-0.04508\\ &=0.1681,\\ SE(Y)&=\sqrt{0.1681}=0.4100. \end{aligned}

For illustration, if the log prediction is approximately normal, a 95% interval for the parameter-uncertainty-based prediction on the original scale is \(\exp\{\hat{Y}\pm1.96SE(Y)\}\). Substitution gives \(\exp\{4.00295\pm1.96(0.4100)\}\), or approximately 24.5 to 122 L/h. This interval is much more informative about parameter-driven uncertainty than the single plug-in value of 54.76 L/h. It is not automatically an interval for an individual patient’s clearance: that would require including relevant individual-level variation in the predictive distribution.

The negative covariance reduces the variance here. Ignoring it would give a log-scale variance of about \(0.2132\), rather than \(0.1681\), and a wider interval. The covariance must therefore be retained when it is part of the parameter uncertainty estimate; assuming independent inputs without justification can misstate uncertainty in either direction.

Bootstrap Predictions: Refit, Then Predict

A bootstrap carries parameter-estimation uncertainty through the prediction by repeatedly resampling the original evidence and refitting the model. Each refit produces a parameter set, and applying the translational equation to that set produces one bootstrap prediction. If parameters are correlated, refitting them jointly preserves the dependence represented in the resampled data. The resulting distribution can show skewness and other features that a symmetric normal approximation may miss.

A percentile interval uses quantiles of the bootstrap predictions. It is meaningful only if the resampling scheme matches how the evidence was collected and the refitting procedure is appropriate. For instance, resampling individual observations independently when the data are clustered or longitudinal may break the data structure. A percentile interval also does not repair a misspecified scaling model or account for uncertainty sources that were never resampled.

Worked Example: An Interval From Bootstrap Predictions

Suppose a hypothetical allometric model is refitted after each bootstrap resample, and each refit produces a human clearance prediction in L/h. For a small teaching illustration, suppose 20 predictions, sorted from smallest to largest, are:

26, 31, 34, 38, 41, 44, 47, 49, 52, 54, 57, 60, 64, 69, 74, 80, 88, 97, 110, 128 L/h.

Use the nearest-rank empirical quantile convention: for a probability \(p\), select sorted rank \(\lceil 20p\rceil\). A central 80% percentile interval uses probabilities 0.10 and 0.90. The lower rank is \(\lceil20(0.10)\rceil=2\), whose prediction is 31 L/h. The upper rank is \(\lceil20(0.90)\rceil=18\), whose prediction is 97 L/h. Thus, this illustrative bootstrap interval is 31 to 97 L/h. The median of these 20 predictions is the average of ranks 10 and 11, \((54+57)/2=55.5\) L/h.

The interval describes the bootstrap distribution of the model’s human predictions under the chosen resampling procedure. With only 20 replicates its endpoints are coarse and unstable; an actual analysis would generally use many more replicates and assess endpoint stability. The bootstrap median need not equal the prediction from the original fitted parameter estimates.

Simulation-Based Prediction Intervals

Simulation offers a direct way to propagate uncertain inputs through a nonlinear prediction. Specify distributions for the uncertain parameters and, when the target requires it, additional predictive components such as residual variation. Draw a joint parameter set, evaluate the human prediction, add any specified predictive variation, and repeat. Quantiles of the resulting predictions form an interval. The distributions and dependence structure must be justified; drawing each input independently when estimates are correlated can distort the result.

Worked Example: Adding Parameter and Residual Uncertainty

For a hypothetical human clearance prediction, suppose the uncertain log-scale prediction has center \(\log(60\text{ L/h})\) and parameter standard deviation 0.250. Suppose a future individual’s log clearance also has independent residual variation with standard deviation 0.200. Assume both components are normal, centered as specified, and independent. Compare a plug-in value, a parameter-only interval, and a 90% predictive interval for an individual.

At the central parameter value, the plug-in prediction is \(60\) L/h. A parameter-only 90% interval is \(\exp\{\log(60)\pm1.645(0.250)\}\) L/h, approximately 40.0 to 90.2 L/h. This interval varies parameters but does not include the stated individual residual variation.

For an individual prediction, independence and normality imply that the two log-scale variances add: \(0.250^2+0.200^2=0.1025\). The total log-scale standard deviation is \(\sqrt{0.1025}=0.3202\). Therefore, the 90% predictive interval is:

\begin{aligned} \exp\{\log(60)\pm1.645(0.3202)\} &=\exp\{\log(60)\pm0.5267\}\\ &\approx (35.4,\ 101.6)\text{ L/h}. \end{aligned}

For comparison, omitting parameter uncertainty and using only residual variation would give \(\exp\{\log(60)\pm1.645(0.200)\}\), or approximately 43.2 to 83.4 L/h. The difference illustrates how parameter uncertainty widens the predictive distribution here. The calculation depends on the stated independence and normality assumptions; a simulation could instead draw from supported non-normal or correlated distributions.

Assumptions, Diagnostics, and Alternatives

An uncertainty interval is only as credible as the model and uncertainty representation that produce it. Check that the scaling relation is appropriate for the species, dose range, and quantity being predicted; that the fitted parameters are identifiable enough for the intended extrapolation; and that the uncertainty distribution reflects the available evidence. A narrow interval can be misleading if it conditions on a fixed model structure while plausible alternative structures or model discrepancy are omitted.

For a simulation or bootstrap analysis, inspect the distribution of predictions, not just its endpoints. Look for skewness, extreme tails, multimodality, failed model fits, and predictions driven by a small number of extreme parameter draws. Check convergence or numerical failures, repeat the simulation with more draws to assess Monte Carlo stability, and verify that correlated inputs are sampled jointly. For a local variance approximation, compare with simulation when feasible, particularly when \(g\) is strongly nonlinear or parameter uncertainty is large.

The main alternatives serve different purposes. A plug-in prediction is concise and useful as a reference, but does not quantify parameter uncertainty. A delta-method approximation is fast and transparent near the estimate, but can be inaccurate for nonlinear or non-normal prediction distributions. A bootstrap uses resampled evidence and refitting, but relies on a suitable resampling design and may not represent structural uncertainty or future individual variation. Parametric simulation can propagate several specified components and preserve their dependence, but its conclusions are conditional on the distributions and model assumptions supplied. None of these methods automatically covers uncertainty that has been left out.

Key takeaway: Report a plug-in prediction as a central model output, not as a complete description of translational risk. State what uncertainty the accompanying interval includes, how it was propagated, and what important sources remain outside it.

Common Pitfalls

Calling every interval a human-outcome prediction interval. An interval based only on fitted-parameter uncertainty does not include individual variability unless that component was explicitly incorporated.
Assuming uncertainty always shifts the prediction upward. Curvature can shift a mean in either direction, and the plug-in value is not guaranteed to be low. The shift depends on the function and uncertainty distribution.
Ignoring dependence among parameters. Covariance terms can materially change the variance of the translated prediction. Use a joint covariance estimate or jointly sampled parameter sets when warranted.
Treating a large number of simulations as evidence of a sound model. More draws reduce Monte Carlo noise, not structural misspecification, biased data, or unsupported extrapolation.
Confusing an interval with a guarantee. A model-based 90% prediction interval is not a promise that every future human value will fall inside it; its interpretation depends on the model and assumptions being adequate.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A model has uncertain parameters \(\theta\), but a report gives only \(g(\hat{\theta})\). What important information is missing?
If the parameter uncertainty is not centered at \(\hat{\theta}\), what term must be retained in the second-order approximation to \(E\{g(\theta)\}\)?
Why might an individual-level predictive interval be wider than a parameter-only interval?
In a bootstrap analysis, why should correlated parameters generally be refitted and propagated jointly rather than sampled independently?
A simulation interval is unchanged when the number of draws is increased, but model diagnostics show systematic prediction errors across species. What does the stable interval establish, and what does it not establish?

Solutions. (1) The point prediction omits the distribution or interval of predictions across plausible parameter values; it also does not show any other predictive variation that may matter. (2) Retain the first-order term \(\nabla g(\hat{\theta})^T(\mu-\hat{\theta})\), and in the second-order term use \(\Sigma+(\mu-\hat{\theta})(\mu-\hat{\theta})^T\). (3) It may include individual residual or between-individual variation in addition to parameter uncertainty. (4) Independent sampling discards the covariance structure, which can alter the distribution of the model output. (5) Stability supports that Monte Carlo sampling noise is small for that calculation; it does not establish that the model is adequate or that the interval is calibrated to real human outcomes.