Handling Failed Parameter Draws

Why Failed Draws Need Explicit Treatment

When propagating parameter uncertainty through a fitted NLME model, each parameter draw is an input to a model evaluation. A draw may produce a valid concentration-time profile and derived outcome, or the model evaluation may fail: a parameter may be outside the model's mathematical domain, an equation may be undefined, or a numerical solver may not return a finite solution. These outcomes are not interchangeable. In particular, a failed evaluation is not automatically evidence that the parameter draw itself should be removed.

The central requirement is to preserve the meaning of the simulation. A summary over only successful evaluations describes outcomes conditional on successful evaluation. If success depends on the sampled parameters, this summary may differ systematically from the intended uncertainty summary. The failure count, reasons, and any resolution therefore belong in the simulation record, not just in a software log.

Define What Counts as Failure

Before simulation, specify the parameters, covariates, dose, time points, model equations, and output being evaluated. Then define failure in relation to that target. Examples include an undefined expression, a solver reporting that it did not converge, or an output containing NaN or infinity where a finite value is required. An output that is surprising, extreme, or outside a preferred clinical range is not necessarily a computational failure. It may be a valid implication of the model and draw, or it may signal a model limitation that should be investigated separately.

A useful first distinction is between domain failure and numerical failure. Domain failure occurs when a draw violates a mathematical or scientific constraint required by the model, such as a negative clearance in a model that requires positive clearance. Numerical failure occurs when the mathematical problem is defined, but the computational procedure does not successfully evaluate it—for example, a solver reaches its step limit for a stiff differential equation. A third category is an output validation failure: the software returns values, but they violate a prespecified mathematical check, such as a concentration that must be nonnegative under the model.

The classification should be based on the model and implementation, not on whether the result is convenient. For example, an unusually high but finite exposure is not a failed draw merely because it is inconvenient for a summary. Conversely, replacing a non-finite value with zero does not make the evaluation successful; it changes the output without a model-based justification.

Assumptions and Failure Accounting

A failure rate is meaningful only relative to a defined set of attempted draws and a consistent evaluation procedure. Let \(F\) be the number of draws that remain failed after any prespecified diagnostic retries. The empirical failure fraction is

The denominator is the number of distinct parameter draws attempted, not the number of successful solver calls. If a draw is retried with a more appropriate solver, it remains one parameter draw. Report both the initial and final failure counts if retries are used, so that the retry policy and its effect are visible.

This accounting assumes that the target parameter-draw distribution and the evaluation rules are stated clearly. If some draws lie outside the model's valid parameter space, the analyst must decide how the uncertainty distribution should represent that space. The decision is part of defining the simulation, not a post hoc way to improve the results. The same applies to a model that is mathematically defined for a draw but has a scientifically implausible implication: that concern should be reported and assessed rather than silently recoded as a numerical failure.

Why Dropping Failures Can Change the Summary

Suppose an outcome \(Y\) is defined for every draw in the intended target distribution, but the computation succeeds only for some parameter values. Let \(S\) denote the event that evaluation succeeds. Averaging only successful evaluations estimates an average conditional on \(S\), not the unconditional target average. The distinction follows directly from the definition of conditional expectation:

where \(I(S)\) equals 1 when evaluation succeeds and 0 otherwise. If success is related to \(Y\) or to parameters associated with \(Y\), then \(E(Y\mid S)\) need not equal \(E(Y)\). Even when the success fraction is high, a small set of failures can matter if those draws correspond to extreme outcomes. If the model genuinely has no defined output for part of the draw distribution, an unconditional mean may not be defined as intended; the target and parameter distribution need reconsideration.

This is why replacing failed draws by new draws until a fixed number of successful evaluations is obtained is not automatically harmless. It samples from the original draw distribution conditional on evaluation success. That conditional distribution can differ from the one originally specified. Redrawing is defensible only if the target distribution is explicitly defined as restricted to an acceptable parameter region, and the restriction is justified and reported.

Constraints, Transformations, and Numerical Remedies

A constraint should reflect the parameterization and scientific or mathematical domain of the model. Parameters that must be positive are often represented on an unconstrained scale and transformed back to the natural scale. For example, if \(\eta\) is a draw on a log scale, then \(CL=\exp(\eta)\) is positive for every finite \(\eta\). This prevents negative clearance values caused by drawing directly from an unconstrained normal approximation on the clearance scale. It does not, however, establish that the log-scale uncertainty distribution is the correct one; the scale and distribution must agree with the fitted model and the intended uncertainty representation.

For parameters with other constraints, a suitable transformation or a constrained distribution may be needed. For covariance matrices, for example, draws must satisfy the matrix properties required by the model. A repair such as changing negative eigenvalues after a draw can alter both marginal uncertainty and dependence. It should not be applied as an undocumented numerical fix. A constrained or reparameterized approach should be specified as part of the draw procedure.

When the draw is in the model's domain, investigate numerical failure using the error message and the parameter values. Check units, initial conditions, time grid, tolerances, and solver settings. A solver appropriate for a stiff system may resolve a failure that a general-purpose method cannot. Any retry should use a defined procedure and should be checked for numerical stability—for example, by confirming that important outputs are materially unchanged under a tighter tolerance. A retry can resolve a computational failure; it cannot make an invalid parameter draw scientifically valid.

Diagnostics, Alternatives, and Limitations

A useful diagnostic sequence is to inspect failures before summarizing outcomes. Group them by failure category and parameter region; examine whether they cluster near a constraint boundary or occur at extreme parameter combinations; and check whether the same draw fails reproducibly. A reproducible domain violation suggests a draw-distribution or model-domain problem. A failure that disappears with an appropriate solver or a tighter, stable evaluation procedure suggests numerical difficulty. Unexplained non-finite results require investigation of the equations and implementation rather than automatic replacement.

The main alternatives have different implications. Retaining only successful draws is simple, but the resulting summaries describe the successful subset and can be biased for the intended target. Redrawing until success likewise changes the sampled distribution when success depends on parameters. Transforming or constraining parameters before evaluation can prevent invalid draws, but it defines a particular draw distribution and must be justified. Improving the numerical method can resolve computational failures without changing the target distribution, provided the model is defined and the evaluation is verified. None of these approaches justifies hiding unresolved failures.

Failure accounting also has limits. A low failure fraction does not prove that the successful draws represent the intended distribution, and a high fraction is not automatically evidence that every failed draw is scientifically impossible. Software may report failure for a solvable problem, while a numerical result may appear finite despite being inaccurate. The distinction requires checking the model, parameter support, and numerical evaluation. Report the unresolved fraction and avoid presenting a successful-draw summary as an unconditional summary when its interpretation is conditional.

Common Mistake

A common mistake is: “These draws failed, so they are outliers; I will delete them and calculate the summaries from the rest.” That reasoning confuses evaluation status with scientific relevance. A failed draw may be invalid under the model, or the evaluation may have failed for computational reasons; the label alone does not decide which. The correction is to classify the failure, apply a prespecified and justified remedy where possible, and report unresolved failures using the number of distinct attempted draws as the denominator. If a summary excludes unresolved draws, state that it describes successful evaluations and assess whether that conditioning changes its interpretation.