Nonfinite Individual Likelihood Contributions

Why Inspect Individual Contributions?

An optimizer usually reports one objective value for the whole data set. That value can reveal a failure, but it may not reveal which subject or calculation caused it. A single NaN or infinity can make the total objective unusable, while a failure in one subject can be obscured by looking only at a summary or optimizer message. The practical task is therefore to find the first nonfinite value in the calculation for each affected subject.

In a pharmacometric model, an individual contribution can depend on a predicted concentration, an observation variance, a distribution’s support, and any transformations used to calculate a density or probability. The same outward symptom—such as a NaN objective—can arise from very different sources. Tracing the calculation in order helps distinguish an invalid model input or parameter combination from a numerical evaluation problem.

Definition: Let \(Y_i\) denote the observations for subject \(i\), and let \(\theta\) denote the model parameters. The individual likelihood contribution \(L_i(\theta)\) is the probability mass or density assigned to that subject’s observed data by the model. Its individual log-likelihood contribution is \(\ell_i(\theta)=\log L_i(\theta)\), when this logarithm is defined. If subjects are independent under the model, the total log likelihood is \(\ell(\theta)=\sum_{i=1}^{N}\ell_i(\theta)\).

A subject contribution may be a joint contribution for several observations, or a marginal contribution that accounts for subject-level random effects. The diagnostic principle is the same: record the intermediate quantities used to evaluate that contribution. The sum formula requires independence across subjects under the model; it does not imply that repeated observations within one subject are independent.

For a continuous observation, \(L_i\) is a density value, not the probability of observing exactly that value. A density can exceed 1, so a positive log-density is not automatically an error. A density with respect to concentration measured in mg/L is expressed per mg/L; changing the concentration unit changes its numerical value. Accordingly, report a log-density as a numerical log-density, without attaching concentration units to it.

A Reference Calculation: Normal Observation Error

Suppose one observation \(y_i\), conditional on the model prediction \(f_i\), has a normal distribution with variance \(v_i\):

Y_i\mid\theta\sim N(f_i,v_i),\qquad v_i>0.

The assumptions for this calculation are that \(f_i\) is finite, \(v_i\) is finite and strictly positive, and the observation is in the domain of the normal model. The normal density is

p(y_i\mid\theta)=\frac{1}{\sqrt{2\pi v_i}} \exp\left(-\frac{(y_i-f_i)^2}{2v_i}\right).

Taking the logarithm of the positive density gives the individual log-likelihood contribution:

\ell_i(\theta)=-\frac{1}{2}\left[\log(2\pi v_i)+\frac{(y_i-f_i)^2}{v_i}\right].

This form exposes several checks: the variance must be positive before the logarithm or division is evaluated; the prediction and observation must be finite; and the standardized residual calculation must remain representable. The formula also helps locate a failure: a nonfinite logarithm of the variance points to a different issue from a nonfinite squared residual.

Worked Example: Checking a Finite Normal Contribution

For a hypothetical concentration observation, let \(y_i=12\) mg/L, let the model prediction be \(f_i=10\) mg/L, and let the residual standard deviation be \(2\) mg/L. Thus \(v_i=(2\text{ mg/L})^2=4\text{ (mg/L)}^2\). All inputs are finite and \(v_i>0\), so the normal log-density calculation is defined.

Substitute into the formula:

\ell_i=-\frac{1}{2}\left[\log(2\pi\cdot4)+\frac{(12-10)^2}{4}\right] =-\frac{1}{2}\left[\log(25.1327)+1\right].

Since \(\log(25.1327)\approx3.2242\), the arithmetic is

\ell_i\approx-\frac{1}{2}(3.2242+1)=-2.112.

The log-density is approximately \(-2.112\), with no concentration unit attached. The corresponding density is approximately \(\exp(-2.112)=0.1210\) per mg/L. The prediction, variance, and resulting contribution are all finite, so this calculation does not show the nonfinite-contribution failure being investigated.

Trace the First Invalid Quantity

A useful diagnostic is to evaluate a suspect subject’s contribution in the same order as the model. Record the input data, parameter values, model prediction, variance or other distribution parameters, transformed arguments, and final contribution. The first nonfinite or out-of-domain intermediate is usually more informative than the final NaN. When possible, repeat the evaluation outside the optimizer at a fixed parameter vector, so the calculation can be examined without changing parameters between diagnostic steps.

Identify the subject and evaluation point.
Record the subject identifier, observations, covariates, doses, and the exact parameter values at which the contribution becomes nonfinite.

Check model outputs first.
Verify that predictions and any integrated states are finite and plausible at the observation times. If a prediction is already NaN or infinite, do not begin by debugging the final density formula.

Check distribution parameters and domain.
Confirm required conditions such as positive variance, valid probabilities, and finite degrees of freedom or shape parameters, as applicable to the chosen distribution.

Check transformations and the contribution itself.
Inspect logarithms, divisions, powers, and density or mass calculations. Record the first operation that produces an invalid value, not only the final objective.

Compare subject-level and total results.
Confirm whether a particular \(\ell_i\) is nonfinite or whether finite contributions become nonfinite only when aggregated. Keep the per-subject record with the diagnostic run.

A NaN often signals an undefined operation, such as taking a logarithm of an invalid argument, or a nonfinite input propagated through later arithmetic. Positive infinity can arise from overflow or an invalid limiting calculation; negative infinity can arise when an evaluated contribution is driven beyond the representable range. These signs are clues, not diagnoses: inspect the intermediate values and their domains.

Worked Example: A Nonfinite Prediction Reaches the Likelihood

Consider a hypothetical subject with an observed concentration of \(8\) mg/L. The observation model is normal, with a valid residual variance \(v_i=1\text{ (mg/L)}^2\), but the model prediction returned by an upstream calculation is NaN. The density formula requires a finite prediction. Substitution therefore yields

\ell_i=-\frac{1}{2}\left[\log(2\pi\cdot1)+\frac{(8-\mathrm{NaN})^2}{1}\right]=\mathrm{NaN}.

The variance is finite and positive, so it is not the first failure. The residual \(8-f_i\) is already invalid because \(f_i\) is NaN. The appropriate diagnostic is to trace how the prediction was produced: for example, check the model output at the observation time and the status of any numerical calculation feeding it. Replacing the NaN with an arbitrary concentration would hide the source rather than repair it.

Invalid Variances and Distribution Domains

A variance in a normal observation model must be strictly positive. A parameterization that does not enforce this condition can return a negative value at some parameter settings, even if it works at others. For example, a variance expression \(v_i=\sigma^2(1+\beta x_i)\) is valid only when \(\sigma^2>0\) and \(1+\beta x_i>0\). The condition must hold for every subject at every parameter vector evaluated by the optimizer, not merely at the initial estimate.

Worked Example: A Negative Observation Variance

Suppose a hypothetical model uses \(v_i=\sigma^2(1+\beta x_i)\), with \(\sigma^2=0.25\text{ (mg/L)}^2\), \(\beta=0.8\), and a subject covariate \(x_i=-2\). The factor in parentheses is

1+\beta x_i=1+(0.8)(-2)=-0.6, \qquad v_i=0.25(-0.6)=-0.15\text{ (mg/L)}^2.

This violates the normal-model condition \(v_i>0\). In the log-density, \(\log(2\pi v_i)\) is not a real-valued logarithm, and the variance is also used as a divisor. The resulting NaN or other nonfinite value is therefore downstream of an invalid variance, not evidence that the observation itself is impossible. The modeler should check the variance parameterization and its constraints over the observed covariate range; silently replacing the negative variance by its absolute value would define a different model.

Other distributions have their own domains. A binomial probability must be between 0 and 1, inclusive, and its trial and event counts must be valid integers with \(0\leq k_i\leq n_i\). A calculation that violates these conditions is not a valid binomial likelihood evaluation. Check the distribution’s support and parameter constraints before attributing a nonfinite result to the optimizer.

Worked Example: An Invalid Binomial Probability

Suppose a hypothetical adverse-event model has \(n_i=10\) trials and \(k_i=3\) observed events. A probability calculation mistakenly returns \(p_i=1.2\), outside the binomial domain. The log likelihood expression, when its arguments are valid, is

\ell_i=\log {n_i\choose k_i}+k_i\log(p_i)+(n_i-k_i)\log(1-p_i).

Here, \(\log(1-p_i)=\log(-0.2)\) is not real-valued. Numerically, that operation produces an invalid value, so the individual contribution is NaN. The observed counts themselves satisfy \(0\leq3\leq10\); the immediate failure is the calculated probability. Trace its construction and confirm that the model uses an appropriate probability transformation and parameter range. Do not interpret the resulting NaN as a meaningful likelihood value.

Distinguishing Failure Modes and Choosing a Response

The most useful distinction is between an invalid quantity and a valid quantity that is poorly represented numerically. A negative variance or probability above 1 violates a distribution condition. A NaN prediction may come from an earlier invalid model operation. By contrast, very large or small but finite intermediate values may indicate loss of numerical range; inspect them before deciding whether the underlying mathematical expression is itself invalid. Earlier material on overflow and underflow and numerical integration error provides further background on those numerical mechanisms.

There are several practical checks that help prevent misdiagnosis:

Check every subject, not only the first one. A covariate value or observation pattern can make a distribution parameter invalid for only a subset of subjects.
Preserve the first failure. If a prediction is already nonfinite, later density calculations add little information until the prediction path is understood.
Separate evaluation from aggregation. A finite set of contributions can still produce a nonfinite total through arithmetic or representation issues; inspect individual values and the sum.
Do not confuse a positive log-density with a failure. Continuous densities can exceed 1. Check finiteness and model conditions, not the sign alone.
Do not patch invalid values silently. Clipping a probability, forcing a variance to be positive, or substituting a finite value changes or obscures the calculation unless that operation is explicitly part of the statistical model.

An alternative to inspecting subject-level contributions is to rely on the optimizer’s final status or the total objective. Those summaries are useful for detecting that something went wrong, but they usually do not identify the first invalid calculation. Likewise, inspecting only residuals may miss a distribution-parameter problem. A subject-level trace is more direct; a model-wide parameter and input audit is complementary because the same invalid parameterization may affect several subjects.

Key takeaway: A nonfinite individual contribution is a diagnostic signal, not an explanation. Locate the affected subject, evaluate its calculation in order, and identify the first prediction, distribution parameter, transformation, or arithmetic operation that fails its required condition.

Check Your Understanding

Answer each question, then compare with the solutions.

For a normal observation model, what two conditions on the prediction and variance should be checked before evaluating the log-density?
A normal-model calculation has a finite prediction but returns variance \(-0.04\text{ (mg/L)}^2\). What is the first model condition violated, and why is changing the sign of the variance not a neutral fix?
A binomial subject has valid counts \(k_i=2\) and \(n_i=8\), but the model returns \(p_i=1.05\). Identify the immediate invalid calculation in the log likelihood.
One subject’s contribution is NaN, and the total objective is NaN. What should be inspected first: the optimizer’s final message or the subject’s intermediate calculations? Explain.
A normal log-density is positive but finite. Does that alone establish a failure? Why or why not?

Solutions. (1) The prediction must be finite, and the variance must be finite and strictly positive. (2) The condition \(v_i>0\) is violated. Replacing the variance by its absolute value changes the specified variance calculation and hence the model; it does not diagnose or justify that change. (3) The binomial probability is outside its valid range. In particular, \(\log(1-p_i)=\log(-0.05)\) is not real-valued, so the contribution is invalid. (4) Inspect the affected subject’s intermediate calculations first, in order, to find the earliest failure; the optimizer message reports the symptom rather than its source. (5) No. A continuous density can exceed 1, giving a positive finite log-density. Check finiteness and the model’s distribution conditions rather than the sign of the log-density alone.