Why the Estimation Target Matters
A population PK model connects concentration observations to population parameters while allowing individual parameters to vary. The same dataset and structural model can produce different estimates if the analysis targets a likelihood maximum, a penalized likelihood maximum, or a Bayesian posterior. The differences are not merely software settings: each target answers a different mathematical question about the data, parameters, and assumptions.
For example, a penalty can pull an estimate toward a chosen reference value, while a Bayesian analysis uses prior distributions to form a distribution for unknown quantities. A maximum likelihood analysis instead chooses the parameter values under which the observed data have the greatest likelihood. To compare these targets correctly, first specify how the population model defines the likelihood.
Likelihood in a Nonlinear Mixed-Effects PK Model
Let \(y_i\) denote the concentration observations for subject \(i\), collected at that subject’s sampling times and on the model’s specified observation scale. Let \(\theta\) denote population fixed effects, \(\eta_i\) the subject-specific random effects, \(\Omega\) their covariance matrix, and \(\Sigma\) the residual-error parameters. A nonlinear mixed-effects model specifies a conditional data density \(p(y_i\mid\eta_i,\theta,\Sigma)\) and a random-effects density \(p(\eta_i\mid\Omega)\). A common assumption is that subjects are independent conditional on the population parameters and that the random effects are independently drawn from a mean-zero multivariate normal distribution with covariance \(\Omega\). The conditional data model must also specify its residual-error distribution and dependence structure.
The marginal density for subject \(i\) is obtained by averaging the conditional data density over the random-effects distribution. This integration accounts for the fact that \(\eta_i\) is unobserved rather than a known subject-specific input. The population likelihood is the product of these subject-level marginal densities.
The integral is generally not available in closed form for a nonlinear PK model. Estimation methods therefore evaluate or approximate it numerically. Maximum likelihood estimation (ML) chooses the population parameters that maximize the marginal likelihood, or equivalently its logarithm. It does not maximize a separate conditional likelihood at each subject’s most favorable random-effect value.
Worked Example: Integrating a Random Effect
Worked Example: Integrating a Random Effect
Suppose a simplified hypothetical log-concentration observation follows \(y_i=\theta+\eta_i+\epsilon_i\), with \(\eta_i\sim N(0,0.09)\) and \(\epsilon_i\sim N(0,0.04)\), independently. Consider one hypothetical observation with \(y_i-\theta=0.40\) log-concentration units. Find its marginal density under the specified model.
Because the sum of independent normal variables is normal, \(\eta_i+\epsilon_i\sim N(0,0.09+0.04)=N(0,0.13)\). Thus the marginal distribution of \(y_i-\theta\) is normal with variance \(0.13\), and its density at \(0.40\) is
Numerically, \(1/\sqrt{0.8168}\approx1.106\), and \(\exp(-0.6154)\approx0.5404\). Evaluating the density using the unrounded inputs gives approximately \(0.5980\) per log-concentration unit. This is a density, not the probability that the observation equals exactly \(0.40\); probabilities require integrating density over an interval.
The random-effect variance contributes to the marginal variance even though \(\eta_i\) was not observed. Omitting that integration and treating the random effect as known would represent a different likelihood target.
Penalized Likelihood and Bayesian Targets
A penalized likelihood modifies the log likelihood by subtracting a penalty function \(J(\psi)\), where \(\psi\) denotes whichever parameters are penalized. With tuning weight \(\lambda\geq0\), the target is \(\ell(\psi)-\lambda J(\psi)\). The penalty may discourage extreme parameter values, complex models, or undesirable combinations. Its form and scale are modeling choices; they must be reported because they affect the estimate. If \(J\) is constant in the parameters, the penalty does not change the maximizer.
A Bayesian analysis specifies a prior density \(p(\psi)\) for unknown parameters and combines it with the likelihood. The posterior density is proportional to the likelihood times the prior. A maximum a posteriori estimate (MAP) maximizes the posterior density, equivalently the log likelihood plus the log prior. A Bayesian analysis, however, usually targets the full posterior distribution, not only its mode. Posterior means, intervals, and probabilities summarize that distribution and need not equal summaries based on a frequentist likelihood.
A penalty is mathematically equivalent to a log prior for the purpose of finding a mode only when the penalty is proportional to the negative log of a specified prior density, with the corresponding scaling. That equivalence does not make every penalized analysis a Bayesian analysis. Bayesian inference requires a probability model for the prior and uses the resulting posterior distribution; a penalty may be adopted only as an optimization device. Prior normalizing constants that do not depend on the parameters do not change a MAP estimate, but they can matter for quantities such as marginal likelihoods used in model comparison.
In a mixed-effects model, the random-effects distribution is already part of the population likelihood. It is not the same thing as a prior on all population parameters. A Bayesian version may additionally assign priors to \(\theta\), \(\Omega\), and \(\Sigma\), and may infer the subject effects jointly. The analysis must state which quantities are assigned priors and whether the subject effects are integrated out or retained as unknowns.
Worked Example: A Prior Changes the Target
Worked Example: A Prior Changes the Target
Suppose a hypothetical PK analysis yields a local normal likelihood for a dimensionless log-effect \(\theta\), centered at \(2.00\) with variance \(0.25\). This is a simplified likelihood approximation for illustrating the targets, not a claim that every nonlinear PK likelihood is exactly normal. The log likelihood, up to a constant, is
The ML estimate is the likelihood center, \(\hat{\theta}_{ML}=2.00\). Now apply the quadratic penalty \(J(\theta)=\theta^2/2\) with \(\lambda=1\). Differentiating the penalized target and setting the derivative to zero gives
The penalty pulls the estimate toward zero, while the data likelihood is unchanged. If instead \(\theta\sim N(0,1)\) is specified as a prior, its log density contributes \(-\theta^2/2\) plus a constant, so the MAP estimate is also \(1.60\). In this normal example, the posterior variance is \((1/0.25+1/1)^{-1}=0.20\), and its standard deviation is \(\sqrt{0.20}=0.4472\). A normal-approximation 95% posterior interval is
Under the stated prior and likelihood approximation, this interval summarizes the posterior uncertainty in the dimensionless log-effect. It is not a 95% frequentist confidence interval, nor does it imply a 95% probability for a fixed parameter under a frequentist interpretation. The matching penalized and MAP point estimates here arise from the particular quadratic penalty and normal prior; the inferential summaries are not thereby interchangeable.
Conditional Subject Estimates Are Not the Marginal Target
For fixed population parameters, a conditional estimate of \(\eta_i\) uses the subject’s data together with the random-effects distribution. A conditional mode maximizes \(p(y_i\mid\eta_i,\theta,\Sigma)p(\eta_i\mid\Omega)\) over \(\eta_i\). This is a subject-level optimization. It differs from the population ML target, which integrates over \(\eta_i\) before optimizing population parameters. A conditional mode can be useful for describing or predicting individual effects, but substituting the mode into the likelihood generally does not equal evaluating the marginal likelihood.
Worked Example: Conditional Mode Versus Marginal Density
Consider one hypothetical log-concentration deviation \(y=1.00\) from a fixed population prediction. Let \(y=\eta+\epsilon\), with \(\eta\sim N(0,0.25)\) and \(\epsilon\sim N(0,0.25)\), independently. Find the conditional mode of \(\eta\) and the marginal density of \(y\).
The conditional log density, omitting constants, is \(-[(1-\eta)^2+\eta^2]/(2(0.25))\). Maximizing it is equivalent to minimizing \((1-\eta)^2+\eta^2\). Differentiation gives \(-2(1-\eta)+2\eta=0\), so the conditional mode is \(\hat{\eta}=0.50\). The conditional posterior distribution is normal with variance \((1/0.25+1/0.25)^{-1}=0.125\); this reflects both the observation and the random-effects distribution.
For the marginal density, integrate over \(\eta\). The sum \(\eta+\epsilon\) has variance \(0.25+0.25=0.50\), so \(y\sim N(0,0.50)\). The density at \(y=1.00\) is
per log-concentration unit. The conditional mode \(0.50\) is an estimate of the subject effect; \(0.2076\) is the marginal density contribution of the observation under the population model. They are different quantities with different roles.
Choosing, Checking, and Interpreting a Target
The target should match the inferential purpose. Marginal ML is a likelihood-based approach for estimating population parameters under the specified model. Penalized likelihood can stabilize an optimization or express a deliberate preference for constrained or less complex estimates, but the penalty must be justified and its influence assessed. Bayesian analysis makes prior information explicit and yields a posterior distribution, with conclusions potentially sensitive to the prior when the data are weakly informative.
| Target | What is optimized or summarized | Important qualification |
|---|---|---|
| Marginal ML | Marginal likelihood after integrating subject random effects | Depends on the data model, random-effects distribution, and numerical integration |
| Penalized likelihood | Log likelihood minus a stated penalty | Penalty form and weight change the point estimate; it need not define a probability distribution |
| Bayesian | Posterior distribution; MAP is one possible point summary | Requires priors, and posterior conclusions can depend on them |
Diagnostics should investigate both the optimizer and the target’s assumptions. For likelihood-based fits, examine convergence messages, objective stability across reasonable starting values, parameter boundaries, and whether numerical integration is sufficiently accurate. An apparently converged objective does not establish that the model is adequate or that a maximum is global. Check whether estimates or objective values change materially when integration accuracy is increased. For penalized fits, also assess sensitivity to the penalty form and weight, and report the unpenalized likelihood separately when useful. For Bayesian fits, assess prior sensitivity and use appropriate computational diagnostics for the posterior sampler or approximation; convergence of an optimizer alone is not a posterior diagnostic.
All three targets rely on a defensible structural and observation model. A misspecified residual-error distribution, an inadequate random-effects distribution, or an unsuitable PK structural model can distort estimates regardless of the optimization target. The marginal likelihood may be difficult to compute in nonlinear models, and approximate integration can affect the objective. A penalty can conceal weak identification by making an estimate look stable, while a prior can contribute substantial information where the likelihood is flat. Neither stability nor a narrow interval alone demonstrates that the data identify a parameter.
Common Mistake: Calling Every Shrinkage Term a Prior
A frequent error is to say that any term pulling an estimate toward zero is “Bayesian” or that a penalized estimate is automatically a posterior mean. The correction is to identify the actual objective. A stated penalty changes the maximized criterion; a specified prior contributes to a posterior distribution, whose mode may coincide with the penalized estimate under particular conditions. Even then, the posterior mean and interval generally require more than the mode. In a mixed-effects model, also distinguish the random-effects distribution used to define the population likelihood from priors placed on population parameters.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- In a nonlinear mixed-effects model, why does the marginal likelihood integrate over \(\eta_i\) rather than simply treating its conditional mode as observed?
- If a quadratic penalty is exactly proportional to the negative log of a normal prior, which point estimates can coincide, and which Bayesian summaries are not thereby determined?
- For the normal example with likelihood center \(2.00\), likelihood variance \(0.25\), and prior \(N(0,1)\), what are the posterior mean and posterior standard deviation?
- What does the density \(0.2076\) in the conditional-mode example represent, and what does the value \(0.50\) represent?
- Name two checks particularly relevant to numerical likelihood estimation and one check relevant to a Bayesian target.
Solutions. (1) The random effect is unobserved; the marginal likelihood averages the conditional data density over its specified population distribution, whereas a mode substitutes one value and generally gives a different criterion. (2) The penalized maximum and MAP can coincide; the posterior mean, interval, and other distributional summaries require the posterior distribution and are not specified by the mode alone. (3) The posterior mean is \(1.60\), and its standard deviation is \(\sqrt{0.20}=0.4472\). (4) The density is the marginal density of the observation under the model, per log-concentration unit; \(0.50\) is the conditional mode of the subject random effect. (5) Check objective stability across starting values and sensitivity to integration accuracy; for a Bayesian target, assess prior sensitivity and computational convergence diagnostics.