Likelihood Comparisons for NLME Models

What a Maximized Likelihood Summarizes

An NLME model describes observations through both population parameters and subject-level random effects. Comparing two fitted models often begins with their likelihoods: how well, at their respective fitted parameter values, do they describe the observations that were actually recorded? A likelihood comparison is meaningful only when both fits refer to the same observations and use the same definition of likelihood.

For subject \(i\), let \(y_i\) denote the vector of observed responses at that subject’s recorded times and let \(\eta_i\) denote the subject-specific random effects. A typical model specifies the response distribution conditional on \(\eta_i\), as well as a population distribution for \(\eta_i\). The marginal subject likelihood integrates over the unobserved random effects. If subjects are independent, the likelihood for all \(N\) subjects is the product of those marginal contributions.

Definition: For a specified NLME model and a specified set of observations, the likelihood \(L(\theta)\) is the joint probability mass or density of those observations, viewed as a function of the model parameters \(\theta\). The maximized likelihood is \(L(\hat{\theta})\), where \(\hat{\theta}\) is a parameter value that maximizes \(L(\theta)\) over the model’s permitted parameter space. Its logarithm, \(\ell(\hat{\theta})=\log L(\hat{\theta})\), is the maximized log-likelihood.

In the usual marginal likelihood for independent subjects, the contribution of subject \(i\) is

L_i(\theta)=\int p(y_i\mid \eta_i,\theta)\,p(\eta_i\mid\theta)\,d\eta_i, \qquad L(\theta)=\prod_{i=1}^{N}L_i(\theta), \qquad \ell(\theta)=\sum_{i=1}^{N}\log L_i(\theta).

Here \(p(y_i\mid\eta_i,\theta)\) describes the subject’s observations conditional on the random effects, while \(p(\eta_i\mid\theta)\) describes the random-effects distribution. The integral accounts for uncertainty about the unobserved \(\eta_i\); it is not generally the same as evaluating the conditional response density at an estimated subject effect. The formula assumes the chosen response and random-effects distributions are appropriate, subjects are independent under the model, and the observations included in each \(y_i\) are defined consistently.

For continuous responses, a likelihood is a density, not the probability of observing one exact numerical data vector. Its numerical value can depend on the units in which responses are expressed. Therefore, the practical comparison is between likelihoods calculated from the same recorded data representation under a common likelihood definition. Log-likelihoods are convenient because products become sums and differences are easier to interpret.

What a Likelihood Difference Means

Suppose two models, A and B, are fitted to the same observations with the same likelihood definition, and both optimization routines find their respective maxima. A larger maximized likelihood means that, at its fitted parameters, that model assigns a higher density to the observed data under that likelihood. The log-likelihood difference is

\Delta \ell=\ell_B(\hat{\theta}_B)-\ell_A(\hat{\theta}_A), \qquad \frac{L_B(\hat{\theta}_B)}{L_A(\hat{\theta}_A)}=\exp(\Delta\ell).

This ratio describes the relative maximized likelihoods for the observed data under the two specified models. It is not the probability that either model is true, nor the probability that the observed data would recur. A more flexible model may achieve a higher maximized likelihood simply because it has more ways to adapt to the sample. Likelihood alone does not balance fit against complexity or establish which model should be preferred for a scientific purpose.

Worked Example: Comparing Two Fits to the Same Records

Suppose a hypothetical concentration data set has 36 observations from 12 subjects. Two candidate NLME models are fitted using the same 36 response-time records, the same subject identifiers and covariates, and the same marginal likelihood calculation. Assume both optimizations converged successfully to their maxima. Model A has maximized log-likelihood \(-128.4\), and model B has maximized log-likelihood \(-125.7\).

The log-likelihood difference, using B minus A, is

\Delta\ell=(-125.7)-(-128.4)=2.7.

Thus the ratio of maximized likelihoods is

\frac{L_B(\hat{\theta}_B)}{L_A(\hat{\theta}_A)} =\exp(2.7)\approx 14.88.

For these observations and this common likelihood definition, model B’s maximized likelihood is about 14.9 times model A’s. This is a statement about relative fit at the maxima only. It does not mean there is a 14.9-fold probability that B is correct, and it does not account for model complexity. The comparison also depends on the stated conditions: if the records, likelihood approximation, or likelihood type differed, this calculation would not support the same interpretation.

Comparability Requires the Same Observations

The likelihood is a function of the observed data as well as the model. If two fits use different observations, their likelihoods summarize different data vectors. Their raw values, or their difference, do not provide a direct comparison of fit to one common data set. A model that omits difficult-to-fit records can have a higher likelihood for the remaining observations without fitting the full intended data better.

“Same observations” means more than the same number of rows. Check that the fits use the same subjects and the same response records, including the same times, outcomes, covariates, and treatment information. Also check how missing values, below-quantification observations, censoring, exclusions, and weights were handled. For longitudinal data, omitting one time point changes a subject’s response vector and hence that subject’s likelihood contribution.

Worked Example: A Higher Value From Fewer Records

Imagine a hypothetical data set with 40 concentration records from 10 subjects. Model A is fitted to all 40 records and has maximized log-likelihood \(-90.0\). Model B is fitted only to 32 records after eight low concentrations are excluded, and has maximized log-likelihood \(-70.0\).

The arithmetic difference is

(-70.0)-(-90.0)=20.0, \qquad \exp(20.0)\approx 4.852\times 10^8.

That numerical ratio is not a valid likelihood comparison between the two fits: its numerator is based on 32 records and its denominator on 40. The likelihood for model B is not the likelihood of the same observed data used for model A. The correct next step is to refit both models to the same defined set of 40 records, if the models can accommodate them, using the same handling of those observations. If exclusion is scientifically required, both models should be fitted to the same 32 records and the resulting comparison should be understood as applying to that reduced data set.

Comparability Requires the Same Likelihood Definition

Even with identical records, likelihood values can differ because the calculations target different quantities. A marginal likelihood integrates over random effects, as in the definition above. A conditional likelihood instead evaluates response densities conditional on specified random effects. These likelihoods answer different mathematical questions; a marginal likelihood value should not be directly contrasted with a conditional likelihood value as though both summarized the same data density.

Likewise, maximum likelihood (ML) and restricted maximum likelihood (REML) are distinct likelihood-based estimation criteria. ML is based on the likelihood of the observed responses under the model. REML uses a restricted likelihood constructed to account for estimation of fixed effects when estimating covariance parameters. In particular, REML likelihood values are generally not suitable for direct comparisons between models with different fixed-effects structures. A comparison should use a common estimation criterion and likelihood target appropriate to the models being compared. The later material on fixed-effect tests with ML and REML develops that issue further.

Other choices must also match. For example, a likelihood calculated with one approximation to the random-effects integral should not be treated as directly comparable to one calculated using a different approximation without justification. Similarly, implementations may differ in whether they retain constants in a log-likelihood. If the same constant is omitted from both fits, their difference is unchanged; if the constants differ, the reported difference can be shifted and misleading.

Worked Example: Identical Data but Different Likelihood Targets

Suppose two hypothetical outputs use the same 24 observations. Model A reports a marginal ML maximized log-likelihood of \(-52.6\), while model B reports a conditional log-likelihood of \(-50.1\). Subtracting the reported numbers gives

(-50.1)-(-52.6)=2.5, \qquad \exp(2.5)\approx 12.18.

The arithmetic is correct, but interpreting 12.18 as a relative likelihood is not justified: the two values arise from different likelihood targets. To make a defensible comparison, calculate both models using the same likelihood definition. For instance, if both marginal ML fits then yielded \(-52.6\) and \(-51.8\), respectively, their difference would be \(0.8\), and the ratio would be \(\exp(0.8)\approx 2.226\). That ratio would describe their relative maximized marginal likelihoods for the same observations, assuming the same approximation and consistent constants.

A Short Justification: Why a Shared Constant Cancels

Software may report a log-likelihood with a data-dependent constant removed. This is harmless for a difference only when the same constant is removed from both models. Let the full log-likelihood for model \(j\) be \(\ell_j\), and suppose both reported values omit the same constant \(C\), so that \(\ell_j^{*}=\ell_j-C\). Then

\ell_B^{*}-\ell_A^{*} =(\ell_B-C)-(\ell_A-C) =\ell_B-\ell_A.

The difference is preserved. If instead the software reports \(\ell_B^{*}=\ell_B-C_B\) and \(\ell_A^{*}=\ell_A-C_A\) with \(C_B\ne C_A\), then

\ell_B^{*}-\ell_A^{*} =(\ell_B-\ell_A)-(C_B-C_A),

so the reported difference is shifted. This is why matching the likelihood definition and reporting convention matters, not merely copying two numbers labeled “log-likelihood.”

Practical Checks and Common Pitfalls

Before interpreting a likelihood comparison, verify the fit itself and the basis of comparison. The following checks help distinguish a substantive difference from a data or computation mismatch.

Check the records. Confirm the same subjects and response-time records are included, with consistent covariates, censoring, missing-data rules, and weights.
Check the likelihood target. Confirm both values are marginal or both are conditional, and that both use the same ML or REML criterion when relevant.
Check computation. Confirm convergence, inspect parameter estimates and boundary behavior, and check that the optimizer did not stop at a poor local solution. Where appropriate, use different starting values to see whether the same maximum is recovered.
Check numerical method. Verify that the same integration or likelihood approximation and compatible software conventions were used. A small reported difference may be within numerical error.
Check interpretation. Remember that a higher maximized likelihood indicates better fit at the maximum on the stated scale, not a complexity-adjusted selection or a probability that the model is true.

A common mistake is to say, “Model B has a higher log-likelihood, so it is the better model.” The reasoning is incomplete. First establish that both fits concern the same observations and likelihood definition. Then report the comparison as evidence about relative maximized fit, while recognizing that complexity, scientific plausibility, diagnostics, and predictive performance may also matter. Information criteria and formal tests address different questions; the likelihood comparison itself should not be mistaken for either.

Comparison	What it summarizes	Key caution
Maximized likelihood	Fit to the observed data at the fitted parameter values	Requires the same data and likelihood definition; does not penalize complexity by itself
Residual and model diagnostics	Whether important patterns remain unexplained under a fitted model	A likelihood value alone does not reveal the form of a mismatch
Predictive evaluation	Performance for observations not used to fit the model	Answers a prediction question, not simply which model has the larger in-sample maximum
Information criteria	A fit-and-complexity summary under a specified criterion	Requires its own assumptions and careful calculation; it is not the raw likelihood difference

Key takeaway: A maximized likelihood summarizes how well a specified model fits a specified data set under a specified likelihood definition. Compare values only after verifying that the observations, likelihood target, computational method, and reporting convention are aligned.

Check Your Understanding

For each question, first decide what the likelihood values can support, then compare your reasoning with the solutions.

Two models use the same 50 records and the same marginal ML definition. Their maximized log-likelihoods are \(-200.0\) and \(-197.5\). What is the likelihood ratio for the second model relative to the first, and what does it mean?
Why is it not a valid direct comparison if one model is fitted to all observations and another excludes records below the quantification limit?
Two models use identical records, but one reported value is marginal ML and the other is conditional likelihood. Is exponentiating their log-likelihood difference a meaningful relative likelihood? Explain.
If the same constant is omitted from both reported log-likelihoods, does their difference change? Show why.
Name two computation or data checks to make before interpreting a small difference in maximized log-likelihood.

Solutions. (1) The difference is \((-197.5)-(-200.0)=2.5\), so the ratio is \(\exp(2.5)\approx 12.18\). For these records and this common likelihood definition, the second model’s maximized likelihood is about 12.18 times the first’s; this is not a probability that the second model is true. (2) The likelihoods are calculated for different observed data vectors, so their ratio does not compare fit to one common data set. Both models must use the same specified records and handling. (3) No. Marginal and conditional likelihoods are different targets, so the difference does not yield an interpretable relative likelihood. (4) No: if both values omit \(C\), then \((\ell_B-C)-(\ell_A-C)=\ell_B-\ell_A\). (5) Suitable checks include verifying identical included records and data handling, confirming the likelihood definitions and approximations match, checking convergence, and testing whether different starting values recover the same maximum.