Why Combine the Observation Model and the Random-Effect Distribution?
For an individual, the observations are informative about the individual’s random effects, while the population distribution constrains which random-effect values are plausible. The conditional likelihood calculation combines both sources of information. Its maximizer is a value of the random effects that best balances the individual’s data against the population distribution, with the population parameters held fixed.
This is different from evaluating the marginal likelihood, which averages over possible random effects by integration. The marginal likelihood as an integral is the relevant earlier material; here the focus is the individual-level density being integrated and the value of the random effects that maximizes it. A conditional mode is therefore a point summary, not the integrated contribution of an individual to the population likelihood.
Deriving the Conditional Log Density
The factorization follows from the definition of a conditional density: the joint density of observations and random effects is the density of the random effects multiplied by the density of the observations conditional on those effects. It does not require observations to be independent without conditioning. Any residual dependence can be represented in the conditional observation model.
For a common Gaussian formulation, suppose \(\eta_i\sim N(0,\Omega)\) and, conditional on \(\eta_i\), \(y_i\sim N(f_i(\eta_i;\theta),R_i(\eta_i;\theta))\). Here \(f_i\) is the vector of model predictions, \(\Omega\) is the random-effect covariance matrix, and \(R_i\) is the conditional residual covariance matrix. Assume \(\Omega\) is positive definite and \(R_i\) is positive definite for the parameter and random-effect values under consideration. With \(n_i\) observations and \(q\) random effects, the log density is:
The first line of contributions measures how well the predictions at \(\eta_i\) describe the observed data, scaled by the residual covariance. The remaining terms describe the population plausibility of \(\eta_i\). If a covariance matrix depends on \(\eta_i\), its determinant term also depends on \(\eta_i\) and must be retained when maximizing. Even when a term is constant in \(\eta_i\), retaining it gives the actual log density rather than merely an objective that has the same maximizer.
The conditional density of random effects given the observations is obtained by normalizing the joint density over \(\eta_i\):
For fixed \(y_i\) and \(\theta\), the denominator does not vary with \(\eta_i\). Thus maximizing the conditional log density is equivalent to maximizing \(p(\eta_i\mid y_i,\theta)\). This equivalence explains the phrase “conditional mode”: it is a mode of the random-effect distribution after conditioning on that individual’s observations.
Worked Example: Adding the Gaussian Log-Density Contributions
Consider one hypothetical observation \(y_i=1.70\). At the random-effect value \(\eta_i=0.20\), let the model prediction be \(f_i=1.50\), with residual variance \(R_i=0.49\). Suppose \(\eta_i\sim N(0,0.08588)\). Calculate the conditional log density at this value.
For a scalar Gaussian observation, the observation contribution is \(-\frac{1}{2}\log(2\pi R_i)-\frac{(y_i-f_i)^2}{2R_i}\). Substituting the values gives:
The random-effect contribution is \(-\frac{1}{2}\log(2\pi\cdot0.08588)-\frac{0.20^2}{2(0.08588)}\approx0.075581\). Therefore the conditional log density, including both contributions, is approximately \(-0.603080+0.075581=-0.527499\).
This is the log of a joint density evaluated at the specified observation and random-effect value. It is not itself a probability. To find a conditional mode, one would compare the log density across candidate values of \(\eta_i\), rather than interpreting this single value in isolation.
Defining and Characterizing the Mode
The conditional mode is an optimization over the random effects for one individual, not an optimization over the population parameters. In general it is written:
If the maximizing value is in the interior of the allowed parameter space and the log density is differentiable, a necessary condition is that the gradient with respect to \(\eta_i\) is zero. At a twice-differentiable interior local maximum, the Hessian is necessarily negative semidefinite. A negative-definite Hessian is sufficient for a strict local maximum under the usual regularity conditions, but it is not necessary: a strict maximum can have zero curvature in some direction.
These conditions do not by themselves prove that a candidate is the global mode. The log density may have multiple local maxima, or the maximum may occur at a boundary if the allowed random-effect space is constrained. A numerical optimizer’s convergence message is therefore not, on its own, evidence that the global maximizer has been found.
Worked Example: An Analytic Mode in a Scalar Model
Suppose the hypothetical observation model is \(y_i\mid\eta_i\sim N(1.8+\eta_i,0.25)\), the population model is \(\eta_i\sim N(0,0.64)\), and the observed value is \(y_i=2.4\). Find the conditional mode.
Ignoring terms constant in \(\eta_i\), the conditional log density is \(-\frac{(2.4-1.8-\eta_i)^2}{2(0.25)}-\frac{\eta_i^2}{2(0.64)}\). Its derivative is:
Setting this derivative to zero gives \(\hat{\eta}_i=\frac{0.64}{0.64+0.25}(2.4-1.8)\). Substituting and calculating, \(\hat{\eta}_i=(0.64/0.89)(0.60)=0.4315\), to four significant figures.
The second derivative is \(-1/0.25-1/0.64=-5.5625\), which is negative. Since the log density is a concave quadratic in \(\eta_i\), this stationary point is the unique global maximum. The mode is less than the observation-only residual \(2.4-1.8=0.60\): the population distribution pulls the estimate toward zero. This is a conditional random-effect estimate, not an estimate of a population parameter.
Worked Example: Finding a Nonlinear Conditional Mode
Consider a hypothetical scalar model \(y_i\mid\eta_i\sim N(e^{\eta_i},0.09)\), with \(\eta_i\sim N(0,0.25)\) and observed value \(y_i=1.50\). Find the mode by solving the score equation and check its curvature.
Apart from constants, the log density is \(-\frac{(1.50-e^{\eta_i})^2}{2(0.09)}-\frac{\eta_i^2}{2(0.25)}\). Its derivative is:
At \(\eta_i=0.344\), \(e^{\eta_i}\approx1.41058\), so the score is approximately \(\frac{(1.50-1.41058)(1.41058)}{0.09}-\frac{0.344}{0.25} \approx1.4015-1.376=0.0255\). At \(\eta_i=0.345\), \(e^{\eta_i}\approx1.41199\), and the score is approximately \(\frac{(1.50-1.41199)(1.41199)}{0.09}-1.380\approx1.38077-1.380=0.00077\). The score crosses zero just above \(0.345\); solving more closely gives \(\hat{\eta}_i\approx0.34503\).
The second derivative is \(\frac{e^{\eta_i}(1.50-2e^{\eta_i})}{0.09}-4\). At \(\eta_i=0.3449\), \(e^{\eta_i}\approx1.412\), giving a second derivative of about \(\frac{1.412(1.50-2.824)}{0.09}-4\approx-24.8\). The curvature is negative, supporting a strict local maximum. In this example it is also the unique stationary point: the derivative of the score is negative for all \(\eta_i\). The nonlinear prediction makes a closed-form solution unavailable, so numerical root finding or optimization is needed.
Numerical Checks, Alternatives, and Limitations
In practice, the conditional log density is optimized separately for each individual at fixed \(\theta\). Its evaluation should use the full specified observation and random-effect densities. Several checks help reveal errors or unreliable optimization:
- Check the objective. Confirm that the observation contribution and random-effect contribution are both included. Omitting the random-effect density changes the target from the conditional mode to an observation-only fit.
- Check scaling and covariance terms. Residual and random-effect covariance matrices must be positive definite for the Gaussian log density to be defined. Include determinant terms, especially when a covariance depends on \(\eta_i\).
- Check stationarity and curvature. At an interior solution, the score should be close to zero. Inspect the Hessian and compare the objective at nearby values. A negative-semidefinite Hessian is necessary at a twice-differentiable interior maximum; a negative-definite Hessian provides stronger local evidence.
- Check sensitivity to starting values. Repeat optimization from multiple starts when the objective is nonlinear. Different converged solutions with different log densities indicate multiple local maxima or optimization difficulty.
- Check boundaries and model behavior. If random effects are constrained, evaluate boundary candidates. If the log density becomes numerically unstable or does not have a finite maximum, investigate the model specification and parameter constraints rather than treating the optimizer output as reliable.
The main alternative summaries answer different questions. The marginal likelihood integrates over random effects, retaining their full distributional contribution instead of selecting one value. The conditional mode selects the most density-supported value for a particular individual. A conditional mean instead averages over the conditional distribution and need not equal its mode, especially when that distribution is asymmetric or multimodal. An observation-only fit maximizes \(p(y_i\mid\eta_i,\theta)\) without the population density and therefore does not generally estimate the conditional mode.
| Quantity | Operation over random effects | What it represents |
|---|---|---|
| Conditional mode | Maximize \(p(y_i\mid\eta_i,\theta)p(\eta_i\mid\theta)\) | A maximizing random-effect value for the individual, at fixed population parameters |
| Marginal likelihood contribution | Integrate \(p(y_i\mid\eta_i,\theta)p(\eta_i\mid\theta)\) over \(\eta_i\) | The individual’s observation density after accounting for random-effect uncertainty |
| Conditional mean | Average \(\eta_i\) under \(p(\eta_i\mid y_i,\theta)\) | The mean random-effect value under the conditional distribution |
A mode is not a complete description of conditional uncertainty: it gives neither the spread nor the shape of the conditional distribution. It can also be sensitive to model assumptions, sparse observations, and nonlinear prediction functions. In particular, a high conditional density at one random-effect value does not establish that the value is known precisely. The conditional mode is useful as a defined optimization target, but it should not be confused with integrating over uncertainty or with an independently estimated population effect.
Check Your Understanding
Answer each question using the definitions and calculations above. Brief solutions follow the questions.
- Write the conditional joint density for one individual in terms of the observation density and random-effect density.
- Why does maximizing the conditional joint density over \(\eta_i\) also maximize the conditional density \(p(\eta_i\mid y_i,\theta)\)?
- In the scalar Gaussian worked example, what term pulls the mode toward zero, and what was the mode to four significant figures?
- At a twice-differentiable interior local maximum, what Hessian condition is necessary? Is negative definiteness necessary?
- Give one reason why a numerical optimizer’s convergence message does not establish that it found the global mode.
Solutions. (1) \(p(y_i,\eta_i\mid\theta)=p(y_i\mid\eta_i,\theta)p(\eta_i\mid\theta)\). (2) The denominator in \(p(\eta_i\mid y_i,\theta)\) is an integral that is constant in \(\eta_i\), so it does not change the maximizing value. (3) The Gaussian population density contributes the penalty \(-\eta_i^2/(2\cdot0.64)\), pulling toward zero; the mode is \(0.4315\). (4) The Hessian must be negative semidefinite. Negative definiteness is sufficient for a strict local maximum under usual regularity conditions, but is not necessary. (5) The objective can have multiple local maxima, and an optimizer may converge to one of them depending on its starting value.