Why Inspect a Profile Likelihood?
A fixed-effect estimate can be very large because the likelihood keeps improving as the coefficient increases, rather than because the data identify a genuinely large finite effect. A profile likelihood makes this behavior visible: it shows how well the model can fit when one coefficient is held at a sequence of values and the remaining parameters are re-estimated. In a binary GLMM, this means accounting for the random effects through the model’s marginal likelihood, not simply plotting a conditional likelihood with the random effects fixed at estimated values.
A profile that rises and then declines has a finite peak. A profile that approaches its best value along a tail toward large positive or negative coefficient values is a warning sign: the coefficient may be diverging in a separating direction. The shape is diagnostic evidence, not a verdict by itself. Weak information, a flat but finite peak, and numerical problems can also make a profile look unusually flat.
Definition in a Binary GLMM
For a binary response \(Y_{ij}\) in cluster \(i\), consider a logistic mixed model with fixed-effect vector \(\boldsymbol{\beta}\), random-effect vector \(\mathbf{b}_i\), and random-effect covariance parameters \(\boldsymbol{\gamma}\). Conditional on \(\mathbf{b}_i\), the responses are Bernoulli with \(\operatorname{logit}\{P(Y_{ij}=1\mid \mathbf{b}_i)\}=\mathbf{x}_{ij}^{\mathsf T}\boldsymbol{\beta}+\mathbf{z}_{ij}^{\mathsf T}\mathbf{b}_i\). Assume clusters are independent, the random effects have the specified distribution (often a mean-zero multivariate normal distribution), and the conditional response model is correctly specified. The marginal likelihood integrates over the random effects.
Here \(p_{ij}(\mathbf{b}_i)\) is the conditional success probability specified by the model, and \(f\) is the random-effect density. For a chosen fixed-effect coordinate \(\beta_k\), the profile log likelihood is obtained by fixing that coordinate and maximizing over all other parameters, including the other fixed effects and any estimated random-effect covariance parameters.
The likelihood-ratio distance from the best attainable fit at a fixed value \(c\) is \(2\{\ell^*-\ell_p(c)\}\). If the best fit is attained only as \(c\) tends to infinity, \(\ell^*\) is a supremum rather than the log likelihood at a finite estimate. In practice, a profile is approximated on a finite grid, so the displayed curve can only suggest this limiting behavior.
Why Separation Can Produce a Plateau
The likelihood mechanism can be seen by following a coefficient along a separating direction. Let \(\mathbf{d}\) be a direction in the fixed-effect space such that, for each observation, the signed fixed-effect contribution \((2y_{ij}-1)\mathbf{x}_{ij}^{\mathsf T}\mathbf{d}\) is strictly positive. Along \(\boldsymbol{\beta}(t)=\boldsymbol{\beta}_0+t\mathbf{d}\), with \(t\) increasing, the fixed-effect linear predictor moves toward the observed outcome for every observation.
For any finite random-effect realization, the fixed-effect contribution eventually dominates the random-effect contribution. Thus each conditional Bernoulli probability for the observed outcome approaches one. The conditional likelihood is bounded between zero and one; applying dominated convergence to the random-effect integral shows that the marginal likelihood also approaches its limiting value along this path. This explains why integrating over random effects does not automatically prevent a fixed-effect coefficient from diverging. If some observations have zero signed contribution, their probabilities do not necessarily approach one along this path; the likelihood can still approach a finite ceiling, as in quasi-complete separation.
The argument describes a path in parameter space. A profile maximizes over nuisance parameters at every fixed coefficient value, so its value is at least as high as the likelihood along any particular path with that coefficient. The profile may therefore approach the same ceiling, or another nuisance-parameter configuration may produce an even higher one. The curve should be interpreted using the actual marginal model and the nuisance-parameter optimization used to construct it.
Worked Profiles With a Random Intercept
The following examples use a deliberately simple binary GLMM to make the profile calculations transparent. Each observation belongs to its own cluster, with random intercept \(u\sim N(0,\sigma^2)\); \(\sigma^2\) is fixed at a known positive value, and the intercept \(\alpha\) is profiled. This is a valid, simplified marginal model, but it is not a substitute for profiling covariance parameters in an applied analysis. Define \(m(\eta)=E[\operatorname{logit}^{-1}(\eta+u)]\). Symmetry of the normal distribution and the identity \(\operatorname{logit}^{-1}(-v)=1-\operatorname{logit}^{-1}(v)\) imply \(m(0)=1/2\), \(m(-\eta)=1-m(\eta)\), and \(m(\eta)\to1\) as \(\eta\to\infty\).
Worked Example: Complete Separation and a Profile Supremum
Suppose four independent clusters each contribute one observation with \(x=0,y=0\), and four more each contribute one observation with \(x=1,y=1\). Use \(\operatorname{logit}\{P(Y=1\mid u)\}=\alpha+\beta x+u\), with the positive coefficient \(\beta\) profiled. Every outcome agrees with the direction of increasing \(\beta\).
After integrating out each one-observation random intercept, the likelihood is \(L(\alpha,\beta)=\{1-m(\alpha)\}^{4}\{m(\alpha+\beta)\}^{4}\). At \(\beta=0\), the success probability \(p=m(\alpha)\) is the same for all observations, so \(L=p^4(1-p)^4\). Since \(p(1-p)\leq 1/4\), with equality at \(p=1/2\), the profiled likelihood is \(L_p(0)=(1/4)^4=1/256=0.003906\). Therefore \(\ell_p(0)=\log(1/256)=-5.545\).
Now take \(\alpha=-\beta/2\). The success probability for \(x=1\) is \(m(\beta/2)\), and the failure probability for \(x=0\) is also \(m(\beta/2)\), by symmetry. Thus this parameter path has likelihood \(m(\beta/2)^8\), which tends to one as \(\beta\to\infty\). No Bernoulli likelihood can exceed one, so the global likelihood supremum is one, or \(\ell^*=0\). The profile at \(\beta=0\) is below that supremum by the likelihood-ratio distance \(2\{0-(-5.545)\}=11.09\). The coefficient has no finite maximizing value: the likelihood keeps improving toward its supremum as \(\beta\) increases.
The random-intercept variance is positive in this example. The pattern arises because each cluster’s random effect is finite with probability one, while the fixed contribution \(\beta x\) grows without bound for the \(x=1\) observations. The marginal integration therefore does not remove the separating direction.
Worked Example: Quasi-Complete Separation and a Nonzero Ceiling
Now suppose there is one observation with \(x=0,y=0\), one with \(x=0,y=1\), and one with \(x=1,y=1\), again with independent one-observation clusters and the same known random-intercept variance. The likelihood is \(L(\alpha,\beta)=m(\alpha)\{1-m(\alpha)\}m(\alpha+\beta)\).
The first two observations contribute at most \(1/4\), because \(p(1-p)\leq1/4\) for every probability \(p\). The last contribution is at most one. Hence \(L(\alpha,\beta)\leq1/4\) for all parameter values. Along the path \(\alpha=0,\ \beta\to\infty\), the first two contributions equal \(1/2\) and \(1/2\), while \(m(\beta)\to1\). Consequently, the profile likelihood approaches the ceiling \(1/4\), or a log-likelihood ceiling of \(\log(1/4)=-1.386\), without attaining it at a finite \(\beta\).
At \(\beta=0\), set \(p=m(\alpha)\). Then \(L=p^2(1-p)\), which is maximized at \(p=2/3\): differentiating gives \(2p(1-p)-p^2=p(2-3p)=0\), and the interior maximum is \(p=2/3\). The profiled likelihood is therefore \(4/27=0.1481\), with log likelihood \(-1.910\). Its likelihood-ratio distance from the limiting supremum is \(2\log\{(1/4)/(4/27)\}=2\log(27/16)=1.046\). The profile approaches a nonzero ceiling because the \(x=0\) observations have conflicting outcomes, while the \(x=1\) success can be fitted increasingly well.
Worked Example: Overlap Produces a Finite Peak
Suppose there is one success and one failure at each predictor value, \(x=0\) and \(x=1\). The likelihood is \(L(\alpha,\beta)=m(\alpha)\{1-m(\alpha)\}m(\alpha+\beta)\{1-m(\alpha+\beta)\}\). Each success-failure pair contributes at most \(1/4\), so \(L\leq1/16=0.06250\). This bound is attained at \(\alpha=0,\beta=0\), since then both probabilities equal \(1/2\). The log likelihood at the finite maximum is \(\log(1/16)=-2.773\).
A tail with \(\beta\to+\infty\) cannot retain this maximum. If \(\alpha\) stays bounded, the \(x=1\) success-failure pair has probability tending to one for one outcome and zero for the other, so its product tends to zero. If \(\alpha\) instead moves toward \(-\infty\) to keep \(\alpha+\beta\) bounded, the \(x=0\) pair has product tending to zero. Thus the likelihood falls away from its finite peak in the positive tail. Both outcomes occur at both predictor values, so this example has overlap rather than the outcome pattern that drove the earlier profiles toward a limiting ceiling.
Constructing and Reading a Profile in Practice
For a fitted GLMM, use the same marginal likelihood definition and integration method throughout the profile calculation. For each grid value \(c\), set \(\beta_k=c\), re-optimize the other fixed effects, and re-estimate any covariance parameters that are part of the model. Plot \(\ell_p(c)\), or plot the relative likelihood-ratio distance \(2\{\ell^*-\ell_p(c)\}\), against \(c\). A standard profile-likelihood workflow is:
State the predictor coding and which tail is being examined. A reversal of the predictor’s coding reverses the sign of the coefficient direction.
At each fixed coefficient value, maximize over the nuisance parameters. Extend the grid far enough to see whether the profile turns down or continues to approach its best value.
Check whether the tail’s likelihood-ratio distance continues to shrink toward zero, or instead increases as the profile falls away from a finite peak.
Repeat tail fits from sensible starting values and assess whether the integration and optimization are sufficiently accurate to resolve the observed change in log likelihood.
A profile is especially informative when it is asymmetric: the coefficient may be well constrained in one direction but poorly constrained in the other. A tail that keeps improving toward the best attainable fit is more specific evidence of a non-finite estimate than a large standard error alone. The earlier material on score and information diagnostics for separation considers complementary local signals; profile likelihood adds a view across a range of coefficient values rather than only near a fitted point.
Common Mistakes and Limits
- Calling any flat profile separation. A shallow but finite peak can reflect weak information. Look for a tail that approaches the best attainable likelihood as the coefficient moves outward, and check the outcome pattern and model specification.
- Profiling the wrong likelihood. Holding estimated random effects fixed gives a conditional calculation, not the marginal profile likelihood defined here. Integrate over random effects and re-optimize the nuisance parameters appropriate to the fitted model.
- Treating the last grid point as infinity. A finite grid cannot establish that a coefficient truly diverges. Extend the grid and check whether the profile’s tail behavior persists; report the diagnostic as evidence, not proof.
- Ignoring numerical accuracy. Integration error or an inadequately optimized nuisance fit can create a spurious plateau, kink, or rise. If successive fits or integration settings materially change the curve, its shape is not yet a reliable diagnostic.
Compared with a Wald diagnostic, a profile examines likelihood behavior away from a single estimate and can reveal a long, asymmetric tail. Its cost is repeated model fitting, and its interpretation still depends on the assumed model and likelihood calculation. A profile sign alone does not identify the cause of a numerical fitting difficulty, establish a causal effect, or justify routine likelihood-ratio reference distributions when the best fit is not attained at a finite parameter value.
Check Your Understanding
Answer each question using the profile-likelihood ideas above. Brief solutions follow the questions.
- In the complete-separation example, what was the log likelihood at \(\beta=0\), and what likelihood-ratio distance separated it from the limiting supremum?
- Why did the quasi-complete example have a limiting likelihood of \(1/4\), rather than one?
- In a binary GLMM, what does the profile calculation do with the random effects and nuisance parameters?
- Why is a flat profile tail not, by itself, conclusive proof of separation?
Solutions. (1) The likelihood at zero was \(1/256\), so \(\ell_p(0)=\log(1/256)=-5.545\); the limiting log likelihood was zero, giving distance \(2(0+5.545)=11.09\). (2) The conflicting \(x=0\) outcomes contribute at most \(p(1-p)=1/4\); the \(x=1\) success contribution can tend to one, but cannot erase that remaining limitation. (3) The marginal likelihood integrates over the random effects, then the profile fixes the chosen coefficient and maximizes over the other fixed effects and estimated covariance parameters. (4) Weak information, inaccurate integration, or imperfect nuisance optimization can also produce a flat-looking tail; the profile must be checked alongside the data pattern and numerical stability.