Profile Likelihood as a Confidence-Interval Method

Why Profile the Likelihood?

An NLME estimate is often accompanied by a standard error and a Wald interval. That interval uses the estimate and its local curvature to make a symmetric range on the parameter scale. This can be a poor description of uncertainty when the likelihood is skewed, when a parameter is constrained, or when other parameters can adjust substantially as the parameter of interest changes. Profile likelihood offers a different route: for each candidate value of the target parameter, it asks how well the model can fit after the remaining parameters are re-estimated.

The result is a likelihood curve for one parameter that retains the best-fitting contribution of the nuisance parameters at every point. A confidence interval can then be based on how far the likelihood at a candidate value falls below its maximum. The curve need not be symmetric around the estimate, although the method does not guarantee a reliable interval if the model or likelihood is poorly behaved.

Definition and Likelihood-Ratio Interval

Let $\theta=(\psi,\lambda)$ denote the model parameters, where $\psi$ is one scalar parameter of interest and $\lambda$ is the vector of all nuisance parameters. Nuisance parameters are parameters that contribute to the likelihood but are not the target of the interval. In an NLME model, they may include other fixed effects, random-effects covariance parameters, and residual-error parameters.

For independent subjects, the marginal likelihood is formed from each subject’s likelihood after integrating over that subject’s random effects. If $y_i$ denotes subject $i$’s data and $b_i$ their random effects, a typical contribution is $\int f(y_i\mid b_i,\theta)g(b_i\mid\theta)\,db_i$, and the subject contributions are multiplied. The profile definition below applies to this specified likelihood; it does not change the model or the data.

Definition: The profile log-likelihood for $\psi$ is $$ \ell_p(\psi)=\max_{\lambda\in\Lambda(\psi)}\ell(\psi,\lambda), $$ where $\ell(\psi,\lambda)$ is the log-likelihood and $\Lambda(\psi)$ is the set of nuisance-parameter values allowed when $\psi$ is fixed. The maximization must be over the allowed parameter space. The profile log-likelihood records the best achievable log-likelihood at each fixed value of $\psi$.

Let $(\hat\psi,\hat\lambda)$ be a global maximum of the full log-likelihood, with value $\hat\ell=\ell(\hat\psi,\hat\lambda)$. If the profile maximization is also global, then $\ell_p(\hat\psi)=\hat\ell$. A likelihood-ratio confidence set includes candidate values whose profile log-likelihood is not too far below that maximum:

D(\psi)=2\{\hat\ell-\ell_p(\psi)\}, \qquad C=\{\psi:D(\psi)\leq c\}.

Here $D(\psi)$ is the likelihood-ratio statistic for comparing the best fit with $\psi$ fixed to the unrestricted best fit. Under regularity conditions and for sufficiently informative data, $D(\psi)$ is approximately distributed as a chi-square random variable with one degree of freedom when the candidate value is the true value. Thus a nominal 95% interval commonly uses $c$ equal to the 95th percentile of that distribution, approximately 3.84. The resulting interval is an approximation to 95% repeated-sampling coverage, not a statement that there is a 95% probability the fixed parameter lies in the observed interval.

The regularity conditions matter: the target should be identifiable and interior to its parameter space, the likelihood should be sufficiently smooth, and the fitted model and likelihood calculations should be appropriate. Small samples, boundaries, weak identification, or a poorly approximated marginal likelihood can undermine the usual calibration. In such situations, the cutoff may require a different calibration; profiling alone does not make the reference distribution exact.

Why the Profile Can Avoid Wald Symmetry

A Wald interval is built from a local approximation near the maximum. In its usual form, it is $\hat\psi\pm z_{1-\alpha/2}SE(\hat\psi)$, where the standard error comes from estimated local curvature. This produces equal distances above and below $\hat\psi$ on the chosen parameter scale. If the actual likelihood falls away more slowly on one side than the other, or changes shape farther from the maximum, that symmetry is an imposed approximation rather than a feature of the full likelihood.

A short local derivation explains the connection. Suppose the full log-likelihood has a smooth, approximately quadratic shape near its interior maximum. If the nuisance parameters are optimized at each nearby $\psi$, the resulting profile also has a local quadratic approximation:

\ell_p(\psi)\approx \hat\ell-\frac{1}{2}I_p(\hat\psi)(\psi-\hat\psi)^2, \qquad D(\psi)\approx I_p(\hat\psi)(\psi-\hat\psi)^2,

where $I_p(\hat\psi)$ is the local profile curvature. Setting the approximate likelihood-ratio statistic equal to $c$ gives $|\psi-\hat\psi|\approx\sqrt{c/I_p(\hat\psi)}$, a symmetric interval. This is why Wald and profile intervals can be similar when the likelihood is well approximated by a quadratic near its maximum. Away from that local approximation, the actual profile can bend differently on either side, and the likelihood-ratio interval can be asymmetric.

Worked Example: A Profile Interval for Clearance

Suppose a hypothetical NLME analysis estimates typical clearance, $CL$, as 5.0 L/h. The fitted profile log-likelihood has its maximum at that value. The following hypothetical profile values are measured relative to the maximum, so $\ell_p(5.0)=0$. Construct an approximate 95% likelihood-ratio interval using $c=3.84$.

CL (L/h)	Profile log-likelihood relative to maximum
3.0	-1.92
2.5	-2.10
5.0	0
7.0	-1.70
7.5	-2.05

The cutoff on the log-likelihood scale is $c/2=3.84/2=1.92$ below the maximum. At $CL=3.0$ L/h, the log-likelihood drop is 1.92, so the lower endpoint is 3.0 L/h in these sampled values. The upper endpoint falls between 7.0 and 7.5 L/h. Linear interpolation of the displayed profile values estimates the crossing:

CL_{\text{upper}}\approx 7.0+ \frac{1.92-1.70}{2.05-1.70}(7.5-7.0) =7.0+\frac{0.22}{0.35}(0.5) \approx 7.31\text{ L/h}.

The approximate profile interval is therefore 3.0 to 7.31 L/h. The interpolation is only an approximation to the crossing; an actual analysis would evaluate the profile more finely near each endpoint. Under the stated regularity and calibration conditions, the interval has approximate 95% coverage in repeated use of the procedure. It is not a 95% probability statement about this fixed clearance value.

Why Nuisance Parameters Must Be Allowed to Adjust

At each candidate $\psi$, the profile maximizes over $\lambda$, rather than evaluating the likelihood with nuisance parameters held at their full-model estimates. This distinction is central: if the nuisance parameters can compensate for a change in $\psi$, fixing them can make the likelihood appear to fall too quickly and can misrepresent uncertainty about $\psi$.

Worked Example: Re-Estimating a Correlated Nuisance Parameter

Consider a hypothetical local likelihood surface for clearance $CL$ and volume $V$, with estimates 5 L/h and 20 L, respectively. Suppose the surface is quadratic with standard-error scales 1.5 L/h and 4 L and correlation $\rho=0.8$. Define $x=(CL-5)/1.5$ and $y=(V-20)/4$. The likelihood drop from its maximum is one-half of

Q(x,y)=\frac{x^2-2\rho xy+y^2}{1-\rho^2}.

Evaluate the candidate $CL=6.5$ L/h. Then $x=(6.5-5)/1.5=1$. For a fixed $x$, maximizing the log-likelihood means minimizing $Q$ over $y$. Differentiating the numerator with respect to $y$ gives $-2\rho x+2y=0$, so the minimizing value is $y=\rho x=0.8$, corresponding to $V=20+4(0.8)=23.2$ L. At this value, $Q=x^2=1$, and the log-likelihood drop is $Q/2=0.5$.

If $V$ were incorrectly held at 20 L, then $y=0$, and $Q=1/(1-0.8^2)=1/0.36=2.778$. The log-likelihood drop would be $2.778/2=1.389$, rather than 0.5. This example shows why the profile includes the best-fitting nuisance value at each candidate clearance. The numerical surface is deliberately simple; in a fitted NLME model the profile is defined by the same maximization principle even when the likelihood is not quadratic.

Comparing a Profile Interval With a Wald Interval

The following hypothetical example illustrates how a profile interval can respect a constrained parameter space while a Wald interval on the original scale does not. It does not imply that every profile interval is better calibrated, or that a Wald interval must fail whenever it crosses a boundary.

Worked Example: A Bounded Bioavailability Estimate

Suppose a hypothetical NLME model estimates bioavailability $F=0.72$, with a standard error of 0.18 on the original scale. A conventional 95% Wald interval is:

0.72\pm1.96(0.18)=0.72\pm0.3528=(0.3672,\;1.0728).

Bioavailability is constrained to the range 0 to 1, so the upper Wald endpoint is outside the parameter space. Now suppose the hypothetical profile log-likelihood relative to its maximum is $-1.92$ at both $F=0.38$ and $F=0.94$, and drops below $-1.92$ beyond those crossings. With $c=3.84$, those crossings give an approximate profile interval of 0.38 to 0.94. The profile follows the specified likelihood and allowed values rather than extending symmetrically past 1.

This result depends on the likelihood and its calibration, not merely on the parameter bound. The profile interval is not automatically exact, and a different parameterization or analysis may be appropriate in some settings. Its key advantage here is that the interval is obtained from the likelihood across the parameter’s allowed domain rather than from a symmetric standard-error rule.

Assumptions, Diagnostics, and Limitations

A profile interval is meaningful only if the fitted likelihood represents the intended NLME model and the optimization at each candidate value finds the relevant maximum. In practice, examine whether the unrestricted fit is stable, whether the profiled log-likelihood reaches its maximum near the full-model estimate, and whether the profile values change smoothly as the candidate parameter changes. Sudden jumps, irregular discontinuities, or different results from repeated fits can indicate numerical optimization problems or competing modes; they should not be interpreted automatically as genuine likelihood shape.

Also verify that the parameter constraints are respected and that the same data, likelihood definition, and model are used throughout. Profile endpoints can be sensitive to weak information in the data: a shallow profile may produce distant or poorly determined crossings. A boundary or nonidentifiable parameter can invalidate the usual chi-square calibration. In such cases, report the issue and consider whether a different calibration or uncertainty method is justified rather than treating the nominal interval as reliable by default.

The main alternative is a Wald interval, which is computationally convenient and can be adequate when the likelihood is locally close to quadratic and the estimate is well away from constraints. Bootstrap intervals use repeated resampling or simulation to estimate the sampling distribution and can reflect features not represented by a local quadratic approximation, but they require many successful refits and can have their own finite-sample and computational limitations. A profile interval instead traces likelihood support while allowing nuisance parameters to adjust. It can reveal asymmetry, but it still depends on the chosen likelihood, the fitting procedure, and an appropriate likelihood-ratio calibration.

Common pitfall: Holding nuisance parameters fixed at their full-model estimates and calling the resulting curve a profile is incorrect. A profile requires maximizing over the nuisance parameters separately at each candidate value of the target parameter. Also, an asymmetric profile does not by itself prove that an interval has correct coverage: the likelihood and the cutoff’s regularity conditions still need to be appropriate.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In one sentence, define the profile log-likelihood for a scalar parameter $\psi$.
At a candidate value, the profile log-likelihood is 2.3 units below its maximum. What is the likelihood-ratio statistic $D(\psi)$? Would it be included using a cutoff of 3.84?
Why can holding nuisance parameters fixed give a different likelihood drop from profiling them?
A Wald interval for a parameter constrained to be between 0 and 1 has an upper endpoint of 1.08. Does this prove that a profile interval is correctly calibrated? Explain.

Solutions. (1) It is the maximum log-likelihood over all allowed nuisance-parameter values at each fixed value of $\psi$. (2) $D(\psi)=2(2.3)=4.6$, which exceeds 3.84, so the candidate is not included in the stated likelihood-ratio set. (3) Nuisance parameters may adjust to partly compensate for the candidate value; profiling finds that best fit, while fixing them does not. (4) No. The Wald endpoint shows that the symmetric interval extends beyond the allowed range, but profile calibration still depends on the likelihood, optimization, and the validity of the reference cutoff’s assumptions.